Chris Lamb • Holger Levsen • Maria Glukhova •
Steven Chamberlain • Vagrant Cascadian •
Valerie Young • Ximin Luo
The incomplete team, with apologies to $YOU
• Alexis Bienvenüe
• Alexander Couzens
• Andrew Ayer
• Asheesh Laroia
• Bernhard M. Wiedemann
• Boyuan Yang
• Chris Lamb
• Chris West
• Christoph Berg
• Clint Adams
• Dafydd Harries
• Daniel Kahn Gillmor
• Daniel Shahaf
• Daniel Stender
• David Suarez
• Drew Fisher
• Emmanuel Bourg
• Emanuel Bronshtein
• Esa Peuha
• Fabian Wolff
• Guillem Jover
• Hans-Christoph Steiner
• Harlan Lieberman-Berg
• Helmut Grohne
• Holger Levsen
• Jelmer Vernooij
• Juan Picca
• Justin Cappos
• Maria Glukhova
• Mathieu Bridon
• Mattia Rizzolo
• Nicolas Boulenguez
• Niels Thykier
• Niko Tyni
• Paul Wise
• Peter De Wachter
• Philip Rinn
• Reiner Herrmann
What is the goal of Reproducible Builds?
Prove binary came from source code.
Why do we want to prove this?
The binary could have been:
- ...compiled by a malicious actor.
- ...compiled with a compromised compiler.
How do we achieve Reproducible Builds?
In Debian, two branches of work:
- 1. Compilation of binary program should be deterministic.
- 2. Build environment of any binary program should be reproducible.
How far we've come..!
|First rebuild in 2013||24% packages reproducible|
|June 2017||94% packages reproducible|
How far we'll need to go..!
|sometime||100% packages reproducible|
|sometime||tools to actually verify that in practice|
Check the progress
Technical & other security benefits
Predictable OpenID secret
'OpenIDConsumerSecret' => '639098210478536',
'cgibin' => '/usr/lib/cgi-bin/gbrowse',
'conf' => '/etc/gbrowse',
- Every installation shares the same secret!
Random chars in manpages
-This manual page documents the usageoof WikipediaFS.
+This manual page documents the usage of WikipediaFS.
memcpy(&buf, &buf, strlen(buf)-1);
memcpy(3): The memory areas must not overlap
n\\011" → "
\111" → maps to capital "I"
- memcpy(&buf, &buf, strlen(buf)-1);
+ memmove(&buf, &buf, strlen(buf)-1);
Fails to build 0.46% of the time
x = f(u('abc'), 16)
y = f(u('abc'), 16)
self.assertEqual(sorted(set(x)), [u('a'), u('b'), u('c')])
AssertionError: Lists differ: [u'a', u'b'] != [u'a', u'b', u'c']
(3C2)*(2/3)16 – (3C1)*(1/3)16 =~ 0.46%
Reproducible Builds Summit
December 2016, Berlin
- Software Freedom Conservancy
A build is reproducible if given the same source code, build environment and build instructions, any party can recreate bit-by-bit identical copies of all specified artifacts.
Reproducible Builds Summit 2016
- .buildinfo files for RPMs
- Cross Distro Collaboration
- increased amd64+i386 ressources
- added arm64
- expanded armhf, up to 29 boards!
- some arm64 boards building armhf, with issues
- some more projects, more collaboration
OpenSuSE on board
- Bernhard submitting a lot of patches upstream, pick some examples
Examples of more active distros/projects
- Guix, Fdroid, LEDE, Coreboot, but also in-toto, Tails…
Updates on build path
- GCC patch fixed 1800 packages but getting some resistance from upstream...
reprotest - overview and updates
- What: run commands under varying build environments and check
their output for reproducibility. Features:
- Running inside virtual containers (e.g. ...)
- Presets for convenience, atm only Deb packages
- Reduce diff with autopkgtest, aim to deduplicate (ximin)
- Make it distro-independent, first ArchLinux (santiago)
diffoscope - overview
- What exactly makes two files different?
- Recursively unpacks archives, decompresses PDF files, disassembles binaries etc
- Converts various file formats to human-readable form
- Reports differences in form of plain text, HTML, RST, JSON or Markdown
- Try it online: try.diffoscope.org
diffoscope - updates 1/2
- Now works better with huge diffs (like GCC)
- possible to control how detailed the output gets
- reuse previously generated output saved in JSON format
- 10s of speed optimisations (via Tails)
- Progress bar displayed when diffoscope runs in terminal
- --exclude, --max-container-depth and other ways to control behaviour
diffoscope - updates 2/2
- Better logging and debugging utilities
- New formats supported for comparison: APK, OGG, .dtb, R object files (.rds, .rdb), PGP files, .docx, .odt, ...
- New output formats: RST, Markdown, JSON
- Visual comparison of images (JPEG, ICO, PNG, GIF)
FTP archive and dak
- Publish buildinfo files in the official archive, requires some dak patches.
- Then, run tests.r-b.org rebuilds against actual Debian binaries, and encourage third parties to do the same.
- steven began to work in this area with repro-build.pl (the topic of his "Fun with .buildinfo" talk yesterday)
- We had not been testing against actual archive binaries, hope to fix this soon.
- Recently, required packages NMU (by ximin), now all reproducible except GCC.
- Next, begin our wider NMU campaign, for packages with long-pending patches.
- Eventually aim for build-essential and key-packages.
- "Packages should be reproducible" (#844431).
- we'll need to define reproducibilty
- reproducibile in a fairly controlled way / sane environment - not everywhere
- define requirements / exceptions: same buld environment + same options + same path
- mention .buildinfo files and missing processes+tools
- Should we have this in policy now? Do we agree that Debian is ready for this, as a should which still needs work and non complying is a normal bug for now…
- UI/workflow for APT to notify users about unreproducible packages (#863622).
- sbuild, pbuilder
How can I help?
- Join our lovely team!
- Check your packages on
- Merge patches & push them upstream
- Fix toolchain issues (Java, TeX,