Here's what happened in the Reproducible Builds effort between Sunday April 16 and Saturday April 22 2017:

Upcoming events

  • On April 26th Chris Lamb will give a talk at foss-north 2017 in Gothenburg, Sweden on Reproducible Builds.

  • Between May 5th-7th the Reproducible Builds Hackathon 2017 will take place in Hamburg, Germany.

  • On May 13th Chris Lamb will give a talk at OSCAL'17 in Tirana, Albania on Reproducible Builds.

Reproducible work in other projects

Packages reviewed and fixed, and bugs filed

Chris Lamb:

Chris West:

Reviews of unreproducible packages

37 package reviews have been added, 64 have been updated and 16 have been removed in this week, adding to our knowledge about identified issues.

One issue type has been updated:

Two issue types have been added:

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Chris Lamb (2)

diffoscope development

Misc.

This week's edition was written by Chris Lamb, Vagrant Cascadian & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Posted 2017-04-25 07:38:47 UTC Tags: reproducible builds

Here's what happened in the Reproducible Builds effort between Sunday April 9 and Saturday April 15 2017:

Upcoming events

On April 26th Chris Lamb will give a talk at foss-north 2017 in Gothenburg, Sweden on Reproducible Builds.

Media coverage

Jake Edge wrote a summary of Vagrant Cascadian's talk on Reproducible Builds at LibrePlanet.

Toolchain development and fixes

Ximin Luo forwarded patches to GCC for BUILD_PATH_PREFIX_MAP support.

With this patch to backported to GCC-6, as well as a patched dpkg to set the environment variable, he scheduled ~3,300 packages that are unreproducible in unstable-amd64 but reproducible in testing-amd64 - because we vary the build path in the former but not latter case. Our infrastructure ran these in just under 3 days, and we reproduced ~1,700 extra packages.

This is about 6.5% of ~26,100 Debian source packages, and about 1/2 of the ones whose irreproducibility is due to build-path issues. Most of the rest are not related to GCC, such as things built by R, OCaml, Erlang, LLVM, PDF IDs, etc.

(The dip afterwards, in the graph linked above, is due to reverting back to an unpatched GCC-6, but we'll be rebasing the patch continually over the next few weeks so the graph should stay up.)

Packages reviewed and fixed, and bugs filed

Chris Lamb:

Chris West:

Reviews of unreproducible packages

38 package reviews have been added, 111 have been updated and 85 have been removed in this week, adding to our knowledge about identified issues.

6 issue types have been updated:

Added:

Updated:

Removed:

diffoscope development

Development continued in git on the experimental branch:

Chris Lamb:

  • Don't crash on invalid archives (#833697)
  • Tidy up some other code

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Chris Lamb (3)
  • Chris West (1)

Misc.

This week's edition was written by Ximin Luo, Chris Lamb & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Posted 2017-04-19 21:00:46 UTC Tags: reproducible builds

Here's what happened in the Reproducible Builds effort between Sunday April 2 and Saturday April 8 2017:

Media coverage

Toolchain development and fixes

Reviews of unreproducible packages

27 package reviews have been added, 14 have been updated and 17 have been removed in this week, adding to our knowledge about identified issues.

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Aaron M. Ucko (1)
  • Adrian Bunk (1)
  • Chris Lamb (2)

tests.reproducible-builds.org

Misc.

This week's edition was written by Chris Lamb, Vagrant Cascadian & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Posted 2017-04-11 07:27:27 UTC Tags: reproducible builds

Here's what happened in the Reproducible Builds effort between Sunday March 26 and Saturday April 1 2017:

Media coverage

Sylvain Beucler wrote a follow-up post Practical basics of reproducible builds 2, which like last weeks article is about his experiences making software build reproducibly.

Reproducible work in other projects

Colin Watson started writing a patch to make launchpad store .buildinfo files. (It's not yet deployed.)

Toolchain development and fixes

Ximin Luo continued to work on BUILD_PATH_PREFIX_MAP patches for GCC 6 and dpkg.

Packages reviewed and fixed, and bugs filed

Chris Lamb:

Mattia Rizzolo:

Reviews of unreproducible packages

49 package reviews have been added, 25 have been updated and 42 have been removed in this week, adding to our knowledge about identified issues.

1 issue type has been updated:

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Chris Lamb (4)
  • Mattia Rizzolo (1)

diffoscope development

diffoscope 81 was uploaded to experimental by Chris Lamb. It included contributions from:

  • Chris Lamb
    • Correct meaningless "1234-content" metadata when introspecting files within archives. This was a regression since #854723 due to the use of auto-incrementing on-disk filenames. (Closes: #858223)
  • Ximin Luo
    • Improve ISO9660/DOS/MBR check.

reprotest development

reprotest development continued in git, including contributions from:

  • Ximin Luo:
    • Preserve directory structure when copying artifacts.

buildinfo.debian.net development

buildinfo.debian.net development continued in git, including contributions from:

  • Chris Lamb:
    • Tidy rejection of supported formats.
    • Don't parse "Format:" header as the source package version.

reproducible-website development

Holger switched reproducible-builds.org and www.reproducible-builds.org to letsencrypt certificates.

Misc.

This week's edition was written by Ximin Luo and Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Posted 2017-04-06 22:29:29 UTC Tags: reproducible builds

Welcome to the 100th issue of this weekly news! We hope you have been enjoying our posts and would love to receive some feedback via our mailing list!

Anyway, here's what happened in the Reproducible Builds effort between Sunday March 19 and Saturday March 25 2017:

Reproducible Builds Hackathon Hamburg 2017

The Reproducible Builds Hamburg Hackathon 2017 (or RB-HH-2017 for short) is a 3-day hacking event taking place May 5th-7th in the CCC Hamburg Hackerspace inside Frappant, as collective art space located witin a historical monument in Hamburg, Germany. The event is open to everyone and we still have some free seats. If you wish to attend, please register your interest as soon as possible.

Media coverage

Packages reviewed and fixed, and bugs filed

Chris Lamb:

Reviews of unreproducible packages

30 package reviews have been added, 13 have been updated and 2 have been removed in this week, adding to our knowledge about identified issues.

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Chris Lamb (2)

diffoscope development

buildinfo.debian.net development

Misc.

This week's edition was written by Chris Lamb & Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Posted 2017-03-28 07:37:58 UTC Tags: reproducible builds

Here's what happened in the Reproducible Builds effort between Sunday March 12 and Saturday March 18 2017:

Upcoming events

Reproducible Builds Hackathon Hamburg 2017

The Reproducible Builds Hamburg Hackathon 2017, or RB-HH-2017 for short is a 3 day hacking event taking place May 5th-7th in the CCC Hamburg Hackerspace located inside Frappant, as collective art space located in a historical monument in Hamburg, Germany.

The aim of the hackathon is to spent some days working on Reproducible Builds in every distribution and project. The event is open to anybody interested on working on Reproducible Builds issues, with or without prior experience!

Accomodation is available and travel sponsorship may be available by agreement. Please register your interest as soon as possible.

Reproducible Builds Summit Berlin 2016

This is just a quick note, that all the pads we've written during the Berlin summit in December 2016 are now online (thanks to Holger), nicely complementing the report by Aspiration Tech.

Request For Comments for new specification: BUILD_PATH_PREFIX_MAP

Ximin Luo posted a draft version of our BUILD_PATH_PREFIX_MAP specification for passing build-time paths between high-level and low-level build tools. This is meant to help eliminate irreproducibility caused by different paths being used at build time. At the time of writing, this affects an estimated 15-20% of 25000 Debian packages.

This is a continuation of an older proposal SOURCE_PREFIX_MAP, which has been updated based on feedback on our patches from GCC upstream, attendees of our Berlin 2016 summit, and participants on our mailing list. Thanks to everyone that contributed!

The specification also contains runnable source code examples and test cases; see our git repo.

Please comment on this draft ASAP - we plan to release version 1.0 of this in a few weeks.

Toolchain changes

  • #857632 apt: ignore the currently running kernel if attempting a reproducible build (Chris Lamb)
  • #857803 shadow: Make the sp_lstchg shadow field reproducible. (Chris Lamb)
  • #857892 fontconfig: please make the cache files reproducible (Chris Lamb)

Packages reviewed and fixed, and bugs filed

Chris Lamb:

Reviews of unreproducible packages

5 package reviews have been added, 274 have been updated and 800 have been removed in this week, adding to our knowledge about identified issues.

1 issue type has been added:

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Chris Lamb (5)
  • Mattia Rizzolo (1)

diffoscope development

diffoscope 79 and 80 were uploaded to experimental by Chris Lamb. It included contributions from:

Chris Lamb:

  • Ensure that we really are using ImageMagick. (Closes: #857940)
  • Extract SquashFS images in one go rather than per-file, speeding up (eg.) Tails ISO comparison by ~10x.
  • Support newer versions of cbfstool to avoid test failures. (Closes: #856446)
  • Skip icc test that varies on endian if the Debian-specific patch is not present. (Closes: #856447)
  • Compare GIF images using gifbuild. (Closes: #857610)
  • Various other code quality, build and UI improvements.

Maria Glukhova:

  • Improve AndroidManifest.xml comparison for APK files. (Closes: #850758)

strip-nondeterminism development

strip-nondeterminism 0.032-1 was uploaded to unstable by Chris Lamb. It included contributions from:

Chris Lamb:

  • Fix a possible endless loop while stripping ar files due to trusting the file's file size data. Thanks to Tobias Stoeckmann for the report, patch and testcase. (Closes: #857975)
  • Add support for testing files we should reject.

tests.reproducible-builds.org

Misc.

This week's edition was written by Ximin Luo, Holger Levsen and Chris Lamb & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Posted 2017-03-21 18:44:20 UTC Tags: reproducible builds

Here's what happened in the Reproducible Builds effort between Sunday March 5 and Saturday March 11 2017:

Upcoming events

Reproducible Builds Hackathon Hamburg

The Reproducible Builds Hamburg Hackathon 2017, or RB-HH-2017 for short, is a 3 day hacking event taking place in the CCC Hamburg Hackerspace located inside the Frappant, which is collective art space located in a historical monument in Hamburg, Germany.

The aim of the hackathon is to spent some days working on Reproducible Builds in every distribution and project. The event is open to anybody interested on working on Reproducible Builds issues in any distro or project, with or without prio experience!

Packages filed

Chris Lamb:

Toolchain development

  • Guillem Jover uploaded dpkg 1.18.23 to unstable, declaring .buildinfo format 1.0 as "stable".

  • Jams McCoy uploaded devscripts 2.17.2 to unstable addingd support for .buildinfo files to the debsign utility via patches from Ximin Luo and Guillem Jover.

  • Hans-Christoph Steiner noted that the first reproducibility-related patch in the Android SDK was marked as confirmed.

Reviews of unreproducible packages

39 package reviews have been added, 7 have been updated and 9 have been removed in this week, adding to our knowledge about identified issues.

2 issue types have been added:

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Chris Lamb (2)

buildinfo.debian.net development

reproducible-website development

tests.reproducible-builds.org

  • Hans-Christoph Steiner gave a progress report on testing F-Droid: we now have a complete vagrant workflow working in nested KVM! So we can provision a new KVM guest, then package it using vagrant box all inside of a KVM guest (which is a profitbricks build node). So we finally have a working setup on jenkins.debian.net. Next up is fixing bugs in our libvirt snapshoting support.
  • Then Hans-Christoph was also able to enable building of all F-Droid apps in our setup, though this is still work in progress…
  • Daniel Shahaf spotted a subtile error in our FreeBSD sudoers configuration and as a result the FreeBSD reproducibility results are back.
  • Holger once again adjusted the Debian armhf scheduling frequency, to cope with the ever increasing amount of armhf builds.
  • Mattia spotted a refactoring error which resulted in no maintenance mails for a week.
  • Holger also spent some time on improving IRC notifications further, though there is still some improvements to be made.

Misc.

This week's edition was written by Chris Lamb, Holger Levsen, Vagrant Cascadian & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Posted 2017-03-14 06:41:54 UTC Tags: reproducible builds

Here's what happened in the Reproducible Builds effort between Sunday February 26 and Saturday March 4 2017:

Upcoming Events

Ed Maste will present Reproducible Builds in FreeBSD at AsiaBSDCon 2017.

Ximin Luo will present Reproducible builds, its uses and the future at Open Source Days in Copenhagen on March 18.

Holger Levsen will give a talk at the German Unix User Group's "Frühjahrsfachgespräch" in Darmstadt, Germany, about Reproducible Builds everywhere on March 23.

Verifying Software Freedom with Reproducible Builds will be presented by Vagrant Cascadian at Libreplanet2017 in Boston, March 25th-26th.

Media coverage

Aspiration Tech published a very detailed report on our Reproducible Builds World Summit 2016 in Berlin.

Reproducible work in other projects

Duncan published a very thorough post on the Rust Programming Language Forum about reproducible builds in the Rust compiler and toolchain.

In particular, he produced a table recording the reproducibility of different build products under different individual variations, totalling 187 build+variation combinations.

Packages reviewed and fixed, and bugs filed

Chris Lamb:

Dhole:

Reviews of unreproducible packages

60 package reviews have been added, 8 have been updated and 13 have been removed in this week, adding to our knowledge about identified issues.

1 issue type has been added:

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Chris Lamb (3)

diffoscope development

diffoscope 78 was uploaded to unstable and jessie-backports by Mattia Rizzolo. It included contributions from:

  • Chris Lamb:
    • Make tests that call xxd work on jessie again. (Closes: #855239)
    • tests: Move normalize_zeros to more generic utils.data module.
  • Brett Smith:
    • comparators.json: Catch bad JSON errors on Python pre-3.5. (Closes: #855233)
  • Ed Maste:
    • Use BSD-style stat(1) on FreeBSD. (Closes: #855169)

In addition, the following changes were made on the experimental branch:

  • Chris Lamb (4):
    • Tidy cbfs tests.
    • Correct "exercice" -> "exercise" typo.
    • Support newer versions of cbfstool to avoid test failure. (Closes: #856446)
    • Skip icc test that varies on endian if the (Debian-specific) patch is not present. (Closes: #856447)

reproducible-website development

  • anonmos1:
    • Replace root with 0 when giving UIDs/GIDs to GNU tar.
  • Holger Levsen and Chris Lamb:
    • Publish report by Aspiration Tech about RWS Berlin 2016.

tests.reproducible-builds.org

  • Ed Maste continued his work on testing FreeBSD for reproducibility but hasn't reached the magical 100% mark yet.
  • Holger Levsen adjusted the Debian builders scheduling frequency, mostly to adopt to armhf having become faster due to the two new nodes.

Misc.

This week's edition was written by Ximin Luo, Chris Lamb, Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Posted 2017-03-10 08:41:59 UTC Tags: reproducible builds

Here's what happened in the Reproducible Builds effort between Sunday February 19 and Saturday February 25 2017:

Reproducible work in other projects

Upcoming Events

Introduction to Reproducible Builds will be presented by Vagrant Cascadian at Scale15x in Pasadena, California, March 5th.

On March 23rd Holger Levsen will give a talk at the German Unix User Group's "Frühjahrsfachgespräch" about Reproducible Builds everywhere.

Verifying Software Freedom with Reproducible Builds will be presented by Vagrant Cascadian at Libreplanet2017 in Boston, March 25th-26th.

Packages reviewed and fixed, and bugs filed

Chris Lamb:

Reviews of unreproducible packages

9 package reviews have been added, 3 have been updated and 1 has been removed in this week, adding to our knowledge about identified issues.

Weekly QA work

During our reproducibility testing, the following FTBFS bugs have been detected and reported by:

  • Chris Lamb (4)

diffoscope development

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.

  • diffoscope 77 was unblocked by the release team for stretch.
  • Mattia Rizzolo uploaded 77~bpo8+1 to jessie-backports.

buildinfo.debian.net development

buildinfo.debian.net is our experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.

Website development

tests.reproducible-builds.org

  • Ed Maste made the upcoming FreeBSD release almost 100% reproducible (see above).
  • Holger Levsen added the number of configured and running builder jobs to the performance stats page.
  • Holger Levsen improved the scheduler, so that untested packages and versions are tried sooner.
  • Holger added logging for submitting .buildinfo files to `buildinfo.debian.net and added notification about this failure.
  • Holger also made some minor improvements to the generated HTML.

Misc.

This week's edition was written by Chris Lamb, Ed Maste & Levsen and reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Posted 2017-02-28 20:25:01 UTC Tags: reproducible builds

Here's what happened in the Reproducible Builds effort between Sunday February 12 and Saturday February 18 2017:

Upcoming Events

The Reproducible Build Zoo will be presented by Vagrant Cascadian at the Embedded Linux Conference in Portland, Oregon, February 22nd.

Introduction to Reproducible Builds will be presented by Vagrant Cascadian at Scale15x in Pasadena, California, March 5th.

Toolchain development and fixes

Ximin Luo posted a preliminary spec for BUILD_PATH_PREFIX_MAP, bringing together work and research from previous weeks.

Ximin refactored and consolidated much of our existing documentation on both SOURCE_DATE_EPOCH and BUILD_PATH_PREFIX_MAP into one unified page, Standard Environment Variables, with extended discussion on related solutions and how these all fit into people's ideas of what reproducible builds should look like in the long term. The specific pages for each variable still remain, at Timestamps Proposal and Build Path Proposal, only without content that was previously duplicated on both pages.

Ximin filed #855282 against devscripts for debsign(1) to support buildinfo files, and wrote an initial series of patches for it with some further additions from Guillem Jover.

Packages reviewed and fixed, and bugs filed

Chris Lamb:

Reviews of unreproducible packages

35 package reviews have been added, 1 have been updated and 17 have been removed in this week, adding to our knowledge about identified issues.

1 issue type has been added:

Weekly QA work

During our reproducibility testing, the following FTBFS bugs have been detected and reported by:

  • Chris Lamb (2)

diffoscope development

diffoscope 77 was uploaded to unstable by Mattia Rizzolo. It included contributions from:

  • Chris Lamb:
    • Some fixes to tests and testing config
    • Don't track archive directory locations, a better fix for CVE-2017-0359.
    • Add --exclude option. Closes: #854783
  • Mattia Rizzolo:
    • Add my key to debian/upstream/signing-key.asc
    • Add CVE-2017-0359 to the changelog of v76
  • Ximin Luo:
    • When extracting archives, try to keep directory sizes small

strip-nondeterminism development

strip-nondeterminism 0.031-1 was uploaded to unstable by Chris Lamb. It included contributions from:

  • Chris Lamb:
    • Make the tests less brittle, by not testing for stat(2) blksize and blocks. #854937

strip-nondeterminism 0.031-1~bpo8+1 was uploaded to jessie-backports by Mattia.

tests.reproducible-builds.org

  • Vagrant Cascadian and Holger Levsen set up two new armhf nodes for Debian tests, p64b and p64c running on pine64 boards with an arm64 kernel and armhf userland. This introduces kernel variations to armhf.
  • Holger also added new setup & maintenance jobs, plus 6 new builder jobs for Debian armhf.
  • Hans-Christoph Steiner continued work on setting up reproducible tests for F-Droid, now with daily tests for faster progress. These tests are now also using the Android SDK from Debian/stretch packages.
  • Mattia Rizzolo added IRC notification to the job testing for mismatches between diffoscope's pypi and Debian archive versions.
  • Mattia also improved the tempfile handling of the Debian builder jobs.
  • Since we've deployed pbuilder 0.228.4 everywhere, Mattia could also simplify the pbuilder configuration and reenable the build directory name variation for Debian reproducibility tests.

Misc.

This week's edition was written by Ximin Luo & Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Posted 2017-02-21 18:25:00 UTC Tags: reproducible builds

This blog is powered by ikiwiki.