Here's what happened in the Reproducible Builds effort between Sunday September 24 and Saturday September 30 2017:

Development and fixes in key packages

Kai Harries did an initial packaging of the Nix package manager for Debian. You can track his progress in #877019.

Uploads in Debian:

Packages reviewed and fixed, and bugs filed

Patches sent upstream:

Reproducible bugs (with patches) filed in Debian:

QA bugs filed in Debian:

Reviews of unreproducible packages

103 package reviews have been added, 153 have been updated and 78 have been removed in this week, adding to our knowledge about identified issues.

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Adrian Bunk (177)
  • Andreas Beckmann (2)
  • Daniel Schepler (1)

diffoscope development

Mattia Rizzolo uploaded version 87 to stretch-backports.

  • Holger Levsen:
    • Bump standards version to 4.1.1, no changes needed.

strip-nondeterminism development

  • Holger Levsen:
    • Bump Standards-Version to 4.1.1, no changes needed.

reprotest development

  • Ximin Luo:
    • New features:
      • Add a --env-build option for testing different env vars. (In-progress, requires the python-rstr package awaiting entry into Debian.)
      • Add a --source-pattern option to restrict copying of source_root.
    • Usability improvements:
      • Improve error messages in some common scenarios.
      • Output hashes after a successful --auto-build.
      • Print a warning message if we reproduced successfully but didn't vary everything.
      • Update examples in documentation.
    • Have dpkg-source extract to different build dir iff varying the build-path.
    • Pass --debug to diffoscope if verbosity >= 2.
    • Pass --exclude-directory-metadata to diffoscope(1) by default.
    • Much refactoring to support the other work and several minor bug fixes.
  • Holger Levsen:
    • Bump standards version to 4.1.1, no changes needed.

tests.reproducible-builds.org

  • Holger Levsen:
    • Fix scheduler to not send empty scheduling notifications in the rare cases nothing has been scheduled.
    • Fix colors in 'amount of packages build each day on $ARCH' graphs.

reproducible-website development

  • Holger Levsen:
    • Fix up HTML syntax
    • Announce that RWS3 will happen at Betahaus, Berlin

Misc.

This week's edition was written by Ximin Luo, Bernhard M. Wiedemann, Holger Levsen and Chris Lamb & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Posted 2017-10-03 18:15:32 UTC Tags: reproducible builds

Here's what happened in the Reproducible Builds effort between Sunday October 1 and Saturday October 7 2017:

Media coverage

Documentation updates

Packages reviewed and fixed, and bugs filed

Reviews of unreproducible packages

32 package reviews have been added, 46 have been updated and 62 have been removed in this week, adding to our knowledge about identified issues.

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Adrian Bunk (27)

diffoscope development

strip-nondeterminism development

Rob Browning noticed that strip-nondeterminism was causing serious performance regressions in the Clojure programming language within Debian. After some discussion, Chris Lamb also posted a query to debian-devel in case there were any other programming languages that might be suffering from the same problem.

reprotest development

Versions 0.7.1 and 0.7.2 were uploaded to unstable by Ximin Luo:

  • New features:
    • Add a --auto-build option to try to determine which specific variations cause unreproducibility.
    • Add a --source-pattern option to restrict copying of source_root, and set this automatically in our presets.
  • Usability improvements:
    • Improve error messages in some common scenarios.
      • Fiving a source_root or build_command that doesn't exist
      • Using reprotest with default settings after not installing Recommends
    • Output hashes after a successful --auto-build.
    • Print a warning message if we reproduced successfully but didn't vary everything.
  • Fix varying both umask and user_group at the same time.
  • Have dpkg-source extract to different build dir if varying the build-path.
  • Pass --exclude-directory-metadata to diffoscope(1) by default as this is the majority use-case.
  • Various bug fixes to get the basic dsc+schroot example working.

It included contributions already covered by posts of the previous weeks, as well as new ones from:

tests.reproducible-builds.org

Misc.

This week's edition was written by Bernhard M. Wiedemann, Chris Lamb, Holger Levsen, Mattia Rizzolo & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Posted 2017-10-10 08:08:10 UTC Tags: reproducible builds

Here's what happened in the Reproducible Builds effort between Sunday October 8 and Saturday October 14 2017:

Upcoming events

  • On Saturday 21st October, Holger Levsen will present at All Systems Go! in Berlin, Germany on reproducible builds.

  • On Tuesday 24th October, Chris Lamb will present at All Things Open 2017 in Raleigh, NC, USA on reproducible builds.

  • On Wednesday 25th October, Holger Levsen will present at the Open Source Summit Europe in Prague, Czech Republic on reproducible builds.

  • From October 31st - November 2nd we will be holding the 3rd Reproducible Builds summit in Berlin. If you are working in the field of reproducible builds, you should totally be there. Please contact us if you have any questions! Quoting from the public invitation mail:

    These dates are inclusive, ie. the summit will be 3 full days from "9 to 5".
    Best arrive on Monday October 30th and leave on the evening of Thursday, 3rd
    at the earliest.
    
    
    Meeting content
    ===============
    
    The exact content of the meeting is going to be shaped by the
    participants, but here are the main goals:
    
     - Update & exchange about the status of reproducible builds in various
       projects.
     - Establish spaces for more strategic and long-term thinking than is possible
       in virtual channels.
     - Improve collaboration both between and inside projects.
     - Expand the scope and reach of reproducible builds to more projects.
     - Brainstorming / Designing several things, eg:
      - designing tools enabling end-users to get the most benefits from
        reproducible builds.
      - design of back-ends needed for that.
     - Work together and hack on solutions.
    
    There will be a huge variety of topics to be discussed. To give a few
    examples:
    - continuing design and development work on .buildinfo infrastructure
    - build-path issues everywhere
    - future directions for diffoscope, reprotest & strip-nondeterminism
    - reproducing signed artifacts such as RPMs
    - discussing formats and tools we can share
    - sharing proposals for standards and documentation helpful to spreading the
      reproducible effort
    - and many many more.
    
    Please think about what you want discuss, brainstorm & learn about at this
    meeting!
    
    
    Schedule
    ========
    
    Preliminary schedule for the three days:
    
    9:00 Welcome and breakfast
    9:30 Meeting starts
    12:30 Lunch
    17:00 End of the official schedule
    
    Gunner and Beatrice from Aspiration will help running the meeting. We will
    collect your input in subsequent emails to make the best of everyone's time.
    Feel free to start thinking about what you want to achieve there. We will also
    adjust topics as the meeting goes.
    
    Please note that we are very likely to spend large parts of the meeting away
    from laptops and closer to post-it notes. So make sure you've answered any
    critical emails *before* Tuesday morning! :)
    

Reproducible work in other projects

Pierre Pronchery reported that that he has built the foundations for doing more reproducibility work in NetBSD.

Packages fixed

Upstream bugs and patches:

  • Bernhard M. Wiedemann:
    • qutim used RANDOM which is unpredictable and unreproducible.
    • dpdk used locale-dependent sort.

Reproducibility non-maintainer uploads in Debian:

QA fixes in Debian:

Reviews of unreproducible packages

6 package reviews have been added, 30 have been updated and 37 have been removed in this week, adding to our knowledge about identified issues.

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Adrian Bunk (40)
  • Eric Valette (1)
  • Markus Koschany (1)

diffoscope development

  • Ximin Luo:
    • Containers: diff the metadata of containers in one central location in the code, so that deep-diff works between all combinations of different container types. This lets us finally close #797759.
    • Tests: add a complete set of cases to test all pairs of container types.
  • Chris Lamb:
    • Temporarily skip the test for ps2ascii(1) in ghostscript > 9.21 which now outputs text in a slightly different format.
    • UI wording improvements.

reprotest development

Version 0.7.3 was uploaded to unstable by Ximin Luo. It included contributions already covered by posts of the previous weeks, as well as new ones:

  • Ximin Luo:
    • Add a --env-build option for testing builds under different sets of environment variables. This is meant to help the discussion over at #876055 about how we should deal with different types of environment variables in a stricter definition of reproducibility.
    • UI and logging tweaks and improvements.
    • Simplify the _shell_ast module and merge it into shell_syn.

Misc.

This week's edition was written by Ximin Luo, Chris Lamb and Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Posted 2017-10-17 19:29:02 UTC Tags: reproducible builds

Here's what happened in the Reproducible Builds effort between Sunday October 15 and Saturday October 21 2017:

  • The Tails project published a report on how they made their ISO images reproducible.

  • dpkg 1.19.0 was uploaded, including support for:

    • Ordering the "unused substitution" warnings to prevent superfluous differences between logs of package builds on the Reproducible Builds test framework. (#870221)

    • A new Build-Kernel-Version field in .buildinfo files that can be generated with a new dpkg-genbuildinfo --always-include-kernel option. (#873937)

Past events

Upcoming events

New York University sessions

A three week session will be held at New York University to work on reproducibilty issues in conjunction with the reproducible builds community. Students from the Application Security course will be working for two weeks to work on the reproducible builds effort.

  • On Tuesday 24th Oct Ed Maste from FreeBSD will be presenting some reproducible builds work for students.

  • On From Tuesday 24th of October to Monday 7th of November students will work on fixing reproducibility issues brought up by the community. A milestone presentation will be held by Santiago Torres-Arias and Preston Moore.

  • On Tuesday 7th November Holger Levsen will join the NYU team to wrap up the work.

Packages reviewed and fixed, and bugs filed

The following reproducible builds-related NMUs were accepted:

Patches sent upstream:

Reviews of unreproducible packages

41 package reviews have been added, 119 have been updated and 54 have been removed in this week, adding to our knowledge about identified issues. 2 issue types were removed as they were fixed:

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Aaron M. Ucko (1)
  • Adrian Bunk (49)
  • Anthony DeRobertis (1)
  • Daniel Schepler (1)
  • Gilles Filippini (1)
  • James Cowgill (1)
  • Matthias Klose (1)
  • Matthias Klumpp (1)
  • Nobuhiro Iwamatsu (1)

diffoscope development

strip-nondeterminism development

Version 0.039-1 was uploaded to unstable by Chris Lamb. It included contributions already covered by posts of the previous weeks, including:

  • Chris Lamb:
    • Clojure considers the .class file to be stale if it shares the same timestamp of the .clj. We thus adjust the timestamps of the .clj to always be younger. (#877418)
    • dh_strip_nondeterminism: Log which handler processed a file. (#876140)
    • bin/strip-nondeterminism: Print a warning in --verbose mode if no canonical time specified.
    • Use HTTPS URI in debian/watch.

reprotest development

tests.reproducible-builds.org

  • Holger Levsen:

    • Install rustc on Jenkins for the reproducible-html-build-path-prefix-map-spec job.
  • Mattia Rizzolo:

    • health_check: Include the running kernel version when reporting multiple kernel installed in /boot.

Website updates

Misc.

This week's edition was written by Bernhard M. Wiedemann, Chris Lamb, Holger Levsen, Santiago Torres & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Posted 2017-10-24 12:53:12 UTC Tags: reproducible builds