Here's what happened in the Reproducible Builds effort between Sunday October 22 and Saturday October 28 2017:

Past Events

Upcoming/current events

Documentation updates

Bernhard Wiedemann started The Unreproducible Package which "is meant as a practical way to demonstrate the various ways that software can break reproducible builds using just low level primitives without requiring external existing programs that implement these primitives themselves.

It is structured so that one subdirectory demonstrates one class of issues in some variants observed in the wild."

Reproducible work in other projects

Hush, a fork of ZCash, opened an issue into reproducible builds.

A new tag was added to lintian (lint checker for Debian packages) to ensure that changelog entry timestamps are strictly increasing. This avoids certain real-world issues with identical timestamps, documented in Debian #843773.

Packages reviewed and fixed, and bugs filed

Patches sent upstream:

  • Bernhard M. Wiedemann:
    • gtranslator, embedded build timestamps
    • libgda, embedded build timestamps
    • mariadb, embedded build timestamps
    • nim, embedded build timestamps

Debian bug reports:

Reviews of unreproducible packages

14 package reviews have been added, 35 have been updated and 28 have been removed in this week, adding to our knowledge about identified issues.

1 issue type has been updated:

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Adrian Bunk (4)

strip-nondeterminism development

Version 0.040-1 was uploaded to unstable by Mattia Rizzolo. It included contributions already covered by posts of the previous weeks, as well as new ones from:

  • Mattia Rizzolo:
    • png.pm: Don't open the original file in write mode

reprotest development

Development continued in git:

  • Ximin Luo:
    • New features:
      • Support a domain_host variation.
      • Support a --print-sudoers feature.
    • Documentation:
      • Note some caveats about the existing git versions as a self-reminder not to release it yet.
      • Updates about our assumptions and rearrange sudo into its own section.
    • Bug fixes:
      • main: When dropping privs, make sure the user can still move in theroot.
      • tests: fix, need to preserve env for su
      • build: Don't fail when the build produces a broken symlink
      • main, presets: Properly drop privs when running the build. (Closes: #877813)
    • Code quality:
      • Improve logging to try to get to the bottom of the jenkins failures
      • Tweak tests to avoid some build dependencies
      • build: Name temporary directories after reprotest not autopkgtest

buildinfo.debian.net development

Development continued in git:

  • Chris Lamb:
    • New features:
      • Add API endpoint to fetch specific .buildinfo files for a certain package/version/architecture, and optimise it. (Closes: #25)
    • Bug fixes:
      • Always show SHA256, regardless of viewport size. (Closes: #27)
      • Actually filter by source package (!)

reproducible-website development

  • Holger Levsen:
    • RWS3 Berlin 2017:
      • Add CoyIM, Arch Linux, LEDE, LEAP, subuser.org, Bazel, coreboot.
      • Make some sponsors visible.
      • Add short paragraph explaining that registration is mandatory.

Misc.

This week's edition was written by Ximin Luo, Chris Lamb, Bernhard M. Wiedemann and Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Posted 2017-11-03 17:58:12 UTC Tags: reproducible builds

Here's what happened in the Reproducible Builds effort between Sunday October 29 and Saturday November 4 2017:

Past events

  • From October 31st — November 2nd we held the 3rd Reproducible Builds summit in Berlin, Germany. A full, in-depth report will be posted in the next week or so.

Upcoming events

  • On November 8th Jonathan Bustillos Osornio (jathan) will present at CubaConf Havana.

  • On November 17th Chris Lamb will present at Open Compliance Summit, Yokohama, Japan on how reproducible builds ensures the long-term sustainability of technology infrastructure.

Reproducible work in other projects

Packages reviewed and fixed, and bugs filed

Reviews of unreproducible packages

7 package reviews have been added, 43 have been updated and 47 have been removed in this week, adding to our knowledge about identified issues.

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Adrian Bunk (44)
  • Andreas Moog (1)
  • Lucas Nussbaum (7)
  • Steve Langasek (1)

Documentation updates

diffoscope development

Version 88 was uploaded to unstable by Mattia Rizzolo. It included contributions (already covered by posts of the previous weeks) from:

  • Mattia Rizzolo
    • tests/comparators/dtb: compatibility with version 1.4.5. (Closes: #880279)
  • Chris Lamb
    • comparators:
      • binwalk: improve names in output of "internal" members. #877525
      • Omit misleading "any of" prefix when only complaining about one module in ImportError messages.
    • Don't crash on malformed "md5sums" files. (Closes: #877473)
    • tests/comparators:
      • ps: ps2ascii > 9.21 now varies on timezone, so skip this test for now.
      • dtby: only parse the version number, not any "-dirty" suffix.
    • debian/watch: Use HTTPS URI.
  • Ximin Luo
    • comparators:
      • utils/file: Diff container metadata centrally. This fixes a last remaining bug in fuzzy-matching across containers. (Closes: #797759)
      • Fix all the affected comparators after the above change.
  • Holger Levsen
    • Bump Standards-Version to 4.1.1, no changes needed.

strip-nondeterminism development

Version 0.040-1 was uploaded to unstable by Mattia Rizzolo. It included contributions already covered by posts of the previous weeks, as well as new ones from:


Version 0.5.2-2 was uploaded to unstable by Holger Levsen.

It included contributions already covered by posts of the previous weeks, as well as new ones from:

reprotest development

buildinfo.debian.net development

tests.reproducible-builds.org

  • Mattia Rizzolo:
    • archlinux: enable schroot building on pb4 as well
    • archlinux: don't install the deprecated abs tool
    • archlinux: try to re-enable one schroot creation job
  • lynxis
    • lede: replace TMPDIR -> RESULTSDIR
    • lede: openwrt_get_banner(): use locals instead of globals
    • lede: add newline to $CONFIG
    • lede: show git log -1 in jenkins log
  • Holger Levsen:
    • lede: add very simple landing page
  • Juliana Oliveira Rodrigues
    • archlinux: adds pacman-git dependencies
  • kpcyrd
    • archlinux: disable signature verification when running in the future
    • archlinux: use pacman-git until the next release
    • archlinux: make pacman fail less early
    • archlinux: use sudo to prepare chroot
    • archlinux: remove -rf for regular file
    • archlinux: avoid possible TOCTOU issue
    • archlinux: Try to fix tar extraction
    • archlinux: fix sha1sums parsing

Misc.

This week's edition was written by Bernhard M. Wiedemann, Chris Lamb, Mattia Rizzolo & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Here's what happened in the Reproducible Builds effort between Sunday November 5 and Saturday November 11 2017:

Upcoming events

On November 17th Chris Lamb will present at Open Compliance Summit, Yokohama, Japan on how reproducible builds ensures the long-term sustainability of technology infrastructure.

We plan to hold an assembly at 34C3 - hope to see you there!

LEDE CI tests

Thanks to the work of lynxis, Mattia and h01ger, we're now testing all LEDE packages in our setup. This is our first result for the ar71xx target: "502 (100.0%) out of 502 built images and 4932 (94.8%) out of 5200 built packages were reproducible in our test setup." - see below for details how this was achieved.

Bootstrapping and Diverse Double Compilation

As a follow-up of a discussion on bootstrapping compilers we had on the Berlin summit, Bernhard and Ximin worked on a Proof of Concept for Diverse Double Compilation of tinycc (aka tcc).

Ximin Luo did a successful diverse-double compilation of tinycc git HEAD using gcc-7.2.0, clang-4.0.1, icc-18.0.0 and pgcc-17.10-0 (pgcc needs to triple-compile it). More variations are planned for the future, with the eventual aim to reproduce the same binaries cross-distro, and extend it to test GCC itself.

Packages reviewed and fixed, and bugs filed

Patches filed upstream:

  • Bernhard M. Wiedemann:
    • clang - ASLR affects objective-C binaries.
  • Chris Lamb:
    • nbsphinx (merged) - Random UUIDs used as element selectors.
    • stardicter (merged) - SOURCE_DATE_EPOCH support.
    • stetl - Build path in documentation.

Patches filed in Debian:

Patches filed in OpenSUSE:

  • Bernhard M. Wiedemann:
    • i4l-base (merged) - Uninitialized memory written to output.

Reviews of unreproducible packages

73 package reviews have been added, 88 have been updated and 40 have been removed in this week, adding to our knowledge about identified issues.

4 issue types have been updated:

Weekly QA work

During our reproducibility testing, FTBFS bugs have been detected and reported by:

  • Adrian Bunk (69)
  • Andreas Beckmann (3)
  • Dmitry Shachnev (1)
  • Graham Inggs (1)

diffoscope development

Mattia Rizzolo uploaded version 88~bpo9+1 to stretch-backports.

reprotest development

  • Ximin Luo:
    • build: add comment that util-linux confirmed bug in nsenter, awaiting fix.
    • Make --print-sudoers work for --env-build as well.

reproducible-website development

  • Holger Levsen:
    • rws3: add OTF as sponsor
    • rws3: add F-Droid, riot-os.org
  • Chris Lamb:
    • Move the "contribute" page from the Debian wiki to /contribute/ on our main website.
  • Eitan Adler:
    • Fix typo in FreeBSD mailing list.

theunreproduciblepackage development

tests.reproducible-builds.org in detail

  • Mattia Rizzolo:

    • reproducible archlinux: enable debugging mode
    • reproducible archlinux: don't use hidden files for the package lists
    • reproducible fedora: don't use hidden files for the package lists
    • udd-query: move from public-udd-mirror.xvm.mit.edu to udd-mirror.debian.net
    • udd-query: remove the temporary file with a trap in case this script is called with the wrong argument, and in case of failures, etc, the temporary file would be left around otherwise
    • reproducible debian: schroot-create: drop the reproducible gpg keyring into /etc/apt/trusted.gpg.d/ instead of using apt-key add
    • reproducible debian: setup_pbuilder: drop the reproducible gpg keyring into /etc/apt/trusted.gpg.d/ instead of using apt-key add
    • reprodocible debian: setup_pbuilder: stop installing gnupg2 in our chroot, not needed anymore now
    • Mattia also merged and deployed some commits from others this week.
  • Alexander 'lynxis' Couzens

    • reproducible_lede: use correct place/variable to save results: Results on remote nodes are expected to be under $TMPDIR, which defined by openwrt_build. RESULTSDIR is undefined on the remote node
    • reproducible_lede: enable building all packages again, after it was disabled to improve the debug speed.
    • reproducible_lede: correct given path for node_cleanup_tmpdirs & node_save_logs- reproducible_lede: enable CONFIG_BUILDBOT to reduce inodes while building.
  • kpcyrd:

    • reproducible-archlinux: try porting abs to asp
    • reproducible-archlinux: explicitly sync packages
    • reproducible-archlinux: use sudo for pacman
  • Hans-Christoph Steiner:

    • reproducible fdroid: point jenkins to canonical URL
    • reproducible_fdroid: separate testsuite into its own job
    • reproducible fdroid: sync upstream script names with jenkins.debian.net, make things self-documenting by reusing the same names everywhere.
    • reproducible_fdroid_test: make script executable
  • Chris Lamb:

    • Move some IRC announcements to #debian-reproducible-changes.
  • Holger Levsen:

    • reproducible LEDE: try to deal gracefully with problems and report
    • as usual, Holger merged many of the above commits and deployed them.

Misc.

This week's edition was written by Ximin Luo, Bernhard M. Wiedemann, Chris Lamb and Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.