Reproducible Builds in September 2023

Thu, 12 Oct 2023 18:22:23 +0200

Welcome to the September 2023 report from the Reproducible Builds project

In these reports, we outline the most important things that we have been up to over the past month. As a quick recap, whilst anyone may inspect the source code of free software for malicious flaws, almost all software is distributed to end users as pre-compiled binaries.

Andreas Herrmann gave a talk at All Systems Go 2023 titled “Fast, correct, reproducible builds with Nix and Bazel”. Quoting from the talk description:

You will be introduced to Google’s open source build system Bazel, and will learn how it provides fast builds, how correctness and reproducibility is relevant, and how Bazel tries to ensure correctness. But, we will also see where Bazel falls short in ensuring correctness and reproducibility. You will [also] learn about the purely functional package manager Nix and how it approaches correctness and build isolation. And we will see where Bazel has an advantage over Nix when it comes to providing fast feedback during development.

Andreas also shows how you can get the best of both worlds and combine Nix and Bazel, too. A video of the talk is available.

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, Chris Lamb fixed compatibility with file(1) version 5.45 […] and updated some documentation […]. In addition, Vagrant Cascadian extended support for GNU Guix […][…] and updated the version in that distribution as well. […].

Yet another reminder that our upcoming Reproducible Builds Summit is set to take place from October 31st — November 2nd 2023 in Hamburg, Germany.

If you haven’t been before, our summits are a unique gathering that brings together attendees from diverse projects, united by a shared vision of advancing the Reproducible Builds effort. During this enriching event, participants will have the opportunity to engage in discussions, establish connections and exchange ideas to drive progress in this vital field.

If you’re interested in joining us this year, please make sure to read the event page, the news item, or the invitation email that Mattia Rizzolo sent out recently, all of which have more details about the event and location.

We are also still looking for sponsors to support the event, so please reach out to the organising team if you are able to help. Also note that PackagingCon 2023 is taking place in Berlin just before our summit.

On the Reproducible Builds website, Greg Chabala updated the JVM-related documentation to update a link to the BUILDSPEC.md file. […] And Fay Stegerman fixed the builds failing because of a YAML syntax error.

Distribution work

In Debian, this month:

Debian bug #940234 was originally opened in September 2019 by Aurelien Jarno to request that Debian “add a new requirement that repeatedly building the source package in the same environment produces identical .dsc file modulo the GPG signature”. This month, however, Russ Allbery closed the bug due to concerns about the viability of source reproducibility.

10 reviews of Debian packages were added, 56 were updated and 68 were removed this month adding to our knowledge about identified issues.

Bastian Blank posted a mail to a number of Debian mailing lists titled “Upcoming changes to Debian Linux kernel packages”. Under Kernel modules will be signed with an ephemeral key, Bastian wrote: “Yes, this will make the build unreproducible, but no better solution currently exists.”

Vagrant Cascadian posted about various experiments with verification builds for Debian and snapshotting the Debian archive

September saw F-Droid add ten new reproducible apps, and one existing app switched to reproducible builds. In addition, two reproducible apps were archived and one was disabled for a current total of 199 apps published with Reproducible Builds and using the upstream developer’s signature. […] In addition, an extensive blog post was posted on f-droid.org titled “Reproducible builds, signing keys, and binary repos”.

Upstream patches

The Reproducible Builds project detects, dissects and attempts to fix as many currently-unreproducible packages as possible. We endeavour to send all of our patches upstream where appropriate. This month, we wrote a large number of such patches, including:

openSUSE monthly

Bernhard M. Wiedemann:

linuxsampler (benchmarking issue)

antlr3 (date)

rpm (embeds too many build details)

seamonkey (date)

conky (date and ordering-related issue)

lsp-plugins-shared (date/copyright year issue)

build-compare

helix (ASLR-related non-determinism)

intel-graphics-compiler (ASLR)

Chris Lamb:

#1052950 filed against sphinxcontrib-mermaid.

#1052970 filed against mkdocs-material.

#1052989 filed against apophenia.

#1053205 filed against lapackpp.

#1053263 filed against blaspp.

Fridrich Strba applied 124 Java-related patches into openSUSE:

mysql-connector-java, java-21-openjdk, apache-ivy, maven-assembly-plugin, eclipse, antlr3, groovy18, hbci4java, ini4j, hppc, checkstyle, glassfish-jaxb, tycho, xmvn, mockito, languagetool, json-lib, jnr-unixsocket, jnr-ffi, jnr-enxio, jboss-jaxrs-2.0-api, istack-commons, rxtx-java, glassfish-jaxb, glassfish-hk2, findbugs, docker-client-java, maven, xmvn-connector-ivy, xmlstreambuffer, checkstyle, cglib, bean-validation-api, aws-sdk-java, javapackages-tools, ant, scala, osgi-service-log, jmdns, xml-security, super-csv, osgi-service-jdbc, msv, junit5, jsr-311, jersey, itextpdf, httpcomponents-asyncclient, ed25519-java, jnacl, javaparser, picocli, freemarker, extra166y, javaparser, xstream, woodstox-core, uom-lib, unit-api, uncommons-maths, tycho, treelayout, tiger-types, super-csv, stax-ex, stax2-api, sqlite-jdbc, reflectasm, prometheus-simpleclient-java, powermock, paranamer, opennlp, netty3, mybatis, morfologik-stemming, minlog, maven-archetype, mariadb-java-client, logback, kryo, jsonp, jopt-simple, jnr-posix, jnr-constants, jnr-a64asm, jfreechart, jffi, jetty-schemas, jetty-minimal, jeromq, jctools, jcsp, jboss-websocket-1.0-api, jboss-marshalling, jboss-logmanager, jboss-logging, javaewah, jatl, janino, jackson-modules-base, jackson-jaxrs-providers, jackson-datatypes-collections, jackson-dataformat-xml, jackson-dataformats-text, jackson-dataformats-binary, indriya, google-gson, glassfish-websocket-api, glassfish-transaction-api, glassfish-jsp, glassfish-jax-rs-api, glassfish-hk2, glassfish-fastinfoset, felix-scr, felix-gogo-shell, felix-gogo-command, disruptor, apache-commons-ognl, apache-commons-math, apache-commons-csv, antlr4, jettison, sisu, maven

Pol Dellaiera:

Since version 2.6.4, Composer (the PHP package manager) is now fully reproducible. See the work in the corresponding PR.

Testing framework

The Reproducible Builds project operates a comprehensive testing framework (available at tests.reproducible-builds.org) in order to check packages and other artifacts for reproducibility. In September, a number of changes were made by Holger Levsen:

Disable armhf and i386 builds due to Debian bug #1052257. […][…][…][…]

Run diffoscope with a lower ionice priority. […]

Log every build in a simple text file […] and create persistent stamp files when running diffoscope to ease debugging […].

Run schedulers one hour after dinstall again. […]

Temporarily use diffoscope from the host, and not from a schroot running the tested suite. […][…]

Fail the diffoscope distribution test if the diffoscope version cannot be determined. […]

Fix a spelling error in the ‘email to IRC’ gateway. […]

Force (and document) the reconfiguration of all jobs, due to the recent rise of zombies. […][…][…][…]

Deal with a rare condition when killing processes which should not be there. […]

Install the Debian backports kernel in an attempt to address Debian bug #1052257. […][…]

In addition, Mattia Rizzolo fixed a call to diffoscope --version (as suggested by Fay Stegerman on our mailing list) […], worked on an openQA credential issue […] and also made some changes to the machine-readable reproducible metadata, reproducible-tracker.json […]. Lastly, Roland Clobus added instructions for manual configuration of the openQA secrets […].