[bug#28016] [PATCH] Reproducible issue for libjpeg-turbo

[bug#28016] [PATCH] Reproducible issue for libjpeg-turbo

[Z. Ren]

[-- Attachment #1: Type: text/plain, Size: 492 bytes --]

Hi!
While conducting a study inspired by the "reproducible builds" practice [1], we have noticed that the package libjpeg-turbo could not be built reproducibly.

After investigation, we observe that the unreproducibility is caused by the "configure" script, in which the output of `date` is captured. 

The attached patch substitutes the `date` according to the SOUR_DATE_EPOCH.  Once applied, libjpeg-turbo could be built deterministically.

 [1]: https://wiki.debian.org/ReproducibleBuilds

[-- Attachment #2: 0001-reproducible-fix-for-libjpeg-turbo.patch --]
[-- Type: application/octet-stream, Size: 1402 bytes --]

From 17a33f6199d47cde1bd2f76849899fe5d39f763c Mon Sep 17 00:00:00 2001
From: Z. Ren <zren@dlut.edu.cn>
Date: Tue, 8 Aug 2017 14:05:25 +0800
Subject: [PATCH] reproducible fix for libjpeg-turbo

---
 gnu/packages/image.scm | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/image.scm b/gnu/packages/image.scm
index 63e3fa504..854a85ae2 100644
--- a/gnu/packages/image.scm
+++ b/gnu/packages/image.scm
@@ -1122,7 +1122,21 @@ PNG, and performs PNG integrity checks and corrections.")
     (native-inputs
      `(("nasm" ,nasm)))
     (arguments
-     '(#:test-target "test"))
+     '(#:test-target "test"
+       #:modules ((srfi srfi-19)
+                  (guix build utils)
+                  (guix build gnu-build-system))
+       #:phases
+       (modify-phases %standard-phases
+        (add-before 'configure 'reproducible
+         (lambda _
+          (let ((source-date-epoch
+           (time-utc->date
+            (make-time time-utc 0 (string->number
+             (getenv "SOURCE_DATE_EPOCH"))))))
+              (substitute* "configure"
+               (("`date .*`") (date->string source-date-epoch "'{~Y~m~d}'"))
+              )))))))
     (home-page "http://www.libjpeg-turbo.org/")
     (synopsis "SIMD-accelerated JPEG image handling library")
     (description "libjpeg-turbo is a JPEG image codec that accelerates baseline
-- 
2.11.0

[bug#28016] [PATCH] Reproducible issue for libjpeg-turbo

[Z. Ren]

[-- Attachment #1: Type: text/plain, Size: 1350 bytes --]

Hi,

There was a typo (redundant brackets around SOURCE_DATE_EPOCH) in the previous submitted patch, sorry that I'm not very familiar with the Guix system. I attach the modified patch. Following the suggestion of Leo (https://lists.gnu.org/archive/html/help-guix/2017-08/msg00053.html Thanks for the suggestion), I investigate the source code of libjpeg-turbo. It appears that the upstream package supports the argument --with-build-date. Maybe this is more suitable for fixing the unreproducible problem?

Based on this idea, I tried setting configure-flags with SOURCE_DATE_EPOCH. Unfortunately, the flag is not assigned properly with the following snippet:

         #:configure-flags
         (list (string-append "--with-build-date="
          (date->string
           (time-utc->date
            (make-time time-utc 0
             (string->number (getenv "SOURCE_DATE_EPOCH")))) "'~Y~m~d'")))

Is it because the environmental variable SOURCE_DATE_EPOCH is not yet set at the time of #:configure-flags? I read the recipe of other packages, maybe "replace 'configure" is the right solution? Again, sorry that I'm not very familiar with Guix, and haven't come up with the patch yet. However, it would be appreciated if it could be confirmed whether the unreproducibility is caused by the `date` command in the onfigure script.

Best regards,

Ren

[-- Attachment #2: 0001-reproducible.patch --]
[-- Type: application/octet-stream, Size: 1401 bytes --]

From 530bc7b42c91fe17a30a5e4007e381541a095a3e Mon Sep 17 00:00:00 2001
From: Z. Ren <zren@dlut.edu.cn>
Date: Wed, 16 Aug 2017 10:35:40 +0800
Subject: [PATCH] reproducible fix for libjpeg-turbo

---
 gnu/packages/image.scm | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/image.scm b/gnu/packages/image.scm
index 95a4c91b1..e4c2ad620 100644
--- a/gnu/packages/image.scm
+++ b/gnu/packages/image.scm
@@ -1120,7 +1120,21 @@ PNG, and performs PNG integrity checks and corrections.")
     (native-inputs
      `(("nasm" ,nasm)))
     (arguments
-     '(#:test-target "test"))
+     '(#:test-target "test"
+       #:modules ((srfi srfi-19)
+                  (guix build utils)
+                  (guix build gnu-build-system))
+       #:phases
+       (modify-phases %standard-phases
+        (add-before 'configure 'reproducible
+         (lambda _
+          (let ((source-date-epoch
+           (time-utc->date
+            (make-time time-utc 0 (string->number
+             (getenv "SOURCE_DATE_EPOCH"))))))
+              (substitute* "configure"
+               (("`date .*`") (date->string source-date-epoch "'~Y~m~d'"))
+              )))))))
     (home-page "http://www.libjpeg-turbo.org/")
     (synopsis "SIMD-accelerated JPEG image handling library")
     (description "libjpeg-turbo is a JPEG image codec that accelerates baseline
-- 
2.13.2

bug#28016: [PATCH] Reproducible issue for libjpeg-turbo

[Ludovic Courtès]

Hi,

"Z. Ren" <zren@dlut.edu.cn> skribis:

> There was a typo (redundant brackets around SOURCE_DATE_EPOCH) in the previous submitted patch, sorry that I'm not very familiar with the Guix system. I attach the modified patch. Following the suggestion of Leo (https://lists.gnu.org/archive/html/help-guix/2017-08/msg00053.html Thanks for the suggestion), I investigate the source code of libjpeg-turbo. It appears that the upstream package supports the argument --with-build-date. Maybe this is more suitable for fixing the unreproducible problem?
>
> Based on this idea, I tried setting configure-flags with SOURCE_DATE_EPOCH. Unfortunately, the flag is not assigned properly with the following snippet:
>
>          #:configure-flags
>          (list (string-append "--with-build-date="
>           (date->string
>            (time-utc->date
>             (make-time time-utc 0
>              (string->number (getenv "SOURCE_DATE_EPOCH")))) "'~Y~m~d'")))
>
> Is it because the environmental variable SOURCE_DATE_EPOCH is not yet
> set at the time of #:configure-flags?

Yes, that’s correct.

I’ve pushed a simple version that does:

  #:configure-flags (list "--with-build-date=1970-01-01")

It’s good enough because we set SOURCE_DATE_EPOCH to 0 anyway.

Thank you for working on reproducibility issues!

Ludo’.