Set FORCE_SOURCE_DATE=1 by default

Set FORCE_SOURCE_DATE=1 by default

[Vagrant Cascadian]

[-- Attachment #1.1: Type: text/plain, Size: 1897 bytes --]

So, guix sets SOURCE_DATE_EPOCH=1 by default in
guix/build/gnu-build-system.scm, which is great!

This allows guix packages in many cases to build packages reproducibly,
with a curious side-effect that takes us all back to the early 70s in
some corner-cases (or even late 60s, dependent on timezone).

That said, some projects (such as texlive) might be worried about
messing with time too much (I get it, lots of cautionary sci-fi
stories!), and so you *also* need FORCE_SOURCE_DATE=1 to be set in order
to respect SOURCE_DATE_EPOCH.

I am quite biased on the subject and always felt that setting
SOURCE_DATE_EPOCH wouldn't happen by accident and it alone should be
sufficient to say "please use this for your timestamps, really."

I am not aware of use-cases where SOURCE_DATE_EPOCH is intentially set
but FORCE_SOURCE_DATE is intentionally not set... in order to embed the
current time. But that is kind of neither here nor there.


Setting this on a package-by-package basis can be a bit tedious, wastes
precious lines that could be spent on more useful code, and needs to be
done for every existing and new package that might need it, such as:

  964cdd57fa037fec4917ac76725b0a65d47483bc gnu: discrover: Use reproducible timestamps.
  a04a987450908a84fa5fde0caa25a6a50027c73c gnu: itpp: Avoid embedding build dates.

I haven't done a full inventory, but many texlive packages should become
considerably more reproducible with FORCE_SOURE_DATE=1 set, as well as
other packages that use texlive to generate documentation in various
formats.


FORCE_SOURCE_DATE is mentioned in the documentation about
SOURCE_DATE_EPOCH:

  https://reproducible-builds.org/docs/source-date-epoch/


I am making the modest proposal of Guix setting this variable by
default! Patch attached for your perusal.

I'm guessing this would be world-rebuild endeavor, hello core-updates!

Thoughts?


live well,
  vagrant

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-guix-gnu-build-system-Set-FORCE_SOURCE_DATE-in-set-S.patch --]
[-- Type: text/x-diff, Size: 1506 bytes --]

From 7a39330b56934accef14b5e2ac003e211c7c6c5b Mon Sep 17 00:00:00 2001
From: Vagrant Cascadian <vagrant@reproducible-builds.org>
Date: Fri, 10 Jun 2022 16:12:59 -0700
Subject: [PATCH] guix: gnu-build-system: Set FORCE_SOURCE_DATE in
 set-SOURCE-DATE-EPOCH phase.

* guix/build/gnu-build-system.scm (set-SOURCE-DATE-EPOCH): Set
  FORCE_SOURCE_DATE=1. Update URL.
---
 guix/build/gnu-build-system.scm | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm
index d84411c090..42a6041798 100644
--- a/guix/build/gnu-build-system.scm
+++ b/guix/build/gnu-build-system.scm
@@ -56,10 +56,11 @@ (define time-monotonic time-tai))
   (else #t))
 
 (define* (set-SOURCE-DATE-EPOCH #:rest _)
-  "Set the 'SOURCE_DATE_EPOCH' environment variable.  This is used by tools
-that incorporate timestamps as a way to tell them to use a fixed timestamp.
-See https://reproducible-builds.org/specs/source-date-epoch/."
-  (setenv "SOURCE_DATE_EPOCH" "1"))
+  "Set the 'SOURCE_DATE_EPOCH' and 'FORCE_SOURCE_DATE' environment variables.
+This is used by tools that incorporate timestamps as a way to tell them to use
+a fixed timestamp.  See https://reproducible-builds.org/docs/source-date-epoch/."
+  (setenv "SOURCE_DATE_EPOCH" "1")
+  (setenv "FORCE_SOURCE_DATE" "1"))
 
 (define (first-subdirectory directory)
   "Return the file name of the first sub-directory of DIRECTORY or false, when
-- 
2.35.1


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Set FORCE_SOURCE_DATE=1 by default

[Ludovic Courtès]

Hi!

Vagrant Cascadian <vagrant@reproducible-builds.org> skribis:

> From 7a39330b56934accef14b5e2ac003e211c7c6c5b Mon Sep 17 00:00:00 2001
> From: Vagrant Cascadian <vagrant@reproducible-builds.org>
> Date: Fri, 10 Jun 2022 16:12:59 -0700
> Subject: [PATCH] guix: gnu-build-system: Set FORCE_SOURCE_DATE in
>  set-SOURCE-DATE-EPOCH phase.
>
> * guix/build/gnu-build-system.scm (set-SOURCE-DATE-EPOCH): Set
>   FORCE_SOURCE_DATE=1. Update URL.

[...]

>  (define* (set-SOURCE-DATE-EPOCH #:rest _)
> -  "Set the 'SOURCE_DATE_EPOCH' environment variable.  This is used by tools
> -that incorporate timestamps as a way to tell them to use a fixed timestamp.
> -See https://reproducible-builds.org/specs/source-date-epoch/."
> -  (setenv "SOURCE_DATE_EPOCH" "1"))
> +  "Set the 'SOURCE_DATE_EPOCH' and 'FORCE_SOURCE_DATE' environment variables.
> +This is used by tools that incorporate timestamps as a way to tell them to use
> +a fixed timestamp.  See https://reproducible-builds.org/docs/source-date-epoch/."
> +  (setenv "SOURCE_DATE_EPOCH" "1")
> +  (setenv "FORCE_SOURCE_DATE" "1"))

I’d mention above that FORCE_SOURCE_DATE is honored exclusively by
TeX Live.

It’s a bit of a bummer that we have to do that here, but as you point
out, TeX Live can be used pretty much in any package and we’d rather not
track every possible issue by hand.

I think it can go to ‘core-updates’.

Thanks!

Ludo’.

Re: Set FORCE_SOURCE_DATE=1 by default

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 2035 bytes --]

On 2022-06-15, Ludovic Courtès wrote:
> Vagrant Cascadian <vagrant@reproducible-builds.org> skribis:
>
>> From 7a39330b56934accef14b5e2ac003e211c7c6c5b Mon Sep 17 00:00:00 2001
>> From: Vagrant Cascadian <vagrant@reproducible-builds.org>
>> Date: Fri, 10 Jun 2022 16:12:59 -0700
>> Subject: [PATCH] guix: gnu-build-system: Set FORCE_SOURCE_DATE in
>>  set-SOURCE-DATE-EPOCH phase.
>>
>> * guix/build/gnu-build-system.scm (set-SOURCE-DATE-EPOCH): Set
>>   FORCE_SOURCE_DATE=1. Update URL.
>
> [...]
>
>>  (define* (set-SOURCE-DATE-EPOCH #:rest _)
>> -  "Set the 'SOURCE_DATE_EPOCH' environment variable.  This is used by tools
>> -that incorporate timestamps as a way to tell them to use a fixed timestamp.
>> -See https://reproducible-builds.org/specs/source-date-epoch/."
>> -  (setenv "SOURCE_DATE_EPOCH" "1"))
>> +  "Set the 'SOURCE_DATE_EPOCH' and 'FORCE_SOURCE_DATE' environment variables.
>> +This is used by tools that incorporate timestamps as a way to tell them to use
>> +a fixed timestamp.  See https://reproducible-builds.org/docs/source-date-epoch/."
>> +  (setenv "SOURCE_DATE_EPOCH" "1")
>> +  (setenv "FORCE_SOURCE_DATE" "1"))
>
> I’d mention above that FORCE_SOURCE_DATE is honored exclusively by
> TeX Live.

It's the only *known* case so far, but in theory could be others, even
though it is strongly discouraged in the reproducible builds
documentation. That tripped me up a bit in how to word it, which is why
I changed the URL from the specification which doesn't mention
FORCE_SOURCE_DATE on principle, to the documentation URL which does, out
of pragmatism.

Suggestions for better wording welcome!


> It’s a bit of a bummer that we have to do that here, but as you point
> out, TeX Live can be used pretty much in any package and we’d rather not
> track every possible issue by hand.
>
> I think it can go to ‘core-updates’.

Wheee!

Ok, will wait a bit for some improved wording suggestions from the
community at large...


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Set FORCE_SOURCE_DATE=1 by default

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 3386 bytes --]

On 2022-06-15, Ludovic Courtès wrote:
> Vagrant Cascadian <vagrant@reproducible-builds.org> skribis:
>> From 7a39330b56934accef14b5e2ac003e211c7c6c5b Mon Sep 17 00:00:00 2001
>> From: Vagrant Cascadian <vagrant@reproducible-builds.org>
>> Date: Fri, 10 Jun 2022 16:12:59 -0700
>> Subject: [PATCH] guix: gnu-build-system: Set FORCE_SOURCE_DATE in
>>  set-SOURCE-DATE-EPOCH phase.
>>
>> * guix/build/gnu-build-system.scm (set-SOURCE-DATE-EPOCH): Set
>>   FORCE_SOURCE_DATE=1. Update URL.
>
> [...]
>
>>  (define* (set-SOURCE-DATE-EPOCH #:rest _)
>> -  "Set the 'SOURCE_DATE_EPOCH' environment variable.  This is used by tools
>> -that incorporate timestamps as a way to tell them to use a fixed timestamp.
>> -See https://reproducible-builds.org/specs/source-date-epoch/."
>> -  (setenv "SOURCE_DATE_EPOCH" "1"))
>> +  "Set the 'SOURCE_DATE_EPOCH' and 'FORCE_SOURCE_DATE' environment variables.
>> +This is used by tools that incorporate timestamps as a way to tell them to use
>> +a fixed timestamp.  See https://reproducible-builds.org/docs/source-date-epoch/."
>> +  (setenv "SOURCE_DATE_EPOCH" "1")
>> +  (setenv "FORCE_SOURCE_DATE" "1"))
>
> I’d mention above that FORCE_SOURCE_DATE is honored exclusively by
> TeX Live.

I am having trouble explaining it, partly because I don't really believe
in it and kind of want to just leave that up to the URL... that said:

diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm
index d84411c090..d69f8c42fd 100644
--- a/guix/build/gnu-build-system.scm
+++ b/guix/build/gnu-build-system.scm
@@ -56,10 +56,13 @@ (define time-monotonic time-tai))
   (else #t))
 
 (define* (set-SOURCE-DATE-EPOCH #:rest _)
-  "Set the 'SOURCE_DATE_EPOCH' environment variable.  This is used by tools
-that incorporate timestamps as a way to tell them to use a fixed timestamp.
-See https://reproducible-builds.org/specs/source-date-epoch/."
-  (setenv "SOURCE_DATE_EPOCH" "1"))
+  "Set the 'SOURCE_DATE_EPOCH' and 'FORCE_SOURCE_DATE' environment variables.
+This is used by tools that incorporate timestamps as a way to tell them to use
+a fixed timestamp.  Setting 'FORCE_SOURCE_DATE' is needed in order for TeX
+Live to respect 'SOURCE_DATE_EPOCH'. See
+https://reproducible-builds.org/docs/source-date-epoch/."
+  (setenv "SOURCE_DATE_EPOCH" "1")
+  (setenv "FORCE_SOURCE_DATE" "1"))
 
 (define (first-subdirectory directory)
   "Return the file name of the first sub-directory of DIRECTORY or false, when


Not really happy with it ... both variables are basically needed to make
SOURCE_DATE_EPOCH effective, and it's not clear to me what it really
adds to the statement to call out TeX Live explicitly... especially
given that other tools *might* actually do the same... even though I
sure hope we can contain the problem to TeX Live.

Would renaming it to set-SOURCE-DATE-EPOCH-and-FORCE-SOURCE-DATE add
anything? or come up with a generic name? or having both
set-SOURCE-DATE-EPOCH and set-FORCE-SOURCE-DATE as separate functions?
Or a more generic description?


> It’s a bit of a bummer that we have to do that here, but as you point
> out, TeX Live can be used pretty much in any package and we’d rather not
> track every possible issue by hand.

Agreed.


> I think it can go to ‘core-updates’.

I hope so too!


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Set FORCE_SOURCE_DATE=1 by default

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 2613 bytes --]

On 2022-06-21, Vagrant Cascadian wrote:
> On 2022-06-15, Ludovic Courtès wrote:
>> Vagrant Cascadian <vagrant@reproducible-builds.org> skribis:
>>> From 7a39330b56934accef14b5e2ac003e211c7c6c5b Mon Sep 17 00:00:00 2001
>>> From: Vagrant Cascadian <vagrant@reproducible-builds.org>
>>> Date: Fri, 10 Jun 2022 16:12:59 -0700
>>> Subject: [PATCH] guix: gnu-build-system: Set FORCE_SOURCE_DATE in
>>>  set-SOURCE-DATE-EPOCH phase.
>>>
>>> * guix/build/gnu-build-system.scm (set-SOURCE-DATE-EPOCH): Set
>>>   FORCE_SOURCE_DATE=1. Update URL.
>>
>> [...]
>>
>>>  (define* (set-SOURCE-DATE-EPOCH #:rest _)
>>> -  "Set the 'SOURCE_DATE_EPOCH' environment variable.  This is used by tools
>>> -that incorporate timestamps as a way to tell them to use a fixed timestamp.
>>> -See https://reproducible-builds.org/specs/source-date-epoch/."
>>> -  (setenv "SOURCE_DATE_EPOCH" "1"))
>>> +  "Set the 'SOURCE_DATE_EPOCH' and 'FORCE_SOURCE_DATE' environment variables.
>>> +This is used by tools that incorporate timestamps as a way to tell them to use
>>> +a fixed timestamp.  See https://reproducible-builds.org/docs/source-date-epoch/."
>>> +  (setenv "SOURCE_DATE_EPOCH" "1")
>>> +  (setenv "FORCE_SOURCE_DATE" "1"))
>>
>> I’d mention above that FORCE_SOURCE_DATE is honored exclusively by
>> TeX Live.

Still not terribly happy with it, but I think I can cope with this
updated version:

diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm
index d84411c090..cbfbd2a209 100644
--- a/guix/build/gnu-build-system.scm
+++ b/guix/build/gnu-build-system.scm
@@ -56,10 +56,14 @@ (define time-monotonic time-tai))
   (else #t))
 
 (define* (set-SOURCE-DATE-EPOCH #:rest _)
-  "Set the 'SOURCE_DATE_EPOCH' environment variable.  This is used by tools
-that incorporate timestamps as a way to tell them to use a fixed timestamp.
-See https://reproducible-builds.org/specs/source-date-epoch/."
-  (setenv "SOURCE_DATE_EPOCH" "1"))
+  "Set the 'SOURCE_DATE_EPOCH' and 'FORCE_SOURCE_DATE' environment variables.
+This is used by tools that incorporate timestamps as a way to tell them to use
+a fixed timestamp.  Setting 'FORCE_SOURCE_DATE' is needed in order for TeX
+Live to respect 'SOURCE_DATE_EPOCH', which is used to generate documentation
+in various packages. See
+https://reproducible-builds.org/docs/source-date-epoch/."
+  (setenv "SOURCE_DATE_EPOCH" "1")
+  (setenv "FORCE_SOURCE_DATE" "1"))
 
 (define (first-subdirectory directory)
   "Return the file name of the first sub-directory of DIRECTORY or false, when


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Set FORCE_SOURCE_DATE=1 by default

[Maxim Cournoyer]

Hi Vagrant,

Vagrant Cascadian <vagrant@reproducible-builds.org> writes:

> So, guix sets SOURCE_DATE_EPOCH=1 by default in
> guix/build/gnu-build-system.scm, which is great!
>
> This allows guix packages in many cases to build packages reproducibly,
> with a curious side-effect that takes us all back to the early 70s in
> some corner-cases (or even late 60s, dependent on timezone).
>
> That said, some projects (such as texlive) might be worried about
> messing with time too much (I get it, lots of cautionary sci-fi
> stories!), and so you *also* need FORCE_SOURCE_DATE=1 to be set in order
> to respect SOURCE_DATE_EPOCH.

That seems ridiculous.  Has anyone tried getting in touch with them to
get their arguments about why inventing another variable that means the
same thing was necessary?

I'd much prefer challenging that stance than "endorsing" it in Guix :-).
I think it'd be OK to reluctantly add it in as a stop-gap fix in Guix,
but *only* after opening an issue to discuss it upstream and linking to
that issue in Guix.

Thanks,

Maxim

Re: Set FORCE_SOURCE_DATE=1 by default

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 1994 bytes --]

On 2022-06-21, Maxim Cournoyer wrote:
> Vagrant Cascadian <vagrant@reproducible-builds.org> writes:
>> So, guix sets SOURCE_DATE_EPOCH=1 by default in
>> guix/build/gnu-build-system.scm, which is great!
>>
>> This allows guix packages in many cases to build packages reproducibly,
>> with a curious side-effect that takes us all back to the early 70s in
>> some corner-cases (or even late 60s, dependent on timezone).
>>
>> That said, some projects (such as texlive) might be worried about
>> messing with time too much (I get it, lots of cautionary sci-fi
>> stories!), and so you *also* need FORCE_SOURCE_DATE=1 to be set in order
>> to respect SOURCE_DATE_EPOCH.
>
> That seems ridiculous.  Has anyone tried getting in touch with them to
> get their arguments about why inventing another variable that means the
> same thing was necessary?

Yes, there were some fairly long threads about it and I have little hope
that revisiting it would change much; it was originally implemented as a
texlive specific variable, which was changed to the FORCE_SOURCE_DATE
variable to at least avoid the danger of every project inventing their
own name-brand variables...


> I'd much prefer challenging that stance than "endorsing" it in Guix :-).
> I think it'd be OK to reluctantly add it in as a stop-gap fix in Guix,
> but *only* after opening an issue to discuss it upstream and linking to
> that issue in Guix.

I get it. I really do. It kind of grates at me every time I think about
this.

I know it really is not great and seems quite suboptimal to me, but I
don't personally think rehashing the arguments will be a productive use
of time for anyone...

I think the pragmatism of making more packages reproducible by conceding
to set FORCE_SOURCE_DATE is the appropriate way forward; I agree it
feels silly or even maybe would go so far as to say a bit "wrong".

If someone wanted to make yet another environment variable, that would
seem a line in the sand worth drawing...


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Set FORCE_SOURCE_DATE=1 by default

[Ludovic Courtès]

Vagrant Cascadian <vagrant@reproducible-builds.org> skribis:

> On 2022-06-21, Maxim Cournoyer wrote:

[...]

>> I'd much prefer challenging that stance than "endorsing" it in Guix :-).
>> I think it'd be OK to reluctantly add it in as a stop-gap fix in Guix,
>> but *only* after opening an issue to discuss it upstream and linking to
>> that issue in Guix.
>
> I get it. I really do. It kind of grates at me every time I think about
> this.

:-)

Another option would be to patch TeX Live, assuming the
FORCE_SOURCE_DATE bit is well isolated in a single file.  Would that
seem reasonable/feasible/desirable to you, Vagrant?

Ludo’.

Re: Set FORCE_SOURCE_DATE=1 by default

[Maxim Cournoyer]

Hi,

Vagrant Cascadian <vagrant@reproducible-builds.org> writes:

> On 2022-06-21, Maxim Cournoyer wrote:
>> Vagrant Cascadian <vagrant@reproducible-builds.org> writes:
>>> So, guix sets SOURCE_DATE_EPOCH=1 by default in
>>> guix/build/gnu-build-system.scm, which is great!
>>>
>>> This allows guix packages in many cases to build packages reproducibly,
>>> with a curious side-effect that takes us all back to the early 70s in
>>> some corner-cases (or even late 60s, dependent on timezone).
>>>
>>> That said, some projects (such as texlive) might be worried about
>>> messing with time too much (I get it, lots of cautionary sci-fi
>>> stories!), and so you *also* need FORCE_SOURCE_DATE=1 to be set in order
>>> to respect SOURCE_DATE_EPOCH.
>>
>> That seems ridiculous.  Has anyone tried getting in touch with them to
>> get their arguments about why inventing another variable that means the
>> same thing was necessary?
>
> Yes, there were some fairly long threads about it and I have little hope
> that revisiting it would change much; it was originally implemented as a
> texlive specific variable, which was changed to the FORCE_SOURCE_DATE
> variable to at least avoid the danger of every project inventing their
> own name-brand variables...
>
>
>> I'd much prefer challenging that stance than "endorsing" it in Guix :-).
>> I think it'd be OK to reluctantly add it in as a stop-gap fix in Guix,
>> but *only* after opening an issue to discuss it upstream and linking to
>> that issue in Guix.
>
> I get it. I really do. It kind of grates at me every time I think about
> this.
>
> I know it really is not great and seems quite suboptimal to me, but I
> don't personally think rehashing the arguments will be a productive use
> of time for anyone...
>
> I think the pragmatism of making more packages reproducible by conceding
> to set FORCE_SOURCE_DATE is the appropriate way forward; I agree it
> feels silly or even maybe would go so far as to say a bit "wrong".

Perhaps to show our stand here we could patch our copy of pdftex with
's/FORCE_SOURCE_DATE/SOURCE_DATE_EPOCH/', lest we end up with a grocery
list of *SOURCE_DATE* variable variants.  Even reproducible-builds.org
discourage its use (directed at upstream rather than downstream, but
still) [0]:

    If for some reason you’re still conflicted on suddenly changing the
    meaning of your “now()” function and desire another switch other than
    SOURCE_DATE_EPOCH being set or not, the texlive project came up with the
    variable FORCE_SOURCE_DATE; when that environment variable is set to 1
    cases that wouldn’t normally obey SOURCE_DATE_EPOCH will do. We strongly
    discourage the usage of such variable; SOURCE_DATE_EPOCH is meant to be
    already a flag forcing a particular timestamp to be used.

It'd still be nice to have the link to the upstream discussion that led
using a FORCE_SOURCE_DATE variable mentioned in a comment above where
we'd do this trivial substitution; do you have a link to it?  I couldn't
find it.

Thanks,

Maxim

[0]  https://reproducible-builds.org/docs/source-date-epoch/

Re: Set FORCE_SOURCE_DATE=1 by default

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 1180 bytes --]

On 2022-06-22, Ludovic Courtès wrote:
> Vagrant Cascadian <vagrant@reproducible-builds.org> skribis:
>
>> On 2022-06-21, Maxim Cournoyer wrote:
>
> [...]
>
>>> I'd much prefer challenging that stance than "endorsing" it in Guix :-).
>>> I think it'd be OK to reluctantly add it in as a stop-gap fix in Guix,
>>> but *only* after opening an issue to discuss it upstream and linking to
>>> that issue in Guix.
>>
>> I get it. I really do. It kind of grates at me every time I think about
>> this.
>
> :-)
>
> Another option would be to patch TeX Live, assuming the
> FORCE_SOURCE_DATE bit is well isolated in a single file.  Would that
> seem reasonable/feasible/desirable to you, Vagrant?

Only problem I see is indefinitely maintaining a patch to TeX Live that
upstream is unlikely to take ... but other than that, this sounds great!

We'd also get a clearer idea of which packages are affected, rather than
setting it globally and rebuilding everything...

I'll also try to dig up at least one of the threads where there was some
discussion with upstream... hopefully without getting too caught up in
that old rabbit hole.

live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Time namespace for build sandbox (was Re: Set FORCE_SOURCE_DATE=1 by default)

[Zhu Zihao]

[-- Attachment #1: Type: text/plain, Size: 375 bytes --]


Can we make some experiments on the Linux time namespace for build
sandbox?

This can mock the real system with our desired value, maybe a good
solution for the reproduciblity on Linux machine.

Ref: https://man7.org/linux/man-pages/man7/time_namespaces.7.html
-- 
Retrieve my PGP public key:

  gpg --recv-keys D47A9C8B2AE3905B563D9135BE42B352A9F6821F

Zihao

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]

Re: Time namespace for build sandbox (was Re: Set FORCE_SOURCE_DATE=1 by default)

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 562 bytes --]

On 2022-06-22, Zhu Zihao wrote:
> Can we make some experiments on the Linux time namespace for build
> sandbox?
>
> This can mock the real system with our desired value, maybe a good
> solution for the reproduciblity on Linux machine.
>
> Ref: https://man7.org/linux/man-pages/man7/time_namespaces.7.html

My understanding is this only affects the time reported by things like
"uptime" and not "date" ... would be happy to have it demonstrated
otherwise!

In theory support could be added in the upstream kernel for the
real-time-clock...


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Time namespace for build sandbox (was Re: Set FORCE_SOURCE_DATE=1 by default)

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 701 bytes --]

Zhu Zihao schreef op wo 22-06-2022 om 23:16 [+0800]:
> 
> Can we make some experiments on the Linux time namespace for build
> sandbox?
> 
> This can mock the real system with our desired value, maybe a good
> solution for the reproduciblity on Linux machine.
> 
> Ref: https://man7.org/linux/man-pages/man7/time_namespaces.7.html

There are some comments on time namespace at
<https://issues.guix.gnu.org/44559#3>.  Unfortunately, it only accepts
offsets in a limited range, so you can't just reset the clock to 1970.

There was also some talk on avoiding the no-CLOCK_REALTIME problem at
<https://logs.guix.gnu.org/guix/2022-06-08.log> with syscall rewriting.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Set FORCE_SOURCE_DATE=1 by default

[Maxim Cournoyer]

Hi Vagrant,

Vagrant Cascadian <vagrant@reproducible-builds.org> writes:

> On 2022-06-22, Ludovic Courtès wrote:
>> Vagrant Cascadian <vagrant@reproducible-builds.org> skribis:
>>
>>> On 2022-06-21, Maxim Cournoyer wrote:
>>
>> [...]
>>
>>>> I'd much prefer challenging that stance than "endorsing" it in Guix :-).
>>>> I think it'd be OK to reluctantly add it in as a stop-gap fix in Guix,
>>>> but *only* after opening an issue to discuss it upstream and linking to
>>>> that issue in Guix.
>>>
>>> I get it. I really do. It kind of grates at me every time I think about
>>> this.
>>
>> :-)
>>
>> Another option would be to patch TeX Live, assuming the
>> FORCE_SOURCE_DATE bit is well isolated in a single file.  Would that
>> seem reasonable/feasible/desirable to you, Vagrant?
>
> Only problem I see is indefinitely maintaining a patch to TeX Live that
> upstream is unlikely to take ... but other than that, this sounds great!

I wouldn't expect this simple substitution should be too difficult to
maintain :-).

> We'd also get a clearer idea of which packages are affected, rather than
> setting it globally and rebuilding everything...

I don't think it's of critical importance; what we care about here is
having the software reproducible, not knowing exactly which esoteric
environment variable enabled it.

> I'll also try to dig up at least one of the threads where there was some
> discussion with upstream... hopefully without getting too caught up in
> that old rabbit hole.

That'd be nice.

Thanks!

Maxim

Re: Set FORCE_SOURCE_DATE=1 by default

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 2454 bytes --]

On 2022-06-22, Maxim Cournoyer wrote:
> Vagrant Cascadian <vagrant@reproducible-builds.org> writes:
>> On 2022-06-21, Maxim Cournoyer wrote:
>>> Vagrant Cascadian <vagrant@reproducible-builds.org> writes:
>>>> That said, some projects (such as texlive) might be worried about
>>>> messing with time too much (I get it, lots of cautionary sci-fi
>>>> stories!), and so you *also* need FORCE_SOURCE_DATE=1 to be set in order
>>>> to respect SOURCE_DATE_EPOCH.
>>>
>>> That seems ridiculous.  Has anyone tried getting in touch with them to
>>> get their arguments about why inventing another variable that means the
>>> same thing was necessary?
>>
>> Yes, there were some fairly long threads about it and I have little hope
>> that revisiting it would change much; it was originally implemented as a
>> texlive specific variable, which was changed to the FORCE_SOURCE_DATE
>> variable to at least avoid the danger of every project inventing their
>> own name-brand variables...

I have tracked it down to these threads:

  https://www.tug.org/pipermail/tex-k/2016-May/thread.html#2691
  https://www.tug.org/pipermail/tex-k/2016-May/thread.html#2712
  https://www.tug.org/pipermail/tex-k/2016-June/thread.html#2721


>>> I'd much prefer challenging that stance than "endorsing" it in Guix :-).
>>> I think it'd be OK to reluctantly add it in as a stop-gap fix in Guix,
>>> but *only* after opening an issue to discuss it upstream and linking to
>>> that issue in Guix.
...
>> I think the pragmatism of making more packages reproducible by conceding
>> to set FORCE_SOURCE_DATE is the appropriate way forward; I agree it
>> feels silly or even maybe would go so far as to say a bit "wrong".
>
> Perhaps to show our stand here we could patch our copy of pdftex with
> 's/FORCE_SOURCE_DATE/SOURCE_DATE_EPOCH/', lest we end up with a grocery
> list of *SOURCE_DATE* variable variants.

Sure, with some technical details fixed up, as I think they are
functionally different, in that FORCE_SOURCE_DATE is a boolean, and
SOURCE_DATE_EPOCH is an integer, though ... Guix sets
SOURCE_DATE_EPOCH=1 ... so it might just work by dumb luck! Though There
may be some rare packages that need SOURCE_DATE_EPOCH to be some larger
value...  "It can't possibly be 1970, this program was first written in
2002, there must be some error, failing build..."

At any rate, if diverging from upstream Tex Live is how Guix wants to
handle this, I'm all for it!

live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Set FORCE_SOURCE_DATE=1 by default

[Maxim Cournoyer]

Hi Vagrant,

[...]

>> Perhaps to show our stand here we could patch our copy of pdftex with
>> 's/FORCE_SOURCE_DATE/SOURCE_DATE_EPOCH/', lest we end up with a grocery
>> list of *SOURCE_DATE* variable variants.
>
> Sure, with some technical details fixed up, as I think they are
> functionally different, in that FORCE_SOURCE_DATE is a boolean, and
> SOURCE_DATE_EPOCH is an integer, though ... Guix sets
> SOURCE_DATE_EPOCH=1 ... so it might just work by dumb luck! Though There
> may be some rare packages that need SOURCE_DATE_EPOCH to be some larger
> value...  "It can't possibly be 1970, this program was first written in
> 2002, there must be some error, failing build..."
>
> At any rate, if diverging from upstream Tex Live is how Guix wants to
> handle this, I'm all for it!

Seems we have a small consensus here (Ludo, you and myself); would you
like to give it a try?

Thanks,

Maxim

Re: Set FORCE_SOURCE_DATE=1 by default

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 2497 bytes --]

On 2022-06-23, Maxim Cournoyer wrote:
>>> Perhaps to show our stand here we could patch our copy of pdftex with
>>> 's/FORCE_SOURCE_DATE/SOURCE_DATE_EPOCH/', lest we end up with a grocery
>>> list of *SOURCE_DATE* variable variants.
>>
>> Sure, with some technical details fixed up, as I think they are
>> functionally different, in that FORCE_SOURCE_DATE is a boolean, and
>> SOURCE_DATE_EPOCH is an integer, though ... Guix sets
>> SOURCE_DATE_EPOCH=1 ... so it might just work by dumb luck! Though There
>> may be some rare packages that need SOURCE_DATE_EPOCH to be some larger
>> value...  "It can't possibly be 1970, this program was first written in
>> 2002, there must be some error, failing build..."
>>
>> At any rate, if diverging from upstream Tex Live is how Guix wants to
>> handle this, I'm all for it!
>
> Seems we have a small consensus here (Ludo, you and myself); would you
> like to give it a try?

My first naive attempt appeared to build, but failed lots of test
suites... so that clearly needs some more work!

diff --git a/gnu/packages/tex.scm b/gnu/packages/tex.scm
index f6b4c25595..e8c99d202b 100644
--- a/gnu/packages/tex.scm
+++ b/gnu/packages/tex.scm
@@ -395,6 +395,12 @@ (define-public texlive-bin
                      (("srcdir/tests/pprecA-0.ind pprecA-0.ind1 \\|\\| exit 1")
                       "srcdir/tests/pprecA-0.ind pprecA-0.ind1 || exit 77")))))
              '())
+         (add-after 'unpack 'default-to-force-source-date
+           ;; https://lists.gnu.org/archive/html/guix-devel/2022-06/msg00330.html
+           (lambda _
+             ;; texk/web2c/lib/texmfmp.c:  string sde_texprim = getenv ("FORCE_SOURCE_DATE");
+             (substitute* "texk/web2c/lib/texmfmp.c"
+                (("getenv ..FORCE_SOURCE_DATE..") "1"))))
          (add-after 'unpack 'unpack-texlive-extra
            (lambda* (#:key inputs #:allow-other-keys)
              (mkdir "texlive-extra")


No idea if this approach would have any side-effects ... there is some
code that checks if both FORCE_SOURCE_DATE and SOURCE_DATE_EPOCH are set
... but i *think* this is the only spot where it directly checks if
FORCE_SOURCE_DATE is set. It would also be good to support
FORCE_SOURCE_DATE=0 if there is in fact some real-world use-case with
SOURCE_DATE_EPOCH is set and FORCE_SOURCE_DATE should be "0".

I'll try with tests disabled to see if it at least fixes the issue in
the two packages (itpp, discrover) that manually set FORCE_SOURCE_DATE.


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Set FORCE_SOURCE_DATE=1 by default

[Ludovic Courtès]

Hi!

Vagrant Cascadian <vagrant@reproducible-builds.org> skribis:

> --- a/gnu/packages/tex.scm
> +++ b/gnu/packages/tex.scm
> @@ -395,6 +395,12 @@ (define-public texlive-bin
>                       (("srcdir/tests/pprecA-0.ind pprecA-0.ind1 \\|\\| exit 1")
>                        "srcdir/tests/pprecA-0.ind pprecA-0.ind1 || exit 77")))))
>               '())
> +         (add-after 'unpack 'default-to-force-source-date
> +           ;; https://lists.gnu.org/archive/html/guix-devel/2022-06/msg00330.html
> +           (lambda _
> +             ;; texk/web2c/lib/texmfmp.c:  string sde_texprim = getenv ("FORCE_SOURCE_DATE");
> +             (substitute* "texk/web2c/lib/texmfmp.c"
> +                (("getenv ..FORCE_SOURCE_DATE..") "1"))))

That looks reasonable to me.  Let us know how your testing goes!

Ludo’.

Re: Set FORCE_SOURCE_DATE=1 by default

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 1112 bytes --]

On 2022-07-04, Ludovic Courtès wrote:
> Vagrant Cascadian <vagrant@reproducible-builds.org> skribis:
>> --- a/gnu/packages/tex.scm
>> +++ b/gnu/packages/tex.scm
>> @@ -395,6 +395,12 @@ (define-public texlive-bin
>>                       (("srcdir/tests/pprecA-0.ind pprecA-0.ind1 \\|\\| exit 1")
>>                        "srcdir/tests/pprecA-0.ind pprecA-0.ind1 || exit 77")))))
>>               '())
>> +         (add-after 'unpack 'default-to-force-source-date
>> +           ;; https://lists.gnu.org/archive/html/guix-devel/2022-06/msg00330.html
>> +           (lambda _
>> +             ;; texk/web2c/lib/texmfmp.c:  string sde_texprim = getenv ("FORCE_SOURCE_DATE");
>> +             (substitute* "texk/web2c/lib/texmfmp.c"
>> +                (("getenv ..FORCE_SOURCE_DATE..") "1"))))
>
> That looks reasonable to me.  Let us know how your testing goes!

Reasonable, perhaps, but it failed to actually work when I tried it last
month...

Will give it another shot with fresh eyes, although if anyone else wants
to give it a shot, that would be nice too! :)

live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]