* bug#44112: SBCL is not reproducible @ 2020-10-21 8:12 ` zimoun 2020-10-21 12:41 ` Guillaume Le Vaillant 2022-11-25 17:22 ` bug#44112: Hendursaga via Bug reports for GNU Guix 0 siblings, 2 replies; 15+ messages in thread From: zimoun @ 2020-10-21 8:12 UTC (permalink / raw) To: 44112; +Cc: Pierre Neidhardt Dear, Using Guix 58af4c9, the package ’sbcl’ seems not-reproducible. The output of “guix challenge” is: --8<---------------cut here---------------start------------->8--- differing files: /lib/sbcl/contrib/sb-rt.fasl /lib/sbcl/contrib/uiop.fasl /lib/sbcl/contrib/sb-cover.fasl /lib/sbcl/contrib/sb-capstone.fasl /lib/sbcl/contrib/sb-executable.fasl /lib/sbcl/contrib/sb-bsd-sockets.fasl /lib/sbcl/contrib/sb-simple-streams.fasl /lib/sbcl/contrib/sb-queue.fasl /lib/sbcl/contrib/sb-gmp.fasl /lib/sbcl/contrib/sb-aclrepl.fasl /lib/sbcl/contrib/sb-sprof.fasl /lib/sbcl/contrib/sb-grovel.fasl /lib/sbcl/contrib/sb-posix.fasl /lib/sbcl/contrib/asdf.fasl /lib/sbcl/contrib/sb-mpfr.fasl /lib/sbcl/contrib/sb-cltl2.fasl /lib/sbcl/contrib/sb-concurrency.fasl /lib/sbcl/contrib/sb-md5.fasl /lib/sbcl/contrib/sb-introspect.fasl /lib/sbcl/contrib/sb-rotate-byte.fasl /lib/sbcl/sbcl.core /bin/sbcl /share/sbcl/src/runtime/coreparse.o /share/sbcl/src/runtime/sbcl 1 store items were analyzed: - 0 (0.0%) were identical - 1 (100.0%) differed - 0 (0.0%) were inconclusive --8<---------------cut here---------------end--------------->8--- which is confirmed by the experimental Data Service webpage: https://data.guix-patches.cbaines.net/revision/4e0498c4d3f5ef03ac5afa4a3e570205c10c28c5/package-derivation-outputs?search_query=sbcl&output_consistency=not-matching&system=x86_64-linux&target=none&after_path=&limit_results=&all_results=on I do not know if the patches in ’staging’ will fix this. Note that this issue does not imply that the build system ’asdf-build-system/sbcl’ is or will be not reproducible. However, this issues cuts any Diverse Double Compiling attempt. All the best, simon ^ permalink raw reply [flat|nested] 15+ messages in thread
* bug#44112: SBCL is not reproducible 2020-10-21 8:12 ` bug#44112: SBCL is not reproducible zimoun @ 2020-10-21 12:41 ` Guillaume Le Vaillant 2020-10-21 13:37 ` zimoun ` (2 more replies) 2022-11-25 17:22 ` bug#44112: Hendursaga via Bug reports for GNU Guix 1 sibling, 3 replies; 15+ messages in thread From: Guillaume Le Vaillant @ 2020-10-21 12:41 UTC (permalink / raw) To: zimoun; +Cc: 44112, Pierre Neidhardt [-- Attachment #1: Type: text/plain, Size: 1558 bytes --] zimoun <zimon.toutoune@gmail.com> skribis: > Using Guix 58af4c9, the package ’sbcl’ seems not-reproducible. > > [...] > > I do not know if the patches in ’staging’ will fix this. > > Note that this issue does not imply that the build system > ’asdf-build-system/sbcl’ is or will be not reproducible. However, this > issues cuts any Diverse Double Compiling attempt. IIRC, when compiling, SBCL puts the timestamp of the source file in the compiled file. It's not a problem when just doing basic compilation because the source files' timestamps have been set to "1970-01-01". However, some packages generate some source files at build time, usually containing things like data type sizes fetched from system header in order to use C libraries with FFI. The timestamp of a generated file is the current time, therefore the build is not reproducible. IIRC, SBCL itself is built in 2 stages. First its core is compiled using another Common Lisp implementation (currently clisp in Guix), then the complete SBCL is compiled using the core compiled in stage 1. There is probably also an embedded timestamp issue here (coming for clisp, from SBCL, or both) causing the reproducibility issue. The patches currently in the staging branch don't have any effect on the generation of source files or on the format of the compiled files, so they will not help with this issue. Removing this source file timestamp from compiled files would simplify things. Maybe nothing really depends on it and it would be possible... [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 247 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* bug#44112: SBCL is not reproducible 2020-10-21 12:41 ` Guillaume Le Vaillant @ 2020-10-21 13:37 ` zimoun 2020-10-21 13:43 ` zimoun 2020-10-21 14:32 ` Pierre Neidhardt 2021-01-19 16:48 ` zimoun 2 siblings, 1 reply; 15+ messages in thread From: zimoun @ 2020-10-21 13:37 UTC (permalink / raw) To: Guillaume Le Vaillant; +Cc: 44112, Pierre Neidhardt Hi, Thank you for the explanations. On Wed, 21 Oct 2020 at 14:42, Guillaume Le Vaillant <glv@posteo.net> wrote: > However, some packages generate some source files at build time, usually > containing things like data type sizes fetched from system header in > order to use C libraries with FFI. The timestamp of a generated file > is the current time, therefore the build is not reproducible. This is an issue. Do you think it is affordable to fix these timestamps to 1970-01-01? > IIRC, SBCL itself is built in 2 stages. First its core is compiled > using another Common Lisp implementation (currently clisp in Guix), then > the complete SBCL is compiled using the core compiled in stage 1. There > is probably also an embedded timestamp issue here (coming for clisp, > from SBCL, or both) causing the reproducibility issue. Yes. But I have replaced this "clisp" by ECL or by SBCL itself. Still unreproducible. Out-of-scope with this bug report, my aim is to have a fixed point: - first compile SBCL with CLISP (using the 2 stages you describe): produce SBCL-A - second recompile SBCL with the previous SBCL-A (again using the 2 stages): produce SBCL-B - third recompile SBCL with the previous SBCL-B (again using the 2 stages): produces SBCL-C The binary SBCL-A is not deterministic probably because of CLISP (and CLISP should be fixed but that's another story :-)). However, SBCL-B and SBCL-C must be deterministic. And ideally bit-to-bit identical but that's another story. :-) And they are not; from my experiments at least. Even, you could do the same procedure replacing CLISP by ECL, producing SBCL-{A,B,C}-bis. Then SBCL-C and SBCL-C-bis should be bit-to-bit identical. > Removing this source file timestamp from compiled files would simplify > things. Maybe nothing really depends on it and it would be possible... Thanks for the explanation. All the best, simon ^ permalink raw reply [flat|nested] 15+ messages in thread
* bug#44112: SBCL is not reproducible 2020-10-21 13:37 ` zimoun @ 2020-10-21 13:43 ` zimoun 2020-10-21 14:32 ` Pierre Neidhardt 0 siblings, 1 reply; 15+ messages in thread From: zimoun @ 2020-10-21 13:43 UTC (permalink / raw) To: Guillaume Le Vaillant; +Cc: 44112, Pierre Neidhardt On Wed, 21 Oct 2020 at 15:39, zimoun <zimon.toutoune@gmail.com> wrote: > > IIRC, SBCL itself is built in 2 stages. First its core is compiled > > using another Common Lisp implementation (currently clisp in Guix), then > > the complete SBCL is compiled using the core compiled in stage 1. There > > is probably also an embedded timestamp issue here (coming for clisp, > > from SBCL, or both) causing the reproducibility issue. In addition, GCC is also involved in the party. And I have also replaced it by Clang with the same effect. Cheers, simon ^ permalink raw reply [flat|nested] 15+ messages in thread
* bug#44112: SBCL is not reproducible 2020-10-21 13:43 ` zimoun @ 2020-10-21 14:32 ` Pierre Neidhardt 2020-10-21 15:13 ` zimoun 0 siblings, 1 reply; 15+ messages in thread From: Pierre Neidhardt @ 2020-10-21 14:32 UTC (permalink / raw) To: zimoun, Guillaume Le Vaillant; +Cc: 44112 [-- Attachment #1: Type: text/plain, Size: 217 bytes --] zimoun <zimon.toutoune@gmail.com> writes: > In addition, GCC is also involved in the party. And I have also > replaced it by Clang with the same effect. Where? -- Pierre Neidhardt https://ambrevar.xyz/ [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 511 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* bug#44112: SBCL is not reproducible 2020-10-21 14:32 ` Pierre Neidhardt @ 2020-10-21 15:13 ` zimoun 2020-10-21 16:45 ` Guillaume Le Vaillant 0 siblings, 1 reply; 15+ messages in thread From: zimoun @ 2020-10-21 15:13 UTC (permalink / raw) To: Pierre Neidhardt; +Cc: 44112 On Wed, 21 Oct 2020 at 16:32, Pierre Neidhardt <mail@ambrevar.xyz> wrote: > > In addition, GCC is also involved in the party. And I have also > > replaced it by Clang with the same effect. > > Where? --8<---------------cut here---------------start------------->8--- (define-module (ddc-sbcl) #:use-module (guix packages) #:use-module (guix utils) #:use-module ((guix build utils) #:select (modify-phases)) #:use-module (srfi srfi-1) #:use-module (ice-9 match) #:use-module (gnu packages lisp) #:use-module (gnu packages commencement) #:use-module (gnu packages llvm)) (define (make-sbcl name string-compil compil cc) (package (inherit sbcl) (name name) (arguments (substitute-keyword-arguments (package-arguments sbcl) ((#:phases phases) `(modify-phases ,phases (replace 'build (lambda* (#:key outputs #:allow-other-keys) (setenv "CC" ,cc) (invoke "sh" "make.sh" ,string-compil (string-append "--prefix=" (assoc-ref outputs "out")) "--dynamic-space-size=2Gb" "--with-sb-core-compression" "--with-sb-xref-for-internals"))))))) (native-inputs `(,@(alist-delete "clisp" (package-native-inputs sbcl)) ("compil" ,compil))))) (define-public sbcl-A (package-with-c-toolchain (make-sbcl "sbcl-A" "clisp" clisp "gcc") `(("gcc" ,gcc-toolchain)))) ;fail with 6 (define-public sbcl-B (package-with-c-toolchain (make-sbcl "sbcl-B" "ecl" ecl "clang") `(("clang" ,clang-toolchain)))) (define-public sbcl-AA (package-with-c-toolchain (make-sbcl "sbcl-AA" "sbcl" sbcl-A "gcc") `(("gcc" ,gcc-toolchain)))) (define-public sbcl-BB (package-with-c-toolchain (make-sbcl "sbcl-BB" "sbcl" sbcl-B "clang") `(("clang" ,clang-toolchain)))) (define-public sbcl-CC (package-with-c-toolchain (make-sbcl "sbcl-CC" "sbcl" sbcl-A "clang") `(("clang" ,clang-toolchain)))) (define-public sbcl-DD (package-with-c-toolchain (make-sbcl "sbcl-DD" "sbcl" sbcl-B "gcc") `(("gcc" ,gcc-toolchain)))) --8<---------------cut here---------------end--------------->8--- ^ permalink raw reply [flat|nested] 15+ messages in thread
* bug#44112: SBCL is not reproducible 2020-10-21 15:13 ` zimoun @ 2020-10-21 16:45 ` Guillaume Le Vaillant 2020-10-21 16:55 ` Pierre Neidhardt 2020-10-22 11:19 ` zimoun 0 siblings, 2 replies; 15+ messages in thread From: Guillaume Le Vaillant @ 2020-10-21 16:45 UTC (permalink / raw) To: zimoun; +Cc: 44112, Pierre Neidhardt [-- Attachment #1: Type: text/plain, Size: 503 bytes --] zimoun <zimon.toutoune@gmail.com> skribis: >> Removing this source file timestamp from compiled files would simplify >> things. Maybe nothing really depends on it and it would be possible... > >Thanks for the explanation. A developer of SBCL agrees that the timestamp should be removed [1], but currently Slime has a function depending on it. I asked if this could be fixed [2]. We'll see... [1] https://sourceforge.net/p/sbcl/mailman/message/37133640/ [2] https://github.com/slime/slime/issues/583 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 247 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* bug#44112: SBCL is not reproducible 2020-10-21 16:45 ` Guillaume Le Vaillant @ 2020-10-21 16:55 ` Pierre Neidhardt 2020-10-22 11:19 ` zimoun 1 sibling, 0 replies; 15+ messages in thread From: Pierre Neidhardt @ 2020-10-21 16:55 UTC (permalink / raw) To: Guillaume Le Vaillant, zimoun; +Cc: 44112 [-- Attachment #1: Type: text/plain, Size: 426 bytes --] Guillaume Le Vaillant <glv@posteo.net> writes: > A developer of SBCL agrees that the timestamp should be removed [1], but > currently Slime has a function depending on it. I asked if this could > be fixed [2]. We'll see... > > [1] https://sourceforge.net/p/sbcl/mailman/message/37133640/ > [2] https://github.com/slime/slime/issues/583 Thank you so much for asking! -- Pierre Neidhardt https://ambrevar.xyz/ [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 511 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* bug#44112: SBCL is not reproducible 2020-10-21 16:45 ` Guillaume Le Vaillant 2020-10-21 16:55 ` Pierre Neidhardt @ 2020-10-22 11:19 ` zimoun 2020-10-22 12:13 ` Pierre Neidhardt 1 sibling, 1 reply; 15+ messages in thread From: zimoun @ 2020-10-22 11:19 UTC (permalink / raw) To: Guillaume Le Vaillant; +Cc: 44112, Pierre Neidhardt Hi Guillaume, Thank you for asking upstream. On Wed, 21 Oct 2020 at 18:45, Guillaume Le Vaillant <glv@posteo.net> wrote: > A developer of SBCL agrees that the timestamp should be removed [1], but > currently Slime has a function depending on it. I asked if this could > be fixed [2]. We'll see... > > [1] https://sourceforge.net/p/sbcl/mailman/message/37133640/ Ouch! Hard to read when not subscribed. :-) > [2] https://github.com/slime/slime/issues/583 If upstream does not want to patch, do you think it is doable to patch SLIME and maintain the patch with the Guix package? Thanks, simon ^ permalink raw reply [flat|nested] 15+ messages in thread
* bug#44112: SBCL is not reproducible 2020-10-22 11:19 ` zimoun @ 2020-10-22 12:13 ` Pierre Neidhardt 0 siblings, 0 replies; 15+ messages in thread From: Pierre Neidhardt @ 2020-10-22 12:13 UTC (permalink / raw) To: zimoun, Guillaume Le Vaillant; +Cc: 44112 [-- Attachment #1: Type: text/plain, Size: 336 bytes --] zimoun <zimon.toutoune@gmail.com> writes: > If upstream does not want to patch, do you think it is doable to patch > SLIME and maintain the patch with the Guix package? Accepted or not, we also need to ping the SLY maintainer to apply the same change. (SLY is a fork of SLIME.) -- Pierre Neidhardt https://ambrevar.xyz/ [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 511 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* bug#44112: SBCL is not reproducible 2020-10-21 12:41 ` Guillaume Le Vaillant 2020-10-21 13:37 ` zimoun @ 2020-10-21 14:32 ` Pierre Neidhardt 2021-01-19 16:48 ` zimoun 2 siblings, 0 replies; 15+ messages in thread From: Pierre Neidhardt @ 2020-10-21 14:32 UTC (permalink / raw) To: Guillaume Le Vaillant, zimoun; +Cc: 44112 [-- Attachment #1: Type: text/plain, Size: 338 bytes --] Guillaume Le Vaillant <glv@posteo.net> writes: > IIRC, SBCL itself is built in 2 stages. First its core is compiled > using another Common Lisp implementation (currently clisp in Guix), then > the complete SBCL is compiled using the core compiled in stage 1. Yes, this is correct. -- Pierre Neidhardt https://ambrevar.xyz/ [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 511 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* bug#44112: SBCL is not reproducible 2020-10-21 12:41 ` Guillaume Le Vaillant 2020-10-21 13:37 ` zimoun 2020-10-21 14:32 ` Pierre Neidhardt @ 2021-01-19 16:48 ` zimoun 2021-01-23 9:19 ` Guillaume Le Vaillant 2 siblings, 1 reply; 15+ messages in thread From: zimoun @ 2021-01-19 16:48 UTC (permalink / raw) To: Guillaume Le Vaillant; +Cc: 44112, Pierre Neidhardt Hi, On Wed, 21 Oct 2020 at 14:41, Guillaume Le Vaillant <glv@posteo.net> wrote: > zimoun <zimon.toutoune@gmail.com> skribis: > >> Using Guix 58af4c9, the package ’sbcl’ seems not-reproducible. [...] > Removing this source file timestamp from compiled files would simplify > things. Maybe nothing really depends on it and it would be possible... Any news from upstream? Still unreproducible for me. All the best, simon ^ permalink raw reply [flat|nested] 15+ messages in thread
* bug#44112: SBCL is not reproducible 2021-01-19 16:48 ` zimoun @ 2021-01-23 9:19 ` Guillaume Le Vaillant 0 siblings, 0 replies; 15+ messages in thread From: Guillaume Le Vaillant @ 2021-01-23 9:19 UTC (permalink / raw) To: zimoun; +Cc: 44112, Pierre Neidhardt [-- Attachment #1: Type: text/plain, Size: 789 bytes --] zimoun <zimon.toutoune@gmail.com> skribis: > Hi, > > On Wed, 21 Oct 2020 at 14:41, Guillaume Le Vaillant <glv@posteo.net> wrote: >> zimoun <zimon.toutoune@gmail.com> skribis: >> >>> Using Guix 58af4c9, the package ’sbcl’ seems not-reproducible. > > [...] > >> Removing this source file timestamp from compiled files would simplify >> things. Maybe nothing really depends on it and it would be possible... > > Any news from upstream? Still unreproducible for me. > > > All the best, > simon So far Slime still uses the function getting the source file timestamp, so these timestamps can't be removed from SBCL. A patch for Slime would be accepted (see [2]), but as far as I know nobody has worked on it yet. [2] https://github.com/slime/slime/issues/583 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 247 bytes --] ^ permalink raw reply [flat|nested] 15+ messages in thread
* bug#44112: 2020-10-21 8:12 ` bug#44112: SBCL is not reproducible zimoun 2020-10-21 12:41 ` Guillaume Le Vaillant @ 2022-11-25 17:22 ` Hendursaga via Bug reports for GNU Guix 2022-12-09 18:50 ` bug#44112: zimoun 1 sibling, 1 reply; 15+ messages in thread From: Hendursaga via Bug reports for GNU Guix @ 2022-11-25 17:22 UTC (permalink / raw) To: 44112 The patch[1] has been closed for some time, is there any other pending patches blocking going forward? [1] https://github.com/slime/slime/issues/583 ^ permalink raw reply [flat|nested] 15+ messages in thread
* bug#44112: 2022-11-25 17:22 ` bug#44112: Hendursaga via Bug reports for GNU Guix @ 2022-12-09 18:50 ` zimoun 0 siblings, 0 replies; 15+ messages in thread From: zimoun @ 2022-12-09 18:50 UTC (permalink / raw) To: Hendursaga, 44112; +Cc: glv Hi, Thank you for the follow up. On Fri, 25 Nov 2022 at 12:22, Hendursaga via Bug reports for GNU Guix <bug-guix@gnu.org> wrote: > The patch[1] has been closed for some time, is there any other pending patches blocking going forward? > > [1] https://github.com/slime/slime/issues/583 Indeed, but the package sbcl is not yet reproducible. The remaining issue is about PDF. Using Guix 28d1e51, I get: --8<---------------cut here---------------start------------->8--- $ diff -r --no-dereference /gnu/store/0q9z6j5ml4rkfrg57axznasdjfm7x803-sbcl-2.2.11-doc{,-check} Binary files /gnu/store/0q9z6j5ml4rkfrg57axznasdjfm7x803-sbcl-2.2.11-doc/share/doc/sbcl/sbcl.pdf and /gnu/store/0q9z6j5ml4rkfrg57axznasdjfm7x803-sbcl-2.2.11-doc-check/share/doc/sbcl/sbcl.pdf differ --8<---------------cut here---------------end--------------->8--- Cheers, simon ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2022-12-09 19:03 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <87zgcflzjn.fsf.ref@aol.com>
2020-10-21 8:12 ` bug#44112: SBCL is not reproducible zimoun
2020-10-21 12:41 ` Guillaume Le Vaillant
2020-10-21 13:37 ` zimoun
2020-10-21 13:43 ` zimoun
2020-10-21 14:32 ` Pierre Neidhardt
2020-10-21 15:13 ` zimoun
2020-10-21 16:45 ` Guillaume Le Vaillant
2020-10-21 16:55 ` Pierre Neidhardt
2020-10-22 11:19 ` zimoun
2020-10-22 12:13 ` Pierre Neidhardt
2020-10-21 14:32 ` Pierre Neidhardt
2021-01-19 16:48 ` zimoun
2021-01-23 9:19 ` Guillaume Le Vaillant
2022-11-25 17:22 ` bug#44112: Hendursaga via Bug reports for GNU Guix
2022-12-09 18:50 ` bug#44112: zimoun
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).