* Challenge substitute servers! @ 2015-10-19 23:09 Ludovic Courtès 2015-10-20 0:17 ` Daniel Pimentel ` (2 more replies) 0 siblings, 3 replies; 29+ messages in thread From: Ludovic Courtès @ 2015-10-19 23:09 UTC (permalink / raw) To: guix-devel Hello! I’m happy to announce the new ‘guix challenge’ command! (Documentation below.) The goal of the command is, ideally, to report malicious or corrupt substitute servers. In practice though, many package build processes are still non-deterministic, so chances are that, when a discrepancy reported by ‘guix challenge’, it is due to a non-deterministic build process–something we must fix. Here’s what it reports on my machine (compared to hydra.gnunet.org, which is one of the build machines behind hydra.gnu.org): --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix challenge --substitute-urls=http://hydra.gnunet.org finding garbage collector roots... determining live/dead paths... updating list of substitutes from 'http://hydra.gnunet.org'... 100.0% /gnu/store/1c96zwwg6lnh6v9n3m0q30pg0x2m0v5c-openssl-1.0.2d contents differ: local hash: 0725l22r5jnzazaacncwsvp9kgf42266ayyp814v7djxs7nk963q http://hydra.gnunet.org/nar/1c96zwwg6lnh6v9n3m0q30pg0x2m0v5c-openssl-1.0.2d: 1zy4fmaaqcnjrzzajkdn3f5gmjk754b43qkq47llbyak9z0qjyim /gnu/store/3b7sjkz1ps719fkl53mcxzmjysd95i5c-emacs-deferred-0.3.2 contents differ: local hash: 0hh4d5plz4ph93yqcqvyy8dfbdidwdmy93kikjgnkglxr0lw8qss http://hydra.gnunet.org/nar/3b7sjkz1ps719fkl53mcxzmjysd95i5c-emacs-deferred-0.3.2: 0d0k5sl6zwbwb4yj0ns118bq6nxqaa8d4kr189w8lb7sma4ji5y1 /gnu/store/5zavfkmrax6z93q85q5nifbwkfz4704m-git-2.5.0 contents differ: local hash: 00p3bmryhjxrhpn2gxs2fy0a15lnip05l97205pgbk5ra395hyha http://hydra.gnunet.org/nar/5zavfkmrax6z93q85q5nifbwkfz4704m-git-2.5.0: 069nb85bv4d4a6slrwjdy8v1cn4cwspm3kdbmyb81d6zckj3nq9f /gnu/store/76yq0qvcbjjljk8my6x06ayssph573xx-pius-2.1.1 contents differ: local hash: 0k4v3m9z1zp8xzzizb7d8kjj72f9172xv078sq4wl73vnq9ig3ax http://hydra.gnunet.org/nar/76yq0qvcbjjljk8my6x06ayssph573xx-pius-2.1.1: 1cy25x1a4fzq5rk0pmvc8xhwyffnqz95h2bpvqsz2mpvlbccy0gs /gnu/store/avrrq8sl1ihq0vrhjrilszgmg7ifhqdw-guile-ssh-0.8.0 contents differ: local hash: 1qxjnirrd24djxwrh1wmsdg3qhhigymaqg673nri0d5dn87dihmc http://hydra.gnunet.org/nar/avrrq8sl1ihq0vrhjrilszgmg7ifhqdw-guile-ssh-0.8.0: 1js93xgwdrbq85mzl655dc7f6hb742bmvv46jij7b73npfzzjml9 /gnu/store/bc081jj8rx640ism3gm6wlpb7jamg080-dmd-0.2.01 contents differ: local hash: 1ngmjzscwva6w0wy3ahmq4r6svdpa8pq7f5kz273qld3k2xrg54v http://hydra.gnunet.org/nar/bc081jj8rx640ism3gm6wlpb7jamg080-dmd-0.2.01: 0hfcg2qvdfrzyfgwn1p1j7k0vv93xk5z4y9kr22gyakf9f9jjfix /gnu/store/cmb889dhvnx9kk6gdvqjm14w4j6pqalq-guix-0.8.3.abbe2c6 contents differ: local hash: 0p3mr40xk9q8bw26h8gkz2nx6n0csrmdh05zb1x85i0zs9pfd2f8 http://hydra.gnunet.org/nar/cmb889dhvnx9kk6gdvqjm14w4j6pqalq-guix-0.8.3.abbe2c6: 1aca8zkvcff38y3s9khf565mlcz8xgl8p6c93jbrin87k8z2islq /gnu/store/fkl97li2g7gqyw5bq09q2r0hkxla54lq-ath9k-htc-firmware-1.4.0 contents differ: local hash: 0r4lysb0skx7dpyh1qsk2mamkwwd4a7yldr1yy2dpr7bc0nk4z8n http://hydra.gnunet.org/nar/fkl97li2g7gqyw5bq09q2r0hkxla54lq-ath9k-htc-firmware-1.4.0: 1aia3539xvzaff4s11ga35f7iahch6rbryqzh5adlmx3gffpy2y7 /gnu/store/j0qc3ghc7ajja6k9c35y8ssxjzxrsy95-emacs-debbugs-0.7 contents differ: local hash: 0wfczpznl9pdalf4sp23dfp63pvrk9jsw7znvw8axj24wv9rbk4c http://hydra.gnunet.org/nar/j0qc3ghc7ajja6k9c35y8ssxjzxrsy95-emacs-debbugs-0.7: 19bagzab1d6xabhy3av863ab06zkcxya4lc7q62za74p7qpj112g /gnu/store/n5q0gbn01w5m2ic9rc2xq4kj2faglg6s-emacs-typo-1.1 contents differ: local hash: 1yly4y92vphdsnqnmdqwsz019fy11jkbz31sl6ysly9j3fmnhq2m http://hydra.gnunet.org/nar/n5q0gbn01w5m2ic9rc2xq4kj2faglg6s-emacs-typo-1.1: 1mj1h56pwf5cqbn397crlgsp7hdwjvrbqp4czha429y7pharjapl /gnu/store/q6xnjg9fd7lfhh7rq2l98grlyq7nbcf0-emacs-butler-0.2.4 contents differ: local hash: 14lb6cdfngjjlxrnipq961hbgnhwp47ap904a9mm0dj4q7pj23n2 http://hydra.gnunet.org/nar/q6xnjg9fd7lfhh7rq2l98grlyq7nbcf0-emacs-butler-0.2.4: 0vk41961frfvm5nrkmljhli3ibsz2lv0s09s6m29aiymixx6sw9g --8<---------------cut here---------------end--------------->8--- If we diff as explained in the doc below, we see that the Git discrepancies are due to timestamp in Perl’s POD files as well as sorted-by-inode-number ‘tclIndex’ files. For the Emacs modes, the problem is the autogenerated autoloads files, which include some sort of a timestamp as well. You’re welcome to help fix these issues! Comments welcome! Ludo’. 6.12 Invoking ‘guix challenge’ ============================== Do the binaries provided by this server really correspond to the source code it claims to build? Is this package’s build process deterministic? These are the questions the ‘guix challenge’ command attempts to answer. The former is obviously an important question: Before using a substitute server (*note Substitutes::), you’d rather _verify_ that it provides the right binaries, and thus _challenge_ it. The latter is what enables the former: If package builds are deterministic, then independent builds of the package should yield the exact same result, bit for bit; if a server provides a binary different from the one obtained locally, it may be either corrupt or malicious. We know that the hash that shows up in ‘/gnu/store’ file names is the hash of all the inputs of the process that built that file or directory—compilers, libraries, build scripts, etc. (*note Introduction::). Assuming deterministic build processes, one store file name should map to exactly one build output. ‘guix challenge’ checks whether there is, indeed, a single mapping by comparing the build outputs of several independent builds of any given store item. The command’s output looks like this: $ guix challenge --substitute-urls="http://hydra.gnu.org http://guix.example.org" updating list of substitutes from 'http://hydra.gnu.org'... 100.0% updating list of substitutes from 'http://guix.example.org'... 100.0% /gnu/store/…-openssl-1.0.2d contents differ: local hash: 0725l22r5jnzazaacncwsvp9kgf42266ayyp814v7djxs7nk963q http://hydra.gnu.org/nar/…-openssl-1.0.2d: 0725l22r5jnzazaacncwsvp9kgf42266ayyp814v7djxs7nk963q http://guix.example.org/nar/…-openssl-1.0.2d: 1zy4fmaaqcnjrzzajkdn3f5gmjk754b43qkq47llbyak9z0qjyim /gnu/store/…-git-2.5.0 contents differ: local hash: 00p3bmryhjxrhpn2gxs2fy0a15lnip05l97205pgbk5ra395hyha http://hydra.gnu.org/nar/…-git-2.5.0: 069nb85bv4d4a6slrwjdy8v1cn4cwspm3kdbmyb81d6zckj3nq9f http://guix.example.org/nar/…-git-2.5.0: 0mdqa9w1p6cmli6976v4wi0sw9r4p5prkj7lzfd1877wk11c9c73 /gnu/store/…-pius-2.1.1 contents differ: local hash: 0k4v3m9z1zp8xzzizb7d8kjj72f9172xv078sq4wl73vnq9ig3ax http://hydra.gnu.org/nar/…-pius-2.1.1: 0k4v3m9z1zp8xzzizb7d8kjj72f9172xv078sq4wl73vnq9ig3ax http://guix.example.org/nar/…-pius-2.1.1: 1cy25x1a4fzq5rk0pmvc8xhwyffnqz95h2bpvqsz2mpvlbccy0gs In this example, ‘guix challenge’ first scans the store to determine the set of locally-built derivations—as opposed to store items that were downloaded from a substitute server—and then queries all the substitute servers. It then reports those store items for which the servers obtained a result different from the local build. As an example, ‘guix.example.org’ always gets a different answer. Conversely, ‘hydra.gnu.org’ agrees with local builds, except in the case of Git. This might indicate that the build process of Git is non-deterministic, meaning that its output varies as a function of various things that Guix does not fully control, in spite of building packages in isolated environments (*note Features::). Most common sources of non-determinism include the addition of timestamps in build results, the inclusion of random numbers, and directory listings sorted by inode number. See <http://reproducible.debian.net/howto/>, for more information. To find out what’s wrong with this Git binary, we can do something along these lines (*note Invoking guix archive::): $ wget -q -O - http://hydra.gnu.org/nar/…-git-2.5.0 \ | guix archive -x /tmp/git $ diff -ur /gnu/store/…-git.2.5.0 /tmp/git This command shows the difference between the files resulting from the local build, and the files resulting from the build on ‘hydra.gnu.org’ (*note Comparing and Merging Files: (diffutils)Overview.). The ‘diff’ command works great for text files. When binary files differ, a better option is Diffoscope (http://diffoscope.org/), a tool that helps visualize differences for all kinds of files. Once you’ve done that work, you can tell whether the differences are due to a non-deterministic build process or to a malicious server. We try hard to remove sources of non-determinism in packages to make it easier to verify substitutes, but of course, this is a process, one that involves not just Guix but a large part of the free software community. In the meantime, ‘guix challenge’ is one tool to help address the problem. If you are writing packages for Guix, you are encouraged to check whether ‘hydra.gnu.org’ and other substitute servers obtain the same build result as you did with: $ guix challenge PACKAGE ... where PACKAGE is a package specification such as ‘guile-2.0’ or ‘glibc:debug’. The general syntax is: guix challenge OPTIONS [PACKAGES…] The one option that matters is: ‘--substitute-urls=URLS’ Consider URLS the whitespace-separated list of substitute source URLs to compare to. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Challenge substitute servers! 2015-10-19 23:09 Challenge substitute servers! Ludovic Courtès @ 2015-10-20 0:17 ` Daniel Pimentel 2015-10-20 14:38 ` [PATCHES] emacs: Changes for 'guix challenge' Alex Kost 2015-10-20 17:40 ` Timestamps in ...-autoloads.el files Alex Kost 2 siblings, 0 replies; 29+ messages in thread From: Daniel Pimentel @ 2015-10-20 0:17 UTC (permalink / raw) To: ludo; +Cc: guix-devel, guix-devel-bounces+d4n1=opmbx.org Congratulation Guixs for it! It's a good idea :) Thanks, -- Daniel Pimentel (d4n1 3:) ^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCHES] emacs: Changes for 'guix challenge'. 2015-10-19 23:09 Challenge substitute servers! Ludovic Courtès 2015-10-20 0:17 ` Daniel Pimentel @ 2015-10-20 14:38 ` Alex Kost 2015-10-20 14:58 ` Ludovic Courtès 2015-10-20 17:40 ` Timestamps in ...-autoloads.el files Alex Kost 2 siblings, 1 reply; 29+ messages in thread From: Alex Kost @ 2015-10-20 14:38 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 637 bytes --] Ludovic Courtès (2015-10-20 02:09 +0300) wrote: > Hello! > > I’m happy to announce the new ‘guix challenge’ command! (Documentation > below.) I like new commands! And this one is cool; I didn't think about timestamp non-determinism before, thanks! I'm attaching 2 trivial patches for emacs support of ‘guix challenge’. The first patch is for completing packages in "M-x shell". And the second one is for adding a positional "Packages" argument in "M-x guix RET challenge". These are 2 different things, so I made 2 patches, or should it be a single “emacs: Add support for 'guix challenge'” patch? [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-emacs-Add-shell-completions-for-guix-challenge.patch --] [-- Type: text/x-patch, Size: 945 bytes --] From adaf6ab125d1bfd8e9416edd7d920e663c4c7785 Mon Sep 17 00:00:00 2001 From: Alex Kost <alezost@gmail.com> Date: Tue, 20 Oct 2015 16:42:01 +0300 Subject: [PATCH 1/2] emacs: Add shell completions for 'guix challenge'. * emacs/guix-pcomplete.el (guix-pcomplete-complete-command-arg): Add "challenge" to complete package names for it. --- emacs/guix-pcomplete.el | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/emacs/guix-pcomplete.el b/emacs/guix-pcomplete.el index 4743be5..bd41837 100644 --- a/emacs/guix-pcomplete.el +++ b/emacs/guix-pcomplete.el @@ -209,7 +209,7 @@ group - the argument.") "Complete argument for guix COMMAND." (cond ((member command - '("archive" "build" "graph" "edit" "environment" + '("archive" "build" "challenge" "graph" "edit" "environment" "lint" "refresh" "size")) (while t (pcomplete-here (guix-pcomplete-all-packages)))) -- 2.5.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #3: 0002-emacs-Add-Packages-option-for-guix-challenge-popup.patch --] [-- Type: text/x-patch, Size: 986 bytes --] From 41b2f36081f5c36b56156231a280e183dabd3243 Mon Sep 17 00:00:00 2001 From: Alex Kost <alezost@gmail.com> Date: Tue, 20 Oct 2015 16:43:49 +0300 Subject: [PATCH 2/2] emacs: Add "Packages" option for 'guix challenge' popup. * emacs/guix-command.el (guix-command-rest-argument): Add "challenge". --- emacs/guix-command.el | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/emacs/guix-command.el b/emacs/guix-command.el index 1a42594..e9e46f4 100644 --- a/emacs/guix-command.el +++ b/emacs/guix-command.el @@ -364,7 +364,7 @@ to be modified." :name "-- " :char ?= :option? t args))) (let ((command (car commands))) (cond - ((member command '("archive" "build" "graph" "edit" + ((member command '("archive" "build" "challenge" "graph" "edit" "environment" "lint" "refresh")) (argument :doc "Packages" :fun 'guix-read-package-names-string)) ((string= command "download") -- 2.5.0 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: [PATCHES] emacs: Changes for 'guix challenge'. 2015-10-20 14:38 ` [PATCHES] emacs: Changes for 'guix challenge' Alex Kost @ 2015-10-20 14:58 ` Ludovic Courtès 0 siblings, 0 replies; 29+ messages in thread From: Ludovic Courtès @ 2015-10-20 14:58 UTC (permalink / raw) To: Alex Kost; +Cc: guix-devel Alex Kost <alezost@gmail.com> skribis: > I'm attaching 2 trivial patches for emacs support of ‘guix challenge’. > The first patch is for completing packages in "M-x shell". And the > second one is for adding a positional "Packages" argument in "M-x guix > RET challenge". LGTM! > These are 2 different things, so I made 2 patches, or should it be a > single “emacs: Add support for 'guix challenge'” patch? Either way is fine with me. Thanks for the quick followup! :-) Ludo’. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Timestamps in ...-autoloads.el files 2015-10-19 23:09 Challenge substitute servers! Ludovic Courtès 2015-10-20 0:17 ` Daniel Pimentel 2015-10-20 14:38 ` [PATCHES] emacs: Changes for 'guix challenge' Alex Kost @ 2015-10-20 17:40 ` Alex Kost 2015-10-20 19:38 ` Ludovic Courtès 2 siblings, 1 reply; 29+ messages in thread From: Alex Kost @ 2015-10-20 17:40 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 580 bytes --] Ludovic Courtès (2015-10-20 02:09 +0300) wrote: > If we diff as explained in the doc below, we see that the Git > discrepancies are due to timestamp in Perl’s POD files as well as > sorted-by-inode-number ‘tclIndex’ files. For the Emacs modes, the > problem is the autogenerated autoloads files, which include some sort of > a timestamp as well. > > You’re welcome to help fix these issues! Thanks for the info! Those timestamps are inserted by `autoload-insert-section-header' and we can avoid them by advising this function, for example, like this: [-- Attachment #2: autoload-advice.el --] [-- Type: application/emacs-lisp, Size: 227 bytes --] [-- Attachment #3: Type: text/plain, Size: 658 bytes --] So after putting this code into ‘emacs-generate-autoloads’ procedure from (guix build emacs-utils) module, there will be zeros instead of non-deterministic timestamps. However this will fix only those packages, that use ‘emacs-generate-autoloads’ directly or via ‘emacs-build-system’. But there are also packages that generate autoloads on their own (for example, 'emacs-w3m' or 'guix' itself). What to do for these ones? Perhaps we can make a special 'emacs-build' package (that will advise ‘autoload-insert-section-header’ function somehow) and use it as an input for emacs-packages, or are there other ways? -- Alex ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2015-10-20 17:40 ` Timestamps in ...-autoloads.el files Alex Kost @ 2015-10-20 19:38 ` Ludovic Courtès 2015-10-21 13:05 ` Alex Kost 0 siblings, 1 reply; 29+ messages in thread From: Ludovic Courtès @ 2015-10-20 19:38 UTC (permalink / raw) To: Alex Kost; +Cc: guix-devel Alex Kost <alezost@gmail.com> skribis: > Ludovic Courtès (2015-10-20 02:09 +0300) wrote: > >> If we diff as explained in the doc below, we see that the Git >> discrepancies are due to timestamp in Perl’s POD files as well as >> sorted-by-inode-number ‘tclIndex’ files. For the Emacs modes, the >> problem is the autogenerated autoloads files, which include some sort of >> a timestamp as well. >> >> You’re welcome to help fix these issues! > > Thanks for the info! > > Those timestamps are inserted by `autoload-insert-section-header' and > we can avoid them by advising this function, for example, like this: > > (defun guix-autoload-no-timestamp (fun outbuf autoloads load-name file time) > (funcall fun outbuf autoloads load-name file 0)) > > (advice-add 'autoload-insert-section-header > :around 'guix-autoload-no-timestamp) > > So after putting this code into ‘emacs-generate-autoloads’ procedure > from (guix build emacs-utils) module, there will be zeros instead of > non-deterministic timestamps. Great. > However this will fix only those packages, that use > ‘emacs-generate-autoloads’ directly or via ‘emacs-build-system’. But > there are also packages that generate autoloads on their own (for > example, 'emacs-w3m' or 'guix' itself). What to do for these ones? > > Perhaps we can make a special 'emacs-build' package (that will advise > ‘autoload-insert-section-header’ function somehow) and use it as an > input for emacs-packages, or are there other ways? What about patching Emacs directly? An upstreamable patch would be one that honors the ‘SOURCE_DATE_EPOCH’ variable¹. Alternately, a patch that simply changes ‘autoload-insert-section-header’ to always use zero as the timestamp would work as well, unless this would somehow break functionality. Thoughts? Thanks for looking into it, Ludo’. ¹ https://reproducible-builds.org/specs/source-date-epoch/ ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2015-10-20 19:38 ` Ludovic Courtès @ 2015-10-21 13:05 ` Alex Kost 2015-10-21 16:55 ` Ludovic Courtès 0 siblings, 1 reply; 29+ messages in thread From: Alex Kost @ 2015-10-21 13:05 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1269 bytes --] Ludovic Courtès (2015-10-20 22:38 +0300) wrote: > Alex Kost <alezost@gmail.com> skribis: > [...] >> However this will fix only those packages, that use >> ‘emacs-generate-autoloads’ directly or via ‘emacs-build-system’. But >> there are also packages that generate autoloads on their own (for >> example, 'emacs-w3m' or 'guix' itself). What to do for these ones? >> >> Perhaps we can make a special 'emacs-build' package (that will advise >> ‘autoload-insert-section-header’ function somehow) and use it as an >> input for emacs-packages, or are there other ways? > > What about patching Emacs directly? Indeed! > An upstreamable patch would be one that honors the ‘SOURCE_DATE_EPOCH’ > variable¹. > > Alternately, a patch that simply changes > ‘autoload-insert-section-header’ to always use zero as the timestamp > would work as well, unless this would somehow break functionality. > > Thoughts? I like the idea to honor SOURCE_DATE_EPOCH, so I'm attaching a patch for this. But now I don't know how to make Guix set this variable during the build process :-( Need help. > Thanks for looking into it, > Ludo’. > > ¹ https://reproducible-builds.org/specs/source-date-epoch/ Thanks for the info! [-- Attachment #2: 0001-gnu-emacs-Honor-SOURCE_DATE_EPOCH.patch --] [-- Type: text/x-patch, Size: 3068 bytes --] From b8dc19a65980690a636ad7f9f39b3c84991f4975 Mon Sep 17 00:00:00 2001 From: Alex Kost <alezost@gmail.com> Date: Wed, 21 Oct 2015 15:59:23 +0300 Subject: [PATCH] gnu: emacs: Honor 'SOURCE_DATE_EPOCH'. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Suggested by Ludovic Courtès <ludo@gnu.org>. * gnu/packages/patches/emacs-source-date-epoch.patch: New patch. * gnu-system.am (dist_patch_DATA): Add it. * gnu/packages/emacs.scm (emacs)[source]: Use it. --- gnu-system.am | 1 + gnu/packages/emacs.scm | 3 ++- gnu/packages/patches/emacs-source-date-epoch.patch | 20 ++++++++++++++++++++ 3 files changed, 23 insertions(+), 1 deletion(-) create mode 100644 gnu/packages/patches/emacs-source-date-epoch.patch diff --git a/gnu-system.am b/gnu-system.am index e62fe18..4a20801 100644 --- a/gnu-system.am +++ b/gnu-system.am @@ -437,6 +437,7 @@ dist_patch_DATA = \ gnu/packages/patches/duplicity-test_selection-tmp.patch \ gnu/packages/patches/elfutils-tests-ptrace.patch \ gnu/packages/patches/emacs-exec-path.patch \ + gnu/packages/patches/emacs-source-date-epoch.patch \ gnu/packages/patches/eudev-rules-directory.patch \ gnu/packages/patches/expat-CVE-2015-1283.patch \ gnu/packages/patches/fastcap-mulGlobal.patch \ diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm index 6416b00..9751125 100644 --- a/gnu/packages/emacs.scm +++ b/gnu/packages/emacs.scm @@ -70,7 +70,8 @@ (sha256 (base32 "0kn3rzm91qiswi0cql89kbv6mqn27rwsyjfb8xmwy9m5s8fxfiyx")) - (patches (list (search-patch "emacs-exec-path.patch"))))) + (patches (list (search-patch "emacs-exec-path.patch") + (search-patch "emacs-source-date-epoch.patch"))))) (build-system glib-or-gtk-build-system) (arguments '(#:phases (alist-cons-before diff --git a/gnu/packages/patches/emacs-source-date-epoch.patch b/gnu/packages/patches/emacs-source-date-epoch.patch new file mode 100644 index 0000000..41c03ef --- /dev/null +++ b/gnu/packages/patches/emacs-source-date-epoch.patch @@ -0,0 +1,20 @@ +Honor SOURCE_DATE_EPOCH variable to avoid non-determinism in generated +"autoloads" files. + +--- a/lisp/emacs-lisp/autoload.el ++++ b/lisp/emacs-lisp/autoload.el +@@ -378,8 +378,12 @@ + "Insert the section-header line, + which lists the file name and which functions are in it, etc." + (insert generate-autoload-section-header) +- (prin1 `(autoloads ,autoloads ,load-name ,file ,time) +- outbuf) ++ (let* ((env (getenv "SOURCE_DATE_EPOCH")) ++ (time (if env ++ (seconds-to-time (string-to-number env)) ++ time))) ++ (prin1 `(autoloads ,autoloads ,load-name ,file ,time) ++ outbuf)) + (terpri outbuf) + ;; Break that line at spaces, to avoid very long lines. + ;; Make each sub-line into a comment. -- 2.5.0 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2015-10-21 13:05 ` Alex Kost @ 2015-10-21 16:55 ` Ludovic Courtès 2015-11-02 16:50 ` Alex Kost 2016-05-11 14:53 ` Alex Kost 0 siblings, 2 replies; 29+ messages in thread From: Ludovic Courtès @ 2015-10-21 16:55 UTC (permalink / raw) To: Alex Kost; +Cc: guix-devel Alex Kost <alezost@gmail.com> skribis: > I like the idea to honor SOURCE_DATE_EPOCH, so I'm attaching a patch for > this. But now I don't know how to make Guix set this variable during > the build process :-( Need help. Ahem, eventually, we’ll have to set it in ‘gnu-build’ in (guix build gnu-build-system) in the next ‘core-updates’ cycle. In the interim, we can set it in a phase of ‘emacs-build-system’, which would entail few rebuilds. WDYT? > From b8dc19a65980690a636ad7f9f39b3c84991f4975 Mon Sep 17 00:00:00 2001 > From: Alex Kost <alezost@gmail.com> > Date: Wed, 21 Oct 2015 15:59:23 +0300 > Subject: [PATCH] gnu: emacs: Honor 'SOURCE_DATE_EPOCH'. > MIME-Version: 1.0 > Content-Type: text/plain; charset=UTF-8 > Content-Transfer-Encoding: 8bit > > Suggested by Ludovic Courtès <ludo@gnu.org>. > > * gnu/packages/patches/emacs-source-date-epoch.patch: New patch. > * gnu-system.am (dist_patch_DATA): Add it. > * gnu/packages/emacs.scm (emacs)[source]: Use it. LGTM. > +++ b/gnu/packages/patches/emacs-source-date-epoch.patch > @@ -0,0 +1,20 @@ > +Honor SOURCE_DATE_EPOCH variable to avoid non-determinism in generated > +"autoloads" files. > + > +--- a/lisp/emacs-lisp/autoload.el > ++++ b/lisp/emacs-lisp/autoload.el > +@@ -378,8 +378,12 @@ > + "Insert the section-header line, > + which lists the file name and which functions are in it, etc." > + (insert generate-autoload-section-header) > +- (prin1 `(autoloads ,autoloads ,load-name ,file ,time) > +- outbuf) > ++ (let* ((env (getenv "SOURCE_DATE_EPOCH")) > ++ (time (if env > ++ (seconds-to-time (string-to-number env)) > ++ time))) > ++ (prin1 `(autoloads ,autoloads ,load-name ,file ,time) > ++ outbuf)) > + (terpri outbuf) > + ;; Break that line at spaces, to avoid very long lines. > + ;; Make each sub-line into a comment. Could you also submit it upstream, Cc’ing guix-devel and reproducible-builds@lists.alioth.debian.org? Hopefully that is acceptable. (I searched a bit but didn’t find a similar patch by the Debian Reproducible team, but patch-tracker.debian.org is unreachable.) Thanks! Ludo’. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2015-10-21 16:55 ` Ludovic Courtès @ 2015-11-02 16:50 ` Alex Kost 2015-11-03 13:27 ` Ludovic Courtès 2016-05-11 14:53 ` Alex Kost 1 sibling, 1 reply; 29+ messages in thread From: Alex Kost @ 2015-11-02 16:50 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 469 bytes --] Ludovic Courtès (2015-10-21 19:55 +0300) wrote: > Alex Kost <alezost@gmail.com> skribis: > >> I like the idea to honor SOURCE_DATE_EPOCH, so I'm attaching a patch for >> this. But now I don't know how to make Guix set this variable during >> the build process :-( Need help. > > Ahem, eventually, we’ll have to set it in ‘gnu-build’ in (guix build > gnu-build-system) in the next ‘core-updates’ cycle. So would it be as simple as this (?): [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: source-date-epoch.diff --] [-- Type: text/x-patch, Size: 647 bytes --] diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm index ff7646b..92e15d1 100644 --- a/guix/build/gnu-build-system.scm +++ b/guix/build/gnu-build-system.scm @@ -576,6 +576,9 @@ in order. Return #t if all the PHASES succeeded, #f otherwise." ;; Encoding/decoding errors shouldn't be silent. (fluid-set! %default-port-conversion-strategy 'error) + ;; Avoid non-determinism related to generated timestamps. + (setenv "SOURCE_DATE_EPOCH" "1") + ;; The trick is to #:allow-other-keys everywhere, so that each procedure in ;; PHASES can pick the keyword arguments it's interested in. (every (match-lambda [-- Attachment #3: Type: text/plain, Size: 1597 bytes --] > In the interim, we can set it in a phase of ‘emacs-build-system’, which > would entail few rebuilds. > > WDYT? I think it is not so important to make a temporary workaround for master. What about just fixing it in core-updates? >> +++ b/gnu/packages/patches/emacs-source-date-epoch.patch >> @@ -0,0 +1,20 @@ >> +Honor SOURCE_DATE_EPOCH variable to avoid non-determinism in generated >> +"autoloads" files. >> + >> +--- a/lisp/emacs-lisp/autoload.el >> ++++ b/lisp/emacs-lisp/autoload.el >> +@@ -378,8 +378,12 @@ >> + "Insert the section-header line, >> + which lists the file name and which functions are in it, etc." >> + (insert generate-autoload-section-header) >> +- (prin1 `(autoloads ,autoloads ,load-name ,file ,time) >> +- outbuf) >> ++ (let* ((env (getenv "SOURCE_DATE_EPOCH")) >> ++ (time (if env >> ++ (seconds-to-time (string-to-number env)) >> ++ time))) >> ++ (prin1 `(autoloads ,autoloads ,load-name ,file ,time) >> ++ outbuf)) >> + (terpri outbuf) >> + ;; Break that line at spaces, to avoid very long lines. >> + ;; Make each sub-line into a comment. > > Could you also submit it upstream, Cc’ing guix-devel and > reproducible-builds@lists.alioth.debian.org? Hopefully that is > acceptable. (I searched a bit but didn’t find a similar patch by the > Debian Reproducible team, but patch-tracker.debian.org is unreachable.) I'm afraid it's a too hard task for me, sorry. I wouldn't like to mail to so many places. Sorry for the long delay. -- Alex ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2015-11-02 16:50 ` Alex Kost @ 2015-11-03 13:27 ` Ludovic Courtès 2015-11-14 11:32 ` Alex Kost 0 siblings, 1 reply; 29+ messages in thread From: Ludovic Courtès @ 2015-11-03 13:27 UTC (permalink / raw) To: Alex Kost; +Cc: guix-devel Alex Kost <alezost@gmail.com> skribis: > Ludovic Courtès (2015-10-21 19:55 +0300) wrote: > >> Alex Kost <alezost@gmail.com> skribis: >> >>> I like the idea to honor SOURCE_DATE_EPOCH, so I'm attaching a patch for >>> this. But now I don't know how to make Guix set this variable during >>> the build process :-( Need help. >> >> Ahem, eventually, we’ll have to set it in ‘gnu-build’ in (guix build >> gnu-build-system) in the next ‘core-updates’ cycle. > > So would it be as simple as this (?): > > diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm > index ff7646b..92e15d1 100644 > --- a/guix/build/gnu-build-system.scm > +++ b/guix/build/gnu-build-system.scm > @@ -576,6 +576,9 @@ in order. Return #t if all the PHASES succeeded, #f otherwise." > ;; Encoding/decoding errors shouldn't be silent. > (fluid-set! %default-port-conversion-strategy 'error) > > + ;; Avoid non-determinism related to generated timestamps. > + (setenv "SOURCE_DATE_EPOCH" "1") > + > ;; The trick is to #:allow-other-keys everywhere, so that each procedure in > ;; PHASES can pick the keyword arguments it's interested in. > (every (match-lambda Yes, as simple as this. >> In the interim, we can set it in a phase of ‘emacs-build-system’, which >> would entail few rebuilds. >> >> WDYT? > > I think it is not so important to make a temporary workaround for > master. What about just fixing it in core-updates? Sure, makes sense. It’ll be several weeks before it is merged, though, but we can probably live with that. >>> +++ b/gnu/packages/patches/emacs-source-date-epoch.patch >>> @@ -0,0 +1,20 @@ >>> +Honor SOURCE_DATE_EPOCH variable to avoid non-determinism in generated >>> +"autoloads" files. >>> + >>> +--- a/lisp/emacs-lisp/autoload.el >>> ++++ b/lisp/emacs-lisp/autoload.el >>> +@@ -378,8 +378,12 @@ >>> + "Insert the section-header line, >>> + which lists the file name and which functions are in it, etc." >>> + (insert generate-autoload-section-header) >>> +- (prin1 `(autoloads ,autoloads ,load-name ,file ,time) >>> +- outbuf) >>> ++ (let* ((env (getenv "SOURCE_DATE_EPOCH")) >>> ++ (time (if env >>> ++ (seconds-to-time (string-to-number env)) >>> ++ time))) >>> ++ (prin1 `(autoloads ,autoloads ,load-name ,file ,time) >>> ++ outbuf)) >>> + (terpri outbuf) >>> + ;; Break that line at spaces, to avoid very long lines. >>> + ;; Make each sub-line into a comment. >> >> Could you also submit it upstream, Cc’ing guix-devel and >> reproducible-builds@lists.alioth.debian.org? Hopefully that is >> acceptable. (I searched a bit but didn’t find a similar patch by the >> Debian Reproducible team, but patch-tracker.debian.org is unreachable.) > > I'm afraid it's a too hard task for me, sorry. I wouldn't like to mail > to so many places. Or email only emacs-devel if you prefer. An experienced Emacs hacker like you shouldn’t have to be afraid of that. Thanks! Ludo’. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2015-11-03 13:27 ` Ludovic Courtès @ 2015-11-14 11:32 ` Alex Kost 2015-11-14 15:03 ` Ludovic Courtès 0 siblings, 1 reply; 29+ messages in thread From: Alex Kost @ 2015-11-14 11:32 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 3516 bytes --] Ludovic Courtès (2015-11-03 16:27 +0300) wrote: > Alex Kost <alezost@gmail.com> skribis: > >> Ludovic Courtès (2015-10-21 19:55 +0300) wrote: >> >>> Alex Kost <alezost@gmail.com> skribis: >>> >>>> I like the idea to honor SOURCE_DATE_EPOCH, so I'm attaching a patch for >>>> this. But now I don't know how to make Guix set this variable during >>>> the build process :-( Need help. >>> >>> Ahem, eventually, we’ll have to set it in ‘gnu-build’ in (guix build >>> gnu-build-system) in the next ‘core-updates’ cycle. >> >> So would it be as simple as this (?): >> >> diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm >> index ff7646b..92e15d1 100644 >> --- a/guix/build/gnu-build-system.scm >> +++ b/guix/build/gnu-build-system.scm >> @@ -576,6 +576,9 @@ in order. Return #t if all the PHASES succeeded, #f otherwise." >> ;; Encoding/decoding errors shouldn't be silent. >> (fluid-set! %default-port-conversion-strategy 'error) >> >> + ;; Avoid non-determinism related to generated timestamps. >> + (setenv "SOURCE_DATE_EPOCH" "1") >> + >> ;; The trick is to #:allow-other-keys everywhere, so that each procedure in >> ;; PHASES can pick the keyword arguments it's interested in. >> (every (match-lambda > > Yes, as simple as this. Great, thanks! I see that you used (setenv "SOURCE_DATE_EPOCH" "0") in 'tk-update' commit, so I also changed the value to "0". >>> In the interim, we can set it in a phase of ‘emacs-build-system’, which >>> would entail few rebuilds. >>> >>> WDYT? >> >> I think it is not so important to make a temporary workaround for >> master. What about just fixing it in core-updates? > > Sure, makes sense. It’ll be several weeks before it is merged, though, > but we can probably live with that. So is it OK to push the attached patches to 'core-updates'? >>>> +++ b/gnu/packages/patches/emacs-source-date-epoch.patch >>>> @@ -0,0 +1,20 @@ >>>> +Honor SOURCE_DATE_EPOCH variable to avoid non-determinism in generated >>>> +"autoloads" files. >>>> + >>>> +--- a/lisp/emacs-lisp/autoload.el >>>> ++++ b/lisp/emacs-lisp/autoload.el >>>> +@@ -378,8 +378,12 @@ >>>> + "Insert the section-header line, >>>> + which lists the file name and which functions are in it, etc." >>>> + (insert generate-autoload-section-header) >>>> +- (prin1 `(autoloads ,autoloads ,load-name ,file ,time) >>>> +- outbuf) >>>> ++ (let* ((env (getenv "SOURCE_DATE_EPOCH")) >>>> ++ (time (if env >>>> ++ (seconds-to-time (string-to-number env)) >>>> ++ time))) >>>> ++ (prin1 `(autoloads ,autoloads ,load-name ,file ,time) >>>> ++ outbuf)) >>>> + (terpri outbuf) >>>> + ;; Break that line at spaces, to avoid very long lines. >>>> + ;; Make each sub-line into a comment. >>> >>> Could you also submit it upstream, Cc’ing guix-devel and >>> reproducible-builds@lists.alioth.debian.org? Hopefully that is >>> acceptable. (I searched a bit but didn’t find a similar patch by the >>> Debian Reproducible team, but patch-tracker.debian.org is unreachable.) >> >> I'm afraid it's a too hard task for me, sorry. I wouldn't like to mail >> to so many places. > > Or email only emacs-devel if you prefer. > > An experienced Emacs hacker like you shouldn’t have to be afraid of that. After fighting with myself for all these days, I must admit I'm not brave enough for this task yet, sorry. [-- Attachment #2: 0001-build-system-gnu-Set-SOURCE_DATE_EPOCH.patch --] [-- Type: text/x-patch, Size: 1214 bytes --] From a99fb41a7b84dd28c1a5b3f53cf14ca3c43785e7 Mon Sep 17 00:00:00 2001 From: Alex Kost <alezost@gmail.com> Date: Sat, 14 Nov 2015 14:04:43 +0300 Subject: [PATCH 1/2] build-system/gnu: Set 'SOURCE_DATE_EPOCH'. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Suggested by Ludovic Courtès <ludo@gnu.org>. * guix/build/gnu-build-system.scm (gnu-build): Set SOURCE_DATE_EPOCH for deterministic builds. --- guix/build/gnu-build-system.scm | 3 +++ 1 file changed, 3 insertions(+) diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm index ff7646b..011ccf8 100644 --- a/guix/build/gnu-build-system.scm +++ b/guix/build/gnu-build-system.scm @@ -576,6 +576,9 @@ in order. Return #t if all the PHASES succeeded, #f otherwise." ;; Encoding/decoding errors shouldn't be silent. (fluid-set! %default-port-conversion-strategy 'error) + ;; Avoid non-determinism related to generated timestamps. + (setenv "SOURCE_DATE_EPOCH" "0") + ;; The trick is to #:allow-other-keys everywhere, so that each procedure in ;; PHASES can pick the keyword arguments it's interested in. (every (match-lambda -- 2.5.0 [-- Attachment #3: 0002-gnu-emacs-Honor-SOURCE_DATE_EPOCH.patch --] [-- Type: text/x-patch, Size: 3072 bytes --] From 8ee244b6470b11928aa207cf21527ca33cef31f5 Mon Sep 17 00:00:00 2001 From: Alex Kost <alezost@gmail.com> Date: Wed, 21 Oct 2015 15:59:23 +0300 Subject: [PATCH 2/2] gnu: emacs: Honor 'SOURCE_DATE_EPOCH'. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Suggested by Ludovic Courtès <ludo@gnu.org>. * gnu/packages/patches/emacs-source-date-epoch.patch: New patch. * gnu-system.am (dist_patch_DATA): Add it. * gnu/packages/emacs.scm (emacs)[source]: Use it. --- gnu-system.am | 1 + gnu/packages/emacs.scm | 3 ++- gnu/packages/patches/emacs-source-date-epoch.patch | 20 ++++++++++++++++++++ 3 files changed, 23 insertions(+), 1 deletion(-) create mode 100644 gnu/packages/patches/emacs-source-date-epoch.patch diff --git a/gnu-system.am b/gnu-system.am index d43358d..92370a2 100644 --- a/gnu-system.am +++ b/gnu-system.am @@ -437,6 +437,7 @@ dist_patch_DATA = \ gnu/packages/patches/duplicity-test_selection-tmp.patch \ gnu/packages/patches/elfutils-tests-ptrace.patch \ gnu/packages/patches/emacs-exec-path.patch \ + gnu/packages/patches/emacs-source-date-epoch.patch \ gnu/packages/patches/eudev-rules-directory.patch \ gnu/packages/patches/expat-CVE-2015-1283.patch \ gnu/packages/patches/fastcap-mulGlobal.patch \ diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm index 6416b00..9751125 100644 --- a/gnu/packages/emacs.scm +++ b/gnu/packages/emacs.scm @@ -70,7 +70,8 @@ (sha256 (base32 "0kn3rzm91qiswi0cql89kbv6mqn27rwsyjfb8xmwy9m5s8fxfiyx")) - (patches (list (search-patch "emacs-exec-path.patch"))))) + (patches (list (search-patch "emacs-exec-path.patch") + (search-patch "emacs-source-date-epoch.patch"))))) (build-system glib-or-gtk-build-system) (arguments '(#:phases (alist-cons-before diff --git a/gnu/packages/patches/emacs-source-date-epoch.patch b/gnu/packages/patches/emacs-source-date-epoch.patch new file mode 100644 index 0000000..41c03ef --- /dev/null +++ b/gnu/packages/patches/emacs-source-date-epoch.patch @@ -0,0 +1,20 @@ +Honor SOURCE_DATE_EPOCH variable to avoid non-determinism in generated +"autoloads" files. + +--- a/lisp/emacs-lisp/autoload.el ++++ b/lisp/emacs-lisp/autoload.el +@@ -378,8 +378,12 @@ + "Insert the section-header line, + which lists the file name and which functions are in it, etc." + (insert generate-autoload-section-header) +- (prin1 `(autoloads ,autoloads ,load-name ,file ,time) +- outbuf) ++ (let* ((env (getenv "SOURCE_DATE_EPOCH")) ++ (time (if env ++ (seconds-to-time (string-to-number env)) ++ time))) ++ (prin1 `(autoloads ,autoloads ,load-name ,file ,time) ++ outbuf)) + (terpri outbuf) + ;; Break that line at spaces, to avoid very long lines. + ;; Make each sub-line into a comment. -- 2.5.0 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2015-11-14 11:32 ` Alex Kost @ 2015-11-14 15:03 ` Ludovic Courtès 2015-11-14 20:39 ` Alex Kost 0 siblings, 1 reply; 29+ messages in thread From: Ludovic Courtès @ 2015-11-14 15:03 UTC (permalink / raw) To: Alex Kost; +Cc: guix-devel Alex Kost <alezost@gmail.com> skribis: > Ludovic Courtès (2015-11-03 16:27 +0300) wrote: [...] >>> + ;; Avoid non-determinism related to generated timestamps. >>> + (setenv "SOURCE_DATE_EPOCH" "1") >>> + >>> ;; The trick is to #:allow-other-keys everywhere, so that each procedure in >>> ;; PHASES can pick the keyword arguments it's interested in. >>> (every (match-lambda >> >> Yes, as simple as this. > > Great, thanks! I see that you used (setenv "SOURCE_DATE_EPOCH" "0") in > 'tk-update' commit, so I also changed the value to "0". Oh, good point. I don’t know if it matters much (for Python it seems to make no difference), but I think “1” is the safest choice because the mtime on files in the store is set to 1, not 0. I’ll fix that in ‘tk-update’. >>>> Could you also submit it upstream, Cc’ing guix-devel and >>>> reproducible-builds@lists.alioth.debian.org? Hopefully that is >>>> acceptable. (I searched a bit but didn’t find a similar patch by the >>>> Debian Reproducible team, but patch-tracker.debian.org is unreachable.) >>> >>> I'm afraid it's a too hard task for me, sorry. I wouldn't like to mail >>> to so many places. >> >> Or email only emacs-devel if you prefer. >> >> An experienced Emacs hacker like you shouldn’t have to be afraid of that. > > After fighting with myself for all these days, I must admit I'm not > brave enough for this task yet, sorry. You’re not the one to be sorry. It tells more about the weaknesses of the Emacs developer community than about yours. I’ll post it if you don’t mind. > From a99fb41a7b84dd28c1a5b3f53cf14ca3c43785e7 Mon Sep 17 00:00:00 2001 > From: Alex Kost <alezost@gmail.com> > Date: Sat, 14 Nov 2015 14:04:43 +0300 > Subject: [PATCH 1/2] build-system/gnu: Set 'SOURCE_DATE_EPOCH'. > MIME-Version: 1.0 > Content-Type: text/plain; charset=UTF-8 > Content-Transfer-Encoding: 8bit > > Suggested by Ludovic Courtès <ludo@gnu.org>. > > * guix/build/gnu-build-system.scm (gnu-build): Set SOURCE_DATE_EPOCH for > deterministic builds. [...] > + ;; Avoid non-determinism related to generated timestamps. > + (setenv "SOURCE_DATE_EPOCH" "0") OK with that set to 1. > From 8ee244b6470b11928aa207cf21527ca33cef31f5 Mon Sep 17 00:00:00 2001 > From: Alex Kost <alezost@gmail.com> > Date: Wed, 21 Oct 2015 15:59:23 +0300 > Subject: [PATCH 2/2] gnu: emacs: Honor 'SOURCE_DATE_EPOCH'. > MIME-Version: 1.0 > Content-Type: text/plain; charset=UTF-8 > Content-Transfer-Encoding: 8bit > > Suggested by Ludovic Courtès <ludo@gnu.org>. > > * gnu/packages/patches/emacs-source-date-epoch.patch: New patch. > * gnu-system.am (dist_patch_DATA): Add it. > * gnu/packages/emacs.scm (emacs)[source]: Use it. OK. Thank you! Ludo’. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2015-11-14 15:03 ` Ludovic Courtès @ 2015-11-14 20:39 ` Alex Kost 0 siblings, 0 replies; 29+ messages in thread From: Alex Kost @ 2015-11-14 20:39 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel Ludovic Courtès (2015-11-14 18:03 +0300) wrote: > Alex Kost <alezost@gmail.com> skribis: > >> Ludovic Courtès (2015-11-03 16:27 +0300) wrote: > > [...] > >>>> + ;; Avoid non-determinism related to generated timestamps. >>>> + (setenv "SOURCE_DATE_EPOCH" "1") >>>> + >>>> ;; The trick is to #:allow-other-keys everywhere, so that each procedure in >>>> ;; PHASES can pick the keyword arguments it's interested in. >>>> (every (match-lambda >>> >>> Yes, as simple as this. >> >> Great, thanks! I see that you used (setenv "SOURCE_DATE_EPOCH" "0") in >> 'tk-update' commit, so I also changed the value to "0". > > Oh, good point. I don’t know if it matters much (for Python it seems to > make no difference), but I think “1” is the safest choice because the > mtime on files in the store is set to 1, not 0. I’ll fix that in > ‘tk-update’. I see, thanks for explaining. >>>>> Could you also submit it upstream, Cc’ing guix-devel and >>>>> reproducible-builds@lists.alioth.debian.org? Hopefully that is >>>>> acceptable. (I searched a bit but didn’t find a similar patch by the >>>>> Debian Reproducible team, but patch-tracker.debian.org is unreachable.) >>>> >>>> I'm afraid it's a too hard task for me, sorry. I wouldn't like to mail >>>> to so many places. >>> >>> Or email only emacs-devel if you prefer. >>> >>> An experienced Emacs hacker like you shouldn’t have to be afraid of that. >> >> After fighting with myself for all these days, I must admit I'm not >> brave enough for this task yet, sorry. > > You’re not the one to be sorry. It tells more about the weaknesses of > the Emacs developer community than about yours. Oh, no, no; it's totally me. I have problems with communicating with people. > I’ll post it if you don’t mind. Yes, please, it would be a great relief for me, thank you! >> From a99fb41a7b84dd28c1a5b3f53cf14ca3c43785e7 Mon Sep 17 00:00:00 2001 >> From: Alex Kost <alezost@gmail.com> >> Date: Sat, 14 Nov 2015 14:04:43 +0300 >> Subject: [PATCH 1/2] build-system/gnu: Set 'SOURCE_DATE_EPOCH'. >> MIME-Version: 1.0 >> Content-Type: text/plain; charset=UTF-8 >> Content-Transfer-Encoding: 8bit >> >> Suggested by Ludovic Courtès <ludo@gnu.org>. >> >> * guix/build/gnu-build-system.scm (gnu-build): Set SOURCE_DATE_EPOCH for >> deterministic builds. > > [...] > >> + ;; Avoid non-determinism related to generated timestamps. >> + (setenv "SOURCE_DATE_EPOCH" "0") > > OK with that set to 1. Fixed and pushed, thanks. -- Alex ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2015-10-21 16:55 ` Ludovic Courtès 2015-11-02 16:50 ` Alex Kost @ 2016-05-11 14:53 ` Alex Kost 2016-05-16 12:45 ` Ludovic Courtès 1 sibling, 1 reply; 29+ messages in thread From: Alex Kost @ 2016-05-11 14:53 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel I've just noticed that our generated "…-autoloads.el" files still have unwished timestamps (SOURCE_DATE_EPOCH is not honored). To recap: we use "gnu/packages/patches/emacs-source-date-epoch.patch" to modify 'autoload-insert-section-header' function, and it should work, but it doesn't. IIUC this happens because "autoload.elc" file was compiled using the unpatched "autoload.el", but first things first. Try the following recipe in Emacs (installed with Guix): 1. M-: (setenv "SOURCE_DATE_EPOCH" "1") 2. Generate autoloads for any file with autoload cookies. If you don't have such a file at hand, make "/tmp/f1.el" file with this single line: ;;;###autoload(defun f1 nil) Then: "M-x update-file-autoloads </tmp/f1.el> </tmp/auto1.el>" Now open "/tmp/auto1.el" and you'll see a "bad" timestamp like this one: (22323 17532 313464 85000). But! If you reevaluate 'autoload-insert-section-header' and try again the timestamp will be "good": 3. M-x find-function autoload-insert-section-header 4. Reevaluate it: C-M-x 5. M-x update-file-autoloads </tmp/f1.el> </tmp/auto2.el> Now look at "/tmp/auto2.el": it contains (0 1 0 0) timestamp as expected. I looked at the compiled "autoload.elc" file and if I understood it correctly, it was compiled using the unpatched version of "autoload.el" (because there is no mention of SOURCE_DATE_EPOCH there). But I don't understand how it could happen since patching is performed before building. Any ideas? -- Alex ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2016-05-11 14:53 ` Alex Kost @ 2016-05-16 12:45 ` Ludovic Courtès 2016-05-16 20:52 ` Alex Kost 2016-05-19 8:12 ` Alex Kost 0 siblings, 2 replies; 29+ messages in thread From: Ludovic Courtès @ 2016-05-16 12:45 UTC (permalink / raw) To: Alex Kost; +Cc: guix-devel Hi! Alex Kost <alezost@gmail.com> skribis: > To recap: we use "gnu/packages/patches/emacs-source-date-epoch.patch" to > modify 'autoload-insert-section-header' function, and it should work, > but it doesn't. IIUC this happens because "autoload.elc" file was > compiled using the unpatched "autoload.el", but first things first. Try > the following recipe in Emacs (installed with Guix): > > 1. M-: (setenv "SOURCE_DATE_EPOCH" "1") > > 2. Generate autoloads for any file with autoload cookies. If you don't > have such a file at hand, make "/tmp/f1.el" file with this single > line: > > ;;;###autoload(defun f1 nil) > > Then: "M-x update-file-autoloads </tmp/f1.el> </tmp/auto1.el>" > > Now open "/tmp/auto1.el" and you'll see a "bad" timestamp like this one: > (22323 17532 313464 85000). > > But! If you reevaluate 'autoload-insert-section-header' and try again > the timestamp will be "good": > > 3. M-x find-function autoload-insert-section-header > > 4. Reevaluate it: C-M-x > > 5. M-x update-file-autoloads </tmp/f1.el> </tmp/auto2.el> > > Now look at "/tmp/auto2.el": it contains (0 1 0 0) timestamp as > expected. Weird! > I looked at the compiled "autoload.elc" file and if I understood it > correctly, it was compiled using the unpatched version of "autoload.el" > (because there is no mention of SOURCE_DATE_EPOCH there). Indeed. > But I don't understand how it could happen since patching is performed > before building. Any ideas? I think I have one: --8<---------------cut here---------------start------------->8--- $ git describe v0.10.0-798-g8a7680a $ tar tvf $(./pre-inst-env guix build -S emacs) |grep 'autoload\.el' -rw-r--r-- root/root 37292 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el -rw-r--r-- root/root 37127 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el.orig -rw-r--r-- root/root 22624 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.elc --8<---------------cut here---------------end--------------->8--- Upstream’s tarball already includes those three files. Ideally, we should remove all the .elc files and rebuild them, but maybe there are bootstrapping issues. Would you be willing to give it a try? :-) Thanks, Ludo’. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2016-05-16 12:45 ` Ludovic Courtès @ 2016-05-16 20:52 ` Alex Kost 2016-05-17 9:12 ` Ludovic Courtès 2016-05-19 8:12 ` Alex Kost 1 sibling, 1 reply; 29+ messages in thread From: Alex Kost @ 2016-05-16 20:52 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel Ludovic Courtès (2016-05-16 15:45 +0300) wrote: > Alex Kost <alezost@gmail.com> skribis: [...] >> I looked at the compiled "autoload.elc" file and if I understood it >> correctly, it was compiled using the unpatched version of "autoload.el" >> (because there is no mention of SOURCE_DATE_EPOCH there). > > Indeed. > >> But I don't understand how it could happen since patching is performed >> before building. Any ideas? > > I think I have one: > > $ git describe > v0.10.0-798-g8a7680a > $ tar tvf $(./pre-inst-env guix build -S emacs) |grep 'autoload\.el' > -rw-r--r-- root/root 37292 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el > -rw-r--r-- root/root 37127 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el.orig > -rw-r--r-- root/root 22624 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.elc > > Upstream’s tarball already includes those three files. IIUC this source is after applying our patches (including "emacs-source-date-epoch.patch"): - “autoload.el.orig” is the original file from the upstream; - “autoload.el” is the patched one; - “autoload.elc” is the compiled file also from the upstream (obviously it doesn't honor SOURCE_DATE_EPOCH). Thanks, the mystery is solved now. This is an unpleasant surprise, I didn't know that emacs release comes with the compiled files. > Ideally, we > should remove all the .elc files and rebuild them, but maybe there are > bootstrapping issues. > > Would you be willing to give it a try? :-) I'll see what I can do here. -- Alex ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2016-05-16 20:52 ` Alex Kost @ 2016-05-17 9:12 ` Ludovic Courtès 2016-05-19 16:29 ` Alex Kost 0 siblings, 1 reply; 29+ messages in thread From: Ludovic Courtès @ 2016-05-17 9:12 UTC (permalink / raw) To: Alex Kost; +Cc: guix-devel Alex Kost <alezost@gmail.com> skribis: > Ludovic Courtès (2016-05-16 15:45 +0300) wrote: > >> Alex Kost <alezost@gmail.com> skribis: > [...] >>> I looked at the compiled "autoload.elc" file and if I understood it >>> correctly, it was compiled using the unpatched version of "autoload.el" >>> (because there is no mention of SOURCE_DATE_EPOCH there). >> >> Indeed. >> >>> But I don't understand how it could happen since patching is performed >>> before building. Any ideas? >> >> I think I have one: >> >> $ git describe >> v0.10.0-798-g8a7680a >> $ tar tvf $(./pre-inst-env guix build -S emacs) |grep 'autoload\.el' >> -rw-r--r-- root/root 37292 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el >> -rw-r--r-- root/root 37127 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el.orig >> -rw-r--r-- root/root 22624 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.elc >> >> Upstream’s tarball already includes those three files. > > IIUC this source is after applying our patches (including > "emacs-source-date-epoch.patch"): > > - “autoload.el.orig” is the original file from the upstream; Indeed, this one isn’t present in upstream’s tarball: --8<---------------cut here---------------start------------->8--- $ wget -q -O - ftp://ftp.gnu.org/gnu/emacs/emacs-24.5.tar.xz | tar tJvf - | grep 'autoload\.' -rw-rw-r-- nico/nico 37127 2015-04-02 09:23 emacs-24.5/lisp/emacs-lisp/autoload.el -rw-r--r-- nico/nico 22624 2015-04-08 19:16 emacs-24.5/lisp/emacs-lisp/autoload.elc --8<---------------cut here---------------end--------------->8--- How come we’re introducing this one? I thought ‘patch’ did not produce .orig files unless the patch failed to apply, but here the patch correctly applies, only with a small offset (can be seen by running ‘guix build -S emacs --check’): --8<---------------cut here---------------start------------->8--- patching file lisp/loadup.el patching file lisp/emacs-lisp/autoload.el Hunk #1 succeeded at 361 (offset -17 lines). --8<---------------cut here---------------end--------------->8--- Apparently we have to use ‘--no-backup-if-mismatch’ to avoid that. > Thanks, the mystery is solved now. This is an unpleasant surprise, I > didn't know that emacs release comes with the compiled files. Yeah. :-/ Ludo’. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2016-05-17 9:12 ` Ludovic Courtès @ 2016-05-19 16:29 ` Alex Kost 2016-05-20 12:02 ` Ludovic Courtès 0 siblings, 1 reply; 29+ messages in thread From: Alex Kost @ 2016-05-19 16:29 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1815 bytes --] Ludovic Courtès (2016-05-17 12:12 +0300) wrote: > Alex Kost <alezost@gmail.com> skribis: > >> Ludovic Courtès (2016-05-16 15:45 +0300) wrote: [...] >>> $ git describe >>> v0.10.0-798-g8a7680a >>> $ tar tvf $(./pre-inst-env guix build -S emacs) |grep 'autoload\.el' >>> -rw-r--r-- root/root 37292 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el >>> -rw-r--r-- root/root 37127 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el.orig >>> -rw-r--r-- root/root 22624 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.elc >>> >>> Upstream’s tarball already includes those three files. >> >> IIUC this source is after applying our patches (including >> "emacs-source-date-epoch.patch"): >> >> - “autoload.el.orig” is the original file from the upstream; > > Indeed, this one isn’t present in upstream’s tarball: > > $ wget -q -O - ftp://ftp.gnu.org/gnu/emacs/emacs-24.5.tar.xz | tar tJvf - | grep 'autoload\.' > -rw-rw-r-- nico/nico 37127 2015-04-02 09:23 emacs-24.5/lisp/emacs-lisp/autoload.el > -rw-r--r-- nico/nico 22624 2015-04-08 19:16 emacs-24.5/lisp/emacs-lisp/autoload.elc > > How come we’re introducing this one? I thought ‘patch’ did not produce > .orig files unless the patch failed to apply, but here the patch > correctly applies, only with a small offset (can be seen by running > ‘guix build -S emacs --check’): > > patching file lisp/loadup.el > patching file lisp/emacs-lisp/autoload.el > Hunk #1 succeeded at 361 (offset -17 lines). Indeed, I also didn't know that "patch" produces such .orig files when a patch applies with offset. > Apparently we have to use ‘--no-backup-if-mismatch’ to avoid that. You even found the flag, thanks! So is it OK to apply the attached patch to core-updates? [-- Attachment #2: 0001-packages-Use-no-backup-if-mismatch-for-patching.patch --] [-- Type: text/x-patch, Size: 1503 bytes --] From bcd636eff01f39583e8742b123d51550f9500795 Mon Sep 17 00:00:00 2001 From: Alex Kost <alezost@gmail.com> Date: Thu, 19 May 2016 19:11:58 +0300 Subject: [PATCH] packages: Use '--no-backup-if-mismatch' for patching. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Suggested-by: Ludovic Courtès <ludo@gnu.org> * guix/packages.scm (patch-and-repack)[build]: Use '--no-backup-if-mismatch' patch flag to avoid making *.orig files. --- guix/packages.scm | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/guix/packages.scm b/guix/packages.scm index d62d1f3..a7b502c 100644 --- a/guix/packages.scm +++ b/guix/packages.scm @@ -464,9 +464,11 @@ IMPORTED-MODULES specify modules to use/import for use by SNIPPET." (format (current-error-port) "applying '~a'...~%" patch) ;; Use '--force' so that patches that do not apply perfectly are - ;; rejected. + ;; rejected. Use '--no-backup-if-mismatch' to prevent making + ;; "*.orig" file if a patch is applied with offset. (zero? (system* (string-append #+patch "/bin/patch") - "--force" #+@flags "--input" patch))) + "--force" "--no-backup-if-mismatch" + #+@flags "--input" patch))) (define (first-file directory) ;; Return the name of the first file in DIRECTORY. -- 2.7.3 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2016-05-19 16:29 ` Alex Kost @ 2016-05-20 12:02 ` Ludovic Courtès 2016-05-20 13:38 ` Leo Famulari 2016-05-20 21:14 ` Alex Kost 0 siblings, 2 replies; 29+ messages in thread From: Ludovic Courtès @ 2016-05-20 12:02 UTC (permalink / raw) To: Alex Kost; +Cc: guix-devel Alex Kost <alezost@gmail.com> skribis: > Ludovic Courtès (2016-05-17 12:12 +0300) wrote: > >> Alex Kost <alezost@gmail.com> skribis: >> >>> Ludovic Courtès (2016-05-16 15:45 +0300) wrote: > [...] >>>> $ git describe >>>> v0.10.0-798-g8a7680a >>>> $ tar tvf $(./pre-inst-env guix build -S emacs) |grep 'autoload\.el' >>>> -rw-r--r-- root/root 37292 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el >>>> -rw-r--r-- root/root 37127 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el.orig >>>> -rw-r--r-- root/root 22624 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.elc >>>> >>>> Upstream’s tarball already includes those three files. >>> >>> IIUC this source is after applying our patches (including >>> "emacs-source-date-epoch.patch"): >>> >>> - “autoload.el.orig” is the original file from the upstream; >> >> Indeed, this one isn’t present in upstream’s tarball: >> >> $ wget -q -O - ftp://ftp.gnu.org/gnu/emacs/emacs-24.5.tar.xz | tar tJvf - | grep 'autoload\.' >> -rw-rw-r-- nico/nico 37127 2015-04-02 09:23 emacs-24.5/lisp/emacs-lisp/autoload.el >> -rw-r--r-- nico/nico 22624 2015-04-08 19:16 emacs-24.5/lisp/emacs-lisp/autoload.elc >> >> How come we’re introducing this one? I thought ‘patch’ did not produce >> .orig files unless the patch failed to apply, but here the patch >> correctly applies, only with a small offset (can be seen by running >> ‘guix build -S emacs --check’): >> >> patching file lisp/loadup.el >> patching file lisp/emacs-lisp/autoload.el >> Hunk #1 succeeded at 361 (offset -17 lines). > > Indeed, I also didn't know that "patch" produces such .orig files when a > patch applies with offset. > >> Apparently we have to use ‘--no-backup-if-mismatch’ to avoid that. > > You even found the flag, thanks! So is it OK to apply the attached > patch to core-updates? The patch LGTM, but since this is a full-rebuild change, I prefer keep it for the next core-updates cycle. Let’s try not to forget about it. :-) Thanks, Ludo’. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2016-05-20 12:02 ` Ludovic Courtès @ 2016-05-20 13:38 ` Leo Famulari 2016-05-21 10:49 ` Alex Kost 2016-05-20 21:14 ` Alex Kost 1 sibling, 1 reply; 29+ messages in thread From: Leo Famulari @ 2016-05-20 13:38 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel, Alex Kost On Fri, May 20, 2016 at 02:02:41PM +0200, Ludovic Courtès wrote: > The patch LGTM, but since this is a full-rebuild change, I prefer keep > it for the next core-updates cycle. Let’s try not to forget about it. Should we create "core-updates-next" or something like that? I have a few other patches I don't want to forget about either. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2016-05-20 13:38 ` Leo Famulari @ 2016-05-21 10:49 ` Alex Kost 2016-05-21 21:02 ` Ludovic Courtès 0 siblings, 1 reply; 29+ messages in thread From: Alex Kost @ 2016-05-21 10:49 UTC (permalink / raw) To: Leo Famulari; +Cc: guix-devel Leo Famulari (2016-05-20 16:38 +0300) wrote: > On Fri, May 20, 2016 at 02:02:41PM +0200, Ludovic Courtès wrote: >> The patch LGTM, but since this is a full-rebuild change, I prefer keep >> it for the next core-updates cycle. Let’s try not to forget about it. > > Should we create "core-updates-next" or something like that? I have a > few other patches I don't want to forget about either. Good idea! I agree. -- Alex ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2016-05-21 10:49 ` Alex Kost @ 2016-05-21 21:02 ` Ludovic Courtès 2016-05-24 9:07 ` Alex Kost 0 siblings, 1 reply; 29+ messages in thread From: Ludovic Courtès @ 2016-05-21 21:02 UTC (permalink / raw) To: Alex Kost; +Cc: guix-devel Alex Kost <alezost@gmail.com> skribis: > Leo Famulari (2016-05-20 16:38 +0300) wrote: > >> On Fri, May 20, 2016 at 02:02:41PM +0200, Ludovic Courtès wrote: >>> The patch LGTM, but since this is a full-rebuild change, I prefer keep >>> it for the next core-updates cycle. Let’s try not to forget about it. >> >> Should we create "core-updates-next" or something like that? I have a >> few other patches I don't want to forget about either. > > Good idea! I agree. OK, go ahead then. :-) We’ll rename it to ‘core-updates’ when the current one is deleted. Ludo’. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2016-05-21 21:02 ` Ludovic Courtès @ 2016-05-24 9:07 ` Alex Kost 0 siblings, 0 replies; 29+ messages in thread From: Alex Kost @ 2016-05-24 9:07 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel Ludovic Courtès (2016-05-22 00:02 +0300) wrote: > Alex Kost <alezost@gmail.com> skribis: > >> Leo Famulari (2016-05-20 16:38 +0300) wrote: >> >>> On Fri, May 20, 2016 at 02:02:41PM +0200, Ludovic Courtès wrote: >>>> The patch LGTM, but since this is a full-rebuild change, I prefer keep >>>> it for the next core-updates cycle. Let’s try not to forget about it. >>> >>> Should we create "core-updates-next" or something like that? I have a >>> few other patches I don't want to forget about either. >> >> Good idea! I agree. > > OK, go ahead then. :-) We’ll rename it to ‘core-updates’ when the > current one is deleted. I went ahead. Leo, now it's you turn to commit your patches to "core-updates-next". I based it on master, not sure if it should be based on "core-updates", but anyway, it's done already :-) -- Alex ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2016-05-20 12:02 ` Ludovic Courtès 2016-05-20 13:38 ` Leo Famulari @ 2016-05-20 21:14 ` Alex Kost 1 sibling, 0 replies; 29+ messages in thread From: Alex Kost @ 2016-05-20 21:14 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel Ludovic Courtès (2016-05-20 15:02 +0300) wrote: > Alex Kost <alezost@gmail.com> skribis: > >> Ludovic Courtès (2016-05-17 12:12 +0300) wrote: [...] >>> Apparently we have to use ‘--no-backup-if-mismatch’ to avoid that. >> >> You even found the flag, thanks! So is it OK to apply the attached >> patch to core-updates? > > The patch LGTM, but since this is a full-rebuild change, I prefer keep > it for the next core-updates cycle. Let’s try not to forget about it. > :-) OK, I will not forget, thanks! -- Alex ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2016-05-16 12:45 ` Ludovic Courtès 2016-05-16 20:52 ` Alex Kost @ 2016-05-19 8:12 ` Alex Kost 2016-05-19 12:56 ` Ludovic Courtès 1 sibling, 1 reply; 29+ messages in thread From: Alex Kost @ 2016-05-19 8:12 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 808 bytes --] Ludovic Courtès (2016-05-16 15:45 +0300) wrote: > Alex Kost <alezost@gmail.com> skribis: > >> To recap: we use "gnu/packages/patches/emacs-source-date-epoch.patch" to >> modify 'autoload-insert-section-header' function, and it should work, >> but it doesn't. IIUC this happens because "autoload.elc" file was >> compiled using the unpatched "autoload.el", but first things first. Try >> the following recipe in Emacs (installed with Guix): [...] > Ideally, we > should remove all the .elc files and rebuild them, but maybe there are > bootstrapping issues. The attached patch does this. Happily there were no any issues. Now emacs packages will honor SOURCE_DATE_EPOCH. Even autoloads that comes with emacs itself (loaddefs.el, cl-loaddefs.el, etc.) will have a proper timestamp. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-gnu-emacs-Remove-.elc-from-the-release-tarball.patch --] [-- Type: text/x-patch, Size: 1365 bytes --] From ca571f7631bf77ddc8ad6257fe165b4ff0ef5e6b Mon Sep 17 00:00:00 2001 From: Alex Kost <alezost@gmail.com> Date: Thu, 19 May 2016 11:01:40 +0300 Subject: [PATCH] gnu: emacs: Remove *.elc from the release tarball. * gnu/packages/emacs.scm (emacs)[arguments]: Add 'remove-compiled-elisp' phase. --- gnu/packages/emacs.scm | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm index 32ed722..0c15f63 100644 --- a/gnu/packages/emacs.scm +++ b/gnu/packages/emacs.scm @@ -91,6 +91,15 @@ (substitute* (find-files "." "^Makefile\\.in$") (("/bin/pwd") "pwd")))) + (add-after 'configure 'remove-compiled-elisp + (lambda _ + ;; Emacs comes with compiled elisp (*.elc) and generated + ;; autoloads (*loaddefs.el) files. This does not allow us to use + ;; "emacs-source-date-epoch.patch" effectively, so remove these + ;; files (using 'make bootstrap-clean'), as 'make' will recreate + ;; them. + (with-directory-excursion "lisp" + (zero? (system* "make" "bootstrap-clean"))))) (add-after 'install 'remove-info.info (lambda* (#:key outputs #:allow-other-keys) ;; Remove 'info.info', which is provided by Texinfo <= 6.0. -- 2.7.3 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2016-05-19 8:12 ` Alex Kost @ 2016-05-19 12:56 ` Ludovic Courtès 2016-05-21 10:31 ` Alex Kost 0 siblings, 1 reply; 29+ messages in thread From: Ludovic Courtès @ 2016-05-19 12:56 UTC (permalink / raw) To: Alex Kost; +Cc: guix-devel Alex Kost <alezost@gmail.com> skribis: > Ludovic Courtès (2016-05-16 15:45 +0300) wrote: > >> Alex Kost <alezost@gmail.com> skribis: >> >>> To recap: we use "gnu/packages/patches/emacs-source-date-epoch.patch" to >>> modify 'autoload-insert-section-header' function, and it should work, >>> but it doesn't. IIUC this happens because "autoload.elc" file was >>> compiled using the unpatched "autoload.el", but first things first. Try >>> the following recipe in Emacs (installed with Guix): > [...] >> Ideally, we >> should remove all the .elc files and rebuild them, but maybe there are >> bootstrapping issues. > > The attached patch does this. Happily there were no any issues. Now > emacs packages will honor SOURCE_DATE_EPOCH. Even autoloads that comes > with emacs itself (loaddefs.el, cl-loaddefs.el, etc.) will have a proper > timestamp. Awesome! > From ca571f7631bf77ddc8ad6257fe165b4ff0ef5e6b Mon Sep 17 00:00:00 2001 > From: Alex Kost <alezost@gmail.com> > Date: Thu, 19 May 2016 11:01:40 +0300 > Subject: [PATCH] gnu: emacs: Remove *.elc from the release tarball. > > * gnu/packages/emacs.scm (emacs)[arguments]: Add 'remove-compiled-elisp' > phase. > --- > gnu/packages/emacs.scm | 9 +++++++++ > 1 file changed, 9 insertions(+) > > diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm > index 32ed722..0c15f63 100644 > --- a/gnu/packages/emacs.scm > +++ b/gnu/packages/emacs.scm > @@ -91,6 +91,15 @@ > (substitute* (find-files "." "^Makefile\\.in$") > (("/bin/pwd") > "pwd")))) > + (add-after 'configure 'remove-compiled-elisp > + (lambda _ > + ;; Emacs comes with compiled elisp (*.elc) and generated > + ;; autoloads (*loaddefs.el) files. This does not allow us to use > + ;; "emacs-source-date-epoch.patch" effectively, so remove these > + ;; files (using 'make bootstrap-clean'), as 'make' will recreate > + ;; them. > + (with-directory-excursion "lisp" > + (zero? (system* "make" "bootstrap-clean"))))) I would rather do it in a ‘snippet’ so that ‘guix build -S emacs’ returns the cleaned-up source. However, the snippet would have to duplicate the logic of this makefile rule, which might not be desirable (depends on how complex this rule is). If you think it’s best to keep this way, please push! Thanks, Ludo’. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2016-05-19 12:56 ` Ludovic Courtès @ 2016-05-21 10:31 ` Alex Kost 2016-05-21 21:01 ` Ludovic Courtès 0 siblings, 1 reply; 29+ messages in thread From: Alex Kost @ 2016-05-21 10:31 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 1864 bytes --] Ludovic Courtès (2016-05-19 15:56 +0300) wrote: >> From ca571f7631bf77ddc8ad6257fe165b4ff0ef5e6b Mon Sep 17 00:00:00 2001 >> From: Alex Kost <alezost@gmail.com> >> Date: Thu, 19 May 2016 11:01:40 +0300 >> Subject: [PATCH] gnu: emacs: Remove *.elc from the release tarball. >> >> * gnu/packages/emacs.scm (emacs)[arguments]: Add 'remove-compiled-elisp' >> phase. >> --- >> gnu/packages/emacs.scm | 9 +++++++++ >> 1 file changed, 9 insertions(+) >> >> diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm >> index 32ed722..0c15f63 100644 >> --- a/gnu/packages/emacs.scm >> +++ b/gnu/packages/emacs.scm >> @@ -91,6 +91,15 @@ >> (substitute* (find-files "." "^Makefile\\.in$") >> (("/bin/pwd") >> "pwd")))) >> + (add-after 'configure 'remove-compiled-elisp >> + (lambda _ >> + ;; Emacs comes with compiled elisp (*.elc) and generated >> + ;; autoloads (*loaddefs.el) files. This does not allow us to use >> + ;; "emacs-source-date-epoch.patch" effectively, so remove these >> + ;; files (using 'make bootstrap-clean'), as 'make' will recreate >> + ;; them. >> + (with-directory-excursion "lisp" >> + (zero? (system* "make" "bootstrap-clean"))))) > > I would rather do it in a ‘snippet’ so that ‘guix build -S emacs’ > returns the cleaned-up source. Hm, I didn't think about it; yeah, I agree it would be better, thanks! > However, the snippet would have to duplicate the logic of this makefile > rule, which might not be desirable (depends on how complex this rule > is). If you think it’s best to keep this way, please push! The rule is rather simple, it just removes all compiled and generated files, so attached is the version with snippet. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-gnu-emacs-Remove-.elc-and-autoloads-from-the-tarball.patch --] [-- Type: text/x-patch, Size: 1602 bytes --] From 13c2e7123d3b8f7dda5814c2ac00acfe806cec3b Mon Sep 17 00:00:00 2001 From: Alex Kost <alezost@gmail.com> Date: Thu, 19 May 2016 11:01:40 +0300 Subject: [PATCH] gnu: emacs: Remove *.elc and autoloads from the tarball. * gnu/packages/emacs.scm (emacs)[source]: Add 'snippet' to remove compiled and generated elisp files. --- gnu/packages/emacs.scm | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm index 18898e9..ae02a07 100644 --- a/gnu/packages/emacs.scm +++ b/gnu/packages/emacs.scm @@ -80,7 +80,18 @@ (base32 "0kn3rzm91qiswi0cql89kbv6mqn27rwsyjfb8xmwy9m5s8fxfiyx")) (patches (search-patches "emacs-exec-path.patch" - "emacs-source-date-epoch.patch")))) + "emacs-source-date-epoch.patch")) + (modules '((guix build utils))) + (snippet + ;; Delete the bundled byte-compiled elisp files and + ;; generated autoloads. + '(with-directory-excursion "lisp" + (for-each delete-file + (append (find-files "." "\\.elc$") + (find-files "." "loaddefs\\.el$") + ;; This is the only "autoloads" file that + ;; does not have "*loaddefs.el" name. + '("eshell/esh-groups.el"))))))) (build-system glib-or-gtk-build-system) (arguments `(#:phases -- 2.8.2 ^ permalink raw reply related [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2016-05-21 10:31 ` Alex Kost @ 2016-05-21 21:01 ` Ludovic Courtès 2016-05-24 9:00 ` Alex Kost 0 siblings, 1 reply; 29+ messages in thread From: Ludovic Courtès @ 2016-05-21 21:01 UTC (permalink / raw) To: Alex Kost; +Cc: guix-devel Alex Kost <alezost@gmail.com> skribis: > From 13c2e7123d3b8f7dda5814c2ac00acfe806cec3b Mon Sep 17 00:00:00 2001 > From: Alex Kost <alezost@gmail.com> > Date: Thu, 19 May 2016 11:01:40 +0300 > Subject: [PATCH] gnu: emacs: Remove *.elc and autoloads from the tarball. > > * gnu/packages/emacs.scm (emacs)[source]: Add 'snippet' to remove > compiled and generated elisp files. Perfect, thanks! Ludo’. ^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: Timestamps in ...-autoloads.el files 2016-05-21 21:01 ` Ludovic Courtès @ 2016-05-24 9:00 ` Alex Kost 0 siblings, 0 replies; 29+ messages in thread From: Alex Kost @ 2016-05-24 9:00 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel Ludovic Courtès (2016-05-22 00:01 +0300) wrote: > Alex Kost <alezost@gmail.com> skribis: > >> From 13c2e7123d3b8f7dda5814c2ac00acfe806cec3b Mon Sep 17 00:00:00 2001 >> From: Alex Kost <alezost@gmail.com> >> Date: Thu, 19 May 2016 11:01:40 +0300 >> Subject: [PATCH] gnu: emacs: Remove *.elc and autoloads from the tarball. >> >> * gnu/packages/emacs.scm (emacs)[source]: Add 'snippet' to remove >> compiled and generated elisp files. > > Perfect, thanks! Committed. -- Alex ^ permalink raw reply [flat|nested] 29+ messages in thread
end of thread, other threads:[~2016-05-24 9:07 UTC | newest] Thread overview: 29+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-10-19 23:09 Challenge substitute servers! Ludovic Courtès 2015-10-20 0:17 ` Daniel Pimentel 2015-10-20 14:38 ` [PATCHES] emacs: Changes for 'guix challenge' Alex Kost 2015-10-20 14:58 ` Ludovic Courtès 2015-10-20 17:40 ` Timestamps in ...-autoloads.el files Alex Kost 2015-10-20 19:38 ` Ludovic Courtès 2015-10-21 13:05 ` Alex Kost 2015-10-21 16:55 ` Ludovic Courtès 2015-11-02 16:50 ` Alex Kost 2015-11-03 13:27 ` Ludovic Courtès 2015-11-14 11:32 ` Alex Kost 2015-11-14 15:03 ` Ludovic Courtès 2015-11-14 20:39 ` Alex Kost 2016-05-11 14:53 ` Alex Kost 2016-05-16 12:45 ` Ludovic Courtès 2016-05-16 20:52 ` Alex Kost 2016-05-17 9:12 ` Ludovic Courtès 2016-05-19 16:29 ` Alex Kost 2016-05-20 12:02 ` Ludovic Courtès 2016-05-20 13:38 ` Leo Famulari 2016-05-21 10:49 ` Alex Kost 2016-05-21 21:02 ` Ludovic Courtès 2016-05-24 9:07 ` Alex Kost 2016-05-20 21:14 ` Alex Kost 2016-05-19 8:12 ` Alex Kost 2016-05-19 12:56 ` Ludovic Courtès 2016-05-21 10:31 ` Alex Kost 2016-05-21 21:01 ` Ludovic Courtès 2016-05-24 9:00 ` Alex Kost
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).