Challenge substitute servers!

Challenge substitute servers!

[Ludovic Courtès]

Hello!

I’m happy to announce the new ‘guix challenge’ command!  (Documentation
below.)

The goal of the command is, ideally, to report malicious or corrupt
substitute servers.  In practice though, many package build processes
are still non-deterministic, so chances are that, when a discrepancy
reported by ‘guix challenge’, it is due to a non-deterministic build
process–something we must fix.

Here’s what it reports on my machine (compared to hydra.gnunet.org,
which is one of the build machines behind hydra.gnu.org):

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix challenge --substitute-urls=http://hydra.gnunet.org
finding garbage collector roots...
determining live/dead paths...
updating list of substitutes from 'http://hydra.gnunet.org'... 100.0%
/gnu/store/1c96zwwg6lnh6v9n3m0q30pg0x2m0v5c-openssl-1.0.2d contents differ:
  local hash: 0725l22r5jnzazaacncwsvp9kgf42266ayyp814v7djxs7nk963q
  http://hydra.gnunet.org/nar/1c96zwwg6lnh6v9n3m0q30pg0x2m0v5c-openssl-1.0.2d: 1zy4fmaaqcnjrzzajkdn3f5gmjk754b43qkq47llbyak9z0qjyim
/gnu/store/3b7sjkz1ps719fkl53mcxzmjysd95i5c-emacs-deferred-0.3.2 contents differ:
  local hash: 0hh4d5plz4ph93yqcqvyy8dfbdidwdmy93kikjgnkglxr0lw8qss
  http://hydra.gnunet.org/nar/3b7sjkz1ps719fkl53mcxzmjysd95i5c-emacs-deferred-0.3.2: 0d0k5sl6zwbwb4yj0ns118bq6nxqaa8d4kr189w8lb7sma4ji5y1
/gnu/store/5zavfkmrax6z93q85q5nifbwkfz4704m-git-2.5.0 contents differ:
  local hash: 00p3bmryhjxrhpn2gxs2fy0a15lnip05l97205pgbk5ra395hyha
  http://hydra.gnunet.org/nar/5zavfkmrax6z93q85q5nifbwkfz4704m-git-2.5.0: 069nb85bv4d4a6slrwjdy8v1cn4cwspm3kdbmyb81d6zckj3nq9f
/gnu/store/76yq0qvcbjjljk8my6x06ayssph573xx-pius-2.1.1 contents differ:
  local hash: 0k4v3m9z1zp8xzzizb7d8kjj72f9172xv078sq4wl73vnq9ig3ax
  http://hydra.gnunet.org/nar/76yq0qvcbjjljk8my6x06ayssph573xx-pius-2.1.1: 1cy25x1a4fzq5rk0pmvc8xhwyffnqz95h2bpvqsz2mpvlbccy0gs
/gnu/store/avrrq8sl1ihq0vrhjrilszgmg7ifhqdw-guile-ssh-0.8.0 contents differ:
  local hash: 1qxjnirrd24djxwrh1wmsdg3qhhigymaqg673nri0d5dn87dihmc
  http://hydra.gnunet.org/nar/avrrq8sl1ihq0vrhjrilszgmg7ifhqdw-guile-ssh-0.8.0: 1js93xgwdrbq85mzl655dc7f6hb742bmvv46jij7b73npfzzjml9
/gnu/store/bc081jj8rx640ism3gm6wlpb7jamg080-dmd-0.2.01 contents differ:
  local hash: 1ngmjzscwva6w0wy3ahmq4r6svdpa8pq7f5kz273qld3k2xrg54v
  http://hydra.gnunet.org/nar/bc081jj8rx640ism3gm6wlpb7jamg080-dmd-0.2.01: 0hfcg2qvdfrzyfgwn1p1j7k0vv93xk5z4y9kr22gyakf9f9jjfix
/gnu/store/cmb889dhvnx9kk6gdvqjm14w4j6pqalq-guix-0.8.3.abbe2c6 contents differ:
  local hash: 0p3mr40xk9q8bw26h8gkz2nx6n0csrmdh05zb1x85i0zs9pfd2f8
  http://hydra.gnunet.org/nar/cmb889dhvnx9kk6gdvqjm14w4j6pqalq-guix-0.8.3.abbe2c6: 1aca8zkvcff38y3s9khf565mlcz8xgl8p6c93jbrin87k8z2islq
/gnu/store/fkl97li2g7gqyw5bq09q2r0hkxla54lq-ath9k-htc-firmware-1.4.0 contents differ:
  local hash: 0r4lysb0skx7dpyh1qsk2mamkwwd4a7yldr1yy2dpr7bc0nk4z8n
  http://hydra.gnunet.org/nar/fkl97li2g7gqyw5bq09q2r0hkxla54lq-ath9k-htc-firmware-1.4.0: 1aia3539xvzaff4s11ga35f7iahch6rbryqzh5adlmx3gffpy2y7
/gnu/store/j0qc3ghc7ajja6k9c35y8ssxjzxrsy95-emacs-debbugs-0.7 contents differ:
  local hash: 0wfczpznl9pdalf4sp23dfp63pvrk9jsw7znvw8axj24wv9rbk4c
  http://hydra.gnunet.org/nar/j0qc3ghc7ajja6k9c35y8ssxjzxrsy95-emacs-debbugs-0.7: 19bagzab1d6xabhy3av863ab06zkcxya4lc7q62za74p7qpj112g
/gnu/store/n5q0gbn01w5m2ic9rc2xq4kj2faglg6s-emacs-typo-1.1 contents differ:
  local hash: 1yly4y92vphdsnqnmdqwsz019fy11jkbz31sl6ysly9j3fmnhq2m
  http://hydra.gnunet.org/nar/n5q0gbn01w5m2ic9rc2xq4kj2faglg6s-emacs-typo-1.1: 1mj1h56pwf5cqbn397crlgsp7hdwjvrbqp4czha429y7pharjapl
/gnu/store/q6xnjg9fd7lfhh7rq2l98grlyq7nbcf0-emacs-butler-0.2.4 contents differ:
  local hash: 14lb6cdfngjjlxrnipq961hbgnhwp47ap904a9mm0dj4q7pj23n2
  http://hydra.gnunet.org/nar/q6xnjg9fd7lfhh7rq2l98grlyq7nbcf0-emacs-butler-0.2.4: 0vk41961frfvm5nrkmljhli3ibsz2lv0s09s6m29aiymixx6sw9g
--8<---------------cut here---------------end--------------->8---

If we diff as explained in the doc below, we see that the Git
discrepancies are due to timestamp in Perl’s POD files as well as
sorted-by-inode-number ‘tclIndex’ files.  For the Emacs modes, the
problem is the autogenerated autoloads files, which include some sort of
a timestamp as well.

You’re welcome to help fix these issues!

Comments welcome!

Ludo’.



6.12 Invoking ‘guix challenge’
==============================

Do the binaries provided by this server really correspond to the source
code it claims to build?  Is this package’s build process deterministic?
These are the questions the ‘guix challenge’ command attempts to answer.

   The former is obviously an important question: Before using a
substitute server (*note Substitutes::), you’d rather _verify_ that it
provides the right binaries, and thus _challenge_ it.  The latter is
what enables the former: If package builds are deterministic, then
independent builds of the package should yield the exact same result,
bit for bit; if a server provides a binary different from the one
obtained locally, it may be either corrupt or malicious.

   We know that the hash that shows up in ‘/gnu/store’ file names is the
hash of all the inputs of the process that built that file or
directory—compilers, libraries, build scripts, etc.  (*note
Introduction::).  Assuming deterministic build processes, one store file
name should map to exactly one build output.  ‘guix challenge’ checks
whether there is, indeed, a single mapping by comparing the build
outputs of several independent builds of any given store item.

   The command’s output looks like this:

     $ guix challenge --substitute-urls="http://hydra.gnu.org http://guix.example.org"
     updating list of substitutes from 'http://hydra.gnu.org'... 100.0%
     updating list of substitutes from 'http://guix.example.org'... 100.0%
     /gnu/store/…-openssl-1.0.2d contents differ:
       local hash: 0725l22r5jnzazaacncwsvp9kgf42266ayyp814v7djxs7nk963q
       http://hydra.gnu.org/nar/…-openssl-1.0.2d: 0725l22r5jnzazaacncwsvp9kgf42266ayyp814v7djxs7nk963q
       http://guix.example.org/nar/…-openssl-1.0.2d: 1zy4fmaaqcnjrzzajkdn3f5gmjk754b43qkq47llbyak9z0qjyim
     /gnu/store/…-git-2.5.0 contents differ:
       local hash: 00p3bmryhjxrhpn2gxs2fy0a15lnip05l97205pgbk5ra395hyha
       http://hydra.gnu.org/nar/…-git-2.5.0: 069nb85bv4d4a6slrwjdy8v1cn4cwspm3kdbmyb81d6zckj3nq9f
       http://guix.example.org/nar/…-git-2.5.0: 0mdqa9w1p6cmli6976v4wi0sw9r4p5prkj7lzfd1877wk11c9c73
     /gnu/store/…-pius-2.1.1 contents differ:
       local hash: 0k4v3m9z1zp8xzzizb7d8kjj72f9172xv078sq4wl73vnq9ig3ax
       http://hydra.gnu.org/nar/…-pius-2.1.1: 0k4v3m9z1zp8xzzizb7d8kjj72f9172xv078sq4wl73vnq9ig3ax
       http://guix.example.org/nar/…-pius-2.1.1: 1cy25x1a4fzq5rk0pmvc8xhwyffnqz95h2bpvqsz2mpvlbccy0gs

In this example, ‘guix challenge’ first scans the store to determine the
set of locally-built derivations—as opposed to store items that were
downloaded from a substitute server—and then queries all the substitute
servers.  It then reports those store items for which the servers
obtained a result different from the local build.

   As an example, ‘guix.example.org’ always gets a different answer.
Conversely, ‘hydra.gnu.org’ agrees with local builds, except in the case
of Git.  This might indicate that the build process of Git is
non-deterministic, meaning that its output varies as a function of
various things that Guix does not fully control, in spite of building
packages in isolated environments (*note Features::).  Most common
sources of non-determinism include the addition of timestamps in build
results, the inclusion of random numbers, and directory listings sorted
by inode number.  See <http://reproducible.debian.net/howto/>, for more
information.

   To find out what’s wrong with this Git binary, we can do something
along these lines (*note Invoking guix archive::):

     $ wget -q -O - http://hydra.gnu.org/nar/…-git-2.5.0 \
        | guix archive -x /tmp/git
     $ diff -ur /gnu/store/…-git.2.5.0 /tmp/git

   This command shows the difference between the files resulting from
the local build, and the files resulting from the build on
‘hydra.gnu.org’ (*note Comparing and Merging Files:
(diffutils)Overview.).  The ‘diff’ command works great for text files.
When binary files differ, a better option is Diffoscope
(http://diffoscope.org/), a tool that helps visualize differences for
all kinds of files.

   Once you’ve done that work, you can tell whether the differences are
due to a non-deterministic build process or to a malicious server.  We
try hard to remove sources of non-determinism in packages to make it
easier to verify substitutes, but of course, this is a process, one that
involves not just Guix but a large part of the free software community.
In the meantime, ‘guix challenge’ is one tool to help address the
problem.

   If you are writing packages for Guix, you are encouraged to check
whether ‘hydra.gnu.org’ and other substitute servers obtain the same
build result as you did with:

     $ guix challenge PACKAGE

...  where PACKAGE is a package specification such as ‘guile-2.0’ or
‘glibc:debug’.

   The general syntax is:

     guix challenge OPTIONS [PACKAGES…]

   The one option that matters is:

‘--substitute-urls=URLS’
     Consider URLS the whitespace-separated list of substitute source
     URLs to compare to.

Re: Challenge substitute servers!

[Daniel Pimentel]

Congratulation Guixs for it!

It's a good idea :)

Thanks,

-- 
Daniel Pimentel (d4n1 3:)

[PATCHES] emacs: Changes for 'guix challenge'.

[Alex Kost]

[-- Attachment #1: Type: text/plain, Size: 637 bytes --]

Ludovic Courtès (2015-10-20 02:09 +0300) wrote:

> Hello!
>
> I’m happy to announce the new ‘guix challenge’ command!  (Documentation
> below.)

I like new commands!  And this one is cool; I didn't think about
timestamp non-determinism before, thanks!

I'm attaching 2 trivial patches for emacs support of ‘guix challenge’.
The first patch is for completing packages in "M-x shell".  And the
second one is for adding a positional "Packages" argument in "M-x guix
RET challenge".  These are 2 different things, so I made 2 patches, or
should it be a single “emacs: Add support for 'guix challenge'” patch?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-emacs-Add-shell-completions-for-guix-challenge.patch --]
[-- Type: text/x-patch, Size: 945 bytes --]

From adaf6ab125d1bfd8e9416edd7d920e663c4c7785 Mon Sep 17 00:00:00 2001
From: Alex Kost <alezost@gmail.com>
Date: Tue, 20 Oct 2015 16:42:01 +0300
Subject: [PATCH 1/2] emacs: Add shell completions for 'guix challenge'.

* emacs/guix-pcomplete.el (guix-pcomplete-complete-command-arg): Add
  "challenge" to complete package names for it.
---
 emacs/guix-pcomplete.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/emacs/guix-pcomplete.el b/emacs/guix-pcomplete.el
index 4743be5..bd41837 100644
--- a/emacs/guix-pcomplete.el
+++ b/emacs/guix-pcomplete.el
@@ -209,7 +209,7 @@ group - the argument.")
   "Complete argument for guix COMMAND."
   (cond
    ((member command
-            '("archive" "build" "graph" "edit" "environment"
+            '("archive" "build" "challenge" "graph" "edit" "environment"
               "lint" "refresh" "size"))
     (while t
       (pcomplete-here (guix-pcomplete-all-packages))))
-- 
2.5.0


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0002-emacs-Add-Packages-option-for-guix-challenge-popup.patch --]
[-- Type: text/x-patch, Size: 986 bytes --]

From 41b2f36081f5c36b56156231a280e183dabd3243 Mon Sep 17 00:00:00 2001
From: Alex Kost <alezost@gmail.com>
Date: Tue, 20 Oct 2015 16:43:49 +0300
Subject: [PATCH 2/2] emacs: Add "Packages" option for 'guix challenge' popup.

* emacs/guix-command.el (guix-command-rest-argument): Add "challenge".
---
 emacs/guix-command.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/emacs/guix-command.el b/emacs/guix-command.el
index 1a42594..e9e46f4 100644
--- a/emacs/guix-command.el
+++ b/emacs/guix-command.el
@@ -364,7 +364,7 @@ to be modified."
                      :name "-- " :char ?= :option? t args)))
     (let ((command (car commands)))
       (cond
-       ((member command '("archive" "build" "graph" "edit"
+       ((member command '("archive" "build" "challenge" "graph" "edit"
                           "environment" "lint" "refresh"))
         (argument :doc "Packages" :fun 'guix-read-package-names-string))
        ((string= command "download")
-- 
2.5.0

Re: [PATCHES] emacs: Changes for 'guix challenge'.

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> I'm attaching 2 trivial patches for emacs support of ‘guix challenge’.
> The first patch is for completing packages in "M-x shell".  And the
> second one is for adding a positional "Packages" argument in "M-x guix
> RET challenge".

LGTM!

> These are 2 different things, so I made 2 patches, or should it be a
> single “emacs: Add support for 'guix challenge'” patch?

Either way is fine with me.

Thanks for the quick followup!  :-)

Ludo’.

Timestamps in ...-autoloads.el files

[Alex Kost]

[-- Attachment #1: Type: text/plain, Size: 580 bytes --]

Ludovic Courtès (2015-10-20 02:09 +0300) wrote:

> If we diff as explained in the doc below, we see that the Git
> discrepancies are due to timestamp in Perl’s POD files as well as
> sorted-by-inode-number ‘tclIndex’ files.  For the Emacs modes, the
> problem is the autogenerated autoloads files, which include some sort of
> a timestamp as well.
>
> You’re welcome to help fix these issues!

Thanks for the info!

Those timestamps are inserted by `autoload-insert-section-header' and
we can avoid them by advising this function, for example, like this:


[-- Attachment #2: autoload-advice.el --]
[-- Type: application/emacs-lisp, Size: 227 bytes --]

[-- Attachment #3: Type: text/plain, Size: 658 bytes --]


So after putting this code into ‘emacs-generate-autoloads’ procedure
from (guix build emacs-utils) module, there will be zeros instead of
non-deterministic timestamps.

However this will fix only those packages, that use
‘emacs-generate-autoloads’ directly or via ‘emacs-build-system’.  But
there are also packages that generate autoloads on their own (for
example, 'emacs-w3m' or 'guix' itself).  What to do for these ones?

Perhaps we can make a special 'emacs-build' package (that will advise
‘autoload-insert-section-header’ function somehow) and use it as an
input for emacs-packages, or are there other ways?

-- 
Alex

Re: Timestamps in ...-autoloads.el files

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2015-10-20 02:09 +0300) wrote:
>
>> If we diff as explained in the doc below, we see that the Git
>> discrepancies are due to timestamp in Perl’s POD files as well as
>> sorted-by-inode-number ‘tclIndex’ files.  For the Emacs modes, the
>> problem is the autogenerated autoloads files, which include some sort of
>> a timestamp as well.
>>
>> You’re welcome to help fix these issues!
>
> Thanks for the info!
>
> Those timestamps are inserted by `autoload-insert-section-header' and
> we can avoid them by advising this function, for example, like this:
>
> (defun guix-autoload-no-timestamp (fun outbuf autoloads load-name file time)
>   (funcall fun outbuf autoloads load-name file 0))
>
> (advice-add 'autoload-insert-section-header
>             :around 'guix-autoload-no-timestamp)
>
> So after putting this code into ‘emacs-generate-autoloads’ procedure
> from (guix build emacs-utils) module, there will be zeros instead of
> non-deterministic timestamps.

Great.

> However this will fix only those packages, that use
> ‘emacs-generate-autoloads’ directly or via ‘emacs-build-system’.  But
> there are also packages that generate autoloads on their own (for
> example, 'emacs-w3m' or 'guix' itself).  What to do for these ones?
>
> Perhaps we can make a special 'emacs-build' package (that will advise
> ‘autoload-insert-section-header’ function somehow) and use it as an
> input for emacs-packages, or are there other ways?

What about patching Emacs directly?

An upstreamable patch would be one that honors the ‘SOURCE_DATE_EPOCH’
variable¹.

Alternately, a patch that simply changes
‘autoload-insert-section-header’ to always use zero as the timestamp
would work as well, unless this would somehow break functionality.

Thoughts?

Thanks for looking into it,
Ludo’.

¹ https://reproducible-builds.org/specs/source-date-epoch/

Re: Timestamps in ...-autoloads.el files

[Alex Kost]

[-- Attachment #1: Type: text/plain, Size: 1269 bytes --]

Ludovic Courtès (2015-10-20 22:38 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
[...]
>> However this will fix only those packages, that use
>> ‘emacs-generate-autoloads’ directly or via ‘emacs-build-system’.  But
>> there are also packages that generate autoloads on their own (for
>> example, 'emacs-w3m' or 'guix' itself).  What to do for these ones?
>>
>> Perhaps we can make a special 'emacs-build' package (that will advise
>> ‘autoload-insert-section-header’ function somehow) and use it as an
>> input for emacs-packages, or are there other ways?
>
> What about patching Emacs directly?

Indeed!

> An upstreamable patch would be one that honors the ‘SOURCE_DATE_EPOCH’
> variable¹.
>
> Alternately, a patch that simply changes
> ‘autoload-insert-section-header’ to always use zero as the timestamp
> would work as well, unless this would somehow break functionality.
>
> Thoughts?

I like the idea to honor SOURCE_DATE_EPOCH, so I'm attaching a patch for
this.  But now I don't know how to make Guix set this variable during
the build process :-(  Need help.

> Thanks for looking into it,
> Ludo’.
>
> ¹ https://reproducible-builds.org/specs/source-date-epoch/

Thanks for the info!


[-- Attachment #2: 0001-gnu-emacs-Honor-SOURCE_DATE_EPOCH.patch --]
[-- Type: text/x-patch, Size: 3068 bytes --]

From b8dc19a65980690a636ad7f9f39b3c84991f4975 Mon Sep 17 00:00:00 2001
From: Alex Kost <alezost@gmail.com>
Date: Wed, 21 Oct 2015 15:59:23 +0300
Subject: [PATCH] gnu: emacs: Honor 'SOURCE_DATE_EPOCH'.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Suggested by Ludovic Courtès <ludo@gnu.org>.

* gnu/packages/patches/emacs-source-date-epoch.patch: New patch.
* gnu-system.am (dist_patch_DATA): Add it.
* gnu/packages/emacs.scm (emacs)[source]: Use it.
---
 gnu-system.am                                      |  1 +
 gnu/packages/emacs.scm                             |  3 ++-
 gnu/packages/patches/emacs-source-date-epoch.patch | 20 ++++++++++++++++++++
 3 files changed, 23 insertions(+), 1 deletion(-)
 create mode 100644 gnu/packages/patches/emacs-source-date-epoch.patch

diff --git a/gnu-system.am b/gnu-system.am
index e62fe18..4a20801 100644
--- a/gnu-system.am
+++ b/gnu-system.am
@@ -437,6 +437,7 @@ dist_patch_DATA =						\
   gnu/packages/patches/duplicity-test_selection-tmp.patch	\
   gnu/packages/patches/elfutils-tests-ptrace.patch		\
   gnu/packages/patches/emacs-exec-path.patch			\
+  gnu/packages/patches/emacs-source-date-epoch.patch		\
   gnu/packages/patches/eudev-rules-directory.patch		\
   gnu/packages/patches/expat-CVE-2015-1283.patch		\
   gnu/packages/patches/fastcap-mulGlobal.patch			\
diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 6416b00..9751125 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -70,7 +70,8 @@
              (sha256
               (base32
                "0kn3rzm91qiswi0cql89kbv6mqn27rwsyjfb8xmwy9m5s8fxfiyx"))
-             (patches (list (search-patch "emacs-exec-path.patch")))))
+             (patches (list (search-patch "emacs-exec-path.patch")
+                            (search-patch "emacs-source-date-epoch.patch")))))
     (build-system glib-or-gtk-build-system)
     (arguments
      '(#:phases (alist-cons-before
diff --git a/gnu/packages/patches/emacs-source-date-epoch.patch b/gnu/packages/patches/emacs-source-date-epoch.patch
new file mode 100644
index 0000000..41c03ef
--- /dev/null
+++ b/gnu/packages/patches/emacs-source-date-epoch.patch
@@ -0,0 +1,20 @@
+Honor SOURCE_DATE_EPOCH variable to avoid non-determinism in generated
+"autoloads" files.
+
+--- a/lisp/emacs-lisp/autoload.el
++++ b/lisp/emacs-lisp/autoload.el
+@@ -378,8 +378,12 @@
+   "Insert the section-header line,
+ which lists the file name and which functions are in it, etc."
+   (insert generate-autoload-section-header)
+-  (prin1 `(autoloads ,autoloads ,load-name ,file ,time)
+-	 outbuf)
++  (let* ((env  (getenv "SOURCE_DATE_EPOCH"))
++         (time (if env
++                   (seconds-to-time (string-to-number env))
++                 time)))
++    (prin1 `(autoloads ,autoloads ,load-name ,file ,time)
++           outbuf))
+   (terpri outbuf)
+   ;; Break that line at spaces, to avoid very long lines.
+   ;; Make each sub-line into a comment.
-- 
2.5.0

Re: Timestamps in ...-autoloads.el files

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> I like the idea to honor SOURCE_DATE_EPOCH, so I'm attaching a patch for
> this.  But now I don't know how to make Guix set this variable during
> the build process :-(  Need help.

Ahem, eventually, we’ll have to set it in ‘gnu-build’ in (guix build
gnu-build-system) in the next ‘core-updates’ cycle.

In the interim, we can set it in a phase of ‘emacs-build-system’, which
would entail few rebuilds.

WDYT?

> From b8dc19a65980690a636ad7f9f39b3c84991f4975 Mon Sep 17 00:00:00 2001
> From: Alex Kost <alezost@gmail.com>
> Date: Wed, 21 Oct 2015 15:59:23 +0300
> Subject: [PATCH] gnu: emacs: Honor 'SOURCE_DATE_EPOCH'.
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
>
> Suggested by Ludovic Courtès <ludo@gnu.org>.
>
> * gnu/packages/patches/emacs-source-date-epoch.patch: New patch.
> * gnu-system.am (dist_patch_DATA): Add it.
> * gnu/packages/emacs.scm (emacs)[source]: Use it.

LGTM.

> +++ b/gnu/packages/patches/emacs-source-date-epoch.patch
> @@ -0,0 +1,20 @@
> +Honor SOURCE_DATE_EPOCH variable to avoid non-determinism in generated
> +"autoloads" files.
> +
> +--- a/lisp/emacs-lisp/autoload.el
> ++++ b/lisp/emacs-lisp/autoload.el
> +@@ -378,8 +378,12 @@
> +   "Insert the section-header line,
> + which lists the file name and which functions are in it, etc."
> +   (insert generate-autoload-section-header)
> +-  (prin1 `(autoloads ,autoloads ,load-name ,file ,time)
> +-	 outbuf)
> ++  (let* ((env  (getenv "SOURCE_DATE_EPOCH"))
> ++         (time (if env
> ++                   (seconds-to-time (string-to-number env))
> ++                 time)))
> ++    (prin1 `(autoloads ,autoloads ,load-name ,file ,time)
> ++           outbuf))
> +   (terpri outbuf)
> +   ;; Break that line at spaces, to avoid very long lines.
> +   ;; Make each sub-line into a comment.

Could you also submit it upstream, Cc’ing guix-devel and
reproducible-builds@lists.alioth.debian.org?  Hopefully that is
acceptable.  (I searched a bit but didn’t find a similar patch by the
Debian Reproducible team, but patch-tracker.debian.org is unreachable.)

Thanks!

Ludo’.

Re: Timestamps in ...-autoloads.el files

[Alex Kost]

[-- Attachment #1: Type: text/plain, Size: 469 bytes --]

Ludovic Courtès (2015-10-21 19:55 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> I like the idea to honor SOURCE_DATE_EPOCH, so I'm attaching a patch for
>> this.  But now I don't know how to make Guix set this variable during
>> the build process :-(  Need help.
>
> Ahem, eventually, we’ll have to set it in ‘gnu-build’ in (guix build
> gnu-build-system) in the next ‘core-updates’ cycle.

So would it be as simple as this (?):


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: source-date-epoch.diff --]
[-- Type: text/x-patch, Size: 647 bytes --]

diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm
index ff7646b..92e15d1 100644
--- a/guix/build/gnu-build-system.scm
+++ b/guix/build/gnu-build-system.scm
@@ -576,6 +576,9 @@ in order.  Return #t if all the PHASES succeeded, #f otherwise."
   ;; Encoding/decoding errors shouldn't be silent.
   (fluid-set! %default-port-conversion-strategy 'error)
 
+  ;; Avoid non-determinism related to generated timestamps.
+  (setenv "SOURCE_DATE_EPOCH" "1")
+
   ;; The trick is to #:allow-other-keys everywhere, so that each procedure in
   ;; PHASES can pick the keyword arguments it's interested in.
   (every (match-lambda

[-- Attachment #3: Type: text/plain, Size: 1597 bytes --]


> In the interim, we can set it in a phase of ‘emacs-build-system’, which
> would entail few rebuilds.
>
> WDYT?

I think it is not so important to make a temporary workaround for
master.  What about just fixing it in core-updates?

>> +++ b/gnu/packages/patches/emacs-source-date-epoch.patch
>> @@ -0,0 +1,20 @@
>> +Honor SOURCE_DATE_EPOCH variable to avoid non-determinism in generated
>> +"autoloads" files.
>> +
>> +--- a/lisp/emacs-lisp/autoload.el
>> ++++ b/lisp/emacs-lisp/autoload.el
>> +@@ -378,8 +378,12 @@
>> +   "Insert the section-header line,
>> + which lists the file name and which functions are in it, etc."
>> +   (insert generate-autoload-section-header)
>> +-  (prin1 `(autoloads ,autoloads ,load-name ,file ,time)
>> +-	 outbuf)
>> ++  (let* ((env  (getenv "SOURCE_DATE_EPOCH"))
>> ++         (time (if env
>> ++                   (seconds-to-time (string-to-number env))
>> ++                 time)))
>> ++    (prin1 `(autoloads ,autoloads ,load-name ,file ,time)
>> ++           outbuf))
>> +   (terpri outbuf)
>> +   ;; Break that line at spaces, to avoid very long lines.
>> +   ;; Make each sub-line into a comment.
>
> Could you also submit it upstream, Cc’ing guix-devel and
> reproducible-builds@lists.alioth.debian.org?  Hopefully that is
> acceptable.  (I searched a bit but didn’t find a similar patch by the
> Debian Reproducible team, but patch-tracker.debian.org is unreachable.)

I'm afraid it's a too hard task for me, sorry.  I wouldn't like to mail
to so many places.

Sorry for the long delay.

-- 
Alex

Re: Timestamps in ...-autoloads.el files

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2015-10-21 19:55 +0300) wrote:
>
>> Alex Kost <alezost@gmail.com> skribis:
>>
>>> I like the idea to honor SOURCE_DATE_EPOCH, so I'm attaching a patch for
>>> this.  But now I don't know how to make Guix set this variable during
>>> the build process :-(  Need help.
>>
>> Ahem, eventually, we’ll have to set it in ‘gnu-build’ in (guix build
>> gnu-build-system) in the next ‘core-updates’ cycle.
>
> So would it be as simple as this (?):
>
> diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm
> index ff7646b..92e15d1 100644
> --- a/guix/build/gnu-build-system.scm
> +++ b/guix/build/gnu-build-system.scm
> @@ -576,6 +576,9 @@ in order.  Return #t if all the PHASES succeeded, #f otherwise."
>    ;; Encoding/decoding errors shouldn't be silent.
>    (fluid-set! %default-port-conversion-strategy 'error)
>  
> +  ;; Avoid non-determinism related to generated timestamps.
> +  (setenv "SOURCE_DATE_EPOCH" "1")
> +
>    ;; The trick is to #:allow-other-keys everywhere, so that each procedure in
>    ;; PHASES can pick the keyword arguments it's interested in.
>    (every (match-lambda

Yes, as simple as this.

>> In the interim, we can set it in a phase of ‘emacs-build-system’, which
>> would entail few rebuilds.
>>
>> WDYT?
>
> I think it is not so important to make a temporary workaround for
> master.  What about just fixing it in core-updates?

Sure, makes sense.  It’ll be several weeks before it is merged, though,
but we can probably live with that.

>>> +++ b/gnu/packages/patches/emacs-source-date-epoch.patch
>>> @@ -0,0 +1,20 @@
>>> +Honor SOURCE_DATE_EPOCH variable to avoid non-determinism in generated
>>> +"autoloads" files.
>>> +
>>> +--- a/lisp/emacs-lisp/autoload.el
>>> ++++ b/lisp/emacs-lisp/autoload.el
>>> +@@ -378,8 +378,12 @@
>>> +   "Insert the section-header line,
>>> + which lists the file name and which functions are in it, etc."
>>> +   (insert generate-autoload-section-header)
>>> +-  (prin1 `(autoloads ,autoloads ,load-name ,file ,time)
>>> +-	 outbuf)
>>> ++  (let* ((env  (getenv "SOURCE_DATE_EPOCH"))
>>> ++         (time (if env
>>> ++                   (seconds-to-time (string-to-number env))
>>> ++                 time)))
>>> ++    (prin1 `(autoloads ,autoloads ,load-name ,file ,time)
>>> ++           outbuf))
>>> +   (terpri outbuf)
>>> +   ;; Break that line at spaces, to avoid very long lines.
>>> +   ;; Make each sub-line into a comment.
>>
>> Could you also submit it upstream, Cc’ing guix-devel and
>> reproducible-builds@lists.alioth.debian.org?  Hopefully that is
>> acceptable.  (I searched a bit but didn’t find a similar patch by the
>> Debian Reproducible team, but patch-tracker.debian.org is unreachable.)
>
> I'm afraid it's a too hard task for me, sorry.  I wouldn't like to mail
> to so many places.

Or email only emacs-devel if you prefer.

An experienced Emacs hacker like you shouldn’t have to be afraid of that.

Thanks!

Ludo’.

Re: Timestamps in ...-autoloads.el files

[Alex Kost]

[-- Attachment #1: Type: text/plain, Size: 3516 bytes --]

Ludovic Courtès (2015-11-03 16:27 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> Ludovic Courtès (2015-10-21 19:55 +0300) wrote:
>>
>>> Alex Kost <alezost@gmail.com> skribis:
>>>
>>>> I like the idea to honor SOURCE_DATE_EPOCH, so I'm attaching a patch for
>>>> this.  But now I don't know how to make Guix set this variable during
>>>> the build process :-(  Need help.
>>>
>>> Ahem, eventually, we’ll have to set it in ‘gnu-build’ in (guix build
>>> gnu-build-system) in the next ‘core-updates’ cycle.
>>
>> So would it be as simple as this (?):
>>
>> diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm
>> index ff7646b..92e15d1 100644
>> --- a/guix/build/gnu-build-system.scm
>> +++ b/guix/build/gnu-build-system.scm
>> @@ -576,6 +576,9 @@ in order.  Return #t if all the PHASES succeeded, #f otherwise."
>>    ;; Encoding/decoding errors shouldn't be silent.
>>    (fluid-set! %default-port-conversion-strategy 'error)
>>  
>> +  ;; Avoid non-determinism related to generated timestamps.
>> +  (setenv "SOURCE_DATE_EPOCH" "1")
>> +
>>    ;; The trick is to #:allow-other-keys everywhere, so that each procedure in
>>    ;; PHASES can pick the keyword arguments it's interested in.
>>    (every (match-lambda
>
> Yes, as simple as this.

Great, thanks!  I see that you used (setenv "SOURCE_DATE_EPOCH" "0") in
'tk-update' commit, so I also changed the value to "0".

>>> In the interim, we can set it in a phase of ‘emacs-build-system’, which
>>> would entail few rebuilds.
>>>
>>> WDYT?
>>
>> I think it is not so important to make a temporary workaround for
>> master.  What about just fixing it in core-updates?
>
> Sure, makes sense.  It’ll be several weeks before it is merged, though,
> but we can probably live with that.

So is it OK to push the attached patches to 'core-updates'?

>>>> +++ b/gnu/packages/patches/emacs-source-date-epoch.patch
>>>> @@ -0,0 +1,20 @@
>>>> +Honor SOURCE_DATE_EPOCH variable to avoid non-determinism in generated
>>>> +"autoloads" files.
>>>> +
>>>> +--- a/lisp/emacs-lisp/autoload.el
>>>> ++++ b/lisp/emacs-lisp/autoload.el
>>>> +@@ -378,8 +378,12 @@
>>>> +   "Insert the section-header line,
>>>> + which lists the file name and which functions are in it, etc."
>>>> +   (insert generate-autoload-section-header)
>>>> +-  (prin1 `(autoloads ,autoloads ,load-name ,file ,time)
>>>> +-	 outbuf)
>>>> ++  (let* ((env  (getenv "SOURCE_DATE_EPOCH"))
>>>> ++         (time (if env
>>>> ++                   (seconds-to-time (string-to-number env))
>>>> ++                 time)))
>>>> ++    (prin1 `(autoloads ,autoloads ,load-name ,file ,time)
>>>> ++           outbuf))
>>>> +   (terpri outbuf)
>>>> +   ;; Break that line at spaces, to avoid very long lines.
>>>> +   ;; Make each sub-line into a comment.
>>>
>>> Could you also submit it upstream, Cc’ing guix-devel and
>>> reproducible-builds@lists.alioth.debian.org?  Hopefully that is
>>> acceptable.  (I searched a bit but didn’t find a similar patch by the
>>> Debian Reproducible team, but patch-tracker.debian.org is unreachable.)
>>
>> I'm afraid it's a too hard task for me, sorry.  I wouldn't like to mail
>> to so many places.
>
> Or email only emacs-devel if you prefer.
>
> An experienced Emacs hacker like you shouldn’t have to be afraid of that.

After fighting with myself for all these days, I must admit I'm not
brave enough for this task yet, sorry.


[-- Attachment #2: 0001-build-system-gnu-Set-SOURCE_DATE_EPOCH.patch --]
[-- Type: text/x-patch, Size: 1214 bytes --]

From a99fb41a7b84dd28c1a5b3f53cf14ca3c43785e7 Mon Sep 17 00:00:00 2001
From: Alex Kost <alezost@gmail.com>
Date: Sat, 14 Nov 2015 14:04:43 +0300
Subject: [PATCH 1/2] build-system/gnu: Set 'SOURCE_DATE_EPOCH'.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Suggested by Ludovic Courtès <ludo@gnu.org>.

* guix/build/gnu-build-system.scm (gnu-build): Set SOURCE_DATE_EPOCH for
  deterministic builds.
---
 guix/build/gnu-build-system.scm | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/guix/build/gnu-build-system.scm b/guix/build/gnu-build-system.scm
index ff7646b..011ccf8 100644
--- a/guix/build/gnu-build-system.scm
+++ b/guix/build/gnu-build-system.scm
@@ -576,6 +576,9 @@ in order.  Return #t if all the PHASES succeeded, #f otherwise."
   ;; Encoding/decoding errors shouldn't be silent.
   (fluid-set! %default-port-conversion-strategy 'error)
 
+  ;; Avoid non-determinism related to generated timestamps.
+  (setenv "SOURCE_DATE_EPOCH" "0")
+
   ;; The trick is to #:allow-other-keys everywhere, so that each procedure in
   ;; PHASES can pick the keyword arguments it's interested in.
   (every (match-lambda
-- 
2.5.0


[-- Attachment #3: 0002-gnu-emacs-Honor-SOURCE_DATE_EPOCH.patch --]
[-- Type: text/x-patch, Size: 3072 bytes --]

From 8ee244b6470b11928aa207cf21527ca33cef31f5 Mon Sep 17 00:00:00 2001
From: Alex Kost <alezost@gmail.com>
Date: Wed, 21 Oct 2015 15:59:23 +0300
Subject: [PATCH 2/2] gnu: emacs: Honor 'SOURCE_DATE_EPOCH'.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Suggested by Ludovic Courtès <ludo@gnu.org>.

* gnu/packages/patches/emacs-source-date-epoch.patch: New patch.
* gnu-system.am (dist_patch_DATA): Add it.
* gnu/packages/emacs.scm (emacs)[source]: Use it.
---
 gnu-system.am                                      |  1 +
 gnu/packages/emacs.scm                             |  3 ++-
 gnu/packages/patches/emacs-source-date-epoch.patch | 20 ++++++++++++++++++++
 3 files changed, 23 insertions(+), 1 deletion(-)
 create mode 100644 gnu/packages/patches/emacs-source-date-epoch.patch

diff --git a/gnu-system.am b/gnu-system.am
index d43358d..92370a2 100644
--- a/gnu-system.am
+++ b/gnu-system.am
@@ -437,6 +437,7 @@ dist_patch_DATA =						\
   gnu/packages/patches/duplicity-test_selection-tmp.patch	\
   gnu/packages/patches/elfutils-tests-ptrace.patch		\
   gnu/packages/patches/emacs-exec-path.patch			\
+  gnu/packages/patches/emacs-source-date-epoch.patch		\
   gnu/packages/patches/eudev-rules-directory.patch		\
   gnu/packages/patches/expat-CVE-2015-1283.patch		\
   gnu/packages/patches/fastcap-mulGlobal.patch			\
diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 6416b00..9751125 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -70,7 +70,8 @@
              (sha256
               (base32
                "0kn3rzm91qiswi0cql89kbv6mqn27rwsyjfb8xmwy9m5s8fxfiyx"))
-             (patches (list (search-patch "emacs-exec-path.patch")))))
+             (patches (list (search-patch "emacs-exec-path.patch")
+                            (search-patch "emacs-source-date-epoch.patch")))))
     (build-system glib-or-gtk-build-system)
     (arguments
      '(#:phases (alist-cons-before
diff --git a/gnu/packages/patches/emacs-source-date-epoch.patch b/gnu/packages/patches/emacs-source-date-epoch.patch
new file mode 100644
index 0000000..41c03ef
--- /dev/null
+++ b/gnu/packages/patches/emacs-source-date-epoch.patch
@@ -0,0 +1,20 @@
+Honor SOURCE_DATE_EPOCH variable to avoid non-determinism in generated
+"autoloads" files.
+
+--- a/lisp/emacs-lisp/autoload.el
++++ b/lisp/emacs-lisp/autoload.el
+@@ -378,8 +378,12 @@
+   "Insert the section-header line,
+ which lists the file name and which functions are in it, etc."
+   (insert generate-autoload-section-header)
+-  (prin1 `(autoloads ,autoloads ,load-name ,file ,time)
+-	 outbuf)
++  (let* ((env  (getenv "SOURCE_DATE_EPOCH"))
++         (time (if env
++                   (seconds-to-time (string-to-number env))
++                 time)))
++    (prin1 `(autoloads ,autoloads ,load-name ,file ,time)
++           outbuf))
+   (terpri outbuf)
+   ;; Break that line at spaces, to avoid very long lines.
+   ;; Make each sub-line into a comment.
-- 
2.5.0

Re: Timestamps in ...-autoloads.el files

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2015-11-03 16:27 +0300) wrote:

[...]

>>> +  ;; Avoid non-determinism related to generated timestamps.
>>> +  (setenv "SOURCE_DATE_EPOCH" "1")
>>> +
>>>    ;; The trick is to #:allow-other-keys everywhere, so that each procedure in
>>>    ;; PHASES can pick the keyword arguments it's interested in.
>>>    (every (match-lambda
>>
>> Yes, as simple as this.
>
> Great, thanks!  I see that you used (setenv "SOURCE_DATE_EPOCH" "0") in
> 'tk-update' commit, so I also changed the value to "0".

Oh, good point.  I don’t know if it matters much (for Python it seems to
make no difference), but I think “1” is the safest choice because the
mtime on files in the store is set to 1, not 0.  I’ll fix that in
‘tk-update’.

>>>> Could you also submit it upstream, Cc’ing guix-devel and
>>>> reproducible-builds@lists.alioth.debian.org?  Hopefully that is
>>>> acceptable.  (I searched a bit but didn’t find a similar patch by the
>>>> Debian Reproducible team, but patch-tracker.debian.org is unreachable.)
>>>
>>> I'm afraid it's a too hard task for me, sorry.  I wouldn't like to mail
>>> to so many places.
>>
>> Or email only emacs-devel if you prefer.
>>
>> An experienced Emacs hacker like you shouldn’t have to be afraid of that.
>
> After fighting with myself for all these days, I must admit I'm not
> brave enough for this task yet, sorry.

You’re not the one to be sorry.  It tells more about the weaknesses of
the Emacs developer community than about yours.

I’ll post it if you don’t mind.

> From a99fb41a7b84dd28c1a5b3f53cf14ca3c43785e7 Mon Sep 17 00:00:00 2001
> From: Alex Kost <alezost@gmail.com>
> Date: Sat, 14 Nov 2015 14:04:43 +0300
> Subject: [PATCH 1/2] build-system/gnu: Set 'SOURCE_DATE_EPOCH'.
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
>
> Suggested by Ludovic Courtès <ludo@gnu.org>.
>
> * guix/build/gnu-build-system.scm (gnu-build): Set SOURCE_DATE_EPOCH for
>   deterministic builds.

[...]

> +  ;; Avoid non-determinism related to generated timestamps.
> +  (setenv "SOURCE_DATE_EPOCH" "0")

OK with that set to 1.

> From 8ee244b6470b11928aa207cf21527ca33cef31f5 Mon Sep 17 00:00:00 2001
> From: Alex Kost <alezost@gmail.com>
> Date: Wed, 21 Oct 2015 15:59:23 +0300
> Subject: [PATCH 2/2] gnu: emacs: Honor 'SOURCE_DATE_EPOCH'.
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
>
> Suggested by Ludovic Courtès <ludo@gnu.org>.
>
> * gnu/packages/patches/emacs-source-date-epoch.patch: New patch.
> * gnu-system.am (dist_patch_DATA): Add it.
> * gnu/packages/emacs.scm (emacs)[source]: Use it.

OK.

Thank you!

Ludo’.

Re: Timestamps in ...-autoloads.el files

[Alex Kost]

Ludovic Courtès (2015-11-14 18:03 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> Ludovic Courtès (2015-11-03 16:27 +0300) wrote:
>
> [...]
>
>>>> +  ;; Avoid non-determinism related to generated timestamps.
>>>> +  (setenv "SOURCE_DATE_EPOCH" "1")
>>>> +
>>>>    ;; The trick is to #:allow-other-keys everywhere, so that each procedure in
>>>>    ;; PHASES can pick the keyword arguments it's interested in.
>>>>    (every (match-lambda
>>>
>>> Yes, as simple as this.
>>
>> Great, thanks!  I see that you used (setenv "SOURCE_DATE_EPOCH" "0") in
>> 'tk-update' commit, so I also changed the value to "0".
>
> Oh, good point.  I don’t know if it matters much (for Python it seems to
> make no difference), but I think “1” is the safest choice because the
> mtime on files in the store is set to 1, not 0.  I’ll fix that in
> ‘tk-update’.

I see, thanks for explaining.

>>>>> Could you also submit it upstream, Cc’ing guix-devel and
>>>>> reproducible-builds@lists.alioth.debian.org?  Hopefully that is
>>>>> acceptable.  (I searched a bit but didn’t find a similar patch by the
>>>>> Debian Reproducible team, but patch-tracker.debian.org is unreachable.)
>>>>
>>>> I'm afraid it's a too hard task for me, sorry.  I wouldn't like to mail
>>>> to so many places.
>>>
>>> Or email only emacs-devel if you prefer.
>>>
>>> An experienced Emacs hacker like you shouldn’t have to be afraid of that.
>>
>> After fighting with myself for all these days, I must admit I'm not
>> brave enough for this task yet, sorry.
>
> You’re not the one to be sorry.  It tells more about the weaknesses of
> the Emacs developer community than about yours.

Oh, no, no; it's totally me.  I have problems with communicating with
people.

> I’ll post it if you don’t mind.

Yes, please, it would be a great relief for me, thank you!

>> From a99fb41a7b84dd28c1a5b3f53cf14ca3c43785e7 Mon Sep 17 00:00:00 2001
>> From: Alex Kost <alezost@gmail.com>
>> Date: Sat, 14 Nov 2015 14:04:43 +0300
>> Subject: [PATCH 1/2] build-system/gnu: Set 'SOURCE_DATE_EPOCH'.
>> MIME-Version: 1.0
>> Content-Type: text/plain; charset=UTF-8
>> Content-Transfer-Encoding: 8bit
>>
>> Suggested by Ludovic Courtès <ludo@gnu.org>.
>>
>> * guix/build/gnu-build-system.scm (gnu-build): Set SOURCE_DATE_EPOCH for
>>   deterministic builds.
>
> [...]
>
>> +  ;; Avoid non-determinism related to generated timestamps.
>> +  (setenv "SOURCE_DATE_EPOCH" "0")
>
> OK with that set to 1.

Fixed and pushed, thanks.

-- 
Alex

Re: Timestamps in ...-autoloads.el files

[Alex Kost]

I've just noticed that our generated "…-autoloads.el" files still have
unwished timestamps (SOURCE_DATE_EPOCH is not honored).

To recap: we use "gnu/packages/patches/emacs-source-date-epoch.patch" to
modify 'autoload-insert-section-header' function, and it should work,
but it doesn't.  IIUC this happens because "autoload.elc" file was
compiled using the unpatched "autoload.el", but first things first.  Try
the following recipe in Emacs (installed with Guix):

1. M-: (setenv "SOURCE_DATE_EPOCH" "1")

2. Generate autoloads for any file with autoload cookies.  If you don't
   have such a file at hand, make "/tmp/f1.el" file with this single
   line:

   ;;;###autoload(defun f1 nil)

   Then: "M-x update-file-autoloads </tmp/f1.el> </tmp/auto1.el>"

Now open "/tmp/auto1.el" and you'll see a "bad" timestamp like this one:
(22323 17532 313464 85000).

But!  If you reevaluate 'autoload-insert-section-header' and try again
the timestamp will be "good":

3. M-x find-function autoload-insert-section-header

4. Reevaluate it: C-M-x

5. M-x update-file-autoloads </tmp/f1.el> </tmp/auto2.el>

Now look at "/tmp/auto2.el": it contains (0 1 0 0) timestamp as
expected.

I looked at the compiled "autoload.elc" file and if I understood it
correctly, it was compiled using the unpatched version of "autoload.el"
(because there is no mention of SOURCE_DATE_EPOCH there).  But I don't
understand how it could happen since patching is performed before
building.  Any ideas?

-- 
Alex

Re: Timestamps in ...-autoloads.el files

[Ludovic Courtès]

Hi!

Alex Kost <alezost@gmail.com> skribis:

> To recap: we use "gnu/packages/patches/emacs-source-date-epoch.patch" to
> modify 'autoload-insert-section-header' function, and it should work,
> but it doesn't.  IIUC this happens because "autoload.elc" file was
> compiled using the unpatched "autoload.el", but first things first.  Try
> the following recipe in Emacs (installed with Guix):
>
> 1. M-: (setenv "SOURCE_DATE_EPOCH" "1")
>
> 2. Generate autoloads for any file with autoload cookies.  If you don't
>    have such a file at hand, make "/tmp/f1.el" file with this single
>    line:
>
>    ;;;###autoload(defun f1 nil)
>
>    Then: "M-x update-file-autoloads </tmp/f1.el> </tmp/auto1.el>"
>
> Now open "/tmp/auto1.el" and you'll see a "bad" timestamp like this one:
> (22323 17532 313464 85000).
>
> But!  If you reevaluate 'autoload-insert-section-header' and try again
> the timestamp will be "good":
>
> 3. M-x find-function autoload-insert-section-header
>
> 4. Reevaluate it: C-M-x
>
> 5. M-x update-file-autoloads </tmp/f1.el> </tmp/auto2.el>
>
> Now look at "/tmp/auto2.el": it contains (0 1 0 0) timestamp as
> expected.

Weird!

> I looked at the compiled "autoload.elc" file and if I understood it
> correctly, it was compiled using the unpatched version of "autoload.el"
> (because there is no mention of SOURCE_DATE_EPOCH there).

Indeed.

> But I don't understand how it could happen since patching is performed
> before building.  Any ideas?

I think I have one:

--8<---------------cut here---------------start------------->8---
$ git describe 
v0.10.0-798-g8a7680a
$ tar tvf $(./pre-inst-env guix build -S emacs) |grep 'autoload\.el'
-rw-r--r-- root/root     37292 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el
-rw-r--r-- root/root     37127 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el.orig
-rw-r--r-- root/root     22624 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.elc
--8<---------------cut here---------------end--------------->8---

Upstream’s tarball already includes those three files.  Ideally, we
should remove all the .elc files and rebuild them, but maybe there are
bootstrapping issues.

Would you be willing to give it a try?  :-)

Thanks,
Ludo’.

Re: Timestamps in ...-autoloads.el files

[Alex Kost]

Ludovic Courtès (2016-05-16 15:45 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
[...]
>> I looked at the compiled "autoload.elc" file and if I understood it
>> correctly, it was compiled using the unpatched version of "autoload.el"
>> (because there is no mention of SOURCE_DATE_EPOCH there).
>
> Indeed.
>
>> But I don't understand how it could happen since patching is performed
>> before building.  Any ideas?
>
> I think I have one:
>
> $ git describe
> v0.10.0-798-g8a7680a
> $ tar tvf $(./pre-inst-env guix build -S emacs) |grep 'autoload\.el'
> -rw-r--r-- root/root     37292 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el
> -rw-r--r-- root/root     37127 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el.orig
> -rw-r--r-- root/root     22624 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.elc
>
> Upstream’s tarball already includes those three files.

IIUC this source is after applying our patches (including
"emacs-source-date-epoch.patch"):

- “autoload.el.orig” is the original file from the upstream;

- “autoload.el” is the patched one;

- “autoload.elc” is the compiled file also from the upstream (obviously
  it doesn't honor SOURCE_DATE_EPOCH).

Thanks, the mystery is solved now.  This is an unpleasant surprise, I
didn't know that emacs release comes with the compiled files.

> Ideally, we
> should remove all the .elc files and rebuild them, but maybe there are
> bootstrapping issues.
>
> Would you be willing to give it a try?  :-)

I'll see what I can do here.

-- 
Alex

Re: Timestamps in ...-autoloads.el files

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2016-05-16 15:45 +0300) wrote:
>
>> Alex Kost <alezost@gmail.com> skribis:
> [...]
>>> I looked at the compiled "autoload.elc" file and if I understood it
>>> correctly, it was compiled using the unpatched version of "autoload.el"
>>> (because there is no mention of SOURCE_DATE_EPOCH there).
>>
>> Indeed.
>>
>>> But I don't understand how it could happen since patching is performed
>>> before building.  Any ideas?
>>
>> I think I have one:
>>
>> $ git describe
>> v0.10.0-798-g8a7680a
>> $ tar tvf $(./pre-inst-env guix build -S emacs) |grep 'autoload\.el'
>> -rw-r--r-- root/root     37292 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el
>> -rw-r--r-- root/root     37127 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el.orig
>> -rw-r--r-- root/root     22624 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.elc
>>
>> Upstream’s tarball already includes those three files.
>
> IIUC this source is after applying our patches (including
> "emacs-source-date-epoch.patch"):
>
> - “autoload.el.orig” is the original file from the upstream;

Indeed, this one isn’t present in upstream’s tarball:

--8<---------------cut here---------------start------------->8---
$ wget -q -O - ftp://ftp.gnu.org/gnu/emacs/emacs-24.5.tar.xz | tar tJvf - | grep 'autoload\.'
-rw-rw-r-- nico/nico     37127 2015-04-02 09:23 emacs-24.5/lisp/emacs-lisp/autoload.el
-rw-r--r-- nico/nico     22624 2015-04-08 19:16 emacs-24.5/lisp/emacs-lisp/autoload.elc
--8<---------------cut here---------------end--------------->8---

How come we’re introducing this one?  I thought ‘patch’ did not produce
.orig files unless the patch failed to apply, but here the patch
correctly applies, only with a small offset (can be seen by running
‘guix build -S emacs --check’):

--8<---------------cut here---------------start------------->8---
patching file lisp/loadup.el
patching file lisp/emacs-lisp/autoload.el
Hunk #1 succeeded at 361 (offset -17 lines).
--8<---------------cut here---------------end--------------->8---

Apparently we have to use ‘--no-backup-if-mismatch’ to avoid that.

> Thanks, the mystery is solved now.  This is an unpleasant surprise, I
> didn't know that emacs release comes with the compiled files.

Yeah.  :-/

Ludo’.

Re: Timestamps in ...-autoloads.el files

[Alex Kost]

[-- Attachment #1: Type: text/plain, Size: 808 bytes --]

Ludovic Courtès (2016-05-16 15:45 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> To recap: we use "gnu/packages/patches/emacs-source-date-epoch.patch" to
>> modify 'autoload-insert-section-header' function, and it should work,
>> but it doesn't.  IIUC this happens because "autoload.elc" file was
>> compiled using the unpatched "autoload.el", but first things first.  Try
>> the following recipe in Emacs (installed with Guix):
[...]
> Ideally, we
> should remove all the .elc files and rebuild them, but maybe there are
> bootstrapping issues.

The attached patch does this.  Happily there were no any issues.  Now
emacs packages will honor SOURCE_DATE_EPOCH.  Even autoloads that comes
with emacs itself (loaddefs.el, cl-loaddefs.el, etc.) will have a proper
timestamp.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-emacs-Remove-.elc-from-the-release-tarball.patch --]
[-- Type: text/x-patch, Size: 1365 bytes --]

From ca571f7631bf77ddc8ad6257fe165b4ff0ef5e6b Mon Sep 17 00:00:00 2001
From: Alex Kost <alezost@gmail.com>
Date: Thu, 19 May 2016 11:01:40 +0300
Subject: [PATCH] gnu: emacs: Remove *.elc from the release tarball.

* gnu/packages/emacs.scm (emacs)[arguments]: Add 'remove-compiled-elisp'
  phase.
---
 gnu/packages/emacs.scm | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 32ed722..0c15f63 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -91,6 +91,15 @@
              (substitute* (find-files "." "^Makefile\\.in$")
                (("/bin/pwd")
                 "pwd"))))
+         (add-after 'configure 'remove-compiled-elisp
+           (lambda _
+             ;; Emacs comes with compiled elisp (*.elc) and generated
+             ;; autoloads (*loaddefs.el) files.  This does not allow us to use
+             ;; "emacs-source-date-epoch.patch" effectively, so remove these
+             ;; files (using 'make bootstrap-clean'), as 'make' will recreate
+             ;; them.
+             (with-directory-excursion "lisp"
+               (zero? (system* "make" "bootstrap-clean")))))
          (add-after 'install 'remove-info.info
            (lambda* (#:key outputs #:allow-other-keys)
              ;; Remove 'info.info', which is provided by Texinfo <= 6.0.
-- 
2.7.3

Re: Timestamps in ...-autoloads.el files

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2016-05-16 15:45 +0300) wrote:
>
>> Alex Kost <alezost@gmail.com> skribis:
>>
>>> To recap: we use "gnu/packages/patches/emacs-source-date-epoch.patch" to
>>> modify 'autoload-insert-section-header' function, and it should work,
>>> but it doesn't.  IIUC this happens because "autoload.elc" file was
>>> compiled using the unpatched "autoload.el", but first things first.  Try
>>> the following recipe in Emacs (installed with Guix):
> [...]
>> Ideally, we
>> should remove all the .elc files and rebuild them, but maybe there are
>> bootstrapping issues.
>
> The attached patch does this.  Happily there were no any issues.  Now
> emacs packages will honor SOURCE_DATE_EPOCH.  Even autoloads that comes
> with emacs itself (loaddefs.el, cl-loaddefs.el, etc.) will have a proper
> timestamp.

Awesome!

> From ca571f7631bf77ddc8ad6257fe165b4ff0ef5e6b Mon Sep 17 00:00:00 2001
> From: Alex Kost <alezost@gmail.com>
> Date: Thu, 19 May 2016 11:01:40 +0300
> Subject: [PATCH] gnu: emacs: Remove *.elc from the release tarball.
>
> * gnu/packages/emacs.scm (emacs)[arguments]: Add 'remove-compiled-elisp'
>   phase.
> ---
>  gnu/packages/emacs.scm | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
> index 32ed722..0c15f63 100644
> --- a/gnu/packages/emacs.scm
> +++ b/gnu/packages/emacs.scm
> @@ -91,6 +91,15 @@
>               (substitute* (find-files "." "^Makefile\\.in$")
>                 (("/bin/pwd")
>                  "pwd"))))
> +         (add-after 'configure 'remove-compiled-elisp
> +           (lambda _
> +             ;; Emacs comes with compiled elisp (*.elc) and generated
> +             ;; autoloads (*loaddefs.el) files.  This does not allow us to use
> +             ;; "emacs-source-date-epoch.patch" effectively, so remove these
> +             ;; files (using 'make bootstrap-clean'), as 'make' will recreate
> +             ;; them.
> +             (with-directory-excursion "lisp"
> +               (zero? (system* "make" "bootstrap-clean")))))

I would rather do it in a ‘snippet’ so that ‘guix build -S emacs’
returns the cleaned-up source.

However, the snippet would have to duplicate the logic of this makefile
rule, which might not be desirable (depends on how complex this rule
is).  If you think it’s best to keep this way, please push!

Thanks,
Ludo’.

Re: Timestamps in ...-autoloads.el files

[Alex Kost]

[-- Attachment #1: Type: text/plain, Size: 1815 bytes --]

Ludovic Courtès (2016-05-17 12:12 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> Ludovic Courtès (2016-05-16 15:45 +0300) wrote:
[...]
>>> $ git describe
>>> v0.10.0-798-g8a7680a
>>> $ tar tvf $(./pre-inst-env guix build -S emacs) |grep 'autoload\.el'
>>> -rw-r--r-- root/root     37292 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el
>>> -rw-r--r-- root/root     37127 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el.orig
>>> -rw-r--r-- root/root     22624 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.elc
>>>
>>> Upstream’s tarball already includes those three files.
>>
>> IIUC this source is after applying our patches (including
>> "emacs-source-date-epoch.patch"):
>>
>> - “autoload.el.orig” is the original file from the upstream;
>
> Indeed, this one isn’t present in upstream’s tarball:
>
> $ wget -q -O - ftp://ftp.gnu.org/gnu/emacs/emacs-24.5.tar.xz | tar tJvf - | grep 'autoload\.'
> -rw-rw-r-- nico/nico     37127 2015-04-02 09:23 emacs-24.5/lisp/emacs-lisp/autoload.el
> -rw-r--r-- nico/nico     22624 2015-04-08 19:16 emacs-24.5/lisp/emacs-lisp/autoload.elc
>
> How come we’re introducing this one?  I thought ‘patch’ did not produce
> .orig files unless the patch failed to apply, but here the patch
> correctly applies, only with a small offset (can be seen by running
> ‘guix build -S emacs --check’):
>
> patching file lisp/loadup.el
> patching file lisp/emacs-lisp/autoload.el
> Hunk #1 succeeded at 361 (offset -17 lines).

Indeed, I also didn't know that "patch" produces such .orig files when a
patch applies with offset.

> Apparently we have to use ‘--no-backup-if-mismatch’ to avoid that.

You even found the flag, thanks!  So is it OK to apply the attached
patch to core-updates?


[-- Attachment #2: 0001-packages-Use-no-backup-if-mismatch-for-patching.patch --]
[-- Type: text/x-patch, Size: 1503 bytes --]

From bcd636eff01f39583e8742b123d51550f9500795 Mon Sep 17 00:00:00 2001
From: Alex Kost <alezost@gmail.com>
Date: Thu, 19 May 2016 19:11:58 +0300
Subject: [PATCH] packages: Use '--no-backup-if-mismatch' for patching.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Suggested-by: Ludovic Courtès <ludo@gnu.org>

* guix/packages.scm (patch-and-repack)[build]: Use
  '--no-backup-if-mismatch' patch flag to avoid making *.orig files.
---
 guix/packages.scm | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/guix/packages.scm b/guix/packages.scm
index d62d1f3..a7b502c 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -464,9 +464,11 @@ IMPORTED-MODULES specify modules to use/import for use by SNIPPET."
             (format (current-error-port) "applying '~a'...~%" patch)
 
             ;; Use '--force' so that patches that do not apply perfectly are
-            ;; rejected.
+            ;; rejected.  Use '--no-backup-if-mismatch' to prevent making
+            ;; "*.orig" file if a patch is applied with offset.
             (zero? (system* (string-append #+patch "/bin/patch")
-                            "--force" #+@flags "--input" patch)))
+                            "--force" "--no-backup-if-mismatch"
+                            #+@flags "--input" patch)))
 
           (define (first-file directory)
             ;; Return the name of the first file in DIRECTORY.
-- 
2.7.3

Re: Timestamps in ...-autoloads.el files

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2016-05-17 12:12 +0300) wrote:
>
>> Alex Kost <alezost@gmail.com> skribis:
>>
>>> Ludovic Courtès (2016-05-16 15:45 +0300) wrote:
> [...]
>>>> $ git describe
>>>> v0.10.0-798-g8a7680a
>>>> $ tar tvf $(./pre-inst-env guix build -S emacs) |grep 'autoload\.el'
>>>> -rw-r--r-- root/root     37292 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el
>>>> -rw-r--r-- root/root     37127 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.el.orig
>>>> -rw-r--r-- root/root     22624 1970-01-01 01:00 emacs-24.5/lisp/emacs-lisp/autoload.elc
>>>>
>>>> Upstream’s tarball already includes those three files.
>>>
>>> IIUC this source is after applying our patches (including
>>> "emacs-source-date-epoch.patch"):
>>>
>>> - “autoload.el.orig” is the original file from the upstream;
>>
>> Indeed, this one isn’t present in upstream’s tarball:
>>
>> $ wget -q -O - ftp://ftp.gnu.org/gnu/emacs/emacs-24.5.tar.xz | tar tJvf - | grep 'autoload\.'
>> -rw-rw-r-- nico/nico     37127 2015-04-02 09:23 emacs-24.5/lisp/emacs-lisp/autoload.el
>> -rw-r--r-- nico/nico     22624 2015-04-08 19:16 emacs-24.5/lisp/emacs-lisp/autoload.elc
>>
>> How come we’re introducing this one?  I thought ‘patch’ did not produce
>> .orig files unless the patch failed to apply, but here the patch
>> correctly applies, only with a small offset (can be seen by running
>> ‘guix build -S emacs --check’):
>>
>> patching file lisp/loadup.el
>> patching file lisp/emacs-lisp/autoload.el
>> Hunk #1 succeeded at 361 (offset -17 lines).
>
> Indeed, I also didn't know that "patch" produces such .orig files when a
> patch applies with offset.
>
>> Apparently we have to use ‘--no-backup-if-mismatch’ to avoid that.
>
> You even found the flag, thanks!  So is it OK to apply the attached
> patch to core-updates?

The patch LGTM, but since this is a full-rebuild change, I prefer keep
it for the next core-updates cycle.  Let’s try not to forget about it.
:-)

Thanks,
Ludo’.

Re: Timestamps in ...-autoloads.el files

[Leo Famulari]

On Fri, May 20, 2016 at 02:02:41PM +0200, Ludovic Courtès wrote:
> The patch LGTM, but since this is a full-rebuild change, I prefer keep
> it for the next core-updates cycle.  Let’s try not to forget about it.

Should we create "core-updates-next" or something like that? I have a
few other patches I don't want to forget about either.

Re: Timestamps in ...-autoloads.el files

[Alex Kost]

Ludovic Courtès (2016-05-20 15:02 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> Ludovic Courtès (2016-05-17 12:12 +0300) wrote:
[...]
>>> Apparently we have to use ‘--no-backup-if-mismatch’ to avoid that.
>>
>> You even found the flag, thanks!  So is it OK to apply the attached
>> patch to core-updates?
>
> The patch LGTM, but since this is a full-rebuild change, I prefer keep
> it for the next core-updates cycle.  Let’s try not to forget about it.
> :-)

OK, I will not forget, thanks!

-- 
Alex

Re: Timestamps in ...-autoloads.el files

[Alex Kost]

[-- Attachment #1: Type: text/plain, Size: 1864 bytes --]

Ludovic Courtès (2016-05-19 15:56 +0300) wrote:

>> From ca571f7631bf77ddc8ad6257fe165b4ff0ef5e6b Mon Sep 17 00:00:00 2001
>> From: Alex Kost <alezost@gmail.com>
>> Date: Thu, 19 May 2016 11:01:40 +0300
>> Subject: [PATCH] gnu: emacs: Remove *.elc from the release tarball.
>>
>> * gnu/packages/emacs.scm (emacs)[arguments]: Add 'remove-compiled-elisp'
>>   phase.
>> ---
>>  gnu/packages/emacs.scm | 9 +++++++++
>>  1 file changed, 9 insertions(+)
>>
>> diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
>> index 32ed722..0c15f63 100644
>> --- a/gnu/packages/emacs.scm
>> +++ b/gnu/packages/emacs.scm
>> @@ -91,6 +91,15 @@
>>               (substitute* (find-files "." "^Makefile\\.in$")
>>                 (("/bin/pwd")
>>                  "pwd"))))
>> +         (add-after 'configure 'remove-compiled-elisp
>> +           (lambda _
>> +             ;; Emacs comes with compiled elisp (*.elc) and generated
>> +             ;; autoloads (*loaddefs.el) files.  This does not allow us to use
>> +             ;; "emacs-source-date-epoch.patch" effectively, so remove these
>> +             ;; files (using 'make bootstrap-clean'), as 'make' will recreate
>> +             ;; them.
>> +             (with-directory-excursion "lisp"
>> +               (zero? (system* "make" "bootstrap-clean")))))
>
> I would rather do it in a ‘snippet’ so that ‘guix build -S emacs’
> returns the cleaned-up source.

Hm, I didn't think about it; yeah, I agree it would be better, thanks!

> However, the snippet would have to duplicate the logic of this makefile
> rule, which might not be desirable (depends on how complex this rule
> is).  If you think it’s best to keep this way, please push!

The rule is rather simple, it just removes all compiled and generated
files, so attached is the version with snippet.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnu-emacs-Remove-.elc-and-autoloads-from-the-tarball.patch --]
[-- Type: text/x-patch, Size: 1602 bytes --]

From 13c2e7123d3b8f7dda5814c2ac00acfe806cec3b Mon Sep 17 00:00:00 2001
From: Alex Kost <alezost@gmail.com>
Date: Thu, 19 May 2016 11:01:40 +0300
Subject: [PATCH] gnu: emacs: Remove *.elc and autoloads from the tarball.

* gnu/packages/emacs.scm (emacs)[source]: Add 'snippet' to remove
  compiled and generated elisp files.
---
 gnu/packages/emacs.scm | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 18898e9..ae02a07 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -80,7 +80,18 @@
               (base32
                "0kn3rzm91qiswi0cql89kbv6mqn27rwsyjfb8xmwy9m5s8fxfiyx"))
              (patches (search-patches "emacs-exec-path.patch"
-                                      "emacs-source-date-epoch.patch"))))
+                                      "emacs-source-date-epoch.patch"))
+             (modules '((guix build utils)))
+             (snippet
+              ;; Delete the bundled byte-compiled elisp files and
+              ;; generated autoloads.
+              '(with-directory-excursion "lisp"
+                 (for-each delete-file
+                           (append (find-files "." "\\.elc$")
+                                   (find-files "." "loaddefs\\.el$")
+                                   ;; This is the only "autoloads" file that
+                                   ;; does not have "*loaddefs.el" name.
+                                   '("eshell/esh-groups.el")))))))
     (build-system glib-or-gtk-build-system)
     (arguments
      `(#:phases
-- 
2.8.2

Re: Timestamps in ...-autoloads.el files

[Alex Kost]

Leo Famulari (2016-05-20 16:38 +0300) wrote:

> On Fri, May 20, 2016 at 02:02:41PM +0200, Ludovic Courtès wrote:
>> The patch LGTM, but since this is a full-rebuild change, I prefer keep
>> it for the next core-updates cycle.  Let’s try not to forget about it.
>
> Should we create "core-updates-next" or something like that? I have a
> few other patches I don't want to forget about either.

Good idea!  I agree.

-- 
Alex

Re: Timestamps in ...-autoloads.el files

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> From 13c2e7123d3b8f7dda5814c2ac00acfe806cec3b Mon Sep 17 00:00:00 2001
> From: Alex Kost <alezost@gmail.com>
> Date: Thu, 19 May 2016 11:01:40 +0300
> Subject: [PATCH] gnu: emacs: Remove *.elc and autoloads from the tarball.
>
> * gnu/packages/emacs.scm (emacs)[source]: Add 'snippet' to remove
>   compiled and generated elisp files.

Perfect, thanks!

Ludo’.

Re: Timestamps in ...-autoloads.el files

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Leo Famulari (2016-05-20 16:38 +0300) wrote:
>
>> On Fri, May 20, 2016 at 02:02:41PM +0200, Ludovic Courtès wrote:
>>> The patch LGTM, but since this is a full-rebuild change, I prefer keep
>>> it for the next core-updates cycle.  Let’s try not to forget about it.
>>
>> Should we create "core-updates-next" or something like that? I have a
>> few other patches I don't want to forget about either.
>
> Good idea!  I agree.

OK, go ahead then.  :-)  We’ll rename it to ‘core-updates’ when the
current one is deleted.

Ludo’.

Re: Timestamps in ...-autoloads.el files

[Alex Kost]

Ludovic Courtès (2016-05-22 00:01 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> From 13c2e7123d3b8f7dda5814c2ac00acfe806cec3b Mon Sep 17 00:00:00 2001
>> From: Alex Kost <alezost@gmail.com>
>> Date: Thu, 19 May 2016 11:01:40 +0300
>> Subject: [PATCH] gnu: emacs: Remove *.elc and autoloads from the tarball.
>>
>> * gnu/packages/emacs.scm (emacs)[source]: Add 'snippet' to remove
>>   compiled and generated elisp files.
>
> Perfect, thanks!

Committed.

-- 
Alex

Re: Timestamps in ...-autoloads.el files

[Alex Kost]

Ludovic Courtès (2016-05-22 00:02 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> Leo Famulari (2016-05-20 16:38 +0300) wrote:
>>
>>> On Fri, May 20, 2016 at 02:02:41PM +0200, Ludovic Courtès wrote:
>>>> The patch LGTM, but since this is a full-rebuild change, I prefer keep
>>>> it for the next core-updates cycle.  Let’s try not to forget about it.
>>>
>>> Should we create "core-updates-next" or something like that? I have a
>>> few other patches I don't want to forget about either.
>>
>> Good idea!  I agree.
>
> OK, go ahead then.  :-)  We’ll rename it to ‘core-updates’ when the
> current one is deleted.

I went ahead.  Leo, now it's you turn to commit your patches to
"core-updates-next".

I based it on master, not sure if it should be based on "core-updates",
but anyway, it's done already :-)

-- 
Alex