* [PATCH] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. @ 2015-11-16 17:03 Ludovic Courtès 2015-11-29 10:44 ` [PATCH PING] " Ludovic Courtès 0 siblings, 1 reply; 53+ messages in thread From: Ludovic Courtès @ 2015-11-16 17:03 UTC (permalink / raw) To: emacs-devel [-- Attachment #1: Type: text/plain, Size: 353 bytes --] Support bit-for-bit reproducible generation of autoloads. See <https://reproducible-builds.org/specs/source-date-epoch/>. Submitted on behalf of Alex Kost. * lisp/emacs-lisp/autoload.el (autoload-insert-section-header): Check whether the 'SOURCE_DATE_EPOCH' environment variable is defined; use it as the TIME part of the 'autoloads' sexp when it is. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: Type: text/x-patch, Size: 658 bytes --] --- a/lisp/emacs-lisp/autoload.el +++ b/lisp/emacs-lisp/autoload.el @@ -378,8 +378,12 @@ "Insert the section-header line, which lists the file name and which functions are in it, etc." (insert generate-autoload-section-header) - (prin1 `(autoloads ,autoloads ,load-name ,file ,time) - outbuf) + (let* ((env (getenv "SOURCE_DATE_EPOCH")) + (time (if env + (seconds-to-time (string-to-number env)) + time))) + (prin1 `(autoloads ,autoloads ,load-name ,file ,time) + outbuf)) (terpri outbuf) ;; Break that line at spaces, to avoid very long lines. ;; Make each sub-line into a comment. ^ permalink raw reply [flat|nested] 53+ messages in thread
* [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-11-16 17:03 [PATCH] Honor 'SOURCE_DATE_EPOCH' when generating autoloads Ludovic Courtès @ 2015-11-29 10:44 ` Ludovic Courtès 2015-11-29 16:02 ` Eli Zaretskii 2015-11-29 20:22 ` Paul Eggert 0 siblings, 2 replies; 53+ messages in thread From: Ludovic Courtès @ 2015-11-29 10:44 UTC (permalink / raw) To: emacs-devel [-- Attachment #1: Type: text/plain, Size: 432 bytes --] Ping! https://lists.gnu.org/archive/html/emacs-devel/2015-11/msg01547.html Support bit-for-bit reproducible generation of autoloads. See <https://reproducible-builds.org/specs/source-date-epoch/>. Submitted on behalf of Alex Kost. * lisp/emacs-lisp/autoload.el (autoload-insert-section-header): Check whether the 'SOURCE_DATE_EPOCH' environment variable is defined; use it as the TIME part of the 'autoloads' sexp when it is. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: Type: text/x-patch, Size: 658 bytes --] --- a/lisp/emacs-lisp/autoload.el +++ b/lisp/emacs-lisp/autoload.el @@ -378,8 +378,12 @@ "Insert the section-header line, which lists the file name and which functions are in it, etc." (insert generate-autoload-section-header) - (prin1 `(autoloads ,autoloads ,load-name ,file ,time) - outbuf) + (let* ((env (getenv "SOURCE_DATE_EPOCH")) + (time (if env + (seconds-to-time (string-to-number env)) + time))) + (prin1 `(autoloads ,autoloads ,load-name ,file ,time) + outbuf)) (terpri outbuf) ;; Break that line at spaces, to avoid very long lines. ;; Make each sub-line into a comment. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-11-29 10:44 ` [PATCH PING] " Ludovic Courtès @ 2015-11-29 16:02 ` Eli Zaretskii 2015-11-29 16:57 ` Ludovic Courtès 2015-11-29 20:22 ` Paul Eggert 1 sibling, 1 reply; 53+ messages in thread From: Eli Zaretskii @ 2015-11-29 16:02 UTC (permalink / raw) To: Ludovic Courtès; +Cc: emacs-devel > From: ludo@gnu.org (Ludovic Courtès) > Date: Sun, 29 Nov 2015 11:44:36 +0100 > > https://lists.gnu.org/archive/html/emacs-devel/2015-11/msg01547.html > > Support bit-for-bit reproducible generation of autoloads. > See <https://reproducible-builds.org/specs/source-date-epoch/>. > Submitted on behalf of Alex Kost. > > * lisp/emacs-lisp/autoload.el (autoload-insert-section-header): Check > whether the 'SOURCE_DATE_EPOCH' environment variable is defined; use it > as the TIME part of the 'autoloads' sexp when it is. Thanks, but I think we need to discuss this first. First, if this is Alex's code, please show his full name and email address, as they will have to be mentioned in the commit message, and we need to know the status of his legal paperwork, or the lack thereof. Second, I don't see how doing this in only loaddefs.el will help make an Emacs build "reproducible": each of the *.elc files includes the version number of the Emacs binary, which is not necessarily constant across different builds. (I think I also saw some other stuff in *.elc files that can be different between otherwise identical builds.) Then there's the name of the build system and the repository version string that get hard-coded into the binary. There could be other sources of differences. (I'm not mentioning the C compiler, assembler, and linker, as I see similar requests posted to the respective mailing lists.) Finally, this will have to be documented in the ELisp manual, probably in the Internals appendix. Thanks. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-11-29 16:02 ` Eli Zaretskii @ 2015-11-29 16:57 ` Ludovic Courtès 2015-11-29 18:12 ` Eli Zaretskii 2015-11-30 9:22 ` Alex Kost 0 siblings, 2 replies; 53+ messages in thread From: Ludovic Courtès @ 2015-11-29 16:57 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Alex Kost, emacs-devel Eli Zaretskii <eliz@gnu.org> skribis: >> From: ludo@gnu.org (Ludovic Courtès) >> Date: Sun, 29 Nov 2015 11:44:36 +0100 >> >> https://lists.gnu.org/archive/html/emacs-devel/2015-11/msg01547.html >> >> Support bit-for-bit reproducible generation of autoloads. >> See <https://reproducible-builds.org/specs/source-date-epoch/>. >> Submitted on behalf of Alex Kost. >> >> * lisp/emacs-lisp/autoload.el (autoload-insert-section-header): Check >> whether the 'SOURCE_DATE_EPOCH' environment variable is defined; use it >> as the TIME part of the 'autoloads' sexp when it is. > > Thanks, but I think we need to discuss this first. > > First, if this is Alex's code, please show his full name and email > address, as they will have to be mentioned in the commit message, and > we need to know the status of his legal paperwork, or the lack > thereof. First, Alex (Cc’d) asked me explicitly to do it on his behalf because he didn’t want to have to deal with this (IMO unfriendly) feedback personally. I see we’re failing here. https://lists.gnu.org/archive/html/guix-devel/2015-11/msg00378.html Second, I would argue that this is not legally significant. In fact, there’s only one way to do it. > Second, I don't see how doing this in only loaddefs.el will help make > an Emacs build "reproducible": This is obviously the first patch of a series. It does not make Emacs itself bit-reproducible, but it makes Emacs packages bit-reproducible. See <https://lists.gnu.org/archive/html/guix-devel/2015-10/msg00728.html> for more info. Thanks, Ludo’. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-11-29 16:57 ` Ludovic Courtès @ 2015-11-29 18:12 ` Eli Zaretskii 2015-11-29 20:38 ` Ludovic Courtès 2015-11-30 17:00 ` John Wiegley 2015-11-30 9:22 ` Alex Kost 1 sibling, 2 replies; 53+ messages in thread From: Eli Zaretskii @ 2015-11-29 18:12 UTC (permalink / raw) To: Ludovic Courtès, John Wiegley; +Cc: alezost, emacs-devel > From: ludo@gnu.org (Ludovic Courtès) > Cc: emacs-devel@gnu.org, Alex Kost <alezost@gmail.com> > Date: Sun, 29 Nov 2015 17:57:32 +0100 > > Eli Zaretskii <eliz@gnu.org> skribis: > > > First, if this is Alex's code, please show his full name and email > > address, as they will have to be mentioned in the commit message, and > > we need to know the status of his legal paperwork, or the lack > > thereof. > > First, Alex (Cc’d) asked me explicitly to do it on his behalf because he > didn’t want to have to deal with this (IMO unfriendly) feedback > personally. I see we’re failing here. There's some unfortunate mis-communication here, because my feedback wasn't meant to be unfriendly in any way. Apologies if my wording failed me. English is not the first language for either of us. > Second, I would argue that this is not legally significant. I didn't say it was. But we mark small enough contributions we accept from people without legal paperwork specially in the commit log message, so we need to know the status just to decide what to put there. As it turns out, Alex already has an assignment on file for contributing to Emacs. But I couldn't verify that without knowing his full name or email address. > > Second, I don't see how doing this in only loaddefs.el will help make > > an Emacs build "reproducible": > > This is obviously the first patch of a series. It does not make Emacs > itself bit-reproducible, but it makes Emacs packages bit-reproducible. > > See <https://lists.gnu.org/archive/html/guix-devel/2015-10/msg00728.html> > for more info. I'd prefer to have the whole series, to see where this is going, and then commit them in one go. But that's me. John? ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-11-29 18:12 ` Eli Zaretskii @ 2015-11-29 20:38 ` Ludovic Courtès 2015-11-30 17:00 ` John Wiegley 1 sibling, 0 replies; 53+ messages in thread From: Ludovic Courtès @ 2015-11-29 20:38 UTC (permalink / raw) To: Eli Zaretskii; +Cc: John Wiegley, alezost, emacs-devel Eli Zaretskii <eliz@gnu.org> skribis: >> > Second, I don't see how doing this in only loaddefs.el will help make >> > an Emacs build "reproducible": >> >> This is obviously the first patch of a series. It does not make Emacs >> itself bit-reproducible, but it makes Emacs packages bit-reproducible. >> >> See <https://lists.gnu.org/archive/html/guix-devel/2015-10/msg00728.html> >> for more info. > > I'd prefer to have the whole series, to see where this is going, and > then commit them in one go. But that's me. John? I should point out that there’s currently no other patch in the series because we haven’t gotten there yet. ;-) We were looking at making autoload generation deterministic as a first step. For us this fixes non-determinism issues in ~37 packages. Ludo’. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-11-29 18:12 ` Eli Zaretskii 2015-11-29 20:38 ` Ludovic Courtès @ 2015-11-30 17:00 ` John Wiegley 2015-11-30 19:30 ` Ludovic Courtès 1 sibling, 1 reply; 53+ messages in thread From: John Wiegley @ 2015-11-30 17:00 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Ludovic Courtès, alezost, emacs-devel >>>>> Eli Zaretskii <eliz@gnu.org> writes: >> This is obviously the first patch of a series. It does not make Emacs >> itself bit-reproducible, but it makes Emacs packages bit-reproducible. > I'd prefer to have the whole series, to see where this is going, and then > commit them in one go. But that's me. John? Yes, I too would like to see this evolve on a feature branch until the whole series is ready, after which Eli and I can review before moving into master. How does that sound, Alex, Ludovic? John ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-11-30 17:00 ` John Wiegley @ 2015-11-30 19:30 ` Ludovic Courtès 2015-12-07 18:36 ` Philipp Stephani 0 siblings, 1 reply; 53+ messages in thread From: Ludovic Courtès @ 2015-11-30 19:30 UTC (permalink / raw) To: Eli Zaretskii; +Cc: alezost, emacs-devel John Wiegley <jwiegley@gmail.com> skribis: >>>>>> Eli Zaretskii <eliz@gnu.org> writes: > >>> This is obviously the first patch of a series. It does not make Emacs >>> itself bit-reproducible, but it makes Emacs packages bit-reproducible. > >> I'd prefer to have the whole series, to see where this is going, and then >> commit them in one go. But that's me. John? > > Yes, I too would like to see this evolve on a feature branch until the whole > series is ready, after which Eli and I can review before moving into master. > > How does that sound, Alex, Ludovic? From the Guix viewpoint, we need an immediate fix for this issue. So we’ll be using the patch I posted until Emacs has an appropriate solution, which may be along the lines of what Paul suggested. We’ll surely look at making Emacs itself reproducible (and perhaps the Debian folks already have patches floating around), but no ETA. Thanks, Ludo’. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-11-30 19:30 ` Ludovic Courtès @ 2015-12-07 18:36 ` Philipp Stephani 2015-12-07 18:58 ` Paul Eggert 2015-12-07 23:11 ` Ludovic Courtès 0 siblings, 2 replies; 53+ messages in thread From: Philipp Stephani @ 2015-12-07 18:36 UTC (permalink / raw) To: Ludovic Courtès, Eli Zaretskii; +Cc: alezost, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1644 bytes --] Ludovic Courtès <ludo@gnu.org> schrieb am Mo., 30. Nov. 2015 um 20:30 Uhr: > John Wiegley <jwiegley@gmail.com> skribis: > > >>>>>> Eli Zaretskii <eliz@gnu.org> writes: > > > >>> This is obviously the first patch of a series. It does not make Emacs > >>> itself bit-reproducible, but it makes Emacs packages bit-reproducible. > > > >> I'd prefer to have the whole series, to see where this is going, and > then > >> commit them in one go. But that's me. John? > > > > Yes, I too would like to see this evolve on a feature branch until the > whole > > series is ready, after which Eli and I can review before moving into > master. > > > > How does that sound, Alex, Ludovic? > > From the Guix viewpoint, we need an immediate fix for this issue. So > we’ll be using the patch I posted until Emacs has an appropriate > solution, which may be along the lines of what Paul suggested. > > We’ll surely look at making Emacs itself reproducible (and perhaps the > Debian folks already have patches floating around), but no ETA. > I'm interested in reproducible builds as well; do you already have some design ideas how to tackle this problem? So far I've found three areas where Emacs itself builds something that could be reproducible (dumping, generating autoload files, byte compilation), and a couple of data items that cause irreproducibility (host names, timestamps, absolute paths). As a rough sketch, Emacs could be made to detect an environment variable such as EMACS_DETERMINISTIC_BUILD, causing various code paths in the core and in the Elisp files to become deterministic. Would that be feasible? [-- Attachment #2: Type: text/html, Size: 2137 bytes --] ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-07 18:36 ` Philipp Stephani @ 2015-12-07 18:58 ` Paul Eggert 2015-12-07 19:00 ` Philipp Stephani 2015-12-07 23:11 ` Ludovic Courtès 1 sibling, 1 reply; 53+ messages in thread From: Paul Eggert @ 2015-12-07 18:58 UTC (permalink / raw) To: Philipp Stephani, Ludovic Courtès, Eli Zaretskii Cc: alezost, emacs-devel On 12/07/2015 10:36 AM, Philipp Stephani wrote: > As a rough sketch, Emacs could be made to detect an environment > variable such as EMACS_DETERMINISTIC_BUILD, causing various code paths > in the core and in the Elisp files to become deterministic. Would that > be feasible? Let's just fix Emacs so that its builds are deterministic. Long ago perhaps there was a good reason to put cruft like hostnames into build products, but those days are gone. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-07 18:58 ` Paul Eggert @ 2015-12-07 19:00 ` Philipp Stephani 2015-12-07 19:14 ` Paul Eggert 0 siblings, 1 reply; 53+ messages in thread From: Philipp Stephani @ 2015-12-07 19:00 UTC (permalink / raw) To: Paul Eggert, Ludovic Courtès, Eli Zaretskii; +Cc: alezost, emacs-devel [-- Attachment #1: Type: text/plain, Size: 682 bytes --] Paul Eggert <eggert@cs.ucla.edu> schrieb am Mo., 7. Dez. 2015 um 19:58 Uhr: > On 12/07/2015 10:36 AM, Philipp Stephani wrote: > > As a rough sketch, Emacs could be made to detect an environment > > variable such as EMACS_DETERMINISTIC_BUILD, causing various code paths > > in the core and in the Elisp files to become deterministic. Would that > > be feasible? > > Let's just fix Emacs so that its builds are deterministic. Long ago > perhaps there was a good reason to put cruft like hostnames into build > products, but those days are gone. > I don't disagree, but such things are shown very prominently (e.g. in the C-h C-a about screen), so this would be a rather big change. [-- Attachment #2: Type: text/html, Size: 999 bytes --] ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-07 19:00 ` Philipp Stephani @ 2015-12-07 19:14 ` Paul Eggert 2015-12-20 12:48 ` Philipp Stephani 0 siblings, 1 reply; 53+ messages in thread From: Paul Eggert @ 2015-12-07 19:14 UTC (permalink / raw) To: Philipp Stephani; +Cc: emacs-devel On 12/07/2015 11:00 AM, Philipp Stephani wrote: > this would be a rather big change. I suppose that depends on what one means by "big". To me it feels like a fairly small improvement. We've already made some progress along these lines in emacs-25 by removing emacs-build-system from the result of the emacs-version function; we just need to make more improvements like that. It should be in 'master', though, not in 'emacs-25', as it's not fixing a bug. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-07 19:14 ` Paul Eggert @ 2015-12-20 12:48 ` Philipp Stephani 2015-12-20 16:37 ` Eli Zaretskii 0 siblings, 1 reply; 53+ messages in thread From: Philipp Stephani @ 2015-12-20 12:48 UTC (permalink / raw) To: Paul Eggert; +Cc: emacs-devel [-- Attachment #1.1: Type: text/plain, Size: 684 bytes --] Paul Eggert <eggert@cs.ucla.edu> schrieb am Mo., 7. Dez. 2015 um 20:14 Uhr: > On 12/07/2015 11:00 AM, Philipp Stephani wrote: > > this would be a rather big change. > > I suppose that depends on what one means by "big". To me it feels like a > fairly small improvement. We've already made some progress along these > lines in emacs-25 by removing emacs-build-system from the result of the > emacs-version function; we just need to make more improvements like that. > > It should be in 'master', though, not in 'emacs-25', as it's not fixing > a bug. > OK, here's a first patch that replaces system-name with a constant during dumping. That eliminates one source of nondeterminism. [-- Attachment #1.2: Type: text/html, Size: 1029 bytes --] [-- Attachment #2: 0001-Remove-build-system-name-from-dump.patch --] [-- Type: application/octet-stream, Size: 2013 bytes --] From 7f9df3f7f311f82efaec2330b0c46094d1a04d39 Mon Sep 17 00:00:00 2001 From: Philipp Stephani <phst@google.com> Date: Sun, 20 Dec 2015 01:57:34 +0100 Subject: [PATCH] Remove build system name from dump. * sysdep.c (init_system_name): Use a constant if we might dump. * version.el (emacs-build-time): Add a comment to remove the variable. (emacs-build-system): Make variable obsolete, replace its value with a constant. --- lisp/version.el | 9 ++++++--- src/sysdep.c | 7 +++++++ 2 files changed, 13 insertions(+), 3 deletions(-) diff --git a/lisp/version.el b/lisp/version.el index 43103fd..d755874 100644 --- a/lisp/version.el +++ b/lisp/version.el @@ -38,13 +38,16 @@ emacs-minor-version "Minor version number of this version of Emacs. This variable first existed in version 19.23.") +;; FIXME: The next variable should also be a constant. (defconst emacs-build-time (current-time) "Time at which Emacs was dumped out.") -;; I think this should be obsoleted/removed. It's just one more meaningless -;; difference between different builds. It's usually not even an fqdn. -(defconst emacs-build-system (system-name) +;; This was the actual system name before, but that made the dump +;; non-deterministic, so now it's just a constant. +(defconst emacs-build-system "unknown" "Name of the system on which Emacs was built.") +(make-obsolete-variable 'emacs-build-system + "this information is no longer available." "25.2") (defvar motif-version-string) (defvar gtk-version-string) diff --git a/src/sysdep.c b/src/sysdep.c index 1af323e..5bed2f9 100644 --- a/src/sysdep.c +++ b/src/sysdep.c @@ -1399,6 +1399,13 @@ setup_pty (int fd) void init_system_name (void) { + if (might_dump) + { + /* If we're dumping, set the hostname to a literal so that the + dump is deterministic. */ + Vsystem_name = build_pure_c_string ("unknown"); + return; + } char *hostname_alloc = NULL; char *hostname; #ifndef HAVE_GETHOSTNAME -- 2.6.4 ^ permalink raw reply related [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-20 12:48 ` Philipp Stephani @ 2015-12-20 16:37 ` Eli Zaretskii 2015-12-20 18:39 ` Philipp Stephani 0 siblings, 1 reply; 53+ messages in thread From: Eli Zaretskii @ 2015-12-20 16:37 UTC (permalink / raw) To: Philipp Stephani; +Cc: eggert, emacs-devel > From: Philipp Stephani <p.stephani2@gmail.com> > Date: Sun, 20 Dec 2015 12:48:17 +0000 > Cc: emacs-devel@gnu.org > > Paul Eggert <eggert@cs.ucla.edu> schrieb am Mo., 7. Dez. 2015 um 20:14 Uhr: > > On 12/07/2015 11:00 AM, Philipp Stephani wrote: > > this would be a rather big change. > > I suppose that depends on what one means by "big". To me it feels like a > fairly small improvement. We've already made some progress along these > lines in emacs-25 by removing emacs-build-system from the result of the > emacs-version function; we just need to make more improvements like that. > > It should be in 'master', though, not in 'emacs-25', as it's not fixing > a bug. > > OK, here's a first patch that replaces system-name with a constant during > dumping. That eliminates one source of nondeterminism. I'm sorry, but I don't like where this is going. I regret I didn't speak up earlier, but better late than never. In a nutshell, I object to losing valuable information for the benefit of a single specialized use case. I have nothing against supporting reproducible builds as long as this removes information and data that is not important to us humans when developing or debugging Emacs. For example, changes in loaddefs files that remove non-deterministic information from there or replace it with deterministic information are fine with me. But here we are losing information about the build system that I found useful more than once in the past when tracking bugs. We have already removed the build system name from the bug report produced by report-emacs-bug (I'm going to add it back, btw, unless someone provides a _very_ good reason for its removal). Earlier, we removed the last keystrokes, because someone was worried about their privacy, but couldn't be bothered to delete the sensitive parts from the bug report. And your comments indicate that you intend to remove the build time as well, so it will be impossible to say whether a given binary was built before or after some change. Where is all this going? I'm sorry, but I cannot agree to this slippery slope, just so we could support a single specialized use case. Let's support that use case by providing some build-time option, either a configure-time option or a Makefile option, or an environment variable, or something else. If someone presents a convincing case, I might even agree to have the release tarball build by default with that information removed (although I found it useful in the past when reading bug reports). But doing this in the development version is simply a non-starter. When debugging some problem that happened elsewhere, we quite often need every bit of information that could potentially help us, so removing it is out of the question. I appreciate your efforts in providing the patch, but we will have to find another way of supporting reproducible builds. Thanks. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-20 16:37 ` Eli Zaretskii @ 2015-12-20 18:39 ` Philipp Stephani 2015-12-20 19:03 ` Eli Zaretskii 2015-12-20 21:27 ` Paul Eggert 0 siblings, 2 replies; 53+ messages in thread From: Philipp Stephani @ 2015-12-20 18:39 UTC (permalink / raw) To: Eli Zaretskii; +Cc: eggert, emacs-devel [-- Attachment #1: Type: text/plain, Size: 3643 bytes --] Eli Zaretskii <eliz@gnu.org> schrieb am So., 20. Dez. 2015 um 17:37 Uhr: > > From: Philipp Stephani <p.stephani2@gmail.com> > > Date: Sun, 20 Dec 2015 12:48:17 +0000 > > Cc: emacs-devel@gnu.org > > > > Paul Eggert <eggert@cs.ucla.edu> schrieb am Mo., 7. Dez. 2015 um 20:14 > Uhr: > > > > On 12/07/2015 11:00 AM, Philipp Stephani wrote: > > > this would be a rather big change. > > > > I suppose that depends on what one means by "big". To me it feels > like a > > fairly small improvement. We've already made some progress along > these > > lines in emacs-25 by removing emacs-build-system from the result of > the > > emacs-version function; we just need to make more improvements like > that. > > > > It should be in 'master', though, not in 'emacs-25', as it's not > fixing > > a bug. > > > > OK, here's a first patch that replaces system-name with a constant during > > dumping. That eliminates one source of nondeterminism. > > I'm sorry, but I don't like where this is going. I regret I didn't > speak up earlier, but better late than never. > > In a nutshell, I object to losing valuable information for the benefit > of a single specialized use case. I have nothing against supporting > reproducible builds as long as this removes information and data that > is not important to us humans when developing or debugging Emacs. For > example, changes in loaddefs files that remove non-deterministic > information from there or replace it with deterministic information > are fine with me. > > But here we are losing information about the build system that I found > useful more than once in the past when tracking bugs. We have already > removed the build system name from the bug report produced by > report-emacs-bug (I'm going to add it back, btw, unless someone > provides a _very_ good reason for its removal). Earlier, we removed > the last keystrokes, because someone was worried about their privacy, > but couldn't be bothered to delete the sensitive parts from the bug > report. And your comments indicate that you intend to remove the > build time as well, so it will be impossible to say whether a given > binary was built before or after some change. Where is all this > going? > > I'm sorry, but I cannot agree to this slippery slope, just so we could > support a single specialized use case. Let's support that use case by > providing some build-time option, either a configure-time option or a > Makefile option, or an environment variable, or something else. If > someone presents a convincing case, I might even agree to have the > release tarball build by default with that information removed > (although I found it useful in the past when reading bug reports). > But doing this in the development version is simply a non-starter. > When debugging some problem that happened elsewhere, we quite often > need every bit of information that could potentially help us, so > removing it is out of the question. > > I appreciate your efforts in providing the patch, but we will have to > find another way of supporting reproducible builds. > > > Thanks for your feedback. I'm totally happy with making this configurable and turning it off by default. My suggestion would be to use an environment variable EMACS_DETERMINISTIC_BUILD, which is used to initialize a new Boolean variable `deterministic-build'. If the variable is set, the core and Lisp packages would switch to deterministic output. I'd prefer to have this configurable at runtime so that a single Emacs binary can be used to produce both deterministic and non-deterministic output (e.g. for loaddefs). Would that be OK? [-- Attachment #2: Type: text/html, Size: 4432 bytes --] ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-20 18:39 ` Philipp Stephani @ 2015-12-20 19:03 ` Eli Zaretskii 2015-12-20 21:27 ` Paul Eggert 1 sibling, 0 replies; 53+ messages in thread From: Eli Zaretskii @ 2015-12-20 19:03 UTC (permalink / raw) To: Philipp Stephani; +Cc: eggert, emacs-devel > From: Philipp Stephani <p.stephani2@gmail.com> > Date: Sun, 20 Dec 2015 18:39:04 +0000 > Cc: eggert@cs.ucla.edu, emacs-devel@gnu.org > > Thanks for your feedback. I'm totally happy with making this configurable and > turning it off by default. My suggestion would be to use an environment > variable EMACS_DETERMINISTIC_BUILD, which is used to initialize a new Boolean > variable `deterministic-build'. If the variable is set, the core and Lisp > packages would switch to deterministic output. I'd prefer to have this > configurable at runtime so that a single Emacs binary can be used to produce > both deterministic and non-deterministic output (e.g. for loaddefs). Would that > be OK? It's definitely OK with me, but let's wait for a couple of days for comments from others, in case I'm missing something. I'd hate to have you do the work again, only to discover some other factor that needs to be considered. Thanks. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-20 18:39 ` Philipp Stephani 2015-12-20 19:03 ` Eli Zaretskii @ 2015-12-20 21:27 ` Paul Eggert 2015-12-23 1:13 ` John Wiegley 1 sibling, 1 reply; 53+ messages in thread From: Paul Eggert @ 2015-12-20 21:27 UTC (permalink / raw) To: Philipp Stephani, Eli Zaretskii; +Cc: emacs-devel Philipp Stephani wrote: > I'd prefer to have > this configurable at runtime so that a single Emacs binary can be used to > produce both deterministic and non-deterministic outpu I'd rather not have Emacs's behavior depend on yet another environment variable. There are already too many such environment variables and they are a maintenance and configuration pain. Instead, I suggest making it a configure-time option, e.g., ./configure --enable-deterministic-build I suppose Emacs code could also set the new Elisp deterministic-build variable that you're suggesting, to override the configure-time default. But not an environment variable, please. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-20 21:27 ` Paul Eggert @ 2015-12-23 1:13 ` John Wiegley 2015-12-23 16:01 ` Philipp Stephani 0 siblings, 1 reply; 53+ messages in thread From: John Wiegley @ 2015-12-23 1:13 UTC (permalink / raw) To: Paul Eggert; +Cc: Philipp Stephani, Eli Zaretskii, emacs-devel >>>>> Paul Eggert <eggert@cs.ucla.edu> writes: > Instead, I suggest making it a configure-time option, e.g., > ./configure --enable-deterministic-build I'd prefer a configuration options too. > I suppose Emacs code could also set the new Elisp deterministic-build > variable that you're suggesting, to override the configure-time default. But > not an environment variable, please. +1 -- John Wiegley GPG fingerprint = 4710 CF98 AF9B 327B B80F http://newartisans.com 60E1 46C4 BD1A 7AC1 4BA2 ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-23 1:13 ` John Wiegley @ 2015-12-23 16:01 ` Philipp Stephani 2015-12-23 16:39 ` Eli Zaretskii 0 siblings, 1 reply; 53+ messages in thread From: Philipp Stephani @ 2015-12-23 16:01 UTC (permalink / raw) To: Paul Eggert, Eli Zaretskii, emacs-devel [-- Attachment #1.1: Type: text/plain, Size: 516 bytes --] John Wiegley <jwiegley@gmail.com> schrieb am Mi., 23. Dez. 2015 um 02:13 Uhr: > >>>>> Paul Eggert <eggert@cs.ucla.edu> writes: > > > Instead, I suggest making it a configure-time option, e.g., > > ./configure --enable-deterministic-build > > I'd prefer a configuration options too. > > > I suppose Emacs code could also set the new Elisp deterministic-build > > variable that you're suggesting, to override the configure-time default. > But > > not an environment variable, please. > > +1 > > OK, here's a patch. [-- Attachment #1.2: Type: text/html, Size: 934 bytes --] [-- Attachment #2: 0001-Remove-build-system-name-from-deterministic-dumps.patch --] [-- Type: application/octet-stream, Size: 4293 bytes --] From d91d9af6f141a913a34a234b166adc6919ebc69d Mon Sep 17 00:00:00 2001 From: Philipp Stephani <phst@google.com> Date: Sun, 20 Dec 2015 01:57:34 +0100 Subject: [PATCH] Remove build system name from deterministic dumps. * configure.ac (DETERMINISTIC_DUMP): New configuration option. * emacs.c (main): Initialize `deterministic-dump' from the configuration option. (syms_of_emacs): New constant `deterministic-dump'. * sysdep.c (init_system_name): Use a constant if a deterministic dump is requested. * version.el (emacs-build-time): Add a comment to make the build time deterministic if requested. (emacs-build-system): Make variable deterministic if requested. --- configure.ac | 10 ++++++++++ lisp/version.el | 7 ++++--- src/emacs.c | 15 +++++++++++++++ src/sysdep.c | 9 +++++++++ 4 files changed, 38 insertions(+), 3 deletions(-) diff --git a/configure.ac b/configure.ac index 0b7b403..b1b7181 100644 --- a/configure.ac +++ b/configure.ac @@ -542,6 +542,16 @@ AC_ARG_ENABLE(gtk-deprecation-warnings, [Show Gtk+/Gdk deprecation warnings for Gtk+ >= 3.0])], [ac_enable_gtk_deprecation_warnings="${enableval}"],[]) +AC_ARG_ENABLE(deterministic-dump, +[AS_HELP_STRING([--enable-deterministic-dump], + [Make dumping deterministic by removing system-specific + information from the dump, such as host names and + timestamps.])]) +if test "x${enableval}" != xno ; then + AC_DEFINE(DETERMINISTIC_DUMP, 1, + [Define this to make dumping deterministic.]) +fi + dnl This used to use changequote, but, apart from 'changequote is evil' dnl per the autoconf manual, we can speed up autoconf somewhat by quoting dnl the great gob of text. Thus it's not processed for possible expansion. diff --git a/lisp/version.el b/lisp/version.el index 43103fd..6af80c6 100644 --- a/lisp/version.el +++ b/lisp/version.el @@ -38,12 +38,13 @@ emacs-minor-version "Minor version number of this version of Emacs. This variable first existed in version 19.23.") +;; FIXME: The next variable should also be a constant if +;; `deterministic-dump' is t. (defconst emacs-build-time (current-time) "Time at which Emacs was dumped out.") -;; I think this should be obsoleted/removed. It's just one more meaningless -;; difference between different builds. It's usually not even an fqdn. -(defconst emacs-build-system (system-name) +(defconst emacs-build-system + (if deterministic-dump "unknown" (system-name)) "Name of the system on which Emacs was built.") (defvar motif-version-string) diff --git a/src/emacs.c b/src/emacs.c index 2e9f950..ac9e524 100644 --- a/src/emacs.c +++ b/src/emacs.c @@ -925,6 +925,10 @@ main (int argc, char **argv) SET_BINARY (fileno (stdout)); #endif /* MSDOS */ +#ifdef DETERMINISTIC_DUMP + Vdeterministic_dump = Qt; +#endif /* DETERMINISTIC_DUMP */ + /* Skip initial setlocale if LC_ALL is "C", as it's not needed in that case. The build procedure uses this while dumping, to ensure that the dumped Emacs does not have its system locale tables initialized, @@ -2566,6 +2570,17 @@ libraries; only those already known by Emacs will be loaded. */); Vdynamic_library_alist = Qnil; Fput (intern_c_string ("dynamic-library-alist"), Qrisky_local_variable, Qt); + DEFVAR_BOOL ("deterministic-dump", Vdeterministic_dump, + doc: /* If t, attempt to make dumping deterministic by avoiding +sources of nondeterminism such as absolute paths, the hostname, or +timestamps. */); +#ifdef DETERMINISTIC_DUMP + Vdeterministic_dump = Qt; +#else + Vdeterministic_dump = Qnil; +#endif + XSYMBOL (intern_c_string ("deterministic-dump"))->constant = 1; + #ifdef WINDOWSNT Vlibrary_cache = Qnil; staticpro (&Vlibrary_cache); diff --git a/src/sysdep.c b/src/sysdep.c index 1af323e..ea322a0 100644 --- a/src/sysdep.c +++ b/src/sysdep.c @@ -1399,6 +1399,15 @@ setup_pty (int fd) void init_system_name (void) { +#ifdef DETERMINISTIC_DUMP + if (might_dump) + { + /* If we're dumping, set the hostname to a literal so that the + dump is deterministic. */ + Vsystem_name = build_pure_c_string ("unknown"); + return; + } +#endif /* DETERMINISTIC_DUMP */ char *hostname_alloc = NULL; char *hostname; #ifndef HAVE_GETHOSTNAME -- 2.6.3 ^ permalink raw reply related [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-23 16:01 ` Philipp Stephani @ 2015-12-23 16:39 ` Eli Zaretskii 2015-12-23 16:52 ` Philipp Stephani 0 siblings, 1 reply; 53+ messages in thread From: Eli Zaretskii @ 2015-12-23 16:39 UTC (permalink / raw) To: Philipp Stephani; +Cc: eggert, emacs-devel > From: Philipp Stephani <p.stephani2@gmail.com> > Date: Wed, 23 Dec 2015 16:01:30 +0000 > > OK, here's a patch. Thanks. A few minor comments. > +AC_ARG_ENABLE(deterministic-dump, > +[AS_HELP_STRING([--enable-deterministic-dump], > + [Make dumping deterministic by removing system-specific > + information from the dump, such as host names and > + timestamps.])]) I'd prefer calling this "deterministic-build", so that we could use it to solve any other similar problems unrelated to dumping. Maybe even "reproducible-build". (If we do change the option, we should also change the name of the variable and the cpp symbol.) > +(defconst emacs-build-system > + (if deterministic-dump "unknown" (system-name)) > "Name of the system on which Emacs was built.") Can we find a better string, like "elided" or "undisclosed"? "Unknown" sounds too self-derogatory. > + DEFVAR_BOOL ("deterministic-dump", Vdeterministic_dump, > + doc: /* If t, attempt to make dumping deterministic by avoiding I think the usual style is "If non-nil". > +sources of nondeterminism such as absolute paths, the hostname, or GNU coding standards frown on using "path" for anything that isn't PATH-style list of directories. Please use "file name" instead. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-23 16:39 ` Eli Zaretskii @ 2015-12-23 16:52 ` Philipp Stephani 2015-12-23 17:10 ` Eli Zaretskii 2015-12-27 9:53 ` Philipp Stephani 0 siblings, 2 replies; 53+ messages in thread From: Philipp Stephani @ 2015-12-23 16:52 UTC (permalink / raw) To: Eli Zaretskii; +Cc: eggert, emacs-devel [-- Attachment #1: Type: text/plain, Size: 2124 bytes --] Eli Zaretskii <eliz@gnu.org> schrieb am Mi., 23. Dez. 2015 um 17:38 Uhr: > > From: Philipp Stephani <p.stephani2@gmail.com> > > Date: Wed, 23 Dec 2015 16:01:30 +0000 > > > > OK, here's a patch. > > Thanks. A few minor comments. > > > +AC_ARG_ENABLE(deterministic-dump, > > +[AS_HELP_STRING([--enable-deterministic-dump], > > + [Make dumping deterministic by removing system-specific > > + information from the dump, such as host names and > > + timestamps.])]) > > I'd prefer calling this "deterministic-build", so that we could use it > to solve any other similar problems unrelated to dumping. Maybe even > "reproducible-build". > I think we have two categories of builds here: - Builds that get executed when running 'make' in the Emacs directory. This includes dumping and byte compilation of code bundled with Emacs. - Builds that get executed by running an initialized Emacs binary, likely on a different machine. This includes e.g. byte compilation of third-party libraries. I think these two categories are separate enough to have two different variables. This patch only addresses the first category, therefore I didn't want to use the generic term "build", but "dump" is also not quite right. Do we have a proper term for the first category? "bootstrap"? "emacs-build"? > > (If we do change the option, we should also change the name of the > variable and the cpp symbol.) > > > +(defconst emacs-build-system > > + (if deterministic-dump "unknown" (system-name)) > > "Name of the system on which Emacs was built.") > > Can we find a better string, like "elided" or "undisclosed"? > "Unknown" sounds too self-derogatory. > Sure, 'elided' is fine. > > > + DEFVAR_BOOL ("deterministic-dump", Vdeterministic_dump, > > + doc: /* If t, attempt to make dumping deterministic by avoiding > > I think the usual style is "If non-nil". > ok > > > +sources of nondeterminism such as absolute paths, the hostname, or > > GNU coding standards frown on using "path" for anything that isn't > PATH-style list of directories. Please use "file name" instead. > > ok [-- Attachment #2: Type: text/html, Size: 3309 bytes --] ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-23 16:52 ` Philipp Stephani @ 2015-12-23 17:10 ` Eli Zaretskii 2015-12-23 17:38 ` Philipp Stephani 2015-12-27 9:53 ` Philipp Stephani 1 sibling, 1 reply; 53+ messages in thread From: Eli Zaretskii @ 2015-12-23 17:10 UTC (permalink / raw) To: Philipp Stephani; +Cc: eggert, emacs-devel > From: Philipp Stephani <p.stephani2@gmail.com> > Date: Wed, 23 Dec 2015 16:52:08 +0000 > Cc: eggert@cs.ucla.edu, emacs-devel@gnu.org > > I'd prefer calling this "deterministic-build", so that we could use it > to solve any other similar problems unrelated to dumping. Maybe even > "reproducible-build". > > I think we have two categories of builds here: > - Builds that get executed when running 'make' in the Emacs directory. This > includes dumping and byte compilation of code bundled with Emacs. > - Builds that get executed by running an initialized Emacs binary, likely on a > different machine. This includes e.g. byte compilation of third-party > libraries. > I think these two categories are separate enough to have two different > variables. Why not use the same variable/option for both? It sounds simpler to me. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-23 17:10 ` Eli Zaretskii @ 2015-12-23 17:38 ` Philipp Stephani 2015-12-23 21:09 ` Paul Eggert 0 siblings, 1 reply; 53+ messages in thread From: Philipp Stephani @ 2015-12-23 17:38 UTC (permalink / raw) To: Eli Zaretskii; +Cc: eggert, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1259 bytes --] Eli Zaretskii <eliz@gnu.org> schrieb am Mi., 23. Dez. 2015 um 18:10 Uhr: > > From: Philipp Stephani <p.stephani2@gmail.com> > > Date: Wed, 23 Dec 2015 16:52:08 +0000 > > Cc: eggert@cs.ucla.edu, emacs-devel@gnu.org > > > > I'd prefer calling this "deterministic-build", so that we could use > it > > to solve any other similar problems unrelated to dumping. Maybe even > > "reproducible-build". > > > > I think we have two categories of builds here: > > - Builds that get executed when running 'make' in the Emacs directory. > This > > includes dumping and byte compilation of code bundled with Emacs. > > - Builds that get executed by running an initialized Emacs binary, > likely on a > > different machine. This includes e.g. byte compilation of third-party > > libraries. > > I think these two categories are separate enough to have two different > > variables. > > Why not use the same variable/option for both? It sounds simpler to > me. > > I thought that too, but on second thought it seems rather difficult: the variable should be set to nil by default in an initialized Emacs (and probably also in temacs), yet we need a way to set it to t based on the configuration option. The simplest solution seems to be to just use two variables. [-- Attachment #2: Type: text/html, Size: 1847 bytes --] ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-23 17:38 ` Philipp Stephani @ 2015-12-23 21:09 ` Paul Eggert 2015-12-24 3:33 ` Eli Zaretskii 0 siblings, 1 reply; 53+ messages in thread From: Paul Eggert @ 2015-12-23 21:09 UTC (permalink / raw) To: Philipp Stephani, Eli Zaretskii; +Cc: emacs-devel Philipp Stephani wrote: > I thought that too, but on second thought it seems rather difficult: the > variable should be set to nil by default in an initialized Emacs (and > probably also in temacs), yet we need a way to set it to t based on the > configuration option. Doesn't your proposed patch already set deterministic-dump to t based on the configuration option? I don't see the difficulty. > + (if deterministic-dump "unknown" (system-name)) How about changing this to: (when (not deterministic-dump) (system-name)) and document that the value is nil when unavailable? That should be cleaner than the confusion if a system name actually happens to be "unknown". Similarly, init_system_name should leave Vsystem_name alone when DETERMINISTIC_DUMP. > +#ifdef DETERMINISTIC_DUMP Please change this to "if (DETERMINISTIC_DUMP)", and similarly for other uses, and arrange for DETERMINISTIC_DUMP to be a constant false or true. Nowadays we should prefer macros to constants only when they're really necessary, which isn't the case here. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-23 21:09 ` Paul Eggert @ 2015-12-24 3:33 ` Eli Zaretskii 2015-12-24 6:54 ` Paul Eggert 0 siblings, 1 reply; 53+ messages in thread From: Eli Zaretskii @ 2015-12-24 3:33 UTC (permalink / raw) To: Paul Eggert; +Cc: p.stephani2, emacs-devel > Cc: emacs-devel@gnu.org > From: Paul Eggert <eggert@cs.ucla.edu> > Date: Wed, 23 Dec 2015 13:09:21 -0800 > > > + (if deterministic-dump "unknown" (system-name)) > > How about changing this to: > > (when (not deterministic-dump) (system-name)) > > and document that the value is nil when unavailable? Some code out there that expects the value to be a string might become broken by such a change, I think. IOW, I think we shouldn't make this an incompatible change. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-24 3:33 ` Eli Zaretskii @ 2015-12-24 6:54 ` Paul Eggert 2015-12-24 16:18 ` Eli Zaretskii 0 siblings, 1 reply; 53+ messages in thread From: Paul Eggert @ 2015-12-24 6:54 UTC (permalink / raw) To: Eli Zaretskii; +Cc: p.stephani2, emacs-devel Eli Zaretskii wrote: > Some code out there that expects the value to be a string might become > broken by such a change, I think. I doubt whether there is any such code that will break. No code uses the variable within Emacs itself; I just checked. And the variable is not documented to contain only a string, so strictly speaking the implementation can make the variable nil if wants to. We should also obsolete the variable, while we're at it. There's a comment to that effect before the variable. Really, code should not be using emacs-build-system. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-24 6:54 ` Paul Eggert @ 2015-12-24 16:18 ` Eli Zaretskii 0 siblings, 0 replies; 53+ messages in thread From: Eli Zaretskii @ 2015-12-24 16:18 UTC (permalink / raw) To: Paul Eggert; +Cc: p.stephani2, emacs-devel > Cc: p.stephani2@gmail.com, emacs-devel@gnu.org > From: Paul Eggert <eggert@cs.ucla.edu> > Date: Wed, 23 Dec 2015 22:54:04 -0800 > > Eli Zaretskii wrote: > > Some code out there that expects the value to be a string might become > > broken by such a change, I think. > > I doubt whether there is any such code that will break. It doesn't matter. Let's not make changes that gratuitously break backward compatibility. A fixed string is good enough to serve this use case, so no need to make it nil. > No code uses the variable within Emacs itself; I just checked. I plan on adding it to the bug report, where it was before a recent change to emacs-version. This information is sometimes very important when working on bugs. > We should also obsolete the variable, while we're at it. There's a comment to > that effect before the variable. Really, code should not be using > emacs-build-system. It's important information in some cases, and I don't plan to remove it any time soon, sorry. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-23 16:52 ` Philipp Stephani 2015-12-23 17:10 ` Eli Zaretskii @ 2015-12-27 9:53 ` Philipp Stephani 2015-12-27 16:10 ` Eli Zaretskii 2015-12-27 23:26 ` Paul Eggert 1 sibling, 2 replies; 53+ messages in thread From: Philipp Stephani @ 2015-12-27 9:53 UTC (permalink / raw) To: Eli Zaretskii; +Cc: eggert, emacs-devel [-- Attachment #1.1: Type: text/plain, Size: 1163 bytes --] Philipp Stephani <p.stephani2@gmail.com> schrieb am Mi., 23. Dez. 2015 um 17:52 Uhr: > >> (If we do change the option, we should also change the name of the >> variable and the cpp symbol.) >> >> > +(defconst emacs-build-system >> > + (if deterministic-dump "unknown" (system-name)) >> > "Name of the system on which Emacs was built.") >> >> Can we find a better string, like "elided" or "undisclosed"? >> "Unknown" sounds too self-derogatory. >> > > Sure, 'elided' is fine. > > >> >> > + DEFVAR_BOOL ("deterministic-dump", Vdeterministic_dump, >> > + doc: /* If t, attempt to make dumping deterministic by avoiding >> >> I think the usual style is "If non-nil". >> > > ok > > >> >> > +sources of nondeterminism such as absolute paths, the hostname, or >> >> GNU coding standards frown on using "path" for anything that isn't >> PATH-style list of directories. Please use "file name" instead. >> >> > ok > Attached a new patch with the requested changes. It would be great to defer the discussion about naming, string vs. nil etc. until later so that this project can make progress; there's still enough time to discuss until the changes get releases. [-- Attachment #1.2: Type: text/html, Size: 2350 bytes --] [-- Attachment #2: 0001-Remove-build-system-name-from-deterministic-dumps.patch --] [-- Type: application/octet-stream, Size: 4319 bytes --] From 6ac74525e09456e0be32f8a25dcd313af0ac537d Mon Sep 17 00:00:00 2001 From: Philipp Stephani <phst@google.com> Date: Sun, 20 Dec 2015 01:57:34 +0100 Subject: [PATCH] Remove build system name from deterministic dumps. * configure.ac (DETERMINISTIC_DUMP): New configuration option. * emacs.c (main): Initialize `deterministic-dump' from the configuration option. (syms_of_emacs): New constant `deterministic-dump'. * sysdep.c (init_system_name): Use a constant if a deterministic dump is requested. * version.el (emacs-build-time): Add a comment to make the build time deterministic if requested. (emacs-build-system): Make variable deterministic if requested. --- configure.ac | 13 +++++++++++++ lisp/version.el | 7 ++++--- src/emacs.c | 10 ++++++++++ src/sysdep.c | 7 +++++++ 4 files changed, 34 insertions(+), 3 deletions(-) diff --git a/configure.ac b/configure.ac index 0b7b403..abbe7f8 100644 --- a/configure.ac +++ b/configure.ac @@ -542,6 +542,19 @@ AC_ARG_ENABLE(gtk-deprecation-warnings, [Show Gtk+/Gdk deprecation warnings for Gtk+ >= 3.0])], [ac_enable_gtk_deprecation_warnings="${enableval}"],[]) +AC_ARG_ENABLE(deterministic-dump, +[AS_HELP_STRING([--enable-deterministic-dump], + [Make dumping deterministic by removing system-specific + information from the dump, such as host names and + timestamps.])]) +if test "x${enableval}" = xno ; then + AC_DEFINE(DETERMINISTIC_DUMP, false, + [Set this to true to make dumping deterministic.]) +else + AC_DEFINE(DETERMINISTIC_DUMP, true, + [Set this to true to make dumping deterministic.]) +fi + dnl This used to use changequote, but, apart from 'changequote is evil' dnl per the autoconf manual, we can speed up autoconf somewhat by quoting dnl the great gob of text. Thus it's not processed for possible expansion. diff --git a/lisp/version.el b/lisp/version.el index 43103fd..a3f0ea0 100644 --- a/lisp/version.el +++ b/lisp/version.el @@ -38,12 +38,13 @@ emacs-minor-version "Minor version number of this version of Emacs. This variable first existed in version 19.23.") +;; FIXME: The next variable should also be a constant if +;; `deterministic-dump' is t. (defconst emacs-build-time (current-time) "Time at which Emacs was dumped out.") -;; I think this should be obsoleted/removed. It's just one more meaningless -;; difference between different builds. It's usually not even an fqdn. -(defconst emacs-build-system (system-name) +(defconst emacs-build-system + (if deterministic-dump "elided" (system-name)) "Name of the system on which Emacs was built.") (defvar motif-version-string) diff --git a/src/emacs.c b/src/emacs.c index 2e9f950..863473a 100644 --- a/src/emacs.c +++ b/src/emacs.c @@ -925,6 +925,9 @@ main (int argc, char **argv) SET_BINARY (fileno (stdout)); #endif /* MSDOS */ + if (DETERMINISTIC_DUMP) + Vdeterministic_dump = Qt; + /* Skip initial setlocale if LC_ALL is "C", as it's not needed in that case. The build procedure uses this while dumping, to ensure that the dumped Emacs does not have its system locale tables initialized, @@ -2566,6 +2569,13 @@ libraries; only those already known by Emacs will be loaded. */); Vdynamic_library_alist = Qnil; Fput (intern_c_string ("dynamic-library-alist"), Qrisky_local_variable, Qt); + DEFVAR_BOOL ("deterministic-dump", Vdeterministic_dump, + doc: /* If non-nil, attempt to make dumping deterministic by +avoiding sources of nondeterminism such as absolute file names, the +hostname, or timestamps. */); + Vdeterministic_dump = DETERMINISTIC_DUMP ? Qt : Qnil; + XSYMBOL (intern_c_string ("deterministic-dump"))->constant = 1; + #ifdef WINDOWSNT Vlibrary_cache = Qnil; staticpro (&Vlibrary_cache); diff --git a/src/sysdep.c b/src/sysdep.c index 1af323e..5bbf723 100644 --- a/src/sysdep.c +++ b/src/sysdep.c @@ -1399,6 +1399,13 @@ setup_pty (int fd) void init_system_name (void) { + if (DETERMINISTIC_DUMP && (might_dump || ! NILP (Vpurify_flag))) + { + /* If we're dumping, set the hostname to a literal so that the + dump is deterministic. */ + Vsystem_name = build_pure_c_string ("elided"); + return; + } char *hostname_alloc = NULL; char *hostname; #ifndef HAVE_GETHOSTNAME -- 2.6.3 ^ permalink raw reply related [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-27 9:53 ` Philipp Stephani @ 2015-12-27 16:10 ` Eli Zaretskii 2016-02-29 21:52 ` Philipp Stephani 2015-12-27 23:26 ` Paul Eggert 1 sibling, 1 reply; 53+ messages in thread From: Eli Zaretskii @ 2015-12-27 16:10 UTC (permalink / raw) To: Philipp Stephani; +Cc: eggert, emacs-devel > From: Philipp Stephani <p.stephani2@gmail.com> > Date: Sun, 27 Dec 2015 09:53:00 +0000 > Cc: eggert@cs.ucla.edu, emacs-devel@gnu.org > > Attached a new patch with the requested changes. LGTM, thanks. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-27 16:10 ` Eli Zaretskii @ 2016-02-29 21:52 ` Philipp Stephani 2016-03-01 16:36 ` Paul Eggert 0 siblings, 1 reply; 53+ messages in thread From: Philipp Stephani @ 2016-02-29 21:52 UTC (permalink / raw) To: Eli Zaretskii; +Cc: eggert, emacs-devel [-- Attachment #1: Type: text/plain, Size: 378 bytes --] Eli Zaretskii <eliz@gnu.org> schrieb am So., 27. Dez. 2015 um 17:10 Uhr: > > From: Philipp Stephani <p.stephani2@gmail.com> > > Date: Sun, 27 Dec 2015 09:53:00 +0000 > > Cc: eggert@cs.ucla.edu, emacs-devel@gnu.org > > > > Attached a new patch with the requested changes. > > LGTM, thanks. > What's the status of this? Could one of the patches get merged into master? Thanks. [-- Attachment #2: Type: text/html, Size: 859 bytes --] ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2016-02-29 21:52 ` Philipp Stephani @ 2016-03-01 16:36 ` Paul Eggert 2016-03-01 21:46 ` Philipp Stephani 0 siblings, 1 reply; 53+ messages in thread From: Paul Eggert @ 2016-03-01 16:36 UTC (permalink / raw) To: Philipp Stephani, Eli Zaretskii; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 520 bytes --] On 02/29/2016 01:52 PM, Philipp Stephani wrote: > What's the status of this? Could one of the patches get merged into > master? Thanks. Thanks for reminding us. I looked into this. How about if I install the attached pair of patches to start with? The first is yours, the second attempts to make it a bit more build-friendly by making it a runtime switch rather than a compile-time flag. Also, I negated the sense of the switch, since making a build deterministic is the same as disabling some minor features. [-- Attachment #2: 0001-Remove-build-system-name-from-deterministic-dumps.patch --] [-- Type: application/x-patch, Size: 4325 bytes --] [-- Attachment #3: 0002-Deterministic-build-improvements.patch --] [-- Type: application/x-patch, Size: 14061 bytes --] ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2016-03-01 16:36 ` Paul Eggert @ 2016-03-01 21:46 ` Philipp Stephani 2016-03-02 18:24 ` Paul Eggert 0 siblings, 1 reply; 53+ messages in thread From: Philipp Stephani @ 2016-03-01 21:46 UTC (permalink / raw) To: Paul Eggert, Eli Zaretskii; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 659 bytes --] Paul Eggert <eggert@cs.ucla.edu> schrieb am Di., 1. März 2016 um 17:36 Uhr: > On 02/29/2016 01:52 PM, Philipp Stephani wrote: > > What's the status of this? Could one of the patches get merged into > > master? Thanks. > > Thanks for reminding us. I looked into this. How about if I install the > attached pair of patches to start with? The first is yours, the second > attempts to make it a bit more build-friendly by making it a runtime > switch rather than a compile-time flag. Also, I negated the sense of the > switch, since making a build deterministic is the same as disabling some > minor features. > Thanks, both look good to me. [-- Attachment #2: Type: text/html, Size: 957 bytes --] ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2016-03-01 21:46 ` Philipp Stephani @ 2016-03-02 18:24 ` Paul Eggert 2016-03-02 18:30 ` Philipp Stephani 0 siblings, 1 reply; 53+ messages in thread From: Paul Eggert @ 2016-03-02 18:24 UTC (permalink / raw) To: Philipp Stephani, Eli Zaretskii; +Cc: emacs-devel On 03/01/2016 01:46 PM, Philipp Stephani wrote: > Thanks, both look good to me. OK, I installed them into the master. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2016-03-02 18:24 ` Paul Eggert @ 2016-03-02 18:30 ` Philipp Stephani 2016-03-22 13:30 ` Philipp Stephani 0 siblings, 1 reply; 53+ messages in thread From: Philipp Stephani @ 2016-03-02 18:30 UTC (permalink / raw) To: Paul Eggert, Eli Zaretskii; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 221 bytes --] Paul Eggert <eggert@cs.ucla.edu> schrieb am Mi., 2. März 2016 um 19:24 Uhr: > On 03/01/2016 01:46 PM, Philipp Stephani wrote: > > Thanks, both look good to me. > OK, I installed them into the master. > Thanks! [-- Attachment #2: Type: text/html, Size: 494 bytes --] ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2016-03-02 18:30 ` Philipp Stephani @ 2016-03-22 13:30 ` Philipp Stephani 2016-03-22 20:37 ` Paul Eggert 0 siblings, 1 reply; 53+ messages in thread From: Philipp Stephani @ 2016-03-22 13:30 UTC (permalink / raw) To: Paul Eggert, Eli Zaretskii; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 612 bytes --] Philipp Stephani <p.stephani2@gmail.com> schrieb am Mi., 2. März 2016 um 19:30 Uhr: > Paul Eggert <eggert@cs.ucla.edu> schrieb am Mi., 2. März 2016 um > 19:24 Uhr: > >> On 03/01/2016 01:46 PM, Philipp Stephani wrote: >> > Thanks, both look good to me. >> OK, I installed them into the master. >> > > Thanks! > After this patch there are indeed no more nondeterminisms in the .data section. However, there are still ~2000 differing bytes in the ,bss section (generated by unexec). These are hard to debug because they are not covered by the debugging symbols. Any idea where they come from? [-- Attachment #2: Type: text/html, Size: 1193 bytes --] ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2016-03-22 13:30 ` Philipp Stephani @ 2016-03-22 20:37 ` Paul Eggert 2016-03-22 21:46 ` Philipp Stephani 0 siblings, 1 reply; 53+ messages in thread From: Paul Eggert @ 2016-03-22 20:37 UTC (permalink / raw) To: Philipp Stephani, Eli Zaretskii; +Cc: emacs-devel On 03/22/2016 06:30 AM, Philipp Stephani wrote: > there are still ~2000 differing bytes in the ,bss section (generated > by unexec). These are hard to debug because they are not covered by > the debugging symbols. Any idea where they come from? Maybe it's pointer guarding. Try setting LD_POINTER_GUARD=0 in the environment when building Emacs. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2016-03-22 20:37 ` Paul Eggert @ 2016-03-22 21:46 ` Philipp Stephani 2016-03-22 21:58 ` Paul Eggert 2016-03-23 8:03 ` Andreas Schwab 0 siblings, 2 replies; 53+ messages in thread From: Philipp Stephani @ 2016-03-22 21:46 UTC (permalink / raw) To: Paul Eggert, Eli Zaretskii; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 698 bytes --] Paul Eggert <eggert@cs.ucla.edu> schrieb am Di., 22. März 2016 um 21:37 Uhr: > On 03/22/2016 06:30 AM, Philipp Stephani wrote: > > there are still ~2000 differing bytes in the ,bss section (generated > > by unexec). These are hard to debug because they are not covered by > > the debugging symbols. Any idea where they come from? > > Maybe it's pointer guarding. Try setting LD_POINTER_GUARD=0 in the > environment when building Emacs. > Unfortunately there are still ~1000 differing bytes left. Is it possible to determine the source address in temacs/bootstrap-emacs from the position in the ,bss segment? Then maybe we could figure out which data is actually nondeterministic. [-- Attachment #2: Type: text/html, Size: 995 bytes --] ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2016-03-22 21:46 ` Philipp Stephani @ 2016-03-22 21:58 ` Paul Eggert 2016-03-23 8:03 ` Andreas Schwab 1 sibling, 0 replies; 53+ messages in thread From: Paul Eggert @ 2016-03-22 21:58 UTC (permalink / raw) To: Philipp Stephani, Eli Zaretskii; +Cc: emacs-devel On 03/22/2016 02:46 PM, Philipp Stephani wrote: > Is it possible to determine the source address in > temacs/bootstrap-emacs from the position in the ,bss segment? I assume it's possible somehow. You could try running GDB on the dynamic linker. Admittedly not everybody's cup of tea. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2016-03-22 21:46 ` Philipp Stephani 2016-03-22 21:58 ` Paul Eggert @ 2016-03-23 8:03 ` Andreas Schwab 2016-04-06 21:29 ` Philipp Stephani 1 sibling, 1 reply; 53+ messages in thread From: Andreas Schwab @ 2016-03-23 8:03 UTC (permalink / raw) To: Philipp Stephani; +Cc: Eli Zaretskii, Paul Eggert, emacs-devel Philipp Stephani <p.stephani2@gmail.com> writes: > Unfortunately there are still ~1000 differing bytes left. > Is it possible to determine the source address in temacs/bootstrap-emacs > from the position in the ,bss segment? Set a watchpoint. Andreas. -- Andreas Schwab, SUSE Labs, schwab@suse.de GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE 1748 E4D4 88E3 0EEA B9D7 "And now for something completely different." ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2016-03-23 8:03 ` Andreas Schwab @ 2016-04-06 21:29 ` Philipp Stephani 0 siblings, 0 replies; 53+ messages in thread From: Philipp Stephani @ 2016-04-06 21:29 UTC (permalink / raw) To: Andreas Schwab; +Cc: Eli Zaretskii, Paul Eggert, emacs-devel [-- Attachment #1: Type: text/plain, Size: 773 bytes --] Andreas Schwab <schwab@suse.de> schrieb am Mi., 23. März 2016 um 09:03 Uhr: > Philipp Stephani <p.stephani2@gmail.com> writes: > > > Unfortunately there are still ~1000 differing bytes left. > > Is it possible to determine the source address in temacs/bootstrap-emacs > > from the position in the ,bss segment? > > Set a watchpoint. > > Thanks, I've used that (and 'info symbol') to search for a couple more sources of nondeterminism. So far I've found: - various jmp_buf objects; these can most likely be zeroed out before dumping - gl_state; could also be zeroed - consing_since_gc; Fdump_emacs could just call Fgarbage_collect to reset it (though I'm wondering why this difference exists; AFAIK the garbage collector should be pretty deterministic) [-- Attachment #2: Type: text/html, Size: 1171 bytes --] ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-27 9:53 ` Philipp Stephani 2015-12-27 16:10 ` Eli Zaretskii @ 2015-12-27 23:26 ` Paul Eggert 2015-12-28 16:26 ` Eli Zaretskii 1 sibling, 1 reply; 53+ messages in thread From: Paul Eggert @ 2015-12-27 23:26 UTC (permalink / raw) To: Philipp Stephani, Eli Zaretskii; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 1161 bytes --] Philipp Stephani wrote: > It would be great to defer > the discussion about naming, string vs. nil etc. until later so that this > project can make progress; there's still enough time to discuss until the > changes get releases. Better yet, let's just get that stuff done now. I'm attaching a followup that I hope works well enough. The first patch is a copy of yours, except with the commit message reformatted a bit for Emacs standards. The second patch is the followup. The third patch is the two patches combined; this should be easier to review than looking at the individual patches. The basic ideas behind the second patch: * Document the changes in etc/NEWS and in the relevant part of the manual. * Use "deterministic-build", not "deterministic-dump", for the name. I didn't understand the need for two variables here; we can add another one if needed. * Represent unknown values with nil, not with magic strings like "elided" or "unknown". * Make emacs-build-time deterministic, too. * Adjust the output of emacs-version and similar functions to omit the build date. * Remove what appears to be an unnecessary initialization in 'main'. [-- Attachment #2: 0001-Remove-build-system-name-from-deterministic-dumps.patch --] [-- Type: text/x-diff, Size: 4325 bytes --] From a6364c408a156a413d1833992790877f974187d1 Mon Sep 17 00:00:00 2001 From: Philipp Stephani <phst@google.com> Date: Sun, 20 Dec 2015 01:57:34 +0100 Subject: [PATCH 1/2] Remove build system name from deterministic dumps * configure.ac (DETERMINISTIC_DUMP): New configuration option. * lisp/version.el (emacs-build-time): Add a comment to make the build time deterministic if requested. (emacs-build-system): Make variable deterministic if requested. * src/emacs.c (main): Initialize `deterministic-dump' from the configuration option. (syms_of_emacs): New constant `deterministic-dump'. * src/sysdep.c (init_system_name): Use a constant if a deterministic dump is requested. --- configure.ac | 13 +++++++++++++ lisp/version.el | 7 ++++--- src/emacs.c | 10 ++++++++++ src/sysdep.c | 7 +++++++ 4 files changed, 34 insertions(+), 3 deletions(-) diff --git a/configure.ac b/configure.ac index 0b7b403..abbe7f8 100644 --- a/configure.ac +++ b/configure.ac @@ -542,6 +542,19 @@ AC_ARG_ENABLE(gtk-deprecation-warnings, [Show Gtk+/Gdk deprecation warnings for Gtk+ >= 3.0])], [ac_enable_gtk_deprecation_warnings="${enableval}"],[]) +AC_ARG_ENABLE(deterministic-dump, +[AS_HELP_STRING([--enable-deterministic-dump], + [Make dumping deterministic by removing system-specific + information from the dump, such as host names and + timestamps.])]) +if test "x${enableval}" = xno ; then + AC_DEFINE(DETERMINISTIC_DUMP, false, + [Set this to true to make dumping deterministic.]) +else + AC_DEFINE(DETERMINISTIC_DUMP, true, + [Set this to true to make dumping deterministic.]) +fi + dnl This used to use changequote, but, apart from 'changequote is evil' dnl per the autoconf manual, we can speed up autoconf somewhat by quoting dnl the great gob of text. Thus it's not processed for possible expansion. diff --git a/lisp/version.el b/lisp/version.el index 43103fd..a3f0ea0 100644 --- a/lisp/version.el +++ b/lisp/version.el @@ -38,12 +38,13 @@ emacs-minor-version "Minor version number of this version of Emacs. This variable first existed in version 19.23.") +;; FIXME: The next variable should also be a constant if +;; `deterministic-dump' is t. (defconst emacs-build-time (current-time) "Time at which Emacs was dumped out.") -;; I think this should be obsoleted/removed. It's just one more meaningless -;; difference between different builds. It's usually not even an fqdn. -(defconst emacs-build-system (system-name) +(defconst emacs-build-system + (if deterministic-dump "elided" (system-name)) "Name of the system on which Emacs was built.") (defvar motif-version-string) diff --git a/src/emacs.c b/src/emacs.c index 2e9f950..863473a 100644 --- a/src/emacs.c +++ b/src/emacs.c @@ -925,6 +925,9 @@ main (int argc, char **argv) SET_BINARY (fileno (stdout)); #endif /* MSDOS */ + if (DETERMINISTIC_DUMP) + Vdeterministic_dump = Qt; + /* Skip initial setlocale if LC_ALL is "C", as it's not needed in that case. The build procedure uses this while dumping, to ensure that the dumped Emacs does not have its system locale tables initialized, @@ -2566,6 +2569,13 @@ libraries; only those already known by Emacs will be loaded. */); Vdynamic_library_alist = Qnil; Fput (intern_c_string ("dynamic-library-alist"), Qrisky_local_variable, Qt); + DEFVAR_BOOL ("deterministic-dump", Vdeterministic_dump, + doc: /* If non-nil, attempt to make dumping deterministic by +avoiding sources of nondeterminism such as absolute file names, the +hostname, or timestamps. */); + Vdeterministic_dump = DETERMINISTIC_DUMP ? Qt : Qnil; + XSYMBOL (intern_c_string ("deterministic-dump"))->constant = 1; + #ifdef WINDOWSNT Vlibrary_cache = Qnil; staticpro (&Vlibrary_cache); diff --git a/src/sysdep.c b/src/sysdep.c index 1af323e..5bbf723 100644 --- a/src/sysdep.c +++ b/src/sysdep.c @@ -1399,6 +1399,13 @@ setup_pty (int fd) void init_system_name (void) { + if (DETERMINISTIC_DUMP && (might_dump || ! NILP (Vpurify_flag))) + { + /* If we're dumping, set the hostname to a literal so that the + dump is deterministic. */ + Vsystem_name = build_pure_c_string ("elided"); + return; + } char *hostname_alloc = NULL; char *hostname; #ifndef HAVE_GETHOSTNAME -- 2.5.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #3: 0002-Deterministic-build-improvements.patch --] [-- Type: text/x-diff; name="0002-Deterministic-build-improvements.patch", Size: 11151 bytes --] From 63ece9364a72fc981778a93eb4359db1d33fdd74 Mon Sep 17 00:00:00 2001 From: Paul Eggert <eggert@cs.ucla.edu> Date: Sun, 27 Dec 2015 14:50:27 -0800 Subject: [PATCH 2/2] Deterministic build improvements * configure.ac (DETERMINISTIC_BUILD): Rename from DETERMINISTIC_DUMP. All uses changed. Change option to --enable-deterministic-build. * doc/lispref/internals.texi (Building Emacs): * etc/NEWS: Document --enable-deterministic-build. * doc/lispref/intro.texi (Version Info): Remove documentation for emacs-build-time. It is no longer correct, and this variable should not be documented in the intro, any more than emacs-build-system should be documented. * lisp/erc/erc-compat.el (erc-emacs-build-time): Now nil with deterministic builds. * lisp/erc/erc.el (erc-cmd-SV): * lisp/version.el (emacs-version): Do not output build time if deterministic build. * lisp/version.el (emacs-build-time, emacs-build-system): Now nil if deterministic build. * src/emacs.c: Rename var from deterministic-dump to deterministic-build. All uses changed. (main): Remove unnecessary initialization of Vdeterministic_build. (syms_of_emacs): Set Vdeterministic_build to a boolean, not to a Lisp_Object. No need to set it if it is false, since false is the default. * src/sysdep.c (init_system_name): When dumping and deterministic build, set system-name to nil, not to "elided". --- configure.ac | 13 ++++++------- doc/lispref/internals.texi | 8 ++++++++ doc/lispref/intro.texi | 13 ------------- etc/NEWS | 8 ++++++++ lisp/erc/erc-compat.el | 5 ++--- lisp/erc/erc.el | 6 ++++-- lisp/version.el | 23 +++++++++++++---------- src/emacs.c | 15 ++++++--------- src/sysdep.c | 7 +++---- 9 files changed, 50 insertions(+), 48 deletions(-) diff --git a/configure.ac b/configure.ac index abbe7f8..9dce607 100644 --- a/configure.ac +++ b/configure.ac @@ -542,16 +542,15 @@ AC_ARG_ENABLE(gtk-deprecation-warnings, [Show Gtk+/Gdk deprecation warnings for Gtk+ >= 3.0])], [ac_enable_gtk_deprecation_warnings="${enableval}"],[]) -AC_ARG_ENABLE(deterministic-dump, -[AS_HELP_STRING([--enable-deterministic-dump], - [Make dumping deterministic by removing system-specific - information from the dump, such as host names and - timestamps.])]) +AC_ARG_ENABLE(deterministic-build, +[AS_HELP_STRING([--enable-deterministic-build], + [Make the build deterministic by omitting host names, + time stamps, etc. from the output.])]) if test "x${enableval}" = xno ; then - AC_DEFINE(DETERMINISTIC_DUMP, false, + AC_DEFINE(DETERMINISTIC_BUILD, false, [Set this to true to make dumping deterministic.]) else - AC_DEFINE(DETERMINISTIC_DUMP, true, + AC_DEFINE(DETERMINISTIC_BUILD, true, [Set this to true to make dumping deterministic.]) fi diff --git a/doc/lispref/internals.texi b/doc/lispref/internals.texi index e620da0..1b7dcfa 100644 --- a/doc/lispref/internals.texi +++ b/doc/lispref/internals.texi @@ -64,6 +64,14 @@ Building Emacs and are not able to implement dumping, then Emacs must load @file{loadup.el} each time it starts. +@cindex deterministic build +@cindex @option{--enable-deterministic-build} option to @command{configure} + By default the dumped @file{emacs} executable contains details such +as the build time and host name. Use the +@option{--enable-determistic-build} option of @command{configure} to +suppress these details, so that building and installing Emacs twice +from the same sources should result in identical copies of Emacs. + @cindex @file{site-load.el} You can specify additional files to preload by writing a library named @file{site-load.el} that loads them. You may need to rebuild Emacs diff --git a/doc/lispref/intro.texi b/doc/lispref/intro.texi index 865c698..24a7131 100644 --- a/doc/lispref/intro.texi +++ b/doc/lispref/intro.texi @@ -491,19 +491,6 @@ Version Info giving a prefix argument makes @var{here} non-@code{nil}. @end deffn -@defvar emacs-build-time -The value of this variable indicates the time at which Emacs was -built. It is a list of four integers, like the value of -@code{current-time} (@pxref{Time of Day}). - -@example -@group -emacs-build-time - @result{} (20614 63694 515336 438000) -@end group -@end example -@end defvar - @defvar emacs-version The value of this variable is the version of Emacs being run. It is a string such as @code{"23.1.1"}. The last number in this string is not diff --git a/etc/NEWS b/etc/NEWS index 20a1232..e0ddc79 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -25,6 +25,14 @@ otherwise leave it unmarked. \f * Installation Changes in Emacs 25.2 +** New configure option ‘--enable-deterministic-build’ attempts to +build an Emacs that is reproducible; that is, if you build and +install Emacs twice, the second installation is a copy of the first. +Deterministic builds omit the build date from the output of the +emacs-version and erc-cmd-SV functions, and the leave the following +variables nil: emacs-build-system, emacs-build-time, +erc-emacs-build-time. + ** 'configure' detects the kqueue file notification library on *BSD and Mac OS X machines. diff --git a/lisp/erc/erc-compat.el b/lisp/erc/erc-compat.el index d8af692..7c7edd1 100644 --- a/lisp/erc/erc-compat.el +++ b/lisp/erc/erc-compat.el @@ -54,10 +54,10 @@ erc-set-write-file-functions (set (make-local-variable 'write-file-functions) new-val)) (defvar erc-emacs-build-time - (if (stringp emacs-build-time) + (if (or (stringp emacs-build-time) (not emacs-build-time)) emacs-build-time (format-time-string "%Y-%m-%d" emacs-build-time)) - "Time at which Emacs was dumped out.") + "Time at which Emacs was dumped out, or nil if not available.") ;; Emacs 21 and XEmacs do not have user-emacs-directory, but XEmacs ;; has user-init-directory. @@ -164,4 +164,3 @@ erc-subseq ;; indent-tabs-mode: t ;; tab-width: 8 ;; End: - diff --git a/lisp/erc/erc.el b/lisp/erc/erc.el index cd8e427..3ff919a 100644 --- a/lisp/erc/erc.el +++ b/lisp/erc/erc.el @@ -3574,7 +3574,7 @@ gtk-version-string (defun erc-cmd-SV () "Say the current ERC and Emacs version into channel." - (erc-send-message (format "I'm using ERC with %s %s (%s%s) of %s." + (erc-send-message (format "I'm using ERC with %s %s (%s%s)%s." (if (featurep 'xemacs) "XEmacs" "GNU Emacs") emacs-version system-configuration @@ -3595,7 +3595,9 @@ erc-cmd-SV x-toolkit-scroll-bars))) "") (if (featurep 'multi-tty) ", multi-tty" "")) - erc-emacs-build-time)) + (if erc-emacs-build-time + (concat " of " erc-emacs-build-time) + ""))) t) (defun erc-cmd-SM () diff --git a/lisp/version.el b/lisp/version.el index a3f0ea0..a6686e1 100644 --- a/lisp/version.el +++ b/lisp/version.el @@ -38,14 +38,12 @@ emacs-minor-version "Minor version number of this version of Emacs. This variable first existed in version 19.23.") -;; FIXME: The next variable should also be a constant if -;; `deterministic-dump' is t. -(defconst emacs-build-time (current-time) - "Time at which Emacs was dumped out.") +(defconst emacs-build-time (if (not deterministic-build) (current-time)) + "Time at which Emacs was dumped out, or nil if not available.") (defconst emacs-build-system - (if deterministic-dump "elided" (system-name)) - "Name of the system on which Emacs was built.") + (if (not deterministic-build) (system-name)) + "Name of the system on which Emacs was built, or nil if not available.") (defvar motif-version-string) (defvar gtk-version-string) @@ -59,9 +57,7 @@ emacs-version to the system configuration; look at `system-configuration' instead." (interactive "P") (let ((version-string - (format (if (not (called-interactively-p 'interactive)) - "GNU Emacs %s (%s%s%s%s)\n of %s" - "GNU Emacs %s (%s%s%s%s) of %s") + (format "GNU Emacs %s (%s%s%s%s)%s" emacs-version system-configuration (cond ((featurep 'motif) @@ -80,7 +76,14 @@ emacs-version (format ", %s scroll bars" (capitalize (symbol-name x-toolkit-scroll-bars))) "") - (format-time-string "%Y-%m-%d" emacs-build-time)))) + (if emacs-build-time + (format-time-string (concat + (if (called-interactively-p + 'interactive) + "" "\n") + " of %Y-%m-%d") + emacs-build-time) + "")))) (if here (insert version-string) (if (called-interactively-p 'interactive) diff --git a/src/emacs.c b/src/emacs.c index 863473a..e7c153f 100644 --- a/src/emacs.c +++ b/src/emacs.c @@ -925,9 +925,6 @@ main (int argc, char **argv) SET_BINARY (fileno (stdout)); #endif /* MSDOS */ - if (DETERMINISTIC_DUMP) - Vdeterministic_dump = Qt; - /* Skip initial setlocale if LC_ALL is "C", as it's not needed in that case. The build procedure uses this while dumping, to ensure that the dumped Emacs does not have its system locale tables initialized, @@ -2569,12 +2566,12 @@ libraries; only those already known by Emacs will be loaded. */); Vdynamic_library_alist = Qnil; Fput (intern_c_string ("dynamic-library-alist"), Qrisky_local_variable, Qt); - DEFVAR_BOOL ("deterministic-dump", Vdeterministic_dump, - doc: /* If non-nil, attempt to make dumping deterministic by -avoiding sources of nondeterminism such as absolute file names, the -hostname, or timestamps. */); - Vdeterministic_dump = DETERMINISTIC_DUMP ? Qt : Qnil; - XSYMBOL (intern_c_string ("deterministic-dump"))->constant = 1; + DEFVAR_BOOL ("deterministic-build", Vdeterministic_build, + doc: /* If non-nil, make builds deterministic by omitting +host names, time stamps, etc. from the output. */); + if (DETERMINISTIC_BUILD) + Vdeterministic_build = true; + XSYMBOL (intern_c_string ("deterministic-build"))->constant = 1; #ifdef WINDOWSNT Vlibrary_cache = Qnil; diff --git a/src/sysdep.c b/src/sysdep.c index 5bbf723..8147afe 100644 --- a/src/sysdep.c +++ b/src/sysdep.c @@ -1399,11 +1399,10 @@ setup_pty (int fd) void init_system_name (void) { - if (DETERMINISTIC_DUMP && (might_dump || ! NILP (Vpurify_flag))) + if (DETERMINISTIC_BUILD && (might_dump || ! NILP (Vpurify_flag))) { - /* If we're dumping, set the hostname to a literal so that the - dump is deterministic. */ - Vsystem_name = build_pure_c_string ("elided"); + /* Set system-name to nil so that the build is deterministic. */ + Vsystem_name = Qnil; return; } char *hostname_alloc = NULL; -- 2.5.0 [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #4: combined.patch --] [-- Type: text/x-diff; name="combined.patch", Size: 8303 bytes --] diff --git a/configure.ac b/configure.ac index 0b7b403..9dce607 100644 --- a/configure.ac +++ b/configure.ac @@ -542,6 +542,18 @@ AC_ARG_ENABLE(gtk-deprecation-warnings, [Show Gtk+/Gdk deprecation warnings for Gtk+ >= 3.0])], [ac_enable_gtk_deprecation_warnings="${enableval}"],[]) +AC_ARG_ENABLE(deterministic-build, +[AS_HELP_STRING([--enable-deterministic-build], + [Make the build deterministic by omitting host names, + time stamps, etc. from the output.])]) +if test "x${enableval}" = xno ; then + AC_DEFINE(DETERMINISTIC_BUILD, false, + [Set this to true to make dumping deterministic.]) +else + AC_DEFINE(DETERMINISTIC_BUILD, true, + [Set this to true to make dumping deterministic.]) +fi + dnl This used to use changequote, but, apart from 'changequote is evil' dnl per the autoconf manual, we can speed up autoconf somewhat by quoting dnl the great gob of text. Thus it's not processed for possible expansion. diff --git a/doc/lispref/internals.texi b/doc/lispref/internals.texi index e620da0..1b7dcfa 100644 --- a/doc/lispref/internals.texi +++ b/doc/lispref/internals.texi @@ -64,6 +64,14 @@ Building Emacs and are not able to implement dumping, then Emacs must load @file{loadup.el} each time it starts. +@cindex deterministic build +@cindex @option{--enable-deterministic-build} option to @command{configure} + By default the dumped @file{emacs} executable contains details such +as the build time and host name. Use the +@option{--enable-determistic-build} option of @command{configure} to +suppress these details, so that building and installing Emacs twice +from the same sources should result in identical copies of Emacs. + @cindex @file{site-load.el} You can specify additional files to preload by writing a library named @file{site-load.el} that loads them. You may need to rebuild Emacs diff --git a/doc/lispref/intro.texi b/doc/lispref/intro.texi index 865c698..24a7131 100644 --- a/doc/lispref/intro.texi +++ b/doc/lispref/intro.texi @@ -491,19 +491,6 @@ Version Info giving a prefix argument makes @var{here} non-@code{nil}. @end deffn -@defvar emacs-build-time -The value of this variable indicates the time at which Emacs was -built. It is a list of four integers, like the value of -@code{current-time} (@pxref{Time of Day}). - -@example -@group -emacs-build-time - @result{} (20614 63694 515336 438000) -@end group -@end example -@end defvar - @defvar emacs-version The value of this variable is the version of Emacs being run. It is a string such as @code{"23.1.1"}. The last number in this string is not diff --git a/etc/NEWS b/etc/NEWS index 20a1232..e0ddc79 100644 --- a/etc/NEWS +++ b/etc/NEWS @@ -25,6 +25,14 @@ otherwise leave it unmarked. \f * Installation Changes in Emacs 25.2 +** New configure option ‘--enable-deterministic-build’ attempts to +build an Emacs that is reproducible; that is, if you build and +install Emacs twice, the second installation is a copy of the first. +Deterministic builds omit the build date from the output of the +emacs-version and erc-cmd-SV functions, and the leave the following +variables nil: emacs-build-system, emacs-build-time, +erc-emacs-build-time. + ** 'configure' detects the kqueue file notification library on *BSD and Mac OS X machines. diff --git a/lisp/erc/erc-compat.el b/lisp/erc/erc-compat.el index d8af692..7c7edd1 100644 --- a/lisp/erc/erc-compat.el +++ b/lisp/erc/erc-compat.el @@ -54,10 +54,10 @@ erc-set-write-file-functions (set (make-local-variable 'write-file-functions) new-val)) (defvar erc-emacs-build-time - (if (stringp emacs-build-time) + (if (or (stringp emacs-build-time) (not emacs-build-time)) emacs-build-time (format-time-string "%Y-%m-%d" emacs-build-time)) - "Time at which Emacs was dumped out.") + "Time at which Emacs was dumped out, or nil if not available.") ;; Emacs 21 and XEmacs do not have user-emacs-directory, but XEmacs ;; has user-init-directory. @@ -164,4 +164,3 @@ erc-subseq ;; indent-tabs-mode: t ;; tab-width: 8 ;; End: - diff --git a/lisp/erc/erc.el b/lisp/erc/erc.el index cd8e427..3ff919a 100644 --- a/lisp/erc/erc.el +++ b/lisp/erc/erc.el @@ -3574,7 +3574,7 @@ gtk-version-string (defun erc-cmd-SV () "Say the current ERC and Emacs version into channel." - (erc-send-message (format "I'm using ERC with %s %s (%s%s) of %s." + (erc-send-message (format "I'm using ERC with %s %s (%s%s)%s." (if (featurep 'xemacs) "XEmacs" "GNU Emacs") emacs-version system-configuration @@ -3595,7 +3595,9 @@ erc-cmd-SV x-toolkit-scroll-bars))) "") (if (featurep 'multi-tty) ", multi-tty" "")) - erc-emacs-build-time)) + (if erc-emacs-build-time + (concat " of " erc-emacs-build-time) + ""))) t) (defun erc-cmd-SM () diff --git a/lisp/version.el b/lisp/version.el index 43103fd..a6686e1 100644 --- a/lisp/version.el +++ b/lisp/version.el @@ -38,13 +38,12 @@ emacs-minor-version "Minor version number of this version of Emacs. This variable first existed in version 19.23.") -(defconst emacs-build-time (current-time) - "Time at which Emacs was dumped out.") +(defconst emacs-build-time (if (not deterministic-build) (current-time)) + "Time at which Emacs was dumped out, or nil if not available.") -;; I think this should be obsoleted/removed. It's just one more meaningless -;; difference between different builds. It's usually not even an fqdn. -(defconst emacs-build-system (system-name) - "Name of the system on which Emacs was built.") +(defconst emacs-build-system + (if (not deterministic-build) (system-name)) + "Name of the system on which Emacs was built, or nil if not available.") (defvar motif-version-string) (defvar gtk-version-string) @@ -58,9 +57,7 @@ emacs-version to the system configuration; look at `system-configuration' instead." (interactive "P") (let ((version-string - (format (if (not (called-interactively-p 'interactive)) - "GNU Emacs %s (%s%s%s%s)\n of %s" - "GNU Emacs %s (%s%s%s%s) of %s") + (format "GNU Emacs %s (%s%s%s%s)%s" emacs-version system-configuration (cond ((featurep 'motif) @@ -79,7 +76,14 @@ emacs-version (format ", %s scroll bars" (capitalize (symbol-name x-toolkit-scroll-bars))) "") - (format-time-string "%Y-%m-%d" emacs-build-time)))) + (if emacs-build-time + (format-time-string (concat + (if (called-interactively-p + 'interactive) + "" "\n") + " of %Y-%m-%d") + emacs-build-time) + "")))) (if here (insert version-string) (if (called-interactively-p 'interactive) diff --git a/src/emacs.c b/src/emacs.c index 2e9f950..e7c153f 100644 --- a/src/emacs.c +++ b/src/emacs.c @@ -2566,6 +2566,13 @@ libraries; only those already known by Emacs will be loaded. */); Vdynamic_library_alist = Qnil; Fput (intern_c_string ("dynamic-library-alist"), Qrisky_local_variable, Qt); + DEFVAR_BOOL ("deterministic-build", Vdeterministic_build, + doc: /* If non-nil, make builds deterministic by omitting +host names, time stamps, etc. from the output. */); + if (DETERMINISTIC_BUILD) + Vdeterministic_build = true; + XSYMBOL (intern_c_string ("deterministic-build"))->constant = 1; + #ifdef WINDOWSNT Vlibrary_cache = Qnil; staticpro (&Vlibrary_cache); diff --git a/src/sysdep.c b/src/sysdep.c index 1af323e..8147afe 100644 --- a/src/sysdep.c +++ b/src/sysdep.c @@ -1399,6 +1399,12 @@ setup_pty (int fd) void init_system_name (void) { + if (DETERMINISTIC_BUILD && (might_dump || ! NILP (Vpurify_flag))) + { + /* Set system-name to nil so that the build is deterministic. */ + Vsystem_name = Qnil; + return; + } char *hostname_alloc = NULL; char *hostname; #ifndef HAVE_GETHOSTNAME ^ permalink raw reply related [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-27 23:26 ` Paul Eggert @ 2015-12-28 16:26 ` Eli Zaretskii 2015-12-28 18:00 ` Paul Eggert 0 siblings, 1 reply; 53+ messages in thread From: Eli Zaretskii @ 2015-12-28 16:26 UTC (permalink / raw) To: Paul Eggert; +Cc: p.stephani2, emacs-devel > Cc: emacs-devel@gnu.org > From: Paul Eggert <eggert@cs.ucla.edu> > Date: Sun, 27 Dec 2015 15:26:28 -0800 > > * Document the changes in etc/NEWS and in the relevant part of the manual. > > * Use "deterministic-build", not "deterministic-dump", for the name. I didn't > understand the need for two variables here; we can add another one if needed. > > * Represent unknown values with nil, not with magic strings like "elided" or > "unknown". > > * Make emacs-build-time deterministic, too. > > * Adjust the output of emacs-version and similar functions to omit the build date. > > * Remove what appears to be an unnecessary initialization in 'main'. Fine with me in general, with the following comments: > --- a/doc/lispref/intro.texi > +++ b/doc/lispref/intro.texi > @@ -491,19 +491,6 @@ Version Info > giving a prefix argument makes @var{here} non-@code{nil}. > @end deffn > > -@defvar emacs-build-time > -The value of this variable indicates the time at which Emacs was > -built. It is a list of four integers, like the value of > -@code{current-time} (@pxref{Time of Day}). > - > -@example > -@group > -emacs-build-time > - @result{} (20614 63694 515336 438000) > -@end group > -@end example > -@end defvar Please let's nott remove this part, we are not removing the variable (and won't be any time soon). Let's just document that the value can be nil in a "deterministic build". > -;; I think this should be obsoleted/removed. It's just one more meaningless > -;; difference between different builds. It's usually not even an fqdn. > -(defconst emacs-build-system (system-name) > - "Name of the system on which Emacs was built.") > +(defconst emacs-build-system > + (if (not deterministic-build) (system-name)) > + "Name of the system on which Emacs was built, or nil if not available.") This causes an error in "M-x report-emacs-bug", so something should be done here or there. (I don't understand why you are so objected to keeping this a string, even though I specifically stated that I would like to keep it a string. Is it really worth another dispute?) Thanks. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-28 16:26 ` Eli Zaretskii @ 2015-12-28 18:00 ` Paul Eggert 2015-12-28 18:23 ` Eli Zaretskii 0 siblings, 1 reply; 53+ messages in thread From: Paul Eggert @ 2015-12-28 18:00 UTC (permalink / raw) To: Eli Zaretskii; +Cc: p.stephani2, emacs-devel Eli Zaretskii wrote: > Please let's nott remove this part, we are not removing the variable > (and won't be any time soon). Let's just document that the value can > be nil in a "deterministic build". OK, I will add the usual text saying that the variable can be nil if the information is not available. >> -;; I think this should be obsoleted/removed. It's just one more meaningless >> -;; difference between different builds. It's usually not even an fqdn. >> -(defconst emacs-build-system (system-name) >> - "Name of the system on which Emacs was built.") >> +(defconst emacs-build-system >> + (if (not deterministic-build) (system-name)) >> + "Name of the system on which Emacs was built, or nil if not available.") > > This causes an error in "M-x report-emacs-bug", so something should be > done here or there. What sort of error? It seems to work for me (not that I actually went all the way and sent the email). I had to back out the recent IDNA changes, as they broke report-emacs-bug -- perhaps that is what you noticed? > (I don't understand why you are so objected to > keeping this a string Consistency and simplicity. Consistency, because we should treat emacs-build-system consistently with other primitives like process-attributes and memory-info that return nil when the information is not available. Simplicity, because it's simpler for code to test for nil than for some reserved string like "unknown"; you can see examples of this in the most recent patch I proposed. Besides, we shouldn't prevent Emacs from working correctly on a machine whose host name really is "unknown". ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-28 18:00 ` Paul Eggert @ 2015-12-28 18:23 ` Eli Zaretskii 2015-12-29 3:01 ` Paul Eggert 0 siblings, 1 reply; 53+ messages in thread From: Eli Zaretskii @ 2015-12-28 18:23 UTC (permalink / raw) To: Paul Eggert; +Cc: p.stephani2, emacs-devel [-- Attachment #1: Type: text/plain, Size: 2708 bytes --] > Cc: p.stephani2@gmail.com, emacs-devel@gnu.org > From: Paul Eggert <eggert@cs.ucla.edu> > Date: Mon, 28 Dec 2015 10:00:22 -0800 > > Eli Zaretskii wrote: > > > Please let's nott remove this part, we are not removing the variable > > (and won't be any time soon). Let's just document that the value can > > be nil in a "deterministic build". > > OK, I will add the usual text saying that the variable can be nil if the > information is not available. Thanks. > >> -;; I think this should be obsoleted/removed. It's just one more meaningless > >> -;; difference between different builds. It's usually not even an fqdn. > >> -(defconst emacs-build-system (system-name) > >> - "Name of the system on which Emacs was built.") > >> +(defconst emacs-build-system > >> + (if (not deterministic-build) (system-name)) > >> + "Name of the system on which Emacs was built, or nil if not available.") > > > > This causes an error in "M-x report-emacs-bug", so something should be > > done here or there. > > What sort of error? "Wrong type argument: char-or-string-p, nil". See the attached image, and note that the cursor stops where it should have displayed the build system name, and there's nothing in the buffer beyond that point. > It seems to work for me (not that I actually went all the way and > sent the email). No need to send, just modify version.el to give emacs-build-system the nil value, rebuild Emacs, and type "M-x report-emacs-bug RET foo RET". > I had to back out the recent IDNA changes, as they broke report-emacs-bug -- > perhaps that is what you noticed? No, I don't think so. It's the 'insert' call in emacsbug.el that errors out, since it cannot handle a nil argument. > > (I don't understand why you are so objected to > > keeping this a string > > Consistency and simplicity. Consistency, because we should treat > emacs-build-system consistently with other primitives like process-attributes > and memory-info that return nil when the information is not available. > Simplicity, because it's simpler for code to test for nil than for some reserved > string like "unknown"; you can see examples of this in the most recent patch I > proposed. Besides, we shouldn't prevent Emacs from working correctly on a > machine whose host name really is "unknown". That's why I asked it not to be "unknown". But "elided" is different: it tells that we deliberately conceal the build system. I'm okay with other, better, strings, if they explain the situation better. My point is that the reproducible build is not (yet) the standard build, it's more like an anomaly at this time, so I thought we should have the string explain itself. And nil cannot explain anything. [-- Attachment #2: emacs-bug-error.PNG --] [-- Type: image/png, Size: 35220 bytes --] ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-28 18:23 ` Eli Zaretskii @ 2015-12-29 3:01 ` Paul Eggert 2015-12-29 15:38 ` Eli Zaretskii 0 siblings, 1 reply; 53+ messages in thread From: Paul Eggert @ 2015-12-29 3:01 UTC (permalink / raw) To: Eli Zaretskii; +Cc: p.stephani2, emacs-devel [-- Attachment #1: Type: text/plain, Size: 1826 bytes --] Eli Zaretskii wrote: >> >What sort of error? > "Wrong type argument: char-or-string-p, nil". See the attached image, Although you applied the patch to emacs-25, the patch was intended for master. emacs-25 and master disagree here because of emacs-25 commit 2dd3581b78a5fe15e0562b1cc2daa6e3afeeec4a dated December 25 "Restore info about the build host in bug reports". This explains why I did not observe the problem on master, whereas you observed it on emacs-25. I just now installed into emacs-25 the attached, which should future-proof the Dec. 25 change whenever it happens to be merged into master. > the reproducible build is not (yet) the standard > build, it's more like an anomaly at this time, so I thought we should > have the string explain itself. And nil cannot explain anything. That is what documentation is for, no? :-) Having now had some experience with using the deterministic-build API in code, I far prefer using (if FOO ...) to something like (if (string-equal FOO "(elided due to deterministic builds)") ...). Not only would the the latter be longer and harder to read, it would mean I need to go to the documentation and see how the magic-cookie string is spelled and would be likely to make a typo there, which means we would need to give the magic cookie a name like ‘deterministic-build-system-name’ to catch typos so that the code will instead be (if (string-equal FOO deterministic-build-system-name) ...), and then we would need to give similar names to the magic cookies used for other things like emacs-build-time that are elided. This would all be complete overkill. It is much simpler to use nil to represent missing information; this is easier to remember and read and document and it fits squarely within the Emacs Lisp tradition. [-- Attachment #2: 0001-Port-report-emacs-bug-to-deterministic-builds.patch --] [-- Type: text/x-diff, Size: 1198 bytes --] From 691092c904cc293d84e16f2239f51cbb8ff1cbcd Mon Sep 17 00:00:00 2001 From: Paul Eggert <eggert@cs.ucla.edu> Date: Mon, 28 Dec 2015 18:43:09 -0800 Subject: [PATCH] Port report-emacs-bug to deterministic builds * lisp/mail/emacsbug.el (report-emacs-bug): Future-proof the recent "built on" change to deterministic builds where emacs-build-system will be nil. See: http://lists.gnu.org/archive/html/emacs-devel/2015-12/msg01369.html --- lisp/mail/emacsbug.el | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/lisp/mail/emacsbug.el b/lisp/mail/emacsbug.el index ae0e711..ef5e86a 100644 --- a/lisp/mail/emacsbug.el +++ b/lisp/mail/emacsbug.el @@ -242,7 +242,11 @@ report-emacs-bug (let ((txt (delete-and-extract-region (1+ user-point) (point)))) (insert (propertize "\n" 'display txt))) - (insert "\n\nIn " (emacs-version) " built on " emacs-build-system "\n") + (insert "\n\nIn " (emacs-version)) + (if emacs-build-system + (insert " built on " emacs-build-system)) + (insert "\n") + (if (stringp emacs-repository-version) (insert "Repository revision: " emacs-repository-version "\n")) (if (fboundp 'x-server-vendor) -- 2.5.0 ^ permalink raw reply related [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-29 3:01 ` Paul Eggert @ 2015-12-29 15:38 ` Eli Zaretskii 2015-12-29 16:47 ` Paul Eggert 0 siblings, 1 reply; 53+ messages in thread From: Eli Zaretskii @ 2015-12-29 15:38 UTC (permalink / raw) To: Paul Eggert; +Cc: p.stephani2, emacs-devel > Cc: p.stephani2@gmail.com, emacs-devel@gnu.org > From: Paul Eggert <eggert@cs.ucla.edu> > Date: Mon, 28 Dec 2015 19:01:37 -0800 > > I just now installed into emacs-25 the attached, which should future-proof the > Dec. 25 change whenever it happens to be merged into master. Thanks. I still wonder whether this is worth the hassle that will come our way. > > the reproducible build is not (yet) the standard > > build, it's more like an anomaly at this time, so I thought we should > > have the string explain itself. And nil cannot explain anything. > > That is what documentation is for, no? :-) Documentation is for details and for when the code and the built-in doc strings fail to explain themselves. It sounds strange to send users to documentation for such a simple issue. > Having now had some experience with using the deterministic-build API in code, I > far prefer using (if FOO ...) to something like (if (string-equal FOO "(elided > due to deterministic builds)") ...). No, the idea is that with the string you don't need any if's at all. The string speaks for itself when you display it. > It is much simpler to use nil to represent missing information It sounds strange that Emacs should "miss the information" about the system it was built on; that could well sound like a bug to some. Keeping it a string that explains itself was supposed to remove that difficulty. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-29 15:38 ` Eli Zaretskii @ 2015-12-29 16:47 ` Paul Eggert 2015-12-29 17:59 ` Eli Zaretskii 0 siblings, 1 reply; 53+ messages in thread From: Paul Eggert @ 2015-12-29 16:47 UTC (permalink / raw) To: Eli Zaretskii; +Cc: p.stephani2, emacs-devel Eli Zaretskii wrote: > The string speaks for itself when you display it. It does not. A host name could be ‘unknown’ or even ‘elided’. In theory it could be any string. > It sounds strange that Emacs should "miss the information" about the > system it was built on; that could well sound like a bug to some. It is a feature, not a bug. That is, it is a desirable feature of deterministic builds. It is not strange; on the contrary, it is the behavior I expect from other programs that I use. When I run (say) Firefox to browse the web, I do not want Firefox to tell me the host name of the system that it was built on. If Firefox did tell me that, it would be a sign of sloppy engineering on the builder’s part. Any reasonably-competent builder should be able to reproduce a build even if the particular host that built it no longer exists. This is basic software construction. In this sense, the Dec. 25 change that introduced the system build name into the output of report-emacs-bug was a misstep. I am glad that there will eventually be a build-time option to disable this sort of thing. It is an option I plan to use. > It sounds strange to send users to documentation for such a simple issue. No matter what we do here, we will need documentation that users can refer to on occasion. Even simple approaches like nil-if-unknown require some documentation. More-complicated approaches like magic-cookie-if-unknown would require more of it. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-29 16:47 ` Paul Eggert @ 2015-12-29 17:59 ` Eli Zaretskii 2015-12-29 20:33 ` Paul Eggert 0 siblings, 1 reply; 53+ messages in thread From: Eli Zaretskii @ 2015-12-29 17:59 UTC (permalink / raw) To: Paul Eggert; +Cc: p.stephani2, emacs-devel > Cc: p.stephani2@gmail.com, emacs-devel@gnu.org > From: Paul Eggert <eggert@cs.ucla.edu> > Date: Tue, 29 Dec 2015 08:47:09 -0800 > > Eli Zaretskii wrote: > > The string speaks for itself when you display it. > > It does not. A host name could be ‘unknown’ or even ‘elided’. In theory it could > be any string. We could come up with something even less probable. > In this sense, the Dec. 25 change that introduced the system build name into the > output of report-emacs-bug was a misstep. No, it isn't. I needed this information many times when examining bug reports, and I'm not going to give it up. Can I have some minimal respect from fellow developers, and some minimal credit that I do things for a reason? > > It sounds strange to send users to documentation for such a simple issue. > > No matter what we do here, we will need documentation that users can refer to on > occasion. Even simple approaches like nil-if-unknown require some documentation. > More-complicated approaches like magic-cookie-if-unknown would require more of it. You are ducking the issue. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-29 17:59 ` Eli Zaretskii @ 2015-12-29 20:33 ` Paul Eggert 0 siblings, 0 replies; 53+ messages in thread From: Paul Eggert @ 2015-12-29 20:33 UTC (permalink / raw) To: Eli Zaretskii; +Cc: p.stephani2, emacs-devel Eli Zaretskii wrote: > We could come up with something even less probable. We could, yes. It should not be a valid domain name, for example. But whatever magic cookie we came up with, we would need to document it so that user code can avoid displaying it. And to avoid typos it would be better to make it a named constant. And we would need to do something similar, with different named constants, for similar issues such as emacs-build-time. And we would never eliminate the possibility that someone (if only to be contrary, e.g., because they are attacking Emacs) would use the magic-cookie as a host name. All in all, it would be a relatively complicated solution to what should be a simple problem. >> >In this sense, the Dec. 25 change that introduced the system build name into the >> >output of report-emacs-bug was a misstep. > No, it isn't. I needed this information many times when examining bug > reports, and I'm not going to give it up. My comment’s “in the sense” referred to the sense of its previous paragraph (not quoted above) which talked about deterministic builds; it did not refer to building Emacs in general. In some cases it can be useful to have the build hostname embedded in the executable even at the cost of deterministic builds. However, many people (including myself) prefer deterministic builds, and for them embedding the build hostname is counterproductive, and that is why it is useful to have an option to suppress it. > Can I have some minimal > respect from fellow developers, and some minimal credit that I do > things for a reason? Nothing in my commentary was intended to exhibit any disrespect. It was intended to be purely technical commentary. If it seemed otherwise, I apologize. >>> > >It sounds strange to send users to documentation for such a simple issue. >> > >> >No matter what we do here, we will need documentation that users can refer to on >> >occasion. Even simple approaches like nil-if-unknown require some documentation. >> >More-complicated approaches like magic-cookie-if-unknown would require more of it. > You are ducking the issue. I do not see what I am ducking here. Surely you are not advocating leaving the behavior undocumented. Are you objecting to the convention of using nil to represent lack of information? If so, how about if we use some other symbol, such as ‘unknown’? Although that would be a bit more of a pain to document and use, it would also be OK and for the reasons discussed above would be better than a magic-cookie string. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-12-07 18:36 ` Philipp Stephani 2015-12-07 18:58 ` Paul Eggert @ 2015-12-07 23:11 ` Ludovic Courtès 1 sibling, 0 replies; 53+ messages in thread From: Ludovic Courtès @ 2015-12-07 23:11 UTC (permalink / raw) To: Philipp Stephani; +Cc: Eli Zaretskii, alezost, emacs-devel Philipp Stephani <p.stephani2@gmail.com> skribis: > I'm interested in reproducible builds as well; do you already have some > design ideas how to tackle this problem? So far I've found three areas > where Emacs itself builds something that could be reproducible (dumping, > generating autoload files, byte compilation), and a couple of data items > that cause irreproducibility (host names, timestamps, absolute paths). There are two things at stake here: making Emacs builds reproducible, and making Emacs produce deterministic output. If autoload generation is non-deterministic, then every elisp package ends up being non-deterministic. So fixing it is a net win, and that’s the reason why we started with this one. The other issues affect only Emacs itself. Timestamps and the likes are easily addressed I guess. Unexec is likely another story, though. Ludo’. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-11-29 16:57 ` Ludovic Courtès 2015-11-29 18:12 ` Eli Zaretskii @ 2015-11-30 9:22 ` Alex Kost 1 sibling, 0 replies; 53+ messages in thread From: Alex Kost @ 2015-11-30 9:22 UTC (permalink / raw) To: Ludovic Courtès; +Cc: Eli Zaretskii, emacs-devel Ludovic Courtès (2015-11-29 19:57 +0300) wrote: > Eli Zaretskii <eliz@gnu.org> skribis: > >>> From: ludo@gnu.org (Ludovic Courtès) >>> Date: Sun, 29 Nov 2015 11:44:36 +0100 >>> >>> https://lists.gnu.org/archive/html/emacs-devel/2015-11/msg01547.html >>> >>> Support bit-for-bit reproducible generation of autoloads. >>> See <https://reproducible-builds.org/specs/source-date-epoch/>. >>> Submitted on behalf of Alex Kost. >>> >>> * lisp/emacs-lisp/autoload.el (autoload-insert-section-header): Check >>> whether the 'SOURCE_DATE_EPOCH' environment variable is defined; use it >>> as the TIME part of the 'autoloads' sexp when it is. >> >> Thanks, but I think we need to discuss this first. >> >> First, if this is Alex's code, please show his full name and email >> address, as they will have to be mentioned in the commit message, and >> we need to know the status of his legal paperwork, or the lack >> thereof. > > First, Alex (Cc’d) asked me explicitly to do it on his behalf because he > didn’t want to have to deal with this (IMO unfriendly) feedback > personally. Oh, no, no! I had nothing against emacs-devel people, I just feel very uncomfortable communicating with people in general. Sorry for misunderstanding. -- Alex ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-11-29 10:44 ` [PATCH PING] " Ludovic Courtès 2015-11-29 16:02 ` Eli Zaretskii @ 2015-11-29 20:22 ` Paul Eggert 2015-11-29 20:42 ` Ludovic Courtès 1 sibling, 1 reply; 53+ messages in thread From: Paul Eggert @ 2015-11-29 20:22 UTC (permalink / raw) To: Ludovic Courtès, emacs-devel Ludovic Courtès wrote: > * lisp/emacs-lisp/autoload.el (autoload-insert-section-header): Check > whether the 'SOURCE_DATE_EPOCH' environment variable is defined; use it > as the TIME part of the 'autoloads' sexp when it is. Thanks, but this patch is heading down the wrong direction. When we generate an autoloads list, we insert a checksum if we don't trust the date. We should do that here, too. There should be no need for a SOURCE_DATE_EPOCH environment variable, for Emacs. Sorry if this process seems a bit unfriendly, by the way. Reviewing is often a pain. ^ permalink raw reply [flat|nested] 53+ messages in thread
* Re: [PATCH PING] Honor 'SOURCE_DATE_EPOCH' when generating autoloads. 2015-11-29 20:22 ` Paul Eggert @ 2015-11-29 20:42 ` Ludovic Courtès 0 siblings, 0 replies; 53+ messages in thread From: Ludovic Courtès @ 2015-11-29 20:42 UTC (permalink / raw) To: Paul Eggert; +Cc: emacs-devel Paul Eggert <eggert@cs.ucla.edu> skribis: > Ludovic Courtès wrote: > >> * lisp/emacs-lisp/autoload.el (autoload-insert-section-header): Check >> whether the 'SOURCE_DATE_EPOCH' environment variable is defined; use it >> as the TIME part of the 'autoloads' sexp when it is. > > Thanks, but this patch is heading down the wrong direction. When we > generate an autoloads list, we insert a checksum if we don't trust the > date. We should do that here, too. Sure, that sounds even better. Ludo’. ^ permalink raw reply [flat|nested] 53+ messages in thread
end of thread, other threads:[~2016-04-06 21:29 UTC | newest] Thread overview: 53+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-11-16 17:03 [PATCH] Honor 'SOURCE_DATE_EPOCH' when generating autoloads Ludovic Courtès 2015-11-29 10:44 ` [PATCH PING] " Ludovic Courtès 2015-11-29 16:02 ` Eli Zaretskii 2015-11-29 16:57 ` Ludovic Courtès 2015-11-29 18:12 ` Eli Zaretskii 2015-11-29 20:38 ` Ludovic Courtès 2015-11-30 17:00 ` John Wiegley 2015-11-30 19:30 ` Ludovic Courtès 2015-12-07 18:36 ` Philipp Stephani 2015-12-07 18:58 ` Paul Eggert 2015-12-07 19:00 ` Philipp Stephani 2015-12-07 19:14 ` Paul Eggert 2015-12-20 12:48 ` Philipp Stephani 2015-12-20 16:37 ` Eli Zaretskii 2015-12-20 18:39 ` Philipp Stephani 2015-12-20 19:03 ` Eli Zaretskii 2015-12-20 21:27 ` Paul Eggert 2015-12-23 1:13 ` John Wiegley 2015-12-23 16:01 ` Philipp Stephani 2015-12-23 16:39 ` Eli Zaretskii 2015-12-23 16:52 ` Philipp Stephani 2015-12-23 17:10 ` Eli Zaretskii 2015-12-23 17:38 ` Philipp Stephani 2015-12-23 21:09 ` Paul Eggert 2015-12-24 3:33 ` Eli Zaretskii 2015-12-24 6:54 ` Paul Eggert 2015-12-24 16:18 ` Eli Zaretskii 2015-12-27 9:53 ` Philipp Stephani 2015-12-27 16:10 ` Eli Zaretskii 2016-02-29 21:52 ` Philipp Stephani 2016-03-01 16:36 ` Paul Eggert 2016-03-01 21:46 ` Philipp Stephani 2016-03-02 18:24 ` Paul Eggert 2016-03-02 18:30 ` Philipp Stephani 2016-03-22 13:30 ` Philipp Stephani 2016-03-22 20:37 ` Paul Eggert 2016-03-22 21:46 ` Philipp Stephani 2016-03-22 21:58 ` Paul Eggert 2016-03-23 8:03 ` Andreas Schwab 2016-04-06 21:29 ` Philipp Stephani 2015-12-27 23:26 ` Paul Eggert 2015-12-28 16:26 ` Eli Zaretskii 2015-12-28 18:00 ` Paul Eggert 2015-12-28 18:23 ` Eli Zaretskii 2015-12-29 3:01 ` Paul Eggert 2015-12-29 15:38 ` Eli Zaretskii 2015-12-29 16:47 ` Paul Eggert 2015-12-29 17:59 ` Eli Zaretskii 2015-12-29 20:33 ` Paul Eggert 2015-12-07 23:11 ` Ludovic Courtès 2015-11-30 9:22 ` Alex Kost 2015-11-29 20:22 ` Paul Eggert 2015-11-29 20:42 ` Ludovic Courtès
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.