* Size of emacs.pdmp changes between successive installs @ 2019-11-19 16:59 Ulrich Mueller 2019-11-19 17:18 ` Paul Eggert ` (2 more replies) 0 siblings, 3 replies; 15+ messages in thread From: Ulrich Mueller @ 2019-11-19 16:59 UTC (permalink / raw) To: emacs-devel For two successive installs from a clean checkout of the same git commit, emacs.pdmp ends up having slightly different size: $ stat -c %s /usr/libexec/emacs/27.0.50/x86_64-pc-linux-gnu/emacs.pdmp 10563744 $ stat -c %s /usr/libexec/emacs/27.0.50/x86_64-pc-linux-gnu/emacs.pdmp 10563728 Why is that so? Shouldn't it be deterministic? (I've tried option --disable-build-details and disabling parallel make, but neither of them makes a difference.) ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Size of emacs.pdmp changes between successive installs 2019-11-19 16:59 Size of emacs.pdmp changes between successive installs Ulrich Mueller @ 2019-11-19 17:18 ` Paul Eggert 2019-11-19 17:45 ` Eli Zaretskii 2019-11-19 18:16 ` Andreas Schwab 2 siblings, 0 replies; 15+ messages in thread From: Paul Eggert @ 2019-11-19 17:18 UTC (permalink / raw) To: Ulrich Mueller; +Cc: emacs-devel On 11/19/19 8:59 AM, Ulrich Mueller wrote: > Why is that so? Shouldn't it be deterministic? I would think so, yes. What differs about the two pdump files and why? (I realize this might not be an easy question to answer....) ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Size of emacs.pdmp changes between successive installs 2019-11-19 16:59 Size of emacs.pdmp changes between successive installs Ulrich Mueller 2019-11-19 17:18 ` Paul Eggert @ 2019-11-19 17:45 ` Eli Zaretskii 2019-11-19 18:19 ` Andreas Schwab 2019-11-19 18:16 ` Andreas Schwab 2 siblings, 1 reply; 15+ messages in thread From: Eli Zaretskii @ 2019-11-19 17:45 UTC (permalink / raw) To: Ulrich Mueller; +Cc: emacs-devel > From: Ulrich Mueller <ulm@gentoo.org> > Date: Tue, 19 Nov 2019 17:59:58 +0100 > > For two successive installs from a clean checkout of the same git > commit, emacs.pdmp ends up having slightly different size: > > $ stat -c %s /usr/libexec/emacs/27.0.50/x86_64-pc-linux-gnu/emacs.pdmp > 10563744 > > $ stat -c %s /usr/libexec/emacs/27.0.50/x86_64-pc-linux-gnu/emacs.pdmp > 10563728 > > Why is that so? Shouldn't it be deterministic? Please describe step by step what you did to produce each of the installs. The above is not detailed enough to understand what could be the cause of the differences. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Size of emacs.pdmp changes between successive installs 2019-11-19 17:45 ` Eli Zaretskii @ 2019-11-19 18:19 ` Andreas Schwab 0 siblings, 0 replies; 15+ messages in thread From: Andreas Schwab @ 2019-11-19 18:19 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Ulrich Mueller, emacs-devel On Nov 19 2019, Eli Zaretskii wrote: > Please describe step by step what you did to produce each of the > installs. The above is not detailed enough to understand what could > be the cause of the differences. $ make; mv src/emacs.pdump{,.alt}; make $ cmp -cl src/emacs.pdmp{,.alt} Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Size of emacs.pdmp changes between successive installs 2019-11-19 16:59 Size of emacs.pdmp changes between successive installs Ulrich Mueller 2019-11-19 17:18 ` Paul Eggert 2019-11-19 17:45 ` Eli Zaretskii @ 2019-11-19 18:16 ` Andreas Schwab 2019-11-19 20:56 ` Paul Eggert 2019-11-19 23:21 ` Ulrich Mueller 2 siblings, 2 replies; 15+ messages in thread From: Andreas Schwab @ 2019-11-19 18:16 UTC (permalink / raw) To: Ulrich Mueller; +Cc: emacs-devel On Nov 19 2019, Ulrich Mueller wrote: > For two successive installs from a clean checkout of the same git > commit, emacs.pdmp ends up having slightly different size: > > $ stat -c %s /usr/libexec/emacs/27.0.50/x86_64-pc-linux-gnu/emacs.pdmp > 10563744 > > $ stat -c %s /usr/libexec/emacs/27.0.50/x86_64-pc-linux-gnu/emacs.pdmp > 10563728 > > Why is that so? Shouldn't it be deterministic? Try disabling ASLR. That doesn't make it completely reproducible, though. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Size of emacs.pdmp changes between successive installs 2019-11-19 18:16 ` Andreas Schwab @ 2019-11-19 20:56 ` Paul Eggert 2019-11-19 22:14 ` Andreas Schwab 2019-11-19 23:21 ` Ulrich Mueller 1 sibling, 1 reply; 15+ messages in thread From: Paul Eggert @ 2019-11-19 20:56 UTC (permalink / raw) To: Andreas Schwab, Ulrich Mueller; +Cc: emacs-devel On 11/19/19 10:16 AM, Andreas Schwab wrote: > Try disabling ASLR. Emacs already tries to disable ASLR when dumping. Perhaps its attempt isn't working. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Size of emacs.pdmp changes between successive installs 2019-11-19 20:56 ` Paul Eggert @ 2019-11-19 22:14 ` Andreas Schwab 2019-11-19 23:45 ` Ulrich Mueller 0 siblings, 1 reply; 15+ messages in thread From: Andreas Schwab @ 2019-11-19 22:14 UTC (permalink / raw) To: Paul Eggert; +Cc: Ulrich Mueller, emacs-devel On Nov 19 2019, Paul Eggert wrote: > Emacs already tries to disable ASLR when dumping. Only when unexecing. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Size of emacs.pdmp changes between successive installs 2019-11-19 22:14 ` Andreas Schwab @ 2019-11-19 23:45 ` Ulrich Mueller 2019-11-20 2:27 ` Paul Eggert 2019-11-20 9:29 ` Andreas Schwab 0 siblings, 2 replies; 15+ messages in thread From: Ulrich Mueller @ 2019-11-19 23:45 UTC (permalink / raw) To: Andreas Schwab; +Cc: Paul Eggert, emacs-devel >>>>> On Tue, 19 Nov 2019, Andreas Schwab wrote: > On Nov 19 2019, Paul Eggert wrote: >> Emacs already tries to disable ASLR when dumping. > Only when unexecing. Right, with this trivial patch I'm getting a reproducible size of emacs.pdmp, too: --- src/emacs.c~ 2019-11-20 00:27:55.016051255 +0100 +++ src/emacs.c 2019-11-20 00:37:56.740331939 +0100 @@ -1055,7 +1055,7 @@ #endif argc = maybe_disable_address_randomization ( - will_dump_with_unexec_p (), argc, argv); + will_dump_p (), argc, argv); #if defined GNU_LINUX && defined HAVE_UNEXEC if (!initialized) ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Size of emacs.pdmp changes between successive installs 2019-11-19 23:45 ` Ulrich Mueller @ 2019-11-20 2:27 ` Paul Eggert 2019-11-20 11:55 ` Andreas Schwab 2019-11-20 9:29 ` Andreas Schwab 1 sibling, 1 reply; 15+ messages in thread From: Paul Eggert @ 2019-11-20 2:27 UTC (permalink / raw) To: Ulrich Mueller, Andreas Schwab; +Cc: emacs-devel [-- Attachment #1: Type: text/plain, Size: 227 bytes --] >> Only when unexecing. > Right, with this trivial patch I'm getting a reproducible size > of emacs.pdmp, too: Thanks to both of you. I installed the attached, which does something similar and simplifies the code a bit too. [-- Attachment #2: 0001-Make-.pdmp-file-more-reproducible.patch --] [-- Type: text/x-patch, Size: 2873 bytes --] From 9ac586928bbef16b572bcb34bfcbcb5515c2335a Mon Sep 17 00:00:00 2001 From: Paul Eggert <eggert@cs.ucla.edu> Date: Tue, 19 Nov 2019 18:23:01 -0800 Subject: [PATCH] Make .pdmp file more reproducible MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Problem reported by Ulrich Müller <https://lists.gnu.org/r/emacs-devel/2019-11/msg00757.html> and diagnosed by Andreas Schwab <https://lists.gnu.org/r/emacs-devel/2019-11/msg00774.html>. * src/sysdep.c (maybe_disable_address_randomization): Disable ASLR if any kind of dumping, instead of merely if unexec dumping. Omit first arg for simplicity; all uses changed. --- src/emacs.c | 3 +-- src/lisp.h | 4 ++-- src/sysdep.c | 7 +++++-- 3 files changed, 8 insertions(+), 6 deletions(-) diff --git a/src/emacs.c b/src/emacs.c index 21a05d337e..8a6e34deda 100644 --- a/src/emacs.c +++ b/src/emacs.c @@ -1054,8 +1054,7 @@ main (int argc, char **argv) load_pdump (argc, argv); #endif - argc = maybe_disable_address_randomization ( - will_dump_with_unexec_p (), argc, argv); + argc = maybe_disable_address_randomization (argc, argv); #if defined GNU_LINUX && defined HAVE_UNEXEC if (!initialized) diff --git a/src/lisp.h b/src/lisp.h index 1d25add928..e0ae2c4262 100644 --- a/src/lisp.h +++ b/src/lisp.h @@ -4576,10 +4576,10 @@ #define DAEMON_RUNNING (w32_daemon_event != INVALID_HANDLE_VALUE) /* Defined in sysdep.c. */ #ifdef HAVE_PERSONALITY_ADDR_NO_RANDOMIZE -extern int maybe_disable_address_randomization (bool, int, char **); +extern int maybe_disable_address_randomization (int, char **); #else INLINE int -maybe_disable_address_randomization (bool dumping, int argc, char **argv) +maybe_disable_address_randomization (int argc, char **argv) { return argc; } diff --git a/src/sysdep.c b/src/sysdep.c index aa18ee22fd..e34ab2eb58 100644 --- a/src/sysdep.c +++ b/src/sysdep.c @@ -158,14 +158,17 @@ #define ULLONG_MAX TYPE_MAXIMUM (unsigned long long int) /* Try to disable randomization if the current process needs it and does not appear to have it already. */ int -maybe_disable_address_randomization (bool dumping, int argc, char **argv) +maybe_disable_address_randomization (int argc, char **argv) { /* Undocumented Emacs option used only by this function. */ static char const aslr_disabled_option[] = "--__aslr-disabled"; if (argc < 2 || strcmp (argv[1], aslr_disabled_option) != 0) { - bool disable_aslr = dumping; + /* If dumping via unexec, ASLR must be disabled, as otherwise + data may be scattered and undumpable as a simple executable. + If pdumping, disabling ASLR makes the .pdmp file reproducible. */ + bool disable_aslr = will_dump_p (); # ifdef __PPC64__ disable_aslr = true; # endif -- 2.23.0 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: Size of emacs.pdmp changes between successive installs 2019-11-20 2:27 ` Paul Eggert @ 2019-11-20 11:55 ` Andreas Schwab 2019-11-20 19:54 ` Paul Eggert 0 siblings, 1 reply; 15+ messages in thread From: Andreas Schwab @ 2019-11-20 11:55 UTC (permalink / raw) To: Paul Eggert; +Cc: Ulrich Mueller, emacs-devel On Nov 19 2019, Paul Eggert wrote: > + If pdumping, disabling ASLR makes the .pdmp file reproducible. */ That's not true. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Size of emacs.pdmp changes between successive installs 2019-11-20 11:55 ` Andreas Schwab @ 2019-11-20 19:54 ` Paul Eggert 2019-11-20 19:59 ` Paul Eggert 0 siblings, 1 reply; 15+ messages in thread From: Paul Eggert @ 2019-11-20 19:54 UTC (permalink / raw) To: Andreas Schwab; +Cc: Ulrich Mueller, emacs-devel On 11/20/19 3:55 AM, Andreas Schwab wrote: >> + If pdumping, disabling ASLR makes the .pdmp file reproducible. */ > That's not true. Suggested rewording? ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Size of emacs.pdmp changes between successive installs 2019-11-20 19:54 ` Paul Eggert @ 2019-11-20 19:59 ` Paul Eggert 0 siblings, 0 replies; 15+ messages in thread From: Paul Eggert @ 2019-11-20 19:59 UTC (permalink / raw) To: Andreas Schwab; +Cc: Ulrich Mueller, emacs-devel [-- Attachment #1: Type: text/plain, Size: 103 bytes --] On 11/20/19 11:54 AM, Paul Eggert wrote: > Suggested rewording? Never mind, I installed the attached. [-- Attachment #2: 0001-src-sysdep.c-Improve-comment-wording.patch --] [-- Type: text/x-patch, Size: 861 bytes --] From 9ca40c4e701e925eaf4b580b20a261737d578706 Mon Sep 17 00:00:00 2001 From: Paul Eggert <eggert@cs.ucla.edu> Date: Wed, 20 Nov 2019 11:57:17 -0800 Subject: [PATCH] * src/sysdep.c: Improve comment wording. --- src/sysdep.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/sysdep.c b/src/sysdep.c index e34ab2eb58..b436bfe74a 100644 --- a/src/sysdep.c +++ b/src/sysdep.c @@ -167,7 +167,7 @@ maybe_disable_address_randomization (int argc, char **argv) { /* If dumping via unexec, ASLR must be disabled, as otherwise data may be scattered and undumpable as a simple executable. - If pdumping, disabling ASLR makes the .pdmp file reproducible. */ + If pdumping, disabling ASLR lessens differences in the .pdmp file. */ bool disable_aslr = will_dump_p (); # ifdef __PPC64__ disable_aslr = true; -- 2.23.0 ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: Size of emacs.pdmp changes between successive installs 2019-11-19 23:45 ` Ulrich Mueller 2019-11-20 2:27 ` Paul Eggert @ 2019-11-20 9:29 ` Andreas Schwab 2019-11-20 11:42 ` Ulrich Mueller 1 sibling, 1 reply; 15+ messages in thread From: Andreas Schwab @ 2019-11-20 9:29 UTC (permalink / raw) To: Ulrich Mueller; +Cc: Paul Eggert, emacs-devel On Nov 20 2019, Ulrich Mueller wrote: > Right, with this trivial patch I'm getting a reproducible size > of emacs.pdmp, too: Do you get an identical pdmp, though? Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1 "And now for something completely different." ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Size of emacs.pdmp changes between successive installs 2019-11-20 9:29 ` Andreas Schwab @ 2019-11-20 11:42 ` Ulrich Mueller 0 siblings, 0 replies; 15+ messages in thread From: Ulrich Mueller @ 2019-11-20 11:42 UTC (permalink / raw) To: Andreas Schwab; +Cc: Paul Eggert, emacs-devel >>>>> On Wed, 20 Nov 2019, Andreas Schwab wrote: > On Nov 20 2019, Ulrich Mueller wrote: >> Right, with this trivial patch I'm getting a reproducible size >> of emacs.pdmp, too: > Do you get an identical pdmp, though? No, cmp shows between 10 and 30 different bytes between successive .pdmp files. (I have removed src/emacs, src/emacs.pdmp, and src/emacs-27.* before calling make again, so it is not due to an increasing build number.) ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Size of emacs.pdmp changes between successive installs 2019-11-19 18:16 ` Andreas Schwab 2019-11-19 20:56 ` Paul Eggert @ 2019-11-19 23:21 ` Ulrich Mueller 1 sibling, 0 replies; 15+ messages in thread From: Ulrich Mueller @ 2019-11-19 23:21 UTC (permalink / raw) To: Andreas Schwab; +Cc: emacs-devel >>>>> On Tue, 19 Nov 2019, Andreas Schwab wrote: > On Nov 19 2019, Ulrich Mueller wrote: >> For two successive installs from a clean checkout of the same git >> commit, emacs.pdmp ends up having slightly different size: >> >> $ stat -c %s /usr/libexec/emacs/27.0.50/x86_64-pc-linux-gnu/emacs.pdmp >> 10563744 >> >> $ stat -c %s /usr/libexec/emacs/27.0.50/x86_64-pc-linux-gnu/emacs.pdmp >> 10563728 >> >> Why is that so? Shouldn't it be deterministic? > Try disabling ASLR. That doesn't make it completely reproducible, > though. Indeed. After doing "echo -n 0 >/proc/sys/kernel/randomize_va_space" I get at least the same size every time. This is with Linux 4.14.154 and glibc 2.30. ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2019-11-20 19:59 UTC | newest] Thread overview: 15+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2019-11-19 16:59 Size of emacs.pdmp changes between successive installs Ulrich Mueller 2019-11-19 17:18 ` Paul Eggert 2019-11-19 17:45 ` Eli Zaretskii 2019-11-19 18:19 ` Andreas Schwab 2019-11-19 18:16 ` Andreas Schwab 2019-11-19 20:56 ` Paul Eggert 2019-11-19 22:14 ` Andreas Schwab 2019-11-19 23:45 ` Ulrich Mueller 2019-11-20 2:27 ` Paul Eggert 2019-11-20 11:55 ` Andreas Schwab 2019-11-20 19:54 ` Paul Eggert 2019-11-20 19:59 ` Paul Eggert 2019-11-20 9:29 ` Andreas Schwab 2019-11-20 11:42 ` Ulrich Mueller 2019-11-19 23:21 ` Ulrich Mueller
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.