* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL @ 2015-06-21 15:30 Eli Zaretskii 2015-06-25 3:31 ` Paul Eggert 0 siblings, 1 reply; 18+ messages in thread From: Eli Zaretskii @ 2015-06-21 15:30 UTC (permalink / raw To: 20862 To reproduce, on a 32-bit system: ./configure ... --with-wide-int make cd test/automated make This causes the cl-lib-tests to fail in this innocent check: (ert-deftest cl-lib-test-first () (should (null (cl-first '()))) <<<<<<<<<<<<<<<<< The error is "Invalid function ((nil))". The same test runs fine in a 32-bit build with 32-bit ints. While digging into this, I found out that if I compile cl-lib-test.el with an Emacs executable that doesn't use wide ints, the test succeeds. So I compared the .elc files, and they indeed differ. Moreover, by comparing the *.elc files in the entire Emacs tree, I found 3 files that differ: cl-generic.elc, srt-wy.elc, and one file in Eshell whose name I don't remember. Focusing my attention on cl-generic, which might explain why cl-lib-tests fails, how can I debug this? Would presenting disassembly of a part of the byte code where I see differences help someone to come up with ideas? Are there any other directions to investigate this? Did anyone run the test suite successfully in such a 32-bit build? Do all the *.elc files come out identical to a "normal" 32-bit build? TIA In GNU Emacs 25.0.50.189 (i686-pc-mingw32) of 2015-06-21 on HOME-C4E4A596F7 Repository revision: 821a3633564f36857968c7fe2b8bb6681a895905 Windowing system distributor `Microsoft Corp.', version 5.1.2600 Configured using: `configure --prefix=/d/usr --enable-checking=yes,glyphs --with-wide-int 'CFLAGS=-gdwarf-4 -g3 -O0'' Configured features: XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS Important settings: value of $LANG: ENU locale-coding-system: cp1255 Major mode: Lisp Interaction Minor modes in effect: tooltip-mode: t global-eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t Recent messages: For information about GNU Emacs and the GNU system, type C-h C-a. Load-path shadows: None found. Features: (shadow sort gnus-util mail-extr emacsbug message dired format-spec rfc822 mml easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util help-fns mail-prsvr mail-utils time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel dos-w32 ls-lisp disp-table w32-win w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote w32notify w32 multi-tty make-network-process emacs) Memory information: ((conses 16 79448 7064) (symbols 56 18398 0) (miscs 48 36 96) (strings 16 12848 4918) (string-bytes 1 326874) (vectors 16 10639) (vector-slots 8 396234 8479) (floats 8 114 55) (intervals 40 251 67) (buffers 856 11)) ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL 2015-06-21 15:30 bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL Eli Zaretskii @ 2015-06-25 3:31 ` Paul Eggert 2015-06-25 14:30 ` Eli Zaretskii 0 siblings, 1 reply; 18+ messages in thread From: Paul Eggert @ 2015-06-25 3:31 UTC (permalink / raw To: Eli Zaretskii; +Cc: 20862 [-- Attachment #1: Type: text/plain, Size: 862 bytes --] Thanks for reporting that. It appears to be a bug in the garbage collector, and is likely to be hard to reproduce. I couldn't reproduce it, but I did a 'make bootstrap' on Fedora x86-64 (configured with --with-wide-int and compiled with gcc -m32 so it's really x86), and got a core dump in a completely different area that (of course!) went away when I compiled without optimization. Rather than try to debug it directly, I thought about what might have caused the problem, and re-audited the garbage collector with the recent Qnil==0 changes in mind. This did uncover a bug, and the attached patch (which we will need anyway) allowed me to do a "make bootstrap" successfully in the same configuration. I installed this into the master as commit 93f4f67ba93b78e8b31e498e8ce7bce4c8298b76; please give it a try in your setup when you have the time. [-- Attachment #2: 0001-Fix-GC-bugs-with-wide-int-and-Qnil-0.patch --] [-- Type: text/x-diff, Size: 7785 bytes --] From 2bc416753ea9dbdeb27719957076ceb909886bd1 Mon Sep 17 00:00:00 2001 From: Paul Eggert <eggert@cs.ucla.edu> Date: Wed, 24 Jun 2015 20:10:03 -0700 Subject: [PATCH] Fix GC bugs --with-wide-int and Qnil == 0 Use the same alignment for the !USE_LSB_TAG case as for the more-typical USE_LSB_TAG case. The attempt to support arbitrary alignments with !USE_LSB_TAG had subtle bugs in garbage collection once we changed the representation of symbols so that Qnil == 0. Problem reported by Eli Zaretskii (Bug#20862). * src/alloc.c (XMALLOC_HEADER_ALIGNMENT) [XMALLOC_OVERRUN_CHECK]: * src/alloc.c (vector_alignment, union aligned_Lisp_Symbol) (union aligned_Lisp_Misc, maybe_lisp_pointer, pure_alloc): Use same alignment for !USE_LSB_TAG as for USE_LSB_TAG. * src/alloc.c (POINTERS_MIGHT_HIDE_IN_OBJECTS): Remove. This optimization in the !USE_LSB_TAG case is no longer valid when symbols are represented via offsets. Change the only use to assume that pointers might hide in objects. * src/lisp.h (alignas) [!USE_LSB_TAG]: Require support in this case, too. (TAG_SYMOFFSET, XSYMBOL) [!USE_LSB_TAG]: Do not shift the offset. This is OK, because the !USE_LSB_TAG case now applies only when Lisp_Object is wider than void *, so there's no longer any need to shift the offset. Not shifting the offset means that symbol representations have the same alignment as pointers, which the GC assumes. --- src/alloc.c | 59 ++++++++++------------------------------------------------- src/lisp.h | 11 ++--------- 2 files changed, 12 insertions(+), 58 deletions(-) diff --git a/src/alloc.c b/src/alloc.c index a956e95..c9bdcc2 100644 --- a/src/alloc.c +++ b/src/alloc.c @@ -528,12 +528,8 @@ buffer_memory_full (ptrdiff_t nbytes) alignment that Emacs needs for C types and for USE_LSB_TAG. */ #define XMALLOC_BASE_ALIGNMENT alignof (max_align_t) -#if USE_LSB_TAG -# define XMALLOC_HEADER_ALIGNMENT \ - COMMON_MULTIPLE (GCALIGNMENT, XMALLOC_BASE_ALIGNMENT) -#else -# define XMALLOC_HEADER_ALIGNMENT XMALLOC_BASE_ALIGNMENT -#endif +#define XMALLOC_HEADER_ALIGNMENT \ + COMMON_MULTIPLE (GCALIGNMENT, XMALLOC_BASE_ALIGNMENT) #define XMALLOC_OVERRUN_SIZE_SIZE \ (((XMALLOC_OVERRUN_CHECK_SIZE + sizeof (size_t) \ + XMALLOC_HEADER_ALIGNMENT - 1) \ @@ -2730,7 +2726,7 @@ enum { /* Alignment of struct Lisp_Vector objects. */ vector_alignment = COMMON_MULTIPLE (ALIGNOF_STRUCT_LISP_VECTOR, - USE_LSB_TAG ? GCALIGNMENT : 1), + GCALIGNMENT), /* Vector size requests are a multiple of this. */ roundup_size = COMMON_MULTIPLE (vector_alignment, word_size) @@ -3299,15 +3295,13 @@ usage: (make-byte-code ARGLIST BYTE-CODE CONSTANTS DEPTH &optional DOCSTRING INT ***********************************************************************/ /* Like struct Lisp_Symbol, but padded so that the size is a multiple - of the required alignment if LSB tags are used. */ + of the required alignment. */ union aligned_Lisp_Symbol { struct Lisp_Symbol s; -#if USE_LSB_TAG unsigned char c[(sizeof (struct Lisp_Symbol) + GCALIGNMENT - 1) & -GCALIGNMENT]; -#endif }; /* Each symbol_block is just under 1020 bytes long, since malloc @@ -3411,15 +3405,13 @@ Its value is void, and its function definition and property list are nil. */) ***********************************************************************/ /* Like union Lisp_Misc, but padded so that its size is a multiple of - the required alignment when LSB tags are used. */ + the required alignment. */ union aligned_Lisp_Misc { union Lisp_Misc m; -#if USE_LSB_TAG unsigned char c[(sizeof (union Lisp_Misc) + GCALIGNMENT - 1) & -GCALIGNMENT]; -#endif }; /* Allocation of markers and other objects that share that structure. @@ -4628,13 +4620,13 @@ mark_maybe_object (Lisp_Object obj) } /* Return true if P can point to Lisp data, and false otherwise. - USE_LSB_TAG needs Lisp data to be aligned on multiples of GCALIGNMENT. - Otherwise, assume that Lisp data is aligned on even addresses. */ + Symbols are implemented via offsets not pointers, but the offsets + are also multiples of GCALIGNMENT. */ static bool maybe_lisp_pointer (void *p) { - return !((intptr_t) p % (USE_LSB_TAG ? GCALIGNMENT : 2)); + return (uintptr_t) p % GCALIGNMENT == 0; } /* If P points to Lisp data, mark that as live if it isn't already @@ -4722,27 +4714,6 @@ mark_maybe_pointer (void *p) miss objects if __alignof__ were used. */ #define GC_POINTER_ALIGNMENT alignof (void *) -/* Define POINTERS_MIGHT_HIDE_IN_OBJECTS to 1 if marking via C pointers does - not suffice, which is the typical case. A host where a Lisp_Object is - wider than a pointer might allocate a Lisp_Object in non-adjacent halves. - If USE_LSB_TAG, the bottom half is not a valid pointer, but it should - suffice to widen it to to a Lisp_Object and check it that way. */ -#if USE_LSB_TAG || VAL_MAX < UINTPTR_MAX -# if !USE_LSB_TAG && VAL_MAX < UINTPTR_MAX >> GCTYPEBITS - /* If tag bits straddle pointer-word boundaries, neither mark_maybe_pointer - nor mark_maybe_object can follow the pointers. This should not occur on - any practical porting target. */ -# error "MSB type bits straddle pointer-word boundaries" -# endif - /* Marking via C pointers does not suffice, because Lisp_Objects contain - pointer words that hold pointers ORed with type bits. */ -# define POINTERS_MIGHT_HIDE_IN_OBJECTS 1 -#else - /* Marking via C pointers suffices, because Lisp_Objects contain pointer - words that hold unmodified pointers. */ -# define POINTERS_MIGHT_HIDE_IN_OBJECTS 0 -#endif - /* Mark Lisp objects referenced from the address range START+OFFSET..END or END+OFFSET..START. */ @@ -4788,8 +4759,7 @@ mark_memory (void *start, void *end) { void *p = *(void **) ((char *) pp + i); mark_maybe_pointer (p); - if (POINTERS_MIGHT_HIDE_IN_OBJECTS) - mark_maybe_object (XIL ((intptr_t) p)); + mark_maybe_object (XIL ((intptr_t) p)); } } @@ -5148,22 +5118,13 @@ static void * pure_alloc (size_t size, int type) { void *result; -#if USE_LSB_TAG - size_t alignment = GCALIGNMENT; -#else - size_t alignment = alignof (EMACS_INT); - - /* Give Lisp_Floats an extra alignment. */ - if (type == Lisp_Float) - alignment = alignof (struct Lisp_Float); -#endif again: if (type >= 0) { /* Allocate space for a Lisp object from the beginning of the free space with taking account of alignment. */ - result = ALIGN (purebeg + pure_bytes_used_lisp, alignment); + result = ALIGN (purebeg + pure_bytes_used_lisp, GCALIGNMENT); pure_bytes_used_lisp = ((char *)result - (char *)purebeg) + size; } else diff --git a/src/lisp.h b/src/lisp.h index 198f116..c3289c9 100644 --- a/src/lisp.h +++ b/src/lisp.h @@ -278,10 +278,7 @@ error !; #endif #ifndef alignas -# define alignas(alignment) /* empty */ -# if USE_LSB_TAG -# error "USE_LSB_TAG requires alignas" -# endif +# error "alignas not defined" #endif #ifdef HAVE_STRUCT_ATTRIBUTE_ALIGNED @@ -731,9 +728,7 @@ struct Lisp_Symbol /* Yield an integer that contains a symbol tag along with OFFSET. OFFSET should be the offset in bytes from 'lispsym' to the symbol. */ -#define TAG_SYMOFFSET(offset) \ - TAG_PTR (Lisp_Symbol, \ - ((uintptr_t) (offset) >> (USE_LSB_TAG ? 0 : GCTYPEBITS))) +#define TAG_SYMOFFSET(offset) TAG_PTR (Lisp_Symbol, offset) /* XLI_BUILTIN_LISPSYM (iQwhatever) is equivalent to XLI (builtin_lisp_symbol (Qwhatever)), @@ -899,8 +894,6 @@ INLINE struct Lisp_Symbol * XSYMBOL (Lisp_Object a) { uintptr_t i = (uintptr_t) XUNTAG (a, Lisp_Symbol); - if (! USE_LSB_TAG) - i <<= GCTYPEBITS; void *p = (char *) lispsym + i; return p; } -- 2.1.0 ^ permalink raw reply related [flat|nested] 18+ messages in thread
* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL 2015-06-25 3:31 ` Paul Eggert @ 2015-06-25 14:30 ` Eli Zaretskii 2015-06-26 19:59 ` Paul Eggert 0 siblings, 1 reply; 18+ messages in thread From: Eli Zaretskii @ 2015-06-25 14:30 UTC (permalink / raw To: Paul Eggert; +Cc: 20862 > Date: Wed, 24 Jun 2015 20:31:10 -0700 > From: Paul Eggert <eggert@cs.ucla.edu> > CC: 20862@debbugs.gnu.org > > Thanks for reporting that. It appears to be a bug in the garbage collector, and is likely to be hard to reproduce. I couldn't reproduce it, but I did a 'make bootstrap' on Fedora x86-64 (configured with --with-wide-int and compiled with gcc -m32 so it's really x86), and got a core dump in a completely different area that (of course!) went away when I compiled without optimization. Do you remember which Lisp file was being compiled when you got a core dump? I had something similar while trying to debug this: crashes while compiling cedet/srecode/proj-obj.el and sometimes also ibuffer.el. And yes, it's a very elusive crash: it only happens during a full bootstrap, and even repeating the exact same bootstrap in a different directory makes it disappear! But I was luckier than you, in that I did succeed reproducing these crashes in an unoptimized build. By slowly and painfully tracking these crashes, I found out that they are due to an un-interned symbol whose name is "THIS", created by this form: (let ((print-gensym nil) (print-quoted t)) (format "%S" (cons 'fn (cl--make-usage-args orig-args)))) That symbol gets GC'ed (I see "DEAD" in its function cell) while it's still alive, and then Emacs crashes trying to print that symbol's name in the call to 'format', because GC recycles that symbol's name and replaces it with a NULL pointer. So your analysis: > Rather than try to debug it directly, I thought about what might have caused the problem, and re-audited the garbage collector with the recent Qnil==0 changes in mind. This did uncover a bug, and the attached patch (which we will need anyway) allowed me to do a "make bootstrap" successfully in the same configuration. and the changes in the patch related to symbols-as-offsets and their alignment on the stack, make perfect sense to me, because they explain how come stack marking didn't mark this symbol, and thus allowed it to be GC'ed. > I installed this into the master as commit 93f4f67ba93b78e8b31e498e8ce7bce4c8298b76; please give it a try in your setup when you have the time. I did, and the crashes are gone, thanks. The cl-lib-tests also succeed. However, there still seems to be some subtle problem, because the byte-compiled files don't all compare equal. (I've seen the same problem before your patches as well.) I used the following command to find the *.elc files that are different: diff -r -a -u -I"in Emacs version 25\.0\.50" ./lisp ../int32/lisp --exclude="*.el" --exclude="*.el~" | grep -a "^diff " where "../int32/" is the directory where I built Emacs without "--with-wide-int". This reveals differences in the following files: cedet/semantic/texi.elc cedet/semantic/util.elc cedet/srecode/srt-wy.elc emacs-lisp/cl-generic.elc Some of the differences are insignificant (different label numbers used, or different file offsets due to a longer Emacs version string), but others seem to be significant. For example, the byte code of cl--generic-struct-tag in cl-generic.elc has a few different bytes. Likewise with the byte code of semantic-texi-expand-tag in texi.elc, of semantic-something-to-tag-table in util.elc, and of srecode-template-wy--parse-table in srt-wy.elc. The list of *.elc files that differ appears to depend on optimization level: the above list was obtained with -O0; compiling with -O1 leaves only cl-generic.elc and srt-wy.elc different, and compiling with -O2 brings util.elc back, and also adds differences in eshell/esh-proc.elc. Or maybe the actual factor is the specific order in which the files are compiled (I bootstrap with "make -j8"), which determines which other Lisp files are available as *.el or *.elc, because bootstrapping without parallel Make execution again leaves only cl-generic.elc and srt-wy.elc? Do you see something similar on your system? How to go about debugging this? ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL 2015-06-25 14:30 ` Eli Zaretskii @ 2015-06-26 19:59 ` Paul Eggert 2015-06-26 20:39 ` Eli Zaretskii 0 siblings, 1 reply; 18+ messages in thread From: Paul Eggert @ 2015-06-26 19:59 UTC (permalink / raw To: Eli Zaretskii; +Cc: 20862 Eli Zaretskii wrote: > Do you remember which Lisp file was being compiled when you got a core > dump? Sorry, no. I don't think it matters -- if the garbage collector is bad anything can happen. > maybe the actual factor is the specific order > in which the files are compiled (I bootstrap with "make -j8"), Yes, that's plausible. > Do you see something similar on your system? How to go about > debugging this? I expect it'd require someone with expertise in byte-compiling. It's not clear that it's a bug, other than a minor performance issue. I do see something similar on my system. The following files differed in one comparison I did: cedet/semantic/grammar-wy.elc cedet/semantic/wisent/javat-wy.elc cedet/semantic/wisent/js-wy.elc cedet/semantic/wisent/python-wy.elc cedet/srecode/insert.elc cedet/srecode/srt-wy.elc gnus/auth-source.elc gnus/message.elc gnus/mm-decode.elc org/ob-C.elc progmodes/antlr-mode.elc progmodes/cc-langs.elc progmodes/cc-mode.elc textmodes/rst.elc In one case I looked at it appears to be gensym values, which ought to be OK. ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL 2015-06-26 19:59 ` Paul Eggert @ 2015-06-26 20:39 ` Eli Zaretskii 2015-06-26 20:45 ` Paul Eggert 0 siblings, 1 reply; 18+ messages in thread From: Eli Zaretskii @ 2015-06-26 20:39 UTC (permalink / raw To: Paul Eggert; +Cc: 20862 > Date: Fri, 26 Jun 2015 12:59:40 -0700 > From: Paul Eggert <eggert@cs.ucla.edu> > CC: 20862@debbugs.gnu.org > > In one case I looked at it appears to be gensym values, which ought to be OK. Can you elaborate? What are "gensym values"? Thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL 2015-06-26 20:39 ` Eli Zaretskii @ 2015-06-26 20:45 ` Paul Eggert 2015-06-26 21:30 ` Glenn Morris 2015-06-27 7:17 ` Eli Zaretskii 0 siblings, 2 replies; 18+ messages in thread From: Paul Eggert @ 2015-06-26 20:45 UTC (permalink / raw To: Eli Zaretskii; +Cc: 20862 Eli Zaretskii wrote: > Can you elaborate? What are "gensym values"? Generated symbols of some sort. They appear to be like the output that the printer uses when it's printing a recursive data structure. E.g., one .elc file says ‘(#3=#:G66300 e)’ where the other says ‘(#3=#:G44900 e)’. I doubt whether this difference matters, though I haven't investigated. ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL 2015-06-26 20:45 ` Paul Eggert @ 2015-06-26 21:30 ` Glenn Morris 2015-06-27 7:22 ` Eli Zaretskii 2015-06-27 7:17 ` Eli Zaretskii 1 sibling, 1 reply; 18+ messages in thread From: Glenn Morris @ 2015-06-26 21:30 UTC (permalink / raw To: Paul Eggert; +Cc: 20862 Paul Eggert wrote: > Eli Zaretskii wrote: >> Can you elaborate? What are "gensym values"? > > Generated symbols of some sort. They appear to be like the output > that the printer uses when it's printing a recursive data structure. > E.g., one .elc file says '(#3=#:G66300 e)' where the other says > '(#3=#:G44900 e)'. I doubt whether this difference matters, though I > haven't investigated. I think this is because cl--gensym-counter, as used by cl-gensym, is initialized to a random value based on the time. (This is an issue for reproducible builds, if we care about those.) ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL 2015-06-26 21:30 ` Glenn Morris @ 2015-06-27 7:22 ` Eli Zaretskii 2015-06-27 15:56 ` Paul Eggert 0 siblings, 1 reply; 18+ messages in thread From: Eli Zaretskii @ 2015-06-27 7:22 UTC (permalink / raw To: Glenn Morris; +Cc: eggert, 20862 > From: Glenn Morris <rgm@gnu.org> > Cc: Eli Zaretskii <eliz@gnu.org>, 20862@debbugs.gnu.org > Date: Fri, 26 Jun 2015 17:30:26 -0400 > > Paul Eggert wrote: > > > Eli Zaretskii wrote: > >> Can you elaborate? What are "gensym values"? > > > > Generated symbols of some sort. They appear to be like the output > > that the printer uses when it's printing a recursive data structure. > > E.g., one .elc file says '(#3=#:G66300 e)' where the other says > > '(#3=#:G44900 e)'. I doubt whether this difference matters, though I > > haven't investigated. > > I think this is because cl--gensym-counter, as used by cl-gensym, is > initialized to a random value based on the time. (This is an issue for > reproducible builds, if we care about those.) I think the *.elc files should be reproducible, yes. Otherwise, it's hard to know if there's some subtle problem in the byte compiler or its support routines. ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL 2015-06-27 7:22 ` Eli Zaretskii @ 2015-06-27 15:56 ` Paul Eggert 2015-06-27 16:30 ` Eli Zaretskii 2015-06-27 16:36 ` Andreas Schwab 0 siblings, 2 replies; 18+ messages in thread From: Paul Eggert @ 2015-06-27 15:56 UTC (permalink / raw To: Eli Zaretskii, Glenn Morris; +Cc: 20862 Eli Zaretskii wrote: > I think the *.elc files should be reproducible, yes. Otherwise, it's > hard to know if there's some subtle problem in the byte compiler or > its support routines. The simplest fix would be to initialize cl--gensym-counter to 0, as is done in Common Lisp, so that Emacs generates reproducible output. However, the documentation for cl-gensym says this: (Internally, the variable @code{cl--gensym-counter} holds the counter used to generate names. It is incremented after each use. In Common Lisp this is initialized with 0, but this package initializes it with a random time-dependent value to avoid trouble when two files that each used @code{cl-gensym} in their compilation are loaded together. Uninterned symbols become interned when the compiler writes them out to a file and the Emacs loader loads them, so their names have to be treated a bit more carefully than in Common Lisp where uninterned symbols remain uninterned after loading.) This passage is dismaying, as it suggests that it's a matter of luck as to whether trouble is avoided when two files that each used cl-gensym are loaded together. I guess the trouble that's avoided is the usual issue with macro capture. (Maybe we should file another bug report for this....) ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL 2015-06-27 15:56 ` Paul Eggert @ 2015-06-27 16:30 ` Eli Zaretskii 2015-06-27 16:36 ` Andreas Schwab 1 sibling, 0 replies; 18+ messages in thread From: Eli Zaretskii @ 2015-06-27 16:30 UTC (permalink / raw To: Paul Eggert; +Cc: 20862 > Date: Sat, 27 Jun 2015 08:56:30 -0700 > From: Paul Eggert <eggert@cs.ucla.edu> > CC: 20862@debbugs.gnu.org > > This passage is dismaying, as it suggests that it's a matter of luck as to > whether trouble is avoided when two files that each used cl-gensym are loaded > together. Yes, and I think it should not be too hard to come up with a number or string that will not be random, and yet will be unique to each file. > I guess the trouble that's avoided is the usual issue with macro > capture. (Maybe we should file another bug report for this....) Indeed, I think it's worth another bug report. Thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL 2015-06-27 15:56 ` Paul Eggert 2015-06-27 16:30 ` Eli Zaretskii @ 2015-06-27 16:36 ` Andreas Schwab 2015-06-27 18:01 ` Paul Eggert 1 sibling, 1 reply; 18+ messages in thread From: Andreas Schwab @ 2015-06-27 16:36 UTC (permalink / raw To: Paul Eggert; +Cc: 20862 Paul Eggert <eggert@cs.ucla.edu> writes: > Uninterned symbols become interned when the compiler writes them out That isn't true. #: prefixed symbols are uninterned symbols. Andreas. -- Andreas Schwab, schwab@linux-m68k.org GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different." ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL 2015-06-27 16:36 ` Andreas Schwab @ 2015-06-27 18:01 ` Paul Eggert 2015-06-27 18:39 ` Eli Zaretskii 0 siblings, 1 reply; 18+ messages in thread From: Paul Eggert @ 2015-06-27 18:01 UTC (permalink / raw To: Andreas Schwab; +Cc: 20862 [-- Attachment #1: Type: text/plain, Size: 523 bytes --] Andreas Schwab wrote: > That isn't true. #: prefixed symbols are uninterned symbols. Good point, thanks. The doc I quoted predates the #: syntax, and evidently wasn't updated when the #: syntax was introduced and used. Since that doc wording is wrong, and there's now no reason to initialize cl--gensym-counter to anything other than 0, I boldly installed the attached patch. It passes 'make bootstrap' and 'make check' (at least, as well as 'make check' did before -- it always fails for me in epg-roundtrip-2). [-- Attachment #2: 0001-Initialize-cl-gensym-counter-to-0.patch --] [-- Type: text/x-diff, Size: 4035 bytes --] From c66ff5757c6eeb495eeb283f34fc0713d1fb5270 Mon Sep 17 00:00:00 2001 From: Paul Eggert <eggert@cs.ucla.edu> Date: Sat, 27 Jun 2015 10:57:02 -0700 Subject: [PATCH] Initialize cl--gensym-counter to 0 Previously it was initialized to a random value, which made it harder to reproduce earlier Emacs runs. The need for a random value went away when Emacs introduced and used the #: syntax for uninterned symbols (Bug#20862). * doc/misc/cl.texi (Creating Symbols, Common Lisp Compatibility): Document that cl--gensym-counter now starts with 0. * lisp/emacs-lisp/cl-lib.el (cl--gensym-counter): Remove. (cl--random-time): Move to near only remaining use. * lisp/emacs-lisp/cl-macs.el (cl--gensym-counter): Initialize to 0. --- doc/misc/cl.texi | 15 +++------------ lisp/emacs-lisp/cl-lib.el | 15 +++++---------- lisp/emacs-lisp/cl-macs.el | 2 +- 3 files changed, 9 insertions(+), 23 deletions(-) diff --git a/doc/misc/cl.texi b/doc/misc/cl.texi index d7b3f4a..1f38ca9 100644 --- a/doc/misc/cl.texi +++ b/doc/misc/cl.texi @@ -2877,14 +2877,8 @@ their names will not conflict with ``real'' variables in the user's code. (Internally, the variable @code{cl--gensym-counter} holds the counter -used to generate names. It is incremented after each use. In Common -Lisp this is initialized with 0, but this package initializes it with -a random time-dependent value to avoid trouble when two files that -each used @code{cl-gensym} in their compilation are loaded together. -Uninterned symbols become interned when the compiler writes them out -to a file and the Emacs loader loads them, so their names have to be -treated a bit more carefully than in Common Lisp where uninterned -symbols remain uninterned after loading.) +used to generate names. It is initialized with zero and incremented +after each use.) @end defun @defun cl-gentemp &optional x @@ -4543,10 +4537,7 @@ example, local @code{special} declarations, which are purely advisory in Emacs Lisp, do not rigorously obey the scoping rules set down in Steele's book. -The variable @code{cl--gensym-counter} starts out with a pseudo-random -value rather than with zero. This is to cope with the fact that -generated symbols become interned when they are written to and -loaded back from a file. +The variable @code{cl--gensym-counter} starts out with zero. The @code{cl-defstruct} facility is compatible, except that structures are of type @code{:type vector :named} by default rather than some diff --git a/lisp/emacs-lisp/cl-lib.el b/lisp/emacs-lisp/cl-lib.el index b6f3a79..2dd0519 100644 --- a/lisp/emacs-lisp/cl-lib.el +++ b/lisp/emacs-lisp/cl-lib.el @@ -249,16 +249,6 @@ so that they are registered at compile-time as well as run-time." `(progn ,@body)))) ; Avoid loading cl-macs.el for cl-eval-when. -;;; Symbols. - -(defun cl--random-time () - (let* ((time (copy-sequence (current-time-string))) (i (length time)) (v 0)) - (while (>= (cl-decf i) 0) (setq v (+ (* v 3) (aref time i)))) - v)) - -(defvar cl--gensym-counter (* (logand (cl--random-time) 1023) 100)) - - ;;; Numbers. (define-obsolete-function-alias 'cl-floatp-safe 'floatp "24.4") @@ -298,6 +288,11 @@ If true return the decimal value of digit CHAR in RADIX." (let ((n (aref cl-digit-char-table char))) (and n (< n (or radix 10)) n))) +(defun cl--random-time () + (let* ((time (copy-sequence (current-time-string))) (i (length time)) (v 0)) + (while (>= (cl-decf i) 0) (setq v (+ (* v 3) (aref time i)))) + v)) + (defvar cl--random-state (vector 'cl--random-state-tag -1 30 (cl--random-time))) diff --git a/lisp/emacs-lisp/cl-macs.el b/lisp/emacs-lisp/cl-macs.el index 636c543..5bcf088 100644 --- a/lisp/emacs-lisp/cl-macs.el +++ b/lisp/emacs-lisp/cl-macs.el @@ -161,7 +161,7 @@ whether X is known at compile time, macroexpand it completely in ;;; Symbols. -(defvar cl--gensym-counter) +(defvar cl--gensym-counter 0) ;;;###autoload (defun cl-gensym (&optional prefix) "Generate a new uninterned symbol. -- 2.1.0 ^ permalink raw reply related [flat|nested] 18+ messages in thread
* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL 2015-06-27 18:01 ` Paul Eggert @ 2015-06-27 18:39 ` Eli Zaretskii 2015-06-27 18:40 ` Paul Eggert 0 siblings, 1 reply; 18+ messages in thread From: Eli Zaretskii @ 2015-06-27 18:39 UTC (permalink / raw To: Paul Eggert; +Cc: schwab, 20862 > Date: Sat, 27 Jun 2015 11:01:53 -0700 > From: Paul Eggert <eggert@cs.ucla.edu> > CC: Eli Zaretskii <eliz@gnu.org>, Glenn Morris <rgm@gnu.org>, > 20862@debbugs.gnu.org > > I boldly installed the attached patch. It passes 'make bootstrap' > and 'make check' Thanks. Did you try "make -jN", for some reasonably large N? ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL 2015-06-27 18:39 ` Eli Zaretskii @ 2015-06-27 18:40 ` Paul Eggert 2015-06-27 19:41 ` Eli Zaretskii 0 siblings, 1 reply; 18+ messages in thread From: Paul Eggert @ 2015-06-27 18:40 UTC (permalink / raw To: Eli Zaretskii; +Cc: schwab, 20862 Eli Zaretskii wrote: > Did you try "make -jN", for some reasonably large N? I used 'make -j5' on a 4-core host. ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL 2015-06-27 18:40 ` Paul Eggert @ 2015-06-27 19:41 ` Eli Zaretskii 2015-06-28 15:48 ` Eli Zaretskii 0 siblings, 1 reply; 18+ messages in thread From: Eli Zaretskii @ 2015-06-27 19:41 UTC (permalink / raw To: Paul Eggert; +Cc: schwab, 20862 > Date: Sat, 27 Jun 2015 11:40:40 -0700 > From: Paul Eggert <eggert@cs.ucla.edu> > CC: schwab@linux-m68k.org, rgm@gnu.org, 20862@debbugs.gnu.org > > Eli Zaretskii wrote: > > Did you try "make -jN", for some reasonably large N? > > I used 'make -j5' on a 4-core host. Thanks, that's reassuring. ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL 2015-06-27 19:41 ` Eli Zaretskii @ 2015-06-28 15:48 ` Eli Zaretskii 2015-06-28 16:01 ` Paul Eggert 0 siblings, 1 reply; 18+ messages in thread From: Eli Zaretskii @ 2015-06-28 15:48 UTC (permalink / raw To: eggert; +Cc: schwab, 20862 > Date: Sat, 27 Jun 2015 22:41:57 +0300 > From: Eli Zaretskii <eliz@gnu.org> > Cc: schwab@linux-m68k.org, 20862@debbugs.gnu.org > > > Date: Sat, 27 Jun 2015 11:40:40 -0700 > > From: Paul Eggert <eggert@cs.ucla.edu> > > CC: schwab@linux-m68k.org, rgm@gnu.org, 20862@debbugs.gnu.org > > > > Eli Zaretskii wrote: > > > Did you try "make -jN", for some reasonably large N? > > > > I used 'make -j5' on a 4-core host. > > Thanks, that's reassuring. After updating from Savannah, I no longer see any problems related to the original report. So I guess we can close this bug now, right? Thanks. ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL 2015-06-28 15:48 ` Eli Zaretskii @ 2015-06-28 16:01 ` Paul Eggert 0 siblings, 0 replies; 18+ messages in thread From: Paul Eggert @ 2015-06-28 16:01 UTC (permalink / raw To: Eli Zaretskii; +Cc: schwab, 20862-done Eli Zaretskii wrote: > After updating from Savannah, I no longer see any problems related to > the original report. So I guess we can close this bug now, right? Yes, thanks for checking. Done. ^ permalink raw reply [flat|nested] 18+ messages in thread
* bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL 2015-06-26 20:45 ` Paul Eggert 2015-06-26 21:30 ` Glenn Morris @ 2015-06-27 7:17 ` Eli Zaretskii 1 sibling, 0 replies; 18+ messages in thread From: Eli Zaretskii @ 2015-06-27 7:17 UTC (permalink / raw To: Paul Eggert; +Cc: 20862 > Date: Fri, 26 Jun 2015 13:45:24 -0700 > From: Paul Eggert <eggert@cs.ucla.edu> > CC: 20862@debbugs.gnu.org > > Eli Zaretskii wrote: > > Can you elaborate? What are "gensym values"? > > Generated symbols of some sort. They appear to be like the output that the > printer uses when it's printing a recursive data structure. E.g., one .elc file > says ‘(#3=#:G66300 e)’ where the other says ‘(#3=#:G44900 e)’. I doubt whether > this difference matters, though I haven't investigated. Thanks. Those not the differences that I see that worry me. I guess I should look at the disassembly differences next. ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2015-06-28 16:01 UTC | newest] Thread overview: 18+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-06-21 15:30 bug#20862: 25.0.50; 32-bit Emacs configured --with-wide-int miscompiles CL Eli Zaretskii 2015-06-25 3:31 ` Paul Eggert 2015-06-25 14:30 ` Eli Zaretskii 2015-06-26 19:59 ` Paul Eggert 2015-06-26 20:39 ` Eli Zaretskii 2015-06-26 20:45 ` Paul Eggert 2015-06-26 21:30 ` Glenn Morris 2015-06-27 7:22 ` Eli Zaretskii 2015-06-27 15:56 ` Paul Eggert 2015-06-27 16:30 ` Eli Zaretskii 2015-06-27 16:36 ` Andreas Schwab 2015-06-27 18:01 ` Paul Eggert 2015-06-27 18:39 ` Eli Zaretskii 2015-06-27 18:40 ` Paul Eggert 2015-06-27 19:41 ` Eli Zaretskii 2015-06-28 15:48 ` Eli Zaretskii 2015-06-28 16:01 ` Paul Eggert 2015-06-27 7:17 ` Eli Zaretskii
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.