* Re: 64-bit emacs crashes a lot
[not found] ` <8361v6nhdb.fsf@gnu.org>
@ 2013-08-16 11:37 ` Ryan Johnson
2013-08-16 13:10 ` Eli Zaretskii
0 siblings, 1 reply; 27+ messages in thread
From: Ryan Johnson @ 2013-08-16 11:37 UTC (permalink / raw
To: Eli Zaretskii; +Cc: emacs-devel
On 16/08/2013 4:56 AM, Eli Zaretskii wrote:
> I'm not subscribed to this list, so if you want me to reply, please CC
> me explicitly. Besides, this discussion should be moved to
> emacs-devel@gnu.org, since I don't see anything Cygwin specific here
> at this point.
>
>> Date: Thu, 15 Aug 2013 16:55:18 -0400
>> From: Ryan Johnson <ryan.johnson@cs.utoronto.ca>
>>
>> On 15/08/2013 1:10 PM, Eli Zaretskii wrote:
>>>> Date: Thu, 15 Aug 2013 12:58:02 -0400
>>>> From: Ken Brown <kbrown@cornell.edu>
>>>> CC: Eli Zaretskii <eliz@gnu.org>
>>>>
>>>> Eli is the expert on bidi.c (he wrote it). He can probably tell you
>>>> whether you've really bumped into an emacs bug here.
>>> There's nothing wrong with bidi.c here, it just aborts because it is
>>> handed an invalid character codepoint. It would have been useful to
>>> see the value of that character.
>> I guess I would just consider crashing to be overkill for a bad byte on
>> the input stream...
> It's not a crash, it's a deliberate abort. Any invalid codepoint at
> such low level of the Emacs display engine means only one thing: a
> bug, and a grave one at that. Such bugs must be flagged prominently
> and unequivocally, prompting users to report them. We could in
> principle "recover" by substituting some other character, but such
> recovery would only sweep a grave problem under the carpet. Since
> Emacs isn't a safety-critical program, and auto-saves your edits
> before it commits suicide, such recovery feature is deemed
> inappropriate, and detrimental to the general quality of Emacs code in
> the long run.
OK, makes sense.
>> and in any case, if 5-byte UTF-8 is illegal, and
>> worth dying for, wouldn't it make sense to die right away rather than
>> processing it so something else can croak down the road?
> See above: yes, it's worth dying for, because I'm quite sure this is a
> sign of a very serious trouble in the session anyway. Why does it
> matter for you, as a user, whether we abort here or "down the road"?
> The principle is to die as soon as possible, because in many cases
> this allows to identify the culprit faster and easier. IOW, dying
> sooner and faster helps the Emacs maintainers to find and fix problems
> without any real effect on the users.
Re die as soon as possible: We're in violent agreement: I argued
that---if 5-byte UTF-8 is illegal---emacs should abort at the decoding
stage (= earlier) rather than at some later point when it discovers the
too-large value. In any case, though, I can't repro on a debug build so
this whole thing may be moot.
Ryan
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
[not found] ` <8338qangma.fsf@gnu.org>
@ 2013-08-16 11:39 ` Ryan Johnson
0 siblings, 0 replies; 27+ messages in thread
From: Ryan Johnson @ 2013-08-16 11:39 UTC (permalink / raw
To: Eli Zaretskii; +Cc: emacs-devel
On 16/08/2013 5:13 AM, Eli Zaretskii wrote:
> Please move this discussion to emacs-devel@gnu.org.
OK. For history's sake, here's the link back to the cygwin thread:
http://cygwin.com/ml/cygwin/2013-08/msg00275.html
>> Date: Fri, 16 Aug 2013 01:59:41 -0400
>> From: Ryan Johnson <ryan.johnson@cs.utoronto.ca>
>>
>> The variable pending_exact has value 0x0, which would be a Bad Thing...
>> except that the code looks like this:
>>> if (!pending_exact
>>>
>>> /* If last exactn not at current position. */
>>> => || pending_exact + *pending_exact + 1 != b
>>>
>> ... with corresponding assembly code looking very reasonable:
>>> 0x0000000100535cfa <regex_compile+34482>: cmpq $0x0,0x3f8(%rbp)
>>> 0x0000000100535d02 <regex_compile+34490>: je 0x100535eca
>>> <regex_compile+34946>
>>> 0x0000000100535d08 <regex_compile+34496>: mov 0x3f8(%rbp),%rax
>>> => 0x0000000100535d0f <regex_compile+34503>: movzbl (%rax),%eax
>>> 0x0000000100535d12 <regex_compile+34506>: movzbl %al,%eax
>>> 0x0000000100535d15 <regex_compile+34509>: lea 0x1(%rax),%rdx
>>> 0x0000000100535d19 <regex_compile+34513>: mov 0x3f8(%rbp),%rax
>>> 0x0000000100535d20 <regex_compile+34520>: add %rdx,%rax
>>> 0x0000000100535d23 <regex_compile+34523>: cmp %rbx,%rax
>>> 0x0000000100535d26 <regex_compile+34526>: jne 0x100535eca
>>> <regex_compile+34946>
> What is the value in the RAX register at the point of the crash? Is
> it also zero? Or maybe it is some other invalid pointer value?
Also zero, iirc
I tested it before, but my computer seems to have rebooted itself in the
night and the history was lost. Unfortunately, I'm having trouble
getting the debug emacs-nox to crash this morning. The -Og and -O2
builds crash even more often than before, though, usually while trying
to invoke compile for the first time; the stack traces have been total
garbage. I'll have to get back to you when the bug is being more
cooperative...
>> A third crash:
>>> #1 0x0000000100541930 in re_match_2_internal (bufp=0x10095ce20
>>> <searchbufs+2912>, string1=0x0, size1=0, string2=0x6fffff00028 "-*-
>>> mode: compilation; default-directory: \"~/\" -*-\nCompilation started
>>> at Fri Aug 16 01:32:19\n\nls\n#message-20130808-090732#\t
>>> emacs-crash.txt\t\tmusic\n6b8ob06a.default.tar.xz\t\t
>>> emacs-nox.exe."..., size2=355, pos=254, regs=0x10095def0
>>> <search_regs>, stop=317) at regex.c:6217
>>> 6217 abort ();
>> This time, p (the subject of the case statement) points to 0x76b3b6c7,
>> which is the middle of a function (ntdll!RtlFillMemory, though the
>> memory map places that address smack in the middle of kernel32.dll
>> instead). This time it makes perfect sense that the switch statement
>> should fail, but how did p go so wrong?
> What is bufp->buffer at this point, and what is its contents?
I'll let you know once I hit the crash again...
Ryan
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
[not found] ` <834naqnh9t.fsf@gnu.org>
@ 2013-08-16 11:41 ` Ryan Johnson
2013-08-16 13:31 ` Eli Zaretskii
0 siblings, 1 reply; 27+ messages in thread
From: Ryan Johnson @ 2013-08-16 11:41 UTC (permalink / raw
To: emacs-devel
On 16/08/2013 4:58 AM, Eli Zaretskii wrote:
> Again, please move this discussion to emacs-devel.
Back pointer: http://cygwin.com/ml/cygwin/2013-08/msg00273.html
>> Date: Thu, 15 Aug 2013 22:35:54 -0400
>> From: Ken Brown <kbrown@cornell.edu>
>>
>> 1. Invoke 'emacs-nox -Q' in mintty.
>>
>> 2. M-x compile C-a C-k ls RET
>>
>> 3. C-x o
>>
>> 4. Hit 'g' repeatedly.
>>
>> I got it to abort with Fatal error 6 after slightly over 100 repetitions.
>>
>> I then tried the same thing with emacs-X11 (running under X, not in
>> mintty). I hit 'g' 200 times without a problem. I repeated this with
>> emacs-w32, again 200 times without a problem.
>>
>> So there's a bug somewhere. But if it's an emacs bug, it's strange that
>> it only occurs with emacs-nox and not with either of the GUI versions of
>> emacs.
> I suspect that buffer relocation might be the reason. Can you show a
> backtrace from the fatal error in an unoptimized build, with the above
> recipe?
I'll try to get one. Meanwhile, is there a quick way to disable buffer
relocation as a sanity test? Some configure option, perhaps? If the
optimized build still crashes reliably without it, the real problem is
probably something else.
Ryan
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-16 11:37 ` 64-bit emacs crashes a lot Ryan Johnson
@ 2013-08-16 13:10 ` Eli Zaretskii
0 siblings, 0 replies; 27+ messages in thread
From: Eli Zaretskii @ 2013-08-16 13:10 UTC (permalink / raw
To: Ryan Johnson; +Cc: emacs-devel
> Date: Fri, 16 Aug 2013 07:37:53 -0400
> From: Ryan Johnson <ryan.johnson@cs.utoronto.ca>
> CC: emacs-devel@gnu.org
>
> Re die as soon as possible: We're in violent agreement: I argued
> that---if 5-byte UTF-8 is illegal---emacs should abort at the decoding
> stage (= earlier) rather than at some later point when it discovers the
> too-large value.
If Emacs can discover this when it decodes text, it does trigger an
error. But sometimes the buffer is corrupted later, which I think
what happened in your case.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-16 11:41 ` Ryan Johnson
@ 2013-08-16 13:31 ` Eli Zaretskii
2013-08-16 14:16 ` Ryan Johnson
2013-08-16 14:20 ` Ken Brown
0 siblings, 2 replies; 27+ messages in thread
From: Eli Zaretskii @ 2013-08-16 13:31 UTC (permalink / raw
To: Ryan Johnson; +Cc: emacs-devel
> Date: Fri, 16 Aug 2013 07:41:44 -0400
> From: Ryan Johnson <ryan.johnson@cs.utoronto.ca>
>
> On 16/08/2013 4:58 AM, Eli Zaretskii wrote:
> > Again, please move this discussion to emacs-devel.
> Back pointer: http://cygwin.com/ml/cygwin/2013-08/msg00273.html
>
> >> Date: Thu, 15 Aug 2013 22:35:54 -0400
> >> From: Ken Brown <kbrown@cornell.edu>
> >>
> >> 1. Invoke 'emacs-nox -Q' in mintty.
> >>
> >> 2. M-x compile C-a C-k ls RET
> >>
> >> 3. C-x o
> >>
> >> 4. Hit 'g' repeatedly.
> >>
> >> I got it to abort with Fatal error 6 after slightly over 100 repetitions.
> >>
> >> I then tried the same thing with emacs-X11 (running under X, not in
> >> mintty). I hit 'g' 200 times without a problem. I repeated this with
> >> emacs-w32, again 200 times without a problem.
> >>
> >> So there's a bug somewhere. But if it's an emacs bug, it's strange that
> >> it only occurs with emacs-nox and not with either of the GUI versions of
> >> emacs.
> > I suspect that buffer relocation might be the reason. Can you show a
> > backtrace from the fatal error in an unoptimized build, with the above
> > recipe?
> I'll try to get one. Meanwhile, is there a quick way to disable buffer
> relocation as a sanity test?
I don't think so, but maybe it's already disabled in your build. What
do you see on these two lines when configure finishes:
Should Emacs use a relocating allocator for buffers? yes
Should Emacs use mmap(2) for buffer allocation? no
In any case, you could configure with --enable-checking=all, it might
catch the problem earlier.
> If the optimized build still crashes reliably without it, the real
> problem is probably something else.
Maybe, but not necessarily.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-16 13:31 ` Eli Zaretskii
@ 2013-08-16 14:16 ` Ryan Johnson
2013-08-16 14:49 ` Eli Zaretskii
2013-08-16 14:20 ` Ken Brown
1 sibling, 1 reply; 27+ messages in thread
From: Ryan Johnson @ 2013-08-16 14:16 UTC (permalink / raw
To: Eli Zaretskii; +Cc: emacs-devel
On 16/08/2013 9:31 AM, Eli Zaretskii wrote:
>> Date: Fri, 16 Aug 2013 07:41:44 -0400
>> From: Ryan Johnson <ryan.johnson@cs.utoronto.ca>
>>
>> On 16/08/2013 4:58 AM, Eli Zaretskii wrote:
>>> Again, please move this discussion to emacs-devel.
>> Back pointer: http://cygwin.com/ml/cygwin/2013-08/msg00273.html
>>
>>>> Date: Thu, 15 Aug 2013 22:35:54 -0400
>>>> From: Ken Brown <kbrown@cornell.edu>
>>>>
>>>> 1. Invoke 'emacs-nox -Q' in mintty.
>>>>
>>>> 2. M-x compile C-a C-k ls RET
>>>>
>>>> 3. C-x o
>>>>
>>>> 4. Hit 'g' repeatedly.
>>>>
>>>> I got it to abort with Fatal error 6 after slightly over 100 repetitions.
>>>>
>>>> I then tried the same thing with emacs-X11 (running under X, not in
>>>> mintty). I hit 'g' 200 times without a problem. I repeated this with
>>>> emacs-w32, again 200 times without a problem.
>>>>
>>>> So there's a bug somewhere. But if it's an emacs bug, it's strange that
>>>> it only occurs with emacs-nox and not with either of the GUI versions of
>>>> emacs.
>>> I suspect that buffer relocation might be the reason. Can you show a
>>> backtrace from the fatal error in an unoptimized build, with the above
>>> recipe?
>> I'll try to get one. Meanwhile, is there a quick way to disable buffer
>> relocation as a sanity test?
> I don't think so, but maybe it's already disabled in your build. What
> do you see on these two lines when configure finishes:
>
> Should Emacs use a relocating allocator for buffers? yes
> Should Emacs use mmap(2) for buffer allocation? no
>
> In any case, you could configure with --enable-checking=all, it might
> catch the problem earlier.
Should Emacs use the GNU version of malloc? yes
Should Emacs use a relocating allocator for buffers? no
Should Emacs use mmap(2) for buffer allocation? yes
Hmm... during bootstrap I got the following with checking enabled:
> Loading loadup.el (source)...
> Using load-path (/scratch/emacs-24.3/lisp
> /scratch/emacs-24.3/lisp/emacs-lisp /scratch/emacs-24.3/lisp/language
> /scratch/emacs-24.3/lisp/international /scratch/emacs-24.3/lisp/textmodes)
> Loading emacs-lisp/byte-run...
> Loading emacs-lisp/backquote...
> Loading subr...
> Loading version...
> <** snip **>
> Loading international/mule-cmds...
> Loading case-table...
> Loading /scratch/emacs-24.3/lisp/international/charprop.el (source)...
> Loading international/characters...
> static heap exhausted: avail 10485760 used 9751456 failed request 790528
> Makefile:795: recipe for target `bootstrap-emacs.exe' failed
> make[1]: *** [bootstrap-emacs.exe] Error 1
> make[1]: Leaving directory `/scratch/emacs-24.3/src'
> Makefile:334: recipe for target `src' failed
> make: *** [src] Error 2
Without checking I get this output instead (and bootstrap completes):
> Finding pointers to doc strings...
> Finding pointers to doc strings...done
> Dumping under the name emacs
> Static heap usage: 9753632 of 10485760 bytes
> 77714 pure bytes used
Is there a way to grow the static heap?
Ryan
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-16 13:31 ` Eli Zaretskii
2013-08-16 14:16 ` Ryan Johnson
@ 2013-08-16 14:20 ` Ken Brown
2013-08-16 14:24 ` Ryan Johnson
` (2 more replies)
1 sibling, 3 replies; 27+ messages in thread
From: Ken Brown @ 2013-08-16 14:20 UTC (permalink / raw
To: Eli Zaretskii; +Cc: Ryan Johnson, emacs-devel
On 8/16/2013 9:31 AM, Eli Zaretskii wrote:
>> Date: Fri, 16 Aug 2013 07:41:44 -0400
>> From: Ryan Johnson <ryan.johnson@cs.utoronto.ca>
>>
>> On 16/08/2013 4:58 AM, Eli Zaretskii wrote:
>>> Again, please move this discussion to emacs-devel.
>> Back pointer: http://cygwin.com/ml/cygwin/2013-08/msg00273.html
>>
>>>> Date: Thu, 15 Aug 2013 22:35:54 -0400
>>>> From: Ken Brown <kbrown@cornell.edu>
>>>>
>>>> 1. Invoke 'emacs-nox -Q' in mintty.
>>>>
>>>> 2. M-x compile C-a C-k ls RET
>>>>
>>>> 3. C-x o
>>>>
>>>> 4. Hit 'g' repeatedly.
>>>>
>>>> I got it to abort with Fatal error 6 after slightly over 100 repetitions.
>>>>
>>>> I then tried the same thing with emacs-X11 (running under X, not in
>>>> mintty). I hit 'g' 200 times without a problem. I repeated this with
>>>> emacs-w32, again 200 times without a problem.
>>>>
>>>> So there's a bug somewhere. But if it's an emacs bug, it's strange that
>>>> it only occurs with emacs-nox and not with either of the GUI versions of
>>>> emacs.
>>> I suspect that buffer relocation might be the reason. Can you show a
>>> backtrace from the fatal error in an unoptimized build, with the above
>>> recipe?
>> I'll try to get one. Meanwhile, is there a quick way to disable buffer
>> relocation as a sanity test?
>
> I don't think so, but maybe it's already disabled in your build. What
> do you see on these two lines when configure finishes:
>
> Should Emacs use a relocating allocator for buffers? yes
> Should Emacs use mmap(2) for buffer allocation? no
Should Emacs use a relocating allocator for buffers? no
Should Emacs use mmap(2) for buffer allocation? yes
> In any case, you could configure with --enable-checking=all, it might
> catch the problem earlier.
>
>> If the optimized build still crashes reliably without it, the real
>> problem is probably something else.
>
> Maybe, but not necessarily.
FWIW, I just tried this on the trunk, both with and without
optimization. The bug is still there in the optimized build, but I
couldn't trigger it in the unoptimized build. I've also checked that
the bug occurs only in 64-bit Cygwin, not 32-bit Cygwin.
Here's some speculation. On 64-bit Cygwin, the space allocated to the
heap and mmap goes from 0x6:00000000 to 0x700:00000000, with the heap
starting at the bottom and mmaps allocated from the top down. (I've
added the colons for readability). Is it possible that the optimized
build takes shortcuts that are somehow incompatible with this model?
I'm grasping at straws here, but I'm just trying to think of things that
differ between the 32-bit and 64-bit systems.
Ken
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-16 14:20 ` Ken Brown
@ 2013-08-16 14:24 ` Ryan Johnson
2013-08-16 15:03 ` Eli Zaretskii
2013-08-16 15:45 ` Eli Zaretskii
2 siblings, 0 replies; 27+ messages in thread
From: Ryan Johnson @ 2013-08-16 14:24 UTC (permalink / raw
To: Ken Brown; +Cc: Eli Zaretskii, emacs-devel
On 16/08/2013 10:20 AM, Ken Brown wrote:
> On 8/16/2013 9:31 AM, Eli Zaretskii wrote:
>>> Date: Fri, 16 Aug 2013 07:41:44 -0400
>>> From: Ryan Johnson <ryan.johnson@cs.utoronto.ca>
>>>
>>> On 16/08/2013 4:58 AM, Eli Zaretskii wrote:
>>>> Again, please move this discussion to emacs-devel.
>>> Back pointer: http://cygwin.com/ml/cygwin/2013-08/msg00273.html
>>>
>>>>> Date: Thu, 15 Aug 2013 22:35:54 -0400
>>>>> From: Ken Brown <kbrown@cornell.edu>
>>>>>
>>>>> 1. Invoke 'emacs-nox -Q' in mintty.
>>>>>
>>>>> 2. M-x compile C-a C-k ls RET
>>>>>
>>>>> 3. C-x o
>>>>>
>>>>> 4. Hit 'g' repeatedly.
>>>>>
>>>>> I got it to abort with Fatal error 6 after slightly over 100
>>>>> repetitions.
>>>>>
>>>>> I then tried the same thing with emacs-X11 (running under X, not in
>>>>> mintty). I hit 'g' 200 times without a problem. I repeated this
>>>>> with
>>>>> emacs-w32, again 200 times without a problem.
>>>>>
>>>>> So there's a bug somewhere. But if it's an emacs bug, it's
>>>>> strange that
>>>>> it only occurs with emacs-nox and not with either of the GUI
>>>>> versions of
>>>>> emacs.
>>>> I suspect that buffer relocation might be the reason. Can you show a
>>>> backtrace from the fatal error in an unoptimized build, with the above
>>>> recipe?
>>> I'll try to get one. Meanwhile, is there a quick way to disable buffer
>>> relocation as a sanity test?
>>
>> I don't think so, but maybe it's already disabled in your build. What
>> do you see on these two lines when configure finishes:
>>
>> Should Emacs use a relocating allocator for buffers? yes
>> Should Emacs use mmap(2) for buffer allocation? no
>
> Should Emacs use a relocating allocator for buffers? no
> Should Emacs use mmap(2) for buffer allocation? yes
>
>> In any case, you could configure with --enable-checking=all, it might
>> catch the problem earlier.
>>
>>> If the optimized build still crashes reliably without it, the real
>>> problem is probably something else.
>>
>> Maybe, but not necessarily.
>
> FWIW, I just tried this on the trunk, both with and without
> optimization. The bug is still there in the optimized build, but I
> couldn't trigger it in the unoptimized build. I've also checked that
> the bug occurs only in 64-bit Cygwin, not 32-bit Cygwin.
>
> Here's some speculation. On 64-bit Cygwin, the space allocated to the
> heap and mmap goes from 0x6:00000000 to 0x700:00000000, with the heap
> starting at the bottom and mmaps allocated from the top down. (I've
> added the colons for readability). Is it possible that the optimized
> build takes shortcuts that are somehow incompatible with this model?
>
> I'm grasping at straws here, but I'm just trying to think of things
> that differ between the 32-bit and 64-bit systems.
I also suspect that the black magic cast by emacs and cygwin clash
somehow for 64-bit builds... some assumption the one makes that the
other breaks... but who knows which which assumption or which one breaks
it.
I also cannot get the unoptimized build to crash today; I'm not sure
what was going on yesterday that made it unstable, but the unwanted
reboot during the night seems to have cleared up whatever it was.
Ryan
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-16 14:16 ` Ryan Johnson
@ 2013-08-16 14:49 ` Eli Zaretskii
0 siblings, 0 replies; 27+ messages in thread
From: Eli Zaretskii @ 2013-08-16 14:49 UTC (permalink / raw
To: Ryan Johnson; +Cc: emacs-devel
> Date: Fri, 16 Aug 2013 10:16:59 -0400
> From: Ryan Johnson <ryan.johnson@cs.utoronto.ca>
> CC: emacs-devel@gnu.org
>
> Should Emacs use the GNU version of malloc? yes
> Should Emacs use a relocating allocator for buffers? no
> Should Emacs use mmap(2) for buffer allocation? yes
Then I think relocation is not a problem in your case.
> Hmm... during bootstrap I got the following with checking enabled:
> > Loading loadup.el (source)...
> > Using load-path (/scratch/emacs-24.3/lisp
> > /scratch/emacs-24.3/lisp/emacs-lisp /scratch/emacs-24.3/lisp/language
> > /scratch/emacs-24.3/lisp/international /scratch/emacs-24.3/lisp/textmodes)
> > Loading emacs-lisp/byte-run...
> > Loading emacs-lisp/backquote...
> > Loading subr...
> > Loading version...
> > <** snip **>
> > Loading international/mule-cmds...
> > Loading case-table...
> > Loading /scratch/emacs-24.3/lisp/international/charprop.el (source)...
> > Loading international/characters...
> > static heap exhausted: avail 10485760 used 9751456 failed request 790528
> > Makefile:795: recipe for target `bootstrap-emacs.exe' failed
Enlarge the value of STATIC_HEAP_SIZE in src/sheap.c (the __x86_64__
branch).
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-16 14:20 ` Ken Brown
2013-08-16 14:24 ` Ryan Johnson
@ 2013-08-16 15:03 ` Eli Zaretskii
2013-08-16 15:45 ` Eli Zaretskii
2 siblings, 0 replies; 27+ messages in thread
From: Eli Zaretskii @ 2013-08-16 15:03 UTC (permalink / raw
To: Ken Brown; +Cc: ryan.johnson, emacs-devel
> Date: Fri, 16 Aug 2013 10:20:00 -0400
> From: Ken Brown <kbrown@cornell.edu>
> CC: Ryan Johnson <ryan.johnson@cs.utoronto.ca>, emacs-devel@gnu.org
>
> Here's some speculation. On 64-bit Cygwin, the space allocated to the
> heap and mmap goes from 0x6:00000000 to 0x700:00000000, with the heap
> starting at the bottom and mmaps allocated from the top down. (I've
> added the colons for readability). Is it possible that the optimized
> build takes shortcuts that are somehow incompatible with this model?
I don't see how optimizations can take any shortcuts when a function
call is involved. But maybe I'm missing something, and someone else
will have an idea.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-16 14:20 ` Ken Brown
2013-08-16 14:24 ` Ryan Johnson
2013-08-16 15:03 ` Eli Zaretskii
@ 2013-08-16 15:45 ` Eli Zaretskii
2013-08-16 16:51 ` Ryan Johnson
` (2 more replies)
2 siblings, 3 replies; 27+ messages in thread
From: Eli Zaretskii @ 2013-08-16 15:45 UTC (permalink / raw
To: Ken Brown; +Cc: ryan.johnson, emacs-devel
> Date: Fri, 16 Aug 2013 10:20:00 -0400
> From: Ken Brown <kbrown@cornell.edu>
> CC: Ryan Johnson <ryan.johnson@cs.utoronto.ca>, emacs-devel@gnu.org
>
> FWIW, I just tried this on the trunk, both with and without
> optimization. The bug is still there in the optimized build
Can you post a full backtrace from the crashed session?
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-16 15:45 ` Eli Zaretskii
@ 2013-08-16 16:51 ` Ryan Johnson
2013-08-16 17:46 ` Ken Brown
[not found] ` <520E5D71.3020307@cornell.edu>
2 siblings, 0 replies; 27+ messages in thread
From: Ryan Johnson @ 2013-08-16 16:51 UTC (permalink / raw
To: Eli Zaretskii; +Cc: Ken Brown, emacs-devel
On 16/08/2013 11:45 AM, Eli Zaretskii wrote:
>> Date: Fri, 16 Aug 2013 10:20:00 -0400
>> From: Ken Brown <kbrown@cornell.edu>
>> CC: Ryan Johnson <ryan.johnson@cs.utoronto.ca>, emacs-devel@gnu.org
>>
>> FWIW, I just tried this on the trunk, both with and without
>> optimization. The bug is still there in the optimized build
> Can you post a full backtrace from the crashed session?
I finally intercepted a SIGSEGV in gdb, from a debug-mode emacs. It's
pretty hard to repro (100-200 M-x compile cycles), so be patient...
Here's the stack trace (based on the emacs-24.3-4 source package on the
cygwin64 setup).
As usual, we've dereferenced a NULL pointer just after verifying that it
was not NULL. This is becoming a repeating theme; are there any
asynchronous actors in emacs that might invoke GC or otherwise frob
pointers?
Initial analysis follows:
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 1280.0xfc4]
> previous_interval (interval=0x6002181e0) at intervals.c:758
> 758 while (! NULL_RIGHT_CHILD (i))
> (gdb) p i
> $1 = (INTERVAL) 0x0
> (gdb) p interval
> $2 = (INTERVAL) 0x6002181e0
> (gdb) p *interval
> $3 = {total_length = 355, position = 99, left = 0x600218090, right =
> 0x0, up = {interval = 0x600070425, obj = 25770263589}, up_obj = 1,
> gcmarkbit = 0, write_protect = 0, visible = 0, front_sticky = 0,
> rear_sticky = 0, plist = 4307415122}
> (gdb) p interval->left->right
> $5 = (struct interval *) 0x600218138
> (gdb) p interval->left->right->right
> $6 = (struct interval *) 0x600218170
> (gdb) p interval->left->right->right->right
> $7 = (struct interval *) 0x6002181a8
> (gdb) p interval->left->right->right->right->right
> $8 = (struct interval *) 0x0
The relevant disassembly is:
> (gdb) disas previous_interval,+60
> 0x00000001006c1546 <previous_interval+0>: push %rbp
> 0x00000001006c1547 <previous_interval+1>: push %rbx
> 0x00000001006c1548 <previous_interval+2>: sub $0x28,%rsp
> 0x00000001006c154c <previous_interval+6>: lea 0x80(%rsp),%rbp
> 0x00000001006c1554 <previous_interval+14>: mov %rcx,%rax
> 0x00000001006c1557 <previous_interval+17>: test %rax,%rax
> 0x00000001006c155a <previous_interval+20>: jne 0x1006c1566
> <previous_interval+32>
%rax is the variable "interval", and if non-zero we jump to +32
> 0x00000001006c155c <previous_interval+22>: mov $0x0,%eax
> 0x00000001006c1561 <previous_interval+27>: jmpq 0x1006c1757
> <previous_interval+529>
> 0x00000001006c1566 <previous_interval+32>: mov 0x10(%rax),%rdx
> 0x00000001006c156a <previous_interval+36>: test %rdx,%rdx
> 0x00000001006c156d <previous_interval+39>: je 0x1006c15ea
> <previous_interval+164>
%rdx is interval->left, and if non-zero we fall through to +41
> 0x00000001006c156f <previous_interval+41>: mov 0x10(%rax),%rbx
> 0x00000001006c1573 <previous_interval+45>: jmp 0x1006c1579
> <previous_interval+51>
copy interval->left into %rbx as initial value of "i" and jump into loop
at +51
> 0x00000001006c1575 <previous_interval+47>: mov 0x18(%rbx),%rbx
loop body: set i = i->right
> => 0x00000001006c1579 <previous_interval+51>: mov 0x18(%rbx),%rdx
> 0x00000001006c157d <previous_interval+55>: test %rdx,%rdx
> 0x00000001006c1580 <previous_interval+58>: jne 0x1006c1575
> <previous_interval+47>
Loop test: if i->right != 0, execute the loop body, which loads i' =
i->right and then tests i'->right (i->right->right). Clobbers %rdx.
Overall the code seems fine. As usual, we loaded a non-NULL value into a
register (now clobbered, unfortunately) from a memory location that was
then zeroed out before the compiler could reload it into a second register.
The full backtrace is below, and I've got the session frozen in gdb. Let
me know if you want me to dig into anything else.
Ryan
(gdb) bt
#0 previous_interval (interval=0x6002181e0) at intervals.c:758
#1 0x00000001006c5301 in set_point_both (charpos=99, bytepos=99) at
intervals.c:1917
#2 0x00000001006c4a0e in set_point (charpos=99) at intervals.c:1820
#3 0x000000010060d567 in Fgoto_char (position=396) at editfns.c:242
#4 0x000000010069f74d in exec_byte_code (bytestr=4303941777,
vector=4303941957, maxdepth=24, args_template=4307415122, nargs=0,
args=0x0) at bytecode.c:1480
#5 0x0000000100629880 in funcall_lambda (fun=4303941725, nargs=3,
arg_vector=0x227718) at eval.c:3010
#6 0x0000000100628b49 in Ffuncall (nargs=4, args=0x227710) at eval.c:2827
#7 0x0000000100627464 in funcall_nil (nargs=4, args=0x227710) at
eval.c:2324
#8 0x00000001006279bf in run_hook_with_args (nargs=4, args=0x227710,
funcall=0x100627444 <funcall_nil>) at eval.c:2509
#9 0x0000000100627506 in Frun_hook_with_args (nargs=4, args=0x227710)
at eval.c:2370
#10 0x000000010062833d in Ffuncall (nargs=5, args=0x227708) at eval.c:2759
#11 0x000000010069ccce in exec_byte_code (bytestr=4303972225,
vector=4303972469, maxdepth=24, args_template=4307415122, nargs=0,
args=0x0) at bytecode.c:900
#12 0x0000000100629880 in funcall_lambda (fun=4303972157, nargs=3,
arg_vector=0x227d58) at eval.c:3010
#13 0x0000000100628b49 in Ffuncall (nargs=4, args=0x227d50) at eval.c:2827
#14 0x0000000100627464 in funcall_nil (nargs=4, args=0x227d50) at
eval.c:2324
#15 0x00000001006279bf in run_hook_with_args (nargs=4, args=0x227d50,
funcall=0x100627444 <funcall_nil>) at eval.c:2509
#16 0x0000000100627506 in Frun_hook_with_args (nargs=4, args=0x227d50)
at eval.c:2370
#17 0x0000000100599cd0 in signal_after_change (charpos=99, lendel=0,
lenins=257) at insdel.c:2058
#18 0x0000000100594fa9 in insert_from_string (string=25771578241, pos=0,
pos_byte=0, length=257, length_byte=257, inherit=false) at insdel.c:873
#19 0x0000000100613151 in general_insert_function
(insert_func=0x1005944ec <insert>, insert_from_string_func=0x100594ec8
<insert_from_string>, inherit=false, nargs=1, args=0x227ef8) at
editfns.c:2258
#20 0x00000001006131da in Finsert (nargs=1, args=0x227ef8) at editfns.c:2299
#21 0x000000010069f7a3 in exec_byte_code (bytestr=25772122625,
vector=4313861789, maxdepth=28, args_template=4307415122, nargs=0,
args=0x0) at bytecode.c:1486
#22 0x0000000100629880 in funcall_lambda (fun=4313861965, nargs=2,
arg_vector=0x228428) at eval.c:3010
#23 0x0000000100628b49 in Ffuncall (nargs=3, args=0x228420) at eval.c:2827
#24 0x0000000100627415 in Fapply (nargs=2, args=0x2284f0) at eval.c:2312
#25 0x0000000100627ae3 in apply1 (fn=25772023938, arg=4318413830) at
eval.c:2546
#26 0x00000001006af86e in read_process_output_call
(fun_and_args=4318413846) at process.c:5022
#27 0x0000000100623506 in internal_condition_case_1 (bfun=0x1006af7d7
<read_process_output_call>, arg=4318413846, handlers=4307507906,
hfun=0x1006af874 <read_process_output_error_handler>) at eval.c:1327
#28 0x00000001006b00bd in read_process_output (proc=4316041005,
channel=257) at process.c:5221
#29 0x00000001006aedb5 in wait_reading_process_output (time_limit=28,
nsecs=0, read_kbd=-1, do_display=true, wait_for_cell=4307415122,
wait_proc=0x0, just_wait_proc=0) at process.c:4852
#30 0x0000000100412d28 in sit_for (timeout=112, reading=true,
display_option=1) at dispnew.c:5978
#31 0x0000000100542d2d in read_char (commandflag=1, nmaps=2,
maps=0x229f90, prev_event=4307415122, used_mouse_menu=0x22a157,
end_time=0x0) at keyboard.c:2669
#32 0x00000001005587ba in read_key_sequence (keybuf=0x22a400,
bufsize=30, prompt=4307415122, dont_downcase_last=false,
can_return_switch_frame=true, fix_current_buffer=true) at keyboard.c:9231
#33 0x000000010053e9c9 in command_loop_1 () at keyboard.c:1459
#34 0x0000000100623376 in internal_condition_case (bfun=0x10053e29a
<command_loop_1>, handlers=4307507906, hfun=0x10053d7bb <cmd_error>) at
eval.c:1289
#35 0x000000010053de20 in command_loop_2 (ignore=4307415122) at
keyboard.c:1168
#36 0x0000000100622bc2 in internal_catch (tag=4307491714,
func=0x10053ddee <command_loop_2>, arg=4307415122) at eval.c:1060
#37 0x000000010053ddaf in command_loop () at keyboard.c:1147
#38 0x000000010053ceea in recursive_edit_1 () at keyboard.c:779
#39 0x000000010053d371 in Frecursive_edit () at keyboard.c:843
#40 0x000000010053a687 in main (argc=2, argv=0x22ab20) at emacs.c:1532
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
[not found] ` <520E5D71.3020307@cornell.edu>
@ 2013-08-16 17:24 ` Ryan Johnson
2013-08-16 18:55 ` Ken Brown
2013-08-16 19:37 ` Eli Zaretskii
1 sibling, 1 reply; 27+ messages in thread
From: Ryan Johnson @ 2013-08-16 17:24 UTC (permalink / raw
To: Ken Brown; +Cc: Eli Zaretskii, emacs-devel
On 16/08/2013 1:12 PM, Ken Brown wrote:
> On 8/16/2013 11:45 AM, Eli Zaretskii wrote:
>>> Date: Fri, 16 Aug 2013 10:20:00 -0400
>>> From: Ken Brown <kbrown@cornell.edu>
>>> CC: Ryan Johnson <ryan.johnson@cs.utoronto.ca>, emacs-devel@gnu.org
>>>
>>> FWIW, I just tried this on the trunk, both with and without
>>> optimization. The bug is still there in the optimized build
>>
>> Can you post a full backtrace from the crashed session?
>
> I'm attaching two of them. The first was a SEGV, the second was an
> abort.
Nice, those are two I've seen before.
BTW, how did you get the backtrace to dump so much info?
Ryan
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-16 15:45 ` Eli Zaretskii
2013-08-16 16:51 ` Ryan Johnson
@ 2013-08-16 17:46 ` Ken Brown
[not found] ` <520E5D71.3020307@cornell.edu>
2 siblings, 0 replies; 27+ messages in thread
From: Ken Brown @ 2013-08-16 17:46 UTC (permalink / raw
To: Eli Zaretskii; +Cc: ryan.johnson, emacs-devel
[-- Attachment #1: Type: text/plain, Size: 468 bytes --]
On 8/16/2013 11:45 AM, Eli Zaretskii wrote:
>> Date: Fri, 16 Aug 2013 10:20:00 -0400
>> From: Ken Brown <kbrown@cornell.edu>
>> CC: Ryan Johnson <ryan.johnson@cs.utoronto.ca>, emacs-devel@gnu.org
>>
>> FWIW, I just tried this on the trunk, both with and without
>> optimization. The bug is still there in the optimized build
>
> Can you post a full backtrace from the crashed session?
I'm attaching two of them. The first was a SEGV, the second was an abort.
Ken
[-- Attachment #2: gdb1.txt.gz --]
[-- Type: application/octet-stream, Size: 15177 bytes --]
[-- Attachment #3: gdb2.txt.gz --]
[-- Type: application/octet-stream, Size: 16458 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-16 17:24 ` Ryan Johnson
@ 2013-08-16 18:55 ` Ken Brown
0 siblings, 0 replies; 27+ messages in thread
From: Ken Brown @ 2013-08-16 18:55 UTC (permalink / raw
To: Ryan Johnson; +Cc: Eli Zaretskii, emacs-devel
On 8/16/2013 1:24 PM, Ryan Johnson wrote:
> On 16/08/2013 1:12 PM, Ken Brown wrote:
>> On 8/16/2013 11:45 AM, Eli Zaretskii wrote:
>>>> Date: Fri, 16 Aug 2013 10:20:00 -0400
>>>> From: Ken Brown <kbrown@cornell.edu>
>>>> CC: Ryan Johnson <ryan.johnson@cs.utoronto.ca>, emacs-devel@gnu.org
>>>>
>>>> FWIW, I just tried this on the trunk, both with and without
>>>> optimization. The bug is still there in the optimized build
>>>
>>> Can you post a full backtrace from the crashed session?
>>
>> I'm attaching two of them. The first was a SEGV, the second was an
>> abort.
> Nice, those are two I've seen before.
>
> BTW, how did you get the backtrace to dump so much info?
I just typed 'bt full'. I was running gdb from the emacs src directory
(the one containing .gdbinit). I don't know if that's relevant.
Ken
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
[not found] ` <520E5D71.3020307@cornell.edu>
2013-08-16 17:24 ` Ryan Johnson
@ 2013-08-16 19:37 ` Eli Zaretskii
2013-08-16 20:17 ` Eli Zaretskii
2013-08-16 20:33 ` Ken Brown
1 sibling, 2 replies; 27+ messages in thread
From: Eli Zaretskii @ 2013-08-16 19:37 UTC (permalink / raw
To: Ken Brown; +Cc: ryan.johnson, emacs-devel
> Date: Fri, 16 Aug 2013 13:12:17 -0400
> From: Ken Brown <kbrown@cornell.edu>
> CC: ryan.johnson@cs.utoronto.ca, emacs-devel@gnu.org
>
> I'm attaching two of them. The first was a SEGV, the second was an abort.
Both in regex.c.
> Lisp Backtrace:
>
> Breakpoint 1, terminate_due_to_signal (sig=6, backtrace_limit=2147483647)
> at emacs.c:341
> 341 {
> The program being debugged stopped while in a function called from GDB.
> Evaluation of the expression containing the function
> (backtrace_function) will be abandoned.
> When the function is done executing, GDB will silently stop.
Here, GDB called backtrace_function, which triggered an assertion
violation. Which probably means that the array of specbindings
(specpdl) is corrupted.
Is it possible that the stack of the 64-bit Cygwin binary is too
small? Can you try building Emacs with a larger stack?
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-16 19:37 ` Eli Zaretskii
@ 2013-08-16 20:17 ` Eli Zaretskii
2013-08-16 20:33 ` Ken Brown
1 sibling, 0 replies; 27+ messages in thread
From: Eli Zaretskii @ 2013-08-16 20:17 UTC (permalink / raw
To: kbrown; +Cc: ryan.johnson, emacs-devel
> Date: Fri, 16 Aug 2013 22:37:15 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: ryan.johnson@cs.utoronto.ca, emacs-devel@gnu.org
>
> > Lisp Backtrace:
> >
> > Breakpoint 1, terminate_due_to_signal (sig=6, backtrace_limit=2147483647)
> > at emacs.c:341
> > 341 {
> > The program being debugged stopped while in a function called from GDB.
> > Evaluation of the expression containing the function
> > (backtrace_function) will be abandoned.
> > When the function is done executing, GDB will silently stop.
>
> Here, GDB called backtrace_function, which triggered an assertion
> violation. Which probably means that the array of specbindings
> (specpdl) is corrupted.
>
> Is it possible that the stack of the 64-bit Cygwin binary is too
> small? Can you try building Emacs with a larger stack?
Another idea is to look at the value of specpdl_size at the point
where Emacs crashes or aborts. Is it perhaps close to the value of
max_specpdl_size?
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-16 19:37 ` Eli Zaretskii
2013-08-16 20:17 ` Eli Zaretskii
@ 2013-08-16 20:33 ` Ken Brown
2013-08-16 21:20 ` Ryan Johnson
2013-08-17 7:01 ` Eli Zaretskii
1 sibling, 2 replies; 27+ messages in thread
From: Ken Brown @ 2013-08-16 20:33 UTC (permalink / raw
To: Eli Zaretskii; +Cc: ryan.johnson, emacs-devel
On 8/16/2013 3:37 PM, Eli Zaretskii wrote:
>> Date: Fri, 16 Aug 2013 13:12:17 -0400
>> From: Ken Brown <kbrown@cornell.edu>
>> CC: ryan.johnson@cs.utoronto.ca, emacs-devel@gnu.org
>>
>> I'm attaching two of them. The first was a SEGV, the second was an abort.
>
> Both in regex.c.
>
>> Lisp Backtrace:
>>
>> Breakpoint 1, terminate_due_to_signal (sig=6, backtrace_limit=2147483647)
>> at emacs.c:341
>> 341 {
>> The program being debugged stopped while in a function called from GDB.
>> Evaluation of the expression containing the function
>> (backtrace_function) will be abandoned.
>> When the function is done executing, GDB will silently stop.
>
> Here, GDB called backtrace_function, which triggered an assertion
> violation. Which probably means that the array of specbindings
> (specpdl) is corrupted.
>
> Is it possible that the stack of the 64-bit Cygwin binary is too
> small? Can you try building Emacs with a larger stack?
You are brilliant! I added LDFLAGS='-Wl,--stack,4194304' to the
compilation, and the bug is gone. Thank you.
Ryan, can you confirm that this fixes the problem for your real uses of
M-x compile?
Ken
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-16 20:33 ` Ken Brown
@ 2013-08-16 21:20 ` Ryan Johnson
2013-08-17 7:01 ` Eli Zaretskii
1 sibling, 0 replies; 27+ messages in thread
From: Ryan Johnson @ 2013-08-16 21:20 UTC (permalink / raw
To: Ken Brown; +Cc: Eli Zaretskii, emacs-devel
On 16/08/2013 4:33 PM, Ken Brown wrote:
> On 8/16/2013 3:37 PM, Eli Zaretskii wrote:
>>> Date: Fri, 16 Aug 2013 13:12:17 -0400
>>> From: Ken Brown <kbrown@cornell.edu>
>>> CC: ryan.johnson@cs.utoronto.ca, emacs-devel@gnu.org
>>>
>>> I'm attaching two of them. The first was a SEGV, the second was an
>>> abort.
>>
>> Both in regex.c.
>>
>>> Lisp Backtrace:
>>>
>>> Breakpoint 1, terminate_due_to_signal (sig=6,
>>> backtrace_limit=2147483647)
>>> at emacs.c:341
>>> 341 {
>>> The program being debugged stopped while in a function called from GDB.
>>> Evaluation of the expression containing the function
>>> (backtrace_function) will be abandoned.
>>> When the function is done executing, GDB will silently stop.
>>
>> Here, GDB called backtrace_function, which triggered an assertion
>> violation. Which probably means that the array of specbindings
>> (specpdl) is corrupted.
>>
>> Is it possible that the stack of the 64-bit Cygwin binary is too
>> small? Can you try building Emacs with a larger stack?
>
> You are brilliant! I added LDFLAGS='-Wl,--stack,4194304' to the
> compilation, and the bug is gone. Thank you.
>
> Ryan, can you confirm that this fixes the problem for your real uses
> of M-x compile?
I've been using it for an hour or so with no problems. Even the
optimized build refuses to crash in 200-300 M-x compile cycles, which is
enough to satisfy me.
Thanks for all the help, I was pulling my hair out with this one...
Ryan
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-16 20:33 ` Ken Brown
2013-08-16 21:20 ` Ryan Johnson
@ 2013-08-17 7:01 ` Eli Zaretskii
2013-08-17 12:17 ` Ken Brown
1 sibling, 1 reply; 27+ messages in thread
From: Eli Zaretskii @ 2013-08-17 7:01 UTC (permalink / raw
To: Ken Brown; +Cc: ryan.johnson, emacs-devel
> Date: Fri, 16 Aug 2013 16:33:51 -0400
> From: Ken Brown <kbrown@cornell.edu>
> CC: ryan.johnson@cs.utoronto.ca, emacs-devel@gnu.org
>
> > Is it possible that the stack of the 64-bit Cygwin binary is too
> > small? Can you try building Emacs with a larger stack?
>
> You are brilliant!
I wish this were true. I'm just a guy with a lot of gray hair. But
thanks anyway.
> I added LDFLAGS='-Wl,--stack,4194304' to the compilation, and the
> bug is gone. Thank you.
So what size of stack was Emacs using before that?
FWIW, even the 32-bit native Windows build uses twice that number
(8MB), so perhaps enlarge the parameter even more.
Also please note that Ryan couldn't build with --enable-checking=all
because the value of STATIC_HEAP_SIZE in sheap.c was too small. I
suggest to enlarge it as needed.
Thanks.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-17 7:01 ` Eli Zaretskii
@ 2013-08-17 12:17 ` Ken Brown
0 siblings, 0 replies; 27+ messages in thread
From: Ken Brown @ 2013-08-17 12:17 UTC (permalink / raw
To: Eli Zaretskii; +Cc: ryan.johnson, emacs-devel
On 8/17/2013 3:01 AM, Eli Zaretskii wrote:
>> Date: Fri, 16 Aug 2013 16:33:51 -0400
>> From: Ken Brown <kbrown@cornell.edu>
>> I added LDFLAGS='-Wl,--stack,4194304' to the compilation, and the
>> bug is gone. Thank you.
>
> So what size of stack was Emacs using before that?
Whatever is the default on Cygwin; I never used '--stack' to change it
until yesterday. 'ulimit -s' returns 2025, so I guess that means the
default stack is only 2MB. I've asked on the Cygwin list whether this
should be increased.
> FWIW, even the 32-bit native Windows build uses twice that number
> (8MB), so perhaps enlarge the parameter even more.
OK.
> Also please note that Ryan couldn't build with --enable-checking=all
> because the value of STATIC_HEAP_SIZE in sheap.c was too small. I
> suggest to enlarge it as needed.
The reason Ryan had to increase it is that he was building from my
patched sources for the Cygwin distribution. I deliberately reduce the
static heap for that build, because building from the source tarball
doesn't require as big a static heap as a bootstrap. I bootstrapped the
trunk yesterday with --enable-checking=all and the static heap was big
enough for that (but just barely).
Thanks.
Ken
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
@ 2013-08-17 19:43 Angelo Graziosi
2013-08-17 20:16 ` Ken Brown
0 siblings, 1 reply; 27+ messages in thread
From: Angelo Graziosi @ 2013-08-17 19:43 UTC (permalink / raw
To: emacs
Ken Brown wrote:
> I bootstrapped the trunk yesterday with --enable-checking=all
Just for the record...
I tried to bootstrap (on XP 32) with --enable-checking=all, but it fails
always in this way (I tried also with no parallel build):
[...]
Loading window (source)...
Loading /work/emacs/lisp/files.el (source)...
Loading /work/emacs/lisp/emacs-lisp/macroexp.el (source)...
Loading /work/emacs/lisp/emacs-lisp/pcase.el (source)...
Loading /work/emacs/lisp/emacs-lisp/macroexp.el (source)...
Loading /work/emacs/lisp/cus-face.el (source)...
Loading /work/emacs/lisp/faces.el (source)...
Loading /work/emacs/lisp/button.el (source)...
Loading /work/emacs/lisp/startup.el (source)...
Loading /work/emacs/lisp/ldefs-boot.el (source)...
Loading /work/emacs/lisp/emacs-lisp/nadvice.el (source)...
Loading /work/emacs/lisp/minibuffer.el (source)...
../../lisp/minibuffer.el: `with-wrapper-hook' is an obsolete macro (as
of 24.4); use a <foo>-function variable modified by add-function.
Loading /work/emacs/lisp/abbrev.el (source)...
../../lisp/abbrev.el: `with-wrapper-hook' is an obsolete macro (as of
24.4); use a <foo>-function variable modified by add-function.
Loading /work/emacs/lisp/simple.el (source)...
../../lisp/simple.el: `with-wrapper-hook' is an obsolete macro (as of
24.4); use a <foo>-function variable modified by add-function.
Loading /work/emacs/lisp/help.el (source)...
Loading /work/emacs/lisp/jka-cmpr-hook.el (source)...
Loading /work/emacs/lisp/epa-hook.el (source)...
Loading /work/emacs/lisp/international/mule-cmds.el (source)...
Loading /work/emacs/lisp/case-table.el (source)...
Loading /work/emacs/lisp/international/charprop.el (source)...
Loading /work/emacs/lisp/international/characters.el (source)...
Loading /work/emacs/lisp/composite.el (source)...
Loading /work/emacs/lisp/language/chinese.el (source)...
Loading /work/emacs/lisp/language/cyrillic.el (source)...
Loading /work/emacs/lisp/language/indian.el (source)...
Loading /work/emacs/lisp/language/sinhala.el (source)...
Loading /work/emacs/lisp/language/english.el (source)...
Loading /work/emacs/lisp/language/ethiopic.el (source)...
Loading /work/emacs/lisp/language/european.el (source)...
Loading /work/emacs/lisp/language/czech.el (source)...
Loading /work/emacs/lisp/language/slovak.el (source)...
Loading /work/emacs/lisp/language/romanian.el (source)...
Loading /work/emacs/lisp/language/greek.el (source)...
Loading /work/emacs/lisp/language/hebrew.el (source)...
Loading /work/emacs/lisp/language/japanese.el (source)...
Loading /work/emacs/lisp/international/cp51932.el (source)...
Loading /work/emacs/lisp/international/eucjp-ms.el (source)...
Loading /work/emacs/lisp/language/korean.el (source)...
Loading /work/emacs/lisp/language/lao.el (source)...
Loading /work/emacs/lisp/language/tai-viet.el (source)...
Loading /work/emacs/lisp/language/thai.el (source)...
Loading /work/emacs/lisp/language/tibetan.el (source)...
Loading /work/emacs/lisp/language/vietnamese.el (source)...
Loading /work/emacs/lisp/language/misc-lang.el (source)...
Loading /work/emacs/lisp/language/utf-8-lang.el (source)...
Loading /work/emacs/lisp/language/georgian.el (source)...
Loading /work/emacs/lisp/language/khmer.el (source)...
Loading /work/emacs/lisp/language/burmese.el (source)...
Loading /work/emacs/lisp/language/cham.el (source)...
Loading /work/emacs/lisp/indent.el (source)...
Loading /work/emacs/lisp/frame.el (source)...
Loading /work/emacs/lisp/term/tty-colors.el (source)...
Loading /work/emacs/lisp/font-core.el (source)...
Loading /work/emacs/lisp/facemenu.el (source)...
Loading /work/emacs/lisp/emacs-lisp/syntax.el (source)...
Loading /work/emacs/lisp/font-lock.el (source)...
mv: impossibile eseguire stat di "emacs.exe": No such file or directory
Makefile:835: recipe for target `bootstrap-emacs.exe' failed
make[2]: *** [bootstrap-emacs.exe] Error 1
make[2]: uscita dalla directory "/work/emacs/Work/src"
Makefile:379: recipe for target `src' failed
make[1]: *** [src] Error 2
make[1]: uscita dalla directory "/work/emacs/Work"
Makefile:1050: recipe for target `bootstrap' failed
make: *** [bootstrap] Error 2
I tried different revisions which bootstrap fine without the checking
but fail as above when checking is enabled.
The bootstrap fails before it prints the STATIC_HEAP_SIZE information,
taking a lot of time..
Ciao,
Angelo.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-17 19:43 Angelo Graziosi
@ 2013-08-17 20:16 ` Ken Brown
2013-08-17 22:23 ` Angelo Graziosi
0 siblings, 1 reply; 27+ messages in thread
From: Ken Brown @ 2013-08-17 20:16 UTC (permalink / raw
To: Angelo Graziosi; +Cc: emacs
On 8/17/2013 3:43 PM, Angelo Graziosi wrote:
> Ken Brown wrote:
>> I bootstrapped the trunk yesterday with --enable-checking=all
>
> Just for the record...
>
> I tried to bootstrap (on XP 32) with --enable-checking=all, but it fails
> always in this way (I tried also with no parallel build):
>
> [...]
> Loading window (source)...
> Loading /work/emacs/lisp/files.el (source)...
> Loading /work/emacs/lisp/emacs-lisp/macroexp.el (source)...
> Loading /work/emacs/lisp/emacs-lisp/pcase.el (source)...
> Loading /work/emacs/lisp/emacs-lisp/macroexp.el (source)...
> Loading /work/emacs/lisp/cus-face.el (source)...
> Loading /work/emacs/lisp/faces.el (source)...
> Loading /work/emacs/lisp/button.el (source)...
> Loading /work/emacs/lisp/startup.el (source)...
> Loading /work/emacs/lisp/ldefs-boot.el (source)...
> Loading /work/emacs/lisp/emacs-lisp/nadvice.el (source)...
> Loading /work/emacs/lisp/minibuffer.el (source)...
> ../../lisp/minibuffer.el: `with-wrapper-hook' is an obsolete macro (as
> of 24.4); use a <foo>-function variable modified by add-function.
> Loading /work/emacs/lisp/abbrev.el (source)...
> ../../lisp/abbrev.el: `with-wrapper-hook' is an obsolete macro (as of
> 24.4); use a <foo>-function variable modified by add-function.
> Loading /work/emacs/lisp/simple.el (source)...
> ../../lisp/simple.el: `with-wrapper-hook' is an obsolete macro (as of
> 24.4); use a <foo>-function variable modified by add-function.
> Loading /work/emacs/lisp/help.el (source)...
> Loading /work/emacs/lisp/jka-cmpr-hook.el (source)...
> Loading /work/emacs/lisp/epa-hook.el (source)...
> Loading /work/emacs/lisp/international/mule-cmds.el (source)...
> Loading /work/emacs/lisp/case-table.el (source)...
> Loading /work/emacs/lisp/international/charprop.el (source)...
> Loading /work/emacs/lisp/international/characters.el (source)...
> Loading /work/emacs/lisp/composite.el (source)...
> Loading /work/emacs/lisp/language/chinese.el (source)...
> Loading /work/emacs/lisp/language/cyrillic.el (source)...
> Loading /work/emacs/lisp/language/indian.el (source)...
> Loading /work/emacs/lisp/language/sinhala.el (source)...
> Loading /work/emacs/lisp/language/english.el (source)...
> Loading /work/emacs/lisp/language/ethiopic.el (source)...
> Loading /work/emacs/lisp/language/european.el (source)...
> Loading /work/emacs/lisp/language/czech.el (source)...
> Loading /work/emacs/lisp/language/slovak.el (source)...
> Loading /work/emacs/lisp/language/romanian.el (source)...
> Loading /work/emacs/lisp/language/greek.el (source)...
> Loading /work/emacs/lisp/language/hebrew.el (source)...
> Loading /work/emacs/lisp/language/japanese.el (source)...
> Loading /work/emacs/lisp/international/cp51932.el (source)...
> Loading /work/emacs/lisp/international/eucjp-ms.el (source)...
> Loading /work/emacs/lisp/language/korean.el (source)...
> Loading /work/emacs/lisp/language/lao.el (source)...
> Loading /work/emacs/lisp/language/tai-viet.el (source)...
> Loading /work/emacs/lisp/language/thai.el (source)...
> Loading /work/emacs/lisp/language/tibetan.el (source)...
> Loading /work/emacs/lisp/language/vietnamese.el (source)...
> Loading /work/emacs/lisp/language/misc-lang.el (source)...
> Loading /work/emacs/lisp/language/utf-8-lang.el (source)...
> Loading /work/emacs/lisp/language/georgian.el (source)...
> Loading /work/emacs/lisp/language/khmer.el (source)...
> Loading /work/emacs/lisp/language/burmese.el (source)...
> Loading /work/emacs/lisp/language/cham.el (source)...
> Loading /work/emacs/lisp/indent.el (source)...
> Loading /work/emacs/lisp/frame.el (source)...
> Loading /work/emacs/lisp/term/tty-colors.el (source)...
> Loading /work/emacs/lisp/font-core.el (source)...
> Loading /work/emacs/lisp/facemenu.el (source)...
> Loading /work/emacs/lisp/emacs-lisp/syntax.el (source)...
> Loading /work/emacs/lisp/font-lock.el (source)...
> mv: impossibile eseguire stat di "emacs.exe": No such file or directory
> Makefile:835: recipe for target `bootstrap-emacs.exe' failed
> make[2]: *** [bootstrap-emacs.exe] Error 1
> make[2]: uscita dalla directory "/work/emacs/Work/src"
> Makefile:379: recipe for target `src' failed
> make[1]: *** [src] Error 2
> make[1]: uscita dalla directory "/work/emacs/Work"
> Makefile:1050: recipe for target `bootstrap' failed
> make: *** [bootstrap] Error 2
>
> I tried different revisions which bootstrap fine without the checking
> but fail as above when checking is enabled.
>
> The bootstrap fails before it prints the STATIC_HEAP_SIZE information,
> taking a lot of time..
This probably means that the static heap isn't big enough for
--enable-checking=all on 32-bit Cygwin. I'll test it and increase it if
necessary. I may not get to this today, but I'll do it soon. (And you
can do it yourself locally if you're anxious to bootstrap right now.)
Ken
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-17 20:16 ` Ken Brown
@ 2013-08-17 22:23 ` Angelo Graziosi
2013-08-18 17:43 ` Ken Brown
0 siblings, 1 reply; 27+ messages in thread
From: Angelo Graziosi @ 2013-08-17 22:23 UTC (permalink / raw
To: Ken Brown; +Cc: emacs
Il 17/08/2013 22.16, Ken Brown ha scritto:
>
> This probably means that the static heap isn't big enough for
> --enable-checking=all on 32-bit Cygwin. I'll test it and increase it if
> necessary. I may not get to this today, but I'll do it soon. (And you
> can do it yourself locally if you're anxious to bootstrap right now.)
OK, with 18MB it is still bootstrapping and has printed:
...
Finding pointers to doc strings...done
Dumping under the name emacs
Static heap usage: 16099796 of 18874368 bytes
...
but... really should we increase the static heap from 13 to 18 MB also
for a not-checking bootstrap?
Usually, in that case, the build log shows
...
Static heap usage: 10273280 of 13631488 bytes
...
Static heap usage: 5419520 of 13631488 bytes
Doesn't it mean that the "end" product uses only about 5.2 MB?
Ciao,
Angelo.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-17 22:23 ` Angelo Graziosi
@ 2013-08-18 17:43 ` Ken Brown
2013-08-18 19:10 ` Angelo Graziosi
2013-08-18 19:14 ` Eli Zaretskii
0 siblings, 2 replies; 27+ messages in thread
From: Ken Brown @ 2013-08-18 17:43 UTC (permalink / raw
To: Angelo Graziosi; +Cc: emacs
On 8/17/2013 6:23 PM, Angelo Graziosi wrote:
> Il 17/08/2013 22.16, Ken Brown ha scritto:
>>
>> This probably means that the static heap isn't big enough for
>> --enable-checking=all on 32-bit Cygwin. I'll test it and increase it if
>> necessary. I may not get to this today, but I'll do it soon. (And you
>> can do it yourself locally if you're anxious to bootstrap right now.)
>
> OK, with 18MB it is still bootstrapping and has printed:
>
> ...
> Finding pointers to doc strings...done
> Dumping under the name emacs
> Static heap usage: 16099796 of 18874368 bytes
> ...
>
> but... really should we increase the static heap from 13 to 18 MB also
> for a not-checking bootstrap?
No. I made the increase to 18 conditional on ENABLE_CHECKING.
> Usually, in that case, the build log shows
>
> ...
> Static heap usage: 10273280 of 13631488 bytes
> ...
> Static heap usage: 5419520 of 13631488 bytes
>
>
> Doesn't it mean that the "end" product uses only about 5.2 MB?
Yes, but you still need a bigger static heap for bootstrapping. While
bootstrap-emacs is being built, many *.el files get loaded, and this
requires a large static heap. Then bootstrap-emacs byte-compiles those
files. Now when emacs is built, the smaller *.elc files can be used, so
a smaller static heap suffices for that second step.
By the way, it just occurred to me that the "static heap usage" report
isn't very informative. It shows how much of the static heap is in use
at the time emacs (or bootstrap-emacs) is dumped, but there might have
been more in use and then freed at some earlier stage. I changed
sheap.c:report_sheap_usage so that it reports the maximum usage, since
this is what determines how big the static heap needs to be.
Ken
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-18 17:43 ` Ken Brown
@ 2013-08-18 19:10 ` Angelo Graziosi
2013-08-18 19:14 ` Eli Zaretskii
1 sibling, 0 replies; 27+ messages in thread
From: Angelo Graziosi @ 2013-08-18 19:10 UTC (permalink / raw
To: Ken Brown; +Cc: emacs
Il 18/08/2013 19.43, Ken Brown ha scritto:
> On 8/17/2013 6:23 PM, Angelo Graziosi wrote:
>> Il 17/08/2013 22.16, Ken Brown ha scritto:
>>>
>>> This probably means that the static heap isn't big enough for
>>> --enable-checking=all on 32-bit Cygwin. I'll test it and increase it if
>>> necessary. I may not get to this today, but I'll do it soon. (And you
>>> can do it yourself locally if you're anxious to bootstrap right now.)
>>
>> OK, with 18MB it is still bootstrapping and has printed:
>>
>> ...
>> Finding pointers to doc strings...done
>> Dumping under the name emacs
>> Static heap usage: 16099796 of 18874368 bytes
>> ...
>>
>> but... really should we increase the static heap from 13 to 18 MB also
>> for a not-checking bootstrap?
>
> No. I made the increase to 18 conditional on ENABLE_CHECKING.
>
>> Usually, in that case, the build log shows
>>
>> ...
>> Static heap usage: 10273280 of 13631488 bytes
>> ...
>> Static heap usage: 5419520 of 13631488 bytes
>>
>>
>> Doesn't it mean that the "end" product uses only about 5.2 MB?
>
> Yes, but you still need a bigger static heap for bootstrapping. While
> bootstrap-emacs is being built, many *.el files get loaded, and this
> requires a large static heap. Then bootstrap-emacs byte-compiles those
> files. Now when emacs is built, the smaller *.elc files can be used, so
> a smaller static heap suffices for that second step.
>
> By the way, it just occurred to me that the "static heap usage" report
> isn't very informative. It shows how much of the static heap is in use
> at the time emacs (or bootstrap-emacs) is dumped, but there might have
> been more in use and then freed at some earlier stage. I changed
> sheap.c:report_sheap_usage so that it reports the maximum usage, since
> this is what determines how big the static heap needs to be.
All this sounds good.. and interesting.. Thanks!
Ciao,
Angelo.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: 64-bit emacs crashes a lot
2013-08-18 17:43 ` Ken Brown
2013-08-18 19:10 ` Angelo Graziosi
@ 2013-08-18 19:14 ` Eli Zaretskii
1 sibling, 0 replies; 27+ messages in thread
From: Eli Zaretskii @ 2013-08-18 19:14 UTC (permalink / raw
To: Ken Brown; +Cc: emacs-devel, angelo.graziosi
> Date: Sun, 18 Aug 2013 13:43:58 -0400
> From: Ken Brown <kbrown@cornell.edu>
> Cc: emacs <emacs-devel@gnu.org>
>
> > Usually, in that case, the build log shows
> >
> > ...
> > Static heap usage: 10273280 of 13631488 bytes
> > ...
> > Static heap usage: 5419520 of 13631488 bytes
> >
> >
> > Doesn't it mean that the "end" product uses only about 5.2 MB?
>
> Yes, but you still need a bigger static heap for bootstrapping. While
> bootstrap-emacs is being built, many *.el files get loaded, and this
> requires a large static heap. Then bootstrap-emacs byte-compiles those
> files. Now when emacs is built, the smaller *.elc files can be used, so
> a smaller static heap suffices for that second step.
You could define the heap size differently during a bootstrap and
afterwards, if that matters. Just make sure GCC defines some
preprocessor symbol during one or the other.
^ permalink raw reply [flat|nested] 27+ messages in thread
end of thread, other threads:[~2013-08-18 19:14 UTC | newest]
Thread overview: 27+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <51F3151D.7040000@cs.utoronto.ca>
[not found] ` <51F33565.1090406@cornell.edu>
[not found] ` <51F33F52.4060405@cs.utoronto.ca>
[not found] ` <51FB1D9E.5090102@cs.utoronto.ca>
[not found] ` <20130802080211.GA18054@calimero.vinschen.de>
[not found] ` <51FB9228.2020309@cornell.edu>
[not found] ` <51FBA100.90005@cs.utoronto.ca>
[not found] ` <51FD5462.5020400@cs.utoronto.ca>
[not found] ` <51FFBDFF.7040501@cornell.edu>
[not found] ` <51FFC4F2.8080909@cs.utoronto.ca>
[not found] ` <5203D89E.6030801@cornell.edu>
[not found] ` <5203DCCA.1010105@cs.utoronto.ca>
[not found] ` <5205B364.8090007@cs.utoronto.ca>
[not found] ` <52064730.50404@cornell.edu>
[not found] ` <"52065B3C.6060104@cs.utoronto <520CCA41.3000107"@cs.utoronto.ca>
[not found] ` <520D089A.1020806@cornell.edu>
[not found] ` <83ioz6op5v.fsf@gnu.org>
[not found] ` <520D4036.8010303@cs.utoronto.ca>
[not found] ` <8361v6nhdb.fsf@gnu.org>
2013-08-16 11:37 ` 64-bit emacs crashes a lot Ryan Johnson
2013-08-16 13:10 ` Eli Zaretskii
[not found] ` <520D900A.8000907@cornell.edu>
[not found] ` <520DABDC.8020304@cs.utoronto.ca>
[not found] ` <520DBFCD.4080808@cs.utoronto.ca>
[not found] ` <8338qangma.fsf@gnu.org>
2013-08-16 11:39 ` Ryan Johnson
[not found] ` <834naqnh9t.fsf@gnu.org>
2013-08-16 11:41 ` Ryan Johnson
2013-08-16 13:31 ` Eli Zaretskii
2013-08-16 14:16 ` Ryan Johnson
2013-08-16 14:49 ` Eli Zaretskii
2013-08-16 14:20 ` Ken Brown
2013-08-16 14:24 ` Ryan Johnson
2013-08-16 15:03 ` Eli Zaretskii
2013-08-16 15:45 ` Eli Zaretskii
2013-08-16 16:51 ` Ryan Johnson
2013-08-16 17:46 ` Ken Brown
[not found] ` <520E5D71.3020307@cornell.edu>
2013-08-16 17:24 ` Ryan Johnson
2013-08-16 18:55 ` Ken Brown
2013-08-16 19:37 ` Eli Zaretskii
2013-08-16 20:17 ` Eli Zaretskii
2013-08-16 20:33 ` Ken Brown
2013-08-16 21:20 ` Ryan Johnson
2013-08-17 7:01 ` Eli Zaretskii
2013-08-17 12:17 ` Ken Brown
2013-08-17 19:43 Angelo Graziosi
2013-08-17 20:16 ` Ken Brown
2013-08-17 22:23 ` Angelo Graziosi
2013-08-18 17:43 ` Ken Brown
2013-08-18 19:10 ` Angelo Graziosi
2013-08-18 19:14 ` Eli Zaretskii
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.