bug#16901: 24.3.50; emacs_backtrace.txt

bug#16901: 24.3.50; emacs_backtrace.txt

[Drew Adams]

> Or it could be some change in Drew's use patterns, like in the
> window/frame configurations and other customizations, which exposed an
> extant bug.  Or even some changes in the OS and other programs used in
> conjunction with Emacs, for that matter.

I am not aware of any such changes on my side.  Doesn't mean there
is definitely not some minor change that I might be overlooking.
I'm just not aware of having changed anything.

If you like, I can revert to using an older Emacs build, to see if
I still get such crashes.  Let me know.

bug#16901: 24.3.50; emacs_backtrace.txt

[Eli Zaretskii]

> Date: Sat, 1 Mar 2014 09:42:26 -0800 (PST)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: dmantipov@yandex.ru, 16901@debbugs.gnu.org
> 
> If you like, I can revert to using an older Emacs build, to see if
> I still get such crashes.

Not unless you cannot possibly use the latest.  Since the build with
memory allocation checks is definitely on to something, I don't see a
need for downgrading yet; it is much more efficient to try debugging
the problems that are flagged to us.

bug#16901: 24.3.50; emacs_backtrace.txt

[Juanma Barranquero]

On Sat, Mar 1, 2014 at 7:26 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> Not unless you cannot possibly use the latest.

Certainly I cannot give Drew a snapshot from trunk built with
-DGC_MCHECK=1, because it will likely crash upon starting and, as Drew
doesn't run Emacs under GDB, he won't be able to provide us with more
information that "mcheck: memory clobbered before allocated block".

> it is much more efficient to try debugging
> the problems that are flagged to us.

If you rebuild gmalloc.c with an added #define GC_MCHECK, do you see
the mabort calls too?

    J

bug#16901: 24.3.50; emacs_backtrace.txt

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Sun, 2 Mar 2014 05:16:29 +0100
> Cc: Drew Adams <drew.adams@oracle.com>, Dmitry Antipov <dmantipov@yandex.ru>, 16901@debbugs.gnu.org
> 
> If you rebuild gmalloc.c with an added #define GC_MCHECK, do you see
> the mabort calls too?

Yes, I see them, and I'm looking into that.  Which requires me to wade
through some completely obfuscated code first...

bug#16901: 24.3.50; emacs_backtrace.txt

[Juanma Barranquero]

On Sun, Mar 2, 2014 at 6:00 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> Yes, I see them, and I'm looking into that.

Grat.

> Which requires me to wade
> through some completely obfuscated code first...

Oops. Good luck. We'll send a rescue party if you haven't reported
back in a couple months...

bug#16901: 24.3.50; emacs_backtrace.txt

[Eli Zaretskii]

> Date: Sun, 02 Mar 2014 19:00:23 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 16901@debbugs.gnu.org, dmantipov@yandex.ru
> 
> > From: Juanma Barranquero <lekktu@gmail.com>
> > Date: Sun, 2 Mar 2014 05:16:29 +0100
> > Cc: Drew Adams <drew.adams@oracle.com>, Dmitry Antipov <dmantipov@yandex.ru>, 16901@debbugs.gnu.org
> > 
> > If you rebuild gmalloc.c with an added #define GC_MCHECK, do you see
> > the mabort calls too?
> 
> Yes, I see them, and I'm looking into that.  Which requires me to wade
> through some completely obfuscated code first...

I fixed 2 bugs in gmalloc (trunk revision 116643).  One of them was in
the GC_MCHECK code, but the other could have been triggered in a
normal build as well (although a GC_MCHECK build triggered it all the
time).  In a nutshell, gmalloc didn't cope well with aligned
allocations, especially when GC_MCHECK was turned on.

The result survived a full bootstrap, where the original code couldn't
even get past loading the *.el files into bootstrap-emacs during the
initial build of the trunk, and of course the crasher with HELLO
reported by Juanma no longer does.

So I think this is ready for prime time, and let's hope it will reveal
real problems.

P.S. The bugs in gmalloc were so glaring that I'd appreciate if
someone could eyeball my changes, in case I grossly misunderstood the
code.  When I see such bugs in such veteran code, I usually question
my own sanity.  TIA.

bug#16901: 24.3.50; emacs_backtrace.txt

[Juanma Barranquero]

On Mon, Mar 3, 2014 at 5:54 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> The result survived a full bootstrap, where the original code couldn't
> even get past loading the *.el files into bootstrap-emacs during the
> initial build of the trunk, and of course the crasher with HELLO
> reported by Juanma no longer does.

I'm bootstrapping trunk now with #define GC_MCHECK (I add it directly
to gmalloc.c instead of going the configure route because it is easier
to turn it off later ;-)

If everything goes fine, I'll build a GC_MCHECK snapshot for Drew.

> When I see such bugs in such veteran code, I usually question
> my own sanity.

I would question how much that code has been used (for the bug in
GC_MCHECK code); as for the other, it surely has been giving us grief
for years, but not in a consistent enough way. Until now.

    J

bug#16901: 24.3.50; emacs_backtrace.txt

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Mon, 3 Mar 2014 18:28:03 +0100
> Cc: 16901@debbugs.gnu.org, Dmitry Antipov <dmantipov@yandex.ru>
> 
> > When I see such bugs in such veteran code, I usually question
> > my own sanity.
> 
> I would question how much that code has been used (for the bug in
> GC_MCHECK code); as for the other, it surely has been giving us grief
> for years, but not in a consistent enough way. Until now.

I think we rarely, if ever, get unaligned blocks in a production
build.  The bug only shows when malloc returns a 16KB block whose
alignment is not a multiple of 1K.  A GC_MCHECK build does that all
the time, because it reserves the first 8 bytes of every block for a
hidden header.

bug#16901: 24.3.50; emacs_backtrace.txt

[Juanma Barranquero]

On Mon, Mar 3, 2014 at 6:42 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> The bug only shows when malloc returns a 16KB block whose
> alignment is not a multiple of 1K.

That's perhaps the reason the GC crashes were so rare.

bug#16901: 24.3.50; emacs_backtrace.txt

[Ken Brown]

On 3/3/2014 11:54 AM, Eli Zaretskii wrote:
> P.S. The bugs in gmalloc were so glaring that I'd appreciate if
> someone could eyeball my changes, in case I grossly misunderstood the
> code.  When I see such bugs in such veteran code, I usually question
> my own sanity.  TIA.

I may be misunderstanding the code too, but I don't think your fix to 
aligned_alloc is quite right.  If adj == 0 in line 1596, then we've 
allocated much more memory than we needed, and the next call to malloc 
(line 1602) allocates even more.  And if adj == 1 in line 1596, then 
we've allocated exactly as much memory as we needed, so there's no need 
to call malloc again in line 1602.

Ken

bug#16901: 24.3.50; emacs_backtrace.txt

[Eli Zaretskii]

> Date: Mon, 03 Mar 2014 18:20:09 -0500
> From: Ken Brown <kbrown@cornell.edu>
> CC: 16901@debbugs.gnu.org, dmantipov@yandex.ru
> 
> If adj == 0 in line 1596, then we've allocated much more memory than
> we needed, and the next call to malloc (line 1602) allocates even
> more.  And if adj == 1 in line 1596, then we've allocated exactly as
> much memory as we needed, so there's no need to call malloc again in
> line 1602.

Thanks for reviewing.

These are further optimizations, and can (and probably should) be done
in separate commits.  But you aren't saying that the previous code was
correct, are you?

bug#16901: 24.3.50; emacs_backtrace.txt

[Ken Brown]

On 3/3/2014 10:45 PM, Eli Zaretskii wrote:
>> Date: Mon, 03 Mar 2014 18:20:09 -0500
>> From: Ken Brown <kbrown@cornell.edu>
>> CC: 16901@debbugs.gnu.org, dmantipov@yandex.ru
>>
>> If adj == 0 in line 1596, then we've allocated much more memory than
>> we needed, and the next call to malloc (line 1602) allocates even
>> more.  And if adj == 1 in line 1596, then we've allocated exactly as
>> much memory as we needed, so there's no need to call malloc again in
>> line 1602.
>
> Thanks for reviewing.
>
> These are further optimizations, and can (and probably should) be done
> in separate commits.  But you aren't saying that the previous code was
> correct, are you?

No, I think it was clearly wrong.  By accident, however, it probably 
worked most of the time and didn't waste memory, since adj is usually 0.

When/if you do the optimizations I suggested, I think it would clarify 
the code if `adj' were used to represent the actual adjustment needed, 
something like this:

   adj = (uintptr_t) alignment - result % alignment;
   if (adj == alignment)
     adj = 0;

Ken

bug#16901: 24.3.50; emacs_backtrace.txt

[Eli Zaretskii]

> Date: Tue, 04 Mar 2014 09:23:39 -0500
> From: Ken Brown <kbrown@cornell.edu>
> CC: lekktu@gmail.com, 16901@debbugs.gnu.org, dmantipov@yandex.ru
> 
> On 3/3/2014 10:45 PM, Eli Zaretskii wrote:
> >> Date: Mon, 03 Mar 2014 18:20:09 -0500
> >> From: Ken Brown <kbrown@cornell.edu>
> >> CC: 16901@debbugs.gnu.org, dmantipov@yandex.ru
> >>
> >> If adj == 0 in line 1596, then we've allocated much more memory than
> >> we needed, and the next call to malloc (line 1602) allocates even
> >> more.  And if adj == 1 in line 1596, then we've allocated exactly as
> >> much memory as we needed, so there's no need to call malloc again in
> >> line 1602.
> >
> > Thanks for reviewing.
> >
> > These are further optimizations, and can (and probably should) be done
> > in separate commits.

Now done in trunk revision 16661.

> > But you aren't saying that the previous code was correct, are you?
> 
> No, I think it was clearly wrong.  By accident, however, it probably 
> worked most of the time and didn't waste memory, since adj is usually 0.

Yes, when the initial allocation was already aligned, it worked OK.

> When/if you do the optimizations I suggested, I think it would clarify 
> the code if `adj' were used to represent the actual adjustment needed, 
> something like this:
> 
>    adj = (uintptr_t) alignment - result % alignment;
>    if (adj == alignment)
>      adj = 0;

I didn't make this change, but feel free to follow up.

Thanks.

bug#16901: 24.3.50; emacs_backtrace.txt

[Ken Brown]

On 3/4/2014 12:37 PM, Eli Zaretskii wrote:
>> Date: Tue, 04 Mar 2014 09:23:39 -0500
> I didn't make this change, but feel free to follow up.

Done as trunk revision 116662.

Ken

bug#16901: 24.3.50; emacs_backtrace.txt

[Drew Adams]

> > If you like, I can revert to using an older Emacs build, to see if
> > I still get such crashes.
> 
> Not unless you cannot possibly use the latest.  Since the build with
> memory allocation checks is definitely on to something, I don't see a
> need for downgrading yet; it is much more efficient to try debugging
> the problems that are flagged to us.

No, I don't have a problem, in general, with the latest - just those
occasional crashes.  I'll keep using the latest.  Thx.

I just downloaded a new build from Juanma (2014-02-28).  Will use that.

bug#16901: 24.3.50; emacs_backtrace.txt

[Drew Adams]

Are all of these duplicate crash reports (dunno whether the
crashes themselves are duplicates) from the same build or
from builds closely related in time?  If so, shouldn't that
help narrow things down a bit?

My impression is that I am getting lots of crashes with the
latest build I have (from Feb 22).  Maybe some development
just before that build is responsible?

bug#16901: 24.3.50; emacs_backtrace.txt

[Juanma Barranquero]

On Fri, Feb 28, 2014 at 5:00 PM, Drew Adams <drew.adams@oracle.com> wrote:

> My impression is that I am getting lots of crashes with the
> latest build I have (from Feb 22).

At least for GC crashes, your impression is correct. You've had 5
GC-related crashes with the build from revno:116523, which is the same
amount of GC-related crashes you've had with all other builds since
Jan 1 (first snapshot from revno:115827) combined.

> Maybe some development
> just before that build is responsible?

These are the changes between the previous snapshot and the build from
Feb 22, except for the ones obviously unrelated (doc fixes, manual,
tests, etc.).

116523: Juanma Barranquero 2014-02-22 lisp/desktop.el: Do not fail
when desktop-files-not-to-save is nil.
116522: Daniel Colascione 2014-02-21 Build correct secrets pattern
from auth-source pattern
116521: Daniel Colascione 2014-02-21 Additional type checking in secrets API
116518: Juanma Barranquero 2014-02-21 lisp/emacs-lisp/gv.el: Avoid
duplicating entries of defun-declaration-alist.
116517: Stefan Monnier 2014-02-21 * lisp/emacs-lisp/cl-macs.el
(cl-define-compiler-macro): Add indent rule.
116515: Juanma Barranquero 2014-02-21 lisp/whitespace.el: End
obsolescence messages with dot.
116514: Dmitry Gutov 2014-02-21 * lisp/progmodes/ruby-mode.el
(auto-mode-alist): Add missing "or".
116513: Dmitry Gutov 2014-02-21 * lisp/electric.el
(electric-indent-functions-without-reindent):
116512: Juanma Barranquero 2014-02-21 lisp/w32-vars.el
(w32-enable-synthesized-fonts): Mark as obsolete.
116509: martin rudalics 2014-02-21 In with-temp-buffer-window don't
evaluate BODY within with-current-buffer (Bug#16816).
116504: martin rudalics 2014-02-21 Fix handling of
window-min-height/-width (Bug#16738).
116501: Michael Albinus 2014-02-21 * net/tramp.el
(tramp-check-cached-permissions):
116500: Paul Eggert 2014-02-20 Pacify GCC when configuring with
--enable-gcc-warnings.
116499: Daniel Colascione 2014-02-20 [merge] Improve dbus error
handling; detect bus failure
116498: Juanma Barranquero 2014-02-21 lisp/w32-fns.el: Remove
obsolescence declarations for nonexistent vars.
116494: Eli Zaretskii 2014-02-20 Fix excessive calls to
bidi_shelve_cache reported in bug #15555.
116493: Eli Zaretskii 2014-02-20 Fix assertion violation in redisplay.
116492: Eli Zaretskii 2014-02-20 Fix bug #16819 with dereferencing
invalid face pointer.
116491: Michael Albinus 2014-02-20 * net/tramp.el
(ls-lisp-use-insert-directory-program): Declare.
116489: Juanma Barranquero 2014-02-20 lisp/elec-pair.el: Fix bug#16799.
116485: W. Trevor King 2014-02-19 * lisp/term/xterm.el
(xterm--version-handler): Adapt to xterm-280's output.
116484: Juanma Barranquero 2014-02-19 lisp/frameset.el
(frameset-restore): Remove duplicate ids only when needed.
116481: Michael Albinus 2014-02-19 Some Tramp minor fixes, found
during test campaign.
116480: Eli Zaretskii 2014-02-19 Fix bug #16806 with horizontal
scrolling of images when fringes are disabled.
116479: Eli Zaretskii 2014-02-19 Avoid crashes on MS-Windows when JPEG
images are too large.
116478: Juanma Barranquero 2014-02-19 lisp/frameset.el
(frameset--reuse-frame): Remove workaround for bug#16793.
116477: martin rudalics 2014-02-19 In window-state-put allow WINDOW to
refer to an internal window (Bug#16793).
116474: Stefan Monnier 2014-02-18 * lisp/delsel.el (delete-char):
Restore incorrectly erased property.
116473: Juanma Barranquero 2014-02-18 lisp/frameset.el: Workaround bug#16793.
116472: martin rudalics 2014-02-18 Don't set FRAME_PIXEL_HEIGHT and
FRAME_PIXEL_WIDTH in update_various_frame_slots (Bug#16736).
116470: Michael Albinus 2014-02-18 * dbusbind.c (xd_close_bus): Apply
proper check on busobj.
116469: Mirek Kaim 2014-02-17 * configure.ac [HAVE_W32]: Test for ImageMagick.

bug#16901: 24.3.50; emacs_backtrace.txt

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Fri, 28 Feb 2014 18:12:57 +0100
> Cc: Eli Zaretskii <eliz@gnu.org>, Dmitry Antipov <dmantipov@yandex.ru>, 16901@debbugs.gnu.org
> 
> > Maybe some development
> > just before that build is responsible?
> 
> These are the changes between the previous snapshot and the build from
> Feb 22, except for the ones obviously unrelated (doc fixes, manual,
> tests, etc.).

Thanks; I see nothing that immediately stands out.  It could be an
unrelated change which just exposed a bug that was present previously.

Or it could be some change in Drew's use patterns, like in the
window/frame configurations and other customizations, which exposed an
extant bug.  Or even some changes in the OS and other programs used in
conjunction with Emacs, for that matter.

IOW, there's no way to be certain without a recipe, or something like
that.

bug#16901: 24.3.50; emacs_backtrace.txt

[Drew Adams]

Backtrace:
011fbfd9
011fc04a
010f067f
011625d1
0115b8ce
0115b7a0
01161ad2
0116038c
010ee79f
011c154c
0118131d
011809b3
0118022e
010f591a
0117d6d7
010f5d12
01180122
010f5d56
0103edd3
0103eadb
010429bd
0104180f
010f6c34
01104076
010f474d
0117d6d7
010f4082
0117cc84
010f403a
010f37d0
010f398c
010f1b90
010010f9
76fa3366
77ce9f6e
77ce9f41




In GNU Emacs 24.3.50.1 (i686-pc-mingw32)
 of 2014-02-21 on ODIEONE
Repository revision: 116523 lekktu@gmail.com-20140222021049-g04nwe512x430tk5
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
 `configure --prefix=/c/Devel/emacs/binary --enable-checking=yes,glyphs
 'CFLAGS=-O0 -g3' LDFLAGS=-Lc:/Devel/emacs/lib
 CPPFLAGS=-Ic:/Devel/emacs/include'

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1252

Major mode: Dired by date

Minor modes in effect:
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<switch-frame> <switch-frame> <switch-frame> <help-echo> 
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo> 
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo> 
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo> 
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo> 
<menu-bar> <help-menu> <send-emacs-bug-report> C-g 
C-g s M-< <help-echo> <down-mouse-2> <mouse-1> <help-echo> 
<help-echo> <switch-frame> <help-echo> <help-echo> 
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo> 
<menu-bar> <help-menu> <send-emacs-bug-report>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Quit [2 times]

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util help-fns mail-prsvr mail-utils dired oneonone hexrgb cl-macs gv
cl cl-loaddefs cl-lib time-date tooltip electric uniquify ediff-hook
vc-hooks lisp-float-type mwheel dos-w32 ls-lisp w32-common-fns
disp-table w32-win w32-vars tool-bar dnd fontset image regexp-opt fringe
tabulated-list newcomment lisp-mode prog-mode register page menu-bar
rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer nadvice loaddefs button
faces cus-face macroexp files text-properties overlay sha1 md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote make-network-process w32notify w32 multi-tty emacs)

bug#16901: 24.3.50; emacs_backtrace.txt

[Juanma Barranquero]

??
??:0
w32_backtrace at w32fns.c:8431
emacs_abort at w32fns.c:8463
terminate_due_to_signal at emacs.c:378
die at alloc.c:6761
compact_small_strings at alloc.c:1960
sweep_strings at alloc.c:1890
gc_sweep at alloc.c:6333
Fgarbage_collect at alloc.c:5572
maybe_gc at lisp.h:4518
exec_byte_code at bytecode.c:964
funcall_lambda at eval.c:3049
Ffuncall at eval.c:2864
call0 at eval.c:2599
safe_run_hooks_1 at keyboard.c:1872
internal_condition_case at eval.c:1354
safe_run_hook_funcall at keyboard.c:1927
run_hook_with_args at eval.c:2551
safe_run_hooks at keyboard.c:1944
update_menu_bar at xdisp.c:11608
prepare_menu_bars at xdisp.c:11509
redisplay_internal at xdisp.c:13312
redisplay at xdisp.c:12931
read_char at keyboard.c:2567
read_key_sequence at keyboard.c:9075
command_loop_1 at keyboard.c:1449
internal_condition_case at eval.c:1354
command_loop_2 at keyboard.c:1174
internal_catch at eval.c:1118
command_loop at keyboard.c:1153
recursive_edit_1 at keyboard.c:777
Frecursive_edit at keyboard.c:845
main at emacs.c:1646
?? at crt1.c:0
??
??:0
??
??:0
??
??:0

bug#16901: 24.3.50; emacs_backtrace.txt

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Fri, 28 Feb 2014 10:41:48 +0100
> Cc: 16901@debbugs.gnu.org
> 
> w32_backtrace at w32fns.c:8431
> emacs_abort at w32fns.c:8463
> terminate_due_to_signal at emacs.c:378
> die at alloc.c:6761
> compact_small_strings at alloc.c:1960
> sweep_strings at alloc.c:1890
> gc_sweep at alloc.c:6333
> Fgarbage_collect at alloc.c:5572
> maybe_gc at lisp.h:4518

Thanks.

Dmitry, any insights?

bug#16901: 24.3.50; emacs_backtrace.txt

[Juanma Barranquero]

>> compact_small_strings at alloc.c:1960
>> sweep_strings at alloc.c:1890
>> gc_sweep at alloc.c:6333
>> Fgarbage_collect at alloc.c:5572
>> maybe_gc at lisp.h:4518

What it is quite puzzling is that Drew is basically using the same
builds than I do (in quite different ways, of course), and yet I've
got very few gc crashes, perhaps a couple in the last month, while he
seems to get them a lot.

bug#16901: 24.3.50; emacs_backtrace.txt

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Fri, 28 Feb 2014 11:48:48 +0100
> Cc: Dmitry Antipov <antipov@dev.rtsoft.ru>, Drew Adams <drew.adams@oracle.com>, 16901@debbugs.gnu.org
> 
> >> compact_small_strings at alloc.c:1960
> >> sweep_strings at alloc.c:1890
> >> gc_sweep at alloc.c:6333
> >> Fgarbage_collect at alloc.c:5572
> >> maybe_gc at lisp.h:4518
> 
> What it is quite puzzling is that Drew is basically using the same
> builds than I do (in quite different ways, of course), and yet I've
> got very few gc crashes, perhaps a couple in the last month, while he
> seems to get them a lot.

Because it's some usage pattern that triggers this, not the way you
build Emacs.

bug#16901: 24.3.50; emacs_backtrace.txt

[Juanma Barranquero]

On Fri, Feb 28, 2014 at 12:41 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> Because it's some usage pattern that triggers this, not the way you
> build Emacs.

Yes, I understand that. Alas, not many people use extremely
multi-frame setups on Windows like Drew does.

bug#16901: 24.3.50; emacs_backtrace.txt

[Drew Adams]

> What it is quite puzzling is that Drew is basically using the same
> builds than I do (in quite different ways, of course), and yet I've
> got very few gc crashes, perhaps a couple in the last month, while he
> seems to get them a lot.

"It's a gift.  And a curse." - Adrian Monk

bug#16901: 24.3.50; emacs_backtrace.txt

[Dmitry Antipov]

On 02/28/2014 02:43 PM, Eli Zaretskii wrote:

>> From: Juanma Barranquero <lekktu@gmail.com>
>> Date: Fri, 28 Feb 2014 10:41:48 +0100
>> Cc: 16901@debbugs.gnu.org
>>
>> w32_backtrace at w32fns.c:8431
>> emacs_abort at w32fns.c:8463
>> terminate_due_to_signal at emacs.c:378
>> die at alloc.c:6761
>> compact_small_strings at alloc.c:1960
>> sweep_strings at alloc.c:1890
>> gc_sweep at alloc.c:6333
>> Fgarbage_collect at alloc.c:5572
>> maybe_gc at lisp.h:4518
>
> Thanks.
>
> Dmitry, any insights?

As usual, it's hard to say anything without a clean recipe to reproduce.
I realize that GC-related bugs may be very subtle, so all affected users
(especially on MS-Windows and OSX) are pleased to bisect at least; r114795
may be a good starting point.

Now we have an explanation of http://debbugs.gnu.org/cgi/bugreport.cgi?bug=16069.
Suggested fix should help http://debbugs.gnu.org/cgi/bugreport.cgi?bug=16817#5
as well; unfortunately this is X-specific and probably won't help with weird GC
issues on other platforms.

Dmitry

bug#16901: 24.3.50; emacs_backtrace.txt

[Eli Zaretskii]

> Date: Fri, 28 Feb 2014 16:21:25 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> CC: Juanma Barranquero <lekktu@gmail.com>, drew.adams@oracle.com, 
>  16901@debbugs.gnu.org
> 
> As usual, it's hard to say anything without a clean recipe to reproduce.

Would it be possible to install some debugging code that could
pinpoint the problem, or at least give further ideas?  Or is the
problem so subtle that any code which uses strings could be the
culprit?

bug#16901: 24.3.50; emacs_backtrace.txt

[Dmitry Antipov]

[CC: to emacs-devel@ in attempt to initiate a broader discussion]

On 02/28/2014 06:26 PM, Eli Zaretskii wrote:

> Would it be possible to install some debugging code that could
> pinpoint the problem, or at least give further ideas?  Or is the
> problem so subtle that any code which uses strings could be the
> culprit?

Hm... are there crashes around sweep_strings on platforms other
than MS-Windows?

Now I have two crash reports to make me worry about GC. Both are
irregular and looks hard to reproduce:

  - this bug (crash in compact_small_strings, MS-Windows only (?))

  - http://debbugs.gnu.org/cgi/bugreport.cgi?bug=16817#11 - crash
    marking C stack, OSX-only (?)

These crashes may be originated by the same bug (probably irregular
heap corruption). It's known that GC-related crashes may be caused
by freeing fonts during gc_sweep; this is
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=16069, but it should
not affect MS-Windows and OSX (and hopefully I'll fix it soon).

On GNU/Linux, valgrind makes great job in finding memory-related
errors; if there are similar tools for other platforms, it would
be nice to try. And what about using GCC and (sorry RMS) LLVM
sanitizers?

Dmitry

bug#16901: 24.3.50; emacs_backtrace.txt

[Paul Eggert]

On 02/28/2014 07:44 AM, Dmitry Antipov wrote:
> On GNU/Linux, valgrind makes great job in finding memory-related
> errors; if there are similar tools for other platforms, it would
> be nice to try. And what about using GCC ... sanitizers?

gcc -fsanitize=address should work on the trunk, if you configure with 
something like "./configure CFLAGS='-g3 -Og -fsanitize=address' 
CANNOT_DUMP=yes" (assuming recent-enough GCC).  Like valgrind, it 
doesn't work with a dumped Emacs.

bug#16901: 24.3.50; emacs_backtrace.txt

[Eli Zaretskii]

> Date: Fri, 28 Feb 2014 19:44:46 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> CC: 16901@debbugs.gnu.org, lekktu@gmail.com, 
>  Emacs development discussions <emacs-devel@gnu.org>
> 
> Now I have two crash reports to make me worry about GC. Both are
> irregular and looks hard to reproduce:
> 
>   - this bug (crash in compact_small_strings, MS-Windows only (?))
> 
>   - http://debbugs.gnu.org/cgi/bugreport.cgi?bug=16817#11 - crash
>     marking C stack, OSX-only (?)
> 
> These crashes may be originated by the same bug (probably irregular
> heap corruption). It's known that GC-related crashes may be caused
> by freeing fonts during gc_sweep; this is
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=16069, but it should
> not affect MS-Windows and OSX (and hopefully I'll fix it soon).
> 
> On GNU/Linux, valgrind makes great job in finding memory-related
> errors; if there are similar tools for other platforms, it would
> be nice to try. And what about using GCC and (sorry RMS) LLVM
> sanitizers?

This is exacerbated by the fact that Drew, who is the only one that
reports such assertion violations on Windows, doesn't build his Emacs.
So using temacs under valgrind-like tool is not an option in this
case.

I'm not aware of any tools comparable with valgrind that work on
Windows with GCC-generated symbol tables.  However, since Emacs on
Windows uses gmalloc, perhaps Juanma could build Emacs with GC_MCHECK
defined, which might catch the villain closer to the corruption locus.
Last time I hit a segfault in 'free', turning on these checks in
gmalloc allowed me to find the culprit in just a few minutes of
debugging, which is quite impressive for this sort of bugs.

bug#16901: 24.3.50; emacs_backtrace.txt

[Juanma Barranquero]

On Sat, Mar 1, 2014 at 9:47 AM, Eli Zaretskii <eliz@gnu.org> wrote:

> However, since Emacs on
> Windows uses gmalloc, perhaps Juanma could build Emacs with GC_MCHECK
> defined, which might catch the villain closer to the corruption locus.

Yes, of course.

bug#16901: 24.3.50; emacs_backtrace.txt

[Juanma Barranquero]

Just by adding -DGC_MCHECK=1 to CPPFLAGS and reconfiguring/recompiling
I'm getting a lot of these:

mcheck: memory clobbered before allocated block

Where do I go from here?
   J

bug#16901: 24.3.50; emacs_backtrace.txt

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Sat, 1 Mar 2014 11:46:30 +0100
> Cc: Dmitry Antipov <dmantipov@yandex.ru>, 16901@debbugs.gnu.org, 
> 	Emacs developers <emacs-devel@gnu.org>
> 
> Just by adding -DGC_MCHECK=1 to CPPFLAGS and reconfiguring/recompiling
> I'm getting a lot of these:
> 
> mcheck: memory clobbered before allocated block
> 
> Where do I go from here?

These are most probably bugs.  Put a breakpoint where that message is
displayed, and post backtraces if you cannot figure out what is the
problem that triggers them.

bug#16901: 24.3.50; emacs_backtrace.txt

[Juanma Barranquero]

Here's one from emacs -Q:

(gdb) break mabort
Breakpoint 3 at 0x124c388: file gmalloc.c, line 1859.
(gdb) run -Q -f view-hello-file
Starting program: C:\Devel\emacs\repo\trunk\src\emacs-24.3.50.2.exe -Q
-f view-hello-file
[New Thread 8228.0x220c]
[New Thread 8228.0x2320]
[New Thread 8228.0x2530]
[New Thread 8228.0x2e70]
[New Thread 8228.0x2404]

Breakpoint 3, mabort (status=MCHECK_HEAD) at gmalloc.c:1859
1859      switch (status)
(gdb) bt
#0  mabort (status=MCHECK_HEAD) at gmalloc.c:1859
#1  0x0124c17c in checkhdr (hdr=0x5aa43f8) at gmalloc.c:1780
#2  0x0124c1a1 in freehook (ptr=0x5aa4400) at gmalloc.c:1792
#3  0x0124bab9 in e_free (ptr=0x5aa4400) at gmalloc.c:1255
#4  0x0115eb49 in lisp_align_free (block=0x5aa8000) at alloc.c:1197
#5  0x0116570e in gc_sweep () at alloc.c:6415
#6  0x01163dd4 in Fgarbage_collect () at alloc.c:5586
#7  0x010f20b7 in maybe_gc () at lisp.h:4519
#8  0x01183fcb in Ffuncall (nargs=6, args=0x88c460) at eval.c:2766
#9  0x011814fc in internal_condition_case_n (bfun=0x1183f13
<Ffuncall>, nargs=6, args=0x88c460, handlers=58374218, hfun=0x1029dbb
<safe_eval_handler>) at eval.c:1436
#10 0x01029edb in safe_call (nargs=6, func=60982154) at xdisp.c:2609
#11 0x011ea7d1 in autocmp_chars (rule=61169485, charpos=361,
bytepos=428, limit=366, win=0x3ac2218
<__register_frame_info+61612568>,
    face=0x3914d08 <__register_frame_info+59854088>, string=58374186)
at composite.c:918
#12 0x011ebb2a in composition_reseat_it (cmp_it=0x88d544, charpos=365,
bytepos=436, endpos=1, w=0x3ac2218 <__register_frame_info+61612568>,
    face=0x3914d08 <__register_frame_info+59854088>, string=58374186)
at composite.c:1252
#13 0x01038bf8 in next_element_from_buffer (it=0x88d07c) at xdisp.c:8209
#14 0x01035545 in get_next_display_element (it=0x88d07c) at xdisp.c:6796
#15 0x0105c4b1 in display_line (it=0x88d07c) at xdisp.c:19820
#16 0x010511d7 in try_window (window=61612573, pos=..., flags=1) at
xdisp.c:16638
#17 0x0104e7cf in redisplay_window (window=61612573,
just_this_one_p=false) at xdisp.c:16155
#18 0x01047aac in redisplay_window_0 (window=61612573) at xdisp.c:14148
#19 0x011812ba in internal_condition_case_1 (bfun=0x1047a76
<redisplay_window_0>, arg=61612573, handlers=58349798, hfun=0x1047a52
<redisplay_window_error>) at eval.c:1378
#20 0x01047a39 in redisplay_windows (window=61612573) at xdisp.c:14128
#21 0x01046a6f in redisplay_internal () at xdisp.c:13727
#22 0x01044aec in redisplay () at xdisp.c:13013
#23 0x010fa553 in read_char (commandflag=1, map=63501118,
prev_event=58374186, used_mouse_menu=0x88f793, end_time=0x0) at
keyboard.c:2567
#24 0x011079b5 in read_key_sequence (keybuf=0x88f8b0, bufsize=30,
prompt=58374186, dont_downcase_last=false,
can_return_switch_frame=true, fix_current_buffer=true,
    prevent_redisplay=false) at keyboard.c:9079
#25 0x010f8065 in command_loop_1 () at keyboard.c:1449
#26 0x011811a7 in internal_condition_case (bfun=0x10f7ce5
<command_loop_1>, handlers=58445162, hfun=0x10f754b <cmd_error>) at
eval.c:1354
#27 0x010f799a in command_loop_2 (ignore=58374186) at keyboard.c:1174
#28 0x01180754 in internal_catch (tag=58432330, func=0x10f7976
<command_loop_2>, arg=58374186) at eval.c:1118
#29 0x010f7952 in command_loop () at keyboard.c:1153
#30 0x010f70e8 in recursive_edit_1 () at keyboard.c:777
#31 0x010f72a4 in Frecursive_edit () at keyboard.c:845
#32 0x010f54a8 in main (argc=4, argv=0xe03048) at emacs.c:1646

Lisp Backtrace:
"Automatic GC" (0x153ce7c)
"auto-compose-chars" (0x88c464)
"redisplay_internal (C function)" (0x153ce7c)
(gdb)



and then there's one with my setup:

Breakpoint 3, mabort (status=MCHECK_HEAD) at gmalloc.c:1859
1859      switch (status)
(gdb) bt
#0  mabort (status=MCHECK_HEAD) at gmalloc.c:1859
#1  0x0124c17c in checkhdr (hdr=0x583d3f8) at gmalloc.c:1780
#2  0x0124c1a1 in freehook (ptr=0x583d400) at gmalloc.c:1792
#3  0x0124bab9 in e_free (ptr=0x583d400) at gmalloc.c:1255
#4  0x0115eb49 in lisp_align_free (block=0x5841000) at alloc.c:1197
#5  0x0116570e in gc_sweep () at alloc.c:6415
#6  0x01163dd4 in Fgarbage_collect () at alloc.c:5586
#7  0x010f20b7 in maybe_gc () at lisp.h:4519
#8  0x01183fcb in Ffuncall (nargs=4, args=0x88d934) at eval.c:2766
#9  0x011c4e38 in exec_byte_code (bytestr=19622849, vector=19622869,
maxdepth=20, args_template=58374186, nargs=0, args=0x0) at
bytecode.c:919
#10 0x01184ded in funcall_lambda (fun=19622797, nargs=5,
arg_vector=0x12b6bd5) at eval.c:3049
#11 0x01184483 in Ffuncall (nargs=6, args=0x88dc64) at eval.c:2864
#12 0x011c4e38 in exec_byte_code (bytestr=19623113, vector=19623133,
maxdepth=28, args_template=58374186, nargs=0, args=0x0) at
bytecode.c:919
#13 0x01184ded in funcall_lambda (fun=19623093, nargs=2,
arg_vector=0x12b6cdd) at eval.c:3049
#14 0x01184483 in Ffuncall (nargs=3, args=0x88df94) at eval.c:2864
#15 0x011c4e38 in exec_byte_code (bytestr=19623209, vector=19623229,
maxdepth=16, args_template=58374186, nargs=0, args=0x0) at
bytecode.c:919
#16 0x01184ded in funcall_lambda (fun=19623189, nargs=2,
arg_vector=0x12b6d3d) at eval.c:3049
#17 0x01184483 in Ffuncall (nargs=3, args=0x88e2c4) at eval.c:2864
#18 0x011c4e38 in exec_byte_code (bytestr=19633881, vector=19633901,
maxdepth=20, args_template=58374186, nargs=0, args=0x0) at
bytecode.c:919
#19 0x01184ded in funcall_lambda (fun=19633861, nargs=2,
arg_vector=0x12b96ed) at eval.c:3049
#20 0x01184483 in Ffuncall (nargs=3, args=0x88e5f4) at eval.c:2864
#21 0x011c4e38 in exec_byte_code (bytestr=19637193, vector=19637221,
maxdepth=12, args_template=58374186, nargs=0, args=0x0) at
bytecode.c:919
#22 0x011c4288 in Fbyte_code (bytestr=19637193, vector=19637221,
maxdepth=12) at bytecode.c:482
#23 0x01182f0d in eval_sub (form=19637182) at eval.c:2191
#24 0x0118109c in internal_lisp_condition_case (var=58374186,
bodyform=19637182, handlers=19449710) at eval.c:1323
#25 0x011c5d78 in exec_byte_code (bytestr=19637081, vector=19637101,
maxdepth=24, args_template=58374186, nargs=0, args=0x0) at
bytecode.c:1169
#26 0x01184ded in funcall_lambda (fun=19637053, nargs=1,
arg_vector=0x12ba36d) at eval.c:3049
#27 0x01184483 in Ffuncall (nargs=2, args=0x88ecf4) at eval.c:2864
#28 0x011c4e38 in exec_byte_code (bytestr=19961633, vector=19961653,
maxdepth=36, args_template=58374186, nargs=0, args=0x0) at
bytecode.c:919
#29 0x01184ded in funcall_lambda (fun=19961613, nargs=0,
arg_vector=0x1309735) at eval.c:3049
#30 0x01184483 in Ffuncall (nargs=1, args=0x88f034) at eval.c:2864
#31 0x011c4e38 in exec_byte_code (bytestr=19655433, vector=62373741,
maxdepth=24, args_template=0, nargs=0, args=0x88f374) at
bytecode.c:919
#32 0x01184a29 in funcall_lambda (fun=61450973, nargs=0,
arg_vector=0x88f374) at eval.c:2983
#33 0x01184483 in Ffuncall (nargs=1, args=0x88f370) at eval.c:2864
#34 0x01182daf in eval_sub (form=60524278) at eval.c:2157
#35 0x0117eaba in Fprogn (body=60524270) at eval.c:468
#36 0x0117eb22 in unwind_body (body=60524270) at eval.c:482
#37 0x011855d0 in unbind_to (count=4, value=58374186) at eval.c:3306
#38 0x011c4edb in exec_byte_code (bytestr=19654841, vector=19654861,
maxdepth=48, args_template=0, nargs=0, args=0x88f7c0) at
bytecode.c:941
#39 0x01184a29 in funcall_lambda (fun=19654821, nargs=0,
arg_vector=0x88f7c0) at eval.c:2983
#40 0x01184748 in apply_lambda (fun=19654821, args=58374186) at eval.c:2924
#41 0x01183126 in eval_sub (form=60147982) at eval.c:2230
#42 0x01182746 in Feval (form=60147982, lexical=58374186) at eval.c:2003
#43 0x010f79cd in top_level_2 () at keyboard.c:1183
#44 0x011811a7 in internal_condition_case (bfun=0x10f79b0
<top_level_2>, handlers=58445162, hfun=0x10f754b <cmd_error>) at
eval.c:1354
#45 0x010f7a01 in top_level_1 (ignore=58374186) at keyboard.c:1191
#46 0x01180754 in internal_catch (tag=58432330, func=0x10f79cf
<top_level_1>, arg=58374186) at eval.c:1118
#47 0x010f7933 in command_loop () at keyboard.c:1152
#48 0x010f70e8 in recursive_edit_1 () at keyboard.c:777
#49 0x010f72a4 in Frecursive_edit () at keyboard.c:845
#50 0x010f54a8 in main (argc=1, argv=0x982f78) at emacs.c:1646

Lisp Backtrace:
"Automatic GC" (0x153ce7c)
"internal-face-x-get-resource" (0x88d938)
"set-face-attribute-from-resource" (0x88dc68)
"set-face-attributes-from-resources" (0x88df98)
"make-face-x-resource-internal" (0x88e2c8)
"face-spec-recalc" (0x88e5f8)
"byte-code" (0x88e880)
"face-set-after-frame-default" (0x88ecf8)
"frame-notice-user-settings" (0x88f038)
0x3a9aad8 PVEC_COMPILED
"funcall" (0x88f370)
"normal-top-level" (0x88f7c0)
(gdb)


Perfectly repeatable, so if you want me to try anything, just ask.

   J

bug#16901: 24.3.50; emacs_backtrace.txt

[Eli Zaretskii]

> Date: Fri, 28 Feb 2014 19:44:46 +0400
> From: Dmitry Antipov <dmantipov@yandex.ru>
> Cc: 16901@debbugs.gnu.org, lekktu@gmail.com,
>  Emacs development discussions <emacs-devel@gnu.org>
> 
> Now I have two crash reports to make me worry about GC. Both are
> irregular and looks hard to reproduce:
> 
>   - this bug (crash in compact_small_strings, MS-Windows only (?))
> 
>   - http://debbugs.gnu.org/cgi/bugreport.cgi?bug=16817#11 - crash
>     marking C stack, OSX-only (?)
> 
> These crashes may be originated by the same bug (probably irregular
> heap corruption). It's known that GC-related crashes may be caused
> by freeing fonts during gc_sweep; this is
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=16069, but it should
> not affect MS-Windows and OSX (and hopefully I'll fix it soon).
> 
> On GNU/Linux, valgrind makes great job in finding memory-related
> errors; if there are similar tools for other platforms, it would
> be nice to try.

I've run Emacs on Windows under Dr. Memory (http://www.drmemory.org/),
but saw nothing appropriate.  However, there's no reproducer, so it's
hard to know if I did what was needed.  Perhaps you could come up with
a function to execute compact_small_strings, then I'd run that and see
what I get.