bug#10617: 24.0.92; Bidi crash reading a message from emacs-devel

bug#10617: 24.0.92; Bidi crash reading a message from emacs-devel

[Nix]

I just got a bidi crash reading an emacs-devel message in Gnus (bzr
r106941). The crash happened when doing a page-down while viewing the
message archived at
<http://lists.gnu.org/archive/html/emacs-devel/2012-01/msg00824.html>
(but you can't see the bidi goodness there, if it *is* meant to be good
to find the periods transposed to the other end of the line while the
lines themselves still read in L2R, but right-justified. Weird, but
maybe intended, I dunno.)

It is quite clear from the backtrace that the second parameter to
char_table_ref() has been garbaged, apparently being set to 2^32/1000
(again, passing strange).

I still have the coredump: any debugging I can do, just ask. (However,
the thing was compiled with optimization, so debugging is visibly
degraded. I'm just about to upgrade GDB to 7.4: maybe that will help a
bit.)

No recipe from emacs -Q yet (a bit hard given that this is provoked by
Gnus-plus-nnml). Tomorrow I'll try to come up with a reproduction recipe
based on the text of the message alone.

Backtrace:

#0  0x00007f5b0ca808a7 in kill () from /lib/libc.so.6
#1  0x00000000004f8d3c in fatal_error_signal (sig=<optimized out>) at emacs.c:366
#2  fatal_error_signal (sig=<optimized out>) at emacs.c:336
#3  <signal handler called>
#4  0x000000000049a16f in char_table_ref (table=<optimized out>, c=4195289) at chartab.c:237
#5  0x00000000005d3746 in composition_compute_stop_pos (cmp_it=0x7fff80a63e68, charpos=1431, bytepos=1396, endpos=<optimized out>, string=12065314) at composite.c:1065
#6  0x000000000042a7b8 in compute_stop_pos (it=0x7fff80a63600) at xdisp.c:3273
#7  0x0000000000442da9 in compute_stop_pos_backwards (it=0x7fff80a63600) at xdisp.c:7551
#8  next_element_from_buffer (it=0x7fff80a63600) at xdisp.c:7717
#9  0x000000000043a2da in get_next_display_element (it=0x7fff80a63600) at xdisp.c:6387
#10 0x0000000000437f49 in move_it_in_display_line_to (it=0x7fff80a63600, to_charpos=2459, to_x=-1, op=MOVE_TO_POS) at xdisp.c:8029
#11 0x000000000043e610 in move_it_to (it=0x7fff80a63600, to_charpos=2459, to_x=-1, to_y=-1, to_vpos=-1, op=8) at xdisp.c:8621
#12 0x0000000000445474 in start_display (it=0x7fff80a63600, w=<optimized out>, pos=...) at xdisp.c:2830
#13 0x000000000054149b in Fvertical_motion (lines=4, window=<optimized out>) at indent.c:2027
#14 0x0000000000575594 in Ffuncall (nargs=<optimized out>, args=0x7fff80a64418) at eval.c:2990
#15 0x00000000005b1006 in exec_byte_code (bytestr=<optimized out>, vector=<optimized out>, maxdepth=<optimized out>, args_template=<optimized out>, nargs=<optimized out>, args=<optimized out>) at bytecode.c:785
#16 0x0000000000575071 in funcall_lambda (fun=9633805, nargs=<optimized out>, arg_vector=0x7fff80a645f8) at eval.c:3218
#17 0x00000000005753db in Ffuncall (nargs=4, args=0x7fff80a645f0) at eval.c:3048
#18 0x00000000005b1006 in exec_byte_code (bytestr=<optimized out>, vector=<optimized out>, maxdepth=<optimized out>, args_template=<optimized out>, nargs=<optimized out>, args=<optimized out>) at bytecode.c:785
#19 0x0000000000575071 in funcall_lambda (fun=9632957, nargs=<optimized out>, arg_vector=0x7fff80a647b8) at eval.c:3218
#20 0x00000000005753db in Ffuncall (nargs=5, args=0x7fff80a647b0) at eval.c:3048
#21 0x00000000005b1006 in exec_byte_code (bytestr=<optimized out>, vector=<optimized out>, maxdepth=<optimized out>, args_template=<optimized out>, nargs=<optimized out>, args=<optimized out>) at bytecode.c:785
#22 0x0000000000574a03 in eval_sub (form=<optimized out>) at eval.c:2341
#23 0x0000000000577c64 in internal_lisp_condition_case (var=12869650, bodyform=9631438, handlers=9631566) at eval.c:1454
#24 0x00000000005b0201 in exec_byte_code (bytestr=<optimized out>, vector=<optimized out>, maxdepth=<optimized out>, args_template=<optimized out>, nargs=<optimized out>, args=<optimized out>) at bytecode.c:981
#25 0x0000000000575071 in funcall_lambda (fun=9631157, nargs=<optimized out>, arg_vector=0x7fff80a64d18) at eval.c:3218
#26 0x00000000005753db in Ffuncall (nargs=3, args=0x7fff80a64d10) at eval.c:3048
#27 0x0000000000571751 in Fcall_interactively (function=12793922, record_flag=12065314, keys=212423709) at callint.c:852
#28 0x0000000000575581 in Ffuncall (nargs=<optimized out>, args=0x7fff80a64ee0) at eval.c:2994
#29 0x00000000005757e4 in call3 (fn=<optimized out>, arg1=<optimized out>, arg2=<optimized out>, arg3=<optimized out>) at eval.c:2787
#30 0x0000000000509c19 in command_loop_1 () at keyboard.c:1571
#31 0x0000000000573746 in internal_condition_case (bfun=0x509880 <command_loop_1>, handlers=12106194, hfun=0x4fddb0 <cmd_error>) at eval.c:1500
#32 0x00000000004fbfce in command_loop_2 (ignore=<optimized out>) at keyboard.c:1159
#33 0x0000000000573628 in internal_catch (tag=0, func=0x4fbfb0 <command_loop_2>, arg=12065314) at eval.c:1257
#34 0x00000000004fd887 in command_loop () at keyboard.c:1138
#35 recursive_edit_1 () at keyboard.c:758
#36 0x00000000004fdbbc in Frecursive_edit () at keyboard.c:822
#37 0x0000000000411c3d in main (argc=2, argv=<optimized out>) at emacs.c:1715



In GNU Emacs 24.0.92.2 (x86_64-unknown-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2012-01-26 on spindle
Windowing system distributor `The X.Org Foundation', version 11.0.11003901
configured using `configure  '--without-pop' '--without-kerberos' '--without-hesiod' '--without-mmdf' '--with-x-toolkit=lucid' '--with-wide-int' 'NO_FAST_MATH=t''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: C
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_GB.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Summary

Minor modes in effect:
  gnus-mailing-list-mode: t
  iswitchb-mode: t
  show-paren-mode: t
  global-cwarn-mode: t
  global-semanticdb-minor-mode: t
  global-semantic-idle-completions-mode: t
  global-semantic-idle-scheduler-mode: t
  compile-bookmarks-mode: t
  semantic-mode: t
  type-break-mode-line-message-mode: t
  icomplete-mode: t
  recentf-mode: t
  mv-shell-mode: t
  shell-dirtrack-mode: t
  which-function-mode: t
  desktop-save-mode: t
  display-time-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t

Recent input:
y M-x g n u s <return> y <next> <next> <up> <up> <up> 
<up> <up> <up> <up> <up> <up> <up> <up> <up> <up> <up> 
<up> <return> <right> <right> <right> <right> <right> 
<right> <right> <down> E E E E <up> <up> <up> <up> 
<right> <return> q <right> <up> C-r e m a c s - d e 
v e l <left> <down> <return> <return> <right> <right> 
<right> <right> <right> <right> C-s d <backspace> b 
i d i C-s C-r C-r C-s C-s C-s C-s C-a <return> <return> 
SPC <backspace> <backspace> n n n n n n n SPC M-x r 
e p o r t - e <tab> <return>

Recent messages:
Warning: Couldn't read newsgroups descriptions
nnimap read 0k
Expiring articles...done
Mark saved where search started [2 times]
Hit C-g to stop BBDB from annotating.  5 of 6 addresses processed.
Hit C-g to stop BBDB from annotating.  5 of 5 addresses processed.
Hit C-g to stop BBDB from annotating.  5 of 6 addresses processed. [2 times]
Hit C-g to stop BBDB from annotating.  5 of 7 addresses processed.
Hit C-g to stop BBDB from annotating.  5 of 6 addresses processed.
Hit C-g to stop BBDB from annotating.  5 of 5 addresses processed.

Load-path shadows:
/home/nix/lisp/defaults hides /usr/share/emacs/site-lisp/defaults
/home/nix/lisp/emacs/site-wide/site-start hides /usr/share/emacs/site-lisp/site-start
/home/nix/lisp/emacs/site-wide/default hides /usr/share/emacs/site-lisp/default
/home/nix/lisp/emacs/site-wide/scroll-in-place hides /usr/share/emacs/site-lisp/scroll-in-place
/usr/share/emacs/site-lisp/emms/tq hides /usr/share/emacs/24.0.92/lisp/emacs-lisp/tq

Features:
(shadow emacsbug multi-isearch gnus-picon gnus-cite ansi-color
gnus-async gnus-bcklg gnus-salt gnus-dup qp gnus-ml gnus-topic url-cache
url-http url-gw url-auth url-handlers nndraft nnrss xml gnus-agent
gnus-srvr gnus-score score-mode nnvirtual gnus-msg utf-7 gnutls nnimap
parse-time utf7 nnmh nnml nnfolder gnus-cache bbdb-gnus bbdb-snarf netrc
gnus-demon nntp dot-gnus-mail dot-gnus-splits mm-url smtpmail gnus-art
mm-uu mml2015 mm-view mml-smime smime dig dot-gnus-articles dot-gnus-sa
background gnus-sum nnoo gnus-group gnus-undo nnmail mail-source
dot-gnus-bbdb dot-gnus-colourization tc mail-extr gnus-start gnus-spec
gnus-int gnus-range gnus-win gnus gnus-ems nnheader server
semantic/bovine/make semantic/bovine/make-by make-mode vc-git checkdoc
thingatpt semantic/imenu semantic/db-file cedet-files semantic/bovine/c
semantic/decorate/include semantic/db-find semantic/db-ref
semantic/decorate/mode semantic/decorate pulse semantic/bovine/c-by
semantic/lex-spp semantic/bovine/gcc semantic/dep semantic/analyze
semantic/sort semantic/scope semantic/analyze/fcn c-eldoc eldoc
sh-script executable epa-file epa derived epg epg-config generic
site-default dot-emacs dot-emacs-emacs iswitchb xemacs-compat add-log
misc init-music network-stream starttls tls emms-volume
emms-volume-amixer emms-history emms-bookmarks emms-metaplaylist-mode
emms-browser sort emms-playlist-sort emms-last-played emms-playing-time
emms-stream-info emms-streams emms-mode-line emms-cache emms-info
later-do emms-playlist-limit emms-playlist-mode emms-player-mpd tq
emms-player-simple emms-source-playlist emms-source-file dired emms
emms-compat init-message-modes bbdb-expire bbdb-hooks bbdb-com
silly-mail sendmail boxquote rect message rfc822 mml mml-sec mm-decode
mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums
mailabbrev mail-utils gmm-utils mailheader init-time-tracking
timeclock-visualize sgml-mode url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-util url-parse url-vars mailcap
auto-edit-substitute init-prog-modes init-prog-modes-emacs filecache
paren cwarn cc-mode cc-fonts cc-guess cc-menus semantic/db-mode
semantic/db eieio-base semantic/idle semantic/format ezimage
semantic/tag-ls semantic/ctxt htmlfontify cus-edit cus-start cus-load
compile-bookmarks gpicker ffap font-latex latex easy-mmode edmacro
kmacro tex-style tex semantic/util-modes semantic/util semantic
semantic/tag semantic/lex semantic/fw mode-local cedet cc-cmds cc-styles
cc-align cc-engine cc-vars cc-defs miniedit type-break icomplete
site-start-load gawd-keys help-mode view gawd-keys-emacs gawd-mode-frobs
gawd-mode-frobs-emacs windmove recentf tree-widget wid-edit mv-shell
printing ps-print ps-def lpr uptimes pp bbdb timezone browse-kill-ring+
browse-kill-ring tempbuf timeclock igrep grep compile term disp-table
ehelp electric tramp tramp-compat auth-source assoc gnus-util mm-util
mail-prsvr password-cache shell pcomplete comint format-spec
tramp-loaddefs regexp-opt hideshow filladapt gawd-faces gawd-faces-emacs
nix-dark-theme gawd-misc gawd-misc-emacs which-func imenu winner
time-date gawd-lists bbdb-autoloads desktop generic-x uniquify time
advice help-fns advice-preload scroll-in-place site-start-emacs
site-autoloads all-autoloads auctex-autoloads tex-site info
c-eldoc-autoloads compilation-recenter-end-autoloads
compile-bookmarks-autoloads dictionary-autoloads diff-git-autoloads
elk-test-autoloads fringe-helper-autoloads full-ack-autoloads
htmlize-autoloads iresize-autoloads jump-autoloads inflections-autoloads
findr-autoloads lua-mode-autoloads minimap-autoloads mv-shell-autoloads
perspective-autoloads package tabulated-list emms-auto rudel-loaddefs
rudel-backend warnings eieio byte-opt bytecomp byte-compile cconv
macroexp w3m-load apropos-toc cl ring filesets easymenu flash-paren
saveplace redo+ tooltip ediff-hook vc-hooks lisp-float-type mwheel x-win
x-dnd tool-bar dnd fontset image fringe lisp-mode register page menu-bar
rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
cus-face files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote
make-network-process dbusbind dynamic-setting system-font-setting
font-render-setting x-toolkit x multi-tty emacs)

-- 
NULL && (void)

bug#10617: 24.0.92; Bidi crash reading a message from emacs-devel

[Eli Zaretskii]

> From: Nix <nix@esperi.org.uk>
> Emacs: no job too big... no job.
> Date: Thu, 26 Jan 2012 22:40:22 +0000
> 
> I just got a bidi crash reading an emacs-devel message in Gnus (bzr
> r106941).

I'm curious: why do you think this crash has anything to do with bidi?
There are no bidi-related functions anywhere in the backtrace you
show.

> The crash happened when doing a page-down while viewing the
> message archived at
> <http://lists.gnu.org/archive/html/emacs-devel/2012-01/msg00824.html>

FWIW, I read that message without any problems in Rmail, neither with
the current trunk nor with the 24.0.92 pretest.  But that's a 32-bit
build on Windows, while yours is a 64-bit build on GNU/Linux.

> (but you can't see the bidi goodness there, if it *is* meant to be good
> to find the periods transposed to the other end of the line while the
> lines themselves still read in L2R, but right-justified. Weird, but
> maybe intended, I dunno.)

This weird display is mandated by the Unicode Bidirectional Algorithm,
because the quoted part of the message is treated as a single
right-to-left paragraph.  It is a single paragraph because there are
no empty lines in it, and it takes a right-to-left paragraph direction
because the first strong directional character is an Arabic letter,
whose directionality is right to left.

I have an idea for how to make the display more reasonable in this
(quite frequent) use case, but it will have to wait until after the
release of Emacs 24.1, because it could have wide potentially
surprising effects which will need to be carefully considered.

> It is quite clear from the backtrace that the second parameter to
> char_table_ref() has been garbaged, apparently being set to 2^32/1000
> (again, passing strange).

Sorry, I don't believe backtraces from optimized builds, they lie
through their teeth.

> I still have the coredump: any debugging I can do, just ask.

It would be interesting to see it->current, it->position, it->sp, and
it->string in frames #6 and #8.  Also, what do you have in the buffer
at the position(s) shown by it->current and it->position (the
functions in etc/emacs-buffer.gdb might come in handy for finding this
out).

> (However, the thing was compiled with optimization, so debugging is
> visibly degraded. I'm just about to upgrade GDB to 7.4: maybe that
> will help a bit.)
> 
> No recipe from emacs -Q yet (a bit hard given that this is provoked by
> Gnus-plus-nnml). Tomorrow I'll try to come up with a reproduction recipe
> based on the text of the message alone.

A newer GDB will help, but please also try this in an unoptimized
build.  If you can reproduce it there, we will have much better
chances of finding the culprit.

Also, please show the results of "xbacktrace" (starting GDB from the
Emacs src directory should cause that be done automagically).

> In GNU Emacs 24.0.92.2 (x86_64-unknown-linux-gnu, X toolkit, Xaw3d scroll bars)
>  of 2012-01-26 on spindle
> Windowing system distributor `The X.Org Foundation', version 11.0.11003901
> configured using `configure  '--without-pop' '--without-kerberos' '--without-hesiod' '--without-mmdf' '--with-x-toolkit=lucid' '--with-wide-int' 'NO_FAST_MATH=t''

Can you tell whether you built with libraries mentioned in INSTALL
under "Complex Text Layout support libraries", and if so, which
versions thereof?

Also, do you have any problems whatsoever displaying etc/HELLO in its
entirety?

Thanks.

bug#10617: 24.0.92; Bidi crash reading a message from emacs-devel

[Nix]

On 27 Jan 2012, Eli Zaretskii spake thusly:

>> From: Nix <nix@esperi.org.uk>
>> Emacs: no job too big... no job.
>> Date: Thu, 26 Jan 2012 22:40:22 +0000
>> 
>> I just got a bidi crash reading an emacs-devel message in Gnus (bzr
>> r106941).
>
> I'm curious: why do you think this crash has anything to do with bidi?
> There are no bidi-related functions anywhere in the backtrace you
> show.

Oh. Maybe I jumped to conclusions, but the fact that I got it when
viewing a heavily-bidi message roused suspicions too strong to see past
:) emacs is too reliable, that's the problem: if it kept crashing all
the time I'd not make this sort of assumption, but since this is my
first crash in months I assumed that most of it was bug-free!

>> (but you can't see the bidi goodness there, if it *is* meant to be good
>> to find the periods transposed to the other end of the line while the
>> lines themselves still read in L2R, but right-justified. Weird, but
>> maybe intended, I dunno.)
>
> This weird display is mandated by the Unicode Bidirectional Algorithm,
> because the quoted part of the message is treated as a single
> right-to-left paragraph.  It is a single paragraph because there are
> no empty lines in it, and it takes a right-to-left paragraph direction
> because the first strong directional character is an Arabic letter,
> whose directionality is right to left.

It's not a bug then, good :)

>> It is quite clear from the backtrace that the second parameter to
>> char_table_ref() has been garbaged, apparently being set to 2^32/1000
>> (again, passing strange).
>
> Sorry, I don't believe backtraces from optimized builds, they lie
> through their teeth.

Backtraces from optimized GCC builds on x86_64 Linux (and, on modern
GCC's, on i386 too) don't work by doing frame pointer walking anymore,
they do DWARF walking. If that lies, the stack is severely corrupted and
exception handling will also crash: perhaps not terribly relevant for
most C programs but still a sign that keeping this particular data
structure un-fudged-up is considered important. (There are the usual
modifications due to inlining and sibcalls but they are easy to
compensate for.)

So it's a good bit more reliable than it used to be. You can generally
rely on the function names being valid.

Looking at variable values from optimized builds still sucks giant
asteroids through micropipettes though. :( so the parameters in the
backtraces might sometimes be lying or outdated.

>> I still have the coredump: any debugging I can do, just ask.
>
> It would be interesting to see it->current, it->position, it->sp, and
> it->string in frames #6 and #8.

Frame 6:

(gdb) print it->current
$3 = {
  pos = {
    charpos = 1430,
    bytepos = 1394
  },
  overlay_string_index = -1,
  string_pos = {
    charpos = -1,
    bytepos = -1
  },
  dpvec_index = -1
}
(gdb) print it->position
$4 = {
  charpos = 1430,
  bytepos = 1394
}
(gdb) print it->sp
$5 = 0
(gdb) print it->string
$6 = 12065314

Frame 8:

(gdb) print it->current
$7 = {
  pos = {
    charpos = 1430,
    bytepos = 1394
  },
  overlay_string_index = -1,
  string_pos = {
    charpos = -1,
    bytepos = -1
  },
  dpvec_index = -1
}
(gdb) print it->position
$8 = {
  charpos = 1430,
  bytepos = 1394
}
(gdb) print it->sp
$9 = 0
(gdb) print it->string
$10 = 12065314

(i.e. unchanged)

>                                Also, what do you have in the buffer
> at the position(s) shown by it->current and it->position (the
> functions in etc/emacs-buffer.gdb might come in handy for finding this
> out).

I'm afraid optimized-build hell has kicked in here:

There is no member named data.

I suspect this will be undiagnosable unless I can reproduce this in an
unoptimized build :(

>> (However, the thing was compiled with optimization, so debugging is
>> visibly degraded. I'm just about to upgrade GDB to 7.4: maybe that
>> will help a bit.)
>> 
>> No recipe from emacs -Q yet (a bit hard given that this is provoked by
>> Gnus-plus-nnml). Tomorrow I'll try to come up with a reproduction recipe
>> based on the text of the message alone.
>
> A newer GDB will help, but please also try this in an unoptimized
> build.  If you can reproduce it there, we will have much better
> chances of finding the culprit.

I'll try that soon. I doubt we can get much further with this as it
stands :(

> Also, please show the results of "xbacktrace" (starting GDB from the
> Emacs src directory should cause that be done automagically).

"There is no member named data" again. Sigh :(

>> In GNU Emacs 24.0.92.2 (x86_64-unknown-linux-gnu, X toolkit, Xaw3d scroll bars)
>>  of 2012-01-26 on spindle
>> Windowing system distributor `The X.Org Foundation', version 11.0.11003901
>> configured using `configure  '--without-pop' '--without-kerberos' '--without-hesiod' '--without-mmdf' '--with-x-toolkit=lucid' '--with-wide-int' 'NO_FAST_MATH=t''
>
> Can you tell whether you built with libraries mentioned in INSTALL
> under "Complex Text Layout support libraries", and if so, which
> versions thereof?

I didn't build with any of the complex text layout support libraries at
all.

> Also, do you have any problems whatsoever displaying etc/HELLO in its
> entirety?

No.

-- 
NULL && (void)

bug#10617: 24.0.92; Bidi crash reading a message from emacs-devel

[Eli Zaretskii]

> From: Nix <nix@esperi.org.uk>
> Cc: 10617@debbugs.gnu.org
> Emacs: is that a Lisp interpreter in your editor, or are you just happy to see me?
> Date: Mon, 30 Jan 2012 18:14:28 +0000
> 
> On 27 Jan 2012, Eli Zaretskii spake thusly:
> 
> >> From: Nix <nix@esperi.org.uk>
> >> Emacs: no job too big... no job.
> >> Date: Thu, 26 Jan 2012 22:40:22 +0000
> >> 
> >> I just got a bidi crash reading an emacs-devel message in Gnus (bzr
> >> r106941).
> >
> > I'm curious: why do you think this crash has anything to do with bidi?
> > There are no bidi-related functions anywhere in the backtrace you
> > show.
> 
> Oh. Maybe I jumped to conclusions, but the fact that I got it when
> viewing a heavily-bidi message roused suspicions too strong to see past

Arabic text is special in that it uses character compositions, not
only bidi display.  And the crash is inside code that handles
character compositions.

> >> It is quite clear from the backtrace that the second parameter to
> >> char_table_ref() has been garbaged, apparently being set to 2^32/1000
> >> (again, passing strange).
> >
> > Sorry, I don't believe backtraces from optimized builds, they lie
> > through their teeth.
> 
> Backtraces from optimized GCC builds on x86_64 Linux (and, on modern
> GCC's, on i386 too) don't work by doing frame pointer walking anymore,
> they do DWARF walking. If that lies, the stack is severely corrupted and
> exception handling will also crash: perhaps not terribly relevant for
> most C programs but still a sign that keeping this particular data
> structure un-fudged-up is considered important. (There are the usual
> modifications due to inlining and sibcalls but they are easy to
> compensate for.)
> 
> So it's a good bit more reliable than it used to be. You can generally
> rely on the function names being valid.

The problem is that (a) static functions inlined by the compiler don't
always appear in the backtraces, and (b) too many arguments in the
calls are not shown ("optimized out") or shown with incorrect values.

> > It would be interesting to see it->current, it->position, it->sp, and
> > it->string in frames #6 and #8.
> 
> Frame 6:
> 
> (gdb) print it->current
> $3 = {
>   pos = {
>     charpos = 1430,
>     bytepos = 1394
>   },
>   overlay_string_index = -1,
>   string_pos = {
>     charpos = -1,
>     bytepos = -1
>   },
>   dpvec_index = -1
> }
> (gdb) print it->position
> $4 = {
>   charpos = 1430,
>   bytepos = 1394
> }

If bytepos is smaller than charpos, it generally means trouble...

> (gdb) print it->sp
> $5 = 0
> (gdb) print it->string
> $6 = 12065314

What does "xtype" say about this string?  If it says Lisp_String, what
does "xstring" say?

bug#10617: 24.0.92; Bidi crash reading a message from emacs-devel

[Nix]

On 30 Jan 2012, Eli Zaretskii spake thusly:

> From: Nix <nix@esperi.org.uk>
>> > It would be interesting to see it->current, it->position, it->sp, and
>> > it->string in frames #6 and #8.
>> 
>> Frame 6:
>> 
>> (gdb) print it->current
>> $3 = {
>>   pos = {
>>     charpos = 1430,
>>     bytepos = 1394
>>   },
>>   overlay_string_index = -1,
>>   string_pos = {
>>     charpos = -1,
>>     bytepos = -1
>>   },
>>   dpvec_index = -1
>> }
>> (gdb) print it->position
>> $4 = {
>>   charpos = 1430,
>>   bytepos = 1394
>> }
>
> If bytepos is smaller than charpos, it generally means trouble...

I thought maybe the gap accounted for it -- but this is already
gap-compensated, isn't it? So we have characters of size <1 byte there.
(I sort of doubt that.)

>> (gdb) print it->sp
>> $5 = 0
>> (gdb) print it->string
>> $6 = 12065314
>
> What does "xtype" say about this string?  If it says Lisp_String, what
> does "xstring" say?

(gdb) xtype
Lisp_Symbol
(gdb) xstring
$2 = (struct Lisp_String *) 0xb81a20
There is no member named data.

Not very useful.

(gdb) print *((struct Lisp_String *) 0xb81a20)
$9 = {
  intervals = 0x98,
  u = {
    imm = {
      gcmarkbit = 1,
      immbit = 0,
      size = -40,
      size_byte = -37,
      data = "\204\000\000\000\000\000\"\032\270\000\000\000\000\000\362\031\270\000\000\000\000"
    },
    dat = {
      unused = 1,
      size = <error reading variable>
  }
}

Even less useful.

I'll see if I can reproduce this in an unoptimized build...

-- 
NULL && (void)

bug#10617: 24.0.92; Bidi crash reading a message from emacs-devel

[Andreas Schwab]

Nix <nix@esperi.org.uk> writes:

> (gdb) xstring
> $2 = (struct Lisp_String *) 0xb81a20
> There is no member named data.
>
> Not very useful.

If you are changing struct Lisp_String you also have to update the gdb
macros.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#10617: 24.0.92; Bidi crash reading a message from emacs-devel

[Eli Zaretskii]

> From: Nix <nix@esperi.org.uk>
> Cc: 10617@debbugs.gnu.org
> Date: Mon, 30 Jan 2012 19:11:55 +0000
> 
> >> (gdb) print it->string
> >> $6 = 12065314
> >
> > What does "xtype" say about this string?  If it says Lisp_String, what
> > does "xstring" say?
> 
> (gdb) xtype
> Lisp_Symbol
> (gdb) xstring
> $2 = (struct Lisp_String *) 0xb81a20
> There is no member named data.
> 
> Not very useful.

It's a symbol (see above), not a string, so using "xstring" with it is
not useful.  Try "xsymbol" (I'm guessing it's nil).

bug#10617: 24.0.92; Bidi crash reading a message from emacs-devel

[Nix]

On 30 Jan 2012, Eli Zaretskii stated:

>> From: Nix <nix@esperi.org.uk>
>> >> (gdb) print it->string
>> >> $6 = 12065314
>> >
>> > What does "xtype" say about this string?  If it says Lisp_String, what
>> > does "xstring" say?
>> 
>> (gdb) xtype
>> Lisp_Symbol
>> (gdb) xstring
>> $2 = (struct Lisp_String *) 0xb81a20
>> There is no member named data.
>> 
>> Not very useful.
>
> It's a symbol (see above), not a string, so using "xstring" with it is
> not useful.  Try "xsymbol" (I'm guessing it's nil).

Oh, how... obvious. I shouldn't respond to these when exhausted.

Ooo:

(gdb) xsymbol it->string
$2 = (struct Lisp_Symbol *) 0xb81a20
There is no member named data.
(gdb) print *((struct Lisp_Symbol *) 0xb81a20)
$3 = {
  gcmarkbit = 0,
  redirect = SYMBOL_PLAINVAL,
  constant = 1,
  interned = 2,
  declared_special = 0,
  xname = 8697697,
  val = {
    value = 12065314,
    alias = 0xb81a22,
    blv = 0xb81a22,
    fwd = 0xb81a22
  },
  function = 12065266,
  plist = 38661286,
  next = 0x0
}

(gdb) print/x Qnil
$6 = 0xb81a22

So this is clearly actually a forwarded or buffer-localized nil
variable, but redirect has become corrupted so that Emacs thinks,
incorrectly, that it's a value. Right?

bug#10617: 24.0.92; Bidi crash reading a message from emacs-devel

[Eli Zaretskii]

> From: Nix <nix@esperi.org.uk>
> Cc: 10617@debbugs.gnu.org
> Emacs: the definitive fritterware.
> Date: Mon, 30 Jan 2012 21:39:36 +0000
> 
> (gdb) xsymbol it->string
> $2 = (struct Lisp_Symbol *) 0xb81a20
> There is no member named data.
> (gdb) print *((struct Lisp_Symbol *) 0xb81a20)
> $3 = {
>   gcmarkbit = 0,
>   redirect = SYMBOL_PLAINVAL,
>   constant = 1,
>   interned = 2,
>   declared_special = 0,
>   xname = 8697697,
>   val = {
>     value = 12065314,
>     alias = 0xb81a22,
>     blv = 0xb81a22,
>     fwd = 0xb81a22
>   },
>   function = 12065266,
>   plist = 38661286,
>   next = 0x0
> }
> 
> (gdb) print/x Qnil
> $6 = 0xb81a22

You make your life unnecessarily complicated ;-)  Just do the below,
but exactly like shown:

  (gdb) p it->string
  (gdb) xsymbol

bug#10617: 24.0.92; Bidi crash reading a message from emacs-devel

[Nix]

On 31 Jan 2012, Eli Zaretskii stated:

>> (gdb) xsymbol it->string
>> $2 = (struct Lisp_Symbol *) 0xb81a20
>> There is no member named data.
>> (gdb) print *((struct Lisp_Symbol *) 0xb81a20)
>> $3 = {
>>   gcmarkbit = 0,
>>   redirect = SYMBOL_PLAINVAL,
>>   constant = 1,
>>   interned = 2,
>>   declared_special = 0,
>>   xname = 8697697,
>>   val = {
>>     value = 12065314,
>>     alias = 0xb81a22,
>>     blv = 0xb81a22,
>>     fwd = 0xb81a22
>>   },
>>   function = 12065266,
>>   plist = 38661286,
>>   next = 0x0
>> }
>> 
>> (gdb) print/x Qnil
>> $6 = 0xb81a22
>
> You make your life unnecessarily complicated ;-)  Just do the below,
> but exactly like shown:
>
>   (gdb) p it->string
>   (gdb) xsymbol

I did that at the top of the last debugging dump. The results are not so
useful:

(gdb) p it->string
$1 = 12065314
(gdb) xsymbol
$2 = (struct Lisp_Symbol *) 0xb81a20
There is no member named data.

Hence the attempt to decrypt things a bit further above.

-- 
NULL && (void)

bug#10617: 24.0.92; Bidi crash reading a message from emacs-devel

[Eli Zaretskii]

> From: Nix <nix@esperi.org.uk>
> Cc: 10617@debbugs.gnu.org
> Date: Tue, 31 Jan 2012 14:21:22 +0000
> 
> (gdb) xsymbol
> $2 = (struct Lisp_Symbol *) 0xb81a20
> There is no member named data.

You are using a bad .gdbinit file, or somehow botched the definition
of xsymbol or one of the commands it invokes.  How about doing a
"source .gdbinit" after you verify that .gdbinit comes from the same
tree from which you produced the Emacs binary?

bug#10617: 24.0.92; Bidi crash reading a message from emacs-devel

[Nix]

On 31 Jan 2012, Eli Zaretskii verbalised:

>> From: Nix <nix@esperi.org.uk>
>> Cc: 10617@debbugs.gnu.org
>> Date: Tue, 31 Jan 2012 14:21:22 +0000
>> 
>> (gdb) xsymbol
>> $2 = (struct Lisp_Symbol *) 0xb81a20
>> There is no member named data.
>
> You are using a bad .gdbinit file, or somehow botched the definition
> of xsymbol or one of the commands it invokes.  How about doing a
> "source .gdbinit" after you verify that .gdbinit comes from the same
> tree from which you produced the Emacs binary?

Well, I'm in the build tree's src subdirectory and xsymbol is defined,
so I'm not sure where else it could be coming from! It's not like I have
multiple .gdbinits lying around that define xsymbol :) I tried sourcing
./.gdbinit explicitly: it changes nothing.

But unfortunately my attempt to reproduce this bug with the same message
have failed so far (it must be dependent on something else in Emacs
state, probably something transient), and the coredump is naturally not
compatible with an Emacs built without optimization. So this probably
needs to be closed as unreproducible :(

-- 
NULL && (void)