bug#12745: crash in bidi_pop_it during (idle) redisplay

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Paul Eggert]

[I'm forwarding this from emacs-devel; the original is at
 <http://lists.gnu.org/archive/html/emacs-devel/2012-10/msg00743.html>.

 The rest of this message is from Ami Fischman.]

Emacs 24.2 regularly crashes on me with the backtrace below.  I usually
start it with "emacs --daemon" and then run one emacsclient on each of two
X servers (a local one on :0 and a remote one through
NX<http://www.nomachine.com/>).
 This setup will be up/stable for anywhere between an hour and a few days,
and then the server aborts dumping core like this.  FWIW, emacs-version
says:
GNU Emacs 24.2.1 (x86_64-unknown-linux-gnu, X toolkit, Xaw scroll bars)
and I built it using this this configuration:
--with-x-toolkit=lucid --with-xpm --with-jpeg --with-tiff --with-gif
--with-png --with-x --without-dbug --without-gconf

Possibly of note is that this happens when I'm not doing anything with
emacs (usually I'll come back to one of my workstations and find the emacs
process is gone and a corefile has shown up).  Both of the X servers in
question are much longer lived than the emacs processes involved.

Cheers,
-a



(gdb) bt
#0  0x00007fa43b6f1707 in kill () at ../sysdeps/unix/syscall-template.S:82
#1  0x00000000004e93d2 in fatal_error_signal (sig=<optimized out>) at
emacs.c:366
#2  <signal handler called>
#3  0x00007fa43b6f1707 in kill () at ../sysdeps/unix/syscall-template.S:82
#4  0x00000000004e7452 in abort () at emacs.c:394
#5  0x00000000004923b5 in bidi_pop_it (bidi_it=<optimized out>) at
bidi.c:636
#6  0x0000000000447aa1 in pop_it (it=0x7fff2251f1e0) at xdisp.c:5769
#7  0x0000000000451f23 in next_overlay_string (it=0x7fff2251f1e0) at
xdisp.c:5223
#8  0x0000000000426ff7 in set_iterator_to_next (it=0x7fff2251f1e0,
reseat_p=<optimized out>) at xdisp.c:7188
#9  0x00000000004311ed in display_line (it=0x7fff2251f1e0) at xdisp.c:19519
#10 0x00000000004302d8 in try_window (window=<optimized out>, flags=1,
pos=...) at xdisp.c:16137
#11 0x000000000044ae4a in redisplay_window (window=82406597,
just_this_one_p=0) at xdisp.c:15662
#12 0x00000000004514e1 in redisplay_window_0 (window=9054) at xdisp.c:13748
#13 0x000000000055a046 in internal_condition_case_1 (bfun=0x4514c0
<redisplay_window_0>, arg=82406597, handlers=9957798, hfun=<optimized out>)
at eval.c:1552
#14 0x0000000000448376 in redisplay_windows (window=<optimized out>) at
xdisp.c:13728
#15 0x0000000000448385 in redisplay_windows (window=<optimized out>) at
xdisp.c:13720
#16 0x000000000042d3d0 in redisplay_internal () at xdisp.c:13305
#17 0x000000000042fb64 in redisplay_preserve_echo_area (from_where=9054) at
xdisp.c:13556
#18 0x00000000004f1a8a in detect_input_pending_run_timers
(do_display=<optimized out>) at keyboard.c:10512
#19 0x0000000000596414 in wait_reading_process_output
(time_limit=<optimized out>, microsecs=0, read_kbd=-1, do_display=1,
wait_for_cell=9755602, wait_proc=0x0, just_wait_proc=<optimized out>) at
process.c:4738
#20 0x00000000004f0954 in kbd_buffer_get_event (kbp=<optimized out>,
end_time=<optimized out>, used_mouse_menu=0x7fff22528354, kbp=<optimized
out>, end_time=<optimized out>) at keyboard.c:3855
#21 read_char (commandflag=<optimized out>, nmaps=9, maps=<optimized out>,
prev_event=9755602, used_mouse_menu=0x7fff22528354, end_time=<optimized
out>) at keyboard.c:2801
#22 0x00000000004ec170 in read_key_sequence (bufsize=30, keybuf=<optimized
out>, prompt=<optimized out>, dont_downcase_last=<optimized out>,
can_return_switch_frame=<optimized out>, fix_current_buffer=<optimized
out>) at keyboard.c:9328
#23 0x00000000004ea909 in command_loop_1 () at keyboard.c:1449
#24 0x000000000055b680 in internal_condition_case (bfun=0x4ea490
<command_loop_1>, handlers=9807890, hfun=<optimized out>) at eval.c:1514
#25 0x00000000004fbf06 in command_loop_2 (ignore=<optimized out>) at
keyboard.c:1160
#26 0x000000000055b0dc in internal_catch (tag=<optimized out>,
func=0x4fbee0 <command_loop_2>, arg=9755602) at eval.c:1271
#27 0x00000000004e9c39 in command_loop () at keyboard.c:1139
#28 recursive_edit_1 () at keyboard.c:759
#29 0x00000000004e9d55 in Frecursive_edit () at keyboard.c:823
#30 0x00000000004e8acd in main (argc=<error reading variable: Cannot access
memory at address 0x0>, argv=<optimized out>) at emacs.c:1715

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Paul Eggert]

[This is a copy of <http://lists.gnu.org/archive/html/emacs-devel/2012-10/msg00744.html>.]

Date: Sun, 28 Oct 2012 05:58:18 +0200
From: Eli Zaretskii <eliz@gnu.org>

> Date: Sat, 27 Oct 2012 19:25:49 -0700
> From: Ami Fischman <ami@fischman.org>
>
> Emacs 24.2 regularly crashes on me with the backtrace below.

Thank you for your report.

First, please file a bug about this via "M-x report-emacs-bug RET".

Second, please show the value of it->current in frame #7 (inside
next_overlay_string).

Third, could you please try the latest development code, and possibly
compile without optimizations?  Several bugs in this area were fixed
since 24.2 was released, so perhaps this is one of them.

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Paul Eggert]

[This is a copy of
 <http://lists.gnu.org/archive/html/emacs-devel/2012-10/msg00745.html>.]

From: Ami Fischman <ami@fischman.org>
Date: Sat, 27 Oct 2012 21:09:43 -0700

Thanks for your reply, Eli.


> First, please file a bug about this via "M-x report-emacs-bug RET".
>

When? :)
(after the crash, the process is gone, so can't M-x anything; before the
crash, there's nothing interesting to report :))
In case it's interesting, appended the info from r-e-b (from an emacs -Q
instance) to the bottom of this email.


> Second, please show the value of it->current in frame #7 (inside
> next_overlay_string).
>

(gdb) p it->current
$3 = {
  pos = {
    charpos = 1295,
    bytepos = 1295
  },
  overlay_string_index = 0,
  string_pos = {
    charpos = -1,
    bytepos = -1
  },
  dpvec_index = -1
}


> Third, could you please try the latest development code, and possibly
> compile without optimizations?


Will do & report back in a few days (or sooner, if the bug recurs in HEAD).

Cheers,
-a


In GNU Emacs 24.2.1 (x86_64-unknown-linux-gnu, X toolkit, Xaw scroll bars)
 of 2012-09-11 on <REDACTED>
Windowing system distributor `The X.Org Foundation', version 11.0.70000000
Configured using:
 `configure '--prefix=/usr/gmacs-24.2' '--mandir=${prefix}/share/man'
 '--infodir=${prefix}/share/info' '--with-x-toolkit=lucid' '--with-xpm'
 '--with-jpeg' '--with-tiff' '--with-gif' '--with-png' '--with-x'
 '--program-transform-name=s/emacs/gmacs/g' '--without-dbug'
 '--without-gconf' 'CC=clang
 -B/home/fischman/src/chromium/src/third_party/gold/' 'CFLAGS=-Wall -g
 -O2' 'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro'
 'CPPFLAGS=-D_FORTIFY_SOURCE=2''

Important settings:
  value of $LC_ALL:
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
M-x r e p o r <tab> <return>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail regexp-opt rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils time-date tooltip ediff-hook
vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image
fringe lisp-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese hebrew greek romanian slovak czech european ethiopic
indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple
abbrev minibuffer loaddefs button faces cus-face files text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dbusbind
dynamic-setting system-font-setting font-render-setting x-toolkit x
multi-tty emacs)

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Eli Zaretskii]

> Date: Sun, 28 Oct 2012 00:50:08 -0700
> From: Paul Eggert <eggert@cs.ucla.edu>
> 
> > First, please file a bug about this via "M-x report-emacs-bug RET".
> >
> 
> When? :)
> (after the crash, the process is gone, so can't M-x anything; before the
> crash, there's nothing interesting to report :))

That's not true, the important information is about your system,
environment, and packages loaded into Emacs, so a report from the same
kind of session will do.

> In case it's interesting, appended the info from r-e-b (from an emacs -Q
> instance) to the bottom of this email.

Thanks.  But I'd like to see the report from your normal session,
invoked just as you invoke those that crash.

> (gdb) p it->current
> $3 = {
>   pos = {
>     charpos = 1295,
>     bytepos = 1295
>   },
>   overlay_string_index = 0,
>   string_pos = {
>     charpos = -1,
>     bytepos = -1
>   },
>   dpvec_index = -1
> }

So this seems to say that there's at least one overlay string at
buffer position 1295.  Is that reasonable?  What was the current
buffer when this crashed?  You can find that out by typing this at GDB
prompt:

  (gdb) pp current_buffer->name_

(If "pp" doesn't work, type "source /path/to/emacs/src/.gdbinit" and
try again.)

You can display the relevant part of the text of that buffer like
this:

  (gdb) p current_buffer->text->beg[1200]@100

What does this show?

Also, what do the following commands produce?

  (gdb) frame 6
  (gdb) pgrowx it->glyph_row

> > Third, could you please try the latest development code, and possibly
> > compile without optimizations?
> 
> 
> Will do & report back in a few days (or sooner, if the bug recurs in HEAD).

Thanks, but please also show the above information for the version
that did crash.

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Ami Fischman]

[-- Attachment #1: Type: text/plain, Size: 1816 bytes --]

>
> Thanks.  But I'd like to see the report from your normal session,
> invoked just as you invoke those that crash.
>

Of course now I don't have such a session b/c I'm running w/ git HEAD :)

So this seems to say that there's at least one overlay string at
> buffer position 1295.  Is that reasonable?  What was the current
> buffer when this crashed?  You can find that out by typing this at GDB
> prompt:
>   (gdb) pp current_buffer->name_
>

(gdb) pp current_buffer->name_
Cannot access memory at address 0x8b6a00


>   (gdb) p current_buffer->text->beg[1200]@100
>

 (gdb) p current_buffer->text->beg[1200]@100
$1 = "num to avoid later static_cast in\n// PluginInstance.\nenum
MediaKeyError {\n  kUnknownError = 1,\n  kCl"
which tells me the current buffer was an edited version of
http://src.chromium.org/viewvc/chrome/trunk/src/webkit/media/crypto/ppapi/cdm_wrapper.cc?view=markup(which
I can't share in its entirety).  FWIW, there's nothing
non-7-bit-ascii in this file, and nothing that should have triggered any
bidi-specific logic.  It's just a cc-mode C++ file.

Possibly interestingly, if I print p current_buffer->text->beg[0]@100000 to
emit the entire buffer, I see this text starting at char 1675:
http://go", '\000' <repeats 2000 times>, "/b
Those 2000 NULs are definitely out of place (the URL should have started
with http://go/b) but I don't know if that's a debugging artifact, or what.

If I load the modified buffer into my HEAD session (overlays-at 1295)
returns nil.

Also, what do the following commands produce?
>   (gdb) frame 6
>   (gdb) pgrowx it->glyph_row
>
>>

> (gdb) frame 6

> #6  0x0000000000447aa1 in pop_it (it=0x7fff2251f1e0) at xdisp.c:5769

> 5769          bidi_pop_it (&it->bidi_it);

> (gdb) pgrowx it->glyph_row
You can't do that without a process to debug.

Cheers,
-a

[-- Attachment #2: Type: text/html, Size: 3511 bytes --]

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Eli Zaretskii]

> Date: Sun, 28 Oct 2012 12:00:49 -0700
> From: Ami Fischman <ami@fischman.org>
> Cc: 12745@debbugs.gnu.org
> 
> > Thanks.  But I'd like to see the report from your normal session,
> > invoked just as you invoke those that crash.
> >
> 
> Of course now I don't have such a session b/c I'm running w/ git HEAD :)

If it loads the same .emacs, that is good enough.

> So this seems to say that there's at least one overlay string at
> > buffer position 1295.  Is that reasonable?  What was the current
> > buffer when this crashed?  You can find that out by typing this at GDB
> > prompt:
> >   (gdb) pp current_buffer->name_
> >
> 
> (gdb) pp current_buffer->name_
> Cannot access memory at address 0x8b6a00

How about this:

  (gdb) p current_buffer->name_
  (gdb) xtype

(Note: "p", not "pp".)

If the last command says it's a Lisp string, display the contents of
'struct Lisp_String' whose address it shows.

> >   (gdb) p current_buffer->text->beg[1200]@100
> >
> 
>  (gdb) p current_buffer->text->beg[1200]@100
> $1 = "num to avoid later static_cast in\n// PluginInstance.\nenum
> MediaKeyError {\n  kUnknownError = 1,\n  kCl"
> which tells me the current buffer was an edited version of
> http://src.chromium.org/viewvc/chrome/trunk/src/webkit/media/crypto/ppapi/cdm_wrapper.cc?view=markup(which

Did that buffer have any minor mode or some other optional feature
turned on, in addition to C++ Mode?

> FWIW, there's nothing non-7-bit-ascii in this file, and nothing that
> should have triggered any bidi-specific logic.  It's just a cc-mode
> C++ file.

This crash is not about bidi.  It is about handling overlays.  That it
barfs inside a function whose name begins with "bidi" is just sheer
luck--or lack thereof.

> Possibly interestingly, if I print p current_buffer->text->beg[0]@100000 to
> emit the entire buffer, I see this text starting at char 1675:
> http://go", '\000' <repeats 2000 times>, "/b
> Those 2000 NULs are definitely out of place (the URL should have started
> with http://go/b) but I don't know if that's a debugging artifact, or what.

This could be the gap, you should see its position and size like this:

  (gdb) p current_buffer->text->gpt
  (gdb) p current_buffer->text->gap_size

> If I load the modified buffer into my HEAD session (overlays-at 1295)
> returns nil.

That's what I suspected, this is the reason for the abort: Emacs
thinks there's an overlay string at that position, when there is none,
actually.

> > (gdb) frame 6
> 
> > #6  0x0000000000447aa1 in pop_it (it=0x7fff2251f1e0) at xdisp.c:5769
> 
> > 5769          bidi_pop_it (&it->bidi_it);
> 
> > (gdb) pgrowx it->glyph_row
> You can't do that without a process to debug.

So are you debugging a core dump?

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Ami Fischman]

[-- Attachment #1.1: Type: text/plain, Size: 2344 bytes --]

> If it loads the same .emacs, that is good enough.
>

Ok, attached.

> So this seems to say that there's at least one overlay string at
> > > buffer position 1295.  Is that reasonable?  What was the current
> > > buffer when this crashed?  You can find that out by typing this at GDB
> > > prompt:
> > >   (gdb) pp current_buffer->name_
> > (gdb) pp current_buffer->name_
> > Cannot access memory at address 0x8b6a00
> How about this:
>   (gdb) p current_buffer->name_
>   (gdb) xtype
> (Note: "p", not "pp".)
> If the last command says it's a Lisp string, display the contents of
> 'struct Lisp_String' whose address it shows.
>

(gdb) p current_buffer->name_
$22 = 101548849
(gdb) xtype
Lisp_String
(gdb) xstring current_buffer->name_
$23 = (struct Lisp_String *) 0x60d8330
  "cdm_wrapper.cc"


> > >   (gdb) p current_buffer->text->beg[1200]@100
> >  (gdb) p current_buffer->text->beg[1200]@100
> > $1 = "num to avoid later static_cast in\n// PluginInstance.\nenum
> > MediaKeyError {\n  kUnknownError = 1,\n  kCl"
> > which tells me the current buffer was an edited version of
> >
> http://src.chromium.org/viewvc/chrome/trunk/src/webkit/media/crypto/ppapi/cdm_wrapper.cc?view=markup(which
> Did that buffer have any minor mode or some other optional feature
> turned on, in addition to C++ Mode?


See attached b-g-e.txt, in which the current buffer is the same .cc file in
my HEAD session loading the same .emacs as the crashed one.

> Possibly interestingly, if I print p current_buffer->text->beg[0]@100000
> to
> > emit the entire buffer, I see this text starting at char 1675:
> > http://go", '\000' <repeats 2000 times>, "/b
> > Those 2000 NULs are definitely out of place (the URL should have started
> > with http://go/b) but I don't know if that's a debugging artifact, or
> what.
>
> This could be the gap, you should see its position and size like this:
>
>   (gdb) p current_buffer->text->gpt
>   (gdb) p current_buffer->text->gap_size


Yep, looks like it:
(gdb) p current_buffer->text->gpt
$24 = 1684
(gdb) p current_buffer->text->gap_size
$25 = 2000

> > (gdb) frame 6
> > > #6  0x0000000000447aa1 in pop_it (it=0x7fff2251f1e0) at xdisp.c:5769
> > > 5769          bidi_pop_it (&it->bidi_it);
> > > (gdb) pgrowx it->glyph_row
> > You can't do that without a process to debug.
> So are you debugging a core dump?
>

Yes.

-a

[-- Attachment #1.2: Type: text/html, Size: 3916 bytes --]

[-- Attachment #2: b-g-e.txt --]
[-- Type: text/plain, Size: 12508 bytes --]


Important settings:
  value of $LC_ALL: 
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: C++/lah

Minor modes in effect:
  fillcode-mode: t
  fci-mode: t
  flymake-mode: t
  global-pointback-mode: t
  pointback-mode: t
  diff-auto-refine-mode: t
  savehist-mode: t
  winner-mode: t
  persistent-thing-mode: t
  nxhtml-menu-mode: t
  ami-hi-lock-mode: t
  hi-lock-mode: t
  rcirc-track-minor-mode: t
  display-time-mode: t
  desktop-save-mode: t
  global-auto-revert-mode: t
  global-linum-mode: t
  linum-mode: t
  shell-dirtrack-mode: t
  ami-everywhere-keys-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  auto-fill-function: c-do-auto-fill
  transient-mark-mode: t
  abbrev-mode: t

Load-path shadows:
/home/fischman/apps/elisp/org-mode/contrib/lisp/org-panel hides /home/fischman/apps/elisp/nxhtml-2.08-100425/util/org-panel
/home/fischman/apps/elisp/nxhtml-2.08-100425/util/pointback hides ~/.elisp/pointback
/home/fischman/apps/elisp/nxhtml-2.08-100425/related/php-mode hides ~/.elisp/php-mode
/home/fischman/apps/elisp/nxhtml-2.08-100425/related/moz hides ~/.elisp/moz
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-irc hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-irc
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-html hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-html
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-timer hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-timer
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-crypt hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-crypt
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-macs hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-macs
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-feed hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-feed
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-id hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-id
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-faces hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-faces
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-habit hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-habit
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-rmail hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-rmail
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-docview hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-docview
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-footnote hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-footnote
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-bibtex hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-bibtex
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-mhe hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-mhe
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-ctags hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-ctags
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-remember hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-remember
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-mew hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-mew
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-table hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-table
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-beamer hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-beamer
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-install hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-install
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-w3m hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-w3m
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-freemind hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-freemind
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-latex hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-latex
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-publish hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-publish
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-attach hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-attach
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-wl hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-wl
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-archive hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-archive
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-exp-blocks hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-exp-blocks
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-vm hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-vm
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-datetree hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-datetree
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-agenda hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-agenda
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-inlinetask hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-inlinetask
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-gnus hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-gnus
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-clock hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-clock
/home/fischman/apps/elisp/org-mode/contrib/lisp/org-special-blocks hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-special-blocks
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-docbook hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-docbook
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-icalendar hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-icalendar
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-mouse hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-mouse
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-bbdb hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-bbdb
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-mac-message hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-mac-message
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-protocol hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-protocol
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-ascii hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-ascii
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-exp hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-exp
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-list hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-list
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-xoxo hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-xoxo
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-indent hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-indent
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-colview hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-colview
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-plot hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-plot
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-info hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-info
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-mobile hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-mobile
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-jsinfo hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-jsinfo
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-compat hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-compat
/home/fischman/apps/elisp/org-mode/share/emacs/site-lisp/org-src hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/org/org-src
/home/build/public/eng/elisp/third_party/xmltok hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/nxml/xmltok
/home/fischman/apps/elisp/nxhtml-2.08-100425/tests/ert hides /usr/gmacs-20121027/share/emacs/24.2.50/lisp/emacs-lisp/ert

Features:
(shadow sort mail-extr emacsbug message mailabbrev gmm-utils
mailheader sendmail mail-utils vc-dispatcher vc-svn tabify apropos
debug misearch multi-isearch longlines reveal org-wl org-w3m org-vm
org-rmail org-mhe org-mew org-irc org-jsinfo org-infojs org-html
org-exp org-agenda org-info org-gnus org-docview org-bibtex org-bbdb
make-mode sh-script smie cc-langs disp-table fillcode cc-bytecomp
eldoc server fill-column-indicator xcscope find-things-fast
flymake-chromium flymake crel gyp 
goto-last-change pointback tramp tramp-compat tramp-loaddefs
ess-toolbar ess-mous mouseme ess-menu speedbar sb-image ezimage dframe
ess-swv ess-noweb noweb-font-lock-mode essl-bugs essd-omg essl-omg
essd-els essd-sas essl-sas essa-sas executable essd-arc essd-vst
essd-xls essl-lsp essd-sta essl-sta make-regexp essd-sp6 essd-sp5
essd-sp3 essd-r essd-r-args essl-s ess-inf ess-utils ess-mode
noweb-mode ess ess-cust ess-emcs ess-site highlight-symbol shell-pop
magit parse-time smerge-mode diff-mode key-chord memory-usage savehist
paren mic-paren pager rc/pgn-misc winner erin ami-rotate-files
flyspell ispell rc/ami-phone describe-keymap rc/ami-blog
ami-toggle-camel-hyphens persistent-thing-mode help-mode popup-ruler
rc/ami-org org-babel-init org-babel-sh org-babel-emacs-lisp
org-babel-keys org-babel-tangle org-babel-lob org-babel-comint
org-babel-table org-babel-exp org-babel-ref org-babel org-exp-blocks
org-table cal-dst cal-menu calendar cal-loaddefs org org-footnote
org-src org-list org-faces org-compat org-macs foldout noutline
outline wdired dired rc/dos2unix caml-font rc/ami-sql sql w3m-load
nxhtml-autostart nxhtml-autoload majmodpri nxhtml-menu web-autoload
nxhtml-base calc calc-loaddefs calc-macs rc/ami-eudc eudcb-ldap ldap
eudc eudc-options-file cus-edit eudc-vars rc/ami-mailcrypt mml2015
epg-config mml mml-sec mm-decode mm-bodies mm-encode mail-parse
rfc2231 rfc2047 rfc2045 ietf-drums mailcrypt rfc822
rc/ami-just-one-space rc/ami-text-mode filladapt rc/ami-fonts pulse
ps-print ps-def lpr rc/ami-ieverything ido mcomplete rc/ami-windows
windmove bbdb-autoloads bbdb timezone hi-lock uniquify rcirc time
desktop vc-git git-blame format-spec git log-edit pcvs-util
add-log ewoc autorevert linum actionscript-mode edmacro
kmacro code-review snapshot sawzall cc-mode cc-fonts
cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cs
browse-url csearch tree-widget
wid-edit etags p4-files ediff-merg ediff-diff ediff-wind ediff-help
ediff-util ediff-init ediff-mult rotate-among-files rotate-clients
string p4 warnings thingatpt shell pcomplete 
ffap url-parse auth-source eieio byte-opt bytecomp byte-compile cconv
gnus-util mm-util mail-prsvr password-cache url-vars 
gud
cc-defs python-custom advice help-fns advice-preload 
derived cl-macs gv cl cl-lib
time-date assoc grep compile elp python-mode python rx
easymenu comint ring ansi-color ami-everywhere-keys-mode
rc/ami-everywhere-keys-mode easy-mmode cus-start cus-load tooltip
ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list newcomment lisp-mode
register page menu-bar rfn-eshadow timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
dbusbind dynamic-setting system-font-setting font-render-setting
x-toolkit x multi-tty emacs)

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Eli Zaretskii]

> Date: Sun, 28 Oct 2012 21:26:55 -0700
> From: Ami Fischman <ami@fischman.org>
> Cc: 12745@debbugs.gnu.org
> 
> > If it loads the same .emacs, that is good enough.
> >
> 
> Ok, attached.

Thanks.

> (gdb) p current_buffer->name_
> $22 = 101548849
> (gdb) xtype
> Lisp_String
> (gdb) xstring current_buffer->name_
> $23 = (struct Lisp_String *) 0x60d8330
>   "cdm_wrapper.cc"

OK, so we were looking at the correct buffer.

> > Did that buffer have any minor mode or some other optional feature
> > turned on, in addition to C++ Mode?
> 
> 
> See attached b-g-e.txt, in which the current buffer is the same .cc file in
> my HEAD session loading the same .emacs as the crashed one.

Fwew!  There are quite a few minor modes you have turned on that use
overlays to do their job.  Coupled with the fact that the problem
happens after hours of work, I don't think a reproducible recipe will
be possible any time soon.  Hmmm...

> > So are you debugging a core dump?
> >
> 
> Yes.

Well, I hope that now you are running under GDB to begin with, because
debugging a live Emacs process is so much easier than debugging a core
dump.

Thanks.

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Ami Fischman]

[-- Attachment #1: Type: text/plain, Size: 2038 bytes --]

In case it's useful, replacing
printf "%s: %d glyphs\n", ($area == 0 ? "LEFT" : $area == 2 ? "RIGHT" :
"TEXT"), $used
in pgrowx with:
printf "%d glyphs\n", $used
gives me:

(gdb) pgrowx (it->glyph_row)
4 glyphs
  0    0: CHAR[ ] str=3b06db1[0] blev=0,btyp=L w=6 a+d=10+2 face=22
  1    6: CHAR[ ] str=3b06db1[1] blev=0,btyp=L w=6 a+d=10+2 face=22
  2   12: CHAR[3] str=3b06db1[2] blev=0,btyp=L w=6 a+d=10+2 face=22
  3   18: CHAR[9] str=3b06db1[3] blev=0,btyp=L w=6 a+d=10+2 face=22
23 glyphs
  0   24: CHAR[ ] pos=1275 blev=0,btyp=L w=6 a+d=10+2 MB
  1   30: CHAR[ ] pos=1276 blev=0,btyp=L w=6 a+d=10+2 MB
  2   36: CHAR[k] pos=1277 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
  3   42: CHAR[U] pos=1278 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
  4   48: CHAR[n] pos=1279 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
  5   54: CHAR[k] pos=1280 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
  6   60: CHAR[n] pos=1281 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
  7   66: CHAR[o] pos=1282 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
  8   72: CHAR[w] pos=1283 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
  9   78: CHAR[n] pos=1284 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
 10   84: CHAR[E] pos=1285 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
 11   90: CHAR[r] pos=1286 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
 12   96: CHAR[r] pos=1287 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
 13  102: CHAR[o] pos=1288 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
 14  108: CHAR[r] pos=1289 blev=0,btyp=L w=6 a+d=10+2 face=14 MB
 15  114: CHAR[ ] pos=1290 blev=0,btyp=L w=6 a+d=10+2 MB
 16  120: CHAR[=] pos=1291 blev=0,btyp=L w=6 a+d=10+2 MB
 17  126: CHAR[ ] pos=1292 blev=0,btyp=L w=6 a+d=10+2 MB
 18  132: CHAR[1] pos=1293 blev=0,btyp=L w=6 a+d=10+2 MB
 19  138: CHAR[,] pos=1294 blev=0,btyp=L w=6 a+d=10+2 MB
 20  144: CHAR[ ] str=61334f1[0] blev=0,btyp=L w=6 a+d=10+2 MB
 21  150: STRETCH[12+10] str=61335d1[1] w=354 a+d=10+2 MB
 22  504: IMAGE[0] str=61335d1[2] w=6 a+d=10+2 MB slice=0,0,6,12

(I imagine the IMAGE at the end there is
fci-mode<https://github.com/alpaker/Fill-Column-Indicator>
)

Cheers,
-a

[-- Attachment #2: Type: text/html, Size: 2562 bytes --]

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Eli Zaretskii]

> Date: Mon, 29 Oct 2012 10:56:48 -0700
> From: Ami Fischman <ami@fischman.org>
> Cc: 12745@debbugs.gnu.org
> 
> In case it's useful, replacing
> printf "%s: %d glyphs\n", ($area == 0 ? "LEFT" : $area == 2 ? "RIGHT" :
> "TEXT"), $used
> in pgrowx with:
> printf "%d glyphs\n", $used
> gives me:

Yes, it's useful, thanks.

>  19  138: CHAR[,] pos=1294 blev=0,btyp=L w=6 a+d=10+2 MB
>  20  144: CHAR[ ] str=61334f1[0] blev=0,btyp=L w=6 a+d=10+2 MB
>  21  150: STRETCH[12+10] str=61335d1[1] w=354 a+d=10+2 MB
>  22  504: IMAGE[0] str=61335d1[2] w=6 a+d=10+2 MB slice=0,0,6,12
> 
> (I imagine the IMAGE at the end there is
> fci-mode<https://github.com/alpaker/Fill-Column-Indicator>
> )

Yes, and so is the STRETCH glyph.  Can you figure out what is the
string before that, viz.:

>  20  144: CHAR[ ] str=61334f1[0] blev=0,btyp=L w=6 a+d=10+2 MB

This is a display string or an overlay string, but where does it come
from, and which mode puts it there?

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Ami Fischman]

[-- Attachment #1: Type: text/plain, Size: 787 bytes --]

On Mon, Oct 29, 2012 at 11:10 AM, Eli Zaretskii <eliz@gnu.org> wrote:

> Yes, and so is the STRETCH glyph.  Can you figure out what is the
> string before that, viz.:
> >  20  144: CHAR[ ] str=61334f1[0] blev=0,btyp=L w=6 a+d=10+2 MB
> This is a display string or an overlay string, but where does it come
> from, and which mode puts it there?
>

I'm groping in the dark, but AFAICT it's a null str?
(gdb) p/x (it->glyph_row->glyphs[1][20].object)
$39 = 0x61334f1
(gdb) p *(it->glyph_row->glyphs[1][20].object)
$40 = 0
I'm probably Doing It Wrong (tm), though.

The only guess I have is that I had entered a trailing space after that
comma, which would highlight it as trailing whitespace because of this
snippet from my .emacs:
(highlight-regexp "[ \t]+$" 'hi-blue)
using hi-lock-mode.

[-- Attachment #2: Type: text/html, Size: 1219 bytes --]

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Eli Zaretskii]

> Date: Mon, 29 Oct 2012 11:29:28 -0700
> From: Ami Fischman <ami@fischman.org>
> Cc: 12745@debbugs.gnu.org
> 
> I'm groping in the dark, but AFAICT it's a null str?

No, it looks like a string with a blank character.

> (gdb) p/x (it->glyph_row->glyphs[1][20].object)
> $39 = 0x61334f1
> (gdb) p *(it->glyph_row->glyphs[1][20].object)
> $40 = 0

To be sure, try this:

  (gdb) p it->glyph_row->glyphs[1][20].object
  (gdb) xtype

If I'm right, it should tell that the object is a Lisp string, and you
can then display it like you did with the name of the buffer.

> The only guess I have is that I had entered a trailing space after that
> comma, which would highlight it as trailing whitespace because of this
> snippet from my .emacs:
> (highlight-regexp "[ \t]+$" 'hi-blue)
> using hi-lock-mode.

Probably.  But then why isn't there a "face=" part there?  hi-blue is
not the default face, is it?

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Alp Aker]

>> Yes, and so is the STRETCH glyph.  Can you figure out what is the
>> string before that, viz.:
>> >  20  144: CHAR[ ] str=61334f1[0] blev=0,btyp=L w=6 a+d=10+2 MB
>> This is a display string or an overlay string, but where does it come
>> from, and which mode puts it there?
>
>
> I'm groping in the dark, but AFAICT it's a null str?
> (gdb) p/x (it->glyph_row->glyphs[1][20].object)
> $39 = 0x61334f1
> (gdb) p *(it->glyph_row->glyphs[1][20].object)
> $40 = 0

That string is likely also from fci-mode.  The overlay string that
mode puts at the end of each line has the structure <character> +
<stretch glyph> + <image>, where by default <character> is U+E000.  It
shouldn't be null.

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Alp Aker]

> That string is likely also from fci-mode.  The overlay string that
> mode puts at the end of each line has the structure <character> +
> <stretch glyph> + <image>, where by default <character> is U+E000.  It
> shouldn't be null.

And if I'm reading things right, the display table entry for U+E000
should be [32] in this situation.

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Ami Fischman]

[-- Attachment #1: Type: text/plain, Size: 574 bytes --]

More poking:

(gdb) p it->glyph_row->glyphs[1][20].object
$44 = 101922033
(gdb) xtype
Lisp_String
(gdb) xstring it->glyph_row->glyphs[1][20].object
$45 = (struct Lisp_String *) 0x61334f0
  ""

(gdb) p ((struct Lisp_String *)0x61334f0)->data
$48 = (unsigned char *) 0x4fe9388 ""
(gdb) p ((struct Lisp_String *)0x61334f0)->data[0]
$49 = 238 '\356'
(gdb) p ((struct Lisp_String *)0x61334f0)->data[1]
$50 = 128 '\200'
(gdb) p ((struct Lisp_String *)0x61334f0)->data[2]
$51 = 128 '\200'
(gdb) p ((struct Lisp_String *)0x61334f0)->data[3]
$52 = 0 '\000'

[-- Attachment #2: Type: text/html, Size: 876 bytes --]

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Eli Zaretskii]

> Date: Mon, 29 Oct 2012 12:23:25 -0700
> From: Ami Fischman <ami@fischman.org>
> Cc: Eli Zaretskii <eliz@gnu.org>, 12745@debbugs.gnu.org
> 
> (gdb) p it->glyph_row->glyphs[1][20].object
> $44 = 101922033
> (gdb) xtype
> Lisp_String
> (gdb) xstring it->glyph_row->glyphs[1][20].object
> $45 = (struct Lisp_String *) 0x61334f0
>   ""
> 
> (gdb) p ((struct Lisp_String *)0x61334f0)->data
> $48 = (unsigned char *) 0x4fe9388 ""

Thanks, this is clear.

Anyway, the problem happened at position 1295, which I believe is the
newline that ends this line.  I need to think...

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Eli Zaretskii]

> Date: Mon, 29 Oct 2012 15:09:22 -0400
> From: Alp Aker <alptekin.aker@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>, 12745@debbugs.gnu.org
> 
> > That string is likely also from fci-mode.  The overlay string that
> > mode puts at the end of each line has the structure <character> +
> > <stretch glyph> + <image>, where by default <character> is U+E000.  It
> > shouldn't be null.
> 
> And if I'm reading things right, the display table entry for U+E000
> should be [32] in this situation.

Out of curiosity: what is the purpose of having in an overlay this
weird character and then to display it as a space?

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Alp Aker]

> Out of curiosity: what is the purpose of having in an overlay this
> weird character and then to display it as a space?

It's a hack to achieve compatibility with packages that change the
display table entry for the newline character.  When fci-mode is
active it puts, at the end of each line, a stretch glyph and then an
image with the line segment indicating the fill column.  When, e.g.,
whitespace-mode changes the display table entry for newlines to
display a '$' at the end of the line, this would show up *after* the
stretch glyph and image unless we do something to handle that case.
So what fci-mode tries to do is intercept changes to the display table
entry for newlines.  If something like whitespace mode changes newline
display, fci-mode changes the display table back to showing newlines
as blanks, and then updates the display table entry for U+E000 to (in
this case) [?$], so that the end-of-line indication appears where the
user expects it to be.

So the weird character is just a hook on which we can hang display
vectors intended for newlines.  But when the display settings for
newlines are normal it just gets displayed as a space. This is
admittedly a gross manoeuvre, but I couldn't think of any other way to
handle the issue.

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Eli Zaretskii]

> Date: Mon, 29 Oct 2012 22:24:12 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: alptekin.aker@gmail.com, 12745@debbugs.gnu.org
> 
> Anyway, the problem happened at position 1295, which I believe is the
> newline that ends this line.  I need to think...

Ami, can you put a breakpoint in the function init_from_display_pos
and see if it ever breaks in your usage of C++ buffers?  There's
something suspicious going on in this function, but since I cannot
trigger its entry here (I even tried several of the minor modes you
have on, including fci-mode, linum-mode, and hi-lock), I'm not sure if
looking into this isn't a wild goose chase.

If you succeed in triggering the breaks inside that function, please
show the backtrace and the values of it->current and of
it->n_overlay_strings when the function returns.

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Eli Zaretskii]

> Date: Mon, 29 Oct 2012 16:42:23 -0400
> From: Alp Aker <alptekin.aker@gmail.com>
> Cc: ami@fischman.org, 12745@debbugs.gnu.org
> 
> So the weird character is just a hook on which we can hang display
> vectors intended for newlines.  But when the display settings for
> newlines are normal it just gets displayed as a space. This is
> admittedly a gross manoeuvre, but I couldn't think of any other way to
> handle the issue.

Couldn't you simply put the $ (or whatever) in the display string,
where you now put the u+e000 character?

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Ami Fischman]

[-- Attachment #1: Type: text/plain, Size: 269 bytes --]

On Mon, Oct 29, 2012 at 1:57 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> Ami, can you put a breakpoint in the function init_from_display_pos
> and see if it ever breaks in your usage of C++ buffers?


It hasn't in a few minutes of usage.  Will follow up if it does.

-a

[-- Attachment #2: Type: text/html, Size: 551 bytes --]

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Alp Aker]

> Couldn't you simply put the $ (or whatever) in the display string,
> where you now put the u+e000 character?

Probably.  IIRC I found that (a) I needed to put a cursor property on
the overlay string to get sensible cursor movement at the end of
lines, but (b) putting the cursor property on a string that gets
displayed as a stretch glyph via its display property didn't work.  So
I needed to have a blank character there to put a cursor property on,
even when the newline has a nil display table entry.  At that point it
made the code simpler to use that char for visible newline display as
well, via display table manipulation.

But if this turns out to cause too many headaches for the redisplay
routine I can rewrite it along the lines you describe.

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Alp Aker]

>> Couldn't you simply put the $ (or whatever) in the display string,
>> where you now put the u+e000 character?
>
> I needed to have a blank character there to put a cursor property on,
> even when the newline has a nil display table entry.  At that point it
> made the code simpler to use that char for visible newline display as
> well, via display table manipulation.

In other words, my earlier explanation wasn't accurate (it's been a
while since I thought about this code).  The blank character in the
overlay string is there for the sake of the cursor property.  That
it's a weird Private Use Area character instead of a space is just to
simplify the whitespace-mode compatibility hack.

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Eli Zaretskii]

> Date: Mon, 29 Oct 2012 22:24:12 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: alptekin.aker@gmail.com, 12745@debbugs.gnu.org
> 
> > Date: Mon, 29 Oct 2012 12:23:25 -0700
> > From: Ami Fischman <ami@fischman.org>
> > Cc: Eli Zaretskii <eliz@gnu.org>, 12745@debbugs.gnu.org
> > 
> > (gdb) p it->glyph_row->glyphs[1][20].object
> > $44 = 101922033
> > (gdb) xtype
> > Lisp_String
> > (gdb) xstring it->glyph_row->glyphs[1][20].object
> > $45 = (struct Lisp_String *) 0x61334f0
> >   ""
> > 
> > (gdb) p ((struct Lisp_String *)0x61334f0)->data
> > $48 = (unsigned char *) 0x4fe9388 ""
> 
> Thanks, this is clear.
> 
> Anyway, the problem happened at position 1295, which I believe is the
> newline that ends this line.  I need to think...

After some thinking, can you show the values of these members of
'struct it' in frame #7 of the crashed 24.2 session (using the core
file):

   it->n_overlay_strings
   it->overlay_strings_charpos
   it->sp

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Ami Fischman]

[-- Attachment #1: Type: text/plain, Size: 245 bytes --]

Sure:

(gdb) frame 7
#7  0x0000000000451f23 in next_overlay_string (it=0x7fff2251f1e0) at
xdisp.c:5223
5223          pop_it (it);
(gdb) p    it->n_overlay_strings
$60 = 1
(gdb) p   it->overlay_strings_charpos
$61 = 1295
(gdb) p   it->sp
$62 = 0

[-- Attachment #2: Type: text/html, Size: 387 bytes --]

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Eli Zaretskii]

> Date: Tue, 30 Oct 2012 11:29:31 -0700
> From: Ami Fischman <ami@fischman.org>
> Cc: alptekin.aker@gmail.com, 12745@debbugs.gnu.org
> 
> (gdb) frame 7
> #7  0x0000000000451f23 in next_overlay_string (it=0x7fff2251f1e0) at
> xdisp.c:5223
> 5223          pop_it (it);
> (gdb) p    it->n_overlay_strings
> $60 = 1
> (gdb) p   it->overlay_strings_charpos
> $61 = 1295
> (gdb) p   it->sp
> $62 = 0

That's what I thought, but this is normal and doesn't tell anything
about why the problem happened.  Hmm...

Is it correct that the character at buffer position 1295 is a newline?

Also, what are the values of the following?

  it->stack[0].current
  it->stack[0].string
  it->stack[0].method
  it->method

(The second one I expect to be a Lisp string or nil.)

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Ami Fischman]

[-- Attachment #1: Type: text/plain, Size: 672 bytes --]

>
> Is it correct that the character at buffer position 1295 is a newline?
>

Yes.


> Also, what are the values of the following?
>   it->stack[0].current
>

(gdb) p  it->stack[0].current
$63 = {
  pos = {
    charpos = 1295,
    bytepos = 1295
  },
  overlay_string_index = 0,
  string_pos = {
    charpos = -1,
    bytepos = -1
  },
  dpvec_index = -1
}

  it->stack[0].string
>

 (gdb) p   it->stack[0].string
$66 = 9755602
(gdb) xtype
Lisp_Symbol
(gdb) xsymbol  it->stack[0].string
$67 = (struct Lisp_Symbol *) 0x94dbd0
  "nil"

  it->stack[0].method
>

(gdb) p   it->stack[0].method
$68 = GET_FROM_BUFFER


>   it->method
>

(gdb) p it->method
$69 = GET_FROM_BUFFER

[-- Attachment #2: Type: text/html, Size: 1908 bytes --]

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Eli Zaretskii]

> Date: Tue, 30 Oct 2012 14:08:49 -0700
> From: Ami Fischman <ami@fischman.org>
> Cc: alptekin.aker@gmail.com, 12745@debbugs.gnu.org
> 
> > Is it correct that the character at buffer position 1295 is a newline?
> >
> 
> Yes.
> 
> 
> > Also, what are the values of the following?
> >   it->stack[0].current
> >
> 
> (gdb) p  it->stack[0].current
> $63 = {
>   pos = {
>     charpos = 1295,
>     bytepos = 1295
>   },
>   overlay_string_index = 0,
>   string_pos = {
>     charpos = -1,
>     bytepos = -1
>   },
>   dpvec_index = -1
> }
> 
>   it->stack[0].string
> >
> 
>  (gdb) p   it->stack[0].string
> $66 = 9755602
> (gdb) xtype
> Lisp_Symbol
> (gdb) xsymbol  it->stack[0].string
> $67 = (struct Lisp_Symbol *) 0x94dbd0
>   "nil"
> 
>   it->stack[0].method
> >
> 
> (gdb) p   it->stack[0].method
> $68 = GET_FROM_BUFFER
> 
> 
> >   it->method
> >
> 
> (gdb) p it->method
> $69 = GET_FROM_BUFFER

As expected.

I guess we need to wait for another crash, this time in the trunk
code, and take it from there.  I will meanwhile fix
init_from_display_pos, because it's certainly looks broken, and could
well be the root cause for this.

Thanks.

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Ami Fischman]

[-- Attachment #1: Type: text/plain, Size: 328 bytes --]

On Wed, Oct 31, 2012 at 8:46 AM, Eli Zaretskii <eliz@gnu.org> wrote:

> I guess we need to wait for another crash, this time in the trunk code,
> and take it from there.
>

I haven't seen the crash recur in the trunk build so I think we can
probably call this bug fixed in trunk.  I'll definitely follow up if this
re-triggers.

[-- Attachment #2: Type: text/html, Size: 637 bytes --]

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Eli Zaretskii]

> Date: Thu, 1 Nov 2012 15:25:36 -0700
> From: Ami Fischman <ami@fischman.org>
> Cc: Alp Aker <alptekin.aker@gmail.com>, 12745@debbugs.gnu.org
> 
> On Wed, Oct 31, 2012 at 8:46 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > I guess we need to wait for another crash, this time in the trunk code,
> > and take it from there.
> >
> 
> I haven't seen the crash recur in the trunk build so I think we can
> probably call this bug fixed in trunk.  I'll definitely follow up if this
> re-triggers.

OK, that's good news.  Are you running an optimized build or an
unoptimized one?  If the latter, and the problem doesn't show, I
suggest to try the optimized build of the trunk, and see if the crash
recurs.

Thanks.

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Ami Fischman]

[-- Attachment #1: Type: text/plain, Size: 79 bytes --]

Running an -O0 build.  Will rebuild & restart tomorrow and see where I end
up.

[-- Attachment #2: Type: text/html, Size: 119 bytes --]

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Eli Zaretskii]

> Date: Wed, 31 Oct 2012 17:46:48 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: alptekin.aker@gmail.com, 12745@debbugs.gnu.org
> 
> I will meanwhile fix init_from_display_pos, because it's certainly
> looks broken, and could well be the root cause for this.

Now done in revision 110767 on the emacs-24 branch.  Will appear on
the trunk on the next merge.  The patch is below, FYI.

=== modified file 'src/xdisp.c'
--- src/xdisp.c	2012-10-20 21:30:51 +0000
+++ src/xdisp.c	2012-11-03 09:25:34 +0000
@@ -928,6 +928,7 @@ static enum move_it_result
        move_it_in_display_line_to (struct it *, ptrdiff_t, int,
 				   enum move_operation_enum);
 void move_it_vertically_backward (struct it *, int);
+static void get_visually_first_element (struct it *);
 static void init_to_row_start (struct it *, struct window *,
                                struct glyph_row *);
 static int init_to_row_end (struct it *, struct window *,
@@ -3113,6 +3114,40 @@ init_from_display_pos (struct it *it, st
       eassert (STRINGP (it->string));
       it->current.string_pos = pos->string_pos;
       it->method = GET_FROM_STRING;
+      it->end_charpos = SCHARS (it->string);
+      /* Set up the bidi iterator for this overlay string.  */
+      if (it->bidi_p)
+	{
+	  it->bidi_it.string.lstring = it->string;
+	  it->bidi_it.string.s = NULL;
+	  it->bidi_it.string.schars = SCHARS (it->string);
+	  it->bidi_it.string.bufpos = it->overlay_strings_charpos;
+	  it->bidi_it.string.from_disp_str = it->string_from_display_prop_p;
+	  it->bidi_it.string.unibyte = !it->multibyte_p;
+	  bidi_init_it (IT_STRING_CHARPOS (*it), IT_STRING_BYTEPOS (*it),
+			FRAME_WINDOW_P (it->f), &it->bidi_it);
+
+	  /* Synchronize the state of the bidi iterator with
+	     pos->string_pos.  For any string position other than
+	     zero, this will be done automagically when we resume
+	     iteration over the string and get_visually_first_element
+	     is called.  But if string_pos is zero, and the string is
+	     to be reordered for display, we need to resync manually,
+	     since it could be that the iteration state recorded in
+	     pos ended at string_pos of 0 moving backwards in string.  */
+	  if (CHARPOS (pos->string_pos) == 0)
+	    {
+	      get_visually_first_element (it);
+	      if (IT_STRING_CHARPOS (*it) != 0)
+		do {
+		  /* Paranoia.  */
+		  eassert (it->bidi_it.charpos < it->bidi_it.string.schars);
+		  bidi_move_to_visually_next (&it->bidi_it);
+		} while (it->bidi_it.charpos != 0);
+	    }
+	  eassert (IT_STRING_CHARPOS (*it) == it->bidi_it.charpos
+		   && IT_STRING_BYTEPOS (*it) == it->bidi_it.bytepos);
+	}
     }
 
   if (CHARPOS (pos->string_pos) >= 0)
@@ -3122,6 +3157,9 @@ init_from_display_pos (struct it *it, st
 	 IT should already be filled with that string.  */
       it->current.string_pos = pos->string_pos;
       eassert (STRINGP (it->string));
+      if (it->bidi_p)
+	bidi_init_it (IT_STRING_CHARPOS (*it), IT_STRING_BYTEPOS (*it),
+		      FRAME_WINDOW_P (it->f), &it->bidi_it);
     }
 
   /* Restore position in display vector translations, control

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Ami Fischman]

[-- Attachment #1: Type: text/plain, Size: 4047 bytes --]

This morning I had the first crash since updating to
d7ef9678754509d426df5f6f2086ca03f7d68b1c on trunk (which doesn't include
110767's fix to init_from_display_pos), in an emacs that's been running for
10 days.

#0  0x00007f958a006b7b in raise (sig=<optimized out>) at
../nptl/sysdeps/unix/sysv/linux/pt-raise.c:42
#1  0x00000000004cf767 in terminate_due_to_signal (sig=6,
backtrace_limit=<optimized out>) at emacs.c:344
#2  0x00000000004e98c0 in emacs_abort () at sysdep.c:2061
#3  0x0000000000492515 in bidi_pop_it (bidi_it=<optimized out>) at
bidi.c:638
#4  0x00000000004492a1 in pop_it (it=0x7fff2ab33b18) at xdisp.c:5860
#5  0x0000000000453558 in next_overlay_string (it=0x7fff2ab33b18) at
xdisp.c:5309
#6  0x0000000000427a74 in set_iterator_to_next (it=0x7fff2ab33b18,
reseat_p=<optimized out>) at xdisp.c:7279
#7  0x00000000004315df in display_line (it=0x7fff2ab33b18) at xdisp.c:19790
#8  0x0000000000431008 in try_window (window=<optimized out>, flags=1,
pos=...) at xdisp.c:16300
#9  0x000000000044ce04 in redisplay_window (window=161353781,
just_this_one_p=0) at xdisp.c:15826
#10 0x0000000000452ba3 in redisplay_window_0 (window=28326) at xdisp.c:13894
#11 0x0000000000541aeb in internal_condition_case_1 (bfun=0x452b80
<redisplay_window_0>, arg=161353781, handlers=10062150, hfun=<optimized
out>) at eval.c:1326
#12 0x0000000000449b2f in redisplay_windows (window=<optimized out>) at
xdisp.c:13874
#13 0x000000000042e39b in redisplay_internal () at xdisp.c:13453
#14 0x00000000004308c4 in redisplay_preserve_echo_area (from_where=28326)
at xdisp.c:13710
#15 0x00000000004d9105 in detect_input_pending_run_timers
(do_display=<optimized out>) at keyboard.c:10276
#16 0x000000000057bce3 in wait_reading_process_output
(time_limit=<optimized out>, nsecs=<optimized out>, read_kbd=-1,
do_display=true, wait_for_cell=9858002, wait_proc=0x0,
    just_wait_proc=<optimized out>) at process.c:4749
#17 0x00000000004d7d30 in kbd_buffer_get_event (end_time=<optimized out>,
kbp=<optimized out>, used_mouse_menu=0x7fff2ab3cac7) at keyboard.c:3802
#18 read_char (commandflag=<optimized out>, nmaps=8, maps=0x7fff2ab3c930,
prev_event=9858002, used_mouse_menu=0x7fff2ab3cac7, end_time=<optimized
out>) at keyboard.c:2768
#19 0x00000000004d423d in read_key_sequence (bufsize=30, keybuf=<optimized
out>, prompt=<optimized out>, dont_downcase_last=<optimized out>,
can_return_switch_frame=<optimized out>,
    fix_current_buffer=<optimized out>) at keyboard.c:9230
#20 0x00000000004d381a in command_loop_1 () at keyboard.c:1458
#21 0x00000000005419b1 in internal_condition_case (bfun=0x4d2590
<command_loop_1>, handlers=9909682, hfun=<optimized out>) at eval.c:1288
#22 0x00000000004e2946 in command_loop_2 (ignore=<optimized out>) at
keyboard.c:1167
#23 0x0000000000541486 in internal_catch (tag=<optimized out>,
func=0x4e2920 <command_loop_2>, arg=9858002) at eval.c:1059
#24 0x00000000004d1d09 in command_loop () at keyboard.c:1146
#25 recursive_edit_1 () at keyboard.c:778
#26 0x00000000004d1e26 in Frecursive_edit () at keyboard.c:842
#27 0x00000000004d0d69 in main (argc=<error reading variable: Cannot access
memory at address 0x0>, argv=<optimized out>) at emacs.c:1552


Frame #3 aborted b/c bidi_cache_sp is 0.
Frame #4 has:
(gdb) p it->current
$2 = {
  pos = {
    charpos = 99256,
    bytepos = 99256
  },
  overlay_string_index = 0,
  string_pos = {
    charpos = -1,
    bytepos = -1
  },
  dpvec_index = -1
}

(gdb) p current_buffer->name_
$3 = 65944961
(gdb) pp current_buffer->name_
Cannot access memory at address 0x8ce6b8

(gdb) p current_buffer->text->beg[99256]@100
$5 = ' ' <repeats 24 times>, "],\n", ' ' <repeats 22 times>, "}],\n", ' '
<repeats 20 times>, "],\n", ' ' <repeats 20 times>, "'tar"

which tells me this is
common.gypi<http://src.chromium.org/viewvc/chrome/trunk/src/build/common.gypi?view=markup>,
running in gyp-mode (as opposed to the previous report which was in
cc-mode).

Let me know if you think this is worth debugging further or if I should
first sync & rebuild before a further crash will be interesting.

[-- Attachment #2: Type: text/html, Size: 5105 bytes --]

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Eli Zaretskii]

> Date: Fri, 23 Nov 2012 12:14:27 -0800
> From: Ami Fischman <ami@fischman.org>
> Cc: Alp Aker <alptekin.aker@gmail.com>, 12745@debbugs.gnu.org
> 
> This morning I had the first crash since updating to
> d7ef9678754509d426df5f6f2086ca03f7d68b1c on trunk (which doesn't include
> 110767's fix to init_from_display_pos)

If you don't have 110767, then why is it interesting to report this
crash?  That revision was supposed to fix precisely this crash.

The current revision numbers on the emacs-24 branch and on the trunk
are 110941 and 110993, respectively.  So why would you use such an old
version, from more than 3 weeks ago?

> Let me know if you think this is worth debugging further or if I should
> first sync & rebuild before a further crash will be interesting.

I certainly recommend to update to the latest code.

Thanks.

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Ami Fischman]

[-- Attachment #1: Type: text/plain, Size: 801 bytes --]

>
> If you don't have 110767, then why is it interesting to report this

crash?  That revision was supposed to fix precisely this crash.
>

I didn't get the impression from our previous exchange that you had such a
high confidence in the connection (just that you were sure it was fixing a
bug, not that it was necessarily the bug I was tripping over).


> The current revision numbers on the emacs-24 branch and on the trunk
> are 110941 and 110993, respectively.  So why would you use such an old
> version, from more than 3 weeks ago?
>

This bug took hours/days to manifest; I wanted to see whether trunk already
had it fixed.  It seems that it was at least partially fixed since this
time it took a lot longer to manifest.

I'll sync & rebuild and will report back if I get another one of these.

[-- Attachment #2: Type: text/html, Size: 1378 bytes --]

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Eli Zaretskii]

> Date: Sat, 24 Nov 2012 20:52:19 -0800
> From: Ami Fischman <ami@fischman.org>
> Cc: Alp Aker <alptekin.aker@gmail.com>, 12745@debbugs.gnu.org
> 
> > If you don't have 110767, then why is it interesting to report this
> > crash?  That revision was supposed to fix precisely this crash.
> 
> I didn't get the impression from our previous exchange that you had such a
> high confidence in the connection (just that you were sure it was fixing a
> bug, not that it was necessarily the bug I was tripping over).

I wasn't 100% sure because I could not reproduce the original problem.
But from looking at the code that caused assertion violation, I can
see no other reasons than what was fixed in 110767.

> I'll sync & rebuild and will report back if I get another one of these.

Thanks.

bug#12745: crash in bidi_pop_it during (idle) redisplay

[Glenn Morris]

Ami Fischman wrote:

> I'll sync & rebuild and will report back if I get another one of these.

Assuming that no news is good news.