Segfault during fontification of rgrep buffers [HELP NEEDED]

Segfault during fontification of rgrep buffers [HELP NEEDED]

[Herwig Hochleitner]

[-- Attachment #1: Type: text/plain, Size: 11101 bytes --]

Hi,

I've been hunting a segfault during usage of rgrep for some time, now I've
managed to catch it in gdb. From the looks of it, BEGV_ADDR is changed
during a single run of search_buffer, which leaves a stale pointer in a
local variable. I don't know enough about emacs to tell whether this is
expected, or how to fix it. Here is some info from my gdb session, please
tell me anything you might need to further investigate this. I'm also
willing to hand out my core file individually:

(gdb) dir /tmp/emacs-25.1/src/
Source directories searched: /tmp/emacs-25.1/src:$cdir:$cwd

(gdb) bt
#0  re_search_2 (bufp=bufp@entry=0xb9dac0 <searchbufs+5792>,
str1=str1@entry=0x6154ca8
<error: Cannot access memory at address 0x6154ca8>, size1=size1@entry
=1559060,
    str2=str2@entry=0x62d1fec <error: Cannot access memory at address
0x62d1fec>, size2=size2@entry=26, startpos=1556280, range=136,
regs=0xb9c3f0 <search_regs>, stop=1556416) at regex.c:4464
#1  0x00000000005395ef in search_buffer (string=string@entry=48209668,
pos=<optimized out>, pos_byte=<optimized out>, lim=lim@entry=1556364,
lim_byte=lim_byte@entry=1556417, n=1, RE=1, trt=0, inverse_trt=0,
    posix=false) at search.c:1265
#2  0x0000000000539f52 in search_command (string=48209668, bound=<optimized
out>, noerror=44832, count=<optimized out>, direction=direction@entry=1,
RE=RE@entry=1, posix=false) at search.c:1058
#3  0x000000000053a167 in Fre_search_forward (regexp=<optimized out>,
bound=<optimized out>, noerror=<optimized out>, count=<optimized out>) at
search.c:2264
#4  0x00000000005686af in Ffuncall (nargs=4, args=args@entry=0x7ffc312cf088)
at eval.c:2704
#5  0x00000000005a1803 in exec_byte_code (bytestr=<optimized out>,
vector=<optimized out>, maxdepth=<optimized out>,
args_template=args_template@entry=0, nargs=nargs@entry=0, args=<optimized
out>,
    args@entry=0x0) at bytecode.c:880
#6  0x00000000005680cd in funcall_lambda (fun=10112541, nargs=nargs@entry=3,
arg_vector=arg_vector@entry=0x7ffc312cf2c0) at eval.c:2921
#7  0x00000000005684cb in Ffuncall (nargs=4, args=args@entry=0x7ffc312cf2b8)
at eval.c:2754
#8  0x00000000005a1803 in exec_byte_code (bytestr=<optimized out>,
vector=<optimized out>, maxdepth=<optimized out>,
args_template=args_template@entry=0, nargs=nargs@entry=0, args=<optimized
out>,
    args@entry=0x0) at bytecode.c:880
#9  0x00000000005680cd in funcall_lambda (fun=10107237, nargs=nargs@entry=3,
arg_vector=arg_vector@entry=0x7ffc312cf4d0) at eval.c:2921
#10 0x00000000005684cb in Ffuncall (nargs=4, args=args@entry=0x7ffc312cf4c8)
at eval.c:2754
#11 0x00000000005a1803 in exec_byte_code (bytestr=<optimized out>,
vector=<optimized out>, maxdepth=<optimized out>,
args_template=args_template@entry=0, nargs=nargs@entry=0, args=<optimized
out>,
    args@entry=0x0) at bytecode.c:880
#12 0x00000000005680cd in funcall_lambda (fun=10104949, nargs=nargs@entry=2,
arg_vector=arg_vector@entry=0x7ffc312cf6e8) at eval.c:2921
#13 0x00000000005684cb in Ffuncall (nargs=3, args=args@entry=0x7ffc312cf6e0)
at eval.c:2754
#14 0x00000000005a1803 in exec_byte_code (bytestr=<optimized out>,
vector=<optimized out>, maxdepth=<optimized out>, args_template=<optimized
out>, nargs=nargs@entry=11260497, args=<optimized out>,
    args@entry=0x9a8684 <pure+1343396>) at bytecode.c:880
#15 0x000000000056820b in funcall_lambda (fun=140721133517472,
nargs=11260497, nargs@entry=1, arg_vector=0x9a8684 <pure+1343396>,
arg_vector@entry=0x7ffc312cf9e8) at eval.c:2855
#16 0x00000000005684cb in Ffuncall (nargs=2, args=args@entry=0x7ffc312cf9e0)
at eval.c:2754
#17 0x000000000056877c in run_hook_wrapped_funcall (nargs=<optimized out>,
args=0x7ffc312cf9e0) at eval.c:2428
#18 0x0000000000566f1d in run_hook_with_args (nargs=2, args=0x7ffc312cf9e0,
funcall=0x568760 <run_hook_wrapped_funcall>) at eval.c:2509
#19 0x00000000005685e1 in Ffuncall (nargs=3, args=args@entry=0x7ffc312cf9d8)
at eval.c:2673
#20 0x00000000005a1803 in exec_byte_code (bytestr=<optimized out>,
vector=<optimized out>, maxdepth=<optimized out>, args_template=<optimized
out>, nargs=nargs@entry=11260594, args=<optimized out>,
    args@entry=0x9a8604 <pure+1343268>) at bytecode.c:880
#21 0x000000000056820b in funcall_lambda (fun=140721133518064,
nargs=11260594, nargs@entry=2, arg_vector=0x9a8604 <pure+1343268>,
arg_vector@entry=0x7ffc312cfbf0) at eval.c:2855
#22 0x00000000005684cb in Ffuncall (nargs=3, args=args@entry=0x7ffc312cfbe8)
at eval.c:2754
#23 0x00000000005a1803 in exec_byte_code (bytestr=<optimized out>,
vector=<optimized out>, maxdepth=<optimized out>, args_template=<optimized
out>, nargs=nargs@entry=11260309, args=<optimized out>,
    args@entry=0x9a872c <pure+1343564>) at bytecode.c:880
#24 0x000000000056820b in funcall_lambda (fun=140721133518544,
nargs=11260309, nargs@entry=2, arg_vector=0x9a872c <pure+1343564>,
arg_vector@entry=0x7ffc312cfe18) at eval.c:2855
#25 0x00000000005684cb in Ffuncall (nargs=3, args=args@entry=0x7ffc312cfe10)
at eval.c:2754
#26 0x00000000005a1803 in exec_byte_code (bytestr=<optimized out>,
vector=<optimized out>, maxdepth=<optimized out>, args_template=<optimized
out>, nargs=nargs@entry=11260666, args=<optimized out>,
    args@entry=0x9a84bc <pure+1342940>) at bytecode.c:880
#27 0x000000000056820b in funcall_lambda (fun=0, nargs=11260666, nargs@entry=1,
arg_vector=0x9a84bc <pure+1342940>, arg_vector@entry=0x7ffc312d0018) at
eval.c:2855
#28 0x00000000005684cb in Ffuncall (nargs=nargs@entry=2,
args=args@entry=0x7ffc312d0010)
at eval.c:2754
#29 0x0000000000566e80 in internal_condition_case_n (bfun=0x5682c0
<Ffuncall>, nargs=nargs@entry=2, args=args@entry=0x7ffc312d0010,
handlers=handlers@entry=44832, hfun=hfun@entry=0x43dfb0 <safe_eval_handler>)
    at eval.c:1389
#30 0x000000000042ef78 in safe__call (inhibit_quit=inhibit_quit@entry=false,
nargs=nargs@entry=2, func=<optimized out>, ap=ap@entry=0x7ffc312d0090) at
xdisp.c:2558
#31 0x000000000043afac in safe_call (nargs=nargs@entry=2, func=<optimized
out>) at xdisp.c:2574
#32 0x000000000043afe2 in safe_call1 (fn=<optimized out>,
arg=arg@entry=6222870)
at xdisp.c:2585
#33 0x000000000043b110 in handle_fontified_prop (it=0x7ffc312d6300) at
xdisp.c:3805
#34 0x000000000043fdfa in handle_stop (it=it@entry=0x7ffc312d6300) at
xdisp.c:3371
#35 0x00000000004449b2 in next_element_from_buffer (it=0x7ffc312d6300) at
xdisp.c:8321
#36 0x0000000000442d25 in get_next_display_element (it=it@entry=0x7ffc312d6300)
at xdisp.c:6921
#37 0x0000000000444f62 in move_it_in_display_line_to
(it=it@entry=0x7ffc312d6300,
to_charpos=to_charpos@entry=1559034, to_x=to_x@entry=0, op=op@entry=(MOVE_TO_X
| MOVE_TO_POS)) at xdisp.c:8662
#38 0x000000000044716c in move_it_to (it=it@entry=0x7ffc312d6300,
to_charpos=to_charpos@entry=1559034, to_x=to_x@entry=-1, to_y=51,
to_vpos=to_vpos@entry=-1, op=op@entry=10) at xdisp.c:9231
#39 0x000000000044ce7f in pos_visible_p (w=w@entry=0x375b290,
charpos=1559034, x=x@entry=0x7ffc312d9c9c, y=y@entry=0x7ffc312d9ca0,
rtop=rtop@entry=0x7ffc312d9cac, rbot=rbot@entry=0x7ffc312d9cb0,
    rowh=0x7ffc312d9ca4, vpos=0x7ffc312d9ca8) at xdisp.c:1336
#40 0x0000000000464ba9 in redisplay_window (window=58045077,
just_this_one_p=just_this_one_p@entry=false) at xdisp.c:16626
#41 0x000000000046803b in redisplay_window_0 (window=window@entry=58045077)
at xdisp.c:14446
#42 0x0000000000566d6c in internal_condition_case_1 (bfun=bfun@entry=0x468010
<redisplay_window_0>, arg=58045077, handlers=<optimized out>,
hfun=hfun@entry=0x42d230 <redisplay_window_error>) at eval.c:1333
#43 0x0000000000432273 in redisplay_windows (window=58045077) at
xdisp.c:14426
#44 0x0000000000432238 in redisplay_windows (window=58044589) at
xdisp.c:14420
#45 0x0000000000454db9 in redisplay_internal () at xdisp.c:13986
#46 0x0000000000456ede in redisplay_preserve_echo_area
(from_where=from_where@entry=12) at xdisp.c:14279
#47 0x00000000005ad119 in wait_reading_process_output
(time_limit=time_limit@entry=120, nsecs=nsecs@entry=0,
read_kbd=read_kbd@entry=-1, do_display=do_display@entry=true,
wait_for_cell=wait_for_cell@entry=0,
    wait_proc=wait_proc@entry=0x0, just_wait_proc=0) at process.c:5074
#48 0x0000000000422d35 in sit_for (timeout=<optimized out>,
reading=reading@entry=true, display_option=display_option@entry=1) at
dispnew.c:5762
#49 0x00000000004fe971 in read_char (commandflag=commandflag@entry=1,
map=map@entry=76057843, prev_event=0,
used_mouse_menu=used_mouse_menu@entry=0x7ffc312de2bb,
end_time=end_time@entry=0x0) at keyboard.c:2714
#50 0x00000000004ff59b in read_key_sequence
(keybuf=keybuf@entry=0x7ffc312de3e0,
prompt=prompt@entry=0, dont_downcase_last=dont_downcase_last@entry=false,
    can_return_switch_frame=can_return_switch_frame@entry=true,
fix_current_buffer=fix_current_buffer@entry=true,
prevent_redisplay=prevent_redisplay@entry=false, bufsize=30) at
keyboard.c:9063
#51 0x000000000050131e in command_loop_1 () at keyboard.c:1365
#52 0x0000000000566cf6 in internal_condition_case (bfun=bfun@entry=0x5010e0
<command_loop_1>, handlers=handlers@entry=19056, hfun=hfun@entry=0x4f5d20
<cmd_error>) at eval.c:1309
#53 0x00000000004f2744 in command_loop_2 (ignore=ignore@entry=0) at
keyboard.c:1107
#54 0x0000000000566c7b in internal_catch (tag=tag@entry=46224,
func=func@entry=0x4f2720 <command_loop_2>, arg=arg@entry=0) at eval.c:1074
#55 0x00000000004f26ed in command_loop () at keyboard.c:1086
#56 0x00000000004f5913 in recursive_edit_1 () at keyboard.c:692
#57 0x00000000004f5c53 in Frecursive_edit () at keyboard.c:763
#58 0x0000000000418db4 in main (argc=2, argv=0x7ffc312de768) at emacs.c:1626

Lisp Backtrace:
"re-search-forward" (0x312cf090)
"font-lock-fontify-keywords-region" (0x312cf2c0)
"font-lock-default-fontify-region" (0x312cf4d0)
"font-lock-fontify-region" (0x312cf6e8)
0x429efa0 PVEC_COMPILED
"run-hook-wrapped" (0x312cf9e0)
"jit-lock--run-functions" (0x312cfbf0)
"jit-lock-fontify-now" (0x312cfe18)
"jit-lock-function" (0x312d0018)
"redisplay_internal (C function)" (0x0)

(gdb) up
#1  0x00000000005395ef in search_buffer (string=string@entry=48209668,
pos=<optimized out>, pos_byte=<optimized out>, lim=lim@entry=1556364,
lim_byte=lim_byte@entry=1556417, n=1, RE=1, trt=0, inverse_trt=0,
    posix=false) at search.c:1265
1265  val = re_search_2 (bufp, (char *) p1, s1, (char *) p2, s2,

(gdb) print p1
$30 = (unsigned char *) 0x6154ca8 <error: Cannot access memory at address
0x6154ca8>

(gdb) print BEGV_ADDR
$31 = (unsigned char *) 0x5e0dca8 "-*- mode: grep; default-directory:
\"/tmp/alt/\" -*-\nGrep started at Wed Oct 26 17:08:00\n\nfind . -type d
\\( -path \\*/SCCS -o -path \\*/RCS -o -path \\*/CVS -o -path \\*/MCVS -o
-path \\*/.src -o -path \\*/.s"...

So, while BEGV_ADDR is valid, re_search_2 gets called with an invalid
pointer, which is strange because p1 is initialized from BEGV_ADDR a few
lines earlier and there are no locations within search_buffer, where p1 is
updated. That can only mean, that BEGV_ADDR is updated somewhere within
search_buffer. Is the rgrep process supposed to be able to write to (hence
possibly reallocate) its buffer during a single search_buffer call? Can
somebody help me with producing a test case for this for the bug report?


thanks

[-- Attachment #2: Type: text/html, Size: 12068 bytes --]

Re: Segfault during fontification of rgrep buffers [HELP NEEDED]

[Noam Postavsky]

On Wed, Oct 26, 2016 at 12:09 PM, Herwig Hochleitner
<hhochleitner@gmail.com> wrote:
> Hi,
>
> I've been hunting a segfault during usage of rgrep for some time, now I've
> managed to catch it in gdb. From the looks of it, BEGV_ADDR is changed
> during a single run of search_buffer, which leaves a stale pointer in a
> local variable.


This is almost certainly the same bug as #24358. You can try building
with REL_ALLOC=no, or applying
https://debbugs.gnu.org/cgi/bugreport.cgi?msg=225;att=2;bug=24358;filename=v4-0002-Inhibit-buffer-relocation-during-regex-searches.patch.

Re: Segfault during fontification of rgrep buffers [HELP NEEDED]

[Herwig Hochleitner]

[-- Attachment #1: Type: text/plain, Size: 812 bytes --]

2016-10-26 20:36 GMT+02:00 Noam Postavsky <npostavs@users.sourceforge.net>:

> On Wed, Oct 26, 2016 at 12:09 PM, Herwig Hochleitner
> <hhochleitner@gmail.com> wrote:
> > Hi,
> >
> > I've been hunting a segfault during usage of rgrep for some time, now
> I've
> > managed to catch it in gdb. From the looks of it, BEGV_ADDR is changed
> > during a single run of search_buffer, which leaves a stale pointer in a
> > local variable.
>
>
> This is almost certainly the same bug as #24358. You can try building
> with REL_ALLOC=no, or applying
> https://debbugs.gnu.org/cgi/bugreport.cgi?msg=225;att=2;
> bug=24358;filename=v4-0002-Inhibit-buffer-relocation-
> during-regex-searches.patch.
>

That looks like the issue. I'll do a test build, and if successful, will
add the patch to NixOS' emacs 25.1 build.

Thanks!

[-- Attachment #2: Type: text/html, Size: 1549 bytes --]

Re: Segfault during fontification of rgrep buffers [HELP NEEDED]

[Noam Postavsky]

On Wed, Oct 26, 2016 at 2:44 PM, Herwig Hochleitner
<hhochleitner@gmail.com> wrote:
> 2016-10-26 20:36 GMT+02:00 Noam Postavsky <npostavs@users.sourceforge.net>:
>>
>> On Wed, Oct 26, 2016 at 12:09 PM, Herwig Hochleitner
>> <hhochleitner@gmail.com> wrote:
>> > Hi,
>> >
>> > I've been hunting a segfault during usage of rgrep for some time, now
>> > I've
>> > managed to catch it in gdb. From the looks of it, BEGV_ADDR is changed
>> > during a single run of search_buffer, which leaves a stale pointer in a
>> > local variable.
>>
>>
>> This is almost certainly the same bug as #24358. You can try building
>> with REL_ALLOC=no, or applying
>>
>> https://debbugs.gnu.org/cgi/bugreport.cgi?msg=225;att=2;bug=24358;filename=v4-0002-Inhibit-buffer-relocation-during-regex-searches.patch.
>
>
> That looks like the issue. I'll do a test build, and if successful, will add
> the patch to NixOS' emacs 25.1 build.
>
> Thanks!

Actually, there are a few other similar issues in other parts of the
code, so you should to apply all of

http://git.savannah.gnu.org/cgit/emacs.git/patch/?id=71ca4f6a43bad06192cbc4bb8c7a2d69c179b7b0
http://git.savannah.gnu.org/cgit/emacs.git/patch/?id=1047496722a58ef5b736dae64d32adeb58c5055c
http://git.savannah.gnu.org/cgit/emacs.git/patch/?id=96ac0c3ebce825e60595794f99e703ec8302e240
http://git.savannah.gnu.org/cgit/emacs.git/patch/?id=43986d16fb6ad78a627250e14570ea70bdb1f23a

(that last one is the same as what I told you earlier). Certainly just
using REL_ALLOC=no is simpler, but getting confirmation that the
patches work would be useful also.

Re: Segfault during fontification of rgrep buffers [HELP NEEDED]

[Noam Postavsky]

On Wed, Oct 26, 2016 at 3:25 PM, Noam Postavsky
<npostavs@users.sourceforge.net> wrote:
>
> Actually, there are a few other similar issues in other parts of the
> code, so you should to apply all of

Sorry, I missed this one, it has to go first to let 71ca apply

http://git.savannah.gnu.org/cgit/emacs.git/patch/?id=9afea93ed536fb9110ac62b413604cf4c4302199

>
> http://git.savannah.gnu.org/cgit/emacs.git/patch/?id=71ca4f6a43bad06192cbc4bb8c7a2d69c179b7b0
> http://git.savannah.gnu.org/cgit/emacs.git/patch/?id=1047496722a58ef5b736dae64d32adeb58c5055c
> http://git.savannah.gnu.org/cgit/emacs.git/patch/?id=96ac0c3ebce825e60595794f99e703ec8302e240
> http://git.savannah.gnu.org/cgit/emacs.git/patch/?id=43986d16fb6ad78a627250e14570ea70bdb1f23a
>
> (that last one is the same as what I told you earlier). Certainly just
> using REL_ALLOC=no is simpler, but getting confirmation that the
> patches work would be useful also.

On Thu, Oct 27, 2016 at 6:16 PM, Herwig Hochleitner
<hhochleitner@gmail.com> wrote:
> The first patch doesn't apply against 25.1, but the remaining three seem to
> fix rgrep for me.
>
> https://github.com/bendlas/nixpkgs/commit/8afa90d5ab591173b1cfdc149ab4cc9406a74982
> thanks

Re: Segfault during fontification of rgrep buffers [HELP NEEDED]

[Herwig Hochleitner]

[-- Attachment #1: Type: text/plain, Size: 259 bytes --]

I've test driven my emacs with those patches for 2 weeks and not
encountered any more segfaults.
I put them on track for NixOS' emacs 25.1 build:
https://github.com/NixOS/nixpkgs/pull/20305/commits/4d3a1f53418c6c3ae5dca321ca1e8ea92a50a3f5

Thanks!
​

[-- Attachment #2: Type: text/html, Size: 423 bytes --]