* bug#40661: Crash in regex search during redisplay @ 2020-04-16 14:35 Richard Copley 2020-04-16 15:36 ` Eli Zaretskii 0 siblings, 1 reply; 19+ messages in thread From: Richard Copley @ 2020-04-16 14:35 UTC (permalink / raw) To: 40661, dancol Recipe from emacs -Q: Save the text below in a file with extension ".pl". Repeatedly: kill the buffer and visit the file again. (You can use C-x C-v for this.) Emacs eventually encounters a segfault. Backtrace below. The text is reduced from a real program that exhibited the problem. Repeating up to about 20 times is usually enough. You can use a keyboard macro, [C-x ( C-x C-v RET C-x e e e e e], holding down the 'e' key until you get the crash. This affects both the master branch and the release branch. Bisected to this commit: 938d252d1c6c5e2027aa250c649deb024154f936 Commit: Daniel Colascione <dancol@dancol.org> CommitDate: Sat Jun 16 13:46:38 2018 -0700 Make regex matching reentrant; update syntax during match BEGIN TEXT use strict; 000000000000000000000000000000000000000000000000000000; # x sub x { } 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 0000000000000000000000000000000; "", @x; ""; eval { use autodie qw(:all); 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000000000000000000000000000000000000000000; 000000000000000000000; }; END TEXT Thread 1 received signal SIGSEGV, Segmentation fault. rpl_re_search_2 (bufp=<optimized out>, bufp@entry=0x4005f3238 <searchbufs+5432>, str1=str1@entry=0x90307fd <error: Cannot access memory at address 0x90307fd>, size1=<optimized out>, size1@entry=0, str2=str2@entry=0x90307fd <error: Cannot access memory at address 0x90307fd>, size2=size2@entry=2051, startpos=502, startpos@entry=10, range=1, regs=0x400534598 <main_thread+152>, stop=503) at regex-emacs.c:3394 3394 int len = BYTES_BY_CHAR_HEAD (*p); (gdb) bt #0 rpl_re_search_2 (bufp=<optimized out>, bufp@entry=0x4005f3238 <searchbufs+5432>, str1=str1@entry=0x90307fd <error: Cannot access memory at address 0x90307fd>, size1=<optimized out>, size1@entry=0, str2=str2@entry=0x90307fd <error: Cannot access memory at address 0x90307fd>, size2=size2@entry=2051, startpos=502, startpos@entry=10, range=1, regs=0x400534598 <main_thread+152>, stop=503) at regex-emacs.c:3394 #1 0x00000004000ea2c2 in search_buffer_re (string=string@entry=XIL(0x48dc3e4), pos=pos@entry=11, pos_byte=<optimized out>, pos_byte@entry=11, lim=lim@entry=504, lim_byte=lim_byte@entry=504, n=n@entry=1, trt=trt@entry=XIL(0), inverse_trt=inverse_trt@entry=XIL(0), posix=posix@entry=false) at search.c:1233 #2 0x00000004000ee0b1 in search_buffer (string=string@entry=XIL(0x48dc3e4), pos=11, pos_byte=11, lim=lim@entry=504, lim_byte=lim_byte@entry=504, n=n@entry=1, RE=RE@entry=1, trt=XIL(0), inverse_trt=XIL(0), posix=posix@entry=false) at search.c:1505 #3 0x00000004000ee2a8 in search_command (string=XIL(0x48dc3e4), bound=<optimized out>, noerror=XIL(0x30), count=<optimized out>, direction=direction@entry=1, RE=RE@entry=1, posix=posix@entry=false) at lisp.h:1409 #4 0x00000004000ee425 in Fre_search_forward (regexp=<optimized out>, bound=<optimized out>, noerror=<optimized out>, count=<optimized out>) at search.c:2276 #5 0x0000000400120277 in funcall_subr (subr=0x400540540 <Sre_search_forward>, numargs=numargs@entry=3, args=args@entry=0xbf8938) at eval.c:2875 #6 0x000000040011f052 in Ffuncall (nargs=4, args=args@entry=0xbf8930) at lisp.h:2113 #7 0x0000000400157ccc in exec_byte_code (bytestr=<optimized out>, vector=<optimized out>, maxdepth=<optimized out>, args_template=args_template@entry=make_fixnum(770), nargs=nargs@entry=3, args=<optimized out>, args@entry=0xbf8d00) at bytecode.c:633 #8 0x00000004001214f3 in funcall_lambda (fun=XIL(0x3f61f25), nargs=nargs@entry=3, arg_vector=arg_vector@entry=0xbf8d00) at lisp.h:1862 #9 0x000000040011f062 in Ffuncall (nargs=4, args=args@entry=0xbf8cf8) at eval.c:2796 #10 0x0000000400157ccc in exec_byte_code (bytestr=<optimized out>, vector=<optimized out>, maxdepth=<optimized out>, args_template=args_template@entry=make_fixnum(771), nargs=nargs@entry=3, args=<optimized out>, args@entry=0xbf8fc8) at bytecode.c:633 #11 0x00000004001214f3 in funcall_lambda (fun=XIL(0x3f61a65), nargs=nargs@entry=3, arg_vector=arg_vector@entry=0xbf8fc8) at lisp.h:1862 #12 0x000000040011f062 in Ffuncall (nargs=4, args=args@entry=0xbf8fc0) at eval.c:2796 #13 0x0000000400157ccc in exec_byte_code (bytestr=<optimized out>, vector=<optimized out>, maxdepth=<optimized out>, args_template=args_template@entry=make_fixnum(770), nargs=nargs@entry=2, args=<optimized out>, args@entry=0xbf91c8) at bytecode.c:633 #14 0x00000004001214f3 in funcall_lambda (fun=XIL(0x3f61995), nargs=nargs@entry=2, arg_vector=arg_vector@entry=0xbf91c8) at lisp.h:1862 #15 0x000000040011f062 in Ffuncall (nargs=3, args=args@entry=0xbf91c0) at eval.c:2796 #16 0x0000000400157ccc in exec_byte_code (bytestr=<optimized out>, vector=<optimized out>, maxdepth=<optimized out>, args_template=args_template@entry=make_fixnum(257), nargs=nargs@entry=1, args=<optimized out>, args@entry=0xbf9618) at bytecode.c:633 #17 0x00000004001214f3 in funcall_lambda (fun=XIL(0x8c7d7d5), nargs=nargs@entry=1, arg_vector=arg_vector@entry=0xbf9618) at lisp.h:1862 #18 0x000000040011f062 in Ffuncall (nargs=2, args=args@entry=0xbf9610) at eval.c:2796 #19 0x000000040011f16a in run_hook_wrapped_funcall (nargs=<optimized out>, args=0xbf9610) at eval.c:2531 #20 0x000000040011e89c in run_hook_with_args (nargs=2, args=0xbf9610, funcall=funcall@entry=0x40011f14e <run_hook_wrapped_funcall>) at eval.c:2612 #21 0x000000040011e9d6 in Frun_hook_wrapped (nargs=<optimized out>, args=<optimized out>) at eval.c:2546 #22 0x00000004001201f2 in funcall_subr (subr=0x400543f00 <Srun_hook_wrapped>, numargs=numargs@entry=2, args=args@entry=0xbf9610) at eval.c:2847 #23 0x000000040011f052 in Ffuncall (nargs=3, args=args@entry=0xbf9608) at lisp.h:2113 #24 0x0000000400157ccc in exec_byte_code (bytestr=<optimized out>, vector=<optimized out>, maxdepth=<optimized out>, args_template=args_template@entry=make_fixnum(514), nargs=nargs@entry=2, args=<optimized out>, args@entry=0xbf98a0) at bytecode.c:633 #25 0x00000004001214f3 in funcall_lambda (fun=XIL(0x400ce1d), nargs=nargs@entry=2, arg_vector=arg_vector@entry=0xbf98a0) at lisp.h:1862 #26 0x000000040011f062 in Ffuncall (nargs=3, args=args@entry=0xbf9898) at eval.c:2796 #27 0x0000000400157ccc in exec_byte_code (bytestr=<optimized out>, vector=<optimized out>, maxdepth=<optimized out>, args_template=args_template@entry=make_fixnum(512), nargs=nargs@entry=2, args=<optimized out>, args@entry=0xbf9c08) at bytecode.c:633 #28 0x00000004001214f3 in funcall_lambda (fun=XIL(0x400cb2d), nargs=nargs@entry=2, arg_vector=arg_vector@entry=0xbf9c08) at lisp.h:1862 #29 0x000000040011f062 in Ffuncall (nargs=3, args=args@entry=0xbf9c00) at eval.c:2796 #30 0x0000000400157ccc in exec_byte_code (bytestr=<optimized out>, vector=<optimized out>, maxdepth=<optimized out>, args_template=args_template@entry=make_fixnum(257), nargs=nargs@entry=1, args=<optimized out>, args@entry=0xbf9eb8) at bytecode.c:633 #31 0x00000004001214f3 in funcall_lambda (fun=XIL(0x400c775), nargs=nargs@entry=1, arg_vector=arg_vector@entry=0xbf9eb8) at lisp.h:1862 #32 0x000000040011f062 in Ffuncall (nargs=2, args=0xbf9eb0) at eval.c:2796 #33 0x000000040011e5eb in internal_condition_case_n (bfun=0x40011ee70 <Ffuncall>, nargs=nargs@entry=2, args=args@entry=0xbf9eb0, handlers=handlers@entry=XIL(0x30), hfun=hfun@entry=0x40002c8ba <safe_eval_handler>) at eval.c:1435 #34 0x000000040001a09b in safe__call (inhibit_quit=inhibit_quit@entry=false, nargs=nargs@entry=2, func=XIL(0xfffffffc03a118a0), ap=<optimized out>, ap@entry=0xbf9f50 "\006") at lisp.h:1042 #35 0x0000000400028a8a in safe_call (nargs=nargs@entry=2, func=<optimized out>) at xdisp.c:2841 #36 0x0000000400028aa3 in safe_call1 (fn=<optimized out>, arg=arg@entry=make_fixnum(1)) at xdisp.c:2852 #37 0x0000000400028ccf in handle_fontified_prop (it=0xbfa1b0) at xdisp.c:4158 #38 0x000000040002e4d5 in handle_stop (it=0xbfa1b0) at xdisp.c:3686 #39 0x000000040002e5b5 in reseat (it=0xbfa1b0, pos=..., force_p=<optimized out>) at xdisp.c:6934 #40 0x000000040002efff in init_iterator (it=it@entry=0xbfa1b0, w=w@entry=0x56eddd0, charpos=1, bytepos=1, row=<optimized out>, base_face_id=<optimized out>, base_face_id@entry=DEFAULT_FACE_ID) at xdisp.c:3287 #41 0x000000040003606c in start_display (it=it@entry=0xbfa1b0, w=w@entry=0x56eddd0, pos=...) at xdisp.c:3303 #42 0x000000040003ea6f in try_window (window=window@entry=XIL(0x56eddd5), pos=..., flags=flags@entry=1) at xdisp.c:19077 #43 0x0000000400051dea in redisplay_window (window=XIL(0x56eddd5), just_this_one_p=just_this_one_p@entry=false) at xdisp.c:18501 #44 0x00000004000538f9 in redisplay_window_0 (window=<optimized out>) at xdisp.c:16215 #45 0x000000040011e4ed in internal_condition_case_1 (bfun=bfun@entry=0x4000538c6 <redisplay_window_0>, arg=arg@entry=XIL(0x56eddd5), handlers=<optimized out>, hfun=hfun@entry=0x400015902 <redisplay_window_error>) at eval.c:1379 #46 0x000000040001c945 in redisplay_windows (window=XIL(0x56eddd5)) at xdisp.c:16195 #47 0x0000000400044416 in redisplay_internal () at xdisp.c:15663 #48 0x00000004000454a3 in redisplay () at xdisp.c:14891 #49 0x00000004000b49aa in read_char (commandflag=0, map=XIL(0), map@entry=XIL(0x8cbfda3), prev_event=XIL(0), used_mouse_menu=0x0, used_mouse_menu@entry=0xbff4cb, end_time=end_time@entry=0x0) at keyboard.c:2493 #50 0x00000004000b644b in read_key_sequence (keybuf=keybuf@entry=0xbff5d0, prompt=prompt@entry=XIL(0), dont_downcase_last=dont_downcase_last@entry=false, can_return_switch_frame=can_return_switch_frame@entry=true, fix_current_buffer=fix_current_buffer@entry=true, prevent_redisplay=prevent_redisplay@entry=false) at keyboard.c:9534 #51 0x00000004000b7785 in command_loop_1 () at lisp.h:1042 #52 0x000000040011e476 in internal_condition_case (bfun=bfun@entry=0x4000b7552 <command_loop_1>, handlers=handlers@entry=XIL(0x90), hfun=hfun@entry=0x4000ae1e4 <cmd_error>) at eval.c:1355 #53 0x00000004000a95d4 in command_loop_2 (ignore=<optimized out>) at lisp.h:1042 #54 0x000000040011e3e7 in internal_catch (tag=tag@entry=XIL(0xe070), func=func@entry=0x4000a95b8 <command_loop_2>, arg=arg@entry=XIL(0)) at eval.c:1116 #55 0x00000004000a9571 in command_loop () at lisp.h:1042 #56 0x0000000000000000 in ?? () Backtrace stopped: previous frame inner to this frame (corrupt stack?) Lisp Backtrace: "re-search-forward" (0xbf8938) "font-lock-fontify-keywords-region" (0xbf8d00) "font-lock-default-fontify-region" (0xbf8fc8) "font-lock-fontify-region" (0xbf91c8) 0x8c7d7d0 PVEC_COMPILED "run-hook-wrapped" (0xbf9610) "jit-lock--run-functions" (0xbf98a0) "jit-lock-fontify-now" (0xbf9c08) "jit-lock-function" (0xbf9eb8) "redisplay_internal (C function)" (0x0) In GNU Emacs 28.0.50 (build 1, x86_64-w64-mingw32) of 2020-04-16 built on MACHINE Repository revision: d5a7df8c02f04102d50a5cd2290262f59f2b1415 Repository branch: master Windowing system distributor 'Microsoft Corp.', version 10.0.19041 System Description: Microsoft Windows 10 Pro (v10.0.2004.19041.153) Recent messages: For information about GNU Emacs and the GNU system, type C-h C-a. Configured using: 'configure --config-cache --with-modules --without-pop --without-compress-install --without-dbus --without-gconf --without-gsettings 'CFLAGS=-Og -g -ggdb -g3'' Configured features: XPM JPEG TIFF GIF PNG RSVG SOUND NOTIFY W32NOTIFY ACL GNUTLS LIBXML2 HARFBUZZ ZLIB TOOLKIT_SCROLL_BARS MODULES THREADS JSON PDUMPER LCMS2 GMP Important settings: value of $LANG: ENG locale-coding-system: cp1252 Major mode: Perl Minor modes in effect: tooltip-mode: t global-eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t tool-bar-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t line-number-mode: t transient-mark-mode: t Load-path shadows: None found. Features: (shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs format-spec rfc822 mml easymenu mml-sec password-cache epa derived epg epg-config gnus-util rmail rmail-loaddefs text-property-search time-date subr-x seq byte-opt gv bytecomp byte-compile cconv mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils perl-mode tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel dos-w32 ls-lisp disp-table term/w32-win w32-win w32-vars term/common-win tool-bar dnd fontset image regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese composite charscript charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote threads w32notify w32 lcms2 multi-tty make-network-process emacs) Memory information: ((conses 16 47504 14219) (symbols 48 6132 1) (strings 32 17070 1654) (string-bytes 1 523301) (vectors 16 9436) (vector-slots 8 132460 12454) (floats 8 21 229) (intervals 56 238 0) (buffers 992 11)) ^ permalink raw reply [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-16 14:35 bug#40661: Crash in regex search during redisplay Richard Copley @ 2020-04-16 15:36 ` Eli Zaretskii 2020-04-16 16:33 ` Eli Zaretskii 0 siblings, 1 reply; 19+ messages in thread From: Eli Zaretskii @ 2020-04-16 15:36 UTC (permalink / raw) To: Richard Copley; +Cc: 40661 > From: Richard Copley <rcopley@gmail.com> > Date: Thu, 16 Apr 2020 15:35:20 +0100 > > Recipe from emacs -Q: > > Save the text below in a file with extension ".pl". > Repeatedly: kill the buffer and visit the file again. (You can use > C-x C-v for this.) > > Emacs eventually encounters a segfault. Backtrace below. > > The text is reduced from a real program that exhibited the problem. > Repeating up to about 20 times is usually enough. You can use a > keyboard macro, [C-x ( C-x C-v RET C-x e e e e e], holding down the > 'e' key until you get the crash. Looks like GC sometimes kicks in while we are inside re_search_2 and have pointers to buffer text lying around. This seems to fix the crash for me: diff --git a/src/search.c b/src/search.c index 818bb4a..79423be 100644 --- a/src/search.c +++ b/src/search.c @@ -1188,6 +1188,7 @@ search_buffer_re (Lisp_Object string, ptrdiff_t pos, ptrdiff_t pos_byte, ptrdiff_t count = SPECPDL_INDEX (); freeze_buffer_relocation (); freeze_pattern (cache_entry); + inhibit_garbage_collection (); while (n < 0) { ^ permalink raw reply related [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-16 15:36 ` Eli Zaretskii @ 2020-04-16 16:33 ` Eli Zaretskii 2020-04-16 16:42 ` Daniel Colascione 0 siblings, 1 reply; 19+ messages in thread From: Eli Zaretskii @ 2020-04-16 16:33 UTC (permalink / raw) To: rcopley; +Cc: 40661 > Date: Thu, 16 Apr 2020 18:36:36 +0300 > From: Eli Zaretskii <eliz@gnu.org> > Cc: 40661@debbugs.gnu.org > > Looks like GC sometimes kicks in while we are inside re_search_2 Or not. I cannot get a breakpoint inside GC to fire while we are in search_buffer_re, so maybe my hypothesis was wrong. Although the symptoms are all there: when the segfault hits, the pointers passed to re_search_2 are invalid, but BEGV_ADDR and GAP_END_ADDR, from which they are supposed to be computed, are valid (and different). And the patch does seem to avoid the segfaults. But maybe it's just a coincidence or a side effect... ^ permalink raw reply [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-16 16:33 ` Eli Zaretskii @ 2020-04-16 16:42 ` Daniel Colascione 2020-04-16 16:56 ` Richard Copley 2020-04-16 16:57 ` Eli Zaretskii 0 siblings, 2 replies; 19+ messages in thread From: Daniel Colascione @ 2020-04-16 16:42 UTC (permalink / raw) To: Eli Zaretskii, rcopley; +Cc: 40661 On April 16, 2020 9:33:16 AM PDT, Eli Zaretskii <eliz@gnu.org> wrote: >> Date: Thu, 16 Apr 2020 18:36:36 +0300 >> From: Eli Zaretskii <eliz@gnu.org> >> Cc: 40661@debbugs.gnu.org >> >> Looks like GC sometimes kicks in while we are inside re_search_2 > >Or not. I cannot get a breakpoint inside GC to fire while we are in >search_buffer_re, so maybe my hypothesis was wrong. Although the >symptoms are all there: when the segfault hits, the pointers passed to >re_search_2 are invalid, but BEGV_ADDR and GAP_END_ADDR, from which >they are supposed to be computed, are valid (and different). And the >patch does seem to avoid the segfaults. But maybe it's just a >coincidence or a side effect... Try using rr and see where those pointers came from ^ permalink raw reply [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-16 16:42 ` Daniel Colascione @ 2020-04-16 16:56 ` Richard Copley 2020-04-16 17:24 ` Daniel Colascione 2020-04-16 16:57 ` Eli Zaretskii 1 sibling, 1 reply; 19+ messages in thread From: Richard Copley @ 2020-04-16 16:56 UTC (permalink / raw) To: Daniel Colascione; +Cc: 40661 On Thu, 16 Apr 2020 at 17:42, Daniel Colascione <dancol@dancol.org> wrote: > > On April 16, 2020 9:33:16 AM PDT, Eli Zaretskii <eliz@gnu.org> wrote: > >> Date: Thu, 16 Apr 2020 18:36:36 +0300 > >> From: Eli Zaretskii <eliz@gnu.org> > >> Cc: 40661@debbugs.gnu.org > >> > >> Looks like GC sometimes kicks in while we are inside re_search_2 > > > >Or not. I cannot get a breakpoint inside GC to fire while we are in > >search_buffer_re, so maybe my hypothesis was wrong. Although the > >symptoms are all there: when the segfault hits, the pointers passed to > >re_search_2 are invalid, but BEGV_ADDR and GAP_END_ADDR, from which > >they are supposed to be computed, are valid (and different). And the > >patch does seem to avoid the segfaults. But maybe it's just a > >coincidence or a side effect... > > Try using rr and see where those pointers came from It seems clear from "str1=str1@entry=0xc607fd", etc., that they come from the caller, search_buffer_re. The question is, why are they no longer valid after updating syntax? ^ permalink raw reply [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-16 16:56 ` Richard Copley @ 2020-04-16 17:24 ` Daniel Colascione 2020-04-16 19:35 ` Richard Copley 0 siblings, 1 reply; 19+ messages in thread From: Daniel Colascione @ 2020-04-16 17:24 UTC (permalink / raw) To: Richard Copley; +Cc: 40661 On 4/16/20 9:56 AM, Richard Copley wrote: > On Thu, 16 Apr 2020 at 17:42, Daniel Colascione <dancol@dancol.org> wrote: >> >> On April 16, 2020 9:33:16 AM PDT, Eli Zaretskii <eliz@gnu.org> wrote: >>>> Date: Thu, 16 Apr 2020 18:36:36 +0300 >>>> From: Eli Zaretskii <eliz@gnu.org> >>>> Cc: 40661@debbugs.gnu.org >>>> >>>> Looks like GC sometimes kicks in while we are inside re_search_2 >>> >>> Or not. I cannot get a breakpoint inside GC to fire while we are in >>> search_buffer_re, so maybe my hypothesis was wrong. Although the >>> symptoms are all there: when the segfault hits, the pointers passed to >>> re_search_2 are invalid, but BEGV_ADDR and GAP_END_ADDR, from which >>> they are supposed to be computed, are valid (and different). And the >>> patch does seem to avoid the segfaults. But maybe it's just a >>> coincidence or a side effect... >> >> Try using rr and see where those pointers came from > > It seems clear from "str1=str1@entry=0xc607fd", etc., that they come > from the caller, search_buffer_re. The question is, why are they no > longer valid after updating syntax? Right. So let's see what updated the valid pointers and invalidated the invalid ones. ^ permalink raw reply [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-16 17:24 ` Daniel Colascione @ 2020-04-16 19:35 ` Richard Copley 2020-04-17 11:22 ` Eli Zaretskii 0 siblings, 1 reply; 19+ messages in thread From: Richard Copley @ 2020-04-16 19:35 UTC (permalink / raw) To: Daniel Colascione; +Cc: 40661 On Thu, 16 Apr 2020 at 18:24, Daniel Colascione <dancol@dancol.org> wrote: > > On 4/16/20 9:56 AM, Richard Copley wrote: > > On Thu, 16 Apr 2020 at 17:42, Daniel Colascione <dancol@dancol.org> wrote: > >> > >> On April 16, 2020 9:33:16 AM PDT, Eli Zaretskii <eliz@gnu.org> wrote: > >>>> Date: Thu, 16 Apr 2020 18:36:36 +0300 > >>>> From: Eli Zaretskii <eliz@gnu.org> > >>>> Cc: 40661@debbugs.gnu.org > >>>> > >>>> Looks like GC sometimes kicks in while we are inside re_search_2 > >>> > >>> Or not. I cannot get a breakpoint inside GC to fire while we are in > >>> search_buffer_re, so maybe my hypothesis was wrong. Although the > >>> symptoms are all there: when the segfault hits, the pointers passed to > >>> re_search_2 are invalid, but BEGV_ADDR and GAP_END_ADDR, from which > >>> they are supposed to be computed, are valid (and different). And the > >>> patch does seem to avoid the segfaults. But maybe it's just a > >>> coincidence or a side effect... > >> > >> Try using rr and see where those pointers came from > > > > It seems clear from "str1=str1@entry=0xc607fd", etc., that they come > > from the caller, search_buffer_re. The question is, why are they no > > longer valid after updating syntax? > > Right. So let's see what updated the valid pointers and invalidated the > invalid ones. Right, I see. Anyway, I wasn't able to reproduce the bug under GNU/Linux (in order to use rr), or make much progress with GDB on Windows. ^ permalink raw reply [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-16 19:35 ` Richard Copley @ 2020-04-17 11:22 ` Eli Zaretskii 2020-04-17 13:55 ` Eli Zaretskii 2020-04-17 15:28 ` Stefan Monnier 0 siblings, 2 replies; 19+ messages in thread From: Eli Zaretskii @ 2020-04-17 11:22 UTC (permalink / raw) To: Richard Copley, Stefan Monnier; +Cc: 40661 > From: Richard Copley <rcopley@gmail.com> > Date: Thu, 16 Apr 2020 20:35:19 +0100 > Cc: Eli Zaretskii <eliz@gnu.org>, 40661@debbugs.gnu.org > > On Thu, 16 Apr 2020 at 18:24, Daniel Colascione <dancol@dancol.org> wrote: > > > > On 4/16/20 9:56 AM, Richard Copley wrote: > > > On Thu, 16 Apr 2020 at 17:42, Daniel Colascione <dancol@dancol.org> wrote: > > >> > > >> On April 16, 2020 9:33:16 AM PDT, Eli Zaretskii <eliz@gnu.org> wrote: > > >>>> Date: Thu, 16 Apr 2020 18:36:36 +0300 > > >>>> From: Eli Zaretskii <eliz@gnu.org> > > >>>> Cc: 40661@debbugs.gnu.org > > >>>> > > >>>> Looks like GC sometimes kicks in while we are inside re_search_2 > > >>> > > >>> Or not. I cannot get a breakpoint inside GC to fire while we are in > > >>> search_buffer_re, so maybe my hypothesis was wrong. Although the > > >>> symptoms are all there: when the segfault hits, the pointers passed to > > >>> re_search_2 are invalid, but BEGV_ADDR and GAP_END_ADDR, from which > > >>> they are supposed to be computed, are valid (and different). And the > > >>> patch does seem to avoid the segfaults. But maybe it's just a > > >>> coincidence or a side effect... > > >> > > >> Try using rr and see where those pointers came from > > > > > > It seems clear from "str1=str1@entry=0xc607fd", etc., that they come > > > from the caller, search_buffer_re. The question is, why are they no > > > longer valid after updating syntax? > > > > Right. So let's see what updated the valid pointers and invalidated the > > invalid ones. > > Right, I see. Anyway, I wasn't able to reproduce the bug under > GNU/Linux (in order to use rr), or make much progress with GDB on > Windows. I finally succeeded to build a sophisticated enough trap to catch the culprit. It's GC allright. Which is not surprising: the commit pointed out by Richard changed re_match_2_internal to call UPDATE_SYNTAX_TABLE_FORWARD, which calls Lisp, and thus can trigger GC. As seen from the backtrace, GC then calls compact_buffer, which calls enlarge_buffer_text (to shrink buffer text), and on MS-Windows -- and this is the w32-specific part -- we return some memory to the OS and relocate buffer text. Obviously, we cannot allow GC to run while regex routines do their work, because they are passed C pointers to buffer text. The question is, where to disable GC? We could do it inside update_syntax_table_forward, but UPDATE_SYNTAX_TABLE_FORWARD is called from many places that evidently have no problems with GC. So my suggestion would be to disable GC inside re_match_2_internal instead. Comments? Here's the full backtrace I caught: mmap_alloc (var=0x6843168, nbytes=2257) at w32heap.c:676 676 if (*var == NULL) #0 mmap_alloc (var=0x6843168, nbytes=2257) at w32heap.c:676 #1 0x015a7934 in mmap_realloc (var=0x6843168, nbytes=2257) at w32heap.c:784 #2 0x0124d431 in enlarge_buffer_text (b=0x6842f10, delta=-1840) at buffer.c:5049 #3 0x01262107 in make_gap_smaller (nbytes_removed=1840) at insdel.c:549 #4 0x0126221c in make_gap (nbytes_added=-1840) at insdel.c:589 #5 0x01262246 in make_gap_1 (b=0x6842f10, nbytes=-1840) at insdel.c:602 #6 0x012427e8 in compact_buffer (buffer=0x6842f10) at buffer.c:1672 #7 0x01314c2e in garbage_collect () at alloc.c:5877 #8 0x01314b9a in maybe_garbage_collect () at alloc.c:5853 #9 0x0137696d in maybe_gc () at lisp.h:5065 #10 0x013848c2 in Ffuncall (nargs=4, args=0x824360) at eval.c:2778 #11 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000006865a98), vector=XIL(0xa000000006889d58), maxdepth=make_fixnum(7), args_template=make_fixnum(257), nargs=1, args=0x824918) at bytecode.c:633 #12 0x01385af1 in funcall_lambda (fun=XIL(0xa0000000071f2c90), nargs=1, arg_vector=0x824910) at eval.c:2989 #13 0x01384a33 in Ffuncall (nargs=2, args=0x824908) at eval.c:2796 #14 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000006865ad8), vector=XIL(0xa0000000072005a0), maxdepth=make_fixnum(17), args_template=make_fixnum(257), nargs=1, args=0x825058) at bytecode.c:633 #15 0x01385af1 in funcall_lambda (fun=XIL(0xa0000000071e4b40), nargs=1, arg_vector=0x825050) at eval.c:2989 #16 0x01384a33 in Ffuncall (nargs=2, args=0x825048) at eval.c:2796 #17 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000006865a18), vector=XIL(0xa000000006828648), maxdepth=make_fixnum(15), args_template=make_fixnum(514), nargs=2, args=0x825868) at bytecode.c:633 #18 0x01385af1 in funcall_lambda (fun=XIL(0xa000000007167768), nargs=2, arg_vector=0x825858) at eval.c:2989 #19 0x01384a33 in Ffuncall (nargs=3, args=0x825850) at eval.c:2796 #20 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005e847e0), vector=XIL(0xa000000005e84438), maxdepth=make_fixnum(12), args_template=make_fixnum(257), nargs=1, args=0x825e98) at bytecode.c:633 #21 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e84408), nargs=1, arg_vector=0x825e90) at eval.c:2989 #22 0x01384a33 in Ffuncall (nargs=2, args=0x825e88) at eval.c:2796 #23 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005ecf1d8), vector=XIL(0xa000000005ecf0e0), maxdepth=make_fixnum(8), args_template=make_fixnum(257), nargs=1, args=0x826460) at bytecode.c:633 #24 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005ecf0b0), nargs=1, arg_vector=0x826458) at eval.c:2989 #25 0x01384a33 in Ffuncall (nargs=2, args=0x826450) at eval.c:2796 #26 0x0137f3e7 in internal_condition_case_n (bfun=0x13847ec <Ffuncall>, nargs=2, args=0x826450, handlers=XIL(0x30), hfun=0x105a91d <safe_eval_handler>) at eval.c:1435 #27 0x0105ab54 in safe__call (inhibit_quit=false, nargs=2, func=XIL(0x7fb0), ap=0x826514 "") at xdisp.c:2807 #28 0x0105abcd in safe_call (nargs=2, func=XIL(0x7fb0)) at xdisp.c:2822 #29 0x0105ac21 in safe_call1 (fn=XIL(0x7fb0), arg=make_fixnum(505)) at xdisp.c:2833 #30 0x014128bb in parse_sexp_propertize (charpos=504) at syntax.c:480 #31 0x01412aa3 in update_syntax_table_forward (charpos=504, init=false, object=XIL(0)) at syntax.c:513 #32 0x012e9cb5 in UPDATE_SYNTAX_TABLE_FORWARD (charpos=504) at syntax.h:185 #33 0x012e9d39 in UPDATE_SYNTAX_TABLE (charpos=504) at syntax.h:205 #34 0x012f81f9 in re_match_2_internal (bufp=0x1bb43a0 <searchbufs+4064>, string1=0x75a07fd '0' <repeats 59 times>, ";\n", '0' <repeats 60 times>, ";\ n", '0' <repeats 60 times>, ";\n", '0' <repeats 21 times>, ";\n};\n", size1=0, string2=0x75a07fd '0' <repeats 59 times>, ";\n", '0' <repeats 60 times>, ";\ n", '0' <repeats 60 times>, ";\n", '0' <repeats 21 times>, ";\n};\n", size2=2051, pos=502, regs=0x1958074 <main_thread+116>, stop=503) at regex-emacs.c:4780 #35 0x012f39f3 in rpl_re_search_2 (bufp=0x1bb43a0 <searchbufs+4064>, str1=0x75a07fd '0' <repeats 59 times>, ";\n", '0' <repeats 60 times>, ";\n", '0' <repeats 60 times>, ";\n", '0' <repeats 21 times>, ";\n};\n", size1=0, str2=0x75a07fd '0' <repeats 59 times>, ";\n", '0' <repeats 60 times>, ";\n", '0' <repeats 60 times>, ";\n", '0' <repeats 21 times>, ";\n};\n", size2=2051, startpos=502, range=1, regs=0x1958074 <main_thread+116>, stop=503) at regex-emacs.c:3373 #36 0x012dc6e0 in search_buffer_re (string=XIL(0x8000000006865028), pos=11, pos_byte=11, lim=504, lim_byte=504, n=1, trt=XIL(0), inverse_trt=XIL(0), posix=false) at search.c:1244 #37 0x012dd74d in search_buffer (string=XIL(0x8000000006865028), pos=11, pos_byte=11, lim=504, lim_byte=504, n=1, RE=1, trt=XIL(0), inverse_trt=XIL(0), posix=false) at search.c:1506 #38 0x012dbefa in search_command (string=XIL(0x8000000006865028), bound=make_fixnum(504), noerror=XIL(0x30), count=XIL(0), direction=1, RE=1, posix=false) at search.c:1048 #39 0x012df7bc in Fre_search_forward (regexp=XIL(0x8000000006865028), bound=make_fixnum(504), noerror=XIL(0x30), count=XIL(0)) at search.c:2277 #40 0x01385067 in funcall_subr (subr=0x195ebc0 <Sre_search_forward>, numargs=3, args=0x8274d8) at eval.c:2875 #41 0x013849d9 in Ffuncall (nargs=4, args=0x8274d0) at eval.c:2794 #42 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005eed790), vector=XIL(0xa000000005e25f58), maxdepth=make_fixnum(25), args_template=make_fixnum(770), nargs=3, args=0x827c38) at bytecode.c:633 #43 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e25f28), nargs=3, arg_vector=0x827c20) at eval.c:2989 #44 0x01384a33 in Ffuncall (nargs=4, args=0x827c18) at eval.c:2796 #45 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005eee538), vector=XIL(0xa000000005e25ac0), maxdepth=make_fixnum(14), args_template=make_fixnum(771), nargs=3, args=0x828280) at bytecode.c:633 #46 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e25a90), nargs=3, arg_vector=0x828268) at eval.c:2989 #47 0x01384a33 in Ffuncall (nargs=4, args=0x828260) at eval.c:2796 #48 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005eee748), vector=XIL(0xa000000005e259f0), maxdepth=make_fixnum(7), args_template=make_fixnum(770), nargs=2, args=0x8287e8) at bytecode.c:633 #49 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e259c0), nargs=2, arg_vector=0x8287d8) at eval.c:2989 #50 0x01384a33 in Ffuncall (nargs=3, args=0x8287d0) at eval.c:2796 #51 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005e8ab40), vector=XIL(0xa000000007255b70), maxdepth=make_fixnum(10), args_template=make_fixnum(257), nargs=1, args=0x829060) at bytecode.c:633 #52 0x01385af1 in funcall_lambda (fun=XIL(0xa000000007255bc0), nargs=1, arg_vector=0x829058) at eval.c:2989 #53 0x01384a33 in Ffuncall (nargs=2, args=0x829050) at eval.c:2796 #54 0x013837c6 in run_hook_wrapped_funcall (nargs=2, args=0x829050) at eval.c:2531 #55 0x01383ca9 in run_hook_with_args (nargs=2, args=0x829050, funcall=0x138377e <run_hook_wrapped_funcall>) at eval.c:2612 #56 0x01383815 in Frun_hook_wrapped (nargs=2, args=0x829050) at eval.c:2546 #57 0x01384e99 in funcall_subr (subr=0x1960bc0 <Srun_hook_wrapped>, numargs=2, args=0x829050) at eval.c:2847 #58 0x013849d9 in Ffuncall (nargs=3, args=0x829048) at eval.c:2794 #59 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005e8abf0), vector=XIL(0xa000000005e8aac8), maxdepth=make_fixnum(19), args_template=make_fixnum(514), nargs=2, args=0x829670) at bytecode.c:633 #60 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e8aa98), nargs=2, arg_vector=0x829660) at eval.c:2989 #61 0x01384a33 in Ffuncall (nargs=3, args=0x829658) at eval.c:2796 #62 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005e8b048), vector=XIL(0xa000000005e8a818), maxdepth=make_fixnum(27), args_template=make_fixnum(512), nargs=2, args=0x829d58) at bytecode.c:633 #63 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e8a7e8), nargs=2, arg_vector=0x829d48) at eval.c:2989 #64 0x01384a33 in Ffuncall (nargs=3, args=0x829d40) at eval.c:2796 #65 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005e8d5d0), vector=XIL(0xa000000005e8d040), maxdepth=make_fixnum(12), args_template=make_fixnum(257), nargs=1, args=0x82a380) at bytecode.c:633 #66 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005e8d010), nargs=1, arg_vector=0x82a378) at eval.c:2989 #67 0x01384a33 in Ffuncall (nargs=2, args=0x82a370) at eval.c:2796 #68 0x0137f3e7 in internal_condition_case_n (bfun=0x13847ec <Ffuncall>, nargs=2, args=0x82a370, handlers=XIL(0x30), hfun=0x105a91d <safe_eval_handler>) at eval.c:1435 #69 0x0105ab54 in safe__call (inhibit_quit=false, nargs=2, func=XIL(0x42cb120), ap=0x82a434 "") at xdisp.c:2807 #70 0x0105abcd in safe_call (nargs=2, func=XIL(0x42cb120)) at xdisp.c:2822 #71 0x0105ac21 in safe_call1 (fn=XIL(0x42cb120), arg=make_fixnum(1)) at xdisp.c:2833 #72 0x0105e9c7 in handle_fontified_prop (it=0x82a6c0) at xdisp.c:4136 #73 0x0105d271 in handle_stop (it=0x82a6c0) at xdisp.c:3664 #74 0x01069335 in reseat (it=0x82a6c0, pos=..., force_p=true) at xdisp.c:6900 #75 0x0105c4c3 in init_iterator (it=0x82a6c0, w=0x70c0b88, charpos=1, bytepos=1, row=0x68432b0, base_face_id=DEFAULT_FACE_ID) at xdisp.c:3265 #76 0x0105c55b in start_display (it=0x82a6c0, w=0x70c0b88, pos=...) at xdisp.c:3281 #77 0x01091a71 in try_window (window=XIL(0xa0000000070c0b88), pos=..., flags=1) at xdisp.c:19056 #78 0x0108e687 in redisplay_window (window=XIL(0xa0000000070c0b88), just_this_one_p=false) at xdisp.c:18480 #79 0x01086178 in redisplay_window_0 (window=XIL(0xa0000000070c0b88)) at xdisp.c:16194 #80 0x0137f1f7 in internal_condition_case_1 ( bfun=0x108611d <redisplay_window_0>, arg=XIL(0xa0000000070c0b88), handlers=XIL(0xc000000005fd6d40), hfun=0x10860da <redisplay_window_error>) at eval.c:1379 #81 0x0108609d in redisplay_windows (window=XIL(0xa0000000070c0b88)) at xdisp.c:16174 #82 0x010848b0 in redisplay_internal () at xdisp.c:15642 #83 0x0108577c in redisplay_preserve_echo_area (from_where=2) at xdisp.c:15995 #84 0x01019565 in Fredisplay (force=XIL(0)) at dispnew.c:6085 #85 0x01384f7d in funcall_subr (subr=0x195a0e0 <Sredisplay>, numargs=0, args=0x82e128) at eval.c:2867 #86 0x013849d9 in Ffuncall (nargs=1, args=0x82e120) at eval.c:2794 #87 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005d746e0), vector=XIL(0xa000000005d744f0), maxdepth=make_fixnum(7), args_template=make_fixnum(769), nargs=1, args=0x82e638) at bytecode.c:633 #88 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005d744c0), nargs=1, arg_vector=0x82e630) at eval.c:2989 #89 0x013855f9 in apply_lambda (fun=XIL(0xa000000005d744c0), args=XIL(0xc000000000fc65e0), count=10) at eval.c:2926 #90 0x01382c31 in eval_sub (form=XIL(0xc000000000fc65f0)) at eval.c:2318 #91 0x0137afa2 in Fprogn (body=XIL(0xc000000000fc6580)) at eval.c:462 #92 0x013860de in funcall_lambda (fun=XIL(0xc000000000fc55b0), nargs=0, arg_vector=0x82ecd8) at eval.c:3060 #93 0x01384bef in Ffuncall (nargs=1, args=0x82ecd0) at eval.c:2808 #94 0x0136e970 in Ffuncall_interactively (nargs=1, args=0x82ecd0) at callint.c:254 #95 0x01384e99 in funcall_subr (subr=0x1960720 <Sfuncall_interactively>, numargs=1, args=0x82ecd0) at eval.c:2847 #96 0x013849d9 in Ffuncall (nargs=2, args=0x82ecc8) at eval.c:2794 #97 0x013830ff in Fapply (nargs=3, args=0x82ecc8) at eval.c:2377 #98 0x0136f077 in Fcall_interactively (function=XIL(0x4cc4f70), record_flag=XIL(0), keys=XIL(0xa000000006803768)) at callint.c:342 #99 0x01384ff4 in funcall_subr (subr=0x1960740 <Scall_interactively>, numargs=3, args=0x82f010) at eval.c:2872 #100 0x013849d9 in Ffuncall (nargs=4, args=0x82f008) at eval.c:2794 #101 0x01427eb3 in exec_byte_code (bytestr=XIL(0x8000000005ecd2a8), vector=XIL(0xa000000005ecd050), maxdepth=make_fixnum(13), args_template=make_fixnum(1025), nargs=1, args=0x82f610) at bytecode.c:633 #102 0x01385af1 in funcall_lambda (fun=XIL(0xa000000005ecd020), nargs=1, arg_vector=0x82f608) at eval.c:2989 #103 0x01384a33 in Ffuncall (nargs=2, args=0x82f600) at eval.c:2796 #104 0x01383eec in call1 (fn=XIL(0x3f30), arg1=XIL(0x4cc4f70)) at eval.c:2654 #105 0x011e09da in command_loop_1 () at keyboard.c:1463 #106 0x0137f10d in internal_condition_case (bfun=0x11dfd8f <command_loop_1>, handlers=XIL(0x90), hfun=0x11df003 <cmd_error>) at eval.c:1355 #107 0x011df806 in command_loop_2 (ignore=XIL(0)) at keyboard.c:1091 #108 0x0137e2d8 in internal_catch (tag=XIL(0xdfb0), func=0x11df7ca <command_loop_2>, arg=XIL(0)) at eval.c:1116 #109 0x011df785 in command_loop () at keyboard.c:1070 #110 0x011dea8b in recursive_edit_1 () at keyboard.c:714 #111 0x011ded01 in Frecursive_edit () at keyboard.c:786 #112 0x011d361b in main (argc=2, argv=0xa42848) at emacs.c:2054 Lisp Backtrace: "Automatic GC" (0x0) "modify-syntax-entry" (0x824368) "perl-quote-syntax-table" (0x824910) "perl-syntax-propertize-special-constructs" (0x825050) "perl-syntax-propertize-function" (0x825858) "syntax-propertize" (0x825e90) "internal--syntax-propertize" (0x826458) "re-search-forward" (0x8274d8) "font-lock-fontify-keywords-region" (0x827c20) "font-lock-default-fontify-region" (0x828268) "font-lock-fontify-region" (0x8287d8) 0x7255bc0 PVEC_COMPILED "run-hook-wrapped" (0x829050) "jit-lock--run-functions" (0x829660) "jit-lock-fontify-now" (0x829d48) "jit-lock-function" (0x82a378) "redisplay_internal (C function)" (0x0) "redisplay" (0x82e128) "sit-for" (0x82e630) "foo" (0x82ecd8) "funcall-interactively" (0x82ecd0) "call-interactively" (0x82f010) "command-execute" (0x82f608) ^ permalink raw reply [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-17 11:22 ` Eli Zaretskii @ 2020-04-17 13:55 ` Eli Zaretskii 2020-04-17 14:01 ` Daniel Colascione 2020-04-17 15:28 ` Stefan Monnier 1 sibling, 1 reply; 19+ messages in thread From: Eli Zaretskii @ 2020-04-17 13:55 UTC (permalink / raw) To: rcopley; +Cc: monnier, 40661 > Date: Fri, 17 Apr 2020 14:22:00 +0300 > From: Eli Zaretskii <eliz@gnu.org> > Cc: 40661@debbugs.gnu.org > > Obviously, we cannot allow GC to run while regex routines do their > work, because they are passed C pointers to buffer text. The question > is, where to disable GC? We could do it inside > update_syntax_table_forward, but UPDATE_SYNTAX_TABLE_FORWARD is called > from many places that evidently have no problems with GC. So my > suggestion would be to disable GC inside re_match_2_internal instead. Alternatively, we could set the buffer's inhibit_shrinking flag while in re_match_2_internal. Although that flag was introduced for a different purpose: for when we have stuff inside the gap that we don't want to lose. The name of the flag notwithstanding, I'm not sure we want to conflate these two purposes. But maybe it's better than preventing the GC entirely. ^ permalink raw reply [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-17 13:55 ` Eli Zaretskii @ 2020-04-17 14:01 ` Daniel Colascione 0 siblings, 0 replies; 19+ messages in thread From: Daniel Colascione @ 2020-04-17 14:01 UTC (permalink / raw) To: Eli Zaretskii, rcopley; +Cc: monnier, 40661 On April 17, 2020 6:56:00 AM Eli Zaretskii <eliz@gnu.org> wrote: >> Date: Fri, 17 Apr 2020 14:22:00 +0300 >> From: Eli Zaretskii <eliz@gnu.org> >> Cc: 40661@debbugs.gnu.org >> >> Obviously, we cannot allow GC to run while regex routines do their >> work, because they are passed C pointers to buffer text. The question >> is, where to disable GC? We could do it inside >> update_syntax_table_forward, but UPDATE_SYNTAX_TABLE_FORWARD is called >> from many places that evidently have no problems with GC. So my >> suggestion would be to disable GC inside re_match_2_internal instead. > > Alternatively, we could set the buffer's inhibit_shrinking flag while > in re_match_2_internal. Although that flag was introduced for a > different purpose: for when we have stuff inside the gap that we don't > want to lose. The name of the flag notwithstanding, I'm not sure we > want to conflate these two purposes. But maybe it's better than > preventing the GC entirely. I think I'd prefer this approach to inhibiting GC entirely. I can imagine code allocating enough garbage that we really want to get rid of it, and inhibiting shrinking is more conservative. ^ permalink raw reply [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-17 11:22 ` Eli Zaretskii 2020-04-17 13:55 ` Eli Zaretskii @ 2020-04-17 15:28 ` Stefan Monnier 2020-04-17 15:50 ` Eli Zaretskii 1 sibling, 1 reply; 19+ messages in thread From: Stefan Monnier @ 2020-04-17 15:28 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Richard Copley, 40661 > I finally succeeded to build a sophisticated enough trap to catch the > culprit. It's GC allright. Which is not surprising: the commit > pointed out by Richard changed re_match_2_internal to call > UPDATE_SYNTAX_TABLE_FORWARD, which calls Lisp, and thus can trigger > GC. As seen from the backtrace, GC then calls compact_buffer, which > calls enlarge_buffer_text (to shrink buffer text), and on MS-Windows > -- and this is the w32-specific part -- we return some memory to the > OS and relocate buffer text. > > Obviously, we cannot allow GC to run while regex routines do their > work, because they are passed C pointers to buffer text. The question > is, where to disable GC? We could do it inside > update_syntax_table_forward, but UPDATE_SYNTAX_TABLE_FORWARD is called > from many places that evidently have no problems with GC. So my > suggestion would be to disable GC inside re_match_2_internal instead. > > Comments? Looks fine to me. I think a better fix is to move the execution of compact_buffer: there's no reason it has to be done during GC, we just need to do it "every once in a while" and the GC was a convenient point for that. But we could avoid several such problems if we were to run such background tasks elsewhere. It could still be linked to GC, e.g. we could start a timer during the GC so its run at the next opportunity. Stefan ^ permalink raw reply [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-17 15:28 ` Stefan Monnier @ 2020-04-17 15:50 ` Eli Zaretskii 2020-04-17 16:00 ` Stefan Monnier 0 siblings, 1 reply; 19+ messages in thread From: Eli Zaretskii @ 2020-04-17 15:50 UTC (permalink / raw) To: Stefan Monnier; +Cc: rcopley, 40661 > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: Richard Copley <rcopley@gmail.com>, dancol@dancol.org, > 40661@debbugs.gnu.org > Date: Fri, 17 Apr 2020 11:28:49 -0400 > > > Obviously, we cannot allow GC to run while regex routines do their > > work, because they are passed C pointers to buffer text. The question > > is, where to disable GC? We could do it inside > > update_syntax_table_forward, but UPDATE_SYNTAX_TABLE_FORWARD is called > > from many places that evidently have no problems with GC. So my > > suggestion would be to disable GC inside re_match_2_internal instead. > > > > Comments? > > Looks fine to me. So you prefer disabling GC to setting the inhibit_shrinking flag? I tend to agree with Daniel here, FWIW. > I think a better fix is to move the execution of compact_buffer: > there's no reason it has to be done during GC, we just need to do it > "every once in a while" and the GC was a convenient point for that. > But we could avoid several such problems if we were to run such > background tasks elsewhere. It could still be linked to GC, e.g. we > could start a timer during the GC so its run at the next > opportunity. That's a project for another pandemic ;-) I want a simple enough solution for now that we could install on the release branch. ^ permalink raw reply [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-17 15:50 ` Eli Zaretskii @ 2020-04-17 16:00 ` Stefan Monnier 2020-04-18 11:57 ` Eli Zaretskii 0 siblings, 1 reply; 19+ messages in thread From: Stefan Monnier @ 2020-04-17 16:00 UTC (permalink / raw) To: Eli Zaretskii; +Cc: rcopley, 40661 >> > Obviously, we cannot allow GC to run while regex routines do their >> > work, because they are passed C pointers to buffer text. The question >> > is, where to disable GC? We could do it inside >> > update_syntax_table_forward, but UPDATE_SYNTAX_TABLE_FORWARD is called >> > from many places that evidently have no problems with GC. So my >> > suggestion would be to disable GC inside re_match_2_internal instead. >> > Comments? >> Looks fine to me. > So you prefer disabling GC to setting the inhibit_shrinking flag? > I tend to agree with Daniel here, FWIW. No, what I meant was that I'm fine with doing it "inside re_match_2_internal" instead of "inside update_syntax_table_forward". But I prefer if the "it" is setting `inhibit_shrinking` instead of inhibiting the GC altogether. > That's a project for another pandemic ;-) I want a simple enough > solution for now that we could install on the release branch. Oh, for `emacs-27` that's clearly out, indeed. But for `master` it should not be that hard. I think we can reuse `pending_funcalls` for that. Stefan ^ permalink raw reply [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-17 16:00 ` Stefan Monnier @ 2020-04-18 11:57 ` Eli Zaretskii 2020-04-20 18:30 ` Richard Copley 0 siblings, 1 reply; 19+ messages in thread From: Eli Zaretskii @ 2020-04-18 11:57 UTC (permalink / raw) To: Stefan Monnier; +Cc: rcopley, 40661 > From: Stefan Monnier <monnier@iro.umontreal.ca> > Cc: rcopley@gmail.com, dancol@dancol.org, 40661@debbugs.gnu.org > Date: Fri, 17 Apr 2020 12:00:27 -0400 > > >> > Obviously, we cannot allow GC to run while regex routines do their > >> > work, because they are passed C pointers to buffer text. The question > >> > is, where to disable GC? We could do it inside > >> > update_syntax_table_forward, but UPDATE_SYNTAX_TABLE_FORWARD is called > >> > from many places that evidently have no problems with GC. So my > >> > suggestion would be to disable GC inside re_match_2_internal instead. > >> > Comments? > >> Looks fine to me. > > So you prefer disabling GC to setting the inhibit_shrinking flag? > > I tend to agree with Daniel here, FWIW. > > No, what I meant was that I'm fine with doing it "inside > re_match_2_internal" instead of "inside update_syntax_table_forward". > But I prefer if the "it" is setting `inhibit_shrinking` instead of > inhibiting the GC altogether. OK, thank you and Daniel for the feedback, and thanks to Richard for finding this bug in the first place. I installed a fix on the emacs-27 branch, it avoids the crashes in the recipe posted by Richard. Please eyeball the fix, in case I made some stupid mistake (my environment is rather noisy today, so I trust myself even less than I usually do). ^ permalink raw reply [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-18 11:57 ` Eli Zaretskii @ 2020-04-20 18:30 ` Richard Copley 2020-04-20 19:06 ` Eli Zaretskii 0 siblings, 1 reply; 19+ messages in thread From: Richard Copley @ 2020-04-20 18:30 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Stefan Monnier, 40661 On Sat, 18 Apr 2020 at 12:57, Eli Zaretskii <eliz@gnu.org> wrote: > OK, thank you and Daniel for the feedback, and thanks to Richard for > finding this bug in the first place. > > I installed a fix on the emacs-27 branch, it avoids the crashes in the > recipe posted by Richard. Please eyeball the fix, in case I made some > stupid mistake (my environment is rather noisy today, so I trust > myself even less than I usually do). There are a couple of points about the patch that aren't clear to me. I think you have probably thought them already. I hope you don't mind me asking for clarification. I have a little worry about the situation where the procedure of doing a "decoding" entails a regexp buffer search. (I can't make out whether or not "decoding" means executing arbitrary Lisp code.) Is it possible, with your patch, that we might re-enable buffer shrinking sooner than desirable? (This is reiterating a worry you yourself mentioned earlier, "I'm not sure we want to conflate these two purposes".) In the comment, you mention relocation as well as shrinking. Does it make sense to combine this new guard with the existing freeze_buffer_relocation in some way? ^ permalink raw reply [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-20 18:30 ` Richard Copley @ 2020-04-20 19:06 ` Eli Zaretskii 2020-04-20 23:04 ` Richard Copley 0 siblings, 1 reply; 19+ messages in thread From: Eli Zaretskii @ 2020-04-20 19:06 UTC (permalink / raw) To: Richard Copley; +Cc: monnier, 40661 > From: Richard Copley <rcopley@gmail.com> > Date: Mon, 20 Apr 2020 19:30:29 +0100 > Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Daniel Colascione <dancol@dancol.org>, 40661@debbugs.gnu.org > > I have a little worry about the situation where the procedure of doing > a "decoding" entails a regexp buffer search. (I can't make out whether > or not "decoding" means executing arbitrary Lisp code.) This can happen if the coding-system has a post-read-conversion attribute, yes. > Is it possible, with your patch, that we might re-enable buffer > shrinking sooner than desirable? I don't think I understand the scenario you have in mind. If decoding calls regexp search, then we inhibit shrinking inside the regex routines, not outside of them. Which code will re-enable shrinking before the regex routines return? > (This is reiterating a worry you yourself mentioned earlier, "I'm > not sure we want to conflate these two purposes".) coding.c uses this flag only while it runs purely C code. Lisp cannot run at that point. If a coding-system has a post-read-conversion, it will run only after the C decoder finishes. > In the comment, you mention relocation as well as shrinking. It's the relocation that causes the crash, because following the relocation we unmap a portion of the process memory, where previously we had buffer text, from the process's address space. Shrinking of the gap is what triggers the relocation (when the original gap was very large). > Does it make sense to combine this new guard with the existing > freeze_buffer_relocation in some way? I thought about that, but decided against it, for two reasons. First, freeze_buffer_relocation should at some point go away: it is not a no-op only in an Emacs that uses ralloc.c, which currently only the MS-DOS build does, AFAIK. And second, it freezes relocation at a higher level than needed, above the re_match_2_internal function which is the only one that can call Lisp. ^ permalink raw reply [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-20 19:06 ` Eli Zaretskii @ 2020-04-20 23:04 ` Richard Copley 2020-09-30 3:49 ` Lars Ingebrigtsen 0 siblings, 1 reply; 19+ messages in thread From: Richard Copley @ 2020-04-20 23:04 UTC (permalink / raw) To: Eli Zaretskii; +Cc: Stefan Monnier, 40661 On Mon, 20 Apr 2020 at 20:07, Eli Zaretskii <eliz@gnu.org> wrote: > > > From: Richard Copley <rcopley@gmail.com> > > Date: Mon, 20 Apr 2020 19:30:29 +0100 > > Cc: Stefan Monnier <monnier@iro.umontreal.ca>, Daniel Colascione <dancol@dancol.org>, 40661@debbugs.gnu.org > > > > I have a little worry about the situation where the procedure of doing > > a "decoding" entails a regexp buffer search. (I can't make out whether > > or not "decoding" means executing arbitrary Lisp code.) > > This can happen if the coding-system has a post-read-conversion > attribute, yes. > > > Is it possible, with your patch, that we might re-enable buffer > > shrinking sooner than desirable? > > I don't think I understand the scenario you have in mind. If decoding > calls regexp search, then we inhibit shrinking inside the regex > routines, not outside of them. Which code will re-enable shrinking > before the regex routines return? > > (This is reiterating a worry you yourself mentioned earlier, "I'm > > not sure we want to conflate these two purposes".) > What I had in mind was that we do inhibit shrinking outside the regex search routines, for example (coding.c:8005), current_buffer->text->inhibit_shrinking = 1; decode_coding (coding); current_buffer->text->inhibit_shrinking = 0; I was worried about the case where a regex buffer search happens 'half way through' decode_coding. Then something bad might happen during the second half, when inhibit_shrinking is zero. But ... > coding.c uses this flag only while it runs purely C code. Lisp cannot > run at that point. If a coding-system has a post-read-conversion, it > will run only after the C decoder finishes. ... so that's OK then. > > In the comment, you mention relocation as well as shrinking. > > It's the relocation that causes the crash, because following the > relocation we unmap a portion of the process memory, where previously > we had buffer text, from the process's address space. Shrinking of > the gap is what triggers the relocation (when the original gap was > very large). > > > Does it make sense to combine this new guard with the existing > > freeze_buffer_relocation in some way? > > I thought about that, but decided against it, for two reasons. First, > freeze_buffer_relocation should at some point go away: it is not a > no-op only in an Emacs that uses ralloc.c, which currently only the > MS-DOS build does, AFAIK. OK. > And second, it freezes relocation at a > higher level than needed, above the re_match_2_internal function which > is the only one that can call Lisp. Right, but freeze_buffer_relocation was surely done like that on purpose, to hoist it outside the loop in search_buffer_re. Anyway, I'm sure it's no big deal. ^ permalink raw reply [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-20 23:04 ` Richard Copley @ 2020-09-30 3:49 ` Lars Ingebrigtsen 0 siblings, 0 replies; 19+ messages in thread From: Lars Ingebrigtsen @ 2020-09-30 3:49 UTC (permalink / raw) To: Richard Copley; +Cc: Stefan Monnier, 40661 Richard Copley <rcopley@gmail.com> writes: >> And second, it freezes relocation at a >> higher level than needed, above the re_match_2_internal function which >> is the only one that can call Lisp. > > Right, but freeze_buffer_relocation was surely done like that on > purpose, to hoist it outside the loop in search_buffer_re. Anyway, I'm > sure it's no big deal. If I'm skimming this thread correctly, the reported bug was fixed by Eli at the time, and there was then some followup discussion. But there doesn't seem to be anything more to be done here, so I'm closing this bug report. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 19+ messages in thread
* bug#40661: Crash in regex search during redisplay 2020-04-16 16:42 ` Daniel Colascione 2020-04-16 16:56 ` Richard Copley @ 2020-04-16 16:57 ` Eli Zaretskii 1 sibling, 0 replies; 19+ messages in thread From: Eli Zaretskii @ 2020-04-16 16:57 UTC (permalink / raw) To: Daniel Colascione; +Cc: rcopley, 40661 > Date: Thu, 16 Apr 2020 09:42:10 -0700 > CC: 40661@debbugs.gnu.org > From: Daniel Colascione <dancol@dancol.org> > > On April 16, 2020 9:33:16 AM PDT, Eli Zaretskii <eliz@gnu.org> wrote: > Try using rr and see where those pointers came from rr doesn't work on MS-Windows. ^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2020-09-30 3:49 UTC | newest] Thread overview: 19+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2020-04-16 14:35 bug#40661: Crash in regex search during redisplay Richard Copley 2020-04-16 15:36 ` Eli Zaretskii 2020-04-16 16:33 ` Eli Zaretskii 2020-04-16 16:42 ` Daniel Colascione 2020-04-16 16:56 ` Richard Copley 2020-04-16 17:24 ` Daniel Colascione 2020-04-16 19:35 ` Richard Copley 2020-04-17 11:22 ` Eli Zaretskii 2020-04-17 13:55 ` Eli Zaretskii 2020-04-17 14:01 ` Daniel Colascione 2020-04-17 15:28 ` Stefan Monnier 2020-04-17 15:50 ` Eli Zaretskii 2020-04-17 16:00 ` Stefan Monnier 2020-04-18 11:57 ` Eli Zaretskii 2020-04-20 18:30 ` Richard Copley 2020-04-20 19:06 ` Eli Zaretskii 2020-04-20 23:04 ` Richard Copley 2020-09-30 3:49 ` Lars Ingebrigtsen 2020-04-16 16:57 ` Eli Zaretskii
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/emacs.git https://git.savannah.gnu.org/cgit/emacs/org-mode.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.