From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: npostavs@users.sourceforge.net Newsgroups: gmane.emacs.bugs Subject: bug#24358: 25.1.50; re-search-forward errors with "Variable binding depth exceeds max-specpdl-size" Date: Sat, 08 Oct 2016 09:45:20 -0400 Message-ID: <87eg3rvtsf.fsf@users.sourceforge.net> References: <87twe6sx2g.fsf@users.sourceforge.net> <87eg51ng4r.fsf_-_@users.sourceforge.net> <87k2djwumn.fsf@users.sourceforge.net> <83h98nidvd.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Trace: blaine.gmane.org 1475934327 15857 195.159.176.226 (8 Oct 2016 13:45:27 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 8 Oct 2016 13:45:27 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux) Cc: 24358@debbugs.gnu.org, peder@klingenberg.no To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sat Oct 08 15:45:21 2016 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bsrwC-00035f-Gj for geb-bug-gnu-emacs@m.gmane.org; Sat, 08 Oct 2016 15:45:16 +0200 Original-Received: from localhost ([::1]:41191 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bsrwB-0003NQ-6A for geb-bug-gnu-emacs@m.gmane.org; Sat, 08 Oct 2016 09:45:15 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:39579) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bsrw3-0003K2-ON for bug-gnu-emacs@gnu.org; Sat, 08 Oct 2016 09:45:10 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bsrvy-0000sy-NF for bug-gnu-emacs@gnu.org; Sat, 08 Oct 2016 09:45:06 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:42200) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bsrvy-0000su-Jq for bug-gnu-emacs@gnu.org; Sat, 08 Oct 2016 09:45:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1bsrvy-00018H-Fd for bug-gnu-emacs@gnu.org; Sat, 08 Oct 2016 09:45:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: npostavs@users.sourceforge.net Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 08 Oct 2016 13:45:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 24358 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 24358-submit@debbugs.gnu.org id=B24358.14759342944325 (code B ref 24358); Sat, 08 Oct 2016 13:45:02 +0000 Original-Received: (at 24358) by debbugs.gnu.org; 8 Oct 2016 13:44:54 +0000 Original-Received: from localhost ([127.0.0.1]:48390 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bsrvq-00017h-5M for submit@debbugs.gnu.org; Sat, 08 Oct 2016 09:44:54 -0400 Original-Received: from mail-it0-f50.google.com ([209.85.214.50]:38526) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1bsrvo-00017P-7N for 24358@debbugs.gnu.org; Sat, 08 Oct 2016 09:44:52 -0400 Original-Received: by mail-it0-f50.google.com with SMTP id o19so41440375ito.1 for <24358@debbugs.gnu.org>; Sat, 08 Oct 2016 06:44:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=lSC9wzjrUNb997Nim8E/oPU/mTuUlCcWHS19rtBynDI=; b=1HTgZwX/A8Qmor+jnodCUM+3edZYalWpYtTlQouWH1yD2UlNOKE4HQQnKV6oNbwsog 8ZWi79u+f/lUDVe9uNljiCOQAC5GEmUDwoehtRxxeU/SRwpvcqgI4umsBJs5vU1iqEmZ kwWAwPBxGOFBvLz+ziFAxTQN+OyB2AZ4LrTusZvUdNB7361TNkm2hE/JDqG1LbSID4lU 3aTBFxHRlf5SFpw+OPGoJKC5WZf29i8DuTNdtE5oTSLyy6muqgl8ORZJQJHC07X/rxzk +nq+Jhj3VTpLz+/dxaSIN0KjjVZAXNP24rg9aKz5LIfs4un+gSGzqxRmLsG0vdFZ2qev ys5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:cc:subject:references:date :in-reply-to:message-id:user-agent:mime-version; bh=lSC9wzjrUNb997Nim8E/oPU/mTuUlCcWHS19rtBynDI=; b=ZVeslYLESF7128L5yHgy8dK5cQDwmPH5VTDIkECxfJO+OD2a7YLsY4XWVBkEdx7YMo I6Jng8RaYHHCoo1eJaYro6z1LIQT1JqWdmIn9Icq9uFIMX32d1FhJtcOyc9UOZyYvRCG WOyoRy8rTAWFn6RkWcuxxoLiYCu3ESCGIWBp1eVuGnYKPtlBMUh2uuoyt/ZSQ/1j5+4Y /qxjXQrx2ojNGg5ZvilaeteqGyzwJn88qoz7EEqB5kQHmU7oMpnImTefXK+eytHUxJX1 iSLqRJfx1Fmdq13kd68qkoOZVjZyxVHViOSrqOxO7Fe8Cmc5r4HEH1GQ+gWTnUmpWL8y 2yvw== X-Gm-Message-State: AA6/9RlW0ecfJwiuLyhpyCGz10M+eIFVil98pYOtz8Zj2/0hWIjgsIitkn75xnG3/sekHQ== X-Received: by 10.36.123.135 with SMTP id q129mr2904659itc.117.1475934286529; Sat, 08 Oct 2016 06:44:46 -0700 (PDT) Original-Received: from zony ([45.2.7.130]) by smtp.googlemail.com with ESMTPSA id b133sm3055914iti.21.2016.10.08.06.44.45 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 08 Oct 2016 06:44:45 -0700 (PDT) In-Reply-To: <83h98nidvd.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 08 Oct 2016 08:55:18 +0300") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:124206 Archived-At: --=-=-= Content-Type: text/plain Eli Zaretskii writes: >> From: npostavs@users.sourceforge.net >> Date: Fri, 07 Oct 2016 20:29:36 -0400 >> Cc: 24358@debbugs.gnu.org >> >> npostavs@users.sourceforge.net writes: >> > >> >> (I'm also on GNU/Linux, Arch) I get the same max-specpdl-size error with >> >> 25.1.50, with 24.5 (and below) I get (error "Stack overflow in regexp >> >> matcher") >> >> icalendar--read-element has been fixed, but this still reproduces when >> doing (re-search-forward ".*\\(\n.*\\)*" nil t) on the text file given >> in the OP. > > Isn't that "user error"? Yes, but it should give "Stack overflow in regexp matcher", not overflow the lisp stack (or assertion failure). > >> And I'm still seeing an assertion failure due to what looks like >> memory corruption on the emacs-25 branch. > > Details of the assertion? (See also https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24358#8) I tracked the corruption to a malloc call, but I wasn't able to figure out what's happening there. I used the following to debug: Apply the attached bug-24358-hunting.diff and then run gdb --args ./emacs -Q -batch -l ~/src/emacs/bug-24358-regex-max-specpdl.el Where ~/src/emacs/bug-24358-regex-max-specpdl.el is: (with-temp-buffer (insert-file-contents "~/src/emacs/bug-24358-regex-max-specpdl.txt") ; adjust path (goto-char (point-min)) (re-search-forward ".*\\(\n.*\\)*" nil t)) I show some more excerpts in the attached bug-24358-debug.log, but my main finding is that string1 of re_match_2_internal is originally: string1=0x1835980 "DESCRIPTION;LANGUAGE= but then it becomes corrupted during a malloc: Old value = 68 'D' New value = 0 '\000' 0x00007ffff0cc01a7 in __memset_sse2_unaligned_erms () from /usr/lib/libc.so.6 (gdb) bt 13 #0 0x00007ffff0cc01a7 in __memset_sse2_unaligned_erms () from /usr/lib/libc.so.6 #1 0x00000000006d27f5 in r_alloc_sbrk (size=290816) at ralloc.c:848 #2 0x00000000006ced96 in get_contiguous_space (size=290816, position=0x1833000) at gmalloc.c:476 #3 0x00000000006cf92a in _malloc_internal_nolock (size=163840) at gmalloc.c:844 #4 0x00000000006cfe9d in _malloc_internal (size=163840) at gmalloc.c:927 #5 0x00000000006cff1a in gmalloc (size=163840) at gmalloc.c:951 #6 0x00000000006d14e4 in malloc (size=163840) at gmalloc.c:1827 #7 0x00000000005f3e6b in lmalloc (size=163840) at alloc.c:1414 #8 0x00000000005f3356 in xmalloc (size=163840) at alloc.c:821 #9 0x00000000005f38e4 in record_xmalloc (size=163840) at alloc.c:1038 #10 0x00000000005ee233 in re_match_2_internal (bufp=0xd6d650 , string1=0x1835980 "", size1=0, string2=0x1835980 "", size2=40918, pos=0, regs=0xd6deb0 , stop=40918) at regex.c:5844 --=-=-= Content-Type: text/plain Content-Disposition: attachment; filename=bug-24358-hunting.diff Content-Description: changes to hunt down bug 24358 diff --git i/src/.gdbinit w/src/.gdbinit index a4e9f70..d17d1ba 100644 --- i/src/.gdbinit +++ w/src/.gdbinit @@ -1280,3 +1280,22 @@ commands end continue end + +# bug 24315 +break re_match_2_internal if (size2>2000 && size2==stop) +commands + p debug = 1 + continue +end +break debug_spot +commands + watch -l string1[0] + disable 4 + # cond 4 (string1[0] != 'D') + # continue +end +# break debug_malloc if ((mem <= 0x1834980) && (0x1834980 < mem + size)) + + + + diff --git i/src/gmalloc.c w/src/gmalloc.c index 00b8364..5084609 100644 --- i/src/gmalloc.c +++ w/src/gmalloc.c @@ -914,6 +914,10 @@ _malloc_internal_nolock (size_t size) return result; } +void debug_malloc (void* mem, size_t size) +{ +} + void * _malloc_internal (size_t size) { @@ -923,6 +927,7 @@ _malloc_internal (size_t size) result = _malloc_internal_nolock (size); UNLOCK (); + debug_malloc (result, size); return result; } diff --git i/src/regex.c w/src/regex.c index 164eb46..861b800 100644 --- i/src/regex.c +++ w/src/regex.c @@ -828,6 +828,7 @@ extract_number_and_incr (re_char **source) interactively. And if linked with the main program in `main.c' and the other test files, you can run the already-written tests. */ +#define DEBUG #ifdef DEBUG /* We use standard I/O for debugging. */ @@ -838,6 +839,13 @@ extract_number_and_incr (re_char **source) static int debug = -100000; +static void debug_spot (int fail_stack_avail, const char*string1, const char*string2) +{ + extern void r_alloc_check (void); + //r_alloc_check (); + fail_stack_avail++; +} + # define DEBUG_STATEMENT(e) e # define DEBUG_PRINT(...) if (debug > 0) printf (__VA_ARGS__) # define DEBUG_COMPILES_ARGUMENTS @@ -1172,16 +1180,31 @@ print_double_string (re_char *where, re_char *string1, ssize_t size1, printf ("(null)"); else { + int i; if (FIRST_STRING_P (where)) { - for (this_char = where - string1; this_char < size1; this_char++) - putchar (string1[this_char]); + for (i = 0, this_char = where - string1; this_char < size1; i++, this_char++) + { + if (i > 20) + { + putchar ('.'); putchar ('.'); putchar ('.'); + break; + } + putchar (string1[this_char]); + } where = string2; } - for (this_char = where - string2; this_char < size2; this_char++) - putchar (string2[this_char]); + for (i = 0, this_char = where - string2; this_char < size2; i++, this_char++) + { + if (i > 20) + { + putchar ('.'); putchar ('.'); putchar ('.'); + break; + } + putchar (string2[this_char]); + } } } @@ -1533,6 +1556,7 @@ while (REMAINING_AVAIL_SLOTS <= space) { \ of 0 + -1 isn't done as unsigned. */ \ \ DEBUG_STATEMENT (nfailure_points_pushed++); \ + if (debug > 0) debug_spot((fail_stack).avail, string1,string2); \ DEBUG_PRINT ("\nPUSH_FAILURE_POINT:\n"); \ DEBUG_PRINT (" Before push, next avail: %zd\n", (fail_stack).avail); \ DEBUG_PRINT (" size: %zd\n", (fail_stack).size);\ --=-=-= Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename=bug-24358-debug.log Content-Transfer-Encoding: quoted-printable Content-Description: gdb session excerpts The compiled pattern is: The string to match is: "DESCRIPTION;LANGUAGE=3D..= ." 0x144aa80: EXECUTING on_failure_jump_smart 4 (to 0x144aa87). smart default =3D> slow loop. 0x144aa80: EXECUTING on_failure_jump 4 (to 0x144aa87): Thread 1 "emacs" hit Breakpoint 4, debug_spot (fail_stack_avail=3D0,=20 string1=3D0x1835980 "DESCRIPTION;LANGUAGE=3Den-US:Nn Nnnnn\\,\\n\\nNnnn= nnnnn nnn nnn nnnnnn nn nnnnnn\n nnnnnnn nnnn nnnnnnnnn. N nnnn nnnnnnnnn n= n nnn nnnnnnnn nnnnnnn nnn nn nn\n nn nn-nnnnnnn nn Nnnnnnn nn 99.99 NNNN\\= n\\nNnnn "...,=20 string2=3D0x1835980 "DESCRIPTION;LANGUAGE=3Den-US:Nn Nnnnn\\,\\n\\nNnnn= nnnnn nnn nnn nnnnnn nn nnnnnn\n nnnnnnn nnnn nnnnnnnnn. N nnnn nnnnnnnnn n= n nnn nnnnnnnn nnnnnnn nnn nn nn\n nn nn-nnnnnnn nn Nnnnnnn nn 99.99 NNNN\\= n\\nNnnn "...) at regex.c:846 846 fail_stack_avail++; Hardware watchpoint 5: -location string1[0] (gdb) bt 5 #0 debug_spot (fail_stack_avail=3D0,=20 string1=3D0x1835980 "DESCRIPTION;LANGUAGE=3Den-US:Nn Nnnnn\\,\\n\\nNnnn= nnnnn nnn nnn nnnnnn nn nnnnnn\n nnnnnnn nnnn nnnnnnnnn. N nnnn nnnnnnnnn n= n nnn nnnnnnnn nnnnnnn nnn nn nn\n nn nn-nnnnnnn nn Nnnnnnn nn 99.99 NNNN\\= n\\nNnnn "...,=20 string2=3D0x1835980 "DESCRIPTION;LANGUAGE=3Den-US:Nn Nnnnn\\,\\n\\nNnnn= nnnnn nnn nnn nnnnnn nn nnnnnn\n nnnnnnn nnnn nnnnnnnnn. N nnnn nnnnnnnnn n= n nnn nnnnnnnn nnnnnnn nnn nn nn\n nn nn-nnnnnnn nn Nnnnnnn nn 99.99 NNNN\\= n\\nNnnn "...) at regex.c:846 #1 0x00000000005ee090 in re_match_2_internal (bufp=3D0xd6d650 ,=20 string1=3D0x1835980 "DESCRIPTION;LANGUAGE=3Den-US:Nn Nnnnn\\,\\n\\nNnnn= nnnnn nnn nnn nnnnnn nn nnnnnn\n nnnnnnn nnnn nnnnnnnnn. N nnnn nnnnnnnnn n= n nnn nnnnnnnn nnnnnnn nnn nn nn\n nn nn-nnnnnnn nn Nnnnnnn nn 99.99 NNNN\\= n\\nNnnn "..., size1=3D0,=20 string2=3D0x1835980 "DESCRIPTION;LANGUAGE=3Den-US:Nn Nnnnn\\,\\n\\nNnnn= nnnnn nnn nnn nnnnnn nn nnnnnn\n nnnnnnn nnnn nnnnnnnnn. N nnnn nnnnnnnnn n= n nnn nnnnnnnn nnnnnnn nnn nn nn\n nn nn-nnnnnnn nn Nnnnnnn nn 99.99 NNNN\\= n\\nNnnn "..., size2=3D40918, pos=3D0, regs=3D0xd6deb0 , stop= =3D40918) at regex.c:5844 #2 0x00000000005e9022 in re_search_2 (bufp=3D0xd6d650 ,=20 str1=3D0x1835980 "DESCRIPTION;LANGUAGE=3Den-US:Nn Nnnnn\\,\\n\\nNnnnnnn= nn nnn nnn nnnnnn nn nnnnnn\n nnnnnnn nnnn nnnnnnnnn. N nnnn nnnnnnnnn nn n= nn nnnnnnnn nnnnnnn nnn nn nn\n nn nn-nnnnnnn nn Nnnnnnn nn 99.99 NNNN\\n\\= nNnnn "..., size1=3D0,=20 str2=3D0x1835980 "DESCRIPTION;LANGUAGE=3Den-US:Nn Nnnnn\\,\\n\\nNnnnnnn= nn nnn nnn nnnnnn nn nnnnnn\n nnnnnnn nnnn nnnnnnnnn. N nnnn nnnnnnnnn nn n= nn nnnnnnnn nnnnnnn nnn nn nn\n nn nn-nnnnnnn nn Nnnnnnn nn 99.99 NNNN\\n\\= nNnnn "..., size2=3D40918, startpos=3D0, range=3D40918, regs=3D0xd6deb0 ,=20 stop=3D40918) at regex.c:4470 #3 0x00000000005d6c06 in search_buffer (string=3D25301860, pos=3D1, pos_by= te=3D1, lim=3D40891,=20 lim_byte=3D40919, n=3D1, RE=3D1, trt=3D20893029, inverse_trt=3D20483397= , posix=3Dfalse) at search.c:1265 #4 0x00000000005d63a1 in search_command (string=3D25301860, bound=3D0, noe= rror=3D44544, count=3D0,=20 direction=3D1, RE=3D1, posix=3Dfalse) at search.c:1058 (More stack frames follow...) (gdb) cont [...] PUSH_FAILURE_POINT: Before push, next avail: 5115 size: 5120 Push frame index: 5115 Push string 0x1836013: ".nnn>\;\n> +NNNN , string1=3D0x1835980 "",=20 size1=3D0, string2=3D0x1835980 "", size2=3D40918, pos=3D0, regs=3D0xd6d= eb0 , stop=3D40918) at regex.c:5844 #11 0x00000000005e9022 in re_search_2 (bufp=3D0xd6d650 , s= tr1=3D0x1835980 "", size1=3D0,=20 str2=3D0x1835980 "", size2=3D40918, startpos=3D0, range=3D40918, regs= =3D0xd6deb0 , stop=3D40918) at regex.c:4470 #12 0x00000000005d6c06 in search_buffer (string=3D25301860, pos=3D1, pos_by= te=3D1, lim=3D40891,=20 lim_byte=3D40919, n=3D1, RE=3D1, trt=3D20893029, inverse_trt=3D20483397= , posix=3Dfalse) at search.c:1265 (More stack frames follow...) Continuing. Thread 1 "emacs" hit Hardware watchpoint 5: -location string1[0] Old value =3D 0 '\000' New value =3D -34 '\336' 0x00007ffff0d67b64 in __memcpy_ssse3 () from /usr/lib/libc.so.6 (gdb) cont Continuing. Doubled stack; size now: 20480 slots available: 15362 Push frame index: 5118 Push string 0x1836014: "=C2=AA$..." [...] PUSH_FAILURE_POINT: Before push, next avail: 5130 size: 20480 Push frame index: 5130 Push string 0x1836018: "=C2=AA$\..." 0: /on_failure_jump to 7 3: /anychar 4: /jump to 0 7: /stop_memory/1 9: /jump to -8 12: /succeed 13: end of pattern. Push pattern 0x144aa8f:=20 0x144aa92: EXECUTING anychar. character.h:696: Emacs fatal error: assertion failed: CHAR_VALID_P (ch) Thread 1 "emacs" hit Breakpoint 1, terminate_due_to_signal (sig=3D6, backtr= ace_limit=3D2147483647) at emacs.c:354 354 signal (sig, SIG_DFL); (gdb) bt 7 #0 terminate_due_to_signal (sig=3D6, backtrace_limit=3D2147483647) at emac= s.c:354 #1 0x00000000005fdb9b in die (msg=3D0x725888 "CHAR_VALID_P (ch)", file=3D0= x72587c "character.h",=20 line=3D696) at alloc.c:7224 #2 0x000000000056c000 in char_table_translate (obj=3D20893029, ch=3D419517= 8) at character.h:696 #3 0x00000000005eb8db in re_match_2_internal (bufp=3D0xd6d650 ,=20 string1=3D0x1835980 "\336[\203\001", size1=3D0, string2=3D0x1835980 "\3= 36[\203\001", size2=3D40918,=20 pos=3D0, regs=3D0xd6deb0 , stop=3D40918) at regex.c:5454 #4 0x00000000005e9022 in re_search_2 (bufp=3D0xd6d650 ,=20 str1=3D0x1835980 "\336[\203\001", size1=3D0, str2=3D0x1835980 "\336[\20= 3\001", size2=3D40918, startpos=3D0,=20 range=3D40918, regs=3D0xd6deb0 , stop=3D40918) at regex.c:= 4470 #5 0x00000000005d6c06 in search_buffer (string=3D25301860, pos=3D1, pos_by= te=3D1, lim=3D40891,=20 lim_byte=3D40919, n=3D1, RE=3D1, trt=3D20893029, inverse_trt=3D20483397= , posix=3Dfalse) at search.c:1265 #6 0x00000000005d63a1 in search_command (string=3D25301860, bound=3D0, noe= rror=3D44544, count=3D0,=20 direction=3D1, RE=3D1, posix=3Dfalse) at search.c:1058 (More stack frames follow...) --=-=-=--