bug#28329: 25.2; Eshell: 40M+ output: Stack overflow in regexp matcher

bug#28329: 25.2; Eshell: 40M+ output: Stack overflow in regexp matcher

[Pierre Neidhardt]

Today I update my Gentoo:

	> emerge --update --newuse --deep @world

in Eshell.  After a very long output (40M+), the shell started stuttering and hanging, messaging the following errors on every new line of output:

	error in process filter: eshell-watch-for-password-prompt: Stack overflow in regexp matcher
	error in process filter: Stack overflow in regexp matcher

The process became so slow it was unlikely it would ever complete, so I terminated it and had to re-run it in a *term*.



In GNU Emacs 25.2.1 (x86_64-pc-linux-gnu, GTK+ Version 3.22.15)
 of 2017-08-28 built on dhiov23k
Windowing system distributor 'The X.Org Foundation', version 11.0.11903000
System Description:	Gentoo Base System release 2.3

Configured using:
 'configure --prefix=/usr --build=x86_64-pc-linux-gnu
 --host=x86_64-pc-linux-gnu --mandir=/usr/share/man
 --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc
 --localstatedir=/var/lib --disable-dependency-tracking
 --disable-silent-rules --docdir=/usr/share/doc/emacs-25.2
 --htmldir=/usr/share/doc/emacs-25.2/html --libdir=/usr/lib64
 --program-suffix=-emacs-25 --infodir=/usr/share/info/emacs-25
 --localstatedir=/var
 --enable-locallisppath=/etc/emacs:/usr/share/emacs/site-lisp
 --with-gameuser=:gamestat --without-compress-install
 --with-file-notification=inotify --enable-acl --without-dbus
 --without-modules --without-gpm --without-hesiod --without-kerberos
 --without-kerberos5 --with-xml2 --without-selinux --with-gnutls
 --without-wide-int --with-zlib --with-sound=alsa --with-x --without-ns
 --without-gconf --without-gsettings --without-toolkit-scroll-bars
 --with-gif --with-jpeg --with-png --with-rsvg --with-tiff --with-xpm
 --with-imagemagick --with-xft --without-cairo --without-libotf
 --without-m17n-flt --with-x-toolkit=gtk3 --without-xwidgets
 GENTOO_PACKAGE=app-editors/emacs-25.2 'CFLAGS=-march=ivybridge -O2
 -pipe' CPPFLAGS= 'LDFLAGS=-Wl,-O1 -Wl,--as-needed''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND NOTIFY ACL GNUTLS LIBXML2
FREETYPE XFT ZLIB GTK3 X11

Important settings:
  value of $LANG: en_US.utf8
  locale-coding-system: utf-8-unix

bug#28329: 25.2; Eshell: 40M+ output: Stack overflow in regexp matcher

[npostavs]

Pierre Neidhardt <ambrevar@gmail.com> writes:

> Today I update my Gentoo:
>
> 	> emerge --update --newuse --deep @world
>
> in Eshell.  After a very long output (40M+), the shell started stuttering and hanging, messaging the following errors on every new line of output:
>
> 	error in process filter: eshell-watch-for-password-prompt: Stack overflow in regexp matcher
> 	error in process filter: Stack overflow in regexp matcher
>
> The process became so slow it was unlikely it would ever complete, so I terminated it and had to re-run it in a *term*.

Can you get a backtrace if you M-x toggle-debug-on-error?  It would be
helpful to get a sample of output which triggers this.

Does changing the password regexp help?

    (setq eshell-password-prompt-regexp
          (format "^.\\{,80\\}%s.\\{,80\\}:\\s-*\\'"
                  (regexp-opt password-word-equivalents)))

bug#28329: 25.2; Eshell: 40M+ output: Stack overflow in regexp matcher

[Pierre Neidhardt]

Sadly, I had to kill Emacs before I could do it (it was almost
completely hanging).

Thinking of a way of reproduicing this... Maybe create a 40M+ file of
random clang/g++ output and dump it to Eshell's stdout.

-- 
Pierre Neidhardt

bug#28329: 25.2; Eshell: 40M+ output: Stack overflow in regexp matcher

[npostavs]

[-- Attachment #1: Type: text/plain, Size: 26 bytes --]

[forwarding to bug list]


[-- Attachment #2: Type: message/rfc822, Size: 7992 bytes --]

From: Pierre Neidhardt <pe.neidhardt@googlemail.com>
To: npostavs@users.sourceforge.net
Subject: Re: bug#28329: 25.2; Eshell: 40M+ output: Stack overflow in regexp matcher
Date: Sat, 02 Sep 2017 19:02:21 +0100
Message-ID: <87ziadvv0y.fsf@gmail.com>

Alright, I managed to get a stack trace:

Debugger entered--Lisp error: (error "Stack overflow in regexp matcher")
  re-search-forward("\\(\\(?:adgangskode\\|contrase\\(?:\\(?:ny\\|ñ\\)a\\)\\|geslo\\|h\\(?:\\(?:asł\\|esl\\)o\\)\\|iphasiwedi\\|jelszó\\|l\\(?:ozinka\\|ösenord\\)\\|m\\(?:ot de passe\\|ật khẩu\\)\\|pa\\(?:rola\\|s\\(?:ahitza\\|s\\(?: phrase\\|code\\|ord\\|phrase\\|wor[dt]\\)\\|vorto\\)\\)\\|s\\(?:alasana\\|enha\\|laptažodis\\)\\|wachtwoord\\|лозинка\\|пароль\\|ססמה\\|كلمة السر\\|गुप्तशब्द\\|शब्दकूट\\|গুপ্তশব্দ\\|পাসওয়ার্ড\\|ਪਾਸਵਰਡ\\|પાસવર્ડ\\|ପ୍ରବେଶ ସଙ୍କେତ\\|கடவுச்சொல்\\|సంకేతపదము\\|ಗುಪ್ತಪದ\\|അടയാളവാക്ക്\\|රහස්පදය\\|ពាក្យសម្ងាត់\\|パスワード\\|密[码碼]\\|암호\\)\\).*:\\s *\\'" #<marker at 20572261 in *eshell*> t)
  eshell-watch-for-password-prompt()
  run-hooks(eshell-output-filter-functions)
  eshell-run-output-filters()
  eshell-output-filter(#<process emerge> "ium/third_party/WebKit/Source/platform/network/blink_platform.WebSocketHandshakeRequest.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/network/blink_platform.WebSocketHandshakeResponse.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/plugins/blink_platform.PluginData.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/plugins/blink_platform.PluginListBuilder.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/scheduler/blink_platform.CancellableTaskFactory.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/scroll/blink_platform.ProgrammaticScrollAnimator.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/scroll/blink_platform.ScrollAnimatorBase.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/scroll/blink_platform.ScrollAnimatorCompositorCoordinator.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/scroll/blink_platform.ScrollAnimator.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/scroll/blink_platform.ScrollableArea.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/scroll/blink_platform.Scrollbar.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/scroll/blink_platform.ScrollbarTheme.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/scroll/blink_platform.ScrollbarThemeAura.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/scroll/blink_platform.ScrollbarThemeMock.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/scroll/blink_platform.ScrollbarThemeNonMacCommon.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/scroll/blink_platform.ScrollbarThemeOverlay.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/speech/blink_platform.PlatformSpeechSynthesisUtterance.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/speech/blink_platform.PlatformSpeechSynthesisVoice.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/speech/blink_platform.PlatformSpeechSynthesizer.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.BidiCharacterRun.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.BidiContext.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.BidiTextRun.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.DateTimeFormat.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.LineEnding.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.LocaleICU.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.LocaleToScriptMapping.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.PlatformLocale.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.QuotedPrintable.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.SegmentedString.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.StringTruncator.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.TextBoundaries.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.TextBreakIterator.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.TextBreakIteratorICU.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.TextBreakIteratorInternalICU.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.TextCheckerClient.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.TextEncodingDetector.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.TextRun.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.TextStream.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.UnicodeRange.o obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/text/blink_platform.UnicodeUtilities.o ob")

Apparently, the line is just plain too long...
I guess limiting the regexp matcher over, say, the first 256 characters
would fix the issue.

bug#28329: 25.2; Eshell: 40M+ output: Stack overflow in regexp matcher

[Noam Postavsky]

> From: Pierre Neidhardt <pe.neidhardt@googlemail.com>
> Subject: Re: bug#28329: 25.2; Eshell: 40M+ output: Stack overflow in regexp matcher
> To: npostavs@users.sourceforge.net
> Date: Sat, 02 Sep 2017 19:02:21 +0100 (3 weeks, 18 hours, 25 minutes ago)
>
> Alright, I managed to get a stack trace:
>
> Debugger entered--Lisp error: (error "Stack overflow in regexp matcher")
>   re-search-forward("\\(\\(?:adgangskode\\|contrase\\(?:\\(?:ny\\|ñ\\)a\\)\\|geslo\\|h\\(?:\\(?:asł\\|esl\\)o\\)\\|iphasiwedi\\|jelszó\\|l\\(?:ozinka\\|ösenord\\)\\|m\\(?:ot
> de passe\\|ật khẩu\\)\\|pa\\(?:rola\\|s\\(?:ahitza\\|s\\(?:
> phrase\\|code\\|ord\\|phrase\\|wor[dt]\\)\\|vorto\\)\\)\\|s\\(?:alasana\\|enha\\|laptažodis\\)\\|wachtwoord\\|лозинка\\|пароль\\|ססמה\\|كلمة
> السر\\|गुप्तशब्द\\|शब्दकूट\\|গুপ্তশব্দ\\|পাসওয়ার্ড\\|ਪਾਸਵਰਡ\\|પાસવર્ડ\\|ପ୍ରବେଶ
> ସଙ୍କେତ\\|கடவுச்சொல்\\|సంకేతపదము\\|ಗುಪ್ತಪದ\\|അടയാളവാക്ക്\\|රහස්පදය\\|ពាក្យសម្ងាត់\\|
> パスワード\\|密[码碼]\\|암호\\)\\).*:\\s *\\'" #<marker at 20572261 in
> *eshell*> t)
>   eshell-watch-for-password-prompt()
>   run-hooks(eshell-output-filter-functions)
>   eshell-run-output-filters()
>   eshell-output-filter(#<process emerge>
> "ium/third_party/WebKit/Source/platform/network/blink_platform.WebSocketHandshakeRequest.o
> obj/src/3rdparty/chromium/third_party/WebKit/Source/platform/network/blink_platform.WebSocketHandshakeResponse.o
[...]
> ob")
>
> Apparently, the line is just plain too long...
> I guess limiting the regexp matcher over, say, the first 256 characters
> would fix the issue.
> ----------

Can you catch that build output in a file, so it could be reproduced by
running 'cat build.log' in eshell?  I tried copying the fragment you
have in your backtrace a couple of times to make it longer, but I was
unable to trigger the error.  Perhaps the particular contents of the
string matters too (some values might let the matcher short-circuit the
backtracking).

bug#28329: 25.2; Eshell: 40M+ output: Stack overflow in regexp matcher

[Noam Postavsky]

tags 28329 + moreinfo unreproducible
quit

Noam Postavsky <npostavs@users.sourceforge.net> writes:

>> Apparently, the line is just plain too long...
>> I guess limiting the regexp matcher over, say, the first 256 characters
>> would fix the issue.
>> ----------
>
> Can you catch that build output in a file, so it could be reproduced by
> running 'cat build.log' in eshell?  I tried copying the fragment you
> have in your backtrace a couple of times to make it longer, but I was
> unable to trigger the error.  Perhaps the particular contents of the
> string matters too (some values might let the matcher short-circuit the
> backtracking).

ping?

bug#28329: 25.2; Eshell: 40M+ output: Stack overflow in regexp matcher

[Pierre Neidhardt]

Alright, I run into this again while emerging qtwebengine on Gentoo.
Now I think about it, this is the only and systematic cause of the issue
for me.
I suspect that emerging / compiling Chrome or affiliated projects would
also trigger the issue.

This time I managed to save the output, 60M+ big.

I tried to following

- emacs -Q
- M-x eshell
- cat portage.output.log

It took about 2 hours to complete!  While it takes half a second to
paste to the buffer.
Damn! Eshell's filters are slow! :/

That being said, no issue doing that.  So the output "saved after the
fact" does not seem to produce the issue.

-- 
Pierre Neidhardt

Finagle's Seventh Law:
	The perversity of the universe tends toward a maximum.

bug#28329: 25.2; Eshell: 40M+ output: Stack overflow in regexp matcher

[Noam Postavsky]

Pierre Neidhardt <ambrevar@gmail.com> writes:

> That being said, no issue doing that.  So the output "saved after the
> fact" does not seem to produce the issue.

Damn.  I guess the chunking is significant too.  Could you try saving
the output chunks, with this:

    (defvar eshell-chunk-number 0)
    (defconst eshell-output-chunk-dir "eshell-output")
    (make-directory eshell-output-chunk-dir t)

    (defun catch-eshell-output-chunk ()
      (write-region eshell-last-output-block-begin
                    eshell-last-output-end
                    (format "%s/chunk.%d"
                            eshell-output-chunk-dir
                            eshell-chunk-number)
                    nil :quiet)
      (setq eshell-chunk-number (1+ eshell-chunk-number)))

    (add-hook 'eshell-output-filter-functions
              'catch-eshell-output-chunk)

And then afterwards 'cat eshell-output/chunk.*' should hopefully
reproduce it?

bug#28329: 25.2; Eshell: 40M+ output: Stack overflow in regexp matcher

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 257 bytes --]


Will do.  It's a tough one though because for now reproducing involves 1
hour+ of compiling...

--
Pierre Neidhardt

Pascal:
	A programming language named after a man who would turn over
	in his grave if he knew about it.
		-- Datamation, January 15, 1984

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#28329: 25.2; Eshell: 40M+ output: Stack overflow in regexp matcher

[Stefan Kangas]

Pierre Neidhardt <ambrevar@gmail.com> writes:
> Will do.  It's a tough one though because for now reproducing involves 1
> hour+ of compiling...

Hi Pierre,

Did you ever get around to looking into this further?  It seems to me
like Noam Postavsky had some ideas, but that your input would be needed
to make progress.

Thanks,
Stefan Kangas

bug#28329: 25.2; Eshell: 40M+ output: Stack overflow in regexp matcher

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 461 bytes --]

Hi Stefan!

Sorry, I should have updated this thread.
So I never got around reproducing the issue with Gentoo, and soon after
I dropped Gentoo and moved to Guix, with which I've never been able to
re-experience this issue, even on massive Eshell buffers (couple of 10M...).

As far as I'm concerned, this could be closed, but maybe another Gentoo
user out there would be able to tell us more.

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#28329: 25.2; Eshell: 40M+ output: Stack overflow in regexp matcher

[Stefan Kangas]

Pierre Neidhardt <mail@ambrevar.xyz> writes:
> Sorry, I should have updated this thread.
> So I never got around reproducing the issue with Gentoo, and soon after
> I dropped Gentoo and moved to Guix, with which I've never been able to
> re-experience this issue, even on massive Eshell buffers (couple of 10M...).

Thank you for your reply.

> As far as I'm concerned, this could be closed, but maybe another Gentoo
> user out there would be able to tell us more.

OK, now closed.  If anyone else runs into any issues, they can always
open a new bug report.

Best regards,
Stefan Kangas

bug#28329: 25.2; Eshell: 40M+ output: Stack overflow in regexp matcher

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 611 bytes --]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> So I never got around reproducing the issue with Gentoo, and soon after
> I dropped Gentoo and moved to Guix, with which I've never been able to
> re-experience this issue, even on massive Eshell buffers (couple of 10M...).
>
> As far as I'm concerned, this could be closed, but maybe another Gentoo
> user out there would be able to tell us more.

Oh yeah, I still have a possible fix sitting in my local repo.
Attaching here for posterity, but I wouldn't want to apply it unless we
actually know that it would help (because it might break legitimate
matches).


[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 1264 bytes --]

From 7a3bdc88941f90c5506165ddf4b9d12f51ba9ea3 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Sat, 2 Sep 2017 08:17:48 -0400
Subject: [PATCH] [?] Only match password prompts in part of line (Bug#28329)

* lisp/eshell/esh-mode.el (eshell-password-prompt-regexp): Add
beginning of line anchor, and use {,80} instead of * to restrict
matches in long lines.
---
 lisp/eshell/esh-mode.el | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/lisp/eshell/esh-mode.el b/lisp/eshell/esh-mode.el
index 80844c3a64..9f80700c6a 100644
--- a/lisp/eshell/esh-mode.el
+++ b/lisp/eshell/esh-mode.el
@@ -179,7 +179,10 @@ eshell-preoutput-filter-functions
   :group 'eshell-mode)
 
 (defcustom eshell-password-prompt-regexp
-  (format "\\(%s\\)[^:：៖]*[:：៖]\\s *\\'" (regexp-opt password-word-equivalents))
+  ;; Assume prompt will show up near the beginning of a line, this
+  ;; should prevent excessive regexp searches on long lines.
+  (format "^.\\{,80\\}%s[^:：៖]\\{,80\\}[:：៖]\\s-*\\'"
+          (regexp-opt password-word-equivalents))
   "Regexp matching prompts for passwords in the inferior process.
 This is used by `eshell-watch-for-password-prompt'."
   :type 'regexp
-- 
2.11.0

bug#28329: 25.2; Eshell: 40M+ output: Stack overflow in regexp matcher

[Mattias Engdegård]

>(format "\\(%s\\)[^:：៖]*[:：៖]\\s *\\'" (regexp-opt password-word-equivalents))

The [^:：៖] part matches newlines as well; adding \n there should help a bit, too.

The expression can also be shortened by passing t as second argument to regexp-opt, turning the format into a concat.