bug#33784: 27.0.50; some case c-backward-token-2 takes cpu more and emacs hang

bug#33784: 27.0.50; some case c-backward-token-2 takes cpu more and emacs hang

[xh yang]

[-- Attachment #1: Type: text/plain, Size: 4079 bytes --]

follow case:

(let ((str "#define SLOGE(...)\n
((void)__android_log_buf_print(LOG_ID_SYSTEM, ANDROID_LOG_ERROR, LOG_TAG,\n
__VA_ARGS__))"))
(with-temp-buffer
(delay-mode-hooks (funcall 'c++-mode))
(insert str)
(font-lock-ensure)
(buffer-string)
)
)

execute up code, emacs will hang


In GNU Emacs 27.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.18.9)
of 2018-12-18 built on xhyang-ThinkPad-T470p
Repository revision: 1691a51094d35ac4b2c311fa407c6b77eea7a105
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
System Description: Ubuntu 16.04.3 LTS

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Mark set
#("#define SLOGE(...)
((void)__android_log_buf_print(LOG_ID_SYSTEM, ANDROID_LOG_ERROR, LOG_TAG,
__VA_ARGS__))" 0 1 (c-is-sws t c-in-sws t) 1 7 (c-in-sws t) 7 8 (c-in-sws
t) 8 13 (c-in-sws t) 13 19 (c-in-sws t) 19 21 (c-in-sws t c-is-sws t) 21 22
(c-is-sws t))
next-line: End of buffer

Configured using:
'configure --enable-link-time-optimization --without-pop
--without-kerberos --without-kerberos5 --without-hesiod
--with-sound=alsa --with-x-toolkit=gtk3 --with-xpm --with-jpeg
--with-tiff --with-gif --with-png --with-rsvg --with-xml2
--with-imagemagick --with-xft --without-libotf --without-xim
--with-xaw3d --with-dbus --without-gconf --without-gsettings
--without-selinux --with-gnutls --with-zlib --with-modules
--with-file-notification=inotify --without-makeinfo --with-x
--exec-prefix=/usr/'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GLIB NOTIFY
INOTIFY GNUTLS LIBXML2 FREETYPE XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11
XDBE XIM MODULES THREADS LCMS2 GMP

Important settings:
value of $LC_MONETARY: zh_CN.UTF-8
value of $LC_NUMERIC: zh_CN.UTF-8
value of $LC_TIME: zh_CN.UTF-8
value of $LANG: en_US.UTF-8
value of $XMODIFIERS: @im=fcitx
locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
tooltip-mode: t
global-eldoc-mode: t
eldoc-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
tool-bar-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
line-number-mode: t
transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv dired dired-loaddefs format-spec rfc822 mml
mml-sec password-cache epa derived epg epg-config gnus-util rmail
rmail-loaddefs time-date mm-decode mm-bodies mm-encode mail-parse
rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils cc-mode cc-fonts easymenu
cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs
cl-loaddefs cl-lib elec-pair mule-util tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow isearch timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify lcms2 dynamic-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 114827 9593)
(symbols 48 22507 1)
(strings 32 34433 2178)
(string-bytes 1 1049070)
(vectors 16 16742)
(vector-slots 8 532827 13806)
(floats 8 49 68)
(intervals 56 261 1)
(buffers 992 11)
(heap 1024 38686 1099))

[-- Attachment #2: Type: text/html, Size: 4876 bytes --]

bug#33784: 27.0.50; some case c-backward-token-2 takes cpu more and emacs hang

[Alan Mackenzie]

Hello, xh.

In article <mailman.5878.1545108246.1284.bug-gnu-emacs@gnu.org> you wrote:
> [-- text/plain, encoding 7bit, charset: UTF-8, 117 lines --]

> follow case:

> (let ((str "#define SLOGE(...)\n
> ((void)__android_log_buf_print(LOG_ID_SYSTEM, ANDROID_LOG_ERROR, LOG_TAG,\n
> __VA_ARGS__))"))
> (with-temp-buffer
> (delay-mode-hooks (funcall 'c++-mode))
> (insert str)
> (font-lock-ensure)
> (buffer-string)
> )
> )

> execute up code, emacs will hang

Yes.  The problem here is the (font-lock-ensure).  This is being called
even though font-lock-mode is disabled.  I think there should be a check
for this in font-lock-ensure, but there isn't.

So, yes, we have a bug here.  Thanks for taking the trouble to report
it.

Incidentally, the temporary buffer created by with-temp-buffer never has
font-lock-mode enabled, and font-lock-mode is spiked so that it cannot
be enabled in such a buffer.  I don't know why.  Attempting to enable it
fails silently, which is probably another bug.

Incidentally[2], the delay-mode-hooks in your recipe has nothing to do
with the problem.

> In GNU Emacs 27.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.18.9)
> of 2018-12-18 built on xhyang-ThinkPad-T470p
> Repository revision: 1691a51094d35ac4b2c311fa407c6b77eea7a105
> Repository branch: master
> Windowing system distributor 'The X.Org Foundation', version 11.0.11906000
> System Description: Ubuntu 16.04.3 LTS

[ .... ]

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#33784: 27.0.50; some case c-backward-token-2 takes cpu more and emacs hang

[Eli Zaretskii]

> Date: 18 Dec 2018 17:47:16 -0000
> From: Alan Mackenzie <acm@muc.de>
> Cc: 33784@debbugs.gnu.org
> 
> Incidentally, the temporary buffer created by with-temp-buffer never has
> font-lock-mode enabled, and font-lock-mode is spiked so that it cannot
> be enabled in such a buffer.  I don't know why.  Attempting to enable it
> fails silently, which is probably another bug.

It's not a bug, since the temporary buffer starts in Fundamental mode.

bug#33784: 27.0.50; some case c-backward-token-2 takes cpu more and emacs hang

[Alan Mackenzie]

Hello, Eli.

On Tue, Dec 18, 2018 at 20:51:48 +0200, Eli Zaretskii wrote:
> > Date: 18 Dec 2018 17:47:16 -0000
> > From: Alan Mackenzie <acm@muc.de>
> > Cc: 33784@debbugs.gnu.org

> > Incidentally, the temporary buffer created by with-temp-buffer never has
> > font-lock-mode enabled, and font-lock-mode is spiked so that it cannot
> > be enabled in such a buffer.  I don't know why.  Attempting to enable it
> > fails silently, which is probably another bug.

> It's not a bug, since the temporary buffer starts in Fundamental mode.

If fails silently even after M-: c++-mode in that buffer.

The current strategy is silently to ignore (font-lock-mode) when the
buffer name begins with a space.  It feels to me like an easy to
implement, but suboptimal, workaround to a real problem, whatever that
real problem might be.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#33784: 27.0.50; some case c-backward-token-2 takes cpu more and emacs hang

[Eli Zaretskii]

> Date: Tue, 18 Dec 2018 18:55:05 +0000
> Cc: linux.xhyang@gmail.com, 33784@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> The current strategy is silently to ignore (font-lock-mode) when the
> buffer name begins with a space.  It feels to me like an easy to
> implement, but suboptimal, workaround to a real problem, whatever that
> real problem might be.

The real problem is probably performance.  But that's a guess; someone
will have to do the forensics to find out which change did that, and
then try to find related bug reports and/or discussions.

bug#33784: 27.0.50; some case c-backward-token-2 takes cpu more and emacs hang

[Alan Mackenzie]

Hello, Eli.

On Tue, Dec 18, 2018 at 21:23:45 +0200, Eli Zaretskii wrote:
> > Date: Tue, 18 Dec 2018 18:55:05 +0000
> > Cc: linux.xhyang@gmail.com, 33784@debbugs.gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > The current strategy is silently to ignore (font-lock-mode) when the
> > buffer name begins with a space.  It feels to me like an easy to
> > implement, but suboptimal, workaround to a real problem, whatever that
> > real problem might be.

> The real problem is probably performance.  But that's a guess; someone
> will have to do the forensics to find out which change did that, and
> then try to find related bug reports and/or discussions.

Hah!  The change, not to fontify buffers with names beginning with
spaces, was made by Simon Marshall on 1995-12-09.  The list archives only
go back to 2000.  :-(

I suspect what happened was that back last millenium, there was a strong
convention for what a buffer with a name beginning with a space meant,
and it was perfectly OK then not to fontify such a buffer.  Over time,
that convention became diluted such that even buffers created by users
(e.g. by with-temp-buffer) get names starting with a space.  But that's
only a guess.

Maybe the solution would be not to start with-temp-buffer names with a
space.

Indeed page "Buffer Names" in Elisp states "Buffers that are ephemeral
and generally uninteresting to the user have names starting with a
space", which is ambiguous - does a buffer have to be both ephemeral and
generally uninteresting, or will one of these properties do?  Probably
the latter.

Maybe the documentation for with-temp-buffer should be amended to
recommend Lisp programmers not to use it for buffers holding user text.

Or something like that.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#33784: 27.0.50; some case c-backward-token-2 takes cpu more and emacs hang

[Glenn Morris]

> Maybe the solution would be not to start with-temp-buffer names with a
> space.

This sounds totally wrong to me.

> Maybe the documentation for with-temp-buffer should be amended to
> recommend Lisp programmers not to use it for buffers holding user text.

This also sounds wrong to me.

I confess I have no idea what problem you are trying to solve here.

bug#33784: 27.0.50; some case c-backward-token-2 takes cpu more and emacs hang

[Eli Zaretskii]

> Date: Tue, 18 Dec 2018 21:05:07 +0000
> Cc: linux.xhyang@gmail.com, 33784@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > The real problem is probably performance.  But that's a guess; someone
> > will have to do the forensics to find out which change did that, and
> > then try to find related bug reports and/or discussions.
> 
> Hah!  The change, not to fontify buffers with names beginning with
> spaces, was made by Simon Marshall on 1995-12-09.  The list archives only
> go back to 2000.  :-(

Maybe we should simply remove that limitation.  Most temporary buffers
are in Fundamental mode anyway, and a few that aren't should support
font-lock.  If they are rarely or never displayed, we should be fine.

> Maybe the documentation for with-temp-buffer should be amended to
> recommend Lisp programmers not to use it for buffers holding user text.

I believe this is a limitation that is unjustifiably harsh.

bug#33784: 27.0.50; some case c-backward-token-2 takes cpu more and emacs hang

[Alan Mackenzie]

Hello, Glenn.

On Tue, Dec 18, 2018 at 17:05:22 -0500, Glenn Morris wrote:

> > Maybe the solution would be not to start with-temp-buffer names with
> > a space.

> This sounds totally wrong to me.

> > Maybe the documentation for with-temp-buffer should be amended to
> > recommend Lisp programmers not to use it for buffers holding user
> > text.

> This also sounds wrong to me.

> I confess I have no idea what problem you are trying to solve here.

I have lost the clear overview I had of the problem yesterday, I'm
afraid.

There is an assemblage of confusing features around this bug which seem
to be contributing to the reported bug:

(i) A temporary buffer is created and made a C++ Mode buffer.
(ii) Font Lock Mode isn't enabled.
(iii) Although font lock isn't enabled, font-lock-ensure is called.
This hangs.
(iv) An attempt to enable Font Lock Mode on this buffer fails silently,
because the buffer name begins with a space.
(v) It is unclear whether font-lock-ensure is intended to work when Font
Lock Mode is disabled.
(vi) The spiking of Font Lock Mode to fail on "space buffers" is not
documented, and 23 years later, the reason for it is obscure.

Hope that helps, if only a little.

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#33784:

[xh yang]

[-- Attachment #1: Type: text/plain, Size: 367 bytes --]

I found if remove first '\n' from the *str,then run emacs will not hang.*
*May be it has nothing to do with font-lock, just str parsing ?*

(let ((str "#define SLOGE(...)\n
((void)__android_log_buf_print(LOG_ID_SYSTEM, ANDROID_LOG_ERROR, LOG_TAG,\n

change to
(let ((str "#define SLOGE(...)
((void)__android_log_buf_print(LOG_ID_SYSTEM, ANDROID_LOG_ERROR, LOG_TAG,\n

[-- Attachment #2: Type: text/html, Size: 769 bytes --]

bug#33784:

[Alan Mackenzie]

Hello, xh.

On Thu, Dec 20, 2018 at 15:18:49 +0800, xh yang wrote:
> I found if remove first '\n' from the *str,then run emacs will not hang.*
> *May be it has nothing to do with font-lock, just str parsing ?*

Sorry for all the distraction in the last couple of days.  As you
suggested in your opening post, the problem was a simple infinite loop
involving c-backward-token-2.  This was in CC Mode's font-locking code.

I've committed a fix to the emacs-26 branch, and it should find its way
to the master branch within a few days.  I'm closing the bug.

Here is the patch:


# HG changeset patch
# User Alan Mackenzie <acm@muc.de>
# Date 1545307557 0
#      Thu Dec 20 12:05:57 2018 +0000
# Node ID 5319aa054ccb77924e19b836093c9e1b3ff91d4b
# Parent  a3f28b92890acc5289cde497ea1335f4b39239d7
Check result from c-backward-token-2 to avoid infinite loop

This fixes bug #33784.

* cc-fonts.el (c-get-fontification-context): While moving back over enclosing
parentheses, check that c-backward-token-2 actually moves.

diff -r a3f28b92890a -r 5319aa054ccb cc-fonts.el
--- a/cc-fonts.el	Thu Dec 20 12:04:53 2018 +0000
+++ b/cc-fonts.el	Thu Dec 20 12:05:57 2018 +0000
@@ -1255,8 +1255,8 @@
 		(save-excursion
 		  (goto-char match-pos)
 		  (while
-		      (progn (c-backward-token-2)
-			     (eq (char-after) ?\()))
+		      (and (zerop (c-backward-token-2))
+			   (eq (char-after) ?\()))
 		  (looking-at c-arithmetic-op-regexp)))
 	   (cons nil nil))
 	  ;; In a C++ member initialization list.


Thanks, once more, for the bug report.

-- 
Alan Mackenzie (Nuremberg, Germany).