bug#40907: Font-lock-mode hanging in Java buffers

bug#40907: Font-lock-mode hanging in Java buffers

[Adam Van Ymeren]

I've been doing a lot of Java work lately, and I'm having issues with 
emacs hanging and spinning at 100% CPU.  I believe the culprit to be 
font-lock-mode.  C-g doesn't save me either, I usually have to kill 
emacs.  Any advice on how to debug this if C-g doesn't work?

Below is the contents of M-x report-emacs-bug if relevant.  This is a 
custom trisquel emacs package I built using unmodified upstream 26.3 
sources.

Thanks!

-Adam



In GNU Emacs 26.3 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.18.9)
  of 2020-03-27 built on gus
Windowing system distributor 'The X.Org Foundation', version 11.0.11804000
System Description:    Trisquel GNU/Linux 8.0, Flidas


Configured using:
  'configure --build=x86_64-linux-gnu --prefix=/usr
  '--includedir=${prefix}/include' '--mandir=${prefix}/share/man'
  '--infodir=${prefix}/share/info' --sysconfdir=/etc --localstatedir=/var
  --disable-silent-rules '--libdir=${prefix}/lib/x86_64-linux-gnu'
  '--libexecdir=${prefix}/lib/x86_64-linux-gnu' --disable-maintainer-mode
  --disable-dependency-tracking --with-modules --with-xwidgets 'CFLAGS=-g
  -O2 -fstack-protector-strong -Wformat -Werror=format-security'
  'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2'
  'LDFLAGS=-Wl,-Bsymbolic-functions -Wl,-z,relro''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND DBUS GSETTINGS GLIB NOTIFY ACL
LIBSELINUX GNUTLS LIBXML2 FREETYPE XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11
XDBE XIM MODULES THREADS XWIDGETS LIBSYSTEMD LCMS2

Important settings:
   value of $LANG: en_CA.UTF-8
   value of $XMODIFIERS: @im=ibus
   locale-coding-system: utf-8-unix

Major mode: Dired by name

Minor modes in effect:
   diff-auto-refine-mode: t
   tooltip-mode: t
   global-eldoc-mode: t
   electric-indent-mode: t
   mouse-wheel-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   blink-cursor-mode: t
   auto-composition-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   buffer-read-only: t
   line-number-mode: t
   transient-mark-mode: t

Load-path shadows:
/usr/share/emacs/site-lisp/dictionaries-common/ispell hides 
/usr/share/emacs/26.3/lisp/textmodes/ispell
/usr/share/emacs/site-lisp/dictionaries-common/flyspell hides 
/usr/share/emacs/26.3/lisp/textmodes/flyspell
/home/adam2/.emacs.d/elpa/let-alist-1.0.6/let-alist hides 
/usr/share/emacs/26.3/lisp/emacs-lisp/let-alist

Features:
(shadow sort mail-extr emacsbug sendmail ediff-merg ediff-wind
ediff-diff ediff-mult ediff-help ediff-init ediff-util ediff diary-lib
diary-loaddefs cal-iso org-rmail org-mhe org-irc org-info org-gnus nnir
gnus-sum gnus-group gnus-undo gnus-start gnus-cloud nnimap nnmail
mail-source tls gnutls utf7 netrc nnoo parse-time gnus-spec gnus-int
gnus-range message rmc puny rfc822 mml mml-sec epa derived epg mm-decode
mm-bodies mailabbrev gmm-utils mailheader gnus-win gnus nnheader
gnus-util rmail rmail-loaddefs mail-utils wid-edit org-docview doc-view
jka-compr image-mode org-bibtex bibtex org-bbdb org-w3m org-agenda
vc-git diff-mode misearch multi-isearch dired-aux warnings hywconfig
cl-extra help-mode elec-pair ob-js base16-solarized-light-theme
base16-theme esh-var esh-io esh-cmd esh-opt esh-ext esh-proc esh-arg
esh-groups eshell esh-module esh-mode esh-util erc-goodies erc
erc-backend erc-compat ob-screen ob-java js sgml-mode dom imenu
thingatpt cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align
cc-engine cc-vars cc-defs ediprolog hyperbole hinit hibtypes hib-doc-id
hsys-www klink hib-kbd hib-social hib-debbugs hsys-org hactypes hui-mini
hui hui-mouse hmouse-key hargs hui-menu hyrolo-menu hyrolo hmail
hui-jmenu hmouse-sh hmouse-drv hypb locate hui-window pulse hycontrol
hsettings hui-em-but hbut hact hpath subr-x hhist hbdata htz cal-julian
hbmap hmoccur hui-select hvar set hversion hload-path exwm-randr
xcb-randr exwm-config ido exwm exwm-input xcb-keysyms xcb-xkb
exwm-manage exwm-floating xcb-cursor xcb-render exwm-layout
exwm-workspace exwm-core xcb-ewmh xcb-icccm xcb xcb-xproto xcb-types
xcb-debug server ox-odt rng-loc rng-uri rng-parse rng-match rng-dt
rng-util rng-pttrn nxml-parse nxml-ns nxml-enc xmltok nxml-util ox-latex
ox-icalendar ox-html table ox-ascii ox-publish ox org-element avl-tree
generator org advice org-macro org-footnote org-pcomplete pcomplete
org-list org-faces org-entities noutline outline easy-mmode org-version
ob-emacs-lisp ob ob-tangle org-src ob-ref ob-lob ob-table ob-keys
ob-comint comint ansi-color ring org-loaddefs cal-menu calendar
cal-loaddefs ob-exp ob-core org-compat ob-eval org-macs format-spec
elnode dired dired-loaddefs db web time-stamp browse-url json map rx s
url-util mailcap mm-encode mail-parse rfc2231 rfc2047 rfc2045 mm-util
ietf-drums mail-prsvr fakir dotassoc kv noflet cl-indent cl dash ert pp
find-func ewoc debug finder-inf kotl-autoloads info slime-autoloads
package easymenu epg-config url-handlers url-parse auth-source cl-seq
eieio eieio-core cl-macs eieio-loaddefs password-cache url-vars seq
byte-opt gv bytecomp byte-compile cconv edmacro kmacro cl-loaddefs
cl-lib time-date mule-util tooltip eldoc electric uniquify ediff-hook
vc-hooks lisp-float-type mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core term/tty-colors frame cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
xwidget-internal move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 486346 46813)
  (symbols 48 56643 2)
  (miscs 40 265 1145)
  (strings 32 145747 8912)
  (string-bytes 1 4635211)
  (vectors 16 67515)
  (vector-slots 8 1265304 32572)
  (floats 8 322 251)
  (intervals 56 2498 406)
  (buffers 992 23)
  (heap 1024 69920 2977))

bug#40907: Font-lock-mode hanging in Java buffers

[Dmitry Gutov]

On 27.04.2020 20:30, Adam Van Ymeren wrote:
> I've been doing a lot of Java work lately, and I'm having issues with 
> emacs hanging and spinning at 100% CPU.  I believe the culprit to be 
> font-lock-mode.  C-g doesn't save me either, I usually have to kill 
> emacs.  Any advice on how to debug this if C-g doesn't work?

You can send the USR2 signal to the Emacs process to try to interrupt 
what it's doing and enter the debugger:

kill -SIGUSR2 <pid>

To fix this, we'll most likely need the contents of the buffer that has 
the problem. If it's proprietary, you could try recreating the problem 
from scratch, or replacing all identifiers in the code with something 
else, etc.

You could also try a build of Emacs 27 first before doing all that.

bug#40907: Font-lock-mode hanging in Java buffers

[Adam Van Ymeren]

On 2020-04-27 5:38 p.m., Dmitry Gutov wrote:
> On 27.04.2020 20:30, Adam Van Ymeren wrote:
>> I've been doing a lot of Java work lately, and I'm having issues with 
>> emacs hanging and spinning at 100% CPU.  I believe the culprit to be 
>> font-lock-mode.  C-g doesn't save me either, I usually have to kill 
>> emacs.  Any advice on how to debug this if C-g doesn't work?
>
> You can send the USR2 signal to the Emacs process to try to interrupt 
> what it's doing and enter the debugger:
>
> kill -SIGUSR2 <pid>
>
> To fix this, we'll most likely need the contents of the buffer that 
> has the problem. If it's proprietary, you could try recreating the 
> problem from scratch, or replacing all identifiers in the code with 
> something else, etc.
>
> You could also try a build of Emacs 27 first before doing all that.


Wow SIGUSR2 is magical, thanks for that tip.  I reproduced it once 
visiting a particular file but it's not happening reliably for me now.  
Below is the stacktrace and the contents of the associated buffer, but 
it's not reproducing reliably.  Now that I know about SIGUSR2 I'll 
report back with a reliable repro once I figure one out.


Here's the emacs stacktrace:

Debugger entered--entering a function:

* #f(compiled-function () #<bytecode 0xe5f179>)()

   c-parse-state()

   c-syntactic-skip-backward("^;{}" 5344 t)

   c-fl-decl-start(5927)

   c-context-expand-fl-region(5927 6427)

   #f(compiled-function (fn) #<bytecode 
0xec98d5>)(c-context-expand-fl-region)

   mapc(#f(compiled-function (fn) #<bytecode 0xec98d5>) 
(c-context-expand-fl-region))

   c-before-context-fl-expand-region(5927 6427)

   c-font-lock-fontify-region(5927 6427 nil)

   font-lock-fontify-region(5927 6427)

   #f(compiled-function (fun) #<bytecode 
0x14a42f5>)(font-lock-fontify-region)

   run-hook-wrapped(#f(compiled-function (fun) #<bytecode 0x14a42f5>) 
font-lock-fontify-region)

   jit-lock--run-functions(5927 6427)

   jit-lock-fontify-now(5927 6427)

   jit-lock-function(5927)

   redisplay_internal\ \(C\ function\)()


And the buffer contents are given after ====

===============

/**

  * @license

  * Copyright 2017 The FOAM Authors. All Rights Reserved.

  * http://www.apache.org/licenses/LICENSE-2.0

  */

package foam.dao;

import foam.core.*;

import foam.dao.index.*;

import foam.mlang.order.Comparator;

import foam.mlang.predicate.Or;

import foam.mlang.predicate.Predicate;

import foam.mlang.sink.GroupBy;

import foam.nanos.logger.Logger;

import foam.nanos.pm.PM;

import java.util.ArrayList;

import java.util.List;

import java.util.HashSet;

import java.util.Set;

/**

  The MDAO class for an ordering, fast lookup, single value,

  index multiplexer, or any other MDAO select() assistance class.

  The assitance class TreeIndex implements the

  data nodes that hold the indexed items and plan and execute

  queries. For any particular operational Index, there may be

  many IndexNode instances:

  <pre>

  1---------> TreeIndex(id)

  MDAO: AltIndex 2---------> TreeIndex(propA) ---> TreeIndex(id) -------------> ValueIndex

  | 1x AltIndexNode    | 1x TreeIndexNode    | 14x TreeIndexNodes         | (DAO size)x ValueIndexNodes

  (2 alt subindexes)     (14 nodes)             (each has 0-5 nodes)

  </pre>

  The base AltIndex has two complete subindexes (each holds the entire DAO).

  The TreeIndex on property A has created one TreeIndexNode, holding one tree of 14 nodes.

  Each tree node contains a tail instance of the next level down, thus

  the TreeIndex on id has created 14 TreeIndexNodes. Each of those contains some number

  of tree nodes, each holding one tail instance of the ValueIndex at the end of the chain.

  */

  // TODO: clone and freeze objects stored in memory

public class MDAO

   extends AbstractDAO

{

   protected AltIndex index_;

   protected Object   state_ = null;

   protected Object   writeLock_ = new Object();

   protected Set      unindexed_ = new HashSet();

   public MDAO(ClassInfo of) {

     setOf(of);

     index_ = new AltIndex(new TreeIndex((PropertyInfo) this.of_.getAxiomByName("id")));

   }

   public void addIndex(Index index) {

     synchronized ( writeLock_ ) {

       state_ = index_.addIndex(state_, index);

     }

   }

   /** Add an Index which is for a unique value. Use addIndex() if the index is not unique. **/

   public void addUniqueIndex(PropertyInfo... props) {

     Index i = ValueIndex.instance();

     for ( PropertyInfo prop : props ) i = new TreeIndex(prop, i);

     addIndex(i);

   }

   /** Add an Index which is for a non-unique value. The 'id' property is

    * appended to property list to make it unique.

    **/

   public void addIndex(PropertyInfo... props) {

     Index i = new TreeIndex((PropertyInfo) this.of_.getAxiomByName("id"));

     for ( PropertyInfo prop : props ) i = new TreeIndex(prop, i);

     addIndex(i);

   }

   synchronized Object getState() {

     return state_;

   }

   synchronized void setState(Object state) {

     state_ = state;

   }

   public FObject objIn(FObject obj) {

     return obj.fclone().freeze();

   }

   public FObject objOut(FObject obj) {

     return obj;

   }

   public FObject put_(X x, FObject obj) {

     // Clone and freeze outside of lock to minimize time spent under lock

     obj = objIn(obj);

     synchronized ( writeLock_ ) {

       FObject oldValue = find_(x, obj);

       Object  state    = getState();

       if ( oldValue != null ) {

         state = index_.remove(state, oldValue);

       }

       setState(index_.put(state, obj));

     }

     onPut(obj);

     return obj;

   }

   public FObject remove_(X x, FObject obj) {

     if ( obj == null ) return null;

     FObject found;

     synchronized ( writeLock_ ) {

       found = find_(x, obj);

       if ( found != null ) {

         setState(index_.remove(getState(), found));

       }

     }

     if ( found != null ) {

       onRemove(found);

     }

     return found;

   }

   public FObject find_(X x, Object o) {

     Object state;

     state = getState();

     if ( o == null ) return null;

     // TODO: PM unindexed plans

     return objOut(

       getOf().isInstance(o)

         ? (FObject) index_.planFind(state, getPrimaryKey().get(o)).find(state, getPrimaryKey().get(o))

         : (FObject) index_.planFind(state, o).find(state, o)

     );

   }

   public Sink select_(X x, Sink sink, long skip, long limit, Comparator order, Predicate predicate) {

     Logger     logger = (Logger) x.get("logger");

     SelectPlan plan;

     Predicate  simplePredicate = null;

     PM         pm = null;

     // use partialEval to wipe out such useless predicate such as: And(EQ()) ==> EQ(), And(And(EQ()),GT()) ==> And(EQ(),GT())

     if ( predicate != null ) simplePredicate = predicate.partialEval();

     Object state = getState();

     // We handle OR logic by seperate request from MDAO. We return different plan for each parameter of OR logic.

     if ( simplePredicate instanceof Or ) {

       Sink dependSink = new ArraySink();

       // When we have groupBy, order, skip, limit such requirement, we can't do it separately so I replace a array sink to temporarily holde the whole data

       //Then after the plan wa slelect we change it to the origin sink

       int length = ((Or) simplePredicate).getArgs().length;

       List<Plan> planList = new ArrayList<>();

       for ( int i = 0 ; i < length ; i++ ) {

         Predicate arg = ((Or) simplePredicate).getArgs()[i];

         planList.add(index_.planSelect(state, dependSink, 0, AbstractDAO.MAX_SAFE_INTEGER, null, arg));

       }

       plan = new OrPlan(simplePredicate, planList);

     } else {

       plan = index_.planSelect(state, sink, skip, limit, order, simplePredicate);

     }

     if ( state != null && predicate != null && plan.cost() > 10 && plan.cost() >= index_.size(state) ) {

       pm = new PM(this.getClass(), "MDAO:UnindexedSelect:" + getOf().getId());

       if ( ! unindexed_.contains(getOf().getId())) {

         if ( ! predicate.equals(simplePredicate) &&

              logger != null ) {

             logger.debug(String.format("The original predicate was %s but it was simplified to %s.", predicate.toString(), simplePredicate.toString()));

         }

         unindexed_.add(getOf().getId());

         if ( logger != null ) {

           logger.warning("Unindexed search on MDAO", getOf().getId(), simplePredicate.toString());

         }

       }

     }

     plan.select(state, sink, skip, limit, order, simplePredicate);

     if ( pm != null ) pm.log(x);

     sink.eof();

     return sink;

   }

   public void removeAll_(X x, long skip, long limit, Comparator order, Predicate predicate) {

     if ( predicate == null && skip == 0 && limit == MAX_SAFE_INTEGER ) {

       synchronized ( writeLock_ ) {

         setState(null);

       }

     } else {

       super.removeAll_(x, skip, limit, order, predicate);

     }

   }

}

bug#40907: Font-lock-mode hanging in Java buffers

[Alan Mackenzie]

Hello, Adam.

On Mon, Apr 27, 2020 at 18:21:09 +0000, Adam Van Ymeren wrote:

> On 2020-04-27 5:38 p.m., Dmitry Gutov wrote:
> > On 27.04.2020 20:30, Adam Van Ymeren wrote:
> >> I've been doing a lot of Java work lately, and I'm having issues with 
> >> emacs hanging and spinning at 100% CPU.  I believe the culprit to be 
> >> font-lock-mode.  C-g doesn't save me either, I usually have to kill 
> >> emacs.  Any advice on how to debug this if C-g doesn't work?

> > You can send the USR2 signal to the Emacs process to try to interrupt 
> > what it's doing and enter the debugger:

> > kill -SIGUSR2 <pid>

> > To fix this, we'll most likely need the contents of the buffer that 
> > has the problem. If it's proprietary, you could try recreating the 
> > problem from scratch, or replacing all identifiers in the code with 
> > something else, etc.

> > You could also try a build of Emacs 27 first before doing all that.


> Wow SIGUSR2 is magical, thanks for that tip.  I reproduced it once 
> visiting a particular file but it's not happening reliably for me now.  
> Below is the stacktrace and the contents of the associated buffer, but 
> it's not reproducing reliably.  Now that I know about SIGUSR2 I'll 
> report back with a reliable repro once I figure one out.

The buffer copy you sent looks corrupted - it contains lots of
non-breaking spaces (or are these intended?) and it seems to be
double-spaced all the way through.  If it is corrupted, is there any
chance you could post a clean version, possibly by attaching it to your
email?

With that, there's a good chance your stack trace will be enough to be
able to debug this.

There's also a good chance the bug has already been diagnosed and fixed
(for the upcoming Emacs-27.1), but it will need some effort to verify
this.

> Here's the emacs stacktrace:

[ Double spacing removed from this ... ]

> Debugger entered--entering a function:

> * #f(compiled-function () #<bytecode 0xe5f179>)()
>    c-parse-state()
>    c-syntactic-skip-backward("^;{}" 5344 t)
>    c-fl-decl-start(5927)
>    c-context-expand-fl-region(5927 6427)
>    #f(compiled-function (fn) #<bytecode 0xec98d5>)(c-context-expand-fl-region)
>    mapc(#f(compiled-function (fn) #<bytecode 0xec98d5>) (c-context-expand-fl-region))
>    c-before-context-fl-expand-region(5927 6427)
>    c-font-lock-fontify-region(5927 6427 nil)
>    font-lock-fontify-region(5927 6427)
>    #f(compiled-function (fun) #<bytecode 0x14a42f5>)(font-lock-fontify-region)
>    run-hook-wrapped(#f(compiled-function (fun) #<bytecode 0x14a42f5>) font-lock-fontify-region)
>    jit-lock--run-functions(5927 6427)
>    jit-lock-fontify-now(5927 6427)
>    jit-lock-function(5927)
>    redisplay_internal\ \(C\ function\)()

> And the buffer contents are given after ====

> ===============

> /**

>   * @license

>   * Copyright 2017 The FOAM Authors. All Rights Reserved.

>   * http://www.apache.org/licenses/LICENSE-2.0

>   */

> package foam.dao;

> import foam.core.*;

> import foam.dao.index.*;

[ .... ]

-- 
Alan Mackenzie (Nuremberg, Germany).

bug#40907: Font-lock-mode hanging in Java buffers

[Lars Ingebrigtsen]

Alan Mackenzie <acm@muc.de> writes:

> The buffer copy you sent looks corrupted - it contains lots of
> non-breaking spaces (or are these intended?) and it seems to be
> double-spaced all the way through.  If it is corrupted, is there any
> chance you could post a clean version, possibly by attaching it to your
> email?

More information was requested, but no response was given within a
year, so I'm closing this bug report.  If the problem still exists,
please respond to this email and we'll reopen the bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#40907: Font-lock-mode hanging in Java buffers

[Adam Van Ymeren]

Yeah I lost track sorry.  Couldn't get a minimal repo case, if I hit it again I will file a better report.  Sorry for ghosting!

Aug 10, 2021 8:59:36 AM Lars Ingebrigtsen <larsi@gnus.org>:

> Alan Mackenzie <acm@muc.de> writes:
> 
>> The buffer copy you sent looks corrupted - it contains lots of
>> non-breaking spaces (or are these intended?) and it seems to be
>> double-spaced all the way through.  If it is corrupted, is there any
>> chance you could post a clean version, possibly by attaching it to your
>> email?
> 
> More information was requested, but no response was given within a
> year, so I'm closing this bug report.  If the problem still exists,
> please respond to this email and we'll reopen the bug report.
> 
> --
> (domestic pets only, the antidote for overdose, milk.)
>    bloggy blog: http://lars.ingebrigtsen.no