Re: What is the _essential_ difference between lazy-lock and jit-lock?

Re: What is the _essential_ difference between lazy-lock and jit-lock?

[Stefan Monnier]

The following message is a courtesy copy of an article
that has been posted to gnu.emacs.help as well.

[ I suggest we move this to emacs-devel, which I Cc'd. ]

>> I'm pretty sure I've seen this report (although I can't remember it),
>> but it sounds like a different problem than just tuning, more like a
>> real bug, probably a bad-regexp in the font-lock settings of a
>> major-mode (the potentially exponential behavior of the current
>> regexp-engine is a common problem).  This was probably compounded by
>> the fact that inhibit-quit is set during stealth fontification (input
>> is polled instead).

> More info:  When Emacs hangs in this manner, a double C-g followed by
> `fg' often (but not always) brings forth this:

> : Garbage collection in progress; cannot auto-save now
> : but will instead do a real quit after garbage collection ends
> : Abort (and dump core)? (y or n)

> On replying `n', Emacs frees up again.  Left undisturbed it frequently
> goes back into the hung state.

> I conjecture that complicated regexps in font-lock settings are somehow
> creating garbage at a fabulous rate, and jit-lock and the GC are somehow
> thrashing.  Just as soon as jit-lock can be halted and the GC left in
> peace, things return to OK.

Hmm... the regexp-engine does not generate any garbage.  If you're inside
the GC, that means you're not inside the regexp-matcher.  So clearly the
bug is different.  A backtrace (and an xbacktrace) would be helpful.

> I've just noticed something: jit-lock-stealth-load seems to be set by
> default to 200%.  This seems strange.  Should this perhaps be 20%.  ;-)

Have you tried to change it?  200% indeed sounds too high.  But on my
single-user Opteron workstation, with no particular background task
I currently have 16% of CPU use, so a limit of 20% might be a bit low.
Maybe 50% ?

Playing with jit-lock-stealth-nice is also a good idea.  The only problem
with it is that if your machine is very fast and takes 0.001s per chunk,
waiting 0.125s between each chunk leads to ridiculously low CPU usage,
whereas if each chunk takes 1s, then waiting 0.125s between chunks still
leads to more than 80% CPU usage.  So maybe jit-lock-stealth should measure
the time it takes for it to process one chunk and dynamically adapt
jit-lock-stealth-nice so as to produce a given CPU usage.


        Stefan

Re: What is the _essential_ difference between lazy-lock and jit-lock?

[Eli Zaretskii]

> From: Alan Mackenzie<none@example.invalid>
> Newsgroups: gnu.emacs.help
> Date: Mon, 26 Jan 2004 17:53:46 +0000
> 
> > Sorry, no.  And the changes were not in jit-lock.el per se, IIRC, but
> > rather in the font-lock definitions of several modes.
> 
> Hmmm.  Font lock definitions are (usually) just data.  It seems a bit
> strange to blame data for hanging a program, rather than the program for
> not handling the data.

The program does handle the data, it just takes a long time and lots
of CPU power to do it.

> Were you thinking of "ambiguous" regexps, whose
> runtime can explode exponentially w.r.t. the length of the string they're
> examining?

IIRC, one problem was that font-lock definitions forced Emacs to scan
large portions of the buffer in order to decide how to fontify.  When
such scans need to be done backwards for every word, it could take a
lot of CPU cycles.

> For the record, the modes I had loaded when I reported the bug in October
> 2002 were:  C and AWK (both CC Mode 5.28, a released version), Emacs Lisp
> (and Lisp Interaction), Info, Diff, Shell Script, and Fundamental.  Since
> then, C and AWK font lock settings have been entirely rewritten.  Of the
> other modes in this list, were any of them amongst those whose font lock
> definitions were rewritten?

I don't really know ("cvs annotate" should tell you, right?), but
Shell Script mode sounds like a good candidate.