Default of jit-lock-stealth-time

Default of jit-lock-stealth-time

[David Kastrup]

Hi,

I think that for the release, we should set jit-lock-stealth-time to
nil rather than its default of 16.

The reason is that Emacs wastes time fontifying buffers that might
never get displayed.  In addition, for problematic font-lock files,
Emacs' overall response gets sluggish, the cursor display gets
erratic, and system load goes up for no conceivable reason.

The problem is exacerbated when one has lots of buffers open and uses
desktop.el or similar packages.

For example, one of my buffers tends to be texbook.tex for reference
purposes.  This has unbalanced $ signs which make the syntax
highlighting thrash a lot.  It renders my whole system unresponsive
because an unattended Emacs session will at some point of time start
wasting cycles on a file that I have not accessed for days, and likely
will not access for several more days.  Apparently, the stealth
fontification gets stuck without useful progress.  In contrast,
opening the file directly will finish the problematic areas pretty
much immediately.

Other people might have fewer problems, but I would ask any developer
defending this default setting to customize
jit-lock-stealth-verbose
to t and then see in daily use whether he thinks the results a good
idea.

Personally, I am very much of the opinion that we should not enable
_any_ actions by default that can potentially waste considerable
amounts of CPU power in areas not immediately related to user
interaction.

Regardless of the system load: the load may be low at the moment
_exactly_ because the user terminated all processes that would disturb
the interactivity of his current work, and I think it a mistake that
Emacs considers the CPU fit for grabbing for its own purposes unless
the user _explicitly_ asked for it.

Whether it would be desirable to apply more fixes to stealth
fontification, I have no idea.  But the default enabling of it is a
mistake in my book regardless of whether it can be made less onerous.

The problem is exacerbated since it is impossible to guess what the
problem is when your system goes irresponsive:

Setting debug-on-quit will not deliver a backtrace pointing to
jit-lock at all when you press C-g: C-g does not interrupt timers.

There might always be buffers that cause font-lock to behave badly.
Emacs should not attempt fontifying them unless the user puts them
on-screen.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[martin rudalics]

> I think that for the release, we should set jit-lock-stealth-time to
> nil rather than its default of 16.

I think so too.

Re: Default of jit-lock-stealth-time

[Kim F. Storm]

David Kastrup <dak@gnu.org> writes:

> Hi,
>
> I think that for the release, we should set jit-lock-stealth-time to
> nil rather than its default of 16.

I agree.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Default of jit-lock-stealth-time

[Richard Stallman]

    Regardless of the system load: the load may be low at the moment
    _exactly_ because the user terminated all processes that would disturb
    the interactivity of his current work, and I think it a mistake that
    Emacs considers the CPU fit for grabbing for its own purposes unless
    the user _explicitly_ asked for it.

Stealth fontification is intended to stop whenever the user types a
character.  So why would it interfere with the "interactivity" of
Emacs commands?  Is there a bug which prevents stealth fontification
from stopping quickly?

Re: Default of jit-lock-stealth-time

[Stefan Monnier]

> I think that for the release, we should set jit-lock-stealth-time to
> nil rather than its default of 16.

100% agreement.  My .emacs has set it to nil for several years now.


        Stefan

Re: Default of jit-lock-stealth-time

[Stefan Monnier]

>     Regardless of the system load: the load may be low at the moment
>     _exactly_ because the user terminated all processes that would disturb
>     the interactivity of his current work, and I think it a mistake that
>     Emacs considers the CPU fit for grabbing for its own purposes unless
>     the user _explicitly_ asked for it.

> Stealth fontification is intended to stop whenever the user types a
> character.  So why would it interfere with the "interactivity" of
> Emacs commands?  Is there a bug which prevents stealth fontification
> from stopping quickly?

Yes, I know of at least 2 causes:
- it's based on polling and may take a long time before the next poll
- the stealth-fontification causes swapping, and so even if the poll comes
  immediately, pages need to be swapped back in before the user's command
  can complete.


        Stefan

Re: Default of jit-lock-stealth-time

[Romain Francoise]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> 100% agreement.  My .emacs has set it to nil for several years
> now.

Same here.

-- 
Romain Francoise <romain@orebokech.com> | The sea! the sea! the open
it's a miracle -- http://orebokech.com/ | sea! The blue, the fresh, the
                                        | ever free! --Bryan W. Procter

Re: Default of jit-lock-stealth-time

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     Regardless of the system load: the load may be low at the moment
>     _exactly_ because the user terminated all processes that would disturb
>     the interactivity of his current work, and I think it a mistake that
>     Emacs considers the CPU fit for grabbing for its own purposes unless
>     the user _explicitly_ asked for it.
>
> Stealth fontification is intended to stop whenever the user types a
> character.  So why would it interfere with the "interactivity" of
> Emacs commands?

Who is talking about the interactivity of Emacs?  Emacs might not be
the only application on the system.

> Is there a bug which prevents stealth fontification from stopping
> quickly?

Actually, yes.  There are font-locking situations in stealth
fontification that don't finish immediately and even cause quite a bit
of sluggishness for Emacs itself.

But even apart from that, even if all font lock patterns for all modes
worked perfectly, this causes _considerable_ power drain whenever I
restart Emacs, reloading all files using desktop.el.  The fan goes on,
blinking gets sluggish, and I notice that Emacs is eating my batteries
_again_.

Even if this worked perfectly (which it doesn't), we have no business
eating "stealth CPU" power in the default configuration since it is
near impossible for the user (heck, I am not exactly naive, and it was
impossible for me to find this on my own without an explicit pointer
after an error report) to guess just _what_ is eating her CPU.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[David Kastrup]

Romain Francoise <romain@orebokech.com> writes:

> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>
>> 100% agreement.  My .emacs has set it to nil for several years
>> now.
>
> Same here.

Do we have a _single_ developer who still has it on and has observed
_benefits_ from its use?

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[Andreas Schwab]

David Kastrup <dak@gnu.org> writes:

> Do we have a _single_ developer who still has it on

I do.

> and has observed _benefits_ from its use?

At least I didn't notice any negative impact so far.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Default of jit-lock-stealth-time

[Kim F. Storm]

David Kastrup <dak@gnu.org> writes:

> Romain Francoise <romain@orebokech.com> writes:
>
>> Stefan Monnier <monnier@iro.umontreal.ca> writes:
>>
>>> 100% agreement.  My .emacs has set it to nil for several years
>>> now.
>>
>> Same here.
>
> Do we have a _single_ developer who still has it on and has observed
> _benefits_ from its use?

Mine's set to nil too.

I actually forgot all about it until David raised the issue again.

It seems that several developers have set it to nil -- because it
a) doesn't really do any good, and
b) has bad effects on Emacs or the system as a whole

So it seems plain stupid to release it with a setting that will impact
and annoy users and cause a support burden after the release.

I agree that the default should be nil, and IMO a non-nil default
setting for this is a really serious bug.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Default of jit-lock-stealth-time

[David Kastrup]

Andreas Schwab <schwab@suse.de> writes:

> David Kastrup <dak@gnu.org> writes:
>
>> Do we have a _single_ developer who still has it on
>
> I do.
>
>> and has observed _benefits_ from its use?
>
> At least I didn't notice any negative impact so far.

The symptom would be CPU load, and sluggish response of Emacs and the
system in general, when one leaves Emacs unattended for longer amounts
of time.

This is particularly noticeable if the noise level of your system
depends on the CPU load (like increased fan usage).

Would you be likely to notice this on your system?

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[Andreas Schwab]

David Kastrup <dak@gnu.org> writes:

> Andreas Schwab <schwab@suse.de> writes:
>
>> David Kastrup <dak@gnu.org> writes:
>>
>>> Do we have a _single_ developer who still has it on
>>
>> I do.
>>
>>> and has observed _benefits_ from its use?
>>
>> At least I didn't notice any negative impact so far.
>
> The symptom would be CPU load, and sluggish response of Emacs and the
> system in general, when one leaves Emacs unattended for longer amounts
> of time.

I cannot say that I have seen this so far.  The instance of Emacs I'm
currently using was started 41 days ago, and I have another one running
for 69 days (the latter is only used for irc).

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Default of jit-lock-stealth-time

[Juanma Barranquero]

On 2/26/07, David Kastrup <dak@gnu.org> wrote:

I don't have an opinion about the setting of the variable, but FWIW
I'm in Andreas' boat: I have it at default value, and I've never
noticed any ill effect.

> This is particularly noticeable if the noise level of your system
> depends on the CPU load (like increased fan usage).
>
> Would you be likely to notice this on your system?

Not at work's machine, but definitely on my home's laptop. And I
didn't notice it (not that it didn't happen, but if it *did* happen it
wasn't much of an issue).

Perhaps a better metric would be: do you always have lots of buffers?
I don't. Most of the time I have perhaps two or three (non-special)
buffers open. I tend to quickly close any buffer I'm not working on.

             Juanma

Re: Default of jit-lock-stealth-time

[David Kastrup]

"Juanma Barranquero" <lekktu@gmail.com> writes:

> On 2/26/07, David Kastrup <dak@gnu.org> wrote:
>
> I don't have an opinion about the setting of the variable, but FWIW
> I'm in Andreas' boat: I have it at default value, and I've never
> noticed any ill effect.
>
>> This is particularly noticeable if the noise level of your system
>> depends on the CPU load (like increased fan usage).
>>
>> Would you be likely to notice this on your system?
>
> Not at work's machine, but definitely on my home's laptop. And I
> didn't notice it (not that it didn't happen, but if it *did* happen it
> wasn't much of an issue).
>
> Perhaps a better metric would be: do you always have lots of
> buffers?

Yes.

> I don't. Most of the time I have perhaps two or three (non-special)
> buffers open. I tend to quickly close any buffer I'm not working on.

Ok, but working with many buffers is not something particularly
forbidden.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[Jan Djärv]

Juanma Barranquero skrev:
> On 2/26/07, David Kastrup <dak@gnu.org> wrote:
> 
> I don't have an opinion about the setting of the variable, but FWIW
> I'm in Andreas' boat: I have it at default value, and I've never
> noticed any ill effect.

I have the default as well.  I have noticed that Emacs can take some CPU when 
it "should" be idle.  However, for the most part, I can't say I noticed any 
ill effects either.

> 
>> This is particularly noticeable if the noise level of your system
>> depends on the CPU load (like increased fan usage).
>>
>> Would you be likely to notice this on your system?
> 
> Not at work's machine, but definitely on my home's laptop. And I
> didn't notice it (not that it didn't happen, but if it *did* happen it
> wasn't much of an issue).

I notice increased fan noice at work.

> 
> Perhaps a better metric would be: do you always have lots of buffers?
> I don't. Most of the time I have perhaps two or three (non-special)
> buffers open. I tend to quickly close any buffer I'm not working on.

I usually have something like 50-200 buffer.  Reloading the desktop is not 
very fast :-)

	Jan D.

Re: Default of jit-lock-stealth-time

[martin rudalics]

 > So it seems plain stupid to release it with a setting that will impact
 > and annoy users and cause a support burden after the release.

While I agree with all of David's arguments (and even add the one that
a buffer tail is stealthily refontified after every single buffer change
followed by 16 secs idleness) I'm afraid that setting this to nil won't
remove any support burden.  Many major modes still work better when they
are allowed to fontify a buffer from beginning to end.

Also, when a mode uses font-lock to assign `syntax-table' properties,
there's a slight chance that an editing sequence that worked with
stealth fontification turned on will not work any more when stealth
fontification is turned off.  Having fontification assign syntax-table
properties is certainly flawed by design but at the moment there's no
other way to do that automatically.

With other words, setting this to nil may cause some obscure bugs get
reported (and hopefully fixed) earlier ...

martin, who has stealth fontification turned off ever since

Re: Default of jit-lock-stealth-time

[Stefan Monnier]

> While I agree with all of David's arguments (and even add the one that
> a buffer tail is stealthily refontified after every single buffer change
> followed by 16 secs idleness) I'm afraid that setting this to nil won't
> remove any support burden.  Many major modes still work better when they
> are allowed to fontify a buffer from beginning to end.

Stealth fontification doesn't do that anyway.

> Also, when a mode uses font-lock to assign `syntax-table' properties,
> there's a slight chance that an editing sequence that worked with
> stealth fontification turned on will not work any more when stealth
> fontification is turned off.  Having fontification assign syntax-table
> properties is certainly flawed by design but at the moment there's no
> other way to do that automatically.

When there are such bugs, stealth fontification doesn't prevent them.
It may hide them occasionally (non-deterministically), depending on the
specific editing steps.

> With other words, setting this to nil may cause some obscure bugs get
> reported (and hopefully fixed) earlier ...

Indeed, and with a recipe that may be slightly easier to reproduce.


        Stefan

Re: Default of jit-lock-stealth-time

[Juanma Barranquero]

On 2/26/07, David Kastrup <dak@gnu.org> wrote:

> Ok, but working with many buffers is not something particularly
> forbidden.

Of course not. I won't even suggest my use of Emacs is typical at all.

             Juanma

Re: Default of jit-lock-stealth-time

[David Kastrup]

martin rudalics <rudalics@gmx.at> writes:

>> So it seems plain stupid to release it with a setting that will impact
>> and annoy users and cause a support burden after the release.
>
> While I agree with all of David's arguments (and even add the one
> that a buffer tail is stealthily refontified after every single
> buffer change followed by 16 secs idleness) I'm afraid that setting
> this to nil won't remove any support burden.  Many major modes still
> work better when they are allowed to fontify a buffer from beginning
> to end.

But stealth mode does not guarantee that at all.  It is completely
irreproducible whether or not some buffer has been completely
stealthified.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[Kim F. Storm]

David Kastrup <dak@gnu.org> writes:

> martin rudalics <rudalics@gmx.at> writes:
>
>>> So it seems plain stupid to release it with a setting that will impact
>>> and annoy users and cause a support burden after the release.
>>
>> While I agree with all of David's arguments (and even add the one
>> that a buffer tail is stealthily refontified after every single
>> buffer change followed by 16 secs idleness) I'm afraid that setting
>> this to nil won't remove any support burden.  Many major modes still
>> work better when they are allowed to fontify a buffer from beginning
>> to end.
>
> But stealth mode does not guarantee that at all.  It is completely
> irreproducible whether or not some buffer has been completely
> stealthified.

Agree!

I actually think Martin's argument speaks _against_ a non-nil default
value of jit-lock-stealth-time -- no major mode should ever rely on
such convoluted behavior.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Default of jit-lock-stealth-time

[martin rudalics]

 >> ... Many major modes still work better when they
 >>are allowed to fontify a buffer from beginning to end.
 >
 >
 > Stealth fontification doesn't do that anyway.

Hmmm... When I simply open a buffer it fontifies the buffer from the
beginning to the end ...

(defsubst jit-lock-stealth-chunk-start (around)
...
       (let* ((next (text-property-not-all around (point-max) 'fontified t))
...
	     (result (cond ((null start) next)
			   ((null next) start)
			   ((< (- around start) (- next around)) start)
			   (t next))))
	result))))

... next is always the first unfontified position in the buffer.

 >>Also, when a mode uses font-lock to assign `syntax-table' properties,
 >>there's a slight chance that an editing sequence that worked with
 >>stealth fontification turned on will not work any more when stealth
 >>fontification is turned off.  Having fontification assign syntax-table
 >>properties is certainly flawed by design but at the moment there's no
 >>other way to do that automatically.
 >
 >
 > When there are such bugs, stealth fontification doesn't prevent them.
 > It may hide them occasionally (non-deterministically), depending on the
 > specific editing steps.

That's exactly what I wanted to express.

 >>With other words, setting this to nil may cause some obscure bugs get
 >>reported (and hopefully fixed) earlier ...
 >
 >
 > Indeed, and with a recipe that may be slightly easier to reproduce.

We violently agree.

Re: Default of jit-lock-stealth-time

[martin rudalics]

>>But stealth mode does not guarantee that at all.  It is completely
>>irreproducible whether or not some buffer has been completely
>>stealthified.
> 
> 
> Agree!
> 
> I actually think Martin's argument speaks _against_ a non-nil default
> value of jit-lock-stealth-time -- no major mode should ever rely on
> such convoluted behavior.

Obviously.  But we should be prepared to get a few more bug reports when we
set this to nil.

Re: Default of jit-lock-stealth-time

[Richard Stallman]

    Yes, I know of at least 2 causes:
    - it's based on polling and may take a long time before the next poll
    - the stealth-fontification causes swapping, and so even if the poll comes
      immediately, pages need to be swapped back in before the user's command
      can complete.

I see.

The motive for stealth fontification was to avoid delays when moving to
another part of the file.  Is fontification now sufficiently fast
that such delays are not much of an annoyance any more?

Re: Default of jit-lock-stealth-time

[Richard Stallman]

    >     Regardless of the system load: the load may be low at the moment
    >     _exactly_ because the user terminated all processes that would disturb
    >     the interactivity of his current work, and I think it a mistake that
    >     Emacs considers the CPU fit for grabbing for its own purposes unless
    >     the user _explicitly_ asked for it.
    >
    > Stealth fontification is intended to stop whenever the user types a
    > character.  So why would it interfere with the "interactivity" of
    > Emacs commands?

    Who is talking about the interactivity of Emacs?

The first paragraph cited seems to suggest that.

Re: Default of jit-lock-stealth-time

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Date: Mon, 26 Feb 2007 08:35:41 +0100
> 
> Do we have a _single_ developer who still has it on

I use the default value.

> and has observed _benefits_ from its use?

I don't see any problems.  But I almost never use a laptop.

Re: Default of jit-lock-stealth-time

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Date: Mon, 26 Feb 2007 14:52:52 -0500
> Cc: emacs-devel@gnu.org
> 
> The motive for stealth fontification was to avoid delays when moving to
> another part of the file.  Is fontification now sufficiently fast
> that such delays are not much of an annoyance any more?

No, we are definitely not there yet, IMO.

Re: Default of jit-lock-stealth-time

[Stefan Monnier]

>     Yes, I know of at least 2 causes:
>     - it's based on polling and may take a long time before the next poll
>     - the stealth-fontification causes swapping, and so even if the poll comes
>       immediately, pages need to be swapped back in before the user's command
>       can complete.

> I see.

> The motive for stealth fontification was to avoid delays when moving to
> another part of the file.  Is fontification now sufficiently fast
> that such delays are not much of an annoyance any more?

In my opinion, we're definitely there.  I'd even say we've always
been there.  After all, the delay is often present even with stealth
fontification (in those cases where stealth fontification hasn't had
a chance to get here yet), so we always have to make sure that this delay is
acceptable anyway.


        Stefan

Re: Default of jit-lock-stealth-time

[Stefan Monnier]

>> Do we have a _single_ developer who still has it on
> I use the default value.
>> and has observed _benefits_ from its use?
> I don't see any problems.  But I almost never use a laptop.

This doesn't answer the question: have you *observed* *benefits*?


        Stefan

Re: Default of jit-lock-stealth-time

[Kim F. Storm]

martin rudalics <rudalics@gmx.at> writes:

>>>But stealth mode does not guarantee that at all.  It is completely
>>>irreproducible whether or not some buffer has been completely
>>>stealthified.
>>
>>
>> Agree!
>>
>> I actually think Martin's argument speaks _against_ a non-nil default
>> value of jit-lock-stealth-time -- no major mode should ever rely on
>> such convoluted behavior.
>
> Obviously.  But we should be prepared to get a few more bug reports when we
> set this to nil.

That's ok.  

If those bugs are really there - users will see them even when
jit-lock-stealth-time is non-nil.  

Getting bug reports for repeatable bugs is infinitely better than
reports about bugs that "happens from time to time, but I don't know
how to reproduce it".

It will just be much more difficult to debug them, if the bug is
timing dependent, rather than happening consistently.

Again -- a very good reason for the default being nil.

-- 
Kim F. Storm <storm@cua.dk> http://www.cua.dk

Re: Default of jit-lock-stealth-time

[Miles Bader]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
> This doesn't answer the question: have you *observed* *benefits*?

That's a rather silly question though, given that the whole purpose of
stealth fontification is to make fontification of large source files
more "seamless" -- i.e., unobservable...

I've never changed the default, and never noticed any delay or adverse
effects; occasionally it causes my system to not be idle when I thought
it should be idle (why should I care though?).

-Miles

-- 
"Most attacks seem to take place at night, during a rainstorm, uphill,
 where four map sheets join."   -- Anon. British Officer in WW I

Re: Default of jit-lock-stealth-time

[Stefan Monnier]

>> This doesn't answer the question: have you *observed* *benefits*?
> That's a rather silly question though, given that the whole purpose of
> stealth fontification is to make fontification of large source files
> more "seamless" -- i.e., unobservable...

Stealth fontification is not supposed to be unobservable.  Instead it's
supposed to make less observable the fact that fontification is done lazily.

So to observe that stealth fontification provides you with some benefit, you
have to turn it off and see if jit-lock behaves worse.

> I've never changed the default, and never noticed any delay or adverse
> effects; occasionally it causes my system to not be idle when I thought
> it should be idle (why should I care though?).

We know fairly well about the downsides and we know they don't affect
everybody all the time, so this is not the interesting part of
the discussion.


        Stefan

Re: Default of jit-lock-stealth-time

[Miles Bader]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
> So to observe that stealth fontification provides you with some benefit, you
> have to turn it off and see if jit-lock behaves worse.

Sure, but there's little incentive to do so, so the rather heavy-handed
way you jumped on Eli seems out of place.  The implication seemed to be
that if he didn't "observe some benefit", then there likely wasn't any
(which obviously isn't true).

> We know fairly well about the downsides and we know they don't affect
> everybody all the time, so this is not the interesting part of
> the discussion.

It serves to put the strident opposition to stealth fontification in
context, I think.

-Miles

-- 
Americans are broad-minded people.  They'll accept the fact that a person can
be an alcoholic, a dope fiend, a wife beater, and even a newspaperman, but if a
man doesn't drive, there is something wrong with him.  -- Art Buchwald

Re: Default of jit-lock-stealth-time

[Eli Zaretskii]

> Cc: David Kastrup <dak@gnu.org>,  emacs-devel@gnu.org
> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Mon, 26 Feb 2007 16:34:05 -0500
> 
> >> and has observed _benefits_ from its use?
> > I don't see any problems.  But I almost never use a laptop.
> 
> This doesn't answer the question: have you *observed* *benefits*?

It answers the question the best way I could.

Re: Default of jit-lock-stealth-time

[Stefan Monnier]

> Sure, but there's little incentive to do so, so the rather heavy-handed
> way you jumped on Eli seems out of place.

Sorry, I guess it sounded louder than I intended.

> The implication seemed to be that if he didn't "observe some benefit",
> then there likely wasn't any (which obviously isn't true).

We're just trying to find people who have observed benefits.  It looks
like Eli isn't one of them.  Maybe there are such people, but I expect
they're very rare:

I've played a good bit with jit-lock (while working on its code, on
syntax-ppss, and on various major mode's font-lock patterns), using
a relatively modest machine at the time (about 4 years old, but
top-of-the-line when new) running on an Emacs compiled with all known
runtime checks.  I've seen significant delays with jit-lock under some
specific circumstances, and in those circumstances jit-lock-stealth was able
to reduce the occurrence of those cases, but it never seemed enough: the
remaining cases were still too severe.  So in practice I expect every major
mode's font-lock rules to be tuned by the author to avoid such
circumstances, such that jit-lock is fast enough and jit-lock-stealth never
makes a noticeable difference.

>> We know fairly well about the downsides and we know they don't affect
>> everybody all the time, so this is not the interesting part of
>> the discussion.

> It serves to put the strident opposition to stealth fontification in
> context, I think.

Sure.  Jit-lock-stealth has been in wide use since Emacs-21.1 (and the years
of development before), so it's clear that its bad effects are
sufficiently mild.


        Stefan

Re: Default of jit-lock-stealth-time

[Richard Stallman]

More discussion of who has, and who has not, been inconvenienced by
stealth fontification is not really useful, because we already know
that it is a substantial inconvenience for some users.

The question now is whether it _avoids_ a substantial inconvenience
for a substantial fraction of users.

Re: Default of jit-lock-stealth-time

[Richard Stallman]

    > The motive for stealth fontification was to avoid delays when moving to
    > another part of the file.  Is fontification now sufficiently fast
    > that such delays are not much of an annoyance any more?

    No, we are definitely not there yet, IMO.

Would you please tell us more?  What sort of machine are you using,
and how bad are these delays?  We need to compare them with the delays
caused by stealth fontification.

Re: Default of jit-lock-stealth-time

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     Yes, I know of at least 2 causes:
>     - it's based on polling and may take a long time before the next poll
>     - the stealth-fontification causes swapping, and so even if the poll comes
>       immediately, pages need to be swapped back in before the user's command
>       can complete.
>
> I see.
>
> The motive for stealth fontification was to avoid delays when moving
> to another part of the file.

That is the motive for jit-font-lock, but stealth fontification
fontifies _other_ buffers.  While it may also hit the current buffer,
this is not a priority as far as I can tell.

> Is fontification now sufficiently fast that such delays are not much
> of an annoyance any more?

Well, if they _are_ an annoyance, then presumably because of
problematic font lock patterns, and then they will be an annoyance
also using stealth fontification (namely causing high CPU drain and
sluggish behavior), only a non-debuggable, non-deterministic and
non-reproduceable annoyance.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     >     Regardless of the system load: the load may be low at the
>     >     moment _exactly_ because the user terminated all processes
>     >     that would disturb the interactivity of his current work,
>     >     and I think it a mistake that Emacs considers the CPU fit
>     >     for grabbing for its own purposes unless the user
>     >     _explicitly_ asked for it.
>     >
>     > Stealth fontification is intended to stop whenever the user types a
>     > character.  So why would it interfere with the "interactivity" of
>     > Emacs commands?
>
>     Who is talking about the interactivity of Emacs?
>
> The first paragraph cited seems to suggest that.

One can actually accomplish work outside of Emacs, though the concept
may appear outlandish to some.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[David Kastrup]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Sure.  Jit-lock-stealth has been in wide use since Emacs-21.1 (and
> the years of development before), so it's clear that its bad effects
> are sufficiently mild.

Font-lock was turned off by default in versions 21.1 to 21.4 since its
performance impact was not well-behaved enough.  Quite a bit of fixes
went into font-lock in order to make it a defensible choice to enable
it by default.

People were _expecting_ downsides from font-locking, and we told them
so, and they had to enable it manually.

The situation is different for Emacs 22.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

> More discussion of who has, and who has not, been inconvenienced by
> stealth fontification is not really useful, because we already know
> that it is a substantial inconvenience for some users.

And let us not forget that this inconvience is unexpected, gives Emacs
a bad name (since it sucks up CPU even when idle) and is
non-debuggable, since timer functions run with quit turned off.  It is
completely impossible for the average affected user (heck, it was even
for me until Richard gave me a direct pointer to stealth fontification
after a bug report of mine) to find out which of the thousands of
settings inside of Emacs is responsible for the problem.

So it is a bad idea for a default setting.

> The question now is whether it _avoids_ a substantial inconvenience
> for a substantial fraction of users.

That would make it a good idea for a prominently accessible user
option.  I'd still think it a bad idea for a default setting.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

> More discussion of who has, and who has not, been inconvenienced by
> stealth fontification is not really useful, because we already know
> that it is a substantial inconvenience for some users.
>
> The question now is whether it _avoids_ a substantial inconvenience
> for a substantial fraction of users.

Well, one pointer would be if we had somebody who turned it off, only
to turn it back on later again because of getting inconvenienced.  It
does not appear like we have such a specimen around.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> CC: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> Date: Tue, 27 Feb 2007 02:39:15 -0500
> 
>     > The motive for stealth fontification was to avoid delays when moving to
>     > another part of the file.  Is fontification now sufficiently fast
>     > that such delays are not much of an annoyance any more?
> 
>     No, we are definitely not there yet, IMO.
> 
> Would you please tell us more?  What sort of machine are you using,
> and how bad are these delays?  We need to compare them with the delays
> caused by stealth fontification.

I was talking from memory.  I will make some measurements in a few
days and tell the results.

Re: Default of jit-lock-stealth-time

[Eli Zaretskii]

> From: David Kastrup <dak@gnu.org>
> Date: Tue, 27 Feb 2007 09:17:10 +0100
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> >
> > The motive for stealth fontification was to avoid delays when moving
> > to another part of the file.
> 
> That is the motive for jit-font-lock, but stealth fontification
> fontifies _other_ buffers.  While it may also hit the current buffer,
> this is not a priority as far as I can tell.

The current buffer does not need to be a priority because no one says
that the user works only (or mainly) in a single buffer.  I'm as
likely to go to the end of another buffer as I am in the one where I
do my edits.

Re: Default of jit-lock-stealth-time

[Eli Zaretskii]

> Cc: Miles Bader <miles@gnu.org>,  Eli Zaretskii <eliz@gnu.org>,
> 	  emacs-devel@gnu.org
> From: David Kastrup <dak@gnu.org>
> Date: Tue, 27 Feb 2007 09:23:59 +0100
> 
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> 
> > Sure.  Jit-lock-stealth has been in wide use since Emacs-21.1 (and
> > the years of development before), so it's clear that its bad effects
> > are sufficiently mild.
> 
> Font-lock was turned off by default in versions 21.1 to 21.4 since its
> performance impact was not well-behaved enough.

Inaccurate.  First, font-lock was turned off by default in _all_
versions of Emacs since its introduction (in v19.x, AFAIR).  More
importantly, a large majority of users turned it on, so that
jit-lock-stealth _was_, indeed, widely used since v21.1, where
jit-lock was introduced.

> Quite a bit of fixes went into font-lock in order to make it a
> defensible choice to enable it by default.

Inaccurate.  Improvements to jit-lock notwithstanding, the main reason
for turning it on was that we decided the garden-variety CPUs are
nowadays fast enough to fontify buffers without annoyance to most
users.

Re: Default of jit-lock-stealth-time

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

> Inaccurate.  First, font-lock was turned off by default in _all_
> versions of Emacs since its introduction (in v19.x, AFAIR).  More
> importantly, a large majority of users turned it on,

I'd like to see some statistics supporting that.  A large majority of
users does not touch defaults, according to my experience.

> so that jit-lock-stealth _was_, indeed, widely used since v21.1,
> where jit-lock was introduced.

Anyway, when people turned it on, they expected to have drawbacks
(which was why it was not on by default).  But people now get font
lock mode without asking for it.

>> Quite a bit of fixes went into font-lock in order to make it a
>> defensible choice to enable it by default.
>
> Inaccurate.  Improvements to jit-lock notwithstanding, the main
> reason for turning it on was that we decided the garden-variety CPUs
> are nowadays fast enough to fontify buffers without annoyance to
> most users.

Our memories differ then.  I remember distinctly that several
`pathological' cases were cleared up before Richard agreed to enable
font locking by default.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: David Kastrup <dak@gnu.org>
> Date: Tue, 27 Feb 2007 21:59:12 +0100
> 
> > Inaccurate.  First, font-lock was turned off by default in _all_
> > versions of Emacs since its introduction (in v19.x, AFAIR).  More
> > importantly, a large majority of users turned it on,
> 
> I'd like to see some statistics supporting that.  A large majority of
> users does not touch defaults, according to my experience.

One evidence I can offer is the number of discussions between
developers where different people tried many times to convince Richard
to turn font-lock on by default.

Re: Default of jit-lock-stealth-time

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: emacs-devel@gnu.org
>> From: David Kastrup <dak@gnu.org>
>> Date: Tue, 27 Feb 2007 21:59:12 +0100
>> 
>> > Inaccurate.  First, font-lock was turned off by default in _all_
>> > versions of Emacs since its introduction (in v19.x, AFAIR).  More
>> > importantly, a large majority of users turned it on,
>> 
>> I'd like to see some statistics supporting that.  A large majority
>> of users does not touch defaults, according to my experience.
>
> One evidence I can offer is the number of discussions between
> developers where different people tried many times to convince
> Richard to turn font-lock on by default.

Where would have been the point in having it turned on by default if
"a large majority of users" would turn it on manually, anyway?

I see this rather as evidence that people did _not_ turn it on by
default, and that Emacs' popularity suffered because of that.

So I come to quite opposite conclusions given the same, indirect data.
And I don't consider my conclusions at all contrived.  Which would
warrant more direct data before one comes to the conclusion that it is
ok to inflict users with the bad results of stealth fontification.

Personally, I find it a _large_ warning sign that a number of
developers said they turned stealth fontification off because of bad
effects, and not a _single_ one reported turning it back on again
because of opposing bad effects.

So we have _several_ reports of stealth fontification being deemed
unacceptable to experienced Emacs users, and not a single report of
having it turned off being deemed unacceptable.

In addition, it is really _hard_ to find the culprit for the bad
effects of stealth fontification.  I have not been able to figure this
out myself.  And even though it is hard, several other developers
_independently_ figured it out and disabled it.

I think that calls for _very_ good evidence for large negative effects
if it is turned off by default, and I have up to now (including this
mail I reply to) failed to see anything I would consider to be even
remotely close.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[Stefan Monnier]

>> Inaccurate.  First, font-lock was turned off by default in _all_
>> versions of Emacs since its introduction (in v19.x, AFAIR).  More
>> importantly, a large majority of users turned it on,
> I'd like to see some statistics supporting that.  A large majority of
> users does not touch defaults, according to my experience.

My guess is that users who didn't change the default ended up moving to some
other editor, while those who did stick with Emacs ended up turning on
font-lock.  But it doesn't matter.  All that matters is that there is a lot
of experience with font-lock, so it's clear that it wasn't/isn't terribly
badly behaved on average.


        Stefan

Re: Default of jit-lock-stealth-time

[Richard Stallman]

    > The motive for stealth fontification was to avoid delays when moving
    > to another part of the file.

    That is the motive for jit-font-lock, but stealth fontification
    fontifies _other_ buffers.

The reason is still to avoid the need to fontify
when you move to a part of one of those other buffers
that hadn't been displayed in this session.

The question is, is the time saved by that enough to be worth caring
about?

Re: Default of jit-lock-stealth-time

[Richard Stallman]

    Well, if they _are_ an annoyance, then presumably because of
    problematic font lock patterns, and then they will be an annoyance
    also using stealth fontification (namely causing high CPU drain and
    sluggish behavior), only a non-debuggable, non-deterministic and
    non-reproduceable annoyance.

Does everyone agree that substantial delays result only from
problematic font lock patterns?  If so, I agree your conclusion
follows.

Re: Default of jit-lock-stealth-time

[Richard Stallman]

    > The question now is whether it _avoids_ a substantial inconvenience
    > for a substantial fraction of users.

    Well, one pointer would be if we had somebody who turned it off, only
    to turn it back on later again because of getting inconvenienced.  It
    does not appear like we have such a specimen around.

You may be right.  I'm asking anyone who DOES finr stealth
fontification useful to speak up.  Eli Z seemed to say so,
but has not given details.

We will see what he has to say.

Re: Default of jit-lock-stealth-time

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: David Kastrup <dak@gnu.org>
> Date: Tue, 27 Feb 2007 23:02:49 +0100
> >
> > One evidence I can offer is the number of discussions between
> > developers where different people tried many times to convince
> > Richard to turn font-lock on by default.
> 
> Where would have been the point in having it turned on by default if
> "a large majority of users" would turn it on manually, anyway?

To lower the flood of questions posted by newbies asking why they see
no syntax highlight, _before_ they turn it on.

> I see this rather as evidence that people did _not_ turn it on by
> default, and that Emacs' popularity suffered because of that.

Popularity did suffer, but because newbies couldn't imagine Emacs
didn't have syntax highlight.  As soon as they learned Emacs did have
it, they turned it on.

Re: Default of jit-lock-stealth-time

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: emacs-devel@gnu.org
>> From: David Kastrup <dak@gnu.org>
>> Date: Tue, 27 Feb 2007 23:02:49 +0100
>> >
>> > One evidence I can offer is the number of discussions between
>> > developers where different people tried many times to convince
>> > Richard to turn font-lock on by default.
>> 
>> Where would have been the point in having it turned on by default if
>> "a large majority of users" would turn it on manually, anyway?
>
> To lower the flood of questions posted by newbies asking why they see
> no syntax highlight, _before_ they turn it on.

I can't remember such a flood.  The discussion was very much between
seasoned Emacs users as far as I remember.

>> I see this rather as evidence that people did _not_ turn it on by
>> default, and that Emacs' popularity suffered because of that.
>
> Popularity did suffer, but because newbies couldn't imagine Emacs
> didn't have syntax highlight.  As soon as they learned Emacs did
> have it, they turned it on.

There would still be that flood around to look at since newbies tend
to install stable versions of Emacs.  I still see quite a few
questions about Emacs 21.4 on the help lists.  I can't remember font
lock mode being a topic in that context lately, however.

So I quite definitely disagree with the conclusions you draw from
font-lock history.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[Richard Stallman]

    I was talking from memory.  I will make some measurements in a few
    days and tell the results.

I am not sure measurements are relevant here.  The question is
whether the delays avoided by stealth fontification bother you.
If you don't notice them, they don't bother you.

Do they bother you?

Re: Default of jit-lock-stealth-time

[Richard Stallman]

    One evidence I can offer is the number of discussions between
    developers where different people tried many times to convince Richard
    to turn font-lock on by default.

You can't judge anything about most users by looking at the habits
of developers.

Re: Default of jit-lock-stealth-time

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: David Kastrup <dak@gnu.org>
> Date: Wed, 28 Feb 2007 07:35:31 +0100
> >> 
> >> Where would have been the point in having it turned on by default if
> >> "a large majority of users" would turn it on manually, anyway?
> >
> > To lower the flood of questions posted by newbies asking why they see
> > no syntax highlight, _before_ they turn it on.
> 
> I can't remember such a flood.

It was on gnu.emacs.help, not here.

Re: Default of jit-lock-stealth-time

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> CC: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> Date: Wed, 28 Feb 2007 02:27:41 -0500
> 
>     I was talking from memory.  I will make some measurements in a few
>     days and tell the results.
> 
> I am not sure measurements are relevant here.

AFAICT, you asked for them explicitly:

    Would you please tell us more?  What sort of machine are you using,
    and how bad are these delays?  We need to compare them with the delays
    caused by stealth fontification.

> The question is whether the delays avoided by stealth fontification
> bother you.  If you don't notice them, they don't bother you.
> 
> Do they bother you?

My recollection is that they do annoy, but I will need to time things
to give more accurate information instead of hand-waving.  However, if
you don't feel you need to see more or less precise numbers, feel free
to tell me: I have better things to do with my limited free time than
gather measurements no one needs.

Re: Default of jit-lock-stealth-time

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> CC: dak@gnu.org, emacs-devel@gnu.org
> Date: Wed, 28 Feb 2007 02:27:45 -0500
> 
>     One evidence I can offer is the number of discussions between
>     developers where different people tried many times to convince Richard
>     to turn font-lock on by default.
> 
> You can't judge anything about most users by looking at the habits
> of developers.

Habits of developers are irrelevant here: the developers were talking
based on habits of mere mortals, not of their own.

Anyway, I just described the past, and we cannot change the past.  So
it's not useful to continue arguing about what did or didn't happen in
the past, and why.

Re: Default of jit-lock-stealth-time

[Richard Stallman]

    My recollection is that they do annoy, but I will need to time things
    to give more accurate information instead of hand-waving.

I don't need _more accurate_ information, just confirmation.  I would
like you to try turning off stealth fontification and see if you are
frequently annoyed by the need to fontify on the fly.  It does not
need to be a quantitative measurement.

Re: Default of jit-lock-stealth-time

[Stefan Monnier]

>     My recollection is that they do annoy, but I will need to time things
>     to give more accurate information instead of hand-waving.

> I don't need _more accurate_ information, just confirmation.  I would
> like you to try turning off stealth fontification and see if you are
> frequently annoyed by the need to fontify on the fly.  It does not
> need to be a quantitative measurement.

Actually, a better test would be to put

   (if (zerop (random 1)) (setq jit-lock-stealth-time nil))

in your .emacs.  And then check the value of jit-lock-stealth-time every
time you notice a delay (after which, you'll want to restart jit-lock,
which basically means restart Emacs).

This should bring the experiment a bit closer to the "double blind"
gold standard.


        Stefan

Re: Default of jit-lock-stealth-time

[David Kastrup]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>     My recollection is that they do annoy, but I will need to time things
>>     to give more accurate information instead of hand-waving.
>
>> I don't need _more accurate_ information, just confirmation.  I would
>> like you to try turning off stealth fontification and see if you are
>> frequently annoyed by the need to fontify on the fly.  It does not
>> need to be a quantitative measurement.
>
> Actually, a better test would be to put
>
>    (if (zerop (random 1)) (setq jit-lock-stealth-time nil))
>
> in your .emacs.

Do we have a JOKES files for inadvertent jokes?

> And then check the value of jit-lock-stealth-time every time you
> notice a delay (after which, you'll want to restart jit-lock, which
> basically means restart Emacs).
>
> This should bring the experiment a bit closer to the "double blind"
> gold standard.

"double blind" only if he does not notice that (zerop (random 1)) is
always t.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[Lennart Borgman (gmail)]

David Kastrup wrote:

> "double blind" only if he does not notice that (zerop (random 1)) is
> always t.

Ah, you spoiled the double blindness. The one who is using it is not 
supposed to know.

Re: Default of jit-lock-stealth-time

[Stuart D. Herring]

> Actually, a better test would be to put
>
>    (if (zerop (random 1)) (setq jit-lock-stealth-time nil))
>
> in your .emacs.  And then check the value of jit-lock-stealth-time every
> time you notice a delay (after which, you'll want to restart jit-lock,
> which basically means restart Emacs).
>
> This should bring the experiment a bit closer to the "double blind"
> gold standard.

Remember to seed your random number generator first!  (random t) is your
friend.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: Default of jit-lock-stealth-time

[Lennart Borgman (gmail)]

Stuart D. Herring wrote:
>> Actually, a better test would be to put
>>
>>    (if (zerop (random 1)) (setq jit-lock-stealth-time nil))
>>
>> in your .emacs.  And then check the value of jit-lock-stealth-time every
>> time you notice a delay (after which, you'll want to restart jit-lock,
>> which basically means restart Emacs).
>>
>> This should bring the experiment a bit closer to the "double blind"
>> gold standard.
> 
> Remember to seed your random number generator first!  (random t) is your
> friend.


It was not necessary in this particular case but would perhaps had made 
obfuscation better. ;-)

Re: Default of jit-lock-stealth-time

[Stuart D. Herring]

>> Remember to seed your random number generator first!  (random t) is your
>> friend.
>
> It was not necessary in this particular case but would perhaps had made
> obfuscation better. ;-)

Without it, you'll always get the -same- 0 every time you start Emacs. 
That's even less random than getting different 0s.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: Default of jit-lock-stealth-time

[David Kastrup]

"Stuart D. Herring" <herring@lanl.gov> writes:

>>> Remember to seed your random number generator first!  (random t) is your
>>> friend.
>>
>> It was not necessary in this particular case but would perhaps had made
>> obfuscation better. ;-)
>
> Without it, you'll always get the -same- 0 every time you start Emacs. 
> That's even less random than getting different 0s.

Oh great.  Binary sectarianism.  Why am I not surprised that the zeros
appear affected foremost?  Probably because of their lack of respect
of authority.  Whoever heard of a significant leading zero?

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[Juanma Barranquero]

On 3/1/07, David Kastrup <dak@gnu.org> wrote:

> Whoever heard of a significant leading zero?

You've never worked for a phone company, have you? :-)

             Juanma

Re: Default of jit-lock-stealth-time

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> CC: emacs-devel@gnu.org
> Date: Thu, 01 Mar 2007 03:14:08 -0500
> 
>     My recollection is that they do annoy, but I will need to time things
>     to give more accurate information instead of hand-waving.
> 
> I don't need _more accurate_ information, just confirmation.  I would
> like you to try turning off stealth fontification and see if you are
> frequently annoyed by the need to fontify on the fly.

I mostly use very fast (3 Ghz) machines nowadays, which are not easily
forced into annoying delays.  I could try to find a slower machine,
but I need to know what is the clock speed we consider as
``reference'' for such investigations.  That is, with what slow clock
speeds we would still like Emacs to be reasonably responsive?

On a 3-GHz machine, with jit-lock-stealth-time set to nil, I measure a
consistent 5-10% increase in CPU time when paging up thru sufficiently
long Texinfo documents wrt to an already fontified buffer (18%-25%
percent the first time I page up, vs 10%-16% on subsequent attempts).
By contrast, with the default setting of jit-lock-stealth-time I see
only 1-3% of CPU being used while stealth fontification runs in the
background, which is barely distinguishable from a totally idle
machine.

Re: Default of jit-lock-stealth-time

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

> On a 3-GHz machine, with jit-lock-stealth-time set to nil, I measure a
> consistent 5-10% increase in CPU time when paging up thru sufficiently
> long Texinfo documents wrt to an already fontified buffer (18%-25%
> percent the first time I page up, vs 10%-16% on subsequent attempts).
> By contrast, with the default setting of jit-lock-stealth-time I see
> only 1-3% of CPU being used while stealth fontification runs in the
> background, which is barely distinguishable from a totally idle
> machine.

Well, I have a 1.2GHz laptop (but the results were similar for my
previous 600MHz box).

Apart from "pathological" buffers, paging through a file will deliver
font locking fast enough to follow the user action, without causing
the laptop to use the fan.

In contrast, left alone to jit-lock-stealth-time=16, Emacs will
eventually turn to eating 100% of CPU time (not 1-3%), causing the fan
to go on and power drainage to occur.

That does not mean that the problematic files will page through
without noticeable delay when I go through them by hand: they tend to
react sluggishly to editing.  But that is a minor inconvenience
compared to the computer going unresponsible at full CPU power
frequently.

I would consider stealth fontification completely useless as long as
the computer can keep up with the user, which appears to be the case
for your usage.  In that case, investing the power when it is actually
needed seems by far the sanest choice.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[Eli Zaretskii]

> Cc: rms@gnu.org,  emacs-devel@gnu.org
> From: David Kastrup <dak@gnu.org>
> Date: Sat, 03 Mar 2007 14:36:28 +0100
> 
> Apart from "pathological" buffers, paging through a file will deliver
> font locking fast enough to follow the user action

Where ``fast enough'' means what, precisely?

> In contrast, left alone to jit-lock-stealth-time=16, Emacs will
> eventually turn to eating 100% of CPU time (not 1-3%)

How can this happen?  JIT-lock is supposed to fontify at most
jit-lock-chunk-size characters (500 by default), and then refrain from
fontification for at least jit-lock-stealth-nice seconds (0.5 by
default).  How can Emacs take 100% of CPU with these defaults?

Maybe there's a bug?  How much time do you need to wait until Emacs
starts using 100% of CPU on your machine?  I didn't see that, but
maybe I didn't wait long enough.  Also, did you try looking at the
percentage of CPU taken by each application, when the CPU consumption
hits 100%, and if you did, was Emacs indeed consuming 100% or
thereabouts at that time?

> That does not mean that the problematic files will page through
> without noticeable delay when I go through them by hand: they tend to
> react sluggishly to editing.

These delays and sluggishness were what I meant when I talked about
annoyances.  I don't like them.

> I would consider stealth fontification completely useless as long as
> the computer can keep up with the user, which appears to be the case
> for your usage.  In that case, investing the power when it is actually
> needed seems by far the sanest choice.

David, your opinion was heard, loud and clear, several times already.
There's no need to reiterate it.  Doing so just wastes bandwidth.

For my part, I can say that investing a small fraction of CPU power
when none is used is by far wiser than annoying the user with
redisplay delays, if those are noticeable.

Re: Default of jit-lock-stealth-time

[Daniel Brockman]

It would be cool if Emacs could detect whether it's running
on battery power and in that case go easy on the CPU.

-- 
Daniel Brockman <daniel@brockman.se>

Re: Default of jit-lock-stealth-time

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: rms@gnu.org,  emacs-devel@gnu.org
>> From: David Kastrup <dak@gnu.org>
>> Date: Sat, 03 Mar 2007 14:36:28 +0100
>> 
>> Apart from "pathological" buffers, paging through a file will deliver
>> font locking fast enough to follow the user action
>
> Where ``fast enough'' means what, precisely?

The limiting factor is the speed with which the user types scrolling
commands.

>> In contrast, left alone to jit-lock-stealth-time=16, Emacs will
>> eventually turn to eating 100% of CPU time (not 1-3%)
>
> How can this happen?  JIT-lock is supposed to fontify at most
> jit-lock-chunk-size characters (500 by default), and then refrain
> from fontification for at least jit-lock-stealth-nice seconds (0.5
> by default).  How can Emacs take 100% of CPU with these defaults?

It does.  If 0.5 seconds is small compared to the time it takes for
the fontification, CPU load will be close to 100%.  Granted, I get
more like 80% on an extended average, but it certainly is enough to
drain the batteries, keep the fan running and render the system
(inside and outside of Emacs) sluggish.

> Maybe there's a bug?

Sigh.  The problem is that _any_ bad font lock pattern in _any_ buffer
could be responsible for the action, and inside of timers, quit is
inhibited.  If there is a bug, stealth-lock makes its effects appear
regardless of what buffer one is editing, while at the same time very
effectively shielding the bug from being debuggable or traceable to a
cause.

> How much time do you need to wait until Emacs starts using 100% of
> CPU on your machine?  I didn't see that, but maybe I didn't wait
> long enough.

Basically, it starts some time after the 16 seconds.

> Also, did you try looking at the percentage of CPU taken by each
> application, when the CPU consumption hits 100%, and if you did, was
> Emacs indeed consuming 100% or thereabouts at that time?

Yes, Emacs was consuming between 70% and 100% at that time.
Repeatably.  This annoyed me for _months_ without being able to trace
it to a cause (and yes, I tried my hand with debug-on-quit and similar
things: stealth fontification "nicely" works around all that).  Only
after a bug report and Richard's suggestion to check out stealth
fontification was I able to get rid of this nonsense.

After months.  And several other Emacs developers have reported to
have used this setting for similar reasons.

>> That does not mean that the problematic files will page through
>> without noticeable delay when I go through them by hand: they tend
>> to react sluggishly to editing.
>
> These delays and sluggishness were what I meant when I talked about
> annoyances.  I don't like them.

Well, guess what: I don't like them when I am editing _unrelated_
buffer or not editing anything at _all_.  Stealth fontification
acerbates the problem by spreading it around your whole desktop so
that it becomes almost impossible to pinpoint it.  But it _does_ _not_
fix the problematic cases.  It just makes it impossible to attribute
them to their cause.

>> I would consider stealth fontification completely useless as long
>> as the computer can keep up with the user, which appears to be the
>> case for your usage.  In that case, investing the power when it is
>> actually needed seems by far the sanest choice.
>
> David, your opinion was heard, loud and clear, several times
> already.  There's no need to reiterate it.  Doing so just wastes
> bandwidth.

I was commenting on your findings.  Do you mean to say that discussion
of your findings is to be prohibited if that could point to the same
conclusion?

> For my part, I can say that investing a small fraction of CPU power
> when none is used is by far wiser than annoying the user with
> redisplay delays, if those are noticeable.

As I said: they are only noticeable when patterns are at work that
make stealth fontification behave _untolerably bad_, without being
debuggable.

The CPU power is not just invested when "none is used", it drains the
power of the system, it suggests that Emacs is the _only_ application
worthy to use CPU power at any time, anyway, and so it is fine for it
to grab it without being asked.  And even if we ignore all that and
make believe that the fairy world in which laptop batteries and any
application except Emacs itself don't count is reality, you still
gloss over the fact that stealth fontification does not even work
transparently within Emacs itself: Emacs reacts sluggishly and the
cursor blink gets erratical (or disappears altogether) when this
happens, and stealth fontification makes it _impossible_ to debug the
cause.

Sure, "maybe there is a bug".  But such bugs can be in the font lock
patterns of any mode, and there is no point in making them
nondebuggable and forcing their effects upon the users, just because
they are "only bugs".

Apart from which I am still of the opinion that an idle Emacs should
in its default setting _not_ consume any CPU power when no events
occur.

An application that turns into a power and CPU hog when it goes
unsupervised is _not_ going to win any trust contests.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[David Kastrup]

Daniel Brockman <daniel@brockman.se> writes:

> It would be cool if Emacs could detect whether it's running
> on battery power and in that case go easy on the CPU.

You are again assuming that Emacs is the only application worth
grabbing CPU power.

Emacs is intended to be started once and then kept running.  If the
only way to keep it from hogging the CPU is to exit it, this makes
Emacs unsuitable as an editor.

Emacs is an editor, an interactive application: the moments where a
user finds it acceptable for it to consume CPU are those when she it
typing into it.

Stealth fontification may be nice as a user option (though I have
still to see a single case where its effects are not worse than what
it is supposed to cure), but enabling it by default is sheer madness.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[Eli Zaretskii]

> Cc: rms@gnu.org,  emacs-devel@gnu.org
> From: David Kastrup <dak@gnu.org>
> Date: Sat, 03 Mar 2007 16:37:33 +0100
> >> 
> >> Apart from "pathological" buffers, paging through a file will deliver
> >> font locking fast enough to follow the user action
> >
> > Where ``fast enough'' means what, precisely?
> 
> The limiting factor is the speed with which the user types scrolling
> commands.

Does it keep up with the keyboard's autorepeat, for example?

> >> In contrast, left alone to jit-lock-stealth-time=16, Emacs will
> >> eventually turn to eating 100% of CPU time (not 1-3%)
> >
> > How can this happen?  JIT-lock is supposed to fontify at most
> > jit-lock-chunk-size characters (500 by default), and then refrain
> > from fontification for at least jit-lock-stealth-nice seconds (0.5
> > by default).  How can Emacs take 100% of CPU with these defaults?
> 
> It does.  If 0.5 seconds is small compared to the time it takes for
> the fontification, CPU load will be close to 100%.  Granted, I get
> more like 80% on an extended average, but it certainly is enough to
> drain the batteries, keep the fan running and render the system
> (inside and outside of Emacs) sluggish.

It sounds strange that fontification of 500 characters should take
time much longer than 0.5 seconds.  What happens if you increase the
value of jit-lock-stealth-nice, does the load go down proportionally?

> > David, your opinion was heard, loud and clear, several times
> > already.  There's no need to reiterate it.  Doing so just wastes
> > bandwidth.
> 
> I was commenting on your findings.  Do you mean to say that discussion
> of your findings is to be prohibited if that could point to the same
> conclusion?

I didn't mean anything except what I said.  And please cool down, we
are discussing technical issues that aren't supposed to get anyone
worked out.

> > For my part, I can say that investing a small fraction of CPU power
> > when none is used is by far wiser than annoying the user with
> > redisplay delays, if those are noticeable.
> 
> As I said: they are only noticeable when patterns are at work that
> make stealth fontification behave _untolerably bad_, without being
> debuggable.

Then perhaps fixing those bad patterns is a better way of dealing with
the problem.  Can you tell what files exhibit this bad behavior?  I'd
like to see what happens with them on my machines.

> The CPU power is not just invested when "none is used", it drains the
> power of the system, it suggests that Emacs is the _only_ application
> worthy to use CPU power at any time, anyway, and so it is fine for it
> to grab it without being asked.

We have customizations to take care of significant drain of power.  We
were discussing the defaults.  By default, I think Emacs should use up
a small fraction of CPU resources when it fontifies stealthily.  That
is what the default values try to accomplish.

Re: Default of jit-lock-stealth-time

[Richard Stallman]

    I mostly use very fast (3 Ghz) machines nowadays, which are not easily
    forced into annoying delays.  I could try to find a slower machine,
    but I need to know what is the clock speed we consider as
    ``reference'' for such investigations.  That is, with what slow clock
    speeds we would still like Emacs to be reasonably responsive?

I don't know.  I have not kept track of things like that for 15 years.

But I think that if NONE of the developers now finds this annoying,
that problem will be rare enough that it doesn't count for much.

So we can set jit-lock-stealth-time to nil.

Would someone please do that?

Re: Default of jit-lock-stealth-time

[Richard Stallman]

    It would be cool if Emacs could detect whether it's running
    on battery power and in that case go easy on the CPU.

It would be nice, but we shouldn't try to do that now.
Please suggest it again 2 months after Emacs 22 is released.

Re: Default of jit-lock-stealth-time

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: rms@gnu.org,  emacs-devel@gnu.org
>> From: David Kastrup <dak@gnu.org>
>> Date: Sat, 03 Mar 2007 16:37:33 +0100
>> >> 
>> >> Apart from "pathological" buffers, paging through a file will deliver
>> >> font locking fast enough to follow the user action
>> >
>> > Where ``fast enough'' means what, precisely?
>> 
>> The limiting factor is the speed with which the user types scrolling
>> commands.
>
> Does it keep up with the keyboard's autorepeat, for example?

I find that under X11 on my machine, processes tend not to be
decoupled to such a high degree that the autorepeat keeps producing
key presses when they are no longer consumed, so there are no
accumulative bad side effects that immediately accompany the point of
"can't keep up".  For me, it tends to keep up.  Note that I changed
the default just about at the time where I started my rant, but then,
Emacs never really got finished with his high-power stealth
fontification jobs, anyway (which is the reason why they are extremely
annoying), and so could not save me much time.

>> >> In contrast, left alone to jit-lock-stealth-time=16, Emacs will
>> >> eventually turn to eating 100% of CPU time (not 1-3%)
>> >
>> > How can this happen?  JIT-lock is supposed to fontify at most
>> > jit-lock-chunk-size characters (500 by default), and then refrain
>> > from fontification for at least jit-lock-stealth-nice seconds
>> > (0.5 by default).  How can Emacs take 100% of CPU with these
>> > defaults?
>> 
>> It does.  If 0.5 seconds is small compared to the time it takes for
>> the fontification, CPU load will be close to 100%.  Granted, I get
>> more like 80% on an extended average, but it certainly is enough to
>> drain the batteries, keep the fan running and render the system
>> (inside and outside of Emacs) sluggish.
>
> It sounds strange that fontification of 500 characters should take
> time much longer than 0.5 seconds.  What happens if you increase the
> value of jit-lock-stealth-nice, does the load go down
> proportionally?

Since the load numbers are not absolutely stable, this is an
experimentation in futility.

>> > David, your opinion was heard, loud and clear, several times
>> > already.  There's no need to reiterate it.  Doing so just wastes
>> > bandwidth.
>> 
>> I was commenting on your findings.  Do you mean to say that
>> discussion of your findings is to be prohibited if that could point
>> to the same conclusion?
>
> I didn't mean anything except what I said.

Fine, then you were wrong.  I did offer my opinion on your findings
previously, so it could not have been heard before.

> And please cool down, we are discussing technical issues that aren't
> supposed to get anyone worked out.

Well, they _are_ getting my computer worked out, to say the least.

>> As I said: they are only noticeable when patterns are at work that
>> make stealth fontification behave _untolerably bad_, without being
>> debuggable.
>
> Then perhaps fixing those bad patterns is a better way of dealing
> with the problem.

What about "without being debuggable" is hard to understand?  Setting
jit-lock-stealth-verbose to t makes it likely that several files are
affected and in several modes.  One is texbook.tex (but by far not the
only).  But it does not behave similarly bad when viewed directly, and
there is likely some interaction with AUCTeX modes or highlighting
involved, as well.

The problem is that it is _not_ _debuggable_ and affects operations in
_all_ buffers in a non-reproducible way due to stealth fontification.

We really don't want to have a setting that spreads problems with one
buffer/major mode/font lock pattern across _all_ of Emacs' operation.

It makes it close to impossible to ever pinpoint the problem and get
rid of it.

> Can you tell what files exhibit this bad behavior?  I'd like to see
> what happens with them on my machines.

As I said: it depends on several circumstances (like using desktop.el,
so files get loaded without getting displayed, and using several modes
and settings in several combinations).  And it is not reproducible.

>> The CPU power is not just invested when "none is used", it drains
>> the power of the system, it suggests that Emacs is the _only_
>> application worthy to use CPU power at any time, anyway, and so it
>> is fine for it to grab it without being asked.
>
> We have customizations to take care of significant drain of power.
> We were discussing the defaults.  By default, I think Emacs should
> use up a small fraction of CPU resources when it fontifies
> stealthily.

I disagree.  By default, an idle Emacs session should be idle.  That
is what people expect from an interactive application, and certainly
from an editor.  Without a _very_ good reason, no deviation from that
policy makes sense.  One has to be aware that Emacs is a _large_
application.  If it is sitting idle, the normal behavior under memory
pressure is that it gets mostly paged out in order not to disturb
other processes, and that is sensible behavior.  Stealth
fontification, however, will cause almost _everything_ to get paged in
again since it touches every buffer and every major mode and has to
garbage collect in the process as well.

This is _not_ what an idle interactive program should do, in
particularly not one as large as Emacs.

So even if stealth fontification worked always as intended and no
problematic font lock patterns existed on the world, it would _still_
be a bad idea to enable it without explicit request from the user.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     I mostly use very fast (3 Ghz) machines nowadays, which are not easily
>     forced into annoying delays.  I could try to find a slower machine,
>     but I need to know what is the clock speed we consider as
>     ``reference'' for such investigations.  That is, with what slow clock
>     speeds we would still like Emacs to be reasonably responsive?
>
> I don't know.  I have not kept track of things like that for 15 years.
>
> But I think that if NONE of the developers now finds this annoying,
> that problem will be rare enough that it doesn't count for much.
>
> So we can set jit-lock-stealth-time to nil.
>
> Would someone please do that?

I have done that and changed the NEWS entry accordingly, too.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: rms@gnu.org,  emacs-devel@gnu.org
>> From: David Kastrup <dak@gnu.org>
>> Date: Sat, 03 Mar 2007 14:36:28 +0100
>> 
>> Apart from "pathological" buffers, paging through a file will deliver
>> font locking fast enough to follow the user action
>
> Where ``fast enough'' means what, precisely?
>
>> In contrast, left alone to jit-lock-stealth-time=16, Emacs will
>> eventually turn to eating 100% of CPU time (not 1-3%)
>
> How can this happen?  JIT-lock is supposed to fontify at most
> jit-lock-chunk-size characters (500 by default), and then refrain from
> fontification for at least jit-lock-stealth-nice seconds (0.5 by
> default).  How can Emacs take 100% of CPU with these defaults?

Actually, it is more around 75% of CPU time according to "top".
Enough to cause a power drain, keep Emacs mapped to memory, and
obviously in long enough bursts to keep the computer in general and/or
Emacs from being responsive.

> Maybe there's a bug?  How much time do you need to wait until Emacs
> starts using 100% of CPU on your machine?  I didn't see that, but
> maybe I didn't wait long enough.  Also, did you try looking at the
> percentage of CPU taken by each application, when the CPU
> consumption hits 100%, and if you did, was Emacs indeed consuming
> 100% or thereabouts at that time?

Since I just got hit again (I deleted my customization of
jit-lock-stealth-time after checking in the change, but did not
actually recompile here), I tried changing jit-lock-stealth-nice.
Changing it from the default of 0.5 to a value of 3 will cause the CPU
load that top reports Emacs to be taking (when it goes wild) to drop
from about 75% (for 0.5) to about 40% (for 3).  So a totally wild
interpolation would come to the conclusion that the "bad"
fontification attempts take about 75%*0.5sec/25% = 1.5sec or
40%*3sec/60%=2sec each, which is similar enough for such a rough
estimate.

Maybe stealth lock should keep track of the actual time it spends on
fontification and calculate its next jit-lock-stealth-nice value based
on that.  Or should disable stealth lock in buffers that seem not to
behave well.

As I said, the most common culprit at the moment for me seems to be
texbook.tex.gz.  I can send you a copy if you want to, but as I said,
the problem also seems to depend on some third-party packages.

After a few seconds of thrashing, I get the following results from
instrumenting font-latex:

Function Name                              Call Count  Elapsed Time  Average Time
=========================================  ==========  ============  ============
font-latex-match-command-in-braces         1780        3.2092149999  0.0018029297
font-latex-find-matching-close             970         3.0349990000  0.0031288649
font-latex-match-type-declaration          10          3.012727      0.3012727
font-latex-match-simple-command            62693       2.2595180000  3.604...e-05
font-latex-script                          14685       1.3847209999  9.429...e-05
font-latex-match-command-with-arguments    4800        1.2166909999  0.0002534772
font-latex-match-script                    14695       1.1428459999  7.777...e-05
font-latex-match-function                  4210        0.8861020000  0.0002104755
font-latex-commented-outp                  27908       0.7627639999  2.733...e-05
font-latex-unfontify-region                10          0.4944489999  0.0494449

Now 62693 is a large number, and the whole file apparently contains
just 37633 occurences of LaTeX commands.  The stealth fontification
for which I have now profiled just a few seconds, pretty much goes on
indefinitely.  So it would appear that there is something like
quadratic behavior in some circumstances triggered in connection with
stealth fontification.

So yes, there is likely a bug (or whatever one may call something that
takes up too much time) in this particularly severe case, but it is
not triggered outside of stealth fontification, and stealth
fontification makes the problem be pretty much impossible to debug
properly, and triggers it when editing _any_ buffer.

I can send you the file in question, but you'll probably need
font-latex and possible other AUCTeX components for reproducing the
problem.  But it is likely to be a black hole: there are other modes
with possible font locking problems, and it is just not a good idea to
spread their effects across a whole Emacs session.  That way, they
will much less likely be fixed some day.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[Eli Zaretskii]

> Cc: rms@gnu.org,  emacs-devel@gnu.org
> From: David Kastrup <dak@gnu.org>
> Date: Sun, 04 Mar 2007 14:29:48 +0100
> 
> I can send you the file in question

If that's the standard texbook.tex from CTAN, I have it.  FWIW, I
don't see any special problem with it in "emacs -Q".  JIT stealth
still takes 1-3% of CPU on a 3GHz box, when this file is the only one
loaded in a session.

> but you'll probably need font-latex and possible other AUCTeX
> components for reproducing the problem.

Are there other files which exhibit similar problems?  Or did we just
change the defaults because of some singular case, which on top of
that needs unbundled packages to raise its ugly head?

Re: Default of jit-lock-stealth-time

[David Kastrup]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: rms@gnu.org,  emacs-devel@gnu.org
>> From: David Kastrup <dak@gnu.org>
>> Date: Sun, 04 Mar 2007 14:29:48 +0100
>> 
>> I can send you the file in question
>
> If that's the standard texbook.tex from CTAN, I have it.  FWIW, I
> don't see any special problem with it in "emacs -Q".  JIT stealth
> still takes 1-3% of CPU on a 3GHz box, when this file is the only one
> loaded in a session.
>
>> but you'll probably need font-latex and possible other AUCTeX
>> components for reproducing the problem.
>
> Are there other files which exhibit similar problems?  Or did we just
> change the defaults because of some singular case, which on top of
> that needs unbundled packages to raise its ugly head?

I get similar effects with other files (not just TeX files, C mode has
shown similar problems in some iterations), and if you had followed
the discussion closely, you'd have realized that several other
developers have disabled jit-lock-stealth-time previously for their
own problems using it.

I apologize for providing an actual example if it causes such a
reaction.

In case you have forgotten: I have repeatedly stated that there might
very well be problems with some font lock patterns in some modes.  The
problem with stealth fontification is that it makes those problems
non-debuggable, non-deterministic and spreads their bad effects across
all buffers and modes, without the user being able to pinpoint them
any more.

With the current settings, such problems, if they exist, will exhibit
themselves when paging through the affected areas.  People will be
able to report them.  Currently, we instead let Emacs drain the
batteries without a possibility to pinpoint when and why that is
happening.

And again: it is not just me and not just a single file and setup:
_several_ other developers have, after my report, stated that they had
long ago disabled stealth fontification for similar reasons.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[Richard Stallman]

Thanks.

Re: Default of jit-lock-stealth-time

[Richard Stallman]

    So yes, there is likely a bug (or whatever one may call something that
    takes up too much time) in this particularly severe case, but it is
    not triggered outside of stealth fontification, and stealth
    fontification makes the problem be pretty much impossible to debug
    properly, and triggers it when editing _any_ buffer.

Is the bug in font-latex, perhaps?

Re: Default of jit-lock-stealth-time

[David Kastrup]

Richard Stallman <rms@gnu.org> writes:

>     So yes, there is likely a bug (or whatever one may call something that
>     takes up too much time) in this particularly severe case, but it is
>     not triggered outside of stealth fontification, and stealth
>     fontification makes the problem be pretty much impossible to debug
>     properly, and triggers it when editing _any_ buffer.
>
> Is the bug in font-latex, perhaps?

I think it imprudent to call it "the bug": any problem in font
highlighting patterns in any of the loaded buffers/major modes can
have the same, non-traceable effects.  I already said that I have seen
similar things with cc-mode.  Until we can guarantee that all major
modes have efficient font lock patterns/code ("correct" alone does not
cut it), stealth fontification will hide problems while making them
more severe.

After some back and forth with one AUCTeX developer, I have asked him
to install a private branch to font-latex which is likely related to
the problem seen with texbook.tex.  It would be my guess that it might
address the particular performance problem seen with texbook.tex.

So the problem triggered by this particular input file might be gone
with future releases of AUCTeX.

But as I already stated: this was merely the file that broke the
camel's back for me and caused me to report the problem which I was
unable to trace to a cause (stealth fontification _is_ stealthy in
hiding its tracks, if not its effects).

And it certainly was not the file that caused several other developers
to disable stealth fontification before (which they then reported).

-- 
David Kastrup

Re: Default of jit-lock-stealth-time

[Miles Bader]

David Kastrup <dak@gnu.org> writes:
>> If that's the standard texbook.tex from CTAN, I have it.  FWIW, I
>> don't see any special problem with it in "emacs -Q".  JIT stealth
>> still takes 1-3% of CPU on a 3GHz box, when this file is the only one
>> loaded in a session.
>
> I get similar effects with other files (not just TeX files, C mode has
> shown similar problems in some iterations), and if you had followed
> the discussion closely, you'd have realized that several other
> developers have disabled jit-lock-stealth-time previously for their
> own problems using it.

What OS do you use?  How much memory you have?

-Miles

-- 
"Most attacks seem to take place at night, during a rainstorm, uphill,
 where four map sheets join."   -- Anon. British Officer in WW I

Re: Default of jit-lock-stealth-time

[David Kastrup]

Miles Bader <miles@gnu.org> writes:

> David Kastrup <dak@gnu.org> writes:
>>> If that's the standard texbook.tex from CTAN, I have it.  FWIW, I
>>> don't see any special problem with it in "emacs -Q".  JIT stealth
>>> still takes 1-3% of CPU on a 3GHz box, when this file is the only one
>>> loaded in a session.
>>
>> I get similar effects with other files (not just TeX files, C mode has
>> shown similar problems in some iterations), and if you had followed
>> the discussion closely, you'd have realized that several other
>> developers have disabled jit-lock-stealth-time previously for their
>> own problems using it.
>
> What OS do you use?  How much memory you have?

Ubuntu GNU/Linux, 384kB.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: Default of jit-lock-stealth-time

[Miles Bader]

David Kastrup <dak@gnu.org> writes:
>> What OS do you use?  How much memory you have?
>
> Ubuntu GNU/Linux, 384kB.
                       ^^

Gee, no wonder!  :-/

-miles

-- 
"An atheist doesn't have to be someone who thinks he has a proof that there
can't be a god.  He only has to be someone who believes that the evidence
on the God question is at a similar level to the evidence on the werewolf
question."  [John McCarthy]

Re: Default of jit-lock-stealth-time

[David Kastrup]

Miles Bader <miles.bader@necel.com> writes:

> David Kastrup <dak@gnu.org> writes:
>>> What OS do you use?  How much memory you have?
>>
>> Ubuntu GNU/Linux, 384kB.
>                        ^^
>
> Gee, no wonder!  :-/

Uh, 384MB.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum