Font-lock in COMINT modes

Font-lock in COMINT modes

[JD Smith]

In Emacs 22, my idlwave-shell mode font-lock fontifies
comments/strings/etc. without having been asked to. This can be
problematic if you miss a closing quote, and all subsequent text gets
string fontification.  In Emacs 21, the only fontification was comint
coloring the prompt blue.

I don't mind the fontification so much (in fact I like it), it's just
that missing string quotes allow it to run away, latching onto parts
of the subsequent input and output in a way that's hard to cancel.
Perhaps there is an easy way to inhibit scanning through the prompt?

JD

Re: Font-lock in COMINT modes

[JD Smith]

On Wed, 06 Dec 2006 17:57:36 -0700, JD Smith wrote:

> 
> In Emacs 22, my idlwave-shell mode font-lock fontifies
> comments/strings/etc. without having been asked to. This can be
> problematic if you miss a closing quote, and all subsequent text gets
> string fontification.  In Emacs 21, the only fontification was comint
> coloring the prompt blue.
> 
> I don't mind the fontification so much (in fact I like it), it's just
> that missing string quotes allow it to run away, latching onto parts
> of the subsequent input and output in a way that's hard to cancel.
> Perhaps there is an easy way to inhibit scanning through the prompt?

A strange side note.  In attempting to find out why strings and
comments are being fontified in my comint shell, despite the fact
that:

 font-lock-defaults is a variable defined in `font-core.el'. 
 Its value is (nil)

 Local in buffer *idl*; global value is nil

I attempted to instrument `font-lock-default-fontify-region'.  Edebug
couldn't do so, giving this error:

 Debugger entered--Lisp error: (invalid-read-syntax "Expected lambda
 expression")

Re: Font-lock in COMINT modes

[JD Smith]

Here is the change which altered the default behavior in comint modes. 

----------------------------
revision 1.348
date: 2006-09-28 19:09:19 +0000;  author: rms;  state: Exp;  lines: +4 -1
(comint-mode): Bind font-lock-defaults non-nil.
----------------------------

If font-lock-defaults is left nil instead of being explicitly set to
'(nil) as it now is, keywords-only is implicitly true, and strings and
comments are not fontified by default.  With this change, strings and
comments are fontified by default in all comint modes (unless they set
font-lock-defaults on their own).

This patch would restore the earlier behavior.  Since I'm uncertain
whether '(nil) was intended to nullify keywords-only, I'm not sure if the
patch is appropriate (i.e. is '(nil t) practically any different from nil).

*** comint.el	28 Nov 2006 11:25:56 -0700	1.350
--- comint.el	07 Dec 2006 10:54:48 -0700	
***************
*** 653,659 ****
    (make-local-variable 'comint-accum-marker)
    (setq comint-accum-marker (make-marker))
    (make-local-variable 'font-lock-defaults)
!   (setq font-lock-defaults '(nil))
    (add-hook 'change-major-mode-hook 'font-lock-defontify nil t)
    ;; This behavior is not useful in comint buffers, and is annoying
    (set (make-local-variable 'next-line-add-newlines) nil))
--- 653,659 ----
    (make-local-variable 'comint-accum-marker)
    (setq comint-accum-marker (make-marker))
    (make-local-variable 'font-lock-defaults)
!   (setq font-lock-defaults '(nil t))
    (add-hook 'change-major-mode-hook 'font-lock-defontify nil t)
    ;; This behavior is not useful in comint buffers, and is annoying
    (set (make-local-variable 'next-line-add-newlines) nil))

Re: Font-lock in COMINT modes

[Richard Stallman]

    Here is the change which altered the default behavior in comint modes. 

    ----------------------------
    revision 1.348
    date: 2006-09-28 19:09:19 +0000;  author: rms;  state: Exp;  lines: +4 -1
    (comint-mode): Bind font-lock-defaults non-nil.
    ----------------------------

I'm sure I did this to fix a bug, but I can't find any discussion
about it.  Maybe it was so facemenu-enable-faces-p would return nil.

As far as I can see, if font-lock-defaults is nil, that is equivalent
to (nil).  How do you conclude that nil would be equivalent to (nil t)?

But that is a side issue.  If the value (nil t) is better, we can
switch to it.  Do people agree it is better?

Re: Font-lock in COMINT modes

[JD Smith]

On Fri, 08 Dec 2006 20:26:27 -0500, Richard Stallman wrote:

>     Here is the change which altered the default behavior in comint modes.
> 
>     ----------------------------
>     revision 1.348
>     date: 2006-09-28 19:09:19 +0000;  author: rms;  state: Exp;  lines: +4
>     -1 (comint-mode): Bind font-lock-defaults non-nil.
>     ----------------------------
> 
> I'm sure I did this to fix a bug, but I can't find any discussion about
> it.  Maybe it was so facemenu-enable-faces-p would return nil.
> 
> As far as I can see, if font-lock-defaults is nil, that is equivalent to
> (nil).  How do you conclude that nil would be equivalent to (nil t)?

nil and '(nil) are not equivalent, in that the former implicitly
causes only keywords to be fontified, and the latter causes
font-lock-keywords-only to be set locally to nil (since it is a list
without a 2nd entry), which allows string and comments to be
fontified.

This is a telling comment from font-core.el:

  ;; Only do hard work if the mode has specified stuff in
  ;; `font-lock-defaults'.

All the defaults, like keyword-only, are recovered from the
font-lock-defaults list *only* if it is non-nil (e.g. '(nil)).  I
haven't understood how a purely nil font-lock-defaults implicitly
prevents string+comment fontification independent of the keywords-only
setting, but it does.

'(nil t) explicitly sets keywords-only to t, restoring the behavior.
What I don't know is whether nil and '(nil t) differ in other ways.
They clearly exercise different code paths.

JD

Re: Font-lock in COMINT modes

[Stefan Monnier]

> I haven't understood how a purely nil font-lock-defaults implicitly
> prevents string+comment fontification independent of the keywords-only
> setting, but it does.

Because if it's nil font-lock-mode-internal is not activated at all.
AFAIK nil and (nil t) are equivalent (i.e. don't highlight anything), except
that (nil t) takes a lot more CPU to do nothing (it runs all the
font-lock-foo-function hooks, enables jit-lock so as to do nothing
just-in-time rather than eagerly, checks where nothing needs to be done,
rounds up to a whole number of lines the region upon which inaction is
requested, checks if there's a multiline entity and extends the region even
more, so that nothing is done in a multiline awaer way, ...).


        Stefan

Re: Font-lock in COMINT modes

[Richard Stallman]

    This is a telling comment from font-core.el:

      ;; Only do hard work if the mode has specified stuff in
      ;; `font-lock-defaults'.

That conditional must be the reason for this difference in behavior.

Can someone please document what it means for `font-lock-defaults'
to be nil?  Including the effect on facemenu?

Re: Font-lock in COMINT modes

[JD Smith]

On Fri, 08 Dec 2006 23:02:29 -0500, Stefan Monnier wrote:

>> I haven't understood how a purely nil font-lock-defaults implicitly
>> prevents string+comment fontification independent of the keywords-only
>> setting, but it does.
> 
> Because if it's nil font-lock-mode-internal is not activated at all. AFAIK
> nil and (nil t) are equivalent (i.e. don't highlight anything), except
> that (nil t) takes a lot more CPU to do nothing (it runs all the
> font-lock-foo-function hooks, enables jit-lock so as to do nothing
> just-in-time rather than eagerly, checks where nothing needs to be done,
> rounds up to a whole number of lines the region upon which inaction is
> requested, checks if there's a multiline entity and extends the region
> even more, so that nothing is done in a multiline awaer way, ...).

Classic ;).

My presumption is that '(nil) and nil must differ in some other
way (aside from the unintended side effect of changing the default
non-keyword fontification), since the original change from Sept.
explicitly bound f-l-defaults non-nil, which by inference suggests nil and
'(nil t) would exhibit this same difference, whatever it is.

Re: Font-lock in COMINT modes

[Richard Stallman]

    My presumption is that '(nil) and nil must differ in some other
    way (aside from the unintended side effect of changing the default
    non-keyword fontification), since the original change from Sept.
    explicitly bound f-l-defaults non-nil, which by inference suggests nil and
    '(nil t) would exhibit this same difference, whatever it is.

It affects facemenu.  I am pretty sure that was my motive
for making the change.

However, if it also makes things a lot slower, that is not good.
Has some font lock expert verified that this is really happening?

If so, we should set up a way to turn off facemenu for faces while not
turning on the Font Lock mechanism unnecessarily.

One way is for Font Lock to notice the value (nil t)
specially and not turn on the mechanism.

Another way is to have another variable to disable the faces menu
with.  That way is very safe.

Re: Font-lock in COMINT modes

[Richard Stallman]

I listed in FOR-RELEASE that this case is slow:

    You must have gotten that impression from Stefan's 12/08 message:

      if [font-lock-defaults ] is nil font-lock-mode-internal is not
      activated at all.  AFAIK nil and (nil t) are equivalent (i.e. don't
      highlight anything), except that (nil t) takes a lot more CPU to do
      nothing (it runs all the font-lock-foo-function hooks, enables
      jit-lock so as to do nothing just-in-time rather than eagerly,
      checks where nothing needs to be done, rounds up to a whole number
      of lines the region upon which inaction is requested, checks if
      there's a multiline entity and extends the region even more, so that
      nothing is done in a multiline way, ...).

    Even though it is true that font lock jumps through more hoops with
    (nil t), there is no evidence that this slowdown affects the user.


Could you check and see?

Re: Font-lock in COMINT modes

[Chong Yidong]

Richard Stallman <rms@gnu.org> writes:

> I listed in FOR-RELEASE that this case is slow:
>
>     You must have gotten that impression from Stefan's 12/08 message:
>
>       if [font-lock-defaults ] is nil font-lock-mode-internal is not
>       activated at all.  AFAIK nil and (nil t) are equivalent (i.e. don't
>       highlight anything), except that (nil t) takes a lot more CPU to do
>       nothing (it runs all the font-lock-foo-function hooks, enables
>       jit-lock so as to do nothing just-in-time rather than eagerly,
>       checks where nothing needs to be done, rounds up to a whole number
>       of lines the region upon which inaction is requested, checks if
>       there's a multiline entity and extends the region even more, so that
>       nothing is done in a multiline way, ...).
>
>     Even though it is true that font lock jumps through more hoops with
>     (nil t), there is no evidence that this slowdown affects the user.
>
>
> Could you check and see?

I haven't been able to observe any slowdown when font-lock-defaults is
(nil t).  Not surprising, since I don't notice any slowdown when
font-lock-defaults is (something t), i.e. normal font-lock situations!

Maybe someone running on a 486 can observe a difference, who knows.

Re: Font-lock in COMINT modes

[Stefan Monnier]

>> I listed in FOR-RELEASE that this case is slow:
>> 
>> You must have gotten that impression from Stefan's 12/08 message:
>> 
>> if [font-lock-defaults ] is nil font-lock-mode-internal is not
>> activated at all.  AFAIK nil and (nil t) are equivalent (i.e. don't
>> highlight anything), except that (nil t) takes a lot more CPU to do
>> nothing (it runs all the font-lock-foo-function hooks, enables
>> jit-lock so as to do nothing just-in-time rather than eagerly,
>> checks where nothing needs to be done, rounds up to a whole number
>> of lines the region upon which inaction is requested, checks if
>> there's a multiline entity and extends the region even more, so that
>> nothing is done in a multiline way, ...).
>> 
>> Even though it is true that font lock jumps through more hoops with
>> (nil t), there is no evidence that this slowdown affects the user.
>> 
>> 
>> Could you check and see?

> I haven't been able to observe any slowdown when font-lock-defaults is
> (nil t).  Not surprising, since I don't notice any slowdown when
> font-lock-defaults is (something t), i.e. normal font-lock situations!

> Maybe someone running on a 486 can observe a difference, who knows.

Indeed, the "slowdown" should be completely and absolutely neligible, even
on a 16MHz 386sx.


        Stefan

Re: Font-lock in COMINT modes

[Richard Stallman]

    Indeed, the "slowdown" should be completely and absolutely neligible, even
    on a 16MHz 386sx.

I believe you, but if it isn't noticeable, I wish you had not
presented it as a problem before.  I believed you then, too.

Re: Font-lock in COMINT modes

[Stefan Monnier]

>     Indeed, the "slowdown" should be completely and absolutely neligible, even
>     on a 16MHz 386sx.

> I believe you, but if it isn't noticeable, I wish you had not
> presented it as a problem before.  I believed you then, too.

Sorry, I (obviously) didn't mean to present it as a problem. I was just in
a lighthearted mood when I wrote it,


        Stefan