hi-lock-mode doesn't work with emacs -Q.

hi-lock-mode doesn't work with emacs -Q.

[Alan Mackenzie]

Hi, Stefan!

Start emacs -Q.  (Emacs 22.1, of course).  Visit a new file with:

    C-x C-f foo.txt

.  Type this:

    This file is foo.txt.

.  Enable hi-lock-mode and make "foo" a highlightable pattern:

    M-x hi-lock-mode
    C-x w h foo<CR><CR>    ; accept the default hi-yellow face.

.  This highlights the "foo" yellow, as expected.  At the end of the
line, type:

    foo

.  This new "foo" doesn't get highlighted.  It should be.

#########################################################################

Partial diagnosis:
(i) The bug happens even when global-hi-lock-mode is enabled.
(ii) after-change-functions is nil.  It ought to be
  (jit-lock-after-change t).
(iii) font-lock-mode is t.

How can font-lock-mode be t, whilst at the same time there is no
after-change function?  Is this some optimisation in Font Lock mode
that only sets after-change-functions when font-lock-keywords is
non-nil?

hi-lock.el doesn't seem to be violating Font Lock's proper calling
conventions in any way.

Stefan, I think you're the best person to fix this.

-- 
Alan Mackenzie (Ittersbach, Germany).

Re: hi-lock-mode doesn't work with emacs -Q.

[Stefan Monnier]

> How can font-lock-mode be t, whilst at the same time there is no
> after-change function?

Because font-lock-mode-internal is not enabled.

> Is this some optimisation in Font Lock mode that only sets
> after-change-functions when font-lock-keywords is non-nil?

Yes.

> hi-lock.el doesn't seem to be violating Font Lock's proper calling
> conventions in any way.

Indeed.  It's that the `font-lock-mode' function has changed meaning from
Emacs-21 to Emacs-22.  I'm not yet sure how best to fix it.


        Stefan

Re: hi-lock-mode doesn't work with emacs -Q.

[Alan Mackenzie]

Hi, Stefan.

On Thu, Jun 07, 2007 at 09:22:40AM -0400, Stefan Monnier wrote:
> > How can font-lock-mode be t, whilst at the same time there is no
> > after-change function?

> Because font-lock-mode-internal is not enabled.

> > Is this some optimisation in Font Lock mode that only sets
> > after-change-functions when font-lock-keywords is non-nil?

> Yes.

> > hi-lock.el doesn't seem to be violating Font Lock's proper calling
> > conventions in any way.

> Indeed.  It's that the `font-lock-mode' function has changed meaning from
> Emacs-21 to Emacs-22.  I'm not yet sure how best to fix it.

How about always enabling the pertinent after-change function (there are
just two, now, aren't there?), and wrapping it in the test currently in
font-lock-default-function at the comment
  ;; Only do hard work if the mode has specified stuff in
  ;; `font-lock-defaults'.
?  The loss in performance for un-font-locked buffers will be
negligible, surely?

>         Stefan

-- 
Alan.

Re: hi-lock-mode doesn't work with emacs -Q.

[Stefan Monnier]

> .  Enable hi-lock-mode and make "foo" a highlightable pattern:

>     M-x hi-lock-mode
>     C-x w h foo<CR><CR>    ; accept the default hi-yellow face.

> .  This highlights the "foo" yellow, as expected.  At the end of the
> line, type:

>     foo

> .  This new "foo" doesn't get highlighted.  It should be.

I believe the patch below fixes the problem without making things worse and
without requiring changes to hi-lock.el.


        Stefan


--- font-lock.el	08 May 2007 11:55:52 -0400	1.317
+++ font-lock.el	11 Jun 2007 15:56:01 -0400	
@@ -698,6 +698,14 @@
 	 ;; contain the new keywords.
 	 (font-lock-update-removed-keyword-alist mode keywords how))
 	(t
+         (when (and font-lock-mode
+                    (not (or font-lock-keywords font-lock-defaults)))
+           ;; The major mode has not set any keywords, so when we enabled
+           ;; font-lock-mode it only enabled the font-core.el part, not the
+           ;; font-lock-mode-internal.  Try again.
+           (font-lock-mode -1)
+           (set (make-local-variable 'font-lock-defaults) '(nil t))
+           (font-lock-mode 1))
 	 ;; Otherwise set or add the keywords now.
 	 ;; This is a no-op if it has been done already in this buffer
 	 ;; for the correct major mode.

Re: hi-lock-mode doesn't work with emacs -Q.

[Stefan Monnier]

> That patch will certainly make hi-lock work properly.  However, are you
> sure there is nothing else which can necessitate the enabling of the
> after-change function?  Also, there is nothing in the documentation
> forbidding a hacker from changing font-lock-keywords directly, which
> would cause font-lock to fail in other circumstances.

Indeed.

> It seems to me that this two-stage initialisation of f-l-mode brings a
> lot of complexity for very little gain - The f-l-change-function can
> never execute in the background and consume vast amounts of CPU usage.
> It can only ever be called for the current buffer, and if Font Lock is
> enabled, yet f-l-keywords hasn't been set, its runtime is negligible
> anyway.

The truth is: I haven't followed closely the introduction of font-core.el
and how font-lock-mode and font-lock-mode-internal interact.  So I'm not
sure which part of font-core.el's behavior is important and which isn't, so
I prefer to fix things from font-lock.el whre I understand the situation
a bit better.

> Or is there some good reason for this optimisation that I'm missing?  If
> not, I suggest that you simply remove this optimisation and always
> install the after-change function when f-l-mode is started.

That sounds great to me, but that amounts to removing font-core.el, so I'd
need to hear from those who introduced font-core.el.


        Stefan

Re: hi-lock-mode doesn't work with emacs -Q.

[Alan Mackenzie]

Hi, Stefan!

On Mon, Jun 11, 2007 at 04:25:14PM -0400, Stefan Monnier wrote:
> > .  Enable hi-lock-mode and make "foo" a highlightable pattern:

> >     M-x hi-lock-mode
> >     C-x w h foo<CR><CR>    ; accept the default hi-yellow face.

> > .  This highlights the "foo" yellow, as expected.  At the end of the
> > line, type:

> >     foo

> > .  This new "foo" doesn't get highlighted.  It should be.

> I believe the patch below fixes the problem without making things worse
> and without requiring changes to hi-lock.el.


>         Stefan


> --- font-lock.el	08 May 2007 11:55:52 -0400	1.317
> +++ font-lock.el	11 Jun 2007 15:56:01 -0400	
> @@ -698,6 +698,14 @@
>  	 ;; contain the new keywords.
>  	 (font-lock-update-removed-keyword-alist mode keywords how))
>  	(t
> +         (when (and font-lock-mode
> +                    (not (or font-lock-keywords font-lock-defaults)))
> +           ;; The major mode has not set any keywords, so when we enabled
> +           ;; font-lock-mode it only enabled the font-core.el part, not the
> +           ;; font-lock-mode-internal.  Try again.
> +           (font-lock-mode -1)
> +           (set (make-local-variable 'font-lock-defaults) '(nil t))
> +           (font-lock-mode 1))
>  	 ;; Otherwise set or add the keywords now.
>  	 ;; This is a no-op if it has been done already in this buffer
>  	 ;; for the correct major mode.

That patch will certainly make hi-lock work properly.  However, are you
sure there is nothing else which can necessitate the enabling of the
after-change function?  Also, there is nothing in the documentation
forbidding a hacker from changing font-lock-keywords directly, which
would cause font-lock to fail in other circumstances.

My first thought was that instead of switching Font Lock fully off then
fully on again, as you are proposing in your patch, we should just call
font-lock-mode-internal.  But this clashes with the
font-lock-default-function mechanism - font-lock-keywords cannot know
that the current f-l-default-function has used this
"ignition/starter-motor" mechanism.

It seems to me that this two-stage initialisation of f-l-mode brings a
lot of complexity for very little gain - The f-l-change-function can
never execute in the background and consume vast amounts of CPU usage.
It can only ever be called for the current buffer, and if Font Lock is
enabled, yet f-l-keywords hasn't been set, its runtime is negligible
anyway.

Or is there some good reason for this optimisation that I'm missing?  If
not, I suggest that you simply remove this optimisation and always
install the after-change function when f-l-mode is started.

-- 
Alan.