font-lock-keywords: matcher function not working

font-lock-keywords: matcher function not working

[Raffaele Ricciardi]

Hello everybody,

I'm trying to highlight the first parenthesis of each line in Lisp code.

After having read the documentation of `font-lock-keywords', I've
written a matcher function which behaves as required (I've tested it
with `eval-expression').  However, first parentheses are not
highlighted.


Here is my code, tested on GNU Emacs 24.3

;; Tested in the *scratch* buffer after "emacs -Q".

(require 'cl)

(defun* rr-first-non-blank? (&optional (point (point)))
   "Return non-NIL if POINT is on the first non-blank character of the
current line."
   (= point
      (save-excursion
        (back-to-indentation)
        (point))))

(defun rr-match-indented-paren (^end)
   "Matcher for `font-lock-keywords' that matches a parenthesis made
  redundant by indentation."
   (if (and (eql (char-after) ?\()
            (rr-first-non-blank?))
       (re-search-forward "(" ^end)
       nil))

(font-lock-add-keywords 'lisp-interaction-mode
                         '(
                           (rr-match-indented-paren . 
'font-lock-warning-face)
                           ))

(font-lock-fontify-buffer)

Re: font-lock-keywords: matcher function not working

[Raffaele Ricciardi]

I forgot: thanks in advance for your help, of course.

Cheers.

Re: font-lock-keywords: matcher function not working

[Stefan Monnier]

> (defun rr-match-indented-paren (^end)
>   "Matcher for `font-lock-keywords' that matches a parenthesis made
>  redundant by indentation."
>   (if (and (eql (char-after) ?\()
>            (rr-first-non-blank?))
>       (re-search-forward "(" ^end)
>       nil))

> (font-lock-add-keywords 'lisp-interaction-mode
>                         '(
>                           (rr-match-indented-paren
> . 'font-lock-warning-face)
>                           ))

Your rr-match-indented-paren should *search* rather than only check the
current position.

While I'm here, I recommend you (setq font-lock-support-mode nil) then
disable&enable font-lock so that jit-lock won't be used: this makes it
much easier to debug the code, especially when it signals errors (or
course, you also want to set debug-on-error to t).


        Stefan

Re: font-lock-keywords: matcher function not working

[rfflrccrd]

On Thursday, June 27, 2013 7:32:57 PM UTC+2, Stefan Monnier wrote:
> Your rr-match-indented-paren should *search* rather than only check the
> 
> current position.

Thank you for your attention, Stefan.

However, doesn't my matcher function search already?  It checks the
current position first to return NIL for a failed match.  OTOH, when
the check succeeds, the matcher function calls `re-search-forward' to
set `match-data' appropriately.  IMO, this is the expected behaviour
according to the documentation of `font-lock-keywords':

"MATCHER [...] should return non-nil, move point, and set `match-data'
appropriately if it succeeds; like `re-search-forward' would."

Otherwise - that is: when the match fails - MATCHER should do nothing
and return NIL, shouldn't it?

> While I'm here, I recommend you (setq font-lock-support-mode nil) then
> 
> disable&enable font-lock so that jit-lock won't be used: this makes it
> 
> much easier to debug the code, especially when it signals errors (or
> 
> course, you also want to set debug-on-error to t).

Thank you for these tips.

Re: font-lock-keywords: matcher function not working

[Stefan Monnier]

> However, doesn't my matcher function search already?

No, it doesn't.

> It checks the current position first to return NIL for a failed match.

That's not a search.  It just says "doesn't match here" without saying
anything about the rest of the text until ^end, whereas a nil return
value is supposed to mean "nothing found between here and ^end" (just
as a nil return value of `re-search-forward').


        Stefan

Re: font-lock-keywords: matcher function not working

[Raffaele Ricciardi]

On 28/06/13 21:30, Stefan Monnier wrote:
> That's not a search.  It just says "doesn't match here" without saying
> anything about the rest of the text until ^end, whereas a nil return
> value is supposed to mean "nothing found between here and ^end" (just
> as a nil return value of `re-search-forward').

Thank you again, Stefan.  However, I still don't understand. The check
is there because Emacs' regular expressions lack a "look behind"
construct, and the matcher has to check for leading whitespace.  How
is it supposed to look?  I chose to look for leading whitespace
separately, and to choose afterwards whether to return NIL immediately
or to return the result of `re-search-forward' - which at that point
is guaranteed to succeed - but such call is a shortcut to ensure that
the matcher returns information consistent to the requirements of the
font locking engine.  Am I missing something?

Re: font-lock-keywords: matcher function not working

[Stefan Monnier]

> Thank you again, Stefan.  However, I still don't understand.

Your function should search for the first occurrence of the thing you're
looking for, between here and ^end and only if there isn't any should it
return nil.

Instead, your current function only checks whether "here" is a spot that
matches the thing you're looking for.  Instead, when you find out that
"here" is not a match, don't return nil: try to look further instead.
This is typically done with a loop.

In your case, I wouldn't use a function anyway: I'd just use a regexp
like "^[^ \t]*(".


        Stefan

Re: font-lock-keywords: matcher function not working

[Raffaele Ricciardi]

On 29/06/13 04:35, Stefan Monnier wrote:
>> Thank you again, Stefan.  However, I still don't understand.
>
> Your function should search for the first occurrence of the thing you're
> looking for, between here and ^end and only if there isn't any should it
> return nil.
>
> Instead, your current function only checks whether "here" is a spot that
> matches the thing you're looking for.  Instead, when you find out that
> "here" is not a match, don't return nil: try to look further instead.
> This is typically done with a loop.
>
> In your case, I wouldn't use a function anyway: I'd just use a regexp
> like "^[^ \t]*(".

Thank you for your attention, Stefan.  Now I understand what the font lock
engine requires.

Following your suggestion, this snippet performs the desired highlighting:

(font-lock-add-keywords 'lisp-interaction-mode
                         '(
                           ("^[[:blank:]]*\\((\\)" 1 
'font-lock-warning-face)
                           ))

However, unless I restart `lisp-interaction-mode', no highlighting occurs,
neither it does if I just restart `font-lock-mode'.  Why is it so?  I 
was expecting
`font-lock-fontify-buffer' to apply the new rules.

Re: font-lock-keywords: matcher function not working

[Raffaele Ricciardi]

On 29/06/13 10:53, Raffaele Ricciardi wrote:
> However, unless I restart `lisp-interaction-mode', no highlighting occurs,
> neither it does if I just restart `font-lock-mode'.  Why is it so?  I
> was expecting
> `font-lock-fontify-buffer' to apply the new rules.

Meanwhile, for those who are interested, I've discovered that
`font-lock-fontify-buffer' applies immediately any new rules applied to the
current buffer only.  Therefore, while experimenting, you can use for 
example:

(font-lock-add-keywords nil
                         '(
                           ("^[[:blank:]]*\\((\\)" 1 
'font-lock-warning-face)
                           ))

instead of:

(font-lock-add-keywords 'lisp-interaction-mode
                         '(
                           ("^[[:blank:]]*\\((\\)" 1 
'font-lock-warning-face)
                           ))

and then call `font-lock-fontify-buffer' to see the effect.  Of course, 
do not forget to
call `font-lock-remove-keywords' on the old rules, first.