Conditionalized font-locking?

Conditionalized font-locking?

[Stephen Berman]

In a certain mode I'd like to fontify date strings where the date is
earlier than current-time-string differently from date strings of later
dates.  I tried to do this with font-lock-keywords by filtering the
output of the regexp search.  It seems to work, but as soon as a
filtered matching string is fontified, Emacs appears to hang (the gdb
backtrace shows thousands of calls to mark_object).  Hitting `C-x k'
followed by rapidly repeating `C-g' many times eventually gets Emacs
back to normal.

The case with date strings is more complicated than necessary to show
the problem; the following code shows the same problem for a simpler
case, fontifying odd numbers (of course, that can be done with just a
regexp and then there's no problem, but I don't think the date string
case is possible just with a regexp):

--8<---------------cut here---------------start------------->8---
(defvar srb-font-lock-keywords
  (list '(srb-number-matcher 1 font-lock-warning-face t)))

(defun srb-number-matcher (lim)
  "Search for odd numbers within LIM for font-locking."
  (re-search-forward "\\<\\([0-9]+\\)\\>" lim t)
  (let ((num (match-string-no-properties 1)))
    (when (eq (logand (string-to-number num) 1) 1) num)))

(define-derived-mode srb-mode nil "SRB" ()
  "Mode for testing font-locking."
  (set (make-local-variable 'font-lock-defaults)
       '(srb-font-lock-keywords t)))
--8<---------------cut here---------------end--------------->8---

Evaluate this code, switch to a new buffer, type `M-x srb-mode' and then
the number `1': the number gets fontified but Emacs appears to hang.

Am I doing something wrong?  If so, what's the right way to do this?  Or
is what I'm trying to do not possible?

Steve Berman

Re: Conditionalized font-locking?

[Stefan Monnier]

> (defun srb-number-matcher (lim)
>   "Search for odd numbers within LIM for font-locking."
>   (re-search-forward "\\<\\([0-9]+\\)\\>" lim t)
>   (let ((num (match-string-no-properties 1)))
>     (when (eq (logand (string-to-number num) 1) 1) num)))

If the re-search-forward fails, this function will mis-behave.


        Stefan

Re: Conditionalized font-locking?

[Stephen Berman]

On Mon, 07 May 2012 11:05:27 -0400 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote:

>> (defun srb-number-matcher (lim)
>>   "Search for odd numbers within LIM for font-locking."
>>   (re-search-forward "\\<\\([0-9]+\\)\\>" lim t)
>>   (let ((num (match-string-no-properties 1)))
>>     (when (eq (logand (string-to-number num) 1) 1) num)))
>
> If the re-search-forward fails, this function will mis-behave.

D'oh!

Thanks...

Steve Berman

Re: Conditionalized font-locking?

[Stephen Berman]

On 7 May 2012 17:15:34 +0200 "Stephen Berman" <stephen.berman@gmx.net> wrote:

> On Mon, 07 May 2012 11:05:27 -0400 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote:
>
>>> (defun srb-number-matcher (lim)
>>>   "Search for odd numbers within LIM for font-locking."
>>>   (re-search-forward "\\<\\([0-9]+\\)\\>" lim t)
>>>   (let ((num (match-string-no-properties 1)))
>>>     (when (eq (logand (string-to-number num) 1) 1) num)))
>>
>> If the re-search-forward fails, this function will mis-behave.
>
> D'oh!

Unfortunately, I'm still not getting what I want.  It's not unlikely I'm
again overlooking something obvious (though I've looked long and hard,
and tried lots of variations), so if you or someone else can point it
out, I'd be grateful.

Again, what I want is to fontify date-time strings with a special face
if the date in the string is earlier than the current date; moreover,
the time string is optional.  The following code (which is close to what
I'm actually using but with a simpler regexp and filtering condition)
seems to work at first:

--8<---------------cut here---------------start------------->8---
(defvar srb-font-lock-keywords
  (list
   '(srb-date-time-matcher (1 font-lock-warning-face t t))
   '(srb-date-time-matcher (2 font-lock-warning-face t t))))

(defun srb-date-time-matcher (lim)
  "Search for date-time strings within LIM for font-locking."
  (when (re-search-forward (concat "^\\(?1:[0-9][0-9][0-9][0-9]-"
  				   "[0-9][0-9]-[0-9][0-9]\\)"
  				   " \\(?2:[0-9]?[0-9]:[0-9][0-9]\\)?")
                           lim t)
    (let* ((date (match-string-no-properties 1))
    	   (time (match-string-no-properties 2))
	   ;; days-between needs a non-empty time string.
    	   (date-time (concat date " " (or time "00:00"))))
      (when (< (days-between date-time (current-time-string)) 0)
	(concat date " " time)))))

(define-derived-mode srb-mode nil "SRB" ()
  "Mode for testing font-locking."
  (set (make-local-variable 'font-lock-defaults)
       '(srb-font-lock-keywords t)))
--8<---------------cut here---------------end--------------->8---

However, if the buffer contains a string that matches the regexp but
fails the second when-condition, then not only is it -- correctly -- not
fontified but subsequent matching dates that pass the when-condition are
also -- incorrectly -- not fontified.  Here's a test file showing the
problem:

--8<---------------cut here---------------start------------->8---
2012-04-01 10:00 
2012-05-02 
2013-05-01 10:00 
2012-03-10 10:00 
2012-04-01 10:00 
2012-05-02 
2012-03-10 10:00 
2012-04-01 10:00 
2012-05-02 
2012-03-10 10:00 

Local Variables:
mode: srb
End:
--8<---------------cut here---------------end--------------->8---

When I evaluate the code above and then visit this file, I see only the
first two dates in font-lock-warning-face.  However, if I modify the
buffer at one of the dates after the 2013 date that fails the
when-condition, e.g. by typing a space, then that date becomes fontified
(and maybe the following one too, this seems to vary).

(With my actual code and data, some but not all dates after a correctly
non-fontified date are correctly fontified, though I've found no pattern
to explain this; but I hope a solution to the above case will also take
care of this wrinkle.)

I suspect the problem has to do with this (info (elisp)Search-based
Fontification):

     Fontification will call FUNCTION repeatedly with the same limit,
     and with point where the previous invocation left it, until
     FUNCTION fails.  On failure, FUNCTION need not reset point in any
     particular way.

I tried resetting point on failure, but that either didn't have an
effect or it resulted in all dates being fontified.

So how can I get all and only dates that satisfy the when-condition
fontified?

Steve Berman

Re: Conditionalized font-locking?

[Davis Herring]

> (defun srb-date-time-matcher (lim)
>   "Search for date-time strings within LIM for font-locking."
>   (when (re-search-forward (concat "^\\(?1:[0-9][0-9][0-9][0-9]-"
>   				   "[0-9][0-9]-[0-9][0-9]\\)"
>   				   " \\(?2:[0-9]?[0-9]:[0-9][0-9]\\)?")
>                            lim t)
>     (let* ((date (match-string-no-properties 1))
>     	   (time (match-string-no-properties 2))
> 	   ;; days-between needs a non-empty time string.
>     	   (date-time (concat date " " (or time "00:00"))))
>       (when (< (days-between date-time (current-time-string)) 0)
> 	(concat date " " time)))))
> [...]
>      Fontification will call FUNCTION repeatedly with the same limit,
>      and with point where the previous invocation left it, until
>      FUNCTION fails.  On failure, FUNCTION need not reset point in any
>      particular way.

If any of your searches turns up a date you don't like, your function
fails (returns nil), so font-lock gives up.  You need to (unfortunately)
write much the same loop that font-lock is wrapping around your function
so that it finds the first date that it cares about rather than
returning nil.

The easy way to do this if you don't have too many dates being fontified
at one go is just to recursively call yourself when you don't like your
current match:

  (or (< ...) (srb-date-time-matcher lim))

Note that you needn't return any particular non-nil value on success;
it's the match-data that font-lock uses.

Davis

-- 
This product is sold by volume, not by mass.  If it appears too dense or
too sparse, it is because mass-energy conversion has occurred during
shipping.

Re: Conditionalized font-locking?

[Stephen Berman]

On Thu, 10 May 2012 09:30:17 -0600 Davis Herring <herring@lanl.gov> wrote:

>> (defun srb-date-time-matcher (lim)
>>   "Search for date-time strings within LIM for font-locking."
>>   (when (re-search-forward (concat "^\\(?1:[0-9][0-9][0-9][0-9]-"
>>   				   "[0-9][0-9]-[0-9][0-9]\\)"
>>   				   " \\(?2:[0-9]?[0-9]:[0-9][0-9]\\)?")
>>                            lim t)
>>     (let* ((date (match-string-no-properties 1))
>>     	   (time (match-string-no-properties 2))
>> 	   ;; days-between needs a non-empty time string.
>>     	   (date-time (concat date " " (or time "00:00"))))
>>       (when (< (days-between date-time (current-time-string)) 0)
>> 	(concat date " " time)))))
>> [...]
>>      Fontification will call FUNCTION repeatedly with the same limit,
>>      and with point where the previous invocation left it, until
>>      FUNCTION fails.  On failure, FUNCTION need not reset point in any
>>      particular way.
>
> If any of your searches turns up a date you don't like, your function
> fails (returns nil), so font-lock gives up.  You need to (unfortunately)
> write much the same loop that font-lock is wrapping around your function
> so that it finds the first date that it cares about rather than
> returning nil.

Thank you, that was the part I didn't quite understand.

> The easy way to do this if you don't have too many dates being fontified
> at one go is just to recursively call yourself when you don't like your
> current match:
>
>   (or (< ...) (srb-date-time-matcher lim))
>
> Note that you needn't return any particular non-nil value on success;
> it's the match-data that font-lock uses.

This works perfectly, thanks very much!

Steve Berman