Re: regexp font-lock highlighting

[martin rudalics <rudalics@gmx.at>]

 >     Note that I only fixed the endless looping behavior here.  My code is
 >     not suitable when you use a support mode like jit-lock and change the
 >     delimiters of regexp groups.
 >
 > I don't understand that last sentence.  What does it mean to "change
 > the delimiters of regexp groups"?  Could you please show us a concrete
 > test case that still has a problem--and describe the problem exactly?
 >

Sorry, I couldn't understand that sentence either.  It's not so easy to
show a concrete test case since the problem depends on the particular
settings of font-lock and jit-lock.  Consider the following stretch from
`tex-mode.el'.

   (setq paragraph-start
	(concat "[ \t]*\\(\\$\\$\\|"
		"\\\\[][]\\|"
		"\\\\" (regexp-opt (append
				    (mapcar 'car latex-section-alist)
				    '("begin" "label" "end"
				      "item" "bibitem" "newline" "noindent"
				      "newpage" "footnote" "marginpar"
				      "parbox" "caption")) t)
		"\\>\\|\\\\[a-z]*" (regexp-opt '("space" "skip" "page") t)
		"\\>\\)"))

Here the innermost group delimited by "\\(" and "\\)" stretches from the
second to the last line and should get painted accordingly.  If you now
delete the "\\)" the painting should disappear.  If you next undo the
change, the painting generally doesn't reappear since font-lock
refontifies from the line preceding the one where the change occurred -
with `font-lock-lines-before' at its default value - only.  The reason
for this is documented in the Elisp manual:


    *Warning:* Do not design an element of `font-lock-keywords' to match
text which spans lines; this does not work reliably.  While
`font-lock-fontify-buffer' handles multi-line patterns correctly,
updating when you edit the buffer does not, since it considers text one
line at a time.  If you have patterns that typically only span one line
but can occasionally span two or three, such as `<title>...</title>',
you can ask Font Lock to be more careful by setting
`font-lock-multiline' to `t'.  But it still will not work in all cases.


I could avoid painting regexp groups that span multiple lines, but this
would deprive me from highlighting many interesting things.  I could set
`font-lock-lines-before' to a higher value but this would slow down
fontification in general and still not handle all cases.  Anyway, I
believe that I found a solution that does the right thing in most cases
and will send it to you in the next days.