bug#192: regexp does not work as documented

[Stefan Monnier <monnier@iro.umontreal.ca>]

> My latest plan is to do what Chong Yidong suggests, setting up text
> properties so that font-lock DTRT, though it doesn't seem as hard as he
> suggests (I'm still in the naive enthusiasm stage).

Indeed, it shouldn't be that hard.

> I tried adding the font-lock-multiline property to the face property
> list passed to font lock and that did the trick, even with the
> font-lock-multiline variable nil.

That may not be enough.  You'll probably want to do something like what
smerge does:

  (while (re-search-forward <RE> nil t)
    (font-lock-fontify-region (match-beginning 0) (match-end 0)))

this will find all the multiline elements.  And the font-lock-multiline
property you add will make sure that those that were found will not
disappear accidentally because of some later refontification.

> I rather do that than turn on font-lock-multiline because I'm assuming
> that font-lock-multiline is set to nil in most cases for
> a good reason.

Setting the `font-lock-multiline' variable to t has a performance cost.

> I actually thought about properly parsing the regexp, but the effort to
> do that could be spent on making multi-line patterns work properly, at
> least if they don't span too many lines.

If someone wants that, I have a parser that takes a regexp and turns it
into something like `rx' syntax.  It uses my lex.el library (which
takes an `rx'-like input syntax).

> One more thing, multi-line regexp matches don't work properly even
> with font-lock-multiline t when jit-lock is being used in a buffer
> without syntactic fontification and using the default setting of
> jit-lock-contextually, setting it to t gets multi-line fontification
> to work.

The `font-lock-multiline' variable only tells font-lock that if it ever
bumps into a multiline element, it should mark it (with the
font-lock-multiline property) so that it will not re-fontify it as
a whole if it ever needs to refontify it.

So it doesn't solve the problem of "how do I make sure that font-lock
indeed finds the multiline element".  Multiline elements can only be
found when font-locking a large enough piece of text, which tends to
only happen during the initial fontification, or during background or
contextual refontification, or during an explicit call such as in the
above while loop.


        Stefan