Re: will we ever have zero width assertions in regexps?

[Stefan Monnier <monnier@iro.umontreal.ca>]

>>> So you have a REx which is matched against a line, but you want (in
>>> addition to the usual effects of matching) to know whether it "wanted"
>>> the match to overflow into the following line?
>> 
>>> If so, it looks like "reusing the continuation state" would not be a
>>> serious optimization - it would add just a small multiplicative
>>> constant to the "use only the hypothetical bit" scenario...
>> 
>> We could probably make it work with just that extra bit, indeed.
>> But with the full intermediate state, we get to just "start the search
>> with last line's state" instead of having to "start the search from the
>> previous N lines since they all ended with the <wantmore> bit set", so
>> it will happily work with many-lines cases without having to reparse
>> those many lines N times.

> Hmm, I thought about a different scenario: if the bit is set, then one
> switches to a DIFFERENT REx designed for a multi-line case.  Otherwise
> why not just run it against the rest of the buffer, instead of
> one-line?

Because we don't want to match those regexps against the whole buffer
every time the buffer is modified (the buffer may be large).

Also it can be tricky to match only some of the font-lock regexps
against the whole buffer, since font-lock-keywords is normally defined
with the assumption that the regexps are applied in turn, that earlier
ones prevent subsequent ones from being applied and that we start with
a fresh un-highlighted buffer.  So in general, if we want to apply one
of the font-lock-keywords to the whole buffer, we have to do it for all
of them.

BTW, another reason to want a non-backtracking matcher can be seen in
the recent thread "Stack overflow in regexp matcher".


        Stefan