Re: /srv/bzr/emacs/trunk r101338: * lisp/emacs-lisp/syntax.el (syntax-ppss): More sanity check to catch

[Stefan Monnier <monnier@iro.umontreal.ca>]

>> IIRC the reason was so that (syntax-ppss) called from a place like
>> a "column-0 just before an open paren" can correctly indicate if this
>> open paren is within a string/comment while at the same time parsing the
>> subsequent text assuming that this open paren was outside of any
>> string/comment.
> Why? That sounds confusing.

That's the point: it makes it possible to find those sources of
confusion and highlight them.

E.g. I had some font-lock code which would highlight an
open-paren-in-column-0-in-string/comment with the `warning' face.
So such an "incorrect" open paren would still cause incorrect
highlighting, but the `warning' face on it would provide the clue as to
what was the source of the problem.

> I would've bound `syntax-ppss-last` and `syntax-ppss-cache` to nil around
> the region and let it be forced to call the syntax-begin-function.

Right, but that largely defeats the purpose of syntax-ppss (which is to
use caching to speed up (parse-partial-sexp (point-min) (point))).

To give you some background: I think syntax-begin-function is basically
useless.  It's used in very few places (it used to be used in lisp-mode,
but that was disabled recently, it's still used in js-mode, but it
should probably be disabled there as well, and apparently mmm-mode also
uses it, but these are the only cases I know) and is more trouble than
it's worth.  It was meant and is designed as an optimization, but it is
vanishingly often useful.

For your use case, you don't want to provide an optimization.  We should
instead provide a new hook whose use would be mandatory (i.e. which
syntax.el has to obey).

I'm not completely sure what it could/should look like, tho.

One option is to have a hook that takes a (POS . PPSS) pair, which
syntax-ppss intends to use as a starting point for parsing, and return
a new such pair to use instead, where the returned position should
always be >= POS.

This way, syntax-ppss could make full use of its cache, but mmm-mode
could tell it about chunk boundaries (and decide what state to use at
the beginning of a boundary).

The main problem I see with this approach is that this hook would be
called maybe too many times, so we'd want to improve the "fast path"
(i.e. the first branch in syntax-ppss which tries to use
syntax-ppss-last) so it can know when calling this new hook is unneeded.

Maybe for that, the new hook should return not just a new (POS . PPSS)
but also a "next-boundary" so we know we don't need to call this hook
again as long as we're within POS...NEXT-BOUNDARY.


        Stefan