Re: Idea for syntax-ppss. Is it new? Could it be any good?

[Stefan Monnier <monnier@iro.umontreal.ca>]

>> Isn't that what syntax-ppss does?
> It caches the state for several positions, but I don't think they're at
> regular positions.

C-h v syntax-ppss-max-span

It's not exactly perfectly regular, but I don't think the
difference matters.

> I don't understand the detailed workings of the routine at the moment.
> I suspect that the slowness of all the lisp manipulation will outweigh
> the benefit of the caching, but I would confirm or refute that with
> the profiler before doing anything serious.

> partial-parse-sexp is blindingly fast.  To scan an entire 3Mb C buffer
> on my elderly 1.2 GHz Athlon takes 0.27s.  That is why I suspect that
> the lisping in syntax-ppss might need severe optimisation.  But again,
> it's only a hunch.

When I wrote syntax-ppss, my main goal was to never be significantly
slower than parse-partial-sexp.  Even if it's not as fast as it could be
if written in C (which is pretty much obviously true), that's not
a reason to recode it in C.

> What I think really needs doing is to make this function bulletproof: It
> should work on narrowed buffers,

That can be done, tho it needs extra info in order to know how to
interpret the fact that it's narrowed.

> it should give reliable elements 2 and 6,

If you really care about them, then I recommend you fix it in
parse-partial-sexp.

> its cache should be cleared when functions like `modify-syntax-entry'
> are called or parse-sexp-lookup-properties is changed, and the cache
> should be bound to nil on `with-syntax-table'.  I actually think it
> could be useful to maintain several parallel caches, each for a
> different syntax-table (or an equivalence class of syntax tables).  And
> so on.  Basically, I would like `(syntax-ppss)' to tell me with 100%
> reliability, no ifs, no buts, whether I am at top-level, in a comment,
> or in a string.

I think this will result in too many cache flushes and will make the
code too intrusive or too ad-hoc.  I'd rather have
a syntax-ppss-syntax-table (and force parse-sexp-lookup-properties to t)
if you want more reliable results.

> Also, Lennart is asking for it to work nicely with multiple major modes.
> Surely this would be a Good Thing.  Files containing several major modes
> are commonplace (awk or sed embedded within a shell script, html
> embedded within php, ....).

Yes, that's a desirable extension.

> At the moment, CC Mode applies a heuristic maximum size of strings and
> comments, for performance reasons.  Checking for strings and comments is
> done so frequently that the mode uses elaborate internal caches.  It
> would be nice if this cacheing could move to the Emacs core.

You can do it today.  Have you even tried to use syntax-ppss before
asking for it to be improved?

> Again, this isn't something which can be implemented in a weekend, but I
> think it would be worthwhile for Emacs 24.

Other than the multi-major-mode part, it all sounds like very
minor changes.


        Stefan