Re: Clarification needed: syntax-propertize vs font-lock-syntax-table

[Dmitry Gutov <dgutov@yandex.ru>]

On 11/23/2015 07:05 PM, Wedler, Christoph wrote:

> Well, I realized that (with Emacs-24.4) the imenu list in antlr-mode
> wasn't correctly before I put (syntax-propertize (point-max)) into
> antlr-imenu-create-index-function.  I could imagine similar things in
> other modes.

In other functions, you mean. Yes, various facilities that depend on 
syntax properties should call syntax-propertize at appropriate times.

Note that calling (syntax-propertize (point-max)), as js-mode was doing, 
is not a guarantee of the file staying entirely up-to-date WRT syntax 
properties as the user edits it.

Neither is using font-lock: we fontify the file lazily, and thus also 
only propertize the buffer near its visible part.

> I probably define a variable for that which will be t for Emacs below
> emacs-25.x and nil with the next release.

But nothing has changed in Emacs 25 in that regard (I think?). We 
changed js-mode behavior, but mostly because it was a kludge anyway, and 
not because anything else is compensating for it now.

> The other documentation request was for font-lock syntax-table as being
> "syntax-ppss-compatible" (as you put it nicely).

I'd be happy to review the documentation patch. We can also ask Stefan 
to do that.

> Agreed.  How about a function like
>
> (defun syntax-ppss-open-list-positions (ppss)
>    "Get all syntactically open list position found in a syntactic scan.
> PPSS is a scan state, as returned by `parse-partial-sexp' or `syntax-ppss'.
> The start position of the outermost list comes first."
>    (nth 9 ppss))
>
> This function needs to be changed if the internal structure of (nth 9
> ppss) is changed.

Yes, that sounds good (maybe choose a shorter name?). But see above.

> We might need something (an extra syntax flag?) which makes it easy to
> "rewind" the ppss such that
>
>    (equal (syntax-ppss inner-mode-end-position)
>           (syntax-ppss inner-mode-start-position))

That sounds like it might require turning the current "list of openers" 
structure into a list-of-lists, where each element is the current "list 
of openers", plus some boundary information.

Or maybe handle that in some entirely different way, like keeping the 
current spss format, but allow the buffers to override the syntax-ppss 
logic via a newly introduced syntax-ppss-function variable.

Someone could try implementing the latter approach right now, using advice.