bug#31290: Fundamental bugs in syntax-propertize

["Andreas Röhler" <andreas.roehler@easy-emacs.de>]

On 12.05.2018 13:26, Alan Mackenzie wrote:
> Hello, Dmitry.
> 
> On Tue, May 08, 2018 at 15:35:14 +0300, Dmitry Gutov wrote:
>> On 4/28/18 12:08 AM, Alan Mackenzie wrote:
> 
>>> At least that would be true if syntax-propertize--done hadn't been
>>> prematurely and spuriously increased, crudely to prevent an infinite
>>> recursion, falsely indicating to the syntax-ppss infrastructure that the
>>> syntax-table properties have already been applied to the region (BEGIN
>>> END).
> 
>>>       .... but it should not call `syntax-ppss-flush-cache', ....
> 
>>> Why not?  Because syntax-ppss-flush-cache sets syntax-propertize--done
>>> back to its true value, allowing the wrongly allowed syntax-ppss calls at
>>> a later position to cause a recursive loop.
> 
>> Maybe we should "allow" it to loop, in certain cases? Leaving it to be
>> the responsibility of the programmer, to make sure the result doesn't
>> infloop, even if these rules are violated.
> 
> I'm not sure how this could work.  We would need to formalise the rules
> very carefully, to avoid the need to read syntax.{c,el}'s source code.
> 
>>>       .... which means that it should not call `syntax-ppss' on some
>>>       position and later modify the buffer on some earlier position.
> 
>>> This is a bad restriction, because sometimes syntax-table properties can
>>> only be correctly determined by examining the syntax of later buffer
>>> positions.  An example of this is giving the string-fence syntax-table
>>> text property to an unbalanced opening string quote, but not to correctly
>>> matched quotes.
> 
>> I'm not exactly convinced by the given example (why would we use the
>> string-fence in that case?), but it might be better if something like
>> this was possible, indeed.
> 
> String fence can be used to signal to font lock that the delimiter
> (together with the "mismatching" unescaped EOL) should be fontified in
> warning face.
> 
> A better example might be C++ Mode's marking of a "< ... >" pair with
> paren syntax.  This isn't done with syntax-propertize-function (as you
> know), but it would be nice if this were possible.
> 
>>> 2. syntax-propertize-function's are banned from using syntax-ppss, the
>>> documentation instead directing them to use parse-partial-sexp directly.
> 
>> The ones that currently call syntax-ppss, can't simply switch over to
>> parse-partial-sexp without becoming slower due to the lack of cache.
> 
> The cache at the pertinent buffer position doesn't exist at the time:
> consistent syntax-table properties aren't on the preceding buffer
> positions.
> 
>> Before tackling this bug, I'd rather we see a real-world problem that it
>> caused, and pick a particular approach based on it.
> 
> My enhancements for bug#30393: "24.4; cperl-mode: indentation failure -
> Documentation enhancements", where (almost) any change which affects the
> syntactic state is programmed to call syntax-ppss-flush-cache from the C
> level, clashes with the mechanism in this bug report.  Most of the time
> it's fine, but when a change affecting the syntactic state is made from
> inside a synax-propertize-function, Emacs goes into an infinite recursive
> loop.
> 
> This isn't good.
> 
>> But off the top of my head, we could introduce a "stricter but somewhat
>> slower" variation of syntax-ppss to be called inside
>> syntax-propertize-function's, which would treat the values in question
>> more carefully, somehow.
> 
> That's an idea worth exploring.
> 

Hi folks,

from what I've seen month ago just may stress the term fundamental.
Gave up to follow details WRT to check-ins made.

That part needs some person treating the gordic knot according to its 
quality...