bug#22983: syntax-ppss returns wrong result.

[Alan Mackenzie <acm@muc.de>]

Hello, Dmitry.

On Sat, Sep 09, 2017 at 12:44:02 +0300, Dmitry Gutov wrote:
> Hi Alan,

> On 9/7/17 11:45 PM, Alan Mackenzie wrote:

> > The solution I propose is to introduce a second cache into syntax-ppss,
> > and this cache would be used whenever (not (eq (point-min) 1)).
> > Whenever point-min changes, and isn't 1, this second cached would be
> > calculated again from scratch.

> Thanks for writing this up. I think it's a good step, and since it 
> follow the current wording of the docstring, it should be highly 
> compatible with the existing code.

Thanks.

> > This proposal has these advantages:

> > (i) It would make the function deliver what its unchanged doc string
> > says.  This is important, given that syntax-ppss has been very widely
> > used within Emacs, and likely by external packages too; these will
> > typically have assumed the advertised behaviour of the function, without
> > having tested it in narrowed buffers.

> It will also continue to function as expected in mmm-mode, AFAICT, 
> without the need for an "escape hatch" we discussed before.

> > (i) In the case which currently works, namely a non-narrowed buffer,
> > there would be only a minute slow-down (basically, there would be extra
> > code to check point-min and select the cache to use).

> > (ii) The cache for use in a narrowed buffer might well be sufficiently
> > fast in normal use.  If it is not, it could be enhanced readily.

> And since the API doesn't change, and the observable behavior doesn't 
> either (in the vast majority of cases; probably all except the broken 
> ones), we can refine this solution easily, or even swap it for something 
> else, with little cost.

Yes.  I now have a provisional implementation of this new strategy,
which works on the test case for xdisp.c with which I opened the bug.
It seems to be working, generally.  I need to test it more thoroughly.

In the implementation, I have left the function `syntax-ppss' untouched
except for adding a function call to set up the cache right at the
start.  I have refactored syntax-ppss-flush-cache, extracting a function
which is called directly from the cache-selecting code.  Other than
that, there is one new function (which switches the current cache in
use) and a few new variables to keep track of the caches.

> > I think Dmitry also proposed a method of solution some months ago,
> > though I don't remember in detail what it was.  Dmitry, do you still
> > think your solution would work?  If so, please elaborate on it.

> There is a simple patch at 
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=22983#47, but I after some 
> consideration, I now prefer your proposed approach. We've also had some 
> grander ideas about enhancing things further, but those can be added 
> later, after we finally decide.

Yes, I agree.

> I do want to know what Stefan thinks of this subject now, though.

Yes.

> Caveats:

> - This solves the dependency on point-min, but does nothing about the 
> dependency on the current syntax-table (which can change). I'm not 
> necessarily suggesting we try to solve that now, though.

I had some ideas on this back in the spring (about having "indirect
variables") which could be used quickly to "swap out" the current
syntax-table text properties, and (more importantly) quickly swap them
back in.  But that's for another day.

> - Before this change is pushed to master, or shortly after, I'd like to 
> know that it actually fixed the problem Philipp experienced with 
> python-mode, so we can revert 4fbd330. If it was caused by e.g. 
> syntax-table changing, we've not improved much.

I am naturally interested in this, too.  If my patch doesn't fix this
bug, at least it will have removed a layer of fog inhibiting its
investigation.

> All the best,
> Dmitry.

-- 
Alan Mackenzie (Nuremberg, Germany).