On Sun, Jun 26, 2022 at 5:17 AM Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> I *think* I've fixed this, but it's complicated.  Also I could be
> completely wrong.  For what it's worth, I can reproduce the bug without the
> patch and cannot with the patch, which see attached.

AFAICT you've indeed found the origin of the problem.

> If this sounds sensible, then a slightly different patch is needed, because
> `electric-pair--with-uncached-syntax` is used in some contexts where hiding
> `syntax-propertize-function` is the correct behavior.

I think the code deserves a comment when/where it overrides
`syntax-propertize-function` to explain why it's needed.
AFAICT it was introduced in commit
89cfdbf729bc731331358e0efc69547547aa3ca2 but that commit doesn't explain
why it bound it to nil (which I later changed to `ignore`).

Furthermore, the cache could be filled with entries before `start` while
the syntax-table  (and/or `syntax-propertize-function`) is temporarily
changed, so the flush doesn't seem sufficient.  [ It's unlikely, because
usually the cache will have been pre-filled via font-lock and friends,
but it can still occur in corner cases.  ]

IIUC we use `with-syntax-table` there specifically when we want to
provide text-mode style paren matching within comments and strings.
Maybe a good way to avoid problem with syntax-ppss/properties is to
narrow the buffer to the comment/string at the same time as we
`with-syntax-table` and let-bind `syntax-propertize-function`.

Thanks.  Two observations:

FIrst, changing `syntax-propertize-function` from nil to `ignore` was wrong IIUC.  If the function is set, then it is wholly responsible for applying syntax table.  When set to nil the default behavior is used, but when set to `ignore`, that should mean that no syntax is applied at all.  In practice, I don't know what behavior that causes.
 
Second, since `electric-pair--with-uncached-syntax` appears to be used for doing text-mode matching (as you've also observed), maybe we should de-generalize it to do only that.  I think that allows us to make some simplifying assumptions about the state of the world.


        Stefan