bug#49629: 27.2; electric-pair-mode doesn't work for angle brackets in HTML file

[Allen Li <darkfeline@felesatra.moe>]

[-- Attachment #1.1: Type: text/plain, Size: 3592 bytes --]

I *think* I've fixed this, but it's complicated.  Also I could be
completely wrong.  For what it's worth, I can reproduce the bug without the
patch and cannot with the patch, which see attached.

`electric-pair--with-uncached-syntax` hides `syntax-propertize-function`,
and `mhtml-mode` uses `syntax-propertize-function`. AFAIU, let-binding
`syntax-propertize-function` may or may not clear the cached syntax applied
by said function, leading to the current heisenbug.

If this sounds sensible, then a slightly different patch is needed, because
`electric-pair--with-uncached-syntax` is used in some contexts where hiding
`syntax-propertize-function` is the correct behavior.

See second attached patch for an attempt at this approach.

+Noam Postavsky <npostavs@gmail.com> since they added
`electric-pair--with-uncached-syntax`

On Sun, Jun 26, 2022 at 1:41 AM Allen Li <darkfeline@felesatra.moe> wrote:

> On Thu, Dec 9, 2021 at 2:31 AM Allen Li <darkfeline@felesatra.moe> wrote:
>
>> I have bisected this regression
>> to 7fff418edf56244a1fcf54718523aa9b5cb3a854
>>
>> I will cc Stefan on the miniscule chance he still remembers anything
>> about this and can save me time.
>> Otherwise, I will see if I can pinpoint the regression (or if I messed up
>> the bisect).
>>
>
> Posting an update (or non-update) on this.  This regression did seem to be
> introduced by 7fff418edf56244a1fcf54718523aa9b5cb3a854, however I'm pretty
> sure there's nothing wrong with the commit itself, it's just that jiggling
> the code around made this regression reliably occur.
>
>
>>
>> Author: Stefan Monnier <monnier@iro.umontreal.ca>
>> Date:   Fri Nov 29 11:51:48 2019 -0500
>>     * lisp/textmodes/mhtml-mode.el: Fix bug#38372
>>
>>     The `sgml-syntax-propertize-rules` rely on the
>> `sgml--syntax-propertize-ppss`
>>     setup by `sgml-syntax-propertize` so it is not correct/safe to use
>>     them directly like html used to do.
>>
>>     Change `sgml-syntax-propertize` so it can be used by mhtml,
>>     and then adjust mhtml-mode accordingly.
>>
>>     * lisp/textmodes/mhtml-mode.el: Remove redundant `eval-and-compile`.
>>     Only require cl-lib at compile-time.
>>     (mhtml--syntax-propertize): New const, extracted from
>> mhtml-syntax-propertize.
>>     (mhtml-syntax-propertize): Use `sgml-syntax-propertize`.
>>
>>     * lisp/textmodes/sgml-mode.el (sgml--syntax-propertize): New const,
>>     extracted from sgml-syntax-propertize.
>>     (sgml-syntax-propertize): Add optional `rules-function` arg.
>>  lisp/textmodes/mhtml-mode.el | 44
>> ++++++++++++++++++++------------------------
>>  lisp/textmodes/sgml-mode.el  | 13 ++++++++-----
>>  2 files changed, 28 insertions(+), 29 deletions(-)
>>
>> On Sun, Aug 1, 2021 at 10:41 AM Lars Ingebrigtsen <larsi@gnus.org> wrote:
>>
>>> Allen Li <darkfeline@felesatra.moe> writes:
>>>
>>> > Thus, I resorted to actually trying to understand the code.
>>>
>>> Darn, I hate it when that happens.
>>>
>>> > I've tracked down the bug to unexpected behavior from the `scan-sexps`
>>> > call in this part of `electric-pair--balance-info`:
>>>
>>> I can reproduce exactly what you're seeing -- when edebugging, the
>>> problem goes away, etc.
>>>
>>> I put a
>>>
>>>               (redisplay t)
>>>
>>> into the function, and that also made the problem go away, but that's as
>>> far as I've gotten so far.  So the problem does indeed seem to be
>>> something related to a cache/table somewhere not having been updated...
>>>
>>> --
>>> (domestic pets only, the antidote for overdose, milk.)
>>>    bloggy blog: http://lars.ingebrigtsen.no
>>>
>>

[-- Attachment #1.2: Type: text/html, Size: 5031 bytes --]

[-- Attachment #2: 0001-Fix-regression.patch --]
[-- Type: text/x-patch, Size: 861 bytes --]

From 5dd42c108b49097624c1637a562e83953752b0cf Mon Sep 17 00:00:00 2001
From: Allen Li <darkfeline@felesatra.moe>
Date: Sun, 26 Jun 2022 01:47:19 -0700
Subject: [PATCH] Fix regression

---
 lisp/elec-pair.el | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/lisp/elec-pair.el b/lisp/elec-pair.el
index 964d21f11c..b94eab066f 100644
--- a/lisp/elec-pair.el
+++ b/lisp/elec-pair.el
@@ -246,8 +246,7 @@ electric-pair--with-uncached-syntax
 cache is flushed from position START, defaulting to point."
   (declare (debug ((form &optional form) body)) (indent 1))
   (let ((start-var (make-symbol "start")))
-    `(let ((syntax-propertize-function #'ignore)
-           (,start-var ,(or start '(point))))
+    `(let ((,start-var ,(or start '(point))))
        (unwind-protect
            (with-syntax-table ,table
              ,@body)
-- 
2.36.1


[-- Attachment #3: 0001-elec-pair-Fix-bug-incorrectly-hiding-syntax-properti.patch --]
[-- Type: text/x-patch, Size: 3595 bytes --]

From 95c20919bb69d46e6f44a9e330c78e6f1d4a6a8b Mon Sep 17 00:00:00 2001
From: Allen Li <darkfeline@felesatra.moe>
Date: Sun, 26 Jun 2022 02:32:47 -0700
Subject: [PATCH] elec-pair: Fix bug incorrectly hiding
 syntax-propertize-function

Notably, this causes electric-pair-mode to often misbehave in HTML
files when pairing angle brackets.
---
 lisp/elec-pair.el | 22 ++++++++++++++--------
 1 file changed, 14 insertions(+), 8 deletions(-)

diff --git a/lisp/elec-pair.el b/lisp/elec-pair.el
index 964d21f11c..915a6d6333 100644
--- a/lisp/elec-pair.el
+++ b/lisp/elec-pair.el
@@ -237,16 +237,21 @@ electric-pair--insert
         (electric-layout-allow-duplicate-newlines t))
     (self-insert-command 1)))
 
-(cl-defmacro electric-pair--with-uncached-syntax ((table &optional start) &rest body)
+(cl-defmacro electric-pair--with-uncached-syntax ((table &optional start func) &rest body)
   "Like `with-syntax-table', but flush the `syntax-ppss' cache afterwards.
 Use this instead of (with-syntax-table TABLE BODY) when BODY
 contains code which may update the `syntax-ppss' cache.  This
 includes calling `parse-partial-sexp' and any sexp-based movement
 functions when `parse-sexp-lookup-properties' is non-nil.  The
-cache is flushed from position START, defaulting to point."
-  (declare (debug ((form &optional form) body)) (indent 1))
-  (let ((start-var (make-symbol "start")))
-    `(let ((syntax-propertize-function #'ignore)
+cache is flushed from position START, defaulting to point.
+FUNC is an optional function to set to `syntax-propertize-function'."
+  (declare (debug ((form &optional form form) body)) (indent 1))
+  (let ((start-var (make-symbol "start"))
+        (func-var (make-symbol "func")))
+    `(let ((syntax-propertize-function (let ((,func-var ,func))
+                                         (if ,func-var
+                                             ,func-var
+                                           syntax-propertize-function)))
            (,start-var ,(or start '(point))))
        (unwind-protect
            (with-syntax-table ,table
@@ -272,7 +277,7 @@ electric-pair--syntax-ppss
                               (point)))))
     (if s-or-c-start
         (electric-pair--with-uncached-syntax (electric-pair-text-syntax-table
-                                              s-or-c-start)
+                                              s-or-c-start #'ignore)
           (parse-partial-sexp s-or-c-start pos))
       ;; HACK! cc-mode apparently has some `syntax-ppss' bugs
       (if (memq major-mode '(c-mode c++ mode))
@@ -326,7 +331,7 @@ electric-pair--balance-info
                                (condition-case nil
                                    (eq (char-after pos)
                                        (electric-pair--with-uncached-syntax
-                                           (table)
+                                           (table (point) (if string-or-comment #'ignore))
                                          (matching-paren
                                           (char-before
                                            (scan-sexps (point) 1)))))
@@ -356,7 +361,8 @@ electric-pair--balance-info
     (save-excursion
       (while (not outermost)
         (condition-case err
-            (electric-pair--with-uncached-syntax (table)
+            (electric-pair--with-uncached-syntax
+                (table (point) (if string-or-comment #'ignore))
               (scan-sexps (point) (if (> direction 0)
                                       (point-max)
                                     (- (point-max))))
-- 
2.36.1