bug#40844: html mode sometimes fooled by apostrophe

[Stephen Berman <stephen.berman@gmx.net>]

On Sun, 13 Jun 2021 14:21:36 +0200 Lars Ingebrigtsen <larsi@gnus.org> wrote:

> Stephen Berman <stephen.berman@gmx.net> writes:
>
>> I made a silly mistake (it was late and I was tired).  Here is a
>> corrected version:
>
> I can confirm that this patch solves the test cases here.

Thanks for checking.

>> With this patch, when any of the paired-bracket characters is followed
>> by `'' in html-mode, there is indeed no string face fontification on the
>> latter (and following characters).  The following function demonstrates
>> this:
>
> [...]
>
>> I wanted to turn this function into a test, and that's what the
>> commented out lines are supposed to do.  But when I uncomment these
>> lines and call this function with the unpatched (i.e. current) version
>> of sgml-mode-syntax-table, it still shows default face for `'' with all
>> the paired-bracket characters.  Yet when I step through the function
>> with Ediff, I do see some cases with font-lock-string-face.  I don't
>> understand what's going on here.
>
> Might be a timing issue, perhaps?

I tried adding sit-for at different points but it made no difference.

> In any case, the patch is an improvement, so perhaps that should be
> pushed anyway?

Upthread Eli said "some SGML/HTML expert should say if that is TRT".
I'm no such expert so I can't make that decision.  FWIW, I rewrote the
test using ert, and the result is as above: it passes with the patch, as
expected, but also without the patch, even though in the latter case the
test buffer clearly contains characters fontified with
font-lock-string-face.  And just as I wrote above, when stepping through
the ert-deftest using the unpatched sgml-tag-syntax-table, the test does
fail as expected.  Here's the test, in case someone else wants to see if
they can figure it out; I haven't succeeded:

(ert-deftest sgml-test-brackets ()
  "Test fontification of apostrophe preceded by paired-bracket character."
  (let ((buf (get-buffer-create "*sgml-test*"))
	brackets results)
    (map-char-table
     (lambda (key value)
       (setq brackets (cons (list
			     (if (consp key)
				 (list (car key) (cdr key))
			       key)
			     value)
			    brackets)))
     (unicode-property-table-internal 'paired-bracket))
    (setq brackets (delete-dups (flatten-tree brackets)))
    (setq brackets (append brackets (list ?$ ?% ?& ?* ?+ ?/)))
    (with-current-buffer buf
      (erase-buffer)
      (fundamental-mode)
      (while brackets
	(let ((char (string (pop brackets))))
	  (insert (concat "<p>" char "'s</p>\n"))))
      (html-mode)
      (goto-char (point-min))
      (while (not (eobp))
	(goto-char (next-single-char-property-change (point) 'face))
	(let ((val (get-text-property (point) 'face)))
	  (when val
	    (should-not (eq val 'font-lock-string-face))))))))

Steve Berman