* bug#36092: 27.0.50; incorrect highlighting in nXML mode with ' and raw > characters
@ 2019-06-04 15:04 Vincent Lefevre
2019-06-05 1:14 ` Noam Postavsky
0 siblings, 1 reply; 5+ messages in thread
From: Vincent Lefevre @ 2019-06-04 15:04 UTC (permalink / raw)
To: 36092
On the following file, lines 26 and below are not highlighted
(i.e. they get the normal color). This is reproducible with
and without the -Q option.
<?xml version="1.0" encoding="utf-8"?>
<root>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx</a>
<a>
<element>
'>xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
</element>
</a>
</root>
In GNU Emacs 27.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.5)
of 2019-06-04 built on cventin
Repository revision: dd7bc5de3f59237f21e1c4b70f0ba97549ea1fb4
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12004000
System Description: Debian GNU/Linux 10 (buster)
Recent messages:
Loading /home/vlefevre/share/emacs/site-lisp/mutteditor.el (source)...done
Loading time...done
For information about GNU Emacs and the GNU system, type C-h C-a.
Using vacuous schema [2 times]
Error during redisplay: (jit-lock-function 1583) signaled (cl-assertion-failed (>= (cadr sgml--syntax-propertize-ppss) 0))
Configured using:
'configure --prefix=/home/vlefevre/opt/emacs-master'
Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND GPM DBUS GSETTINGS GLIB NOTIFY INOTIFY
ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM THREADS LIBSYSTEMD PDUMPER LCMS2
GMP
Important settings:
value of $LC_COLLATE: POSIX
value of $LC_CTYPE: en_US.UTF-8
value of $LC_TIME: en_DK
value of $LANG: POSIX
locale-coding-system: utf-8-unix
Major mode: nXML
Minor modes in effect:
display-time-mode: t
show-paren-mode: t
tooltip-mode: t
global-eldoc-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
column-number-mode: t
line-number-mode: t
transient-mark-mode: t
Load-path shadows:
None found.
Features:
(shadow sort mail-extr warnings emacsbug message rmc puny dired
dired-loaddefs format-spec rfc822 mml mml-sec password-cache epa derived
epg epg-config gnus-util rmail rmail-loaddefs text-property-search
time-date mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils vc-dispatcher vc-svn rng-xsd xsd-regexp rng-cmpct
rng-nxml rng-valid rng-loc rng-uri rng-parse nxml-parse rng-match rng-dt
rng-util rng-pttrn nxml-ns easymenu nxml-mode nxml-outln nxml-rap
sgml-mode seq byte-opt gv bytecomp byte-compile cconv dom nxml-util
nxml-enc xmltok elec-pair time cus-start cus-load paren cc-styles
cc-align cc-engine cc-vars cc-defs edmacro kmacro cl-loaddefs cl-lib
mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty make-network-process emacs)
Memory information:
((conses 16 81866 9310)
(symbols 48 10247 1)
(strings 32 25966 1934)
(string-bytes 1 826776)
(vectors 16 13514)
(vector-slots 8 158635 8782)
(floats 8 35 19)
(intervals 56 465 0)
(buffers 992 14))
^ permalink raw reply [flat|nested] 5+ messages in thread
* bug#36092: 27.0.50; incorrect highlighting in nXML mode with ' and raw > characters
2019-06-04 15:04 bug#36092: 27.0.50; incorrect highlighting in nXML mode with ' and raw > characters Vincent Lefevre
@ 2019-06-05 1:14 ` Noam Postavsky
2019-06-05 1:48 ` Stefan Monnier
0 siblings, 1 reply; 5+ messages in thread
From: Noam Postavsky @ 2019-06-05 1:14 UTC (permalink / raw)
To: Vincent Lefevre; +Cc: Stefan Monnier, 36092
[-- Attachment #1: Type: text/plain, Size: 573 bytes --]
Vincent Lefevre <vincent@vinc17.net> writes:
> On the following file, lines 26 and below are not highlighted
> (i.e. they get the normal color). This is reproducible with
> and without the -Q option.
It's hitting an assertion failure during syntax-propertizing. The
problem seems to be that the parse-partial-sexp call in
sgml--syntax-propertize-ppss isn't using the right syntax table. I
wonder why it doesn't happen for sgml-mode though (as far I can tell).
Stefan, any thoughts about that?
Patch below fixes it, but it might just be papering over another
problem.
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 2602 bytes --]
From ed59843b7fa456e0614c2c63e1479458642671ea Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Tue, 4 Jun 2019 21:08:26 -0400
Subject: [PATCH] Fix nxml syntax propertizing assertion failure (Bug#36092)
* lisp/textmodes/sgml-mode.el (sgml-syntax-propertize): Use the
syntax-ppss-table syntax-table.
* test/lisp/nxml/nxml-mode-tests.el (nxml-mode->-after-quote): New
test.
---
lisp/textmodes/sgml-mode.el | 17 +++++++++--------
test/lisp/nxml/nxml-mode-tests.el | 15 +++++++++++++++
2 files changed, 24 insertions(+), 8 deletions(-)
diff --git a/lisp/textmodes/sgml-mode.el b/lisp/textmodes/sgml-mode.el
index 0c5d5e56a6..09211e1661 100644
--- a/lisp/textmodes/sgml-mode.el
+++ b/lisp/textmodes/sgml-mode.el
@@ -399,14 +399,15 @@ (eval-and-compile
(defun sgml-syntax-propertize (start end)
"Syntactic keywords for `sgml-mode'."
- (setq sgml--syntax-propertize-ppss (cons start (syntax-ppss start)))
- (cl-assert (>= (cadr sgml--syntax-propertize-ppss) 0))
- (sgml-syntax-propertize-inside end)
- (funcall
- (syntax-propertize-rules sgml-syntax-propertize-rules)
- start end)
- ;; Catch any '>' after the last quote.
- (sgml--syntax-propertize-ppss end))
+ (with-syntax-table syntax-ppss-table
+ (setq sgml--syntax-propertize-ppss (cons start (syntax-ppss start)))
+ (cl-assert (>= (cadr sgml--syntax-propertize-ppss) 0))
+ (sgml-syntax-propertize-inside end)
+ (funcall
+ (syntax-propertize-rules sgml-syntax-propertize-rules)
+ start end)
+ ;; Catch any '>' after the last quote.
+ (sgml--syntax-propertize-ppss end)))
(defun sgml-syntax-propertize-inside (end)
(let ((ppss (syntax-ppss)))
diff --git a/test/lisp/nxml/nxml-mode-tests.el b/test/lisp/nxml/nxml-mode-tests.el
index 70816bb9de..53416b4280 100644
--- a/test/lisp/nxml/nxml-mode-tests.el
+++ b/test/lisp/nxml/nxml-mode-tests.el
@@ -99,5 +99,20 @@ (ert-deftest nxml-mode-prolog-comment ()
(should (nth 4 (syntax-ppss)))
(search-forward "comment3")))
+(ert-deftest nxml-mode->-after-quote ()
+ "Reduction from Bug#36092."
+ (with-temp-buffer
+ (insert "<root>\n"
+ (make-string 1794 ?a) "\n"
+ "'>"
+ (make-string 196 ?a) "\n"
+ "</root>")
+ (nxml-mode)
+ (syntax-propertize 2001)
+ (syntax-propertize (point-max)) ; Triggered an assert failure.
+ ;; Check that last tag is parsed as a tag.
+ (should (= 1 (- (car (syntax-ppss (1- (point-max))))
+ (car (syntax-ppss (point-max))))))))
+
(provide 'nxml-mode-tests)
;;; nxml-mode-tests.el ends here
--
2.11.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* bug#36092: 27.0.50; incorrect highlighting in nXML mode with ' and raw > characters
2019-06-05 1:14 ` Noam Postavsky
@ 2019-06-05 1:48 ` Stefan Monnier
2019-06-05 11:28 ` Vincent Lefevre
0 siblings, 1 reply; 5+ messages in thread
From: Stefan Monnier @ 2019-06-05 1:48 UTC (permalink / raw)
To: Noam Postavsky; +Cc: Vincent Lefevre, 36092
> It's hitting an assertion failure during syntax-propertizing. The
> problem seems to be that the parse-partial-sexp call in
> sgml--syntax-propertize-ppss isn't using the right syntax table.
Hmm...
> (defun sgml-syntax-propertize (start end)
> "Syntactic keywords for `sgml-mode'."
> - (setq sgml--syntax-propertize-ppss (cons start (syntax-ppss start)))
> - (cl-assert (>= (cadr sgml--syntax-propertize-ppss) 0))
> - (sgml-syntax-propertize-inside end)
> - (funcall
> - (syntax-propertize-rules sgml-syntax-propertize-rules)
> - start end)
> - ;; Catch any '>' after the last quote.
> - (sgml--syntax-propertize-ppss end))
> + (with-syntax-table syntax-ppss-table
> + (setq sgml--syntax-propertize-ppss (cons start (syntax-ppss start)))
> + (cl-assert (>= (cadr sgml--syntax-propertize-ppss) 0))
> + (sgml-syntax-propertize-inside end)
> + (funcall
> + (syntax-propertize-rules sgml-syntax-propertize-rules)
> + start end)
> + ;; Catch any '>' after the last quote.
> + (sgml--syntax-propertize-ppss end)))
OMG!
We use syntax-ppss-table in syntax-ppss but not in syntax-propertize!
How dumb could I be!
Of course, it usually works fine because syntax-propertize is usually
called from syntax-ppss, but ... well ... not always,
I just install the patch below into `master` to address that.
Stefan
diff --git a/lisp/emacs-lisp/syntax.el b/lisp/emacs-lisp/syntax.el
index 9c6d5b5829..b6cba44e77 100644
--- a/lisp/emacs-lisp/syntax.el
+++ b/lisp/emacs-lisp/syntax.el
@@ -283,6 +283,9 @@ syntax-propertize-via-font-lock
;; In case it was eval'd/compiled.
(setq keywords font-lock-syntactic-keywords)))))
+(defvar-local syntax-ppss-table nil
+ "Syntax-table to use during `syntax-ppss', if any.")
+
(defun syntax-propertize (pos)
"Ensure that syntax-table properties are set until POS (a buffer point)."
(when (< syntax-propertize--done pos)
@@ -301,47 +304,48 @@ syntax-propertize
#'syntax-ppss-flush-cache 99 t))
(save-excursion
(with-silent-modifications
- (make-local-variable 'syntax-propertize--done) ;Just in case!
- (let* ((start (max (min syntax-propertize--done (point-max))
- (point-min)))
- (end (max pos
- (min (point-max)
- (+ start syntax-propertize-chunk-size))))
- (funs syntax-propertize-extend-region-functions))
- (while funs
- (let ((new (funcall (pop funs) start end))
- ;; Avoid recursion!
- (syntax-propertize--done most-positive-fixnum))
- (if (or (null new)
- (and (>= (car new) start) (<= (cdr new) end)))
- nil
- (setq start (car new))
- (setq end (cdr new))
- ;; If there's been a change, we should go through the
- ;; list again since this new position may
- ;; warrant a different answer from one of the funs we've
- ;; already seen.
- (unless (eq funs
- (cdr syntax-propertize-extend-region-functions))
- (setq funs syntax-propertize-extend-region-functions)))))
- ;; Flush ppss cache between the original value of `start' and that
- ;; set above by syntax-propertize-extend-region-functions.
- (syntax-ppss-flush-cache start)
- ;; Move the limit before calling the function, so the function
- ;; can use syntax-ppss.
- (setq syntax-propertize--done end)
- ;; (message "syntax-propertizing from %s to %s" start end)
- (remove-text-properties start end
- '(syntax-table nil syntax-multiline nil))
- ;; Avoid recursion!
- (let ((syntax-propertize--done most-positive-fixnum))
- (funcall syntax-propertize-function start end))))))))
+ (with-syntax-table syntax-ppss-table
+ (make-local-variable 'syntax-propertize--done) ;Just in case!
+ (let* ((start (max (min syntax-propertize--done (point-max))
+ (point-min)))
+ (end (max pos
+ (min (point-max)
+ (+ start syntax-propertize-chunk-size))))
+ (funs syntax-propertize-extend-region-functions))
+ (while funs
+ (let ((new (funcall (pop funs) start end))
+ ;; Avoid recursion!
+ (syntax-propertize--done most-positive-fixnum))
+ (if (or (null new)
+ (and (>= (car new) start) (<= (cdr new) end)))
+ nil
+ (setq start (car new))
+ (setq end (cdr new))
+ ;; If there's been a change, we should go through the
+ ;; list again since this new position may
+ ;; warrant a different answer from one of the funs we've
+ ;; already seen.
+ (unless (eq funs
+ (cdr syntax-propertize-extend-region-functions))
+ (setq funs syntax-propertize-extend-region-functions)))))
+ ;; Flush ppss cache between the original value of `start' and that
+ ;; set above by syntax-propertize-extend-region-functions.
+ (syntax-ppss-flush-cache start)
+ ;; Move the limit before calling the function, so the function
+ ;; can use syntax-ppss.
+ (setq syntax-propertize--done end)
+ ;; (message "syntax-propertizing from %s to %s" start end)
+ (remove-text-properties start end
+ '(syntax-table nil syntax-multiline nil))
+ ;; Avoid recursion!
+ (let ((syntax-propertize--done most-positive-fixnum))
+ (funcall syntax-propertize-function start end)))))))))
;;; Link syntax-propertize with syntax.c.
(defvar syntax-propertize-chunks
;; We're not sure how far we'll go. In my tests, using chunks of 2000
- ;; brings to overhead to something negligible. Passing ‘charpos’ directly
+ ;; brings the overhead to something negligible. Passing ‘charpos’ directly
;; also works (basically works line-by-line) but results in an overhead which
;; I thought was a bit too high (like around 50%).
2000)
@@ -450,9 +454,6 @@ syntax-ppss--update-stats
(cl-incf (car pair))
(cl-incf (cdr pair) (- new old))))
-(defvar-local syntax-ppss-table nil
- "Syntax-table to use during `syntax-ppss', if any.")
-
(defun syntax-ppss--data ()
(if (eq (point-min) 1)
(progn
^ permalink raw reply related [flat|nested] 5+ messages in thread
* bug#36092: 27.0.50; incorrect highlighting in nXML mode with ' and raw > characters
2019-06-05 1:48 ` Stefan Monnier
@ 2019-06-05 11:28 ` Vincent Lefevre
2019-06-05 13:14 ` Noam Postavsky
0 siblings, 1 reply; 5+ messages in thread
From: Vincent Lefevre @ 2019-06-05 11:28 UTC (permalink / raw)
To: Stefan Monnier; +Cc: Noam Postavsky, 36092
On 2019-06-04 21:48:19 -0400, Stefan Monnier wrote:
> I just install the patch below into `master` to address that.
Thanks. Everything seems fine now.
--
Vincent Lefèvre <vincent@vinc17.net> - Web: <https://www.vinc17.net/>
100% accessible validated (X)HTML - Blog: <https://www.vinc17.net/blog/>
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)
^ permalink raw reply [flat|nested] 5+ messages in thread
* bug#36092: 27.0.50; incorrect highlighting in nXML mode with ' and raw > characters
2019-06-05 11:28 ` Vincent Lefevre
@ 2019-06-05 13:14 ` Noam Postavsky
0 siblings, 0 replies; 5+ messages in thread
From: Noam Postavsky @ 2019-06-05 13:14 UTC (permalink / raw)
To: Vincent Lefevre; +Cc: Stefan Monnier, 36092
tags 36092 fixed
close 36092
quit
Vincent Lefevre <vincent@vinc17.net> writes:
> On 2019-06-04 21:48:19 -0400, Stefan Monnier wrote:
>> I just install the patch below into `master` to address that.
>
> Thanks. Everything seems fine now.
Okay, adding just the test case from patch my above and closing.
7ad5e3915f 2019-06-05T09:10:23-04:00 "; Add test for Bug#36092"
https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=7ad5e3915fe0fad7c8053b53cab1aa7aa74193fe
dec148939a 2019-06-05T07:29:23-04:00 "Do not use syntax-ppss-table in syntax-propertize when nil (Bug#36095)"
https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=dec148939aad8391875e73784af2bcb0a01100bb
0b3982b1a3 2019-06-04T21:48:06-04:00 "* lisp/emacs-lisp/syntax.el: Use syntax-ppss-table for syntax-propertize."
https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=0b3982b1a3892486fd9e4916b9cfafa12ddd9137
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2019-06-05 13:14 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-06-04 15:04 bug#36092: 27.0.50; incorrect highlighting in nXML mode with ' and raw > characters Vincent Lefevre
2019-06-05 1:14 ` Noam Postavsky
2019-06-05 1:48 ` Stefan Monnier
2019-06-05 11:28 ` Vincent Lefevre
2019-06-05 13:14 ` Noam Postavsky
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.