From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.bugs Subject: bug#36092: 27.0.50; incorrect highlighting in nXML mode with ' and raw > characters Date: Tue, 04 Jun 2019 21:48:19 -0400 Message-ID: References: <87zhmxie7n.fsf@vinc17.net> <874l544yva.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="148952"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) Cc: Vincent Lefevre , 36092@debbugs.gnu.org To: Noam Postavsky Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Wed Jun 05 03:49:19 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hYL3F-000cVJ-4z for geb-bug-gnu-emacs@m.gmane.org; Wed, 05 Jun 2019 03:49:17 +0200 Original-Received: from localhost ([127.0.0.1]:34004 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hYL3E-0001so-6A for geb-bug-gnu-emacs@m.gmane.org; Tue, 04 Jun 2019 21:49:16 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:58207) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hYL34-0001sY-KY for bug-gnu-emacs@gnu.org; Tue, 04 Jun 2019 21:49:07 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hYL32-0001pc-Kb for bug-gnu-emacs@gnu.org; Tue, 04 Jun 2019 21:49:06 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:60834) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hYL30-0001jz-OP for bug-gnu-emacs@gnu.org; Tue, 04 Jun 2019 21:49:04 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hYL2z-0006JT-Ts for bug-gnu-emacs@gnu.org; Tue, 04 Jun 2019 21:49:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Stefan Monnier Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 05 Jun 2019 01:49:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36092 X-GNU-PR-Package: emacs Original-Received: via spool by 36092-submit@debbugs.gnu.org id=B36092.155969931324231 (code B ref 36092); Wed, 05 Jun 2019 01:49:01 +0000 Original-Received: (at 36092) by debbugs.gnu.org; 5 Jun 2019 01:48:33 +0000 Original-Received: from localhost ([127.0.0.1]:46144 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hYL2W-0006Ik-SO for submit@debbugs.gnu.org; Tue, 04 Jun 2019 21:48:33 -0400 Original-Received: from mailscanner.iro.umontreal.ca ([132.204.25.50]:8386) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hYL2S-0006IV-Lc for 36092@debbugs.gnu.org; Tue, 04 Jun 2019 21:48:30 -0400 Original-Received: from pmg2.iro.umontreal.ca (localhost.localdomain [127.0.0.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 10C6F811DE; Tue, 4 Jun 2019 21:48:23 -0400 (EDT) Original-Received: from mail01.iro.umontreal.ca (unknown [172.31.2.1]) by pmg2.iro.umontreal.ca (Proxmox) with ESMTP id 3D2108116D; Tue, 4 Jun 2019 21:48:21 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=iro.umontreal.ca; s=mail; t=1559699301; bh=6BPwdNl2t26mT0DxPCS555BKwhVJx0q0dszsN/qRL9o=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=B6QDqozhzJUJr5La3XBLSvD2oi+YzJCyE0Y7KaVdzdL5bcSgd1Y9Xg5Gir9ko4Wmc CAe2IJx8XT+DfN/SI2ikJQDtlPD24GhQLga5zCpcO44EimEyT+bzr23fri80lcke3b QoE/ZzCx71bzKSFz2vIfMaXkPE+Qjo9aq2XQVhyCl+AqQY+xw0BIkSaTyzHhv0U9fD R125wdRwOiiiH3w5uP6wRSlZWCvP8pWRgqKghZXhIjd2xgZ9VuZEWzX++XcaRkL3an QaRLHjB7B5Mge+MHvRJ6eju3NIOnfReDL4x8c+YUR/+xa01+zfzsg3WFxu0cLZmqDF CrkvbwxcG67Sg== Original-Received: from alfajor (unknown [45.72.243.188]) by mail01.iro.umontreal.ca (Postfix) with ESMTPSA id 9BB341203AB; Tue, 4 Jun 2019 21:48:20 -0400 (EDT) In-Reply-To: <874l544yva.fsf@gmail.com> (Noam Postavsky's message of "Tue, 04 Jun 2019 21:14:33 -0400") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:160130 Archived-At: > It's hitting an assertion failure during syntax-propertizing. The > problem seems to be that the parse-partial-sexp call in > sgml--syntax-propertize-ppss isn't using the right syntax table. Hmm... > (defun sgml-syntax-propertize (start end) > "Syntactic keywords for `sgml-mode'." > - (setq sgml--syntax-propertize-ppss (cons start (syntax-ppss start))) > - (cl-assert (>=3D (cadr sgml--syntax-propertize-ppss) 0)) > - (sgml-syntax-propertize-inside end) > - (funcall > - (syntax-propertize-rules sgml-syntax-propertize-rules) > - start end) > - ;; Catch any '>' after the last quote. > - (sgml--syntax-propertize-ppss end)) > + (with-syntax-table syntax-ppss-table > + (setq sgml--syntax-propertize-ppss (cons start (syntax-ppss start))) > + (cl-assert (>=3D (cadr sgml--syntax-propertize-ppss) 0)) > + (sgml-syntax-propertize-inside end) > + (funcall > + (syntax-propertize-rules sgml-syntax-propertize-rules) > + start end) > + ;; Catch any '>' after the last quote. > + (sgml--syntax-propertize-ppss end))) OMG! We use syntax-ppss-table in syntax-ppss but not in syntax-propertize! How dumb could I be! Of course, it usually works fine because syntax-propertize is usually called from syntax-ppss, but ... well ... not always, I just install the patch below into `master` to address that. Stefan diff --git a/lisp/emacs-lisp/syntax.el b/lisp/emacs-lisp/syntax.el index 9c6d5b5829..b6cba44e77 100644 --- a/lisp/emacs-lisp/syntax.el +++ b/lisp/emacs-lisp/syntax.el @@ -283,6 +283,9 @@ syntax-propertize-via-font-lock ;; In case it was eval'd/compiled. (setq keywords font-lock-syntactic-keywords))))) =20 +(defvar-local syntax-ppss-table nil + "Syntax-table to use during `syntax-ppss', if any.") + (defun syntax-propertize (pos) "Ensure that syntax-table properties are set until POS (a buffer point)." (when (< syntax-propertize--done pos) @@ -301,47 +304,48 @@ syntax-propertize #'syntax-ppss-flush-cache 99 t)) (save-excursion (with-silent-modifications - (make-local-variable 'syntax-propertize--done) ;Just in case! - (let* ((start (max (min syntax-propertize--done (point-max)) - (point-min))) - (end (max pos - (min (point-max) - (+ start syntax-propertize-chunk-size)))) - (funs syntax-propertize-extend-region-functions)) - (while funs - (let ((new (funcall (pop funs) start end)) - ;; Avoid recursion! - (syntax-propertize--done most-positive-fixnum)) - (if (or (null new) - (and (>=3D (car new) start) (<=3D (cdr new) end))) - nil - (setq start (car new)) - (setq end (cdr new)) - ;; If there's been a change, we should go through the - ;; list again since this new position may - ;; warrant a different answer from one of the funs we've - ;; already seen. - (unless (eq funs - (cdr syntax-propertize-extend-region-functio= ns)) - (setq funs syntax-propertize-extend-region-functions))= ))) - ;; Flush ppss cache between the original value of `start' and = that - ;; set above by syntax-propertize-extend-region-functions. - (syntax-ppss-flush-cache start) - ;; Move the limit before calling the function, so the function - ;; can use syntax-ppss. - (setq syntax-propertize--done end) - ;; (message "syntax-propertizing from %s to %s" start end) - (remove-text-properties start end - '(syntax-table nil syntax-multiline ni= l)) - ;; Avoid recursion! - (let ((syntax-propertize--done most-positive-fixnum)) - (funcall syntax-propertize-function start end)))))))) + (with-syntax-table syntax-ppss-table + (make-local-variable 'syntax-propertize--done) ;Just in case! + (let* ((start (max (min syntax-propertize--done (point-max)) + (point-min))) + (end (max pos + (min (point-max) + (+ start syntax-propertize-chunk-size)))) + (funs syntax-propertize-extend-region-functions)) + (while funs + (let ((new (funcall (pop funs) start end)) + ;; Avoid recursion! + (syntax-propertize--done most-positive-fixnum)) + (if (or (null new) + (and (>=3D (car new) start) (<=3D (cdr new) end)= )) + nil + (setq start (car new)) + (setq end (cdr new)) + ;; If there's been a change, we should go through the + ;; list again since this new position may + ;; warrant a different answer from one of the funs we'= ve + ;; already seen. + (unless (eq funs + (cdr syntax-propertize-extend-region-funct= ions)) + (setq funs syntax-propertize-extend-region-functions= ))))) + ;; Flush ppss cache between the original value of `start' an= d that + ;; set above by syntax-propertize-extend-region-functions. + (syntax-ppss-flush-cache start) + ;; Move the limit before calling the function, so the functi= on + ;; can use syntax-ppss. + (setq syntax-propertize--done end) + ;; (message "syntax-propertizing from %s to %s" start end) + (remove-text-properties start end + '(syntax-table nil syntax-multiline = nil)) + ;; Avoid recursion! + (let ((syntax-propertize--done most-positive-fixnum)) + (funcall syntax-propertize-function start end))))))))) =20 ;;; Link syntax-propertize with syntax.c. =20 (defvar syntax-propertize-chunks ;; We're not sure how far we'll go. In my tests, using chunks of 2000 - ;; brings to overhead to something negligible. Passing =E2=80=98charpos= =E2=80=99 directly + ;; brings the overhead to something negligible. Passing =E2=80=98charpo= s=E2=80=99 directly ;; also works (basically works line-by-line) but results in an overhead = which ;; I thought was a bit too high (like around 50%). 2000) @@ -450,9 +454,6 @@ syntax-ppss--update-stats (cl-incf (car pair)) (cl-incf (cdr pair) (- new old)))) =20 -(defvar-local syntax-ppss-table nil - "Syntax-table to use during `syntax-ppss', if any.") - (defun syntax-ppss--data () (if (eq (point-min) 1) (progn