From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Alan Mackenzie Newsgroups: gmane.emacs.bugs Subject: bug#36136: [PATCH]: Re: bug#36136: syntax-ppss fails to invalidate its cache on changes to syntax-table text properties Date: Sun, 9 Jun 2019 18:39:12 +0000 Message-ID: <20190609183912.GA17965@ACM> References: <20190608131724.GA4643@ACM> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="160061"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Mutt/1.10.1 (2018-07-13) Cc: Stefan Monnier To: 36136@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sun Jun 09 20:40:46 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1ha2kI-000fNQ-Be for geb-bug-gnu-emacs@m.gmane.org; Sun, 09 Jun 2019 20:40:46 +0200 Original-Received: from localhost ([::1]:37444 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1ha2kB-0008Ri-Gu for geb-bug-gnu-emacs@m.gmane.org; Sun, 09 Jun 2019 14:40:39 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:50467) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1ha2je-0008NR-2m for bug-gnu-emacs@gnu.org; Sun, 09 Jun 2019 14:40:08 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ha2jb-0008Uy-Kq for bug-gnu-emacs@gnu.org; Sun, 09 Jun 2019 14:40:05 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:41831) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ha2ja-0008UI-2y for bug-gnu-emacs@gnu.org; Sun, 09 Jun 2019 14:40:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ha2jZ-0008KW-Vm for bug-gnu-emacs@gnu.org; Sun, 09 Jun 2019 14:40:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Alan Mackenzie Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 09 Jun 2019 18:40:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36136 X-GNU-PR-Package: emacs Original-Received: via spool by 36136-submit@debbugs.gnu.org id=B36136.156010556031966 (code B ref 36136); Sun, 09 Jun 2019 18:40:01 +0000 Original-Received: (at 36136) by debbugs.gnu.org; 9 Jun 2019 18:39:20 +0000 Original-Received: from localhost ([127.0.0.1]:55375 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ha2it-0008JW-Ea for submit@debbugs.gnu.org; Sun, 09 Jun 2019 14:39:19 -0400 Original-Received: from colin.muc.de ([193.149.48.1]:49163 helo=mail.muc.de) by debbugs.gnu.org with smtp (Exim 4.84_2) (envelope-from ) id 1ha2ip-0008JK-4R for 36136@debbugs.gnu.org; Sun, 09 Jun 2019 14:39:16 -0400 Original-Received: (qmail 59666 invoked by uid 3782); 9 Jun 2019 18:39:13 -0000 Original-Received: from acm.muc.de (p2E5D56FA.dip0.t-ipconnect.de [46.93.86.250]) by colin.muc.de (tmda-ofmipd) with ESMTP; Sun, 09 Jun 2019 20:39:12 +0200 Original-Received: (qmail 18017 invoked by uid 1000); 9 Jun 2019 18:39:12 -0000 Content-Disposition: inline In-Reply-To: <20190608131724.GA4643@ACM> X-Delivery-Agent: TMDA/1.1.12 (Macallan) X-Primary-Address: acm@muc.de X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:160294 Archived-At: On Sat, Jun 08, 2019 at 13:17:24 +0000, Alan Mackenzie wrote: > Hello, Emacs. > The syntax-ppss cache is not invalidated when syntax-table text > properties are set or cleared. This is because the invalidation > function, syntax-ppss-flush-cache is invoked only as a before-change > function, but typical (?all) syntax-table property changes happen when > before-change-functions is inactive. > This is a bug. > In my debugging of a CC Mode scenario, a buffer change causes a > syntax-table text property change at an earlier part of the buffer. This > is to do with the change making previously non-matching C++ raw string > identifiers match up. Font lock follows the syntax-ppss state, which > spuriously records that the end part of the buffer is still in a string. > Hence the non-string part of the buffer is still fontified with > font-lock-string-face. > Suggested fix: the functions set_properties, add_properties, > remove_properties in textprop.c should check for changes to, > specifically, syntax-table properties. When these changes are detected, > a hook called something like syntax-table-props-change-alert-hook should > be called (with some appropriate position parameters, tbd). > syntax-ppss-flush-cache will be added to this hook. > -- > Alan Mackenzie (Nuremberg, Germany). The following patch is simpler than my first proposal, following feedback from Eli. It works for me. Stefan, could you look at this, please? Make syntax-ppss react to changes in syntax-table text properties In particular, it trims its cache to the point of such a change. This fixes bug #36136. * src/textprop.c (syntax-propertize--done): New buffer local variable. (set_properties, add_properties, remove_properties): when a syntax-table text property is being changed, reduce syntax-propertize--done to the buffer position. * lisp/emacs-lisp/syntax.el (syntax-ppss--trim-cache): New function extracted from syntax-ppss-flush-cache. (syntax-ppss-flush-cache): Now only modifies syntax-propertize--done and syntax-ppss--done. (syntax-ppss): Calls syntax-ppss--trim-cache and sets syntax-propertize--done. diff --git a/lisp/emacs-lisp/syntax.el b/lisp/emacs-lisp/syntax.el index 9c6d5b5829..6ca27d8e83 100644 --- a/lisp/emacs-lisp/syntax.el +++ b/lisp/emacs-lisp/syntax.el @@ -404,13 +404,10 @@ syntax-ppss-narrow (defvar-local syntax-ppss-narrow-start nil "Start position of the narrowing for `syntax-ppss-narrow'.") -(define-obsolete-function-alias 'syntax-ppss-after-change-function - #'syntax-ppss-flush-cache "27.1") -(defun syntax-ppss-flush-cache (beg &rest ignored) - "Flush the cache of `syntax-ppss' starting at position BEG." - ;; Set syntax-propertize to refontify anything past beg. - (setq syntax-propertize--done (min beg syntax-propertize--done)) - ;; Flush invalid cache entries. +(defun syntax-ppss--trim-cache (beg) + "Flush the cache specific to `syntax-ppss' from position BEG. + +This doesn't include the cache for `syntax-propertize'." (dolist (cell (list syntax-ppss-wide syntax-ppss-narrow)) (pcase cell (`(,last . ,cache) @@ -432,8 +429,16 @@ syntax-ppss-flush-cache ;; (unless cache ;; (remove-hook 'before-change-functions #'syntax-ppss-flush-cache t)) (setcar cell last) - (setcdr cell cache))) - )) + (setcdr cell cache))))) + +(define-obsolete-function-alias 'syntax-ppss-after-change-function + #'syntax-ppss-flush-cache "27.1") +(defun syntax-ppss-flush-cache (beg &rest ignored) + "Flush the cache of `syntax-ppss' starting at position BEG." + ;; Set syntax-propertize to refontify anything past beg. + (setq syntax-propertize--done (min beg syntax-propertize--done)) + ;; Flush invalid cache entries. + (setq syntax-ppss--done (min beg syntax-ppss--done))) ;;; FIXME: Explain this variable. Currently only its last (5th) slot is used. ;;; Perhaps the other slots should be removed? @@ -477,8 +482,10 @@ syntax-ppss running the hook." ;; Default values. (unless pos (setq pos (point))) + (syntax-ppss--trim-cache syntax-ppss--done) (syntax-propertize pos) ;; + (setq syntax-ppss--done (max syntax-ppss--done pos)) (with-syntax-table (or syntax-ppss-table (syntax-table)) (let* ((cell (syntax-ppss--data)) (ppss-last (car cell)) diff --git a/src/textprop.c b/src/textprop.c index ae42c44185..be9e34c889 100644 --- a/src/textprop.c +++ b/src/textprop.c @@ -334,6 +334,9 @@ set_properties (Lisp_Object properties, INTERVAL interval, Lisp_Object object) record_property_change (interval->position, LENGTH (interval), XCAR (sym), XCAR (value), object); + if (EQ (sym, Qsyntax_table) + && (interval->position < syntax_ppss__done)) + syntax_ppss__done = interval->position; } /* For each new property that has no value at all in the old plist, @@ -346,6 +349,9 @@ set_properties (Lisp_Object properties, INTERVAL interval, Lisp_Object object) record_property_change (interval->position, LENGTH (interval), XCAR (sym), Qnil, object); + if (EQ (sym, Qsyntax_table) + && (interval->position < syntax_ppss__done)) + syntax_ppss__done = interval->position; } } @@ -400,6 +406,9 @@ add_properties (Lisp_Object plist, INTERVAL i, Lisp_Object object, { record_property_change (i->position, LENGTH (i), sym1, Fcar (this_cdr), object); + if (EQ (sym1, Qsyntax_table) + && (i->position < syntax_ppss__done)) + syntax_ppss__done = i->position; } /* I's property has a different value -- change it */ @@ -436,6 +445,9 @@ add_properties (Lisp_Object plist, INTERVAL i, Lisp_Object object, { record_property_change (i->position, LENGTH (i), sym1, Qnil, object); + if (EQ (sym1, Qsyntax_table) + && (i->position < syntax_ppss__done)) + syntax_ppss__done = i->position; } set_interval_plist (i, Fcons (sym1, Fcons (val1, i->plist))); changed = true; @@ -470,9 +482,14 @@ remove_properties (Lisp_Object plist, Lisp_Object list, INTERVAL i, Lisp_Object while (CONSP (current_plist) && EQ (sym, XCAR (current_plist))) { if (BUFFERP (object)) - record_property_change (i->position, LENGTH (i), - sym, XCAR (XCDR (current_plist)), - object); + { + record_property_change (i->position, LENGTH (i), + sym, XCAR (XCDR (current_plist)), + object); + if (EQ (sym, Qsyntax_table) + && (i->position < syntax_ppss__done)) + syntax_ppss__done = i->position; + } current_plist = XCDR (XCDR (current_plist)); changed = true; @@ -486,8 +503,13 @@ remove_properties (Lisp_Object plist, Lisp_Object list, INTERVAL i, Lisp_Object if (CONSP (this) && EQ (sym, XCAR (this))) { if (BUFFERP (object)) - record_property_change (i->position, LENGTH (i), - sym, XCAR (XCDR (this)), object); + { + record_property_change (i->position, LENGTH (i), + sym, XCAR (XCDR (this)), object); + if (EQ (sym, Qsyntax_table) + && (i->position < syntax_ppss__done)) + syntax_ppss__done = i->position; + } Fsetcdr (XCDR (tail2), XCDR (XCDR (this))); changed = true; @@ -2319,6 +2341,11 @@ inherits it if NONSTICKINESS is nil. The `front-sticky' and Vtext_property_default_nonsticky = list2 (Fcons (Qsyntax_table, Qt), Fcons (Qdisplay, Qt)); + DEFVAR_INT ("syntax-ppss--done", syntax_ppss__done, + doc: /* Position up to which the `syntax-ppss' cache is valid. */); + syntax_ppss__done = -1; + Fmake_variable_buffer_local (intern ("syntax_ppss__done")); + interval_insert_behind_hooks = Qnil; interval_insert_in_front_hooks = Qnil; staticpro (&interval_insert_behind_hooks); -- Alan Mackenzie (Nuremberg, Germany).