From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Alan Mackenzie Newsgroups: gmane.emacs.bugs Subject: bug#22983: [ Patch ] Re: bug#22983: syntax-ppss returns wrong result. Date: Sun, 10 Sep 2017 11:36:26 +0000 Message-ID: <20170910113626.GB3588@ACM> References: <9fc66395-045c-1984-f530-033c2ff706f6@yandex.ru> <83h8wlz1kf.fsf@gnu.org> <20170902174027.GB4267@ACM> <20170907204502.GC4488@ACM> <69e034d3-7a52-cc81-dc56-e5308ad5dce0@yandex.ru> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: blaine.gmane.org 1505043741 9264 195.159.176.226 (10 Sep 2017 11:42:21 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sun, 10 Sep 2017 11:42:21 +0000 (UTC) User-Agent: Mutt/1.7.2 (2016-11-26) Cc: John Wiegley , 22983@debbugs.gnu.org To: Dmitry Gutov , Philipp Stephani Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sun Sep 10 13:42:15 2017 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dr0ct-0001u3-Og for geb-bug-gnu-emacs@m.gmane.org; Sun, 10 Sep 2017 13:42:11 +0200 Original-Received: from localhost ([::1]:52526 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dr0cz-00034b-Bb for geb-bug-gnu-emacs@m.gmane.org; Sun, 10 Sep 2017 07:42:17 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:40394) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dr0cp-00032w-R6 for bug-gnu-emacs@gnu.org; Sun, 10 Sep 2017 07:42:09 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dr0ck-0008NA-ON for bug-gnu-emacs@gnu.org; Sun, 10 Sep 2017 07:42:07 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:49813) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dr0ck-0008N4-Je for bug-gnu-emacs@gnu.org; Sun, 10 Sep 2017 07:42:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1dr0ck-0004qf-Di for bug-gnu-emacs@gnu.org; Sun, 10 Sep 2017 07:42:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Alan Mackenzie Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sun, 10 Sep 2017 11:42:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 22983 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 22983-submit@debbugs.gnu.org id=B22983.150504369018598 (code B ref 22983); Sun, 10 Sep 2017 11:42:02 +0000 Original-Received: (at 22983) by debbugs.gnu.org; 10 Sep 2017 11:41:30 +0000 Original-Received: from localhost ([127.0.0.1]:58494 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1dr0cE-0004pt-FJ for submit@debbugs.gnu.org; Sun, 10 Sep 2017 07:41:30 -0400 Original-Received: from ocolin.muc.de ([193.149.48.4]:23792 helo=mail.muc.de) by debbugs.gnu.org with smtp (Exim 4.84_2) (envelope-from ) id 1dr0cC-0004pk-TW for 22983@debbugs.gnu.org; Sun, 10 Sep 2017 07:41:29 -0400 Original-Received: (qmail 80459 invoked by uid 3782); 10 Sep 2017 11:41:27 -0000 Original-Received: from acm.muc.de (p548C6F29.dip0.t-ipconnect.de [84.140.111.41]) by colin.muc.de (tmda-ofmipd) with ESMTP; Sun, 10 Sep 2017 13:41:26 +0200 Original-Received: (qmail 5566 invoked by uid 1000); 10 Sep 2017 11:36:26 -0000 Content-Disposition: inline In-Reply-To: <69e034d3-7a52-cc81-dc56-e5308ad5dce0@yandex.ru> X-Delivery-Agent: TMDA/1.1.12 (Macallan) X-Primary-Address: acm@muc.de X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:136726 Archived-At: Hello, Dmitry and Philipp. On Sat, Sep 09, 2017 at 12:44:02 +0300, Dmitry Gutov wrote: > Hi Alan, > On 9/7/17 11:45 PM, Alan Mackenzie wrote: > > The solution I propose is to introduce a second cache into syntax-ppss, > > and this cache would be used whenever (not (eq (point-min) 1)). > > Whenever point-min changes, and isn't 1, this second cached would be > > calculated again from scratch. Here is a patch implementing this. Comments about it would be welcome. [ .... ] > And since the API doesn't change, and the observable behavior doesn't > either (in the vast majority of cases; probably all except the broken > ones), we can refine this solution easily, or even swap it for something > else, with little cost. [ .... ] > Caveats: > - This solves the dependency on point-min, but does nothing about the > dependency on the current syntax-table (which can change). I'm not > necessarily suggesting we try to solve that now, though. > - Before this change is pushed to master, or shortly after, I'd like to > know that it actually fixed the problem Philipp experienced with > python-mode, so we can revert 4fbd330. If it was caused by e.g. > syntax-table changing, we've not improved much. Philipp, any chance of you trying out python mode with this patch but without 4fbd330? diff --git a/lisp/emacs-lisp/syntax.el b/lisp/emacs-lisp/syntax.el index d1d5176944..952ea8bb83 100644 --- a/lisp/emacs-lisp/syntax.el +++ b/lisp/emacs-lisp/syntax.el @@ -386,11 +386,103 @@ syntax-ppss-cache (defvar-local syntax-ppss-last nil "Cache of (LAST-POS . LAST-PPSS).") -(defalias 'syntax-ppss-after-change-function 'syntax-ppss-flush-cache) -(defun syntax-ppss-flush-cache (beg &rest ignored) - "Flush the cache of `syntax-ppss' starting at position BEG." - ;; Set syntax-propertize to refontify anything past beg. - (setq syntax-propertize--done (min beg syntax-propertize--done)) +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; +;; Several caches. +;; +;; Because `syntax-ppss' is equivalent to (parse-partial-sexp +;; (POINT-MIN) x), we need either to empty the cache when we narrow +;; the buffer, which is suboptimal, or we need to use several caches. +;; +;; The implementation which follows uses three caches, the current one +;; (in `syntax-ppss-cache' and `syntax-ppss-last') and two inactive +;; ones (in `syntax-ppss-{cache,last}-{wide,narrow}'), which store the +;; former state of the active cache as it was used in widened and +;; narrowed buffers respectively. There are also the variables +;; `syntax-ppss-max-valid-{wide,narrow}' which hold the maximum +;; position where the caches are valid, due to buffer changes. +;; +;; At the first call to `syntax-ppss' after a widening or narrowing of +;; the buffer, the pertinent inactive cache is swapped into the +;; current cache by calling `syntax-ppss-set-cache'. Note that there +;; is currently just one inactive cache for narrowed buffers, so only +;; one inactive narrowed cache can be stored at a time. +;; +;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; + +(defvar-local syntax-ppss-cache-wide nil + "Holds the value of `syntax-ppss-cache' for a widened buffer.") +(defvar-local syntax-ppss-last-wide nil + "Holds the value of `syntax-ppss-last' for a widened buffer.") +(defvar-local syntax-ppss-max-valid-wide most-positive-fixnum + "The buffer position after which `syntax-ppss-cache-wide' is invalid.") + +(defvar-local syntax-ppss-cache-narrow nil + "Holds the value of `syntax-ppss-cache' for a narrowed buffer.") +(defvar-local syntax-ppss-last-narrow nil + "Holds the value of `syntax-ppss-last' for a narrowed buffer.") +(defvar-local syntax-ppss-max-valid-narrow most-positive-fixnum + "The buffer position after which `syntax-ppss-cache-narrow' is invalid.") + +(defvar-local syntax-ppss-narrow-point-min 1 + "Value of `point-min' for which the stored \"narrow\" cache is valid.") + +(defvar-local syntax-ppss-supremum most-positive-fixnum + "Lowest change position since previous restriction change.") + +(defvar-local syntax-ppss-cache-point-min 1 + "Value of `point-min' for which the current cache is valid.") + +(defun syntax-ppss-set-cache () + "Swap in and out the cache pertinent to the new point-min." + (unless (eq (point-min) syntax-ppss-cache-point-min) + ;; Update the stored `...max-valid' values. + (setq syntax-ppss-max-valid-wide + (if (eq syntax-ppss-cache-point-min 1) + (or (caar syntax-ppss-cache) + 1) + (min syntax-ppss-max-valid-wide syntax-ppss-supremum))) + (setq syntax-ppss-max-valid-narrow + (if (eq syntax-ppss-cache-point-min syntax-ppss-narrow-point-min) + (or (caar syntax-ppss-cache) + syntax-ppss-cache-point-min) + (min syntax-ppss-max-valid-narrow syntax-ppss-supremum))) + (setq syntax-ppss-supremum most-positive-fixnum) + + ;; Store away the current values of the cache. + (cond + ((eq syntax-ppss-cache-point-min 1) + (setq syntax-ppss-cache-wide syntax-ppss-cache + syntax-ppss-last-wide syntax-ppss-last)) + ((eq syntax-ppss-cache-point-min syntax-ppss-narrow-point-min) + (setq syntax-ppss-cache-narrow syntax-ppss-cache + syntax-ppss-last-narrow syntax-ppss-last)) + (syntax-ppss-cache + (setq syntax-ppss-narrow-point-min syntax-ppss-cache-point-min + syntax-ppss-cache-narrow syntax-ppss-cache + syntax-ppss-last-narrow syntax-ppss-last)) + (t nil)) + + ;; Restore/initialize the cache for the new point-min. + (cond + ((eq (point-min) 1) + (setq syntax-ppss-cache syntax-ppss-cache-wide + syntax-ppss-last syntax-ppss-last-wide) + (save-restriction + (widen) + (syntax-ppss-invalidate-cache syntax-ppss-max-valid-wide))) + ((eq (point-min) syntax-ppss-narrow-point-min) + (setq syntax-ppss-cache syntax-ppss-cache-narrow + syntax-ppss-last syntax-ppss-last-narrow) + (save-restriction + (widen) + (syntax-ppss-invalidate-cache syntax-ppss-max-valid-narrow))) + (t + (setq syntax-ppss-cache nil + syntax-ppss-last nil))) + (setq syntax-ppss-cache-point-min (point-min)))) + +(defun syntax-ppss-invalidate-cache (beg &rest ignored) + "Invalidate the cache of `syntax-ppss' starting at position BEG." ;; Flush invalid cache entries. (while (and syntax-ppss-cache (> (caar syntax-ppss-cache) beg)) (setq syntax-ppss-cache (cdr syntax-ppss-cache))) @@ -411,6 +503,16 @@ syntax-ppss-flush-cache ;; (remove-hook 'before-change-functions 'syntax-ppss-flush-cache t)) ) +;; Retain the following two for compatibility reasons. +(defalias 'syntax-ppss-after-change-function 'syntax-ppss-flush-cache) +(defun syntax-ppss-flush-cache (beg &rest ignored) + "Flush the `syntax-ppss' caches and set `syntax-propertize--done'." + (setq syntax-ppss-supremum (min beg syntax-ppss-supremum)) + ;; Ensure the appropriate cache is active. + (syntax-ppss-set-cache) + (setq syntax-propertize--done (min beg syntax-propertize--done)) + (syntax-ppss-invalidate-cache beg ignored)) + (defvar syntax-ppss-stats [(0 . 0.0) (0 . 0.0) (0 . 0.0) (0 . 0.0) (0 . 0.0) (1 . 2500.0)]) (defun syntax-ppss-stats () @@ -434,6 +536,8 @@ syntax-ppss this function is called while `before-change-functions' is temporarily let-bound, or if the buffer is modified without running the hook." + ;; Ensure the appropriate cache is active. + (syntax-ppss-set-cache) ;; Default values. (unless pos (setq pos (point))) (syntax-propertize pos) > All the best, > Dmitry. -- Alan Mackenzie (Nuremberg, Germany).