From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Alan Mackenzie Newsgroups: gmane.emacs.bugs Subject: bug#30393: 24.4; cperl-mode: indentation failure Date: Mon, 12 Feb 2018 18:38:00 +0000 Message-ID: <20180212183800.GA5601@ACM> References: <20180208152552.GL13340@hodi> <20180209175040.63536.qmail@mail.muc.de> <3331f80a-c5aa-5cb9-8088-0a88888bdaca@yandex.ru> <20180210112654.GA4537@ACM> <20180211103610.GA4515@ACM> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: blaine.gmane.org 1518461484 18719 195.159.176.226 (12 Feb 2018 18:51:24 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 12 Feb 2018 18:51:24 +0000 (UTC) User-Agent: Mutt/1.7.2 (2016-11-26) Cc: 30393@debbugs.gnu.org, Dmitry Gutov , Noam Postavsky To: Stefan Monnier Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Mon Feb 12 19:51:20 2018 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1elJBY-0002Da-Lt for geb-bug-gnu-emacs@m.gmane.org; Mon, 12 Feb 2018 19:50:40 +0100 Original-Received: from localhost ([::1]:39084 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1elJDa-0002WK-9a for geb-bug-gnu-emacs@m.gmane.org; Mon, 12 Feb 2018 13:52:46 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:39078) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1elJB2-0000Ya-IV for bug-gnu-emacs@gnu.org; Mon, 12 Feb 2018 13:50:10 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1elJAw-0007cp-33 for bug-gnu-emacs@gnu.org; Mon, 12 Feb 2018 13:50:08 -0500 Original-Received: from debbugs.gnu.org ([208.118.235.43]:60346) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1elJAv-0007cV-VN for bug-gnu-emacs@gnu.org; Mon, 12 Feb 2018 13:50:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1elJAv-00060y-NL for bug-gnu-emacs@gnu.org; Mon, 12 Feb 2018 13:50:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Alan Mackenzie Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 12 Feb 2018 18:50:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 30393 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 30393-submit@debbugs.gnu.org id=B30393.151846134423043 (code B ref 30393); Mon, 12 Feb 2018 18:50:01 +0000 Original-Received: (at 30393) by debbugs.gnu.org; 12 Feb 2018 18:49:04 +0000 Original-Received: from localhost ([127.0.0.1]:40010 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1elJ9y-0005zY-OX for submit@debbugs.gnu.org; Mon, 12 Feb 2018 13:49:04 -0500 Original-Received: from colin.muc.de ([193.149.48.1]:16381 helo=mail.muc.de) by debbugs.gnu.org with smtp (Exim 4.84_2) (envelope-from ) id 1elJ9w-0005z8-4I for 30393@debbugs.gnu.org; Mon, 12 Feb 2018 13:49:00 -0500 Original-Received: (qmail 78966 invoked by uid 3782); 12 Feb 2018 18:48:58 -0000 Original-Received: from acm.muc.de (p548C77BB.dip0.t-ipconnect.de [84.140.119.187]) by colin.muc.de (tmda-ofmipd) with ESMTP; Mon, 12 Feb 2018 19:48:57 +0100 Original-Received: (qmail 5898 invoked by uid 1000); 12 Feb 2018 18:38:00 -0000 Content-Disposition: inline In-Reply-To: X-Delivery-Agent: TMDA/1.1.12 (Macallan) X-Primary-Address: acm@muc.de X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:143215 Archived-At: Hello, Stefan On Sun, Feb 11, 2018 at 17:53:38 -0500, Stefan Monnier wrote: > > OK, but I suspect in practice, this would be impossible to debug for > > lack of reproducibility. > Those problems can be hard to debug, indeed. But an inf-loop should at > least be diagnosed as such fairly easily (even if its origin can be > difficult to track down), so I don't think "in practice I haven't bumped > into this problem yet" just because those problems stay undetected. > > Another definite bug is that the syntax-ppss cache is not flushed when > > the syntax-table is changed, whether with set-syntax-table or > > modify-syntax-entry. > That's right. I haven't bumped into such issues yet, but here (contrary > to the above problem) it might very well be because the error > stays undetected. .... and of course, the need to flush the cache when a syntax-table text property is applied or removed. > > This is critical, now that primitives depend on this cache. > I can see two approaches to solve this problem: > - hook into set-syntax-table and modify-syntax-entry or something > like that. This will make it work right everywhere automatically, but > I'm afraid it could turn out to be difficult to make it efficient > (because of the cost of the tests needed to detect changes and more > importantly because of excessive flushing of the syntax-ppss cache). > - provide new functions to let packages tell syntax-ppss about > such things. E.g. a macro `with-new-syntax-context` (which would > be treated a bit like narrowing, maybe). This would require changes > to packages that suffer from this problem but should give > better performance. > I'd prefer the second option, but at the same time, I'm not completely sure > what are the "typical" problem cases (which makes it hard to come up > with good new functions/macros) other than the case where we use > with-syntax-table (which is sometimes combined with a local narrowing) > but some of those only tweak the "word-vs-symbol-vs-punctuation" > settings so should ideally not flush the syntax-ppss cache. > Also I don't actually know whether the "fully automatic" approach would > actually turn out to be too expensive, it's just a gut feeling. > > Would you please fix this, Stefan. > It's fairly high up on my todo list, but I'm kinda swamped right now. It has occurred to me over the last day or two that I have already solved these problems (basically, with your first approach, hooking into set-syntax-table and friends) in the comment-cache branch, and that the approach taken could be used more or less unchanged in the current master. For set-syntax-table, it compares the old and the new syntax tables to see if they are "literally the same" (i.e. process strings and comments identically) or "literally different", and only in the latter case does it flush the cache. These comparisons, which are expensive, are cached inside the syntax-tables (in "extra slots"). Similarly, on modify-syntax-entry, the cache is flushed iff the change affects literals. Similarly, on setting or deleting a syntax-table text property, the cache is flushed from that point if the change affects literals. This happens regardless of the setting of inhibit-modification-hooks, etc. It is a fact that the vast bulk of libraries which use syntax-ppss use only elements 3, 4, and 8, i.e. the ones relevant to literals, and ignore everything else. For these the scheme outlined above is rigorous. I have timed it in the comment-cache branch, scanning through .../src/xdisp.c displaying each screen, and found no difference to the approach without comment-cache. For those few libraries which do use the full capabilities of the parsing state, we would need to flush the cache on all set-syntax-table's and so on. Maybe. Maybe this would be too expensive in run time. So the interface I propose would be two abnormal hooks, one for "literally important" changes to the syntax, and the other for other changes to the syntax. The hook functions would take an optional argument which would be nil for changes to the syntax table, or a buffer position where a syntax-table property is being changed. Mostly, only the first of these hooks need be used, the standard function on them being syntax-ppss's flush function. Major modes could add syntax-ppss's flush function to the second hook (possibly through some nice interface), should they use the non-literal parts of the parse state. One or two incidental changes would be needed, for example to fix the infinite recursion in printing syntax-tables, caused by the mutual presence of "literally the same/different" syntax tables in the extra slots. This would not be difficult. Then, finally, if we can be bothered, we could put in a mechanism to deal with changes in parse-sexp-lookup-properties and parse-sexp-ignore-comments. What do you think? > Stefan -- Alan Mackenzie (Nuremberg, Germany).