From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Alan Mackenzie Newsgroups: gmane.emacs.bugs Subject: bug#31290: Fundamental bugs in syntax-propertize Date: Sat, 12 May 2018 11:26:12 +0000 Message-ID: <20180512112612.GA4766@ACM> References: <20180427210859.GA6023@ACM> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: blaine.gmane.org 1526124728 16264 195.159.176.226 (12 May 2018 11:32:08 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 12 May 2018 11:32:08 +0000 (UTC) User-Agent: Mutt/1.9.4 (2018-02-28) Cc: 31290@debbugs.gnu.org To: Dmitry Gutov Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sat May 12 13:32:04 2018 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fHSks-00044f-72 for geb-bug-gnu-emacs@m.gmane.org; Sat, 12 May 2018 13:32:02 +0200 Original-Received: from localhost ([::1]:38938 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fHSmz-00074f-9e for geb-bug-gnu-emacs@m.gmane.org; Sat, 12 May 2018 07:34:13 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:53310) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fHSms-00074M-Fn for bug-gnu-emacs@gnu.org; Sat, 12 May 2018 07:34:07 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fHSmp-00067L-AB for bug-gnu-emacs@gnu.org; Sat, 12 May 2018 07:34:06 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:51092) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fHSmp-00067F-5u for bug-gnu-emacs@gnu.org; Sat, 12 May 2018 07:34:03 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1fHSmo-0003ew-P9 for bug-gnu-emacs@gnu.org; Sat, 12 May 2018 07:34:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Alan Mackenzie Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 12 May 2018 11:34:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 31290 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 31290-submit@debbugs.gnu.org id=B31290.152612483114047 (code B ref 31290); Sat, 12 May 2018 11:34:01 +0000 Original-Received: (at 31290) by debbugs.gnu.org; 12 May 2018 11:33:51 +0000 Original-Received: from localhost ([127.0.0.1]:58989 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fHSmc-0003eV-UA for submit@debbugs.gnu.org; Sat, 12 May 2018 07:33:51 -0400 Original-Received: from colin.muc.de ([193.149.48.1]:38221 helo=mail.muc.de) by debbugs.gnu.org with smtp (Exim 4.84_2) (envelope-from ) id 1fHSma-0003eL-Fr for 31290@debbugs.gnu.org; Sat, 12 May 2018 07:33:49 -0400 Original-Received: (qmail 81713 invoked by uid 3782); 12 May 2018 11:33:47 -0000 Original-Received: from acm.muc.de (p5B146231.dip0.t-ipconnect.de [91.20.98.49]) by colin.muc.de (tmda-ofmipd) with ESMTP; Sat, 12 May 2018 13:33:45 +0200 Original-Received: (qmail 6270 invoked by uid 1000); 12 May 2018 11:26:12 -0000 Content-Disposition: inline In-Reply-To: X-Delivery-Agent: TMDA/1.1.12 (Macallan) X-Primary-Address: acm@muc.de X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:146135 Archived-At: Hello, Dmitry. On Tue, May 08, 2018 at 15:35:14 +0300, Dmitry Gutov wrote: > On 4/28/18 12:08 AM, Alan Mackenzie wrote: > > At least that would be true if syntax-propertize--done hadn't been > > prematurely and spuriously increased, crudely to prevent an infinite > > recursion, falsely indicating to the syntax-ppss infrastructure that the > > syntax-table properties have already been applied to the region (BEGIN > > END). > > .... but it should not call `syntax-ppss-flush-cache', .... > > Why not? Because syntax-ppss-flush-cache sets syntax-propertize--done > > back to its true value, allowing the wrongly allowed syntax-ppss calls at > > a later position to cause a recursive loop. > Maybe we should "allow" it to loop, in certain cases? Leaving it to be > the responsibility of the programmer, to make sure the result doesn't > infloop, even if these rules are violated. I'm not sure how this could work. We would need to formalise the rules very carefully, to avoid the need to read syntax.{c,el}'s source code. > > .... which means that it should not call `syntax-ppss' on some > > position and later modify the buffer on some earlier position. > > This is a bad restriction, because sometimes syntax-table properties can > > only be correctly determined by examining the syntax of later buffer > > positions. An example of this is giving the string-fence syntax-table > > text property to an unbalanced opening string quote, but not to correctly > > matched quotes. > I'm not exactly convinced by the given example (why would we use the > string-fence in that case?), but it might be better if something like > this was possible, indeed. String fence can be used to signal to font lock that the delimiter (together with the "mismatching" unescaped EOL) should be fontified in warning face. A better example might be C++ Mode's marking of a "< ... >" pair with paren syntax. This isn't done with syntax-propertize-function (as you know), but it would be nice if this were possible. > > 2. syntax-propertize-function's are banned from using syntax-ppss, the > > documentation instead directing them to use parse-partial-sexp directly. > The ones that currently call syntax-ppss, can't simply switch over to > parse-partial-sexp without becoming slower due to the lack of cache. The cache at the pertinent buffer position doesn't exist at the time: consistent syntax-table properties aren't on the preceding buffer positions. > Before tackling this bug, I'd rather we see a real-world problem that it > caused, and pick a particular approach based on it. My enhancements for bug#30393: "24.4; cperl-mode: indentation failure - Documentation enhancements", where (almost) any change which affects the syntactic state is programmed to call syntax-ppss-flush-cache from the C level, clashes with the mechanism in this bug report. Most of the time it's fine, but when a change affecting the syntactic state is made from inside a synax-propertize-function, Emacs goes into an infinite recursive loop. This isn't good. > But off the top of my head, we could introduce a "stricter but somewhat > slower" variation of syntax-ppss to be called inside > syntax-propertize-function's, which would treat the values in question > more carefully, somehow. That's an idea worth exploring. -- Alan Mackenzie (Nuremberg, Germany).