From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Alan Mackenzie Newsgroups: gmane.emacs.bugs Subject: bug#31290: Fundamental bugs in syntax-propertize Date: Fri, 27 Apr 2018 21:08:59 +0000 Message-ID: <20180427210859.GA6023@ACM> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: blaine.gmane.org 1524863590 20622 195.159.176.226 (27 Apr 2018 21:13:10 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 27 Apr 2018 21:13:10 +0000 (UTC) User-Agent: Mutt/1.9.4 (2018-02-28) To: 31290@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Fri Apr 27 23:13:05 2018 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fCAfw-0005D1-Ei for geb-bug-gnu-emacs@m.gmane.org; Fri, 27 Apr 2018 23:13:04 +0200 Original-Received: from localhost ([::1]:49991 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fCAi3-000318-9C for geb-bug-gnu-emacs@m.gmane.org; Fri, 27 Apr 2018 17:15:15 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:46611) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fCAhv-0002zi-TF for bug-gnu-emacs@gnu.org; Fri, 27 Apr 2018 17:15:08 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fCAhq-00045f-Tc for bug-gnu-emacs@gnu.org; Fri, 27 Apr 2018 17:15:07 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:33493) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fCAhq-00045b-PA for bug-gnu-emacs@gnu.org; Fri, 27 Apr 2018 17:15:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1fCAhq-0005NH-Gf for bug-gnu-emacs@gnu.org; Fri, 27 Apr 2018 17:15:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Alan Mackenzie Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 27 Apr 2018 21:15:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 31290 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.152486364720568 (code B ref -1); Fri, 27 Apr 2018 21:15:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 27 Apr 2018 21:14:07 +0000 Original-Received: from localhost ([127.0.0.1]:41390 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fCAgx-0005Lg-FH for submit@debbugs.gnu.org; Fri, 27 Apr 2018 17:14:07 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:55182) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fCAgv-0005L3-6Q for submit@debbugs.gnu.org; Fri, 27 Apr 2018 17:14:06 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fCAgp-0003ka-2K for submit@debbugs.gnu.org; Fri, 27 Apr 2018 17:13:59 -0400 Original-Received: from lists.gnu.org ([2001:4830:134:3::11]:48842) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fCAgo-0003kQ-VJ for submit@debbugs.gnu.org; Fri, 27 Apr 2018 17:13:59 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:46147) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fCAgn-0002Jr-Ub for bug-gnu-emacs@gnu.org; Fri, 27 Apr 2018 17:13:58 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fCAgi-0003iQ-Vj for bug-gnu-emacs@gnu.org; Fri, 27 Apr 2018 17:13:57 -0400 Original-Received: from colin.muc.de ([193.149.48.1]:31573 helo=mail.muc.de) by eggs.gnu.org with smtp (Exim 4.71) (envelope-from ) id 1fCAgi-0003gc-KK for bug-gnu-emacs@gnu.org; Fri, 27 Apr 2018 17:13:52 -0400 Original-Received: (qmail 77206 invoked by uid 3782); 27 Apr 2018 21:13:47 -0000 Original-Received: from acm.muc.de (p5B14720D.dip0.t-ipconnect.de [91.20.114.13]) by colin.muc.de (tmda-ofmipd) with ESMTP; Fri, 27 Apr 2018 23:13:46 +0200 Original-Received: (qmail 6051 invoked by uid 1000); 27 Apr 2018 21:08:59 -0000 Content-Disposition: inline X-Delivery-Agent: TMDA/1.1.12 (Macallan) X-Primary-Address: acm@muc.de X-detected-operating-system: by eggs.gnu.org: FreeBSD 9.x [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:145793 Archived-At: Hello, Emacs. There are fundamental bugs in syntax-propertize and syntax-propertize-function. The doc string of the latter states: The specified function may call `syntax-ppss' on any position before END, .... This is untrue. True is that syntax-ppss can be called on a position only up to syntax-propertize--done. After this point, the syntax-table properties haven't been applied, so calling syntax-ppss is, in general, going to give a false result. At least that would be true if syntax-propertize--done hadn't been prematurely and spuriously increased, crudely to prevent an infinite recursion, falsely indicating to the syntax-ppss infrastructure that the syntax-table properties have already been applied to the region (BEGIN END). .... but it should not call `syntax-ppss-flush-cache', .... Why not? Because syntax-ppss-flush-cache sets syntax-propertize--done back to its true value, allowing the wrongly allowed syntax-ppss calls at a later position to cause a recursive loop. .... which means that it should not call `syntax-ppss' on some position and later modify the buffer on some earlier position. This is a bad restriction, because sometimes syntax-table properties can only be correctly determined by examining the syntax of later buffer positions. An example of this is giving the string-fence syntax-table text property to an unbalanced opening string quote, but not to correctly matched quotes. ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; The plain fact is that (syntax-ppss pos) calls (syntax-propertize pos), so syntax-propertize cannot itself use syntax-ppss because of the recursive loop thus created. ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Proposed solutions: 1. Major modes' syntax-propertize-function's are somehow given read access to syntax-propertize--done, and may call syntax-ppss up to that point only. syntax-propertize--done is updated only after the syntax-table properties have been applied. Or.... 2. syntax-propertize-function's are banned from using syntax-ppss, the documentation instead directing them to use parse-partial-sexp directly. In either solution, the restriction on using syntax-ppss-flush-cache would no longer be necessary, and there would be no restriction on setting syntax-table text properties at an earlier position than the one currently being analysed. I think solution 2 is the better one. -- Alan Mackenzie (Nuremberg, Germany).