From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Alan Mackenzie Newsgroups: gmane.emacs.bugs Subject: bug#30393: 24.4; cperl-mode: indentation failure - Documentation enhancements Date: Mon, 16 Apr 2018 19:21:37 +0000 Message-ID: <20180416192137.GA5637@ACM> References: <20180210112654.GA4537@ACM> <20180211103610.GA4515@ACM> <20180212183800.GA5601@ACM> <20180305084255.GA4786@ACM> <83zi3msdms.fsf@gnu.org> <20180408105257.GA5350@ACM> <83vad0xlwu.fsf@gnu.org> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: blaine.gmane.org 1523906592 17235 195.159.176.226 (16 Apr 2018 19:23:12 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Mon, 16 Apr 2018 19:23:12 +0000 (UTC) User-Agent: Mutt/1.9.4 (2018-02-28) Cc: dgutov@yandex.ru, 30393@debbugs.gnu.org, monnier@IRO.UMontreal.CA, npostavs@users.sourceforge.net To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Mon Apr 16 21:23:07 2018 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1f89iV-0004Lr-CM for geb-bug-gnu-emacs@m.gmane.org; Mon, 16 Apr 2018 21:23:07 +0200 Original-Received: from localhost ([::1]:38344 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f89ka-0000Me-En for geb-bug-gnu-emacs@m.gmane.org; Mon, 16 Apr 2018 15:25:16 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:39192) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1f89kP-0000Ji-TG for bug-gnu-emacs@gnu.org; Mon, 16 Apr 2018 15:25:07 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1f89kM-0004ro-J4 for bug-gnu-emacs@gnu.org; Mon, 16 Apr 2018 15:25:05 -0400 Original-Received: from debbugs.gnu.org ([208.118.235.43]:48349) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1f89kM-0004re-Ci for bug-gnu-emacs@gnu.org; Mon, 16 Apr 2018 15:25:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1f89kM-0000qj-73 for bug-gnu-emacs@gnu.org; Mon, 16 Apr 2018 15:25:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Alan Mackenzie Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 16 Apr 2018 19:25:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 30393 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 30393-submit@debbugs.gnu.org id=B30393.15239066573208 (code B ref 30393); Mon, 16 Apr 2018 19:25:02 +0000 Original-Received: (at 30393) by debbugs.gnu.org; 16 Apr 2018 19:24:17 +0000 Original-Received: from localhost ([127.0.0.1]:56246 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1f89jd-0000pf-4P for submit@debbugs.gnu.org; Mon, 16 Apr 2018 15:24:17 -0400 Original-Received: from colin.muc.de ([193.149.48.1]:63558 helo=mail.muc.de) by debbugs.gnu.org with smtp (Exim 4.84_2) (envelope-from ) id 1f89jb-0000pX-9j for 30393@debbugs.gnu.org; Mon, 16 Apr 2018 15:24:16 -0400 Original-Received: (qmail 20553 invoked by uid 3782); 16 Apr 2018 19:24:13 -0000 Original-Received: from acm.muc.de (p5B147020.dip0.t-ipconnect.de [91.20.112.32]) by colin.muc.de (tmda-ofmipd) with ESMTP; Mon, 16 Apr 2018 21:24:11 +0200 Original-Received: (qmail 5689 invoked by uid 1000); 16 Apr 2018 19:21:37 -0000 Content-Disposition: inline In-Reply-To: <83vad0xlwu.fsf@gnu.org> X-Delivery-Agent: TMDA/1.1.12 (Macallan) X-Primary-Address: acm@muc.de X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:145461 Archived-At: Hello, Eli. On Mon, Apr 09, 2018 at 21:41:21 +0300, Eli Zaretskii wrote: > > Date: Sun, 8 Apr 2018 10:52:57 +0000 > > Cc: monnier@IRO.UMontreal.CA, 30393@debbugs.gnu.org, dgutov@yandex.ru, > > npostavs@users.sourceforge.net > > From: Alan Mackenzie > > Can we start moving forward with this change, please? > What prevents you from moving forward with it? You already know what > needs to be done to move forward: documentation of the new primitives > and some tests. After mulling it over for several weeks, I now think it would be better just to leave these new primitives out. Although they might be useful, their main use was for me whilst hacking the syntax routines. However, I propose the following documentation changes to go with my patch to the code, these changes clarifying some of the limitations inherent to syntax-ppss, and indicating how my enhancements will work. diff --git a/doc/lispref/syntax.texi b/doc/lispref/syntax.texi index 3327d7855c..b58e2a9a98 100644 --- a/doc/lispref/syntax.texi +++ b/doc/lispref/syntax.texi @@ -430,6 +430,11 @@ Syntax Table Functions This function always returns @code{nil}. The old syntax information in the table for this character is discarded. +Note that if the modification of @var{table} changes the way it parses +comments or strings, the cache used by @code{syntax-ppss} will be +emptied in each buffer whose current syntax table (or its parent, +etc.) is @var{table}. @xref{Position Parse}. + @example @group @exdent @r{Examples:} @@ -502,6 +507,12 @@ Syntax Table Functions @defun set-syntax-table table This function makes @var{table} the syntax table for the current buffer. It returns @var{table}. + +Note that if @var{table} parses comments or strings differently from +the buffer's previous syntax table, the cache used by +@code{syntax-ppss} in this buffer will be emptied. @xref{Position +Parse}. + @end defun @defun syntax-table @@ -523,6 +534,9 @@ Syntax Table Functions more precise: @code{with-syntax-table} temporarily alters the current syntax table of whichever buffer is current at the time the macro execution starts. Other buffers are not affected. + +This macro might empty the cache used by @code{syntax-ppss}, as noted +above under @code{set-syntax-table}. @end defmac @node Syntax Properties @@ -534,6 +548,15 @@ Syntax Properties occurrences in the buffer, by applying a @code{syntax-table} text property. @xref{Text Properties}, for how to apply text properties. +As an alternative to setting @code{syntax-table} text properties +directly, you can use @code{syntax-propertize-function} (see below). +Most major modes supplied with Emacs which use these text properties +use this method for applying them. We strongly recommended you to use +just one of these methods in any Emacs Lisp program, and not to mix +them in the same buffer.@footnote{@code{syntax-propertize-function} +can operate at unpredictable times, and may erase explicitly applied +@code{syntax-table} properties.} + The valid values of @code{syntax-table} text property are: @table @asis @@ -556,6 +579,10 @@ Syntax Properties If this is non-@code{nil}, the syntax scanning functions, like @code{forward-sexp}, pay attention to syntax text properties. Otherwise they use only the current syntax table. + +If you change this variable after buffer initialization time, even if +your Emacs Lisp program doesn't use @code{syntax-ppss}, you should +call @code{syntax-ppss-flush-cache}. @xref{Position Parse}. @end defvar @defvar syntax-propertize-function @@ -695,8 +722,9 @@ Motion via Parsing negative @var{depth} has the effect of moving deeper by @var{-depth} levels of parenthesis. -Scanning ignores comments if @code{parse-sexp-ignore-comments} is -non-@code{nil}. +Scanning skips over comments if @code{parse-sexp-ignore-comments} is +non-@code{nil} (which it usually is for programming language major +modes). If the scan reaches the beginning or end of the accessible part of the buffer before it has scanned over @var{count} parenthetical groupings, @@ -709,7 +737,7 @@ Motion via Parsing It returns the position where the scan stops. If @var{count} is negative, the scan moves backwards. -Scanning ignores comments if @code{parse-sexp-ignore-comments} is +Scanning skips over comments if @code{parse-sexp-ignore-comments} is non-@code{nil}. If the scan reaches the beginning or end of (the accessible part of) the @@ -747,7 +775,7 @@ Position Parse For syntactic analysis, such as in indentation, often the useful thing is to compute the syntactic state corresponding to a given buffer -position. This function does that conveniently. +position. @code{syntax-ppss} does that conveniently. @defun syntax-ppss &optional pos This function returns the parser state that the parser would reach at @@ -769,6 +797,33 @@ Position Parse complete subexpression) and sixth value (minimum parenthesis depth) in the returned parser state are not meaningful. +Note that these caches become invalid when you set a new syntax table +for the buffer, or change an entry for a character in the current +syntax table. If you set or clear a @code{syntax-table} text property +at some buffer position, the caches become invalid for the buffer +portion at and after that position. It is your responsibility to deal +with these situations, either by calling +@code{syntax-ppss-flush-cache} (see below), or by refraining from +using @code{syntax-ppss} while the caches are invalid. An exception +to this is when such a change alters the way comments or strings are +parsed; then, Emacs calls @code{syntax-ppss-flush-cache} +automatically.@footnote{The reason for this is that from Emacs 27, +Emacs uses @code{syntax-ppss} internally in low level primitives such +as @code{forward-comment} and @code{scan-lists}. This automatic +flushing of the cache helps these primitives continue to work +reliably.} + +Changing any of the variables @code{multibyte-syntax-as-symbol}, +@code{parse-sexp-ignore-comments}, @code{comment-end-can-be-escaped} +(@pxref{Control Parsing}), or @code{parse-sexp-lookup-properties} +(@pxref{Syntax Properties}) after buffer initialization will always +leave @code{syntax-ppss}'s caches invalid in the affected buffers. So +will changing a symbol's @code{syntax-table} property when that symbol +is the value of a @code{category} text property somewhere in the +buffer (@pxref{Special Properties}), a practice we don't recommend. +In such cases you must always take one of the actions detailed in the +previous paragraph. + This function has a side effect: it adds a buffer-local entry to @code{before-change-functions} (@pxref{Change Hooks}) for @code{syntax-ppss-flush-cache} (see below). This entry keeps the @@ -952,6 +1007,11 @@ Control Parsing You can use @code{forward-comment} to move forward or backward over one comment or several comments. +If you change any of the above three variables after buffer +initialization time, even if your Emacs Lisp program doesn't use +@code{syntax-ppss}, you should call @code{syntax-ppss-flush-cache}. +@xref{Position Parse}. + @node Syntax Table Internals @section Syntax Table Internals @cindex syntax table internals diff --git a/doc/lispref/text.texi b/doc/lispref/text.texi index ebfa8b9b0f..d8df9fe352 100644 --- a/doc/lispref/text.texi +++ b/doc/lispref/text.texi @@ -3203,6 +3203,14 @@ Special Properties properties of this symbol serve as defaults for the properties of the character. +You might be tempted to put a @code{syntax-table} property onto the +symbol, and change this property's value repeatedly in your Lisp +program, thus changing the syntax of many characters in a buffer at +the same time. We advise against doing this, since it renders the +caches used by @code{syntax-ppss} invalid in a way that Emacs cannot +detect and correct for. If you are going to be doing this, please +consult @ref{Position Parse} and follow the advice there. + @item face @cindex face codes of text @kindex face @r{(text property)} -- Alan Mackenzie (Nuremberg, Germany).