unofficial mirror of bug-gnu-emacs@gnu.org 
 help / color / mirror / code / Atom feed
From: Alan Mackenzie <acm@muc.de>
To: Eli Zaretskii <eliz@gnu.org>
Cc: dgutov@yandex.ru, 30393@debbugs.gnu.org,
	monnier@IRO.UMontreal.CA, npostavs@users.sourceforge.net
Subject: bug#30393: 24.4; cperl-mode: indentation failure - Documentation enhancements
Date: Mon, 16 Apr 2018 19:21:37 +0000	[thread overview]
Message-ID: <20180416192137.GA5637@ACM> (raw)
In-Reply-To: <83vad0xlwu.fsf@gnu.org>

Hello, Eli.

On Mon, Apr 09, 2018 at 21:41:21 +0300, Eli Zaretskii wrote:
> > Date: Sun, 8 Apr 2018 10:52:57 +0000
> > Cc: monnier@IRO.UMontreal.CA, 30393@debbugs.gnu.org, dgutov@yandex.ru,
> >   npostavs@users.sourceforge.net
> > From: Alan Mackenzie <acm@muc.de>

> > Can we start moving forward with this change, please?

> What prevents you from moving forward with it?  You already know what
> needs to be done to move forward: documentation of the new primitives
> and some tests.

After mulling it over for several weeks, I now think it would be better
just to leave these new primitives out.  Although they might be useful,
their main use was for me whilst hacking the syntax routines.

However, I propose the following documentation changes to go with my
patch to the code, these changes clarifying some of the limitations
inherent to syntax-ppss, and indicating how my enhancements will work.



diff --git a/doc/lispref/syntax.texi b/doc/lispref/syntax.texi
index 3327d7855c..b58e2a9a98 100644
--- a/doc/lispref/syntax.texi
+++ b/doc/lispref/syntax.texi
@@ -430,6 +430,11 @@ Syntax Table Functions
 This function always returns @code{nil}.  The old syntax information in
 the table for this character is discarded.
 
+Note that if the modification of @var{table} changes the way it parses
+comments or strings, the cache used by @code{syntax-ppss} will be
+emptied in each buffer whose current syntax table (or its parent,
+etc.) is @var{table}.  @xref{Position Parse}.
+
 @example
 @group
 @exdent @r{Examples:}
@@ -502,6 +507,12 @@ Syntax Table Functions
 @defun set-syntax-table table
 This function makes @var{table} the syntax table for the current buffer.
 It returns @var{table}.
+
+Note that if @var{table} parses comments or strings differently from
+the buffer's previous syntax table, the cache used by
+@code{syntax-ppss} in this buffer will be emptied.  @xref{Position
+Parse}.
+
 @end defun
 
 @defun syntax-table
@@ -523,6 +534,9 @@ Syntax Table Functions
 more precise: @code{with-syntax-table} temporarily alters the current
 syntax table of whichever buffer is current at the time the macro
 execution starts.  Other buffers are not affected.
+
+This macro might empty the cache used by @code{syntax-ppss}, as noted
+above under @code{set-syntax-table}.
 @end defmac
 
 @node Syntax Properties
@@ -534,6 +548,15 @@ Syntax Properties
 occurrences in the buffer, by applying a @code{syntax-table} text
 property.  @xref{Text Properties}, for how to apply text properties.
 
+As an alternative to setting @code{syntax-table} text properties
+directly, you can use @code{syntax-propertize-function} (see below).
+Most major modes supplied with Emacs which use these text properties
+use this method for applying them.  We strongly recommended you to use
+just one of these methods in any Emacs Lisp program, and not to mix
+them in the same buffer.@footnote{@code{syntax-propertize-function}
+can operate at unpredictable times, and may erase explicitly applied
+@code{syntax-table} properties.}
+
   The valid values of @code{syntax-table} text property are:
 
 @table @asis
@@ -556,6 +579,10 @@ Syntax Properties
 If this is non-@code{nil}, the syntax scanning functions, like
 @code{forward-sexp}, pay attention to syntax text properties.
 Otherwise they use only the current syntax table.
+
+If you change this variable after buffer initialization time, even if
+your Emacs Lisp program doesn't use @code{syntax-ppss}, you should
+call @code{syntax-ppss-flush-cache}.  @xref{Position Parse}.
 @end defvar
 
 @defvar syntax-propertize-function
@@ -695,8 +722,9 @@ Motion via Parsing
 negative @var{depth} has the effect of moving deeper by @var{-depth}
 levels of parenthesis.
 
-Scanning ignores comments if @code{parse-sexp-ignore-comments} is
-non-@code{nil}.
+Scanning skips over comments if @code{parse-sexp-ignore-comments} is
+non-@code{nil} (which it usually is for programming language major
+modes).
 
 If the scan reaches the beginning or end of the accessible part of the
 buffer before it has scanned over @var{count} parenthetical groupings,
@@ -709,7 +737,7 @@ Motion via Parsing
 It returns the position where the scan stops.  If @var{count} is
 negative, the scan moves backwards.
 
-Scanning ignores comments if @code{parse-sexp-ignore-comments} is
+Scanning skips over comments if @code{parse-sexp-ignore-comments} is
 non-@code{nil}.
 
 If the scan reaches the beginning or end of (the accessible part of) the
@@ -747,7 +775,7 @@ Position Parse
 
   For syntactic analysis, such as in indentation, often the useful
 thing is to compute the syntactic state corresponding to a given buffer
-position.  This function does that conveniently.
+position.  @code{syntax-ppss} does that conveniently.
 
 @defun syntax-ppss &optional pos
 This function returns the parser state that the parser would reach at
@@ -769,6 +797,33 @@ Position Parse
 complete subexpression) and sixth value (minimum parenthesis depth) in
 the returned parser state are not meaningful.
 
+Note that these caches become invalid when you set a new syntax table
+for the buffer, or change an entry for a character in the current
+syntax table.  If you set or clear a @code{syntax-table} text property
+at some buffer position, the caches become invalid for the buffer
+portion at and after that position.  It is your responsibility to deal
+with these situations, either by calling
+@code{syntax-ppss-flush-cache} (see below), or by refraining from
+using @code{syntax-ppss} while the caches are invalid.  An exception
+to this is when such a change alters the way comments or strings are
+parsed; then, Emacs calls @code{syntax-ppss-flush-cache}
+automatically.@footnote{The reason for this is that from Emacs 27,
+Emacs uses @code{syntax-ppss} internally in low level primitives such
+as @code{forward-comment} and @code{scan-lists}.  This automatic
+flushing of the cache helps these primitives continue to work
+reliably.}
+
+Changing any of the variables @code{multibyte-syntax-as-symbol},
+@code{parse-sexp-ignore-comments}, @code{comment-end-can-be-escaped}
+(@pxref{Control Parsing}), or @code{parse-sexp-lookup-properties}
+(@pxref{Syntax Properties}) after buffer initialization will always
+leave @code{syntax-ppss}'s caches invalid in the affected buffers.  So
+will changing a symbol's @code{syntax-table} property when that symbol
+is the value of a @code{category} text property somewhere in the
+buffer (@pxref{Special Properties}), a practice we don't recommend.
+In such cases you must always take one of the actions detailed in the
+previous paragraph.
+
 This function has a side effect: it adds a buffer-local entry to
 @code{before-change-functions} (@pxref{Change Hooks}) for
 @code{syntax-ppss-flush-cache} (see below).  This entry keeps the
@@ -952,6 +1007,11 @@ Control Parsing
 You can use @code{forward-comment} to move forward or backward over
 one comment or several comments.
 
+If you change any of the above three variables after buffer
+initialization time, even if your Emacs Lisp program doesn't use
+@code{syntax-ppss}, you should call @code{syntax-ppss-flush-cache}.
+@xref{Position Parse}.
+
 @node Syntax Table Internals
 @section Syntax Table Internals
 @cindex syntax table internals
diff --git a/doc/lispref/text.texi b/doc/lispref/text.texi
index ebfa8b9b0f..d8df9fe352 100644
--- a/doc/lispref/text.texi
+++ b/doc/lispref/text.texi
@@ -3203,6 +3203,14 @@ Special Properties
 properties of this symbol serve as defaults for the properties of the
 character.
 
+You might be tempted to put a @code{syntax-table} property onto the
+symbol, and change this property's value repeatedly in your Lisp
+program, thus changing the syntax of many characters in a buffer at
+the same time.  We advise against doing this, since it renders the
+caches used by @code{syntax-ppss} invalid in a way that Emacs cannot
+detect and correct for.  If you are going to be doing this, please
+consult @ref{Position Parse} and follow the advice there.
+
 @item face
 @cindex face codes of text
 @kindex face @r{(text property)}


-- 
Alan Mackenzie (Nuremberg, Germany).





  parent reply	other threads:[~2018-04-16 19:21 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-02-08 15:25 bug#30393: 24.4; cperl-mode: indentation failure paulusm
2018-02-09  1:44 ` Noam Postavsky
     [not found] ` <mailman.8766.1518140709.27995.bug-gnu-emacs@gnu.org>
2018-02-09 17:50   ` Alan Mackenzie
2018-02-10  3:55     ` Noam Postavsky
2018-02-10  8:53     ` Dmitry Gutov
2018-02-10 11:26       ` Alan Mackenzie
2018-02-10 12:08         ` Eli Zaretskii
2018-02-11 12:49           ` Alan Mackenzie
2018-02-11 16:16             ` Eli Zaretskii
2018-02-14 21:00               ` Alan Mackenzie
2018-02-15 17:39                 ` Eli Zaretskii
2018-02-16 11:52                 ` Dmitry Gutov
2018-02-16 17:43                   ` Alan Mackenzie
2018-02-17  2:16                     ` Dmitry Gutov
2018-02-17 10:54                       ` Alan Mackenzie
2018-02-10 14:58         ` Stefan Monnier
2018-02-11 10:36           ` Alan Mackenzie
2018-02-11 22:53             ` Stefan Monnier
2018-02-12 18:38               ` Alan Mackenzie
2018-02-12 20:45                 ` Stefan Monnier
2018-03-05  8:42                   ` Alan Mackenzie
2018-03-05 16:14                     ` Eli Zaretskii
2018-03-06 18:09                       ` Alan Mackenzie
2018-04-08 10:52                       ` Alan Mackenzie
2018-04-09 18:41                         ` Eli Zaretskii
2018-04-10 17:31                           ` Alan Mackenzie
2018-04-16 19:21                           ` Alan Mackenzie [this message]
2018-04-19  7:52                             ` bug#30393: 24.4; cperl-mode: indentation failure - Documentation enhancements Eli Zaretskii
2020-08-22 16:07                               ` Lars Ingebrigtsen
2020-11-03 13:45 ` bug#30393: [PATCH] Add a test to verify that the bug is gone (and a fix for Emacs 26) Harald Jörg
2020-11-03 14:29   ` Lars Ingebrigtsen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://www.gnu.org/software/emacs/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180416192137.GA5637@ACM \
    --to=acm@muc.de \
    --cc=30393@debbugs.gnu.org \
    --cc=dgutov@yandex.ru \
    --cc=eliz@gnu.org \
    --cc=monnier@IRO.UMontreal.CA \
    --cc=npostavs@users.sourceforge.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).