bug#22983: syntax-ppss returns wrong result.

[Alan Mackenzie <acm@muc.de>]

Hello, Dmitry.

On Fri, Mar 18, 2016 at 02:49:34AM +0200, Dmitry Gutov wrote:
> On 03/11/2016 05:15 PM, Alan Mackenzie wrote:

> This patch should make ppss-0 and ppss-1 match:

OK, no bad thing!

But seeing that the function is a new function (its specification has
changed), it will need new test cases, fresh new attempts to break it.

> diff --git a/lisp/emacs-lisp/syntax.el b/lisp/emacs-lisp/syntax.el
> index e20a210..c1b9d84 100644
> --- a/lisp/emacs-lisp/syntax.el
> +++ b/lisp/emacs-lisp/syntax.el
> @@ -371,6 +371,11 @@ syntax-ppss-max-span
>   We try to make sure that cache entries are at least this far apart
>   from each other, to avoid keeping too much useless info.")

> +(defvar syntax-ppss-dont-widen nil
> +  "If non-nil, `syntax-ppss' will work on the non-widened buffer.
> +The code that uses this should create local bindings for
> +`syntax-ppss-cache' and `syntax-ppss-last' too.")
> +

I'm against this bit.  If syntax-ppss-dont-widen is non-nil, the buffer
is narrowed, and the local cache variables are correctly bound and
filled, then something at a low level is going to widen the buffer (and
call back_comment) without knowing to restore the global bindings for
those cache variables.  This could easily give the wrong result and
corrupt the locally bound cache.

I think the only sensible functionality for syntax-ppss is to be
equivalent to (parse-partial-sexp 1 pos).  Then everybody knows where
they stand.  Those pieces of code which actually need a ppss cache with
origin other than 1 could then use a more appropriate specialized
function whose cache wouldn't get mixed up with syntax-ppss's.  (It
could share a lot of code with syntax-ppss).

>   (defvar syntax-begin-function nil
>     "Function to move back outside of any comment/string/paren.
>   This function should move the cursor back to some syntactically safe
> @@ -423,12 +428,21 @@ syntax-ppss
>   in the returned list (counting from 0) cannot be relied upon.
>   Point is at POS when this function returns.

> +IF `syntax-ppss-dont-widen' is nil, the buffer is temporarily
> +widened.
> +
>   It is necessary to call `syntax-ppss-flush-cache' explicitly if
>   this function is called while `before-change-functions' is
>   temporarily let-bound, or if the buffer is modified without
>   running the hook."
>     ;; Default values.
>     (unless pos (setq pos (point)))
> +  (save-restriction
> +    (unless syntax-ppss-dont-widen
> +      (widen))
> +    (syntax-pps--at pos)))
> +
> +(defun syntax-ppss--at (pos)
>     (syntax-propertize pos)
>     ;;
>     (let ((old-ppss (cdr syntax-ppss-last))

-- 
Alan Mackenzie (Nuremberg, Germany).