bug#4920: 23.1; (check-parens) problem with bash script case items unbalanced close paren

bug#4920: 23.1; (check-parens) problem with bash script case items unbalanced close paren

[Jeremy Hall]

Running (check-parens) fails on a bash script with case statements in
because of the unmatched close ")".  Would it be possible to fix this,
or tell me how to work around it?

For example:- 
    case "$SYSNAME" in
	  Darwin)
            # ssss
	    ;;
          Linux)

Many thanks,
Jeremy
mail@jeremy-hall.com


In GNU Emacs 23.1.1 (i686-pc-linux-gnu, GTK+ Version 2.14.4)
 of 2009-09-08 on jeremy
Windowing system distributor `The X.Org Foundation', version
11.0.10503000
configured using `configure  'CFLAGS=-march=native -O3 -s -ffast-math
-fomit-frame-pointer -minline-all-stringops -pipe -fno-ident''

Important settings:
  value of $LC_ALL: C
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_GB.UTF-8
  value of $XMODIFIERS: @im=imsettings
  locale-coding-system: nil
  default-enable-multibyte-characters: t

Major mode: Shell-script

Minor modes in effect:
  cua-mode: t
  recentf-mode: t
  which-function-mode: t
  show-paren-mode: t
  diff-auto-refine-mode: t
  tooltip-mode: t
  tool-bar-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
o c <backspace> SPC " <backspace> J A H <backspace> 
<backspace> " <backspace> <backspace> " J A H SPC c 
a l l SPC r e c u r s SPC u i n <backspace> <backspace> 
<backspace> <backspace> e SPC i n s t a l l SPC l i 
b c a n <backspace> b i n e t " C-j <down> <down> C-u 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> d e 
d u p e <help-echo> <help-echo> <tool-bar> <save-buffer> 
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo> 
<tool-bar> <kill-buffer> <help-echo> <help-echo> <tool-bar> 
<open-file> <help-echo> <down-mouse-5> <mouse-5> <double-down-mouse-5> 
<double-mouse-5> <help-echo> <help-echo> <tool-bar> 
<kill-buffer> <help-echo> <help-echo> <help-echo> <help-echo> 
<help-echo> <tool-bar> <kill-buffer> <help-echo> C-z 
C-z C-z <help-echo> <help-echo> <tool-bar> <kill-buffer> 
<help-echo> <help-echo> <tool-bar> <kill-buffer> <help-echo> 
<help-echo> <help-echo> <menu-bar> <buffer> C-@ <help-echo> 
<down-mouse-5> <mouse-5> <down-mouse-5> <mouse-5> <down-mouse-5> 
<mouse-5> <double-down-mouse-5> <double-mouse-5> <triple-down-mouse-5> 
<triple-mouse-5> <triple-down-mouse-5> <triple-mouse-5> 
<down-mouse-4> <mouse-4> <double-down-mouse-4> <double-mouse-4> 
<triple-down-mouse-4> <triple-mouse-4> <triple-down-mouse-4> 
<triple-mouse-4> <triple-down-mouse-4> <triple-mouse-4> 
<down-mouse-4> <mouse-4> <double-down-mouse-4> <double-mouse-4> 
<triple-down-mouse-4> <triple-mouse-4> <triple-down-mouse-4> 
<triple-mouse-4> <help-echo> <help-echo> <help-echo> 
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo> 
<help-echo> <help-echo> <menu-bar> <file> <Open Recent> 
</home/jah/.emacs> <down-mouse-5> <mouse-5> <double-down-mouse-5> 
<double-mouse-5> <triple-down-mouse-5> <triple-mouse-5> 
<triple-down-mouse-5> <triple-mouse-5> <triple-down-mouse-5> 
<triple-mouse-5> <triple-down-mouse-5> <triple-mouse-5> 
<down-mouse-5> <mouse-5> <double-down-mouse-5> <double-mouse-5> 
<triple-down-mouse-5> <triple-mouse-5> <triple-down-mouse-5> 
<triple-mouse-5> <triple-down-mouse-5> <triple-mouse-5> 
<down-mouse-4> <mouse-4> <double-down-mouse-4> <double-mouse-4> 
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo> 
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo> 
<help-echo> <tool-bar> <kill-buffer> <help-echo> <help-echo> 
<help-echo> <help-echo> <down-mouse-1> <mouse-movement> 
<mouse-1> <help-echo> <help-echo> <help-echo> <help-echo> 
<help-echo> <help-echo> <help-echo> <tool-bar> <open-file> 
<help-echo> <up> <up> <up> <down-mouse-1> <mouse-movement> 
<mouse-1> <down> <down> <down> <down> <down> <down> 
<down> <help-echo> <help-echo> <help-echo> <help-echo> 
<help-echo> <help-echo> <help-echo> <menu-bar> <help-menu> 
<send-emacs-bug-report> <down-mouse-1> <mouse-movement> 
<mouse-1> <help-echo> <help-echo> <help-echo> <help-echo> 
<tool-bar> <kill-buffer> <help-echo> <help-echo> <tool-bar> 
<open-file> <help-echo> <help-echo> <help-echo> <help-echo> 
<help-echo> <help-echo> <help-echo> <help-echo> <help-echo> 
<help-echo> <menu-bar> <help-menu> <send-emacs-bug
-report>

Recent messages:
Note: file is write protected
Setting up indent for shell type bash
setting up indent stuff
Indentation variables are now local.
Indentation setup for shell type bash
newline: Buffer is read-only: #<buffer test.sh>
Setting up indent for shell type bash
setting up indent stuff
Indentation variables are now local.
Indentation setup for shell type bash

bug#4920: 23.1; (check-parens) problem with bash script case items unbalanced close paren

[Chong Yidong]

> Running (check-parens) fails on a bash script with case statements in
> because of the unmatched close ")".  Would it be possible to fix this,
> or tell me how to work around it?

I can't reproduce this: `M-x check-parens RET' on the following file
signals no error.  Please provice a more detailed bug report.

#!/bin/sh

case "$SYSNAME" in
    Darwin)
	;;
    Linux)

bug#4920: 23.1; (check-parens) problem with bash script case items unbalanced close paren

[Stefan Monnier]

>> Running (check-parens) fails on a bash script with case statements in
>> because of the unmatched close ")".  Would it be possible to fix this,
>> or tell me how to work around it?
> I can't reproduce this: `M-x check-parens RET' on the following file
> signals no error.  Please provice a more detailed bug report.

> #!/bin/sh

> case "$SYSNAME" in
>     Darwin)
> 	;;
>     Linux)

Those parentheses get a syntax-table property applied by
font-lock-syntactic-keywords, which means that the behavior will be
(hopefully) correct for the visible part of the text, and maybe for all
the text before that, but not necessarily for the text after window-end.


        Stefan

bug#4920: 23.1; (check-parens) problem with bash script case items unbalanced close paren

[Jeremy Hall]

Hi, 

I have noticed that the behaviour is intermittent.  I have just tried a
file (at the start) when it fails.  The cursor is left on the bracket of
xxxx) .  I moved around the file a bit and it kept failing, then, at the
end of the file it worked.  After that, it worked every where in the
file.  

I see it calls (scan-sexps (point-min) (point-max)) in lisp.el,
so its clearly intended to work over over the entire buffer.

For info, my elisp is:-  (here I just want the function to be aborted
with a message, if the file is incorrect, before the indenting starts).

;;
;; Tidy whitespace in a source file.
;;
(defun jah-src-tidy ()
	"Tidy whitespace in a source file."
	(interactive)
	(jah-set-tabs)
	(message "Tidy leading tabs ...")
	(tabify (point-min) (point-max))
	(message "Tidy trailing whitespace ...")
	(delete-trailing-whitespace)
	(message "Check brackets and quotes ...")
	(save-excursion (check-parens))
	(message "Tidy indentation ...")
	(indent-region (point-min) (point-max))
	(message "Source tidy complete.")
	)

Thanks,
Jeremy

On Sat, 2009-11-14 at 13:11 -0500, Stefan Monnier wrote:
> >> Running (check-parens) fails on a bash script with case statements in
> >> because of the unmatched close ")".  Would it be possible to fix this,
> >> or tell me how to work around it?
> > I can't reproduce this: `M-x check-parens RET' on the following file
> > signals no error.  Please provice a more detailed bug report.
> 
> > #!/bin/sh
> 
> > case "$SYSNAME" in
> >     Darwin)
> > 	;;
> >     Linux)
> 
> Those parentheses get a syntax-table property applied by
> font-lock-syntactic-keywords, which means that the behavior will be
> (hopefully) correct for the visible part of the text, and maybe for all
> the text before that, but not necessarily for the text after window-end.
> 
> 
>         Stefan
> 

bug#4920: 23.1; (check-parens) problem with bash script case items unbalanced close paren

[npostavs]

tags 4920 patch
quit

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> Running (check-parens) fails on a bash script with case statements in
>>> because of the unmatched close ")".  Would it be possible to fix this,
>>> or tell me how to work around it?
>> I can't reproduce this: `M-x check-parens RET' on the following file
>> signals no error.  Please provice a more detailed bug report.
>
>> #!/bin/sh
>
>> case "$SYSNAME" in
>>     Darwin)
>> 	;;
>>     Linux)
>
> Those parentheses get a syntax-table property applied by
> font-lock-syntactic-keywords, which means that the behavior will be
> (hopefully) correct for the visible part of the text, and maybe for all
> the text before that, but not necessarily for the text after
> window-end.

Right, seems to easily reproducible with

    echo '#!/bin/sh' > bug-4920-check-parens.sh
    seq -f '# etc %.0f' 1000 >> bug-4920-check-parens.sh 
    cat >>bug-4920-check-parens.sh <<EOF
         case "$SYSNAME" in
          Darwin)
                # ssss
            ;;
              Linux)
    EOF
    emacs -Q bug-4920-check-parens.sh -f check-parens

And easily fixable by adding a syntax-propertize call at the beginning
of check-parens:

diff --git i/lisp/emacs-lisp/lisp.el w/lisp/emacs-lisp/lisp.el
index ea7cce6..9786429 100644
--- i/lisp/emacs-lisp/lisp.el
+++ w/lisp/emacs-lisp/lisp.el
@@ -719,2 +719,3 @@ check-parens
   (interactive)
+  (syntax-propertize (point-max))
   (condition-case data

bug#4920: 23.1; (check-parens) problem with bash script case items unbalanced close paren

[Stefan Monnier]

> diff --git i/lisp/emacs-lisp/lisp.el w/lisp/emacs-lisp/lisp.el
> index ea7cce6..9786429 100644
> --- i/lisp/emacs-lisp/lisp.el
> +++ w/lisp/emacs-lisp/lisp.el
> @@ -719,2 +719,3 @@ check-parens
>    (interactive)
> +  (syntax-propertize (point-max))
>    (condition-case data

FWIW, the new support in Emacs-25 for calling syntax-propertize directly
from forward-parsing operations should make the above unnecessary (tho
I must admit I haven't tested this).


        Stefan

bug#4920: 23.1; (check-parens) problem with bash script case items unbalanced close paren

[Noam Postavsky]

On Sat, Jul 9, 2016 at 8:11 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> FWIW, the new support in Emacs-25 for calling syntax-propertize directly
> from forward-parsing operations should make the above unnecessary (tho
> I must admit I haven't tested this).

The test I mentioned above still fails on Emacs-25. Where is
syntax-propertize supposed to be called from?

bug#4920: 23.1; (check-parens) problem with bash script case items unbalanced close paren

[Stefan Monnier]

>> FWIW, the new support in Emacs-25 for calling syntax-propertize directly
>> from forward-parsing operations should make the above unnecessary (tho
>> I must admit I haven't tested this).
> The test I mentioned above still fails on Emacs-25.  Where is
> syntax-propertize supposed to be called from?

Things like forward-sexp, parse-partial-sexp, scan-sexps, ...


        Stefan

bug#4920: 23.1; (check-parens) problem with bash script case items unbalanced close paren

[npostavs]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> FWIW, the new support in Emacs-25 for calling syntax-propertize directly
>>> from forward-parsing operations should make the above unnecessary (tho
>>> I must admit I haven't tested this).
>> The test I mentioned above still fails on Emacs-25.  Where is
>> syntax-propertize supposed to be called from?
>
> Things like forward-sexp, parse-partial-sexp, scan-sexps, ...

Oh I see, but it's disabled by default, that's why my the test was still
failing.  Doing

    emacs -Q bug-4920-check-parens.sh --eval '(setq parse-sexp-lookup-properties t)' -f check-parens

succeeds.  Should this variable be set by sh-mode along with the other
syntax propertize settings?

bug#4920: 23.1; (check-parens) problem with bash script case items unbalanced close paren

[Stefan Monnier]

>     emacs -Q bug-4920-check-parens.sh --eval '(setq parse-sexp-lookup-properties t)' -f check-parens

> succeeds.  Should this variable be set by sh-mode along with the other
> syntax propertize settings?

Oh, yes, please, and thanks,


        Stefan

bug#4920: 23.1; (check-parens) problem with bash script case items unbalanced close paren

[npostavs]

[-- Attachment #1: Type: text/plain, Size: 654 bytes --]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>     emacs -Q bug-4920-check-parens.sh --eval '(setq parse-sexp-lookup-properties t)' -f check-parens
>
>> succeeds.  Should this variable be set by sh-mode along with the other
>> syntax propertize settings?
>
> Oh, yes, please, and thanks,

So it was already being set by sh-set-shell, but in only in case
sh-use-smie is nil.  Here is a patch to set it unconditionally.  I think
it's safe to apply to emacs-25 since I see that
parse-sexp-lookup-properties does get set to t at some point anyway (by
font-lock-fontify-syntactic-keywords-region I guess?), so this just
makes it happen a bit earlier.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: patch --]
[-- Type: text/x-diff, Size: 1215 bytes --]

From 28e9ec947f1edf44947b50009b5f3fec23027f45 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@gmail.com>
Date: Thu, 7 Jul 2016 21:14:48 -0400
Subject: [PATCH v1] Make sh-mode always use p-s-lookup-properties

This lets functions which rely on syntax-propertize for parsing
work correctly even before font lock has a chance to run.

* lisp/progmodes/sh-script.el (sh-set-shell): Set
parse-sexp-lookup-properties unconditionally (Bug #4920).
---
 lisp/progmodes/sh-script.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/progmodes/sh-script.el b/lisp/progmodes/sh-script.el
index 10e56d0..5288734 100644
--- a/lisp/progmodes/sh-script.el
+++ b/lisp/progmodes/sh-script.el
@@ -2430,8 +2430,8 @@ sh-set-shell
                       (funcall mksym "rules")
                       :forward-token  (funcall mksym "forward-token")
                       :backward-token (funcall mksym "backward-token")))
+        (setq-local parse-sexp-lookup-properties t)
         (unless sh-use-smie
-          (setq-local parse-sexp-lookup-properties t)
           (setq-local sh-kw-alist (sh-feature sh-kw))
           (let ((regexp (sh-feature sh-kws-for-done)))
             (if regexp
-- 
2.8.0

bug#4920: 23.1; (check-parens) problem with bash script case items unbalanced close paren

[Stefan Monnier]

> sh-use-smie is nil.  Here is a patch to set it unconditionally.

Thanks, looks good.

> I think it's safe to apply to emacs-25 since I see that

I also think it's safe, but I'll let John decide if it's important
enough (it's not super urgent).


        Stefan