bug#18380: 24.3.93; sh-script multiline quoted subshell wreaks havoc with indentation

bug#18380: 24.3.93; sh-script multiline quoted subshell wreaks havoc with indentation

[Carlos Pita]

This will see the { as quoted:

"$(echo  "{")"

while this will not:

"$(echo
   "{")"

after this line indentation is broken as emacs thinks there is an
unclosed open brace.

The problem with multiline quoted subshells is more general, as
remarked by this comment in sh-script.el:

  ;; FIXME: This can (and often does) match multiple lines, yet it makes no
  ;; effort to handle multiline cases correctly, so it ends up being
  ;; rather flaky.

Regards
--
Carlos

GNU Emacs 24.3.93.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.12.2)
 of 2014-08-27 on carlos
Windowing system distributor `The X.Org Foundation', version 11.0.11600000
Configured using:
 `configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-x-toolkit=gtk3 --with-xft
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong
 --param=ssp-buffer-size=4' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro'

Important settings:
  value of $LC_COLLATE: en_US.UTF-8
  value of $LC_CTYPE: en_US.UTF-8
  value of $LC_MESSAGES: en_US.UTF-8
  value of $LC_MONETARY: en_US.UTF-8
  value of $LC_NUMERIC: en_US.UTF-8
  value of $LC_TIME: en_US.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Shell-script

Minor modes in effect:
  shell-dirtrack-mode: t
  sh-electric-here-document-mode: t
  show-paren-mode: t
  eldoc-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  ido-ubiquitous-mode: t
  global-auto-complete-mode: t
  auto-complete-mode: t
  ido-everywhere: t
  winner-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<down> <down> <down> <down> <down> C-w <up> <up> <up> 
<up> <up> SPC C-x C-s <backspace> <down> C-e <backspace> 
<backspace> " <left> d s a d s a C-x C-s <down> <up> 
<up> <left> SPC <down> C-x C-s <up> <backspace> SPC 
C-x C-s <backspace> <down> <right> <right> <right> 
<right> <right> <right> C-x C-s <right> { <up> <up> 
C-e C-d C-d C-d d <backspace> C-d C-d C-d <down> <down> 
<down> <down> <down> } <backspace> <up> <up> <up> <up> 
<up> C-e <C-left> <left> <left> <backspace> <return> 
<return> <tab> SPC SPC SPC SPC <up> <up> <left> C-k 
e c h o <return> <return> SPC SPC <backspace> <backspace> 
<backspace> <backspace> <return> SPC SPC SPC " { " 
) " <return> <up> <up> C-a C-SPC <down> <down> M-w 
<up> <up> <up> <up> <return> C-y <up> <up> C-e C-d 
C-d <down> <down> <backspace> <down> <down> <down> 
C-k C-k C-k C-k C-k C-k C-k C-k C-k C-k x C-s <backspace> 
<backspace> <backspace> <backspace> <up> <up> <down> 
<down> <backspace> <backspace> <return> <return> C-x 
C-s <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <return> C-x C-s <backspace> 
<backspace> <backspace> <backspace> <backspace> } <backspace> 
<up> <up> <up> <up> <return> } <backspace> <backspace> 
<down> <down> <down> x C-s <backspace> <backspace> 
<up> <up> <down> <down> <down> <up> <backspace> } <backspace> 
C-x C-s <up> <up> <return> <backspace> C-x C-s <down> 
<down> <down> } <backspace> <backspace> <backspace> 
<backspace> " <return> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <return> C-a C-k 
<return> <backspace> C-a C-k <tab> <tab> <tab> <tab> 
<tab> s a } C-a C-k s a d s a d s a <tab> C-a C-k <up> 
<up> <up> <up> <up> SPC <down> <up> <backspace> C-SPC 
<tab> C-SPC <down> <down> <down> <down> M-q <down> 
C-/ <up> <up> <up> <up> C-SPC <down> <down> <down> 
<down> M-w <down> <down> M-x r e p o r t - e m <tab> 
<return>

Recent messages:
Saving file /tmp/test.sh...
Wrote /tmp/test.sh
Saving file /tmp/test.sh...
Wrote /tmp/test.sh
Saving file /tmp/test.sh...
Wrote /tmp/test.sh
Auto-saving...done
Mark activated [2 times]
Undo!
Mark activated

Load-path shadows:
~/.emacs.d/lisp/rmail hides /usr/share/emacs/24.3.93/lisp/mail/rmail

Features:
(eieio-opt speedbar sb-image ezimage dframe shadow emacsbug xterm
descr-text tramp-cache tramp tramp-compat tramp-loaddefs trampver shell
view woman man misearch multi-isearch cus-edit cus-start cus-load
sh-script smie executable gnus-draft mailalias smtpmail sendmail oauth2
warnings json plstore nnir url-http url-gw url-auth url-queue face-remap
org-colview diary-lib diary-loaddefs cal-iso org-agenda bookmark
thingatpt tabify image-file org-capture pp rect vc-git org-element
org-rmail org-mhe org-irc org-info org-gnus org-docview doc-view
jka-compr image-mode dired org-bibtex bibtex org-bbdb org-w3m mule-util
shr-color color shr browse-url gnus-dup sort smiley gnus-cite mm-archive
mail-extr gnus-async gnus-bcklg qp gnus-ml disp-table gnus-topic nndraft
nnmh nnfolder utf-7 epa-file epa derived epg gnutls network-stream
starttls nnimap parse-time tls utf7 netrc gnus-agent gnus-srvr
gnus-score score-mode nnvirtual gnus-msg gnus-art mm-uu mml2015
epg-config mm-view mml-smime smime dig nntp gnus-cache gnus-sum nnoo
gnus-group gnus-undo nnmail mail-source google-contacts-message
google-contacts xml url-cache url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf mailcap url-util url-parse
auth-source eieio eieio-core password-cache url-vars google-oauth
gnus-start gnus-spec gnus-int gnus-range message idna rfc822 mml mml-sec
mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
ietf-drums mailabbrev gmm-utils mailheader gnus-win gnus gnus-ems
nnheader gnus-util mail-utils mm-util mail-prsvr wid-edit server flymake
compile paredit paren eldoc cl-macs ob-python ob-R org byte-opt bytecomp
byte-compile cconv org-macro org-footnote org-pcomplete pcomplete
org-list org-faces org-entities time-date noutline outline easy-mmode
org-version ob-emacs-lisp ob ob-tangle org-src ob-ref ob-lob ob-table
ob-keys ob-exp ob-comint comint ansi-color ob-core ob-eval org-compat
org-macs org-loaddefs format-spec find-func cal-menu calendar
cal-loaddefs yasnippet help-mode ido-ubiquitous cl gv advice help-fns
auto-complete-config auto-complete popup imenu-anywhere imenu ido
windmove winner ring edmacro kmacro cl-loaddefs cl-lib info easymenu
package wombat-theme tooltip electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment lisp-mode prog-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dbusbind
gfilenotify dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty emacs)

Memory information:
((conses 16 685711 94948)
 (symbols 48 57707 0)
 (miscs 40 4316 4531)
 (strings 32 245451 4719)
 (string-bytes 1 4862500)
 (vectors 16 51937)
 (vector-slots 8 1688568 35010)
 (floats 8 1010 1797)
 (intervals 56 37504 525)
 (buffers 960 93)
 (heap 1024 68950 5163))

bug#18380: 24.3.93; sh-script multiline quoted subshell wreaks havoc with indentation

[Stefan Monnier]

> This will see the { as quoted:
> "$(echo  "{")"
>
> while this will not:
>
> "$(echo
>    "{")"

Actually, it depends: sometimes it will sometimes it won't :-(

> after this line indentation is broken as Emacs thinks there is an
> unclosed open brace.
>
> The problem with multiline quoted subshells is more general, as
> remarked by this comment in sh-script.el:
>
>   ;; FIXME: This can (and often does) match multiple lines, yet it makes no
>   ;; effort to handle multiline cases correctly, so it ends up being
>   ;; rather flaky.

Indeed, the problem you show is the direct result of the problem alluded
to in the above comment.  I installed the patch below which seems to help.


        Stefan


=== modified file 'lisp/progmodes/sh-script.el'
--- lisp/progmodes/sh-script.el	2014-07-21 01:41:59 +0000
+++ lisp/progmodes/sh-script.el	2014-09-03 00:37:50 +0000
@@ -1051,13 +1051,11 @@
   "Search for a subshell embedded in a string.
 Find all the unescaped \" characters within said subshell, remembering that
 subshells can nest."
-  ;; FIXME: This can (and often does) match multiple lines, yet it makes no
-  ;; effort to handle multiline cases correctly, so it ends up being
-  ;; rather flaky.
   (when (eq ?\" (nth 3 (syntax-ppss))) ; Check we matched an opening quote.
     ;; bingo we have a $( or a ` inside a ""
     (let (;; `state' can be: double-quote, backquote, code.
           (state (if (eq (char-before) ?`) 'backquote 'code))
+          (startpos (point))
           ;; Stacked states in the context.
           (states '(double-quote)))
       (while (and state (progn (skip-chars-forward "^'\\\\\"`$()" limit)
@@ -1088,7 +1086,12 @@
                  (`double-quote nil)
                  (_ (setq state (pop states)))))
           (_ (error "Internal error in sh-font-lock-quoted-subshell")))
-        (forward-char 1)))))
+        (forward-char 1))
+      (when (< startpos (line-beginning-position))
+        (put-text-property startpos (point) 'syntax-multiline t)
+        (add-hook 'syntax-propertize-extend-region-functions
+                  'syntax-propertize-multiline nil t))
+      )))
 
 
 (defun sh-is-quoted-p (pos)

bug#18380: 24.3.93; sh-script multiline quoted subshell wreaks havoc with indentation

[Stefan Kangas]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> This will see the { as quoted:
>> "$(echo  "{")"
>>
>> while this will not:
>>
>> "$(echo
>>    "{")"
>
> Actually, it depends: sometimes it will sometimes it won't :-(
>
>> after this line indentation is broken as Emacs thinks there is an
>> unclosed open brace.
>>
>> The problem with multiline quoted subshells is more general, as
>> remarked by this comment in sh-script.el:
>>
>>   ;; FIXME: This can (and often does) match multiple lines, yet it makes no
>>   ;; effort to handle multiline cases correctly, so it ends up being
>>   ;; rather flaky.
>
> Indeed, the problem you show is the direct result of the problem alluded
> to in the above comment.  I installed the patch below which seems to help.

I can't reproduce the original problem on current master.

Since there also has been no further updates here in 5 years, I'm going
to close this bug report.  If any of this is still an issue, please
reopen the bug report.

Best regards,
Stefan Kangas