bug#6391: 23.2; Wrong face with here-docs in sh-mode

bug#6391: 23.2; Wrong face with here-docs in sh-mode

[Ralf Doering]

sh-mode displays parts of shellscripts with here documents with the
wrong face, if <<- is used. Suppose the following simple script (saved
as here.sh):

--8<---------------cut here---------------start------------->8---
#!/bin/bash
cat <<-+++ >/dev/null

+++

echo Outside here doc.
--8<---------------cut here---------------end--------------->8---

When opened in sh-mode, the part after the here document ("echo ...") is
still displayed with sh-heredoc face. This seems to be caused by '-'
used in the here-doc marker, as changing "<<-+++" to "<<+++" displays
the right face after the here doc. However, '<<-' is valid and needed in my
case. I can confirm this with a recent trunk as of "GNU Emacs 24.0.50.2
(x86_64-unknown-linux-gnu, GTK+ Version 2.18.3) of 2010-05-31 on seere"
in addition to this 23.2.

I already checked with emacs -Q here.sh, this does not change anything
about it, so I guess no configuration related misunderstandings.


In GNU Emacs 23.2.1 (x86_64-unknown-linux-gnu, GTK+ Version 2.18.3)
 of 2010-05-10 on seere
Windowing system distributor `The X.Org Foundation', version 11.0.10604000
configured using `configure  '--prefix=/space/ralf/emacs23.2''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: C
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: de_DE.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  savehist-mode: t
  display-time-mode: t
  tabbar-mwheel-mode: t
  tabbar-mode: t
  tabkey2-mode: t
  which-function-mode: t
  iswitchb-mode: t
  diff-auto-refine-mode: t
  delete-selection-mode: t
  show-paren-mode: t
  cua-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
M-x r e p o <tab> r t <tab> e m <tab> <return>

Recent messages:
Loading /home/ralf/src/elisp/nxhtml/2.08/nxhtml/nxhtml-autoload...done
Nxml/Nxhtml Autostart.el loaded in 0.1 seconds
Loading /home/ralf/src/elisp/nxhtml/2.08/autostart.el (source)...done
Loading /home/ralf/.emacs-gnu23...done
Loading /home/ralf/.custom-gnu23...
Loading time...done
Loading savehist...done
Loading /home/ralf/.custom-gnu23...done
For information about GNU Emacs and the GNU system, type C-h C-a.
Making completion list...

Load-path shadows:
~/erlang/lib/erlang/lib/tools-2.6.5/emacs/erlang hides /usr/share/emacs23/site-lisp/erlang/erlang
~/erlang/lib/erlang/lib/tools-2.6.5/emacs/erlang-start hides /usr/share/emacs23/site-lisp/erlang/erlang-start
/usr/share/emacs23/site-lisp/flim/md4 hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/md4
/usr/share/emacs23/site-lisp/flim/hex-util hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/hex-util
~/locallisp/htmlfontify hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/htmlfontify
/usr/share/emacs23/site-lisp/flim/sha1 hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/sha1
/usr/share/emacs23/site-lisp/dictionaries-common/ispell hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/textmodes/ispell
/usr/share/emacs23/site-lisp/dictionaries-common/flyspell hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/textmodes/flyspell
~/locallisp/js hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/progmodes/js
~/locallisp/org-mode/lisp/org-colview hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-colview
~/locallisp/org-mode/lisp/org-attach hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-attach
~/locallisp/org-mode/lisp/org-exp-blocks hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-exp-blocks
~/locallisp/org-mode/lisp/org-ascii hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-ascii
~/locallisp/org-mode/lisp/org-compat hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-compat
~/locallisp/org-mode/lisp/org-vm hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-vm
~/locallisp/org-mode/lisp/org-agenda hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-agenda
~/locallisp/org-mode/lisp/org-mew hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-mew
~/locallisp/org-mode/lisp/org-docbook hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-docbook
~/locallisp/org-mode/lisp/org-exp hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-exp
~/locallisp/org-mode/lisp/org-crypt hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-crypt
~/locallisp/org-mode/lisp/org-xoxo hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-xoxo
~/locallisp/org-mode/lisp/org-latex hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-latex
~/locallisp/org-mode/lisp/org-w3m hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-w3m
~/locallisp/org-mode/lisp/org hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org
~/locallisp/org-mode/lisp/org-irc hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-irc
~/locallisp/org-mode/lisp/org-gnus hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-gnus
~/locallisp/org-mode/lisp/org-faces hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-faces
~/locallisp/org-mode/lisp/org-list hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-list
~/locallisp/org-mode/lisp/org-feed hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-feed
~/locallisp/org-mode/lisp/org-bbdb hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-bbdb
~/locallisp/org-mode/lisp/org-remember hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-remember
~/locallisp/org-mode/lisp/org-mouse hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-mouse
~/locallisp/org-mode/lisp/org-bibtex hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-bibtex
~/locallisp/org-mode/lisp/org-plot hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-plot
~/locallisp/org-mode/lisp/org-protocol hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-protocol
~/locallisp/org-mode/lisp/org-rmail hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-rmail
~/locallisp/org-mode/lisp/org-publish hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-publish
~/locallisp/org-mode/lisp/org-archive hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-archive
~/locallisp/org-mode/lisp/org-table hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-table
~/locallisp/org-mode/lisp/org-install hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-install
~/locallisp/org-mode/lisp/org-clock hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-clock
~/locallisp/org-mode/lisp/org-footnote hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-footnote
~/locallisp/org-mode/lisp/org-macs hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-macs
~/locallisp/org-mode/lisp/org-src hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-src
~/locallisp/org-mode/lisp/org-habit hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-habit
~/locallisp/org-mode/lisp/org-jsinfo hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-jsinfo
~/locallisp/org-mode/lisp/org-mac-message hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-mac-message
~/locallisp/org-mode/lisp/org-html hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-html
~/locallisp/org-mode/lisp/org-mobile hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-mobile
~/locallisp/org-mode/lisp/org-freemind hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-freemind
~/locallisp/org-mode/lisp/org-datetree hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-datetree
~/locallisp/org-mode/lisp/org-info hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-info
~/locallisp/org-mode/lisp/org-wl hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-wl
~/locallisp/org-mode/lisp/org-mhe hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-mhe
~/locallisp/org-mode/lisp/org-id hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-id
~/locallisp/org-mode/lisp/org-indent hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-indent
~/locallisp/org-mode/lisp/org-icalendar hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-icalendar
~/locallisp/org-mode/lisp/org-inlinetask hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-inlinetask
~/locallisp/org-mode/lisp/org-timer hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/org/org-timer
/usr/share/emacs23/site-lisp/flim/sasl-cram hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/net/sasl-cram
/usr/share/emacs23/site-lisp/flim/sasl-ntlm hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/net/sasl-ntlm
/usr/share/emacs23/site-lisp/flim/sasl-digest hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/net/sasl-digest
/usr/share/emacs23/site-lisp/flim/sasl hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/net/sasl
/usr/share/emacs23/site-lisp/flim/hmac-md5 hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/net/hmac-md5
/usr/share/emacs23/site-lisp/flim/hmac-def hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/net/hmac-def
/usr/share/emacs23/site-lisp/flim/ntlm hides /space/ralf/emacs23.2/share/emacs/23.2/lisp/net/ntlm

Features:
(shadow sort gnus-cite mail-extr gnus-msg gnus-art mm-uu mml2015
epg-config mm-view smime dig gnus-sum nnoo gnus-group gnus-undo nnmail
mail-source format-spec gnus-start gnus-spec gnus-int gnus-range
gnus-win message ecomplete rfc822 mml mml-sec password-cache mm-decode
mm-bodies mm-encode mailcap mail-parse rfc2231 rfc2047 rfc2045 qp
ietf-drums mailabbrev gmm-utils mailheader canlock sha1 sha1-el hex-util
hashcash gnus gnus-ems nnheader gnus-util netrc mail-utils mm-util
mail-prsvr emacsbug help-mode view uniquify savehist time cus-start
cus-load assoc swank-clojure clojure-mode slime-fancy
slime-fontifying-fu slime-package-fu slime-references slime-xref-browser
tree-widget slime-scratch slime-highlight-edits slime-fancy-inspector
slime-c-p-c slime-editing-commands slime-autodoc slime-parse
slime-presentations slime-repl slime-fuzzy slime apropos hyperspec
thingatpt browse-url tabbar tabkey2 appmenu nxhtml-autostart
nxhtml-autoload majmodpri nxhtml-menu web-autoload nxhtml-base distel-ie
edb patmatch erl-service derl erlext epmd net-fsm erl distel erlang
etags compile tempo comint socks preview-latex tex-site auto-loads
sh-script executable hideshow which-func imenu windmove iswitchb
org-protocol org-latex org-export-latex org-beamer org-exp
org-exp-blocks org-agenda org org-footnote org-src org-list org-faces
org-compat org-entities org-macs time-date noutline outline footnote
remember org-install psvn byte-opt bytecomp byte-compile advice
advice-preload derived wid-edit pp elp ediff-merg ediff-diff ediff-wind
ediff-mult ediff-help ediff-init ediff-util dired regexp-opt magit
diff-mode git-blame git help-fns log-edit easymenu easy-mmode ring
pcvs-util add-log ewoc cl cl-19 server delsel ffap edmacro kmacro erin
highlight-current-line paren saveplace cua-base debian-el
debian-el-loaddefs w3m-load path-util poe pym static apel-ver product
erlang-start emacs-goodies-el emacs-goodies-custom
emacs-goodies-loaddefs dpkg-dev-el dpkg-dev-el-loaddefs bbdb-autoloads
tooltip ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd
font-setting tool-bar dnd fontset image fringe lisp-mode register page
menu-bar rfn-eshadow timer select scroll-bar mldrag mouse jit-lock
font-lock syntax facemenu font-core frame cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese hebrew
greek romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev loaddefs button
minibuffer faces cus-face files text-properties overlay md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote make-network-process dbusbind system-font-setting
font-render-setting gtk x-toolkit x multi-tty emacs)

bug#6391: 23.2; Wrong face with here-docs in sh-mode

[Glenn Morris]

Ralf Doering wrote:

> still displayed with sh-heredoc face. This seems to be caused by '-'
> used in the here-doc marker, as changing "<<-+++" to "<<+++" displays
> the right face after the here doc. However, '<<-' is valid and needed in my
> case.

It works fine with a more traditional delimiter such as "EOF".
I'd guess there is a regexp-quote issue, or perhaps a character syntax
issue, with your chosen "+++".

bug#6391: 23.2; Wrong face with here-docs in sh-mode

[Andreas Schwab]

Glenn Morris <rgm@gnu.org> writes:

> Ralf Doering wrote:
>
>> still displayed with sh-heredoc face. This seems to be caused by '-'
>> used in the here-doc marker, as changing "<<-+++" to "<<+++" displays
>> the right face after the here doc. However, '<<-' is valid and needed in my
>> case.
>
> It works fine with a more traditional delimiter such as "EOF".
> I'd guess there is a regexp-quote issue, or perhaps a character syntax
> issue, with your chosen "+++".

See sh-here-doc-open-re.  Only delimiters consisting of word characters
are recognized if unquoted.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#6391: 23.2; Wrong face with here-docs in sh-mode

[Glenn Morris]

Andreas Schwab wrote:

> See sh-here-doc-open-re.  Only delimiters consisting of word characters
> are recognized if unquoted.

Can you propose a better regexp? I do not know what the allowed
characters for heredoc delimiters actually are...

bug#6391: emacs: Wrong face with here-docs in sh-mode

[Dale Mellor]

Subject: Re: Bug#6391: emacs: Wrong face with here-docs in sh-mode

This bug is still present (since 2010!) in emacs-26.3.

Note that many of the previous comments are off the mark; it doesn't
matter what characters make up the end marker, it is simply the
presence of '-' at the start of the opening marker which triggers the 
problem.

It would be really nice if someone would increase the severity of this
bug.  The feature described may not be used very often, but when it is
used the bug is a bit of a nightmare, and means that syntax
highlighting of a shell script has to be turned off completely.

bug#6391: emacs: Wrong face with here-docs in sh-mode

[Noam Postavsky]

[-- Attachment #1: Type: text/plain, Size: 377 bytes --]

Dale Mellor <debian-bugs-ffbwxg@rdmp.org> writes:

> Note that many of the previous comments are off the mark; it doesn't
> matter what characters make up the end marker, it is simply the
> presence of '-' at the start of the opening marker which triggers the 
> problem.

I can't reproduce this.  The previous comments seem correct to me.  See
attached (produced with 26.3).


[-- Attachment #2: screenshot with -EOF and -+++ --]
[-- Type: image/png, Size: 15795 bytes --]

bug#6391: emacs: Wrong face with here-docs in sh-mode

[Dale Mellor]

On Sat, 2020-03-21 at 18:07 +0000, Dale Mellor wrote:
> Subject: Re: Bug#6391: emacs: Wrong face with here-docs in sh-mode
> 
> This bug is still present (since 2010!) in emacs-26.3.
> 
> Note that many of the previous comments are off the mark; it doesn't
> matter what characters make up the end marker, it is simply the
> presence of '-' at the start of the opening marker which triggers
> the 
> problem.
> 
> It would be really nice if someone would increase the severity of
> this
> bug.  The feature described may not be used very often, but when it
> is
> used the bug is a bit of a nightmare, and means that syntax
> highlighting of a shell script has to be turned off completely.

  Okay, I've made the error of combining '\' and '-' in the wrong
order, i.e. cat <<\-EOF...

  I can confirm that when these special characters are used in the
correct sequence, emacs highlighting works as it should, and this bug
is mostly fixed (but not for the '+++' case, which I think is probably
also wrong anyway).

  Sorry for the confusion, the bug can be considered closed as far as I
am concerned.

bug#6391: emacs: Wrong face with here-docs in sh-mode

[Noam Postavsky]

severity 6391 minor
quit

Dale Mellor <debian-bugs-ffbwxg@rdmp.org> writes:

>> It would be really nice if someone would increase the severity of
>> this bug.

>   Okay, I've made the error of combining '\' and '-' in the wrong
> order, i.e. cat <<\-EOF...

>   Sorry for the confusion, the bug can be considered closed as far as I
> am concerned.

I'll leave it open since the original bug still reproduces, but I'll put
the severity back to 'minor'.