bug#54613: 29.0.50; uncomment-region in sh-mode deletes extra char

bug#54613: 29.0.50; uncomment-region in sh-mode deletes extra char

[Filipp Gunbin]

- emacs -Q
- Visit this file:

--8<---------------cut here---------------start------------->8---
#!/bin/bash
curl -s -d @- http://example.org <<EOF \
    | cat -
a=b
EOF
--8<---------------cut here---------------end--------------->8---

- Put point at bol on the second line (where "curl" is)
- M-x comment-region
- M-x uncomment-region
- Notice that `-' after "cat" disappeared

Thanks.


In GNU Emacs 29.0.50 (build 11, x86_64-apple-darwin20.6.0, NS appkit-2022.60 Version 11.6.1 (Build 20G224))
 of 2022-03-28 built on fgunbin.local
Repository revision: 3f19a23c1f60757c54a0ec7d84c625d83766ee08
Repository branch: HEAD
System Description:  macOS 11.6.1

Configured using:
 'configure --enable-check-lisp-object-type --with-file-notification=no'

Configured features:
ACL GLIB GNUTLS LCMS2 LIBXML2 MODULES NS PDUMPER PNG RSVG SQLITE3
THREADS TOOLKIT_SCROLL_BARS WEBP XIM ZLIB

Important settings:
  value of $LC_CTYPE: UTF-8
  value of $LANG: ru_RU.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Shell-script

Minor modes in effect:
  csv-field-index-mode: t
  sh-electric-here-document-mode: t
  shell-dirtrack-mode: t
  minibuffer-depth-indicate-mode: t
  midnight-mode: t
  epa-global-mail-mode: t
  display-time-mode: t
  display-battery-mode: t
  desktop-save-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  file-name-shadow-mode: t
  jit-lock-debug-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow mail-extr emacsbug gnus-async gnus-ml disp-table cursor-sensor
qp imap rfc2104 nndraft nnmh utf-7 epa-file gnutls network-stream nsm
nnml nnfolder nnnil gnus-agent gnus-srvr gnus-score score-mode nnvirtual
gnus-msg gnus-art mm-uu mml2015 mm-view mml-smime smime dig nntp
gnus-cache gnus-sum shr pixel-fill kinsoku url-file url-dired svg dom
gnus-group gnus-undo gnus-start gnus-dbus gnus-cloud nnimap nnmail
mail-source utf7 netrc nnoo gnus-spec gnus-int gnus-range gnus-win gnus
nnheader range term/xterm xterm cus-start csv-mode sort dired-aux
sql-indent vc-git diff-mode vc-dispatcher noutline outline easy-mmode
sh-script smie executable w3m-load w3m doc-view jka-compr image-mode
exif timezone w3m-hist w3m-fb bookmark-w3m w3m-ems w3m-favicon w3m-image
w3m-proc w3m-util vlf-setup advice tramp-hdfs tramp-sh docker-tramp
tramp-cache tramp tramp-loaddefs ...)

Memory information:
((conses 16 250501 21292)
 (symbols 48 23802 6)
 (strings 32 84371 3359)
 (string-bytes 1 3043075)
 (vectors 16 46528)
 (vector-slots 8 1146153 195358)
 (floats 8 289 657)
 (intervals 56 864 89)
 (buffers 992 29))

bug#54613: 29.0.50; uncomment-region in sh-mode deletes extra char

[Filipp Gunbin]

On 28/03/2022 20:19 +0300, Filipp Gunbin wrote:

> - Put point at bol on the second line (where "curl" is)

Here, I meant "and select region up to eob"

bug#54613: 29.0.50; uncomment-region in sh-mode deletes extra char

[Lars Ingebrigtsen]

Filipp Gunbin <fgunbin@fastmail.fm> writes:

> - Notice that `-' after "cat" disappeared

I can reproduce this issue, too.

What happens is that we have the following:

curl -s -d @- http://example.org <<EOF \
# cat -

with point after "cat -".  In that case, (comment-enter-backward) will
skip back past the " -", because comment-end-skip is

"[ 	]*\\(\\s>\\|
\\)"

And...  \s> matches the " -\\'"?  Hm.  I'm not sure what's the real
problem here is.  Should comment-end-skip have that value in shell
scripts?  On the other hand, commenting out bits of a heredoc-introduced
command with a continuation line is pretty hairy in itself.

I've added Stefan to the CCs; perhaps he'll have some comments.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#54613: 29.0.50; uncomment-region in sh-mode deletes extra char

[Filipp Gunbin]

On 29/03/2022 14:59 +0200, Lars Ingebrigtsen wrote:

> Filipp Gunbin <fgunbin@fastmail.fm> writes:
>
>> - Notice that `-' after "cat" disappeared
>
> I can reproduce this issue, too.
>
> What happens is that we have the following:
>
> curl -s -d @- http://example.org <<EOF \
> # cat -
>
> with point after "cat -".  In that case, (comment-enter-backward) will
> skip back past the " -", because comment-end-skip is
>
> "[ 	]*\\(\\s>\\|
> \\)"
>
> And...  \s> matches the " -\\'"?  Hm.  I'm not sure what's the real
> problem here is.  Should comment-end-skip have that value in shell
> scripts?  On the other hand, commenting out bits of a heredoc-introduced
> command with a continuation line is pretty hairy in itself.
>

I'm seeing in debugging that "-" after "cat" gets syntax-class "comment
end", which seems wrong.  Just in buffer, outside of running
uncomment-region, it doesn't happen.

Filipp

bug#54613: 29.0.50; uncomment-region in sh-mode deletes extra char

[Filipp Gunbin]

On 29/03/2022 18:17 +0300, Filipp Gunbin wrote:

> On 29/03/2022 14:59 +0200, Lars Ingebrigtsen wrote:
>
>> Filipp Gunbin <fgunbin@fastmail.fm> writes:
>>
>>> - Notice that `-' after "cat" disappeared
>>
>> I can reproduce this issue, too.
>>
>> What happens is that we have the following:
>>
>> curl -s -d @- http://example.org <<EOF \
>> # cat -
>>
>> with point after "cat -".  In that case, (comment-enter-backward) will
>> skip back past the " -", because comment-end-skip is
>>
>> "[ 	]*\\(\\s>\\|
>> \\)"
>>
>> And...  \s> matches the " -\\'"?  Hm.  I'm not sure what's the real
>> problem here is.  Should comment-end-skip have that value in shell
>> scripts?  On the other hand, commenting out bits of a heredoc-introduced
>> command with a continuation line is pretty hairy in itself.
>>
>
> I'm seeing in debugging that "-" after "cat" gets syntax-class "comment
> end", which seems wrong.  Just in buffer, outside of running
> uncomment-region, it doesn't happen.
>
> Filipp

This is the suspect, in sh-font-lock-open-heredoc:

--8<---------------cut here---------------start------------->8---
      (if (nth 4 ppss)
          ;; The \n not only starts the heredoc but also closes a comment.
          ;; Let's close the comment just before the \n.
          (put-text-property (1- eol) eol 'syntax-table '(12))) ;">"
--8<---------------cut here---------------end--------------->8---

So we put ">" syntax on the "-" char, and it gets removed in
uncomment-region.

Don't know how to fix it, though.

Filipp