bug#49596: 28.0.50; shadowfile.el: `shadow-define-literal-group' is broken

bug#49596: 28.0.50; shadowfile.el: `shadow-define-literal-group' is broken

[Max Brieiev]

The following command:

M-x shadow-define-literal-group RET
/ssh:user@host|ssh:root@otherhost: RET
~/foo.el RET RET

produces the wrong value of `shadow-literal-groups':

("//ssh:user@host|ssh:root@otherhost::~/foo.el")

This is incorrect tramp file name, it has an extra //...:: wrapping
around connection details.

In GNU Emacs 28.0.50 (build 6, x86_64-pc-linux-gnu, GTK+ Version 3.24.29, cairo version 1.17.4)
 of 2021-07-05 built on arch-max
Repository revision: 8028153ec8bb98855233a64833a596b5dce404f4
Repository branch: makepkg
Windowing system distributor 'System Description: Arch Linux

Configured using:
 'configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --mandir=/usr/share/man --with-gameuser=:games
 --with-sound=alsa --with-modules --without-gconf --without-gsettings
 --with-pgtk --enable-link-time-optimization --with-native-compilation
 --with-x-toolkit=gtk3 --without-xaw3d --without-compress-install
 '--program-transform-name=s/\([ec]tags\)/\1.emacs/'
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions
 -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security
 -fstack-clash-protection -fcf-protection -flto -fuse-linker-plugin
 -flto -fuse-linker-plugin'
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM HARFBUZZ JPEG JSON LCMS2
LIBOTF LIBSYSTEMD LIBXML2 MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER
PGTK PNG RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS XIM GTK3
ZLIB

Important settings:
  value of $LC_MESSAGES: 
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: ELisp/l

Minor modes in effect:
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  show-paren-mode: t
  global-company-mode: t
  company-mode: t
  display-battery-mode: t
  shell-dirtrack-mode: t
  recentf-mode: t
  marginalia-mode: t
  savehist-mode: t
  vertico-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/home/max/.emacs.d/elpa/transient-20210701.1116/transient hides /usr/share/emacs/28.0.50/lisp/transient

Features:
(shadow sort mail-extr emacsbug sendmail tramp-adb tramp-archive
tramp-cmds tramp-ftp tramp-gvfs magit-extras goto-addr magit-patch
magit-subtree magit-ediff ediff ediff-merg ediff-mult ediff-wind
ediff-diff ediff-help ediff-init ediff-util face-remap embark-consult
embark loadhist edebug debug backtrace misearch multi-isearch
bug-reference pulse vc-mtn vc-hg vc-bzr vc-src vc-sccs vc-svn vc-cvs
vc-rcs vc consult-vertico consult magit-bookmark bookmark shortdoc ffap
pp cl-print magit-submodule magit-obsolete magit-blame magit-stash
magit-reflog magit-bisect magit-push magit-pull magit-fetch magit-clone
magit-remote magit-commit magit-sequence magit-notes magit-worktree
magit-tag magit-merge magit-branch magit-reset magit-files magit-refs
magit-status magit magit-repos magit-apply magit-wip magit-log
which-func magit-diff smerge-mode diff git-commit log-edit pcvs-util
add-log magit-core magit-autorevert autorevert filenotify magit-margin
magit-transient magit-process with-editor server magit-mode transient
comp comp-cstr warnings magit-git magit-section magit-utils crm dash
vc-git diff-mode vc-dispatcher js imenu cc-mode cc-fonts cc-guess
cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs help-fns
radix-tree shadowfile dired-aux rx mule-util cursor-sensor
company-oddmuse company-keywords company-etags etags fileloop xref
project company-gtags company-dabbrev-code company-dabbrev company-files
company-clang company-capf company-cmake company-semantic
company-template company-bbdb org-element avl-tree generator ol-w3m
ol-rmail ol-mhe ol-irc ol-info org-habit org-agenda org-refile ol-gnus
nnselect gnus-search eieio-opt speedbar ezimage dframe gnus-art mm-uu
mml2015 mm-view mml-smime smime dig gnus-sum gnus-group gnus-undo
gnus-start gnus-dbus gnus-cloud nnimap nnmail mail-source utf7 netrc
nnoo gnus-spec gnus-int gnus-range message rmc rfc822 mml mml-sec epa
derived epg epg-config mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader gnus-win ol-eww eww xdg url-queue
thingatpt shr kinsoku svg dom puny mm-url ol-docview doc-view jka-compr
image-mode exif dired dired-loaddefs ol-bibtex bibtex ol-bbdb
solarized-light-high-contrast-theme solarized-palettes solarized
solarized-faces color yasnippet quail paren gnus nnheader gnus-util
rmail rmail-loaddefs rfc2047 rfc2045 ietf-drums text-property-search
mail-utils mm-util mail-prsvr company pcase battery dbus xml tramp-sh
docker-tramp tramp-cache tramp tramp-loaddefs trampver tramp-integration
files-x tramp-compat shell parse-time iso8601 ls-lisp recentf
tree-widget wid-edit cus-load ob-typescript org ob ob-tangle ob-ref
ob-lob ob-table ob-exp org-macro org-footnote org-src ob-comint
org-pcomplete pcomplete comint ansi-color ring org-list org-faces
org-entities time-date noutline outline org-version ob-emacs-lisp
ob-core ob-eval org-table ol org-keys org-compat org-macs org-loaddefs
format-spec find-func cal-menu calendar cal-loaddefs marginalia edmacro
kmacro savehist vertico orderless web-mode advice disp-table cl-extra
help-mode use-package use-package-ensure use-package-delight
use-package-diminish use-package-bind-key bind-key easy-mmode
use-package-core info package browse-url url url-proxy url-privacy
url-expand url-methods url-history url-cookie url-domsuf url-util
mailcap url-handlers url-parse auth-source cl-seq eieio eieio-core
cl-macs eieio-loaddefs password-cache json subr-x map url-vars seq
byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib iso-transl
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/pgtk-win pgtk-win term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow isearch
easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads dbusbind
inotify dynamic-setting font-render-setting cairo move-toolbar gtk
x-toolkit pgtk lcms2 multi-tty make-network-process native-compile
emacs)

Memory information:
((conses 16 678389 73007)
 (symbols 48 39894 0)
 (strings 32 163496 8895)
 (string-bytes 1 6805995)
 (vectors 16 77462)
 (vector-slots 8 1942308 188779)
 (floats 8 646 523)
 (intervals 56 12549 97)
 (buffers 992 33))

bug#49596: 28.0.50; shadowfile.el: `shadow-define-literal-group' is broken

[Michael Albinus]

Max Brieiev <max.brieiev@gmail.com> writes:

Hi Max,

> The following command:
>
> M-x shadow-define-literal-group RET
> /ssh:user@host|ssh:root@otherhost: RET
> ~/foo.el RET RET
>
> produces the wrong value of `shadow-literal-groups':
>
> ("//ssh:user@host|ssh:root@otherhost::~/foo.el")
>
> This is incorrect tramp file name, it has an extra //...:: wrapping
> around connection details.

Thanks for the report, I confirm the problem. Surprisingly, it isn't
easy to fix. I'll work on this.

Best regards, Michael.

bug#49596: 28.0.50; shadowfile.el: `shadow-define-literal-group' is broken

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Max,

>> The following command:
>>
>> M-x shadow-define-literal-group RET
>> /ssh:user@host|ssh:root@otherhost: RET
>> ~/foo.el RET RET
>>
>> produces the wrong value of `shadow-literal-groups':
>>
>> ("//ssh:user@host|ssh:root@otherhost::~/foo.el")
>>
>> This is incorrect tramp file name, it has an extra //...:: wrapping
>> around connection details.
>
> Thanks for the report, I confirm the problem. Surprisingly, it isn't
> easy to fix. I'll work on this.

Should be fixed now in master. I've extended also the test
shadow-test06-literal-groups in order to cover the problem, and to
ensure it keeps fixed in the future.

Pls report whether everything works fine now for you; I haven't tested
multi-hop remote files.

Best regards, Michael.

bug#49596: 28.0.50; shadowfile.el: `shadow-define-literal-group' is broken

[Max Brieiev]

Hi Michael,

> Pls report whether everything works fine now for you; I haven't tested
> multi-hop remote files.

Now the command adds an extra slash to the file path:

("/ssh:user@host|ssh:root@other:~//foo.el")
--------------------------------^^^--------

bug#49596: 28.0.50; shadowfile.el: `shadow-define-literal-group' is broken

[Michael Albinus]

Max Brieiev <max.brieiev@gmail.com> writes:

> Hi Michael,

Hi Max,

>> Pls report whether everything works fine now for you; I haven't tested
>> multi-hop remote files.
>
> Now the command adds an extra slash to the file path:
>
> ("/ssh:user@host|ssh:root@other:~//foo.el")
> --------------------------------^^^--------

Yep. I've fixed this in master.

Just a note about shadowfile.el: Some years ago, I've adapted it to use
Tramp instead of ange-ftp, which required internal changes. But I don't
use it myself.

So I hope you could help to make it more robust. Pls report any problem
you see with it. And if you have proposals to extend
shadowfile-tests.el, pls tell us.

Best regards, Michael.

bug#49596: 28.0.50; shadowfile.el: `shadow-define-literal-group' is broken

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Max,

>> Now the command adds an extra slash to the file path:
>>
>> ("/ssh:user@host|ssh:root@other:~//foo.el")
>> --------------------------------^^^--------
>
> Yep. I've fixed this in master.

Any progress on testing? If there are no further problems, I'd like to
close the bug.

Best regards, Michael.

bug#49596: 28.0.50; shadowfile.el: `shadow-define-literal-group' is broken

[Max Brieiev]

Hi Michael,

Sorry for being quiet for a long time.
Yes, the issue is fixed.

However, the interface of `shadow-define-literal-group` is somewhat
obscure:

- `M-x shadow-define-literal-group RET RET` will insert `nil` into
  `shadow-literal-groups`, which is probably not correct?

- The first prompt `Host identification or cluster name: ` isn't very
  descriptive. There is not much docs and I had to learn by trial and
  error that it accepts either tramp format like `/ssh:user@host:` for
  remote endpoints or a plain string like `my-linux-box` for local host
  names. And it seems if it can't resolve provided host name it will
  silently insert `nil` into `shadow-literal-groups`, which also doesn't
  seem correct.

That said, it is easier to set `shadow-literal-groups` manually, because
you can manually test accessibility of remote files and then simply
append the resulting tramp file name into the variable.

bug#49596: 28.0.50; shadowfile.el: `shadow-define-literal-group' is broken

[Michael Albinus]

Max Brieiev <max.brieiev@gmail.com> writes:

> Hi Michael,

Hi Max,

> Yes, the issue is fixed.

Thanks for the feedback.

> However, the interface of `shadow-define-literal-group` is somewhat
> obscure:
>
> - `M-x shadow-define-literal-group RET RET` will insert `nil` into
>   `shadow-literal-groups`, which is probably not correct?

Fixed.

> - The first prompt `Host identification or cluster name: ` isn't very
>   descriptive. There is not much docs and I had to learn by trial and
>   error that it accepts either tramp format like `/ssh:user@host:` for
>   remote endpoints or a plain string like `my-linux-box` for local host
>   names. And it seems if it can't resolve provided host name it will
>   silently insert `nil` into `shadow-literal-groups`, which also doesn't
>   seem correct.

I've adapted this. It prompts now for "Site name: ", and it has also a
help echo describing what a site name is (move the mouse over the prompt).
The error is fixed.

> That said, it is easier to set `shadow-literal-groups` manually, because
> you can manually test accessibility of remote files and then simply
> append the resulting tramp file name into the variable.

That's an alternative. But the interactive command shall also work as
expected.

Best regards, Michael.

bug#49596: 28.0.50; shadowfile.el: `shadow-define-literal-group' is broken

[Michael Albinus]

Version: 28.1

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Max,

>> Yes, the issue is fixed.
>
> Thanks for the feedback.
>
>> However, the interface of `shadow-define-literal-group` is somewhat
>> obscure:
>>
>> - `M-x shadow-define-literal-group RET RET` will insert `nil` into
>>   `shadow-literal-groups`, which is probably not correct?
>
> Fixed.
>
>> - The first prompt `Host identification or cluster name: ` isn't very
>>   descriptive. There is not much docs and I had to learn by trial and
>>   error that it accepts either tramp format like `/ssh:user@host:` for
>>   remote endpoints or a plain string like `my-linux-box` for local host
>>   names. And it seems if it can't resolve provided host name it will
>>   silently insert `nil` into `shadow-literal-groups`, which also doesn't
>>   seem correct.
>
> I've adapted this. It prompts now for "Site name: ", and it has also a
> help echo describing what a site name is (move the mouse over the prompt).
> The error is fixed.
>
>> That said, it is easier to set `shadow-literal-groups` manually, because
>> you can manually test accessibility of remote files and then simply
>> append the resulting tramp file name into the variable.
>
> That's an alternative. But the interactive command shall also work as
> expected.

No further comment, so I'm closing the bug. Feel free to report further
problems if available.

Best regards, Michael.