bug#31489: 25.3; Dired unable to open directory "/ssh:example.com"

bug#31489: 25.3; Dired unable to open directory "/ssh:example.com"

[Christoph Michelbach]

If you can't reproduce the problem with the version of Emacs you're running
right away, please read till the end of this email.


When a file's path conflicts with how an external resource is addressed, you can
tell Emacs to open the local path by preceding it with "/:". If you created a
file with

sudo touch "/ssh:example.com/file"

, you could access it via:

C-x C-f /:/ssh:example.com/file RET

However, if you leave "file" out, you don't get to see a dired buffer after
hitting return. Instead, the current windows are retained and the mini buffer
reads:

Host name must not match method "ssh"

The *Messages* buffer says:

tramp-file-name-handler: Host name must not match method "ssh"

You also cannot first navigate to "/" and then enter the directory by moving
point to it and hitting return. The same problem accompanied by the same error
message occurs.


If you use the development version of Emacs, the problem only occurs if the
directory's name ends in a colon. So if you're using the development version,
run

sudo mkdir "/ssh:exaple.com:"

instead to create a folder you can't access via dired. Again, the problem occurs
both when entering its path preceded by "/:" and when hitting return on the
dired buffer of "/".

I used the latest version on the master branch of the mirror on GitHub
(60ff8101449eea3a5ca4961299501efd83d011bd) to confirm that the problem still
exists.

bug#31489: 25.3; Dired unable to open directory "/ssh:example.com"

[Michael Albinus]

[-- Attachment #1: Type: text/plain, Size: 1116 bytes --]

Christoph Michelbach <michelbach94@gmail.com> writes:

Hi Christoph,

> When a file's path conflicts with how an external resource is
> addressed, you can tell Emacs to open the local path by preceding it
> with "/:". If you created a file with
>
> sudo touch "/ssh:example.com/file"
>
> , you could access it via:
>
> C-x C-f /:/ssh:example.com/file RET
>
> However, if you leave "file" out, you don't get to see a dired buffer after
> hitting return. Instead, the current windows are retained and the mini buffer
> reads:
>
> Host name must not match method "ssh"

Thanks for the bug report. The appended patch, towards the master branch
of the Emacs repository, fixes it for me. Could you pls check?

> You also cannot first navigate to "/" and then enter the directory by moving
> point to it and hitting return. The same problem accompanied by the same error
> message occurs.

You should navigate to "/:". Then it works, after applying the patch.

However, before committing the patch I need to run further tests. The
patch let tramp-tests.el fail, which must be investigated further.

Best regards, Michael.


[-- Attachment #2: Type: text/plain, Size: 2367 bytes --]

diff --git a/lisp/files.el b/lisp/files.el
index d98d09bb1e..2478a22046 100644
--- a/lisp/files.el
+++ b/lisp/files.el
@@ -7056,7 +7056,8 @@ file-name-non-special
 		   '(nil 0))))
 	 method
 	 ;; Copy ARGUMENTS so we can replace elements in it.
-	 (arguments (copy-sequence arguments)))
+	 (arguments (copy-sequence arguments))
+         file-name-handler-alist)
     (if (symbolp (car file-arg-indices))
 	(setq method (pop file-arg-indices)))
     ;; Strip off the /: from the file names that have it.
@@ -7064,27 +7065,20 @@ file-name-non-special
       (while (consp file-arg-indices)
 	(let ((pair (nthcdr (car file-arg-indices) arguments)))
 	  (and (car pair)
-	       (string-match "\\`/:" (car pair))
-	       (setcar pair
-		       (if (= (length (car pair)) 2)
-			   "/"
-			 (substring (car pair) 2)))))
+	       (setcar pair (file-name-unquote (car pair)))))
 	(setq file-arg-indices (cdr file-arg-indices))))
     (pcase method
       (`identity (car arguments))
       (`add (file-name-quote (apply operation arguments)))
       (`buffer-file-name
-       (let ((buffer-file-name
-              (if (string-match "\\`/:" buffer-file-name)
-                  (substring buffer-file-name (match-end 0))
-                buffer-file-name)))
+       (let ((buffer-file-name (file-name-unquote buffer-file-name)))
          (apply operation arguments)))
       (`insert-file-contents
        (let ((visit (nth 1 arguments)))
          (unwind-protect
              (apply operation arguments)
            (when (and visit buffer-file-name)
-             (setq buffer-file-name (concat "/:" buffer-file-name))))))
+             (setq buffer-file-name (file-name-quote buffer-file-name))))))
       (`unquote-then-quote
        ;; We can't use `cl-letf' with `(buffer-local-value)' here
        ;; because it wouldn't work during bootstrapping.
@@ -7093,7 +7087,7 @@ file-name-non-special
          ;; `verify-visited-file-modtime' action, which takes a buffer
          ;; as only optional argument.
          (with-current-buffer (or (car arguments) buffer)
-           (let ((buffer-file-name (substring buffer-file-name 2)))
+           (let ((buffer-file-name (file-name-unquote buffer-file-name)))
              ;; Make sure to hide the temporary buffer change from the
              ;; underlying operation.
              (with-current-buffer buffer

bug#31489: 25.3; Dired unable to open directory "/ssh:example.com"

[Christoph Michelbach]

After applying your patch, I can enter the directory with the SSH resource name
by hitting enter on it if and only if I start at "/:".

However, if I start at "/:" instead of "/" in my regular installation of Emacs,
I cannot enter any directory. This error message is generated (I hit return on
the directory "mnt"):

apply: Setting current directory: No such file or directory, /:/

If this didn't work thus far, it's probably a bad idea to force people to start
at "/:" instead of "/".

bug#31489: 25.3; Dired unable to open directory "/ssh:example.com"

[Michael Albinus]

Christoph Michelbach <michelbach94@gmail.com> writes:

Hi Christoph,

> After applying your patch, I can enter the directory with the SSH
> resource name by hitting enter on it if and only if I start at "/:".

Of course. Otherwise, you would try to open "/ssh:example.com:", which is
a Tramp file name.

> If this didn't work thus far, it's probably a bad idea to force people
> to start at "/:" instead of "/".

Meanwhile, I don't believe it is the way to go. If you enter the
directory "/:/ssh:example.com:/", and want to access any file there, none
of the file name handlers would work due to the quoting with "/:". Even
uncompressing a file on the fly doesn't work, like /:/ssh:example.com:/file.gz".

This is not what we want. We want just to hinder Tramp taking care of
"/ssh:example.com:". If you don't use Tramp at all, set `tramp-mode' to
nil in your .emacs. Everything shall work then as expected. This shall
work also with older Emacsen, like 25.3.

If you plan to still use Tramp, tell me so. I will add a variable then,
where you could give a regexp for file names to be ignored by Tramp, like
"\\`/ssh:example\\.com:".

I will use bug#31489 to continue the work on the bug it has uncovered:
If there is a magic function with two file name arguments, and the first
argument is quoted with "/:", then *both* arguments are handled like
they were quoted. Something like (copy-file "/:/ssh:example.com/file"
"/sudo::/etc/file") won't work anymore.

Best regards, Michael.

bug#31489: 25.3; Dired unable to open directory "/ssh:example.com"

[Christoph Michelbach]

On Sat, 2018-05-19 at 20:10 +0200, Michael Albinus wrote:
> > After applying your patch, I can enter the directory with the SSH
> > resource name by hitting enter on it if and only if I start at "/:".
> Of course. Otherwise, you would try to open "/ssh:example.com:", which is
> a Tramp file name.

Yes, but that's only because of how dired-find-file works (or the functions
called by it). The user would expect a dired buffer of the external resource to
be opened if tramp files were displayed in the dired buffer for "/" and they
actually hit return on a tramp file. But they're not, so the user does not hit
return on a tramp file. They hit return on a directory which is part of their
local (V)FS. When the user hits return on some directory, they expect that
directory to be opened. In the current implementation of dired, this is not the
case.

From the user's perspective, these are separate bugs:

1. The user is unable to access some directories by entering their paths.
2. Hitting return on a directory does not load the directory.

If the user has to enter a path different from the actual path to avoid loading
an external resource, that's perfectly acceptable. At some point, what the user
means by their input has to be clarified and if they enter a location,
disambiguating it is their job. But if the user hits enter on a directory, they
just want that directory to be loaded. The function called upon hitting return
should take care of disambiguation.

bug#31489: 25.3; Dired unable to open directory "/ssh:example.com"

[Michael Albinus]

Christoph Michelbach <michelbach94@gmail.com> writes:

Hi Christoph,

> On Sat, 2018-05-19 at 20:10 +0200, Michael Albinus wrote:
>> > After applying your patch, I can enter the directory with the SSH
>> > resource name by hitting enter on it if and only if I start at "/:".
>> Of course. Otherwise, you would try to open "/ssh:example.com:", which is
>> a Tramp file name.
>
> Yes, but that's only because of how dired-find-file works (or the
> functions called by it). The user would expect a dired buffer of the
> external resource to be opened if tramp files were displayed in the
> dired buffer for "/" and they actually hit return on a tramp file. But
> they're not, so the user does not hit return on a tramp file. They hit
> return on a directory which is part of their local (V)FS. When the
> user hits return on some directory, they expect that directory to be
> opened. In the current implementation of dired, this is not the case.

I don't see how this could be avoided. Of course, dired could quote any
directory name with "/:" when opening a directory with a file name
dedicated to Tramp (or another file name handling library). But this
would discard *any* file name handlers in this subdirectory, including
something like uncrompressing files, as jka-compr does, or decrypting
files, whis is performed by epa.

> From the user's perspective, these are separate bugs:
>
> 1. The user is unable to access some directories by entering their paths.
> 2. Hitting return on a directory does not load the directory.
>
> If the user has to enter a path different from the actual path to
> avoid loading an external resource, that's perfectly acceptable. At
> some point, what the user means by their input has to be clarified and
> if they enter a location, disambiguating it is their job. But if the
> user hits enter on a directory, they just want that directory to be
> loaded. The function called upon hitting return should take care of
> disambiguation.

You haven't answered my question: Could you live w/o Tramp, and set
tramp-mode to nil? Or do you want still use Tramp, and I shall extend
Tramp with an "exclude file names" feature?

Both variants would be a solution for this.

Best regards, Michael.

bug#31489: 25.3; Dired unable to open directory "/ssh:example.com"

[Christoph Michelbach]

On Sun, 2018-05-20 at 20:30 +0200, Michael Albinus wrote:
> You haven't answered my question: Could you live w/o Tramp, and set
> tramp-mode to nil? Or do you want still use Tramp, and I shall extend
> Tramp with an "exclude file names" feature?
> 
> Both variants would be a solution for this.

I haven't done so because I'm not suffering from this bug atm. I do use
tramp but I don't have the problem described in the bug report in
practice. I merely wanted to find out to what extent dired can be used
as a file browser and explored some edge cases. While doing so, I found
a few problems and this is the first one (and thus far only one found
while explicitly looking for bugs) I have reported. To evade the
problem in practice, one could simply use symlinks for the folders with
the weird names. Or one could use a single simlink created via

sudo ln -s / /f

and then always start at "/f". Evading this problem in practice is
simple, but it seems to me that HAVING to evade it shows poor
usability.


> I don't see how this could be avoided. Of course, dired could quote any
> directory name with "/:" when opening a directory with a file name
> dedicated to Tramp (or another file name handling library). But this
> would discard *any* file name handlers in this subdirectory, including
> something like uncrompressing files, as jka-compr does, or decrypting
> files, whis is performed by epa.

So the problem is that this way of quoting is also used in different
places, which means that it's not possible to view the contents of
"/ssh:example.com/tarball.tar" in Emacs without removing tramp /
disabling tramp / adding an exception for this specific path?

bug#31489: 25.3; Dired unable to open directory "/ssh:example.com"

[Michael Albinus]

Christoph Michelbach <michelbach94@gmail.com> writes:

Hi Christoph,

> On Sun, 2018-05-20 at 20:30 +0200, Michael Albinus wrote:
>> You haven't answered my question: Could you live w/o Tramp, and set
>> tramp-mode to nil? Or do you want still use Tramp, and I shall extend
>> Tramp with an "exclude file names" feature?
>>
>> Both variants would be a solution for this.
>
> I haven't done so because I'm not suffering from this bug atm. I do use
> tramp but I don't have the problem described in the bug report in
> practice. I merely wanted to find out to what extent dired can be used
> as a file browser and explored some edge cases. While doing so, I found
> a few problems and this is the first one (and thus far only one found
> while explicitly looking for bugs) I have reported. To evade the
> problem in practice, one could simply use symlinks for the folders with
> the weird names. Or one could use a single simlink created via
>
> sudo ln -s / /f
>
> and then always start at "/f".

That was my first idea, when reading your bug report. But it doesn't
work. `file-truename' would convert any file name
"/f/ssh:example.com:/..." into "/ssh:example.com:/...", and we're back
to the problem.

> Evading this problem in practice is simple, but it seems to me that
> HAVING to evade it shows poor usability.

The problem is not related to dired. It comes from how file name
handlers are designed in Emacs. They are active only based on the file
*name*, not on anything else of the related file. That means, any file
*name* which matched Tramp's remote file name syntax, activates Tramp.

>> I don't see how this could be avoided. Of course, dired could quote any
>> directory name with "/:" when opening a directory with a file name
>> dedicated to Tramp (or another file name handling library). But this
>> would discard *any* file name handlers in this subdirectory, including
>> something like uncrompressing files, as jka-compr does, or decrypting
>> files, whis is performed by epa.
>
> So the problem is that this way of quoting is also used in different
> places, which means that it's not possible to view the contents of
> "/ssh:example.com/tarball.tar" in Emacs without removing tramp /
> disabling tramp / adding an exception for this specific path?

Well, viewing the contents of a *.tar file is not performed via a file
name handler, so it is possible to view the contents of
"/:/tmp/foo.tar", for example. See the variable `file-name-handler-alist'
for the file name handlers which are invoked via this mechanism, and
which would be disabled by quoting a file name (prefixing it with "/:").

(On the contrary, in Emacs 27.0.50, viewing the contents of a tar file
by the special file name syntax "/tmp/foo.tar/" could fail, when the
file name is quoted like "/:/tmp/foo.tar/".)

You might read (info "(elisp) Magic File Names") for further
information about the underlying mechanism.

---

Finally, I've pushed a patch to Emacs' master branch which should solve
your problem. Add the following line to your .emacs:

(setq tramp-ignored-file-name-regexp "\\`/ssh:example\\.com:")

That's it. Tramp's documentation tells something about.

Best regards, Michael.

bug#31489: 25.3; Dired unable to open directory "/ssh:example.com"

[Michael Albinus]

Version: 27.1

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Christoph,

> Finally, I've pushed a patch to Emacs' master branch which should solve
> your problem. Add the following line to your .emacs:
>
> (setq tramp-ignored-file-name-regexp "\\`/ssh:example\\.com:")
>
> That's it. Tramp's documentation tells something about.

Additionally, I have enhanced `file-name-non-special' to work proper for
files in a directory like "/:/ssh:example.com:/". This is independent
from the Tramp configuration shown above.

If you have a chance to build Emacs from git, you could test it.

I'm closing the bug.

Best regards, Michael.

bug#31489: 25.3; Dired unable to open directory "/ssh:example.com"

[Christoph Michelbach]

Can't the problems be avoided by establishing that paths to tramp paths can't
start with two slashes?

To paths on Unix-like systems, you can add more slashes. So with that rule in
place, the user could simply start weird paths with two slashes and dired could
be changed to just add another slash to the beginning of the path if the file
the user hit enter on is in "/".

The path "/ssh:example.com:" would be passed to other functions as
"//ssh:example.com:". When Emacs asks the OS to access "//ssh:example.com:",
it's just the same as if it asked it to access "/ssh:example.com:".

bug#31489: 25.3; Dired unable to open directory "/ssh:example.com"

[Michael Albinus]

Christoph Michelbach <michelbach94@gmail.com> writes:

Hi Christoph,

> Can't the problems be avoided by establishing that paths to tramp paths can't
> start with two slashes?
>
> To paths on Unix-like systems, you can add more slashes. So with that rule in
> place, the user could simply start weird paths with two slashes and dired could
> be changed to just add another slash to the beginning of the path if the file
> the user hit enter on is in "/".
>
> The path "/ssh:example.com:" would be passed to other functions as
> "//ssh:example.com:". When Emacs asks the OS to access "//ssh:example.com:",
> it's just the same as if it asked it to access "/ssh:example.com:".

This is changed by `substitute-in-file-name':

(substitute-in-file-name "//ssh:example.com:") => "/ssh:example.com:"

See the docstring of `substitute-in-file-name', and
(info "(emacs) Minibuffer File")

Don't worry, I'm just working on a solution for you.

Best regards, Michael.