bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

[Damien Cassou]

1. emacs -Q
2. M-x browse-url-emacs RET https://gnu.org
3. C-h v default-directory

Actual: "https://"

Expected: a directory such as (1) Emacs' startup directory or (2) a
temporary directory.

In GNU Emacs 26.0.90 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.26)
 of 2018-01-08 built on luz4
Repository revision: 918a052a422c05b5f78fd7c702eb6e3ee189fa18
Windowing system distributor 'Fedora Project', version 11.0.11906000
System Description:	Fedora release 27 (Twenty Seven)

-- 
Damien Cassou
http://damiencassou.seasidehosting.st

"Success is the ability to go from one failure to another without
losing enthusiasm." --Winston Churchill

bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 550 bytes --]

Damien Cassou <damien@cassou.me> writes:

> 1. emacs -Q
> 2. M-x browse-url-emacs RET https://gnu.org
> 3. C-h v default-directory
>
> Actual: "https://"
>
> Expected: a directory such as (1) Emacs' startup directory or (2) a
> temporary directory.

My guess is that "https://gnu.org/" should be the default directory in
this case.  For instance:

  M-x browse-url-emacs RET https://gnu.org/index.html
  C-h v default-directory
  https://gnu.org/

Which looks correct to me, so I'd say that the issue is that the
trailing / is omitted.

Cheers,
Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

[Damien Cassou]

Nicolas Petton <nicolas@petton.fr> writes:
> My guess is that "https://gnu.org/" should be the default directory in
> this case.

do you mean that packages shouldn't rely on default-directory being a
directory? For example, dired-jump says

   dired-insert-directory: Setting current directory: No such file or
   directory, https:/https:/https:/

and eshell has https:/ as directory (in which nothing can be done).

-- 
Damien Cassou
http://damiencassou.seasidehosting.st

"Success is the ability to go from one failure to another without
losing enthusiasm." --Winston Churchill

bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 330 bytes --]

Damien Cassou <damien@cassou.me> writes:


> do you mean that packages shouldn't rely on default-directory being a
> directory?

I meant that `default-directory' doesn't always exist (it's the case
with `C-x C-f /foo/bar/baz.txt' for instance).

Now having "https://" as the default directory looks wrong, I agree with
you.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

[Eli Zaretskii]

> From: Damien Cassou <damien@cassou.me>
> Date: Tue, 13 Feb 2018 13:52:23 +0100
> 
> 1. emacs -Q
> 2. M-x browse-url-emacs RET https://gnu.org
> 3. C-h v default-directory
> 
> Actual: "https://"
> 
> Expected: a directory such as (1) Emacs' startup directory or (2) a
> temporary directory.

Michael, could you please look into fixing this?

bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

> Michael, could you please look into fixing this?

Will do. I'm short in time these days, so it might take some more days
until I could check.

Best regards, Michael.

bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 308 bytes --]

Michael Albinus <michael.albinus@gmx.de> writes:

> Will do. I'm short in time these days, so it might take some more days
> until I could check.

I could give this a try, but I'm not sure what is the expected behavior
(though I guess that the default directory should be set to the
hostname).

Cheers,
Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

[Michael Albinus]

Nicolas Petton <nicolas@petton.fr> writes:

Hi Nico,

>> Will do. I'm short in time these days, so it might take some more days
>> until I could check.
>
> I could give this a try, but I'm not sure what is the expected behavior
> (though I guess that the default directory should be set to the
> hostname).

That would be great!

I haven't started yet, but I guess you must check url-handlers. Look at
this:

(progn
  (require 'url-handlers)
  (file-name-directory "https://gnu.org"))

=> "https://"

In this special case, I would expect "https://gnu.org" as result,
indeed.

Creating a test file test/lisp/url/url-handlers-test.el would be useful.

> Cheers,
> Nico

Best regards, Michael.

bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 180 bytes --]

Michael Albinus <michael.albinus@gmx.de> writes:

> Nicolas Petton <nicolas@petton.fr> writes:

> That would be great!

I'll have some time tonight, I'll work on it.

Cheers,
Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 405 bytes --]

Michael Albinus <michael.albinus@gmx.de> writes:


> That would be great!
>
> I haven't started yet, but I guess you must check url-handlers. Look at
> this:

I started writing unit tests and quickly realized I had to turn on
`url-handler-mode' to actually add url file handlers to
file-name-handler-alist.

I'm completely new to url-handlers.el, and I'm wondering why it is off
by default.

Cheers,
Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

[Robert Pluim]

Nicolas Petton <nicolas@petton.fr> writes:

> I'm completely new to url-handlers.el, and I'm wondering why it is off
> by default.

Probably because it's completely undocumented, so it's entirely
unclear what it actually does.

Robert

bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

[Nicolas Petton]

[-- Attachment #1.1: Type: text/plain, Size: 214 bytes --]

Michael Albinus <michael.albinus@gmx.de> writes:

> That would be great!

Here's a first tentative.  It's probably naive though, as there might be
some corner cases I didn't think of and I'm new to file-handlers.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-Add-URL-handler-for-file-name-directory-Bug-30444.patch --]
[-- Type: text/x-patch, Size: 5113 bytes --]

From ef407140ab8ae6b421156d70028327b0eb69c565 Mon Sep 17 00:00:00 2001
From: Nicolas Petton <nicolas@petton.fr>
Date: Tue, 13 Mar 2018 22:07:08 +0100
Subject: [PATCH] Add URL handler for file-name-directory  (Bug#30444)

* lisp/url/url-handlers.el (url-handler-file-name-directory): New
function which handles special cases for `file-name-directory' and
URLs.
* test/lisp/url/url-handlers-test.el: New file.  Add tests for
`url-handler-file-name-directory'.
---
 lisp/url/url-handlers.el           |  9 +++++
 test/lisp/url/url-handlers-test.el | 68 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 77 insertions(+)
 create mode 100644 test/lisp/url/url-handlers-test.el

diff --git a/lisp/url/url-handlers.el b/lisp/url/url-handlers.el
index 7d0320cb5b..17c76cff75 100644
--- a/lisp/url/url-handlers.el
+++ b/lisp/url/url-handlers.el
@@ -186,6 +186,7 @@ url-file-handler-identity
 (put 'file-name-absolute-p 'url-file-handlers (lambda (&rest ignored) t))
 (put 'expand-file-name 'url-file-handlers 'url-handler-expand-file-name)
 (put 'directory-file-name 'url-file-handlers 'url-handler-directory-file-name)
+(put 'file-name-directory 'url-file-handlers 'url-handler-file-name-directory)
 (put 'unhandled-file-name-directory 'url-file-handlers 'url-handler-unhandled-file-name-directory)
 (put 'file-remote-p 'url-file-handlers 'url-handler-file-remote-p)
 ;; (put 'file-name-as-directory 'url-file-handlers 'url-handler-file-name-as-directory)
@@ -231,6 +232,14 @@ url-handler-unhandled-file-name-directory
       ;; a local process.
       nil)))
 
+(defun url-handler-file-name-directory (dir)
+  (let ((url (url-generic-parse-url dir)))
+    ;; Do not attempt to handle `file' URLs which are local.
+    (if (and (not (equal (url-type url) "file"))
+	     (string-empty-p (url-filename url)))
+	(url-handler-file-name-directory (concat dir "/"))
+      (url-run-real-handler 'file-name-directory (list dir)))))
+
 (defun url-handler-file-remote-p (filename &optional identification _connected)
   (let ((url (url-generic-parse-url filename)))
     (if (and (url-type url) (not (equal (url-type url) "file")))
diff --git a/test/lisp/url/url-handlers-test.el b/test/lisp/url/url-handlers-test.el
new file mode 100644
index 0000000000..01ef18e0c9
--- /dev/null
+++ b/test/lisp/url/url-handlers-test.el
@@ -0,0 +1,68 @@
+;;; url-handlers-test.el --- Test suite for url-handlers.el  -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2018 Free Software Foundation, Inc.
+
+;; Author: Nicolas Petton <nicolas@petton.fr>
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;;; Code:
+
+(require 'ert)
+(require 'url-handlers)
+
+(defmacro with-url-handler-mode (&rest body)
+  "Evaluate BODY with `url-handler-mode' turned on."
+  (declare (indent 0) (debug t))
+  (let ((url-handler-mode-active (make-symbol "url-handler-mode-active")))
+    `(let ((,url-handler-mode-active url-handler-mode))
+       (unwind-protect
+           (progn
+             (unless ,url-handler-mode-active
+               (url-handler-mode))
+             ,@body)
+         (unless ,url-handler-mode-active
+           (url-handler-mode -1))))))
+
+(ert-deftest url-handlers-file-name-directory/preserve-url-types ()
+  (with-url-handler-mode
+    (should (equal (file-name-directory "https://gnu.org/index.html")
+                   "https://gnu.org/"))
+    (should (equal (file-name-directory "http://gnu.org/index.html")
+                   "http://gnu.org/"))
+    (should (equal (file-name-directory "ftp://gnu.org/index.html")
+                   "ftp://gnu.org/"))))
+
+(ert-deftest url-handlers-file-name-directory/sub-directories ()
+  (with-url-handler-mode
+    (should (equal (file-name-directory "https://foo/bar/baz/index.html")
+                   "https://foo/bar/baz/"))))
+
+(ert-deftest url-handlers-file-name-directory/file-urls ()
+  (with-url-handler-mode
+    (should (equal (file-name-directory "file:///foo/bar/baz.txt")
+                   "file:///foo/bar/"))
+    (should (equal (file-name-directory "file:///")
+                   "file:///"))))
+
+;; Regression test for bug#30444
+(ert-deftest url-handlers-file-name-directory/no-filename ()
+  (with-url-handler-mode
+    (should (equal (file-name-directory "https://foo.org")
+                   "https://foo.org/"))))
+
+(provide 'url-handlers-test)
+;;; url-handlers-test.el ends here
-- 
2.14.3


[-- Attachment #1.3: Type: text/plain, Size: 14 bytes --]


Cheers,
Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

[Michael Albinus]

Nicolas Petton <nicolas@petton.fr> writes:

Hi Nico,

> Here's a first tentative.  It's probably naive though, as there might be
> some corner cases I didn't think of and I'm new to file-handlers.

I haven't tested thoroughly, but your tests look OK.

> +    (should (equal (file-name-directory "https://foo.org")
> +                   "https://foo.org/"))))

This is OK, but I would add also

    (should (equal (file-name-directory "https://foo.org/")
                   "https://foo.org/"))))

Maybe you could add also negative tests, for example for
(file-name-directory "not-a-uri://foo.org")

Best regards, Michael.

bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

[Nicolas Petton]

[-- Attachment #1.1: Type: text/plain, Size: 537 bytes --]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Michael,


> I haven't tested thoroughly, but your tests look OK.
>
>> +    (should (equal (file-name-directory "https://foo.org")
>> +                   "https://foo.org/"))))
>
> This is OK, but I would add also
>
>     (should (equal (file-name-directory "https://foo.org/")
>                    "https://foo.org/"))))
>
> Maybe you could add also negative tests, for example for
> (file-name-directory "not-a-uri://foo.org")

Thanks for your feedback.

Here's a revised version.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-Add-URL-handler-for-file-name-directory-Bug-30444.patch --]
[-- Type: text/x-patch, Size: 5679 bytes --]

From a8bc0f60cfc61b390e089166fcd69f267b77c39e Mon Sep 17 00:00:00 2001
From: Nicolas Petton <nicolas@petton.fr>
Date: Tue, 13 Mar 2018 22:07:08 +0100
Subject: [PATCH] Add URL handler for file-name-directory  (Bug#30444)

* lisp/url/url-handlers.el (url-handler-file-name-directory): New
function which handles special cases for `file-name-directory' and
URLs.
* test/lisp/url/url-handlers-test.el: New file.  Add tests for
`url-handler-file-name-directory'.
---
 lisp/url/url-handlers.el           |  9 +++++
 test/lisp/url/url-handlers-test.el | 80 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 89 insertions(+)
 create mode 100644 test/lisp/url/url-handlers-test.el

diff --git a/lisp/url/url-handlers.el b/lisp/url/url-handlers.el
index 7d0320cb5b..17c76cff75 100644
--- a/lisp/url/url-handlers.el
+++ b/lisp/url/url-handlers.el
@@ -186,6 +186,7 @@ url-file-handler-identity
 (put 'file-name-absolute-p 'url-file-handlers (lambda (&rest ignored) t))
 (put 'expand-file-name 'url-file-handlers 'url-handler-expand-file-name)
 (put 'directory-file-name 'url-file-handlers 'url-handler-directory-file-name)
+(put 'file-name-directory 'url-file-handlers 'url-handler-file-name-directory)
 (put 'unhandled-file-name-directory 'url-file-handlers 'url-handler-unhandled-file-name-directory)
 (put 'file-remote-p 'url-file-handlers 'url-handler-file-remote-p)
 ;; (put 'file-name-as-directory 'url-file-handlers 'url-handler-file-name-as-directory)
@@ -231,6 +232,14 @@ url-handler-unhandled-file-name-directory
       ;; a local process.
       nil)))
 
+(defun url-handler-file-name-directory (dir)
+  (let ((url (url-generic-parse-url dir)))
+    ;; Do not attempt to handle `file' URLs which are local.
+    (if (and (not (equal (url-type url) "file"))
+	     (string-empty-p (url-filename url)))
+	(url-handler-file-name-directory (concat dir "/"))
+      (url-run-real-handler 'file-name-directory (list dir)))))
+
 (defun url-handler-file-remote-p (filename &optional identification _connected)
   (let ((url (url-generic-parse-url filename)))
     (if (and (url-type url) (not (equal (url-type url) "file")))
diff --git a/test/lisp/url/url-handlers-test.el b/test/lisp/url/url-handlers-test.el
new file mode 100644
index 0000000000..814c68bac3
--- /dev/null
+++ b/test/lisp/url/url-handlers-test.el
@@ -0,0 +1,80 @@
+;;; url-handlers-test.el --- Test suite for url-handlers.el  -*- lexical-binding: t; -*-
+
+;; Copyright (C) 2018 Free Software Foundation, Inc.
+
+;; Author: Nicolas Petton <nicolas@petton.fr>
+
+;; This program is free software; you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; This program is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+;;; Commentary:
+
+;;; Code:
+
+(require 'ert)
+(require 'url-handlers)
+
+(defmacro with-url-handler-mode (&rest body)
+  "Evaluate BODY with `url-handler-mode' turned on."
+  (declare (indent 0) (debug t))
+  (let ((url-handler-mode-active (make-symbol "url-handler-mode-active")))
+    `(let ((,url-handler-mode-active url-handler-mode))
+       (unwind-protect
+           (progn
+             (unless ,url-handler-mode-active
+               (url-handler-mode))
+             ,@body)
+         (unless ,url-handler-mode-active
+           (url-handler-mode -1))))))
+
+(ert-deftest url-handlers-file-name-directory/preserve-url-types ()
+  (with-url-handler-mode
+    (should (equal (file-name-directory "https://gnu.org/index.html")
+                   "https://gnu.org/"))
+    (should (equal (file-name-directory "http://gnu.org/index.html")
+                   "http://gnu.org/"))
+    (should (equal (file-name-directory "ftp://gnu.org/index.html")
+                   "ftp://gnu.org/"))))
+
+(ert-deftest url-handlers-file-name-directory/should-not-handle-non-url-file-names ()
+  (with-url-handler-mode
+    (should-not (equal (file-name-directory "not-uri://gnu.org")
+                   "not-uri://gnu.org/"))))
+
+(ert-deftest url-handlers-file-name-directory/sub-directories ()
+  (with-url-handler-mode
+    (should (equal (file-name-directory "https://foo/bar/baz/index.html")
+                   "https://foo/bar/baz/"))))
+
+(ert-deftest url-handlers-file-name-directory/file-urls ()
+  (with-url-handler-mode
+    (should (equal (file-name-directory "file:///foo/bar/baz.txt")
+                   "file:///foo/bar/"))
+    (should (equal (file-name-directory "file:///")
+                   "file:///"))))
+
+;; Regression test for bug#30444
+(ert-deftest url-handlers-file-name-directory/no-filename ()
+  (with-url-handler-mode
+    (should (equal (file-name-directory "https://foo.org")
+                   "https://foo.org/"))
+    (should (equal (file-name-directory "https://foo.org/")
+                   "https://foo.org/"))))
+
+(ert-deftest url-handlers-file-name-directory/should-not-handle-non-url-file-names ()
+  (with-url-handler-mode
+    (should-not (equal (file-name-directory "not-uri://gnu.org")
+                   "not-uri://gnu.org/"))))
+
+(provide 'url-handlers-test)
+;;; url-handlers-test.el ends here
-- 
2.14.3


[-- Attachment #1.3: Type: text/plain, Size: 14 bytes --]


Cheers,
Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

[Michael Albinus]

Nicolas Petton <nicolas@petton.fr> writes:

> Hi Michael,

Hi Nico,

> Here's a revised version.

Thanks. I've played with this. Both url-handlers-test.el and
tramp-archive-tests.el (which uses url-handler-mode) do pass. From my
pov, you could commit the patch.

Two remarks, 'tho:

- Require subr-x in url-handlers.el, otherwise there is a compilation
  warning.

- In url-handlers-test.el, test
  url-handlers-file-name-directory/should-not-handle-non-url-file-names
  is defined twice.

Best regards, Michael.

bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 537 bytes --]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Michael,

> Thanks. I've played with this. Both url-handlers-test.el and
> tramp-archive-tests.el (which uses url-handler-mode) do pass. From my
> pov, you could commit the patch.
>
> Two remarks, 'tho:
>
> - Require subr-x in url-handlers.el, otherwise there is a compilation
>   warning.
>
> - In url-handlers-test.el, test
>   url-handlers-file-name-directory/should-not-handle-non-url-file-names
>   is defined twice.

Thanks, I'll fix this and commit it to master.

Cheers,
Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#30444: 26.0.90; browse-url-emacs sets default-directory to "https://"

[Nicolas Petton]

[-- Attachment #1: Type: text/plain, Size: 257 bytes --]

Michael Albinus <michael.albinus@gmx.de> writes:

> Thanks. I've played with this. Both url-handlers-test.el and
> tramp-archive-tests.el (which uses url-handler-mode) do pass. From my
> pov, you could commit the patch.

Done, I'm closing this issue.

Nico

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]