bug#6339: url-filename => "/c:/some/file.txt"

bug#6339: url-filename => "/c:/some/file.txt"

[Lennart Borgman]

(setq x (url-generic-parse-url "file:///c:/some/file.txt"))
(url-filename x) => "/c:/some/file.txt"

Should not that be "c:/some/file.txt"?

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

> Should not that be "c:/some/file.txt"?

Yes.

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Thu, Jun 3, 2010 at 04:38, Lennart Borgman <lennart.borgman@gmail.com> wrote:

> (setq x (url-generic-parse-url "file:///c:/some/file.txt"))
> (url-filename x) => "/c:/some/file.txt"
>
> Should not that be "c:/some/file.txt"?

This is caused by url-generic-parse-url thinking that in
file://HOST/PATH, the path is /PATH, when in fact, according to the
relevant RFCs, "/" is a separator and part of the URI syntax, and PATH
is an absolute path (to be interpreted as starting from / on Unix, but
that's irrelevant). This was extensively discussed in bug#5345.

The subyacent cause of the bug, IMHO, is that many software for POSIX
environments that parses URIs just takes the /PATH part as a whole
because it is easier than removing the slash and then re-adding it.
That breaks on Windows file: URIs, as this bug, and #5345, show.

So the right fix IMO is as simple as the attached patch. The problem
is that it will likely cause trouble for other users of
url-generic-parse-url which wrongly assume that the path part will be
rooted at /.

Another, hacky & ugly, fix is removing the slash only if the PATH part
matches /[A-Z]:.

I'd rather fix it the good way and watch the fireworks.


    Juanma



=== modified file 'lisp/url/url-parse.el'
--- lisp/url/url-parse.el	2010-06-22 16:48:53 +0000
+++ lisp/url/url-parse.el	2010-07-26 11:46:11 +0000
@@ -148,4 +148,5 @@
           ;; 3.3. Path
+	  (when (looking-at "/") (forward-char 1))
           ;; Gross hack to preserve ';' in data URLs
           (setq save-pos (point))

bug#6339: url-filename => "/c:/some/file.txt"

[Lennart Borgman]

On Mon, Jul 26, 2010 at 1:50 PM, Juanma Barranquero <lekktu@gmail.com> wrote:
>
> The subyacent cause of the bug, IMHO, is that many software for POSIX
> environments that parses URIs just takes the /PATH part as a whole
> because it is easier than removing the slash and then re-adding it.
> That breaks on Windows file: URIs, as this bug, and #5345, show.
>
> So the right fix IMO is as simple as the attached patch. The problem
> is that it will likely cause trouble for other users of
> url-generic-parse-url which wrongly assume that the path part will be
> rooted at /.
>
> Another, hacky & ugly, fix is removing the slash only if the PATH part
> matches /[A-Z]:.
>
> I'd rather fix it the good way and watch the fireworks.


I agree.

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

Anyone opposed to trying this patch on the trunk and seeing what breaks?

  J


> === modified file 'lisp/url/url-parse.el'
> --- lisp/url/url-parse.el       2010-06-22 16:48:53 +0000
> +++ lisp/url/url-parse.el       2010-07-26 11:46:11 +0000
> @@ -148,4 +148,5 @@
>           ;; 3.3. Path
> +         (when (looking-at "/") (forward-char 1))
>           ;; Gross hack to preserve ';' in data URLs
>           (setq save-pos (point))
>

bug#6339: url-filename => "/c:/some/file.txt"

[Michael Albinus]

Juanma Barranquero <lekktu@gmail.com> writes:

> Anyone opposed to trying this patch on the trunk and seeing what breaks?

I'm in favour of this patch. The current behaviour could cause trouble
to Tramp.

>   J

Best regards, Michael.

bug#6339: url-filename => "/c:/some/file.txt"

[Lars Magne Ingebrigtsen]

Juanma Barranquero <lekktu@gmail.com> writes:

> On Thu, Jun 3, 2010 at 04:38, Lennart Borgman <lennart.borgman@gmail.com> wrote:
>
>> (setq x (url-generic-parse-url "file:///c:/some/file.txt"))
>> (url-filename x) => "/c:/some/file.txt"
>>
>> Should not that be "c:/some/file.txt"?

[...]

> I'd rather fix it the good way and watch the fireworks.
>
>     Juanma
>
> === modified file 'lisp/url/url-parse.el'
> --- lisp/url/url-parse.el	2010-06-22 16:48:53 +0000
> +++ lisp/url/url-parse.el	2010-07-26 11:46:11 +0000
> @@ -148,4 +148,5 @@
>            ;; 3.3. Path
> +	  (when (looking-at "/") (forward-char 1))
>            ;; Gross hack to preserve ';' in data URLs
>            (setq save-pos (point))

If I say:

(setq x (url-generic-parse-url "file:///home/larsi/foo.txt"))

Then I get

(url-filename x)
=> "/home/larsi/foo.txt"

as expected.  Would your patch break that?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Wed, Sep 21, 2011 at 22:28, Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:

> If I say:
>
> (setq x (url-generic-parse-url "file:///home/larsi/foo.txt"))
>
> Then I get
>
> (url-filename x)
> => "/home/larsi/foo.txt"
>
> as expected.  Would your patch break that?

My patch would return "home/larsi/foo.txt", yes.

But expecting the slash is a "bug" in you expectations, because the
*filename* (the path, according to the RFCs, see below) of

  file:///home/larsi/foo.txt

is not "/home/larsi/foo.txt". The slash is a separator, part of the
URI syntax, and "home/larsi/foo.txt" is an absolute path. It's easier
to see it with the full syntax, in things like

  file://localhost/home/larsi/foo.txt

for example. The fact that url-filename returns the slash is a bug;
just one that nobody has fixed or complained about because it makes
easier to process the path than having to do

  (concat "/" (url-filename "file:///mypath"))

At least, until you have a Windows URI. And no, this is *not* a
Windows problem or a Windows bug, it's a bug that makes life easier
for POSIX at the cost of making it uglier for Windows.

    Juanma


From RFC 1738 "Uniform Resource Locators (URL)"

3.10 FILES

   The file URL scheme is used to designate files accessible on a
   particular host computer. This scheme, unlike most other URL schemes,
   does not designate a resource that is universally accessible over the
   Internet.

   A file URL takes the form:

       file://<host>/<path>

   where <host> is the fully qualified domain name of the system on
   which the <path> is accessible, and <path> is a hierarchical
   directory path of the form <directory>/<directory>/.../<name>.

   For example, a VMS file

     DISK$USER:[MY.NOTES]NOTE123456.TXT

   might become

     <URL:file://vms.host.edu/disk$user/my/notes/note12345.txt>

   As a special case, <host> can be the string "localhost" or the empty
   string; this is interpreted as `the machine from which the URL is
   being interpreted'.

   The file URL scheme is unusual in that it does not specify an
   Internet protocol or access method for such files; as such, its
   utility in network protocols between hosts is limited.

bug#6339: url-filename => "/c:/some/file.txt"

[Lennart Borgman]

On Thu, Sep 22, 2011 at 00:21, Juanma Barranquero <lekktu@gmail.com> wrote:
> On Wed, Sep 21, 2011 at 22:28, Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:
>
>> If I say:
>>
>> (setq x (url-generic-parse-url "file:///home/larsi/foo.txt"))
>>
>> Then I get
>>
>> (url-filename x)
>> => "/home/larsi/foo.txt"
>>
>> as expected.  Would your patch break that?
>
> My patch would return "home/larsi/foo.txt", yes.
>
> But expecting the slash is a "bug" in you expectations, because the
> *filename* (the path, according to the RFCs, see below) of
>
>  file:///home/larsi/foo.txt
>
> is not "/home/larsi/foo.txt". The slash is a separator, part of the
> URI syntax, and "home/larsi/foo.txt" is an absolute path. It's easier
> to see it with the full syntax, in things like
>
>  file://localhost/home/larsi/foo.txt
>
> for example. The fact that url-filename returns the slash is a bug;
> just one that nobody has fixed or complained about because it makes
> easier to process the path than having to do
>
>  (concat "/" (url-filename "file:///mypath"))

Are we not dealing with two different file name syntaxes here that
just happens to look very similar? I.e. the URL file name syntax and
the unix style file name syntax.

url-filename is now returning the file name in unix style. This is, as
you said, practical, but a bit surprising. I suspect there are some
small mixing of those syntaxes elsewhere too. My suggestion would be
to keep them and document them.


> At least, until you have a Windows URI. And no, this is *not* a
> Windows problem or a Windows bug, it's a bug that makes life easier
> for POSIX at the cost of making it uglier for Windows.

Indeed.

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Thu, Sep 22, 2011 at 00:31, Lennart Borgman
<lennart.borgman@gmail.com> wrote:

> Are we not dealing with two different file name syntaxes here that
> just happens to look very similar? I.e. the URL file name syntax and
> the unix style file name syntax.

I would expect that an accessor for the filename slot of a struct
called URL would interpret the struct as a URL, and the "filename" of
the URL does not start with a slash. Why would I expect that
url-filename (url-generic-parse-url, in fact) returns anything "unix
style" specific?

> url-filename is now returning the file name in unix style.

No.

  (url-generic-parse-url "file:///C:/my/file.xt") => [cl-struct-url
"file" nil nil "" 21 "/C:/my/file.txt" nil nil t nil]

That's not a "file name in unix style", that's a bug, clear and
simple. There's no reason why url-generic-parse-url should think that
a URL is in some kind of "unix style". What if it is a VMS filename?

> This is, as
> you said, practical, but a bit surprising. I suspect there are some
> small mixing of those syntaxes elsewhere too. My suggestion would be
> to keep them and document them.

"Keep them and document them" still means that it does not work for
Windows or VMS or anything not POSIX-style. And fixing it just for
these systems means that we keep a bug for POSIX sake and make other
systems jump around hoops. Ugly, to say the least.

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Lennart Borgman]

On Thu, Sep 22, 2011 at 00:45, Juanma Barranquero <lekktu@gmail.com> wrote:
> On Thu, Sep 22, 2011 at 00:31, Lennart Borgman
> <lennart.borgman@gmail.com> wrote:
>
>> Are we not dealing with two different file name syntaxes here that
>> just happens to look very similar? I.e. the URL file name syntax and
>> the unix style file name syntax.
>
> I would expect that an accessor for the filename slot of a struct
> called URL would interpret the struct as a URL, and the "filename" of
> the URL does not start with a slash. Why would I expect that
> url-filename (url-generic-parse-url, in fact) returns anything "unix
> style" specific?
>
>> url-filename is now returning the file name in unix style.
>
> No.
>
>  (url-generic-parse-url "file:///C:/my/file.xt") => [cl-struct-url
> "file" nil nil "" 21 "/C:/my/file.txt" nil nil t nil]
>
> That's not a "file name in unix style", that's a bug, clear and
> simple.

Ah, yes, of course. I thought everyone reading here was clear about that.

> There's no reason why url-generic-parse-url should think that
> a URL is in some kind of "unix style". What if it is a VMS filename?

You are right. I did not mean to imply that the current behaviour is ok.

>> This is, as
>> you said, practical, but a bit surprising. I suspect there are some
>> small mixing of those syntaxes elsewhere too. My suggestion would be
>> to keep them and document them.
>
> "Keep them and document them" still means that it does not work for
> Windows or VMS or anything not POSIX-style. And fixing it just for
> these systems means that we keep a bug for POSIX sake and make other
> systems jump around hoops. Ugly, to say the least.

Of course the behaviour must be fixed too. I meant to write that it is
ok that url-filename returns a system-dependent absolute file name
(for local files).

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Thu, Sep 22, 2011 at 00:51, Lennart Borgman
<lennart.borgman@gmail.com> wrote:

> Of course the behaviour must be fixed too. I meant to write that it is
> ok that url-filename returns a system-dependent absolute file name
> (for local files).

When you do (url-filename (url-generic-parse-url "file:///SOMEPATH"))
you are not checking SOMEPATH against the local filesystem (the path
does not need to exist, for example). So, when you say
"system-dependant" and "local files", what you're really saying is
that SOMEPATH should be analyzed to see whether it is POSIX-style, or
Windows-style, or whatever, to "fix it" or "leave it alone" according
to the detected style.

And yes, we have already discussed that. It is certainly possible. It
is also ugly and ad hoc, and ugly too.

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Lennart Borgman]

On Thu, Sep 22, 2011 at 00:59, Juanma Barranquero <lekktu@gmail.com> wrote:
> On Thu, Sep 22, 2011 at 00:51, Lennart Borgman
> <lennart.borgman@gmail.com> wrote:
>
>> Of course the behaviour must be fixed too. I meant to write that it is
>> ok that url-filename returns a system-dependent absolute file name
>> (for local files).
>
> When you do (url-filename (url-generic-parse-url "file:///SOMEPATH"))
> you are not checking SOMEPATH against the local filesystem (the path
> does not need to exist, for example). So, when you say
> "system-dependant" and "local files", what you're really saying is
> that SOMEPATH should be analyzed to see whether it is POSIX-style, or
> Windows-style, or whatever, to "fix it" or "leave it alone" according
> to the detected style.

The analysis of "file:///SOMEPATH" should probably be system
independent (since it is an URL that is analysed).

But the return value should be system dependant since it is supposed
to be a local file system absolute path.

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Thu, Sep 22, 2011 at 01:15, Lennart Borgman
<lennart.borgman@gmail.com> wrote:

> But the return value should be system dependant since it is supposed
> to be a local file system absolute path.

Do you mean that Emacs running in a POSIX system cannot process file
URLs for Windows local files, for example?

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Lennart Borgman]

On Thu, Sep 22, 2011 at 01:18, Juanma Barranquero <lekktu@gmail.com> wrote:
> On Thu, Sep 22, 2011 at 01:15, Lennart Borgman
> <lennart.borgman@gmail.com> wrote:
>
>> But the return value should be system dependant since it is supposed
>> to be a local file system absolute path.
>
> Do you mean that Emacs running in a POSIX system cannot process file
> URLs for Windows local files, for example?

I am not sure. Are URL style file name context dependent?

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Thu, Sep 22, 2011 at 01:32, Lennart Borgman
<lennart.borgman@gmail.com> wrote:

> I am not sure. Are URL style file name context dependent?

I don't understand your question, but I will be quite surprised if the
answer to my previous question is "yes".

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Lennart Borgman]

On Thu, Sep 22, 2011 at 01:37, Juanma Barranquero <lekktu@gmail.com> wrote:
> On Thu, Sep 22, 2011 at 01:32, Lennart Borgman
> <lennart.borgman@gmail.com> wrote:
>
>> I am not sure. Are URL style file name context dependent?
>
> I don't understand your question, but I will be quite surprised if the
> answer to my previous question is "yes".

Then your answer to my question is that you will be surprised if URL
style file names are context dependent. ;-)

But unfortunately that might not be the right answer... Take this URL file name:

   file:///c:/some/file.txt

On windows that would be "c:/some/file.txt". What would it mean on a
unix system? I guess it would mean "/c:/some/file.txt", but I am not
sure. Perhaps that is an invalid file name on unix?

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Thu, Sep 22, 2011 at 01:46, Lennart Borgman
<lennart.borgman@gmail.com> wrote:

> But unfortunately that might not be the right answer... Take this URL file name:
>
>   file:///c:/some/file.txt
>
> On windows that would be "c:/some/file.txt". What would it mean on a
> unix system? I guess it would mean "/c:/some/file.txt", but I am not
> sure. Perhaps that is an invalid file name on unix?

Quite likely, but IMO you're missing the point.

file:///c:/some/file.txt is a URL, i.e., it's a pointer to some
specific file in some specific place (or files with identical path and
name in different hosts, because the URL is missing the HOST part,
defaulting then to localhost). In fact, it is the URL that points to a
file, in some filesystem, with absolute path "c:/some/file.txt".
Whether that file exists, and whether that path makes sense when you
apply it locally to a POSIX system, i.e., whether you can access a
file with that URL, is irrelevant to the fact that "c:/some/file.txt"
is the path of the URL. You're muddling the waters when you insist in
"context" and "local system" and the like, because that affects to the
*use* of the URL, not its syntax. Splitting a URL into pieces
according to the RFC does not depend on where you do it or how do you
intend to use it.

We wouldn't be having this discussion in an alternative world where
the POSIX path separator was still "/", but the good people who wrote
the URI/URL specs had chosen "#" as the field separator in URLs:
file:##HOST#PATH

  file:###c:/some/file.txt  vs.  file:###some/file.txt

No one would try to split that as "#some/file.txt".

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Lennart Borgman]

On Thu, Sep 22, 2011 at 02:32, Juanma Barranquero <lekktu@gmail.com> wrote:
> On Thu, Sep 22, 2011 at 01:46, Lennart Borgman
> <lennart.borgman@gmail.com> wrote:
>
>> But unfortunately that might not be the right answer... Take this URL file name:
>>
>>   file:///c:/some/file.txt
>>
>> On windows that would be "c:/some/file.txt". What would it mean on a
>> unix system? I guess it would mean "/c:/some/file.txt", but I am not
>> sure. Perhaps that is an invalid file name on unix?
>
> Quite likely, but IMO you're missing the point.
>
> file:///c:/some/file.txt is a URL, i.e., it's a pointer to some
> specific file in some specific place (or files with identical path and
> name in different hosts, because the URL is missing the HOST part,
> defaulting then to localhost). In fact, it is the URL that points to a
> file, in some filesystem, with absolute path "c:/some/file.txt".

Yes, but please remember if you make that interpretation (which I hope
is the correct one) then "c:/some/file.txt" is an absolute local file
name in URL file name syntax.

So if we hold on to that interpretation then maybe we should say (and
document ;-) that url-filename returns such a file name and that it
needs to be translated to the actually used systems file syntax to be
used there.

This would be an incompatible change from the current (faulty)
behaviour, but I think it would be the best.

> Whether that file exists, and whether that path makes sense when you
> apply it locally to a POSIX system, i.e., whether you can access a
> file with that URL, is irrelevant to the fact that "c:/some/file.txt"
> is the path of the URL. You're muddling the waters when you insist in
> "context" and "local system" and the like, because that affects to the
> *use* of the URL, not its syntax. Splitting a URL into pieces
> according to the RFC does not depend on where you do it or how do you
> intend to use it.

Ah, yes. I think we are saying the same thing about the problem here.

> We wouldn't be having this discussion in an alternative world where
> the POSIX path separator was still "/", but the good people who wrote
> the URI/URL specs had chosen "#" as the field separator in URLs:
> file:##HOST#PATH
>
>  file:###c:/some/file.txt  vs.  file:###some/file.txt
>
> No one would try to split that as "#some/file.txt".

That is a good example.

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Thu, Sep 22, 2011 at 02:40, Lennart Borgman
<lennart.borgman@gmail.com> wrote:

> Yes, but please remember if you make that interpretation (which I hope
> is the correct one) then "c:/some/file.txt" is an absolute local file
> name in URL file name syntax.
>
> So if we hold on to that interpretation then maybe we should say (and
> document ;-) that url-filename returns such a file name and that it
> needs to be translated to the actually used systems file syntax to be
> used there.
>
> This would be an incompatible change from the current (faulty)
> behaviour, but I think it would be the best.

Isn't that what I've been saying all along?

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Lennart Borgman]

On Thu, Sep 22, 2011 at 02:59, Juanma Barranquero <lekktu@gmail.com> wrote:
> On Thu, Sep 22, 2011 at 02:40, Lennart Borgman
> <lennart.borgman@gmail.com> wrote:
>
>> Yes, but please remember if you make that interpretation (which I hope
>> is the correct one) then "c:/some/file.txt" is an absolute local file
>> name in URL file name syntax.
>>
>> So if we hold on to that interpretation then maybe we should say (and
>> document ;-) that url-filename returns such a file name and that it
>> needs to be translated to the actually used systems file syntax to be
>> used there.
>>
>> This would be an incompatible change from the current (faulty)
>> behaviour, but I think it would be the best.
>
> Isn't that what I've been saying all along?

If you say so ;-)

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Thu, Sep 22, 2011 at 03:07, Lennart Borgman
<lennart.borgman@gmail.com> wrote:

> If you say so ;-)

Well,

> Yes, but please remember if you make that interpretation (which I hope
> is the correct one) then "c:/some/file.txt" is an absolute local file
> name in URL file name syntax.

This is what the RFC says. I even quoted it.

> So if we hold on to that interpretation then maybe we should say (and
> document ;-) that url-filename returns such a file name

This I didn't say (but any change or unexpected behavior should be documented).

> and that it
> needs to be translated to the actually used systems file syntax to be
> used there.

I explicitly talked about (concat "/" (url-filename ...))

> This would be an incompatible change from the current (faulty)
> behaviour, but I think it would be the best.

This is what I've been saying in pretty much every message from this thread.

So yes, "I say so".

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Lennart Borgman]

On Thu, Sep 22, 2011 at 03:14, Juanma Barranquero <lekktu@gmail.com> wrote:
>
> This is what I've been saying in pretty much every message from this thread.
>
> So yes, "I say so".

And I said in my second message that I agree... ;-)

bug#6339: url-filename => "/c:/some/file.txt"

[Jason Rumney]

Juanma Barranquero <lekktu@gmail.com> writes:

> file:///c:/some/file.txt is a URL, i.e., it's a pointer to some
> specific file in some specific place (or files with identical path and
> name in different hosts, because the URL is missing the HOST part,
> defaulting then to localhost). In fact, it is the URL that points to a
> file, in some filesystem, with absolute path "c:/some/file.txt".

No. That is not an absolute path on most platforms. The absolute path on
POSIX platforms would be /c:/some/file.txt, which is consistant with
other URL schemes where the initial / forms part of the PATH portion of
the URL.

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Sat, Sep 24, 2011 at 05:35, Jason Rumney <jasonr@gnu.org> wrote:

> No. That is not an absolute path on most platforms.

The spec says that the PATH part is an absolute path. You can argue
that, in POSIX, "some/file.txt" is an absolute path, because all PATH
parts of file: URIs must be understood as such (and so, have an
implicit / at the front), or you can argue that these paths should be
propertly rooted (and so, the correct way for a POSIX file: reference
is file:////some.file.txt). But no amount of defining or redefining
"absolute" is going to change the fact that the third slash in
file:/// is a separator, and no part of the PATH, according to the
spec.

> The absolute path on
> POSIX platforms would be /c:/some/file.txt, which is consistant with
> other URL schemes where the initial / forms part of the PATH portion of
> the URL.

I don't know what other URLs schemes say, but certainly RFC 1738 says:

3.3. HTTP

   The HTTP URL scheme is used to designate Internet resources
   accessible using HTTP (HyperText Transfer Protocol).

   The HTTP protocol is specified elsewhere. This specification only
   describes the syntax of HTTP URLs.

   An HTTP URL takes the form:

      http://<host>:<port>/<path>?<searchpart>

   where <host> and <port> are as described in Section 3.1. If :<port>
   is omitted, the port defaults to 80.  No user name or password is
   allowed.  <path> is an HTTP selector, and <searchpart> is a query
   string. The <path> is optional, as is the <searchpart> and its
   preceding "?". If neither <path> nor <searchpart> is present, the "/"
   may also be omitted.

   Within the <path> and <searchpart> components, "/", ";", "?" are
   reserved.  The "/" character may be used within HTTP to designate a
   hierarchical structure.

so it's no different of "file:". The slash is a separator.

If people doesn''t want to change the way Emacs works, and prefer to
add ad hoc fixes for Windows URIs, I can do nothing against it. It's
ugly, but such is life. But let's accept that using that slash is just
a shortcut because it conveniently happens to coincide with the POSIX
slash separator, not because it is really part of the path.

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Sat, Sep 24, 2011 at 10:59, Juanma Barranquero <lekktu@gmail.com> wrote:

> You can argue
> that, in POSIX, "some/file.txt" is an absolute path, because all PATH
> parts of file: URIs must be understood as such (and so, have an
> implicit / at the front),

I mean that "some/file.txt" is an absolute path, in the context of
decoding and splitting an URI, of course. Obviously it is not in
general terms.

That said, I suppose that people who thinks that the third slash is
not a separator, but part of the PATH, should also be arguing in favor
of Windows-style file: URIs being of the form
"file://C:/some/file.txt" (which is clearly incorrect, according to
the spec, and certainly unused in the wild). What does not make sense
is arguing that the value returned by url-generic-parse-url is
correct.

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Lennart Borgman]

On Sat, Sep 24, 2011 at 05:35, Jason Rumney <jasonr@gnu.org> wrote:
> Juanma Barranquero <lekktu@gmail.com> writes:
>
>> file:///c:/some/file.txt is a URL, i.e., it's a pointer to some
>> specific file in some specific place (or files with identical path and
>> name in different hosts, because the URL is missing the HOST part,
>> defaulting then to localhost). In fact, it is the URL that points to a
>> file, in some filesystem, with absolute path "c:/some/file.txt".
>
> No. That is not an absolute path on most platforms.

This is not about absolute path in the platform syntax. It is an
absolute path as understood as part of an URL. That has to be
translated to a platform path.

bug#6339: url-filename => "/c:/some/file.txt"

[Andreas Schwab]

Juanma Barranquero <lekktu@gmail.com> writes:

> I don't know what other URLs schemes say, but certainly RFC 1738 says:
>
> 3.3. HTTP

Note that the interpretation of the local part is scheme dependent, so
you cannot compare file: and http: directly.

RFC 1738 defines fileurl in terms of fpath, which is shared with ftpurl.
In an ftp URL an absolute fpath needs to be written with a leading
escaped slash (%2F).  RFCs 4722 and 5022 consistently use four leading
slashes for an absolute file name.  OTOH there are a few occurrences of
file URLs denoting absolute paths with only three slashes in other RFCs.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#6339: url-filename => "/c:/some/file.txt"

[Richard Stallman]

    > defaulting then to localhost). In fact, it is the URL that points to a
    > file, in some filesystem, with absolute path "c:/some/file.txt".

    No. That is not an absolute path on most platforms.

Please note that the GNU terminology for this is "file name" -- "path"
means a list of directories, only.


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Sat, Sep 24, 2011 at 13:53, Andreas Schwab <schwab@linux-m68k.org> wrote:

> In an ftp URL an absolute fpath needs to be written with a leading
> escaped slash (%2F).  RFCs 4722 and 5022 consistently use four leading
> slashes for an absolute file name.  OTOH there are a few occurrences of
> file URLs denoting absolute paths with only three slashes in other RFCs.

I think we're in agreement: there are several ways to interpret the
PATH part, but in all cases, it does not include the separator slash.

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Andreas Schwab]

Juanma Barranquero <lekktu@gmail.com> writes:

> On Sat, Sep 24, 2011 at 13:53, Andreas Schwab <schwab@linux-m68k.org> wrote:
>
>> In an ftp URL an absolute fpath needs to be written with a leading
>> escaped slash (%2F).  RFCs 4722 and 5022 consistently use four leading
>> slashes for an absolute file name.  OTOH there are a few occurrences of
>> file URLs denoting absolute paths with only three slashes in other RFCs.
>
> I think we're in agreement: there are several ways to interpret the
> PATH part, but in all cases, it does not include the separator slash.

That's not the point.  If you interpret the local part as an absolute
file name you have to prepend a slash after decomposing.  If not, then
file:///etc/hosts wouldn't work as probably intended.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Sun, Sep 25, 2011 at 01:42, Andreas Schwab <schwab@linux-m68k.org> wrote:

> If you interpret the local part as an absolute
> file name you have to prepend a slash after decomposing.  If not, then
> file:///etc/hosts wouldn't work as probably intended.

Of course. But that's not the point. The point is that how do you
interpret the local part is only important *after* decomposing the
URI. It's not url-generic-parse-url's task to do any interpretation.
Its docstring says just this:

  Return an url-struct of the parts of url.
  The CL-style struct contains the following fields:
  TYPE USER PASSWORD HOST PORTSPEC FILENAME TARGET ATTRIBUTES FULLNESS.

It says nothing of interpreting any of it, and specifically FILENAME,
POSIX-style. And if it does interpret it POSIX-style it is wreaking
havoc with Windows file: URIs (that's the whole point of this bug
report).

So, of course I agree with you when you say "If you interpret the
local part as an absolute file name you have to prepend a slash after
decomposing.", or more precisely, with what Lennart said previously:

> This is not about absolute path in the platform syntax. It is an
> absolute path as understood as part of an URL. That has to be
> translated to a platform path.

But this discussion is not about how to use the decomposed URI, is
about whether the decomposition is correct, or buggy (and so should be
fixed). I think the RFCs (the one I quoted and the ones you quoted)
show clearly that the current decomposition is erroneous, though
convenient for POSIX systems.

At this point, I can see only two options:

- Fix the bug, and fix also the places where the return value of
url-generic-parse-url / url-filename is assumed to be a POSIX absolute
path.
- Leave it as is, and implement some ad hoc hack to make it work with
Windows file: URIs.

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Kevin Rodgers]

On 9/21/11 7:14 PM, Juanma Barranquero wrote:
> On Thu, Sep 22, 2011 at 03:07, Lennart Borgman
> <lennart.borgman@gmail.com>  wrote:
>
>> If you say so ;-)
>
> Well,
>
>> Yes, but please remember if you make that interpretation (which I hope
>> is the correct one) then "c:/some/file.txt" is an absolute local file
>> name in URL file name syntax.
>
> This is what the RFC says. I even quoted it.
>
>> So if we hold on to that interpretation then maybe we should say (and
>> document ;-) that url-filename returns such a file name
>
> This I didn't say (but any change or unexpected behavior should be documented).
>
>> and that it
>> needs to be translated to the actually used systems file syntax to be
>> used there.
>
> I explicitly talked about (concat "/" (url-filename ...))

Don't you mean: (expand-file-name (url-filename ...) "/")

>> This would be an incompatible change from the current (faulty)
>> behaviour, but I think it would be the best.
>
> This is what I've been saying in pretty much every message from this thread.
>
> So yes, "I say so".
>
>      Juanma
>
>
>
>


-- 
Kevin Rodgers
Denver, Colorado, USA

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Thu, Oct 6, 2011 at 07:09, Kevin Rodgers <kevin.d.rodgers@gmail.com> wrote:

> Don't you mean: (expand-file-name (url-filename ...) "/")

That's better and more correct, yes.

BTW, I would like to know whether we will finally do anything with
this bug. The fix is clear, is just that apparently some people
dislike it.

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Stefan Monnier]

>> Don't you mean: (expand-file-name (url-filename ...) "/")
> That's better and more correct, yes.
> BTW, I would like to know whether we will finally do anything with
> this bug. The fix is clear, is just that apparently some people
> dislike it.

Could someone (re)send the proposed patch?


        Stefan

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

> Could someone (re)send the proposed patch?

The proposed patch is this, or something similar to this:

=== modified file 'lisp/url/url-parse.el'
--- lisp/url/url-parse.el       2010-06-22 16:48:53 +0000
+++ lisp/url/url-parse.el       2010-07-26 11:46:11 +0000
@@ -148,4 +148,5 @@
          ;; 3.3. Path
+         (when (looking-at "/") (forward-char 1))
          ;; Gross hack to preserve ';' in data URLs
          (setq save-pos (point))

but the gist of it is, make url-generic-parse-url not consider the
slash separator as part of the path. If we agree in that, we can then
decide whether that place is the right one to do it or not.

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Jason Rumney]

Juanma Barranquero <lekktu@gmail.com> writes:

>> Could someone (re)send the proposed patch?
>
> The proposed patch is this, or something similar to this:
>
> === modified file 'lisp/url/url-parse.el'
> --- lisp/url/url-parse.el       2010-06-22 16:48:53 +0000
> +++ lisp/url/url-parse.el       2010-07-26 11:46:11 +0000
> @@ -148,4 +148,5 @@
>           ;; 3.3. Path
> +         (when (looking-at "/") (forward-char 1))
>           ;; Gross hack to preserve ';' in data URLs
>           (setq save-pos (point))
>
> but the gist of it is, make url-generic-parse-url not consider the
> slash separator as part of the path. If we agree in that, we can then
> decide whether that place is the right one to do it or not.

This is a general url-parsing function, changing this to not return
the leading / on the file portion of the url will break every other user
of this function.

This needs to be done for file URLs only, and be system-dependent.

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Thu, Oct 6, 2011 at 16:24, Jason Rumney <jasonr@gnu.org> wrote:

> This is a general url-parsing function, changing this to not return
> the leading / on the file portion of the url will break every other user
> of this function.

Only the ones that incorrectly assume that the slash is part of the
path. The ones that use expand-file-name, as Kevin pointed out, will
continue to work.

> This needs to be done for file URLs only

Perhaps, but for non-file URLs, the slash isn't part of the path
either. That's what the RFCs say.

> and be system-dependent.

That's a hack.

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Jason Rumney]

Juanma Barranquero <lekktu@gmail.com> writes:

> On Thu, Oct 6, 2011 at 16:24, Jason Rumney <jasonr@gnu.org> wrote:
>
>> This is a general url-parsing function, changing this to not return
>> the leading / on the file portion of the url will break every other user
>> of this function.
>
> Only the ones that incorrectly assume that the slash is part of the
> path. The ones that use expand-file-name, as Kevin pointed out, will
> continue to work.

It is incorrect to assume that the slash is NOT part of the file portion
of the URL, when actual usage in http, ftp, gopher and file URLs on
platforms other than Windows and VMS relies on the leading slash being
preserved.

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Thu, Oct 6, 2011 at 16:38, Jason Rumney <jasonr@gnu.org> wrote:

> It is incorrect to assume that the slash is NOT part of the file portion
> of the URL, when actual usage in http, ftp, gopher and file URLs on
> platforms other than Windows and VMS relies on the leading slash being
> preserved.

[Today is my "whole disagreement day", it seems.] It's a fact that the
slash is NOT part of the file portion, whatever shortcuts people have
taken along the years.

That said, someone who really thinks that this is a Windows-specific
problem, even if it is not, can trivially "fix" it with an ugly hack.
Let's hope someone does and we can close this bug report forever.

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Lennart Borgman]

On Thu, Oct 6, 2011 at 16:38, Jason Rumney <jasonr@gnu.org> wrote:
> Juanma Barranquero <lekktu@gmail.com> writes:
>
>> On Thu, Oct 6, 2011 at 16:24, Jason Rumney <jasonr@gnu.org> wrote:
>>
>>> This is a general url-parsing function, changing this to not return
>>> the leading / on the file portion of the url will break every other user
>>> of this function.
>>
>> Only the ones that incorrectly assume that the slash is part of the
>> path. The ones that use expand-file-name, as Kevin pointed out, will
>> continue to work.
>
> It is incorrect to assume that the slash is NOT part of the file portion
> of the URL, when actual usage in http, ftp, gopher and file URLs on
> platforms other than Windows and VMS relies on the leading slash being
> preserved.

Didn't we conclude before that the URL style file part does not
include the leading "/"? Or was there any objection to this?

AFAICS the URL style file part simply does not have the same (as *nix
systems) file syntax. So I agree with Juanma here (which is not too
common ;-).

bug#6339: url-filename => "/c:/some/file.txt"

[Lennart Borgman]

On Thu, Oct 6, 2011 at 16:38, Jason Rumney <jasonr@gnu.org> wrote:
> Juanma Barranquero <lekktu@gmail.com> writes:
>
>> On Thu, Oct 6, 2011 at 16:24, Jason Rumney <jasonr@gnu.org> wrote:
>>
>>> This is a general url-parsing function, changing this to not return
>>> the leading / on the file portion of the url will break every other user
>>> of this function.
>>
>> Only the ones that incorrectly assume that the slash is part of the
>> path. The ones that use expand-file-name, as Kevin pointed out, will
>> continue to work.
>
> It is incorrect to assume that the slash is NOT part of the file portion
> of the URL, when actual usage in http, ftp, gopher and file URLs on
> platforms other than Windows and VMS relies on the leading slash being
> preserved.

Didn't we conclude before that the URL style file part does not
include the leading "/"? Or was there any objection to this?

AFAICS the URL style file part simply does not have the same (as *nix
systems) file syntax. So I agree with Juanma here (which is not too
common ;-).

bug#6339: url-filename => "/c:/some/file.txt"

[Stefan Monnier]

>> Could someone (re)send the proposed patch?
> The proposed patch is this, or something similar to this:

> === modified file 'lisp/url/url-parse.el'
> --- lisp/url/url-parse.el       2010-06-22 16:48:53 +0000
> +++ lisp/url/url-parse.el       2010-07-26 11:46:11 +0000
> @@ -148,4 +148,5 @@
>           ;; 3.3. Path
> +         (when (looking-at "/") (forward-char 1))
>           ;; Gross hack to preserve ';' in data URLs
>           (setq save-pos (point))

> but the gist of it is, make url-generic-parse-url not consider the
> slash separator as part of the path. If we agree in that, we can then
> decide whether that place is the right one to do it or not.

I understand the gist of it.  I'm interested in the actual resulting
code changes: the above sample patch fails to include the corresponding
(expand-file-name (url-filename ...) "/"), for example.


        Stefan

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Thu, Oct 6, 2011 at 17:31, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> I understand the gist of it.

I'm sure of it.

> I'm interested in the actual resulting
> code changes: the above sample patch fails to include the corresponding
> (expand-file-name (url-filename ...) "/"), for example.

There are 41 uses of (url-filename ...) in the Emacs sources, all of
them in url\url*.el except two in net/xesam.el.

I'm unwilling to go looking for things to fix and prepare such a patch
unless there's some consensus that removing the slash is the Right
Thing to Do.

If the decision is that no, it isn't, then I suppose things like these:

---- C:\Devel\emacs\repo\trunk\lisp\url\url-handlers.el
        (or (file-name-directory (url-filename url)) "/")

---- C:\Devel\emacs\repo\trunk\lisp\url\url-parse.el
          (or (url-filename urlobj) "/")

---- C:\Devel\emacs\repo\trunk\lisp\url\url-expand.el
    (if (string-match "^/" (url-filename urlobj))

will have to be checked to see whether they are right for Windows.

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Stefan Monnier]

>> I'm interested in the actual resulting
>> code changes: the above sample patch fails to include the corresponding
>> (expand-file-name (url-filename ...) "/"), for example.
> There are 41 uses of (url-filename ...) in the Emacs sources, all of
> them in url\url*.el except two in net/xesam.el.

But the patch you sent changes url-generic-parse-url so it presumably
can affect many more places that just those that call url-filename.

> I'm unwilling to go looking for things to fix and prepare such a patch
> unless there's some consensus that removing the slash is the Right
> Thing to Do.

I think it's the right thing to do.  But maybe the best patch will need
to rename the `filename' slot and to define url-filename to do the
(expand-file-name (url-somethingelse URL) "/") or something like that.


        Stefan

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Thu, Oct 6, 2011 at 18:49, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

> But the patch you sent changes url-generic-parse-url so it presumably
> can affect many more places that just those that call url-filename.

url-generic-parse-url returns a url struct, and url-filename is the
accessor for the slot. If there's code that uses the slot without
going through the accessor, well... I think that's a bug (or, at the
very least, dangerous and ugly) and sooner or later we'll find about
it.

> I think it's the right thing to do.  But maybe the best patch will need
> to rename the `filename' slot and to define url-filename to do the
> (expand-file-name (url-somethingelse URL) "/") or something like that.

I don't think you can do that, because, as explained before in the
thread, splitting the URL and using its parts are independent actions.
It is entirely possible to split a file URL and then use it in
different systems, isn't it?

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Stefan Monnier]

>> But the patch you sent changes url-generic-parse-url so it presumably
>> can affect many more places that just those that call url-filename.
> url-generic-parse-url returns a url struct, and url-filename is the
> accessor for the slot. If there's code that uses the slot without
> going through the accessor, well... I think that's a bug (or, at the
> very least, dangerous and ugly) and sooner or later we'll find
> about it.

Sorry, yes, the above paragraph was written before I remembered that
url-filename was an accessor.  I just forgot to erase it before sending.

>> I think it's the right thing to do.  But maybe the best patch will need
>> to rename the `filename' slot and to define url-filename to do the
>> (expand-file-name (url-somethingelse URL) "/") or something like that.
> I don't think you can do that, because, as explained before in the
> thread, splitting the URL and using its parts are independent actions.
> It is entirely possible to split a file URL and then use it in
> different systems, isn't it?

Let's not talk about that for now.  Once there's a working patch
there'll be less chance for confusion.


        Stefan

bug#6339: url-filename => "/c:/some/file.txt"

[Chong Yidong]

Ressurrecting this discussion...

I just looked at the relevant code and at RFC 3986, and I think the
problem is that the url object returned by url-generic-parse-url has
badly named slots.  That function ought to be following the RFC, instead
of imposing its own interpretation of things.  So the FILENAME slot,
which is returned by url-filename, should really be two slots, PATH and
QUERY, whose meanings are unambiguously defined in the URI spec.  And
there should be no expectation that the PATH part corresponds to a
filename.

In particular, RFC 3986 explicitly states that

  If a URI contains an authority component, then the path component must
  either be empty or begin with a slash ("/") character.

That is to say, the / is part of the path.

Unfortunately, it's inconvenient to change the contents of the struct
now, since that means changing the arglist of the url-parse-make-urlobj
constructor; that constructor is already called from Tramp, and maybe
other packages.  So I think we should just explicitly state that the
FILENAME slot is really PATH and QUERY together, and wash our hands of
the matter.

This also means that it should be up to callers to convert the FILENAME
slot (i.e. PATH and QUERY) into proper filenames.  The translation from
URIs to filenames is scheme-independent anyway, so it shouldn't be
handled at the level of url-generic-parse-url.

bug#6339: url-filename => "/c:/some/file.txt"

[Juanma Barranquero]

On Wed, May 9, 2012 at 11:04 AM, Chong Yidong <cyd@gnu.org> wrote:

> In particular, RFC 3986 explicitly states that
>
>  If a URI contains an authority component, then the path component must
>  either be empty or begin with a slash ("/") character.
>
> That is to say, the / is part of the path.

This means that it is part of the "path" component of the URI, used to
separate and disambiguate, but does not say anything about how that is
interpreted as a a filesystem path. In this case (with no authority):

  file:///C:/windows/path

"/C:/windows/path" is the "path", but it is obviously not a filesystem path.

> So I think we should just explicitly state that the
> FILENAME slot is really PATH and QUERY together, and wash our hands of
> the matter.
>
> This also means that it should be up to callers to convert the FILENAME
> slot (i.e. PATH and QUERY) into proper filenames.  The translation from
> URIs to filenames is scheme-independent anyway, so it shouldn't be
> handled at the level of url-generic-parse-url.

I don't mind which way we fix it, but I'd be glad if we can avoid
snarky and erroneous "Windows does this wrong" comments in the code
while doing it...

    Juanma

bug#6339: url-filename => "/c:/some/file.txt"

[Chong Yidong]

Juanma Barranquero <lekktu@gmail.com> writes:

> I don't mind which way we fix it, but I'd be glad if we can avoid
> snarky and erroneous "Windows does this wrong" comments in the code
> while doing it...

I've committed a fix to ffap to DTRT, and don't see what else to patch.
Closing this bug; feel free to open a new one if there is a specific
example of a caller doing the wrong thing.