expand-file-name, DOS/Windows, and directory separator

expand-file-name, DOS/Windows, and directory separator

[Eric Abrahamsen]

Hi,

My reading of `expand-file-name' (I don't really speak C) is that, if we
run it over a file path produced by an external process on a Windows
machine -- meaning path strings where the directory separator might be a
backward slash -- it will normalize that separator to a unix-style
forward slash. It looks like fileio.c:1247 calls dostounix_filename, and
I'm assuming that's what that does.

Is that a correct assumption? Can I rely on that behavior? Again this
isn't a file path produced by an Emacs function like `directory-files',
but by an external process returning a list of file names into a process
buffer.

TIA,
Eric

Re: expand-file-name, DOS/Windows, and directory separator

[Stefan Monnier]

Eric Abrahamsen [2022-02-15 09:59:53] wrote:
> My reading of `expand-file-name' (I don't really speak C) is that, if we
> run it over a file path produced by an external process on a Windows

I'll let Eli confirm that this is indeed guaranteed behavior, but I just
want to point out that the GNU convention (which we use) says these
aren't "path"s but "file names".


        Stefan

Re: expand-file-name, DOS/Windows, and directory separator

[Eric Abrahamsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Eric Abrahamsen [2022-02-15 09:59:53] wrote:
>> My reading of `expand-file-name' (I don't really speak C) is that, if we
>> run it over a file path produced by an external process on a Windows
>
> I'll let Eli confirm that this is indeed guaranteed behavior, but I just
> want to point out that the GNU convention (which we use) says these
> aren't "path"s but "file names".

Gotcha, I might have been trying to emphasize that it's the separators I
was interested in. Anyway, point taken.

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Eric Abrahamsen <eric@ericabrahamsen.net>
>> Date: Tue, 15 Feb 2022 09:59:53 -0800
>> 
>> My reading of `expand-file-name' (I don't really speak C) is that, if we
>> run it over a file path produced by an external process on a Windows
>> machine -- meaning path strings where the directory separator might be a
>> backward slash -- it will normalize that separator to a unix-style
>> forward slash. It looks like fileio.c:1247 calls dostounix_filename, and
>> I'm assuming that's what that does.
>> 
>> Is that a correct assumption?
>
> Yes.
>
>> Can I rely on that behavior?
>
> I'd rather you didn't.  Why do you need such an assumption?  Emacs on
> Windows can cope with file names that use any style of slashes.

This is code dealing with search results in Gnus, and the absolute file
names need to be broken up so we can work on their segments. Right now
that's done with regexps, which is ugly and fragile, and I'm just
looking for the confidence that:

(file-name-split (expand-file-name <file> "/"))

Is going to return exactly the segments, no more no less, regardless of
the system or separator type or whether there are multiple separators in
a row, etc etc. No leftover slashes, no empty strings, all that.

(Okay empty strings are fine, I guess `file-name-split' always returns
one for absolute file names.)

Re: expand-file-name, DOS/Windows, and directory separator

[Eli Zaretskii]

> From: Eric Abrahamsen <eric@ericabrahamsen.net>
> Date: Tue, 15 Feb 2022 11:39:47 -0800
> 
> >> Can I rely on that behavior?
> >
> > I'd rather you didn't.  Why do you need such an assumption?  Emacs on
> > Windows can cope with file names that use any style of slashes.
> 
> This is code dealing with search results in Gnus, and the absolute file
> names need to be broken up so we can work on their segments. Right now
> that's done with regexps, which is ugly and fragile, and I'm just
> looking for the confidence that:
> 
> (file-name-split (expand-file-name <file> "/"))

file-name-split is one of the functions that support both styles of
slashes, so you don't need to call expand-file-name at all.  (And "/"
is not really an absolute file name on Windows anyway).

> Is going to return exactly the segments, no more no less, regardless of
> the system or separator type or whether there are multiple separators in
> a row, etc etc. No leftover slashes, no empty strings, all that.
> 
> (Okay empty strings are fine, I guess `file-name-split' always returns
> one for absolute file names.)

It would be a bug for file-name-split (or any other file-name-*
function, really) to fail to recognize the parts of a file name
depending on the style of slashes.  So if you find a case where the
results depend on the slashes, even without running the file name
through expand-file-name, please report that as a bug.

Re: expand-file-name, DOS/Windows, and directory separator

[Eric Abrahamsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Eric Abrahamsen <eric@ericabrahamsen.net>
>> Date: Tue, 15 Feb 2022 11:39:47 -0800
>> 
>> >> Can I rely on that behavior?
>> >
>> > I'd rather you didn't.  Why do you need such an assumption?  Emacs on
>> > Windows can cope with file names that use any style of slashes.
>> 
>> This is code dealing with search results in Gnus, and the absolute file
>> names need to be broken up so we can work on their segments. Right now
>> that's done with regexps, which is ugly and fragile, and I'm just
>> looking for the confidence that:
>> 
>> (file-name-split (expand-file-name <file> "/"))
>
> file-name-split is one of the functions that support both styles of
> slashes, so you don't need to call expand-file-name at all.  (And "/"
> is not really an absolute file name on Windows anyway).

The `expand-file-name' is in there to collapse multiple consecutive
directory separators, which happens in the wild, as `file-name-split'
doesn't do that by itself.

>> Is going to return exactly the segments, no more no less, regardless of
>> the system or separator type or whether there are multiple separators in
>> a row, etc etc. No leftover slashes, no empty strings, all that.
>> 
>> (Okay empty strings are fine, I guess `file-name-split' always returns
>> one for absolute file names.)
>
> It would be a bug for file-name-split (or any other file-name-*
> function, really) to fail to recognize the parts of a file name
> depending on the style of slashes.  So if you find a case where the
> results depend on the slashes, even without running the file name
> through expand-file-name, please report that as a bug.

I'm not sure, as I wasn't able to test with any confidence. Since the
search results are simply plain strings in a buffer as far as Emacs is
concerned, I wasn't sure if the file name functions would give them any
special treatment. Just running this, where I've doubled the slashes so
as not to raise an error:

(file-name-split "C:\\Users\\eric\\random")

Just returns the whole string. But maybe this is a meaningless test.

This code has to handle all the weirdness of different systems and
filesystem layouts and search engines, so I'm being a little paranoid
about it.

Thanks,
Eric

Re: expand-file-name, DOS/Windows, and directory separator

[Eli Zaretskii]

> From: Eric Abrahamsen <eric@ericabrahamsen.net>
> Date: Tue, 15 Feb 2022 12:15:08 -0800
> 
> >> (file-name-split (expand-file-name <file> "/"))
> >
> > file-name-split is one of the functions that support both styles of
> > slashes, so you don't need to call expand-file-name at all.  (And "/"
> > is not really an absolute file name on Windows anyway).
> 
> The `expand-file-name' is in there to collapse multiple consecutive
> directory separators, which happens in the wild, as `file-name-split'
> doesn't do that by itself.

Then do call expand-file-name, but you don't need to assume anything
about slashes in the result.

> Just running this, where I've doubled the slashes so as not to raise
> an error:
> 
> (file-name-split "C:\\Users\\eric\\random")
> 
> Just returns the whole string.

I bet you did that on Unix?  Because here on MS-Windows it returns the
expected

   ("" "Users" "eric" "random")

> This code has to handle all the weirdness of different systems and
> filesystem layouts and search engines, so I'm being a little paranoid
> about it.

You don't need to be paranoid.  Emacs's file-name related primitives
are supposed to handle file names on all the supported systems;
anything else is a bug that should be fixed.

Re: expand-file-name, DOS/Windows, and directory separator

[Eric Abrahamsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Eric Abrahamsen <eric@ericabrahamsen.net>
>> Date: Tue, 15 Feb 2022 12:15:08 -0800
>> 
>> >> (file-name-split (expand-file-name <file> "/"))
>> >
>> > file-name-split is one of the functions that support both styles of
>> > slashes, so you don't need to call expand-file-name at all.  (And "/"
>> > is not really an absolute file name on Windows anyway).
>> 
>> The `expand-file-name' is in there to collapse multiple consecutive
>> directory separators, which happens in the wild, as `file-name-split'
>> doesn't do that by itself.
>
> Then do call expand-file-name, but you don't need to assume anything
> about slashes in the result.

Got it.

>> Just running this, where I've doubled the slashes so as not to raise
>> an error:
>> 
>> (file-name-split "C:\\Users\\eric\\random")
>> 
>> Just returns the whole string.
>
> I bet you did that on Unix?  Because here on MS-Windows it returns the
> expected
>
>    ("" "Users" "eric" "random")

As I was hoping it would! But yes, I only have unix-like boxes to test
on.

>> This code has to handle all the weirdness of different systems and
>> filesystem layouts and search engines, so I'm being a little paranoid
>> about it.
>
> You don't need to be paranoid.  Emacs's file-name related primitives
> are supposed to handle file names on all the supported systems;
> anything else is a bug that should be fixed.

Ignorance breeds magical thinking :) Thanks for the reassurance, I'll go
ahead and update this.

Re: expand-file-name, DOS/Windows, and directory separator

[Stefan Monnier]

> As I was hoping it would! But yes, I only have unix-like boxes to test on.

Side note: last time I tried it (it was some years ago, admittdly), the
Window binary of Emacs worked in WINE.  Not great, but well enough for
simple tests like this one.


        Stefan

Re: expand-file-name, DOS/Windows, and directory separator

[Eric Abrahamsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> As I was hoping it would! But yes, I only have unix-like boxes to test on.
>
> Side note: last time I tried it (it was some years ago, admittdly), the
> Window binary of Emacs worked in WINE.  Not great, but well enough for
> simple tests like this one.

Okay! Now installed.

Re: expand-file-name, DOS/Windows, and directory separator

[Eli Zaretskii]

> From: Eric Abrahamsen <eric@ericabrahamsen.net>
> Date: Tue, 15 Feb 2022 09:59:53 -0800
> 
> My reading of `expand-file-name' (I don't really speak C) is that, if we
> run it over a file path produced by an external process on a Windows
> machine -- meaning path strings where the directory separator might be a
> backward slash -- it will normalize that separator to a unix-style
> forward slash. It looks like fileio.c:1247 calls dostounix_filename, and
> I'm assuming that's what that does.
> 
> Is that a correct assumption?

Yes.

> Can I rely on that behavior?

I'd rather you didn't.  Why do you need such an assumption?  Emacs on
Windows can cope with file names that use any style of slashes.