`rename-file' with leading spaces in filename

`rename-file' with leading spaces in filename

[martin rudalics]

With GNU Emacs 22.0.50.1 (i386-mingw-windows98.3000):

Suppose I have a file called "c:/foo"

Doing M-x rename-file RET c:/foo RET c:/ foo
       M-x rename-file RET c:/ foo RET c:/foo

gets me:

File c:/foo already exists; rename to it anyway? (y or n)

After confirmation Emacs renames as expected, the behavior is, however,
inconsistent since the first rename didn't ask for a confirmation.


Suppose now I have two files called "c:/ foo" (one leading space)
and "c:/  foo" (two leading spaces).

Doing M-x rename-file RET c:/ foo RET c:/foo

gets me:

File c:/foo already exists; rename to it anyway? (y or n)

If I now confirm, the file "c:/  foo" (two leading spaces) is deleted and
"c:/ foo" (one leading space) renamed to "c:/foo".  That's unfriendly.

Re: `rename-file' with leading spaces in filename

[Eli Zaretskii]

> Date: Fri, 30 Dec 2005 15:20:20 +0100
> From: martin rudalics <rudalics@gmx.at>
> 
> With GNU Emacs 22.0.50.1 (i386-mingw-windows98.3000):
> 
> Suppose I have a file called "c:/foo"
> 
> Doing M-x rename-file RET c:/foo RET c:/ foo
>        M-x rename-file RET c:/ foo RET c:/foo
> 
> gets me:
> 
> File c:/foo already exists; rename to it anyway? (y or n)
> 
> After confirmation Emacs renames as expected, the behavior is, however,
> inconsistent since the first rename didn't ask for a confirmation.

I don't have access to Windows 9x/ME anymore to verify my guess, but I
think I know what is the reason: it's the fact that Windows 9x
generates a 8+3 alias for every file whose name violates the DOS
file-naming constraints.

The Windows algorithm for generating the 8+3 alias begins with
removing all the whitespace characters from the file name, and then
replaces the other invalid characters with `_' and truncates the
result to 8+3 limits.  This makes short alias for "c:/ foo" to be --
ta-da! -- "c:/foo".  You should be able to verify that by looking at
the output of the DIR command built into command.com, which shows the
8+3 alias next to the long file name.

In a sense, these two names are like hard links on Posix systems: they
both point to the same data on disk.

If I'm right, then Emacs is not to blame, since the target file indeed
exists (although in this case, no harm would be done by renaming,
since you are renaming the same file).

> Suppose now I have two files called "c:/ foo" (one leading space)
> and "c:/  foo" (two leading spaces).
> 
> Doing M-x rename-file RET c:/ foo RET c:/foo
> 
> gets me:
> 
> File c:/foo already exists; rename to it anyway? (y or n)
> 
> If I now confirm, the file "c:/  foo" (two leading spaces) is deleted and
> "c:/ foo" (one leading space) renamed to "c:/foo".  That's unfriendly.

  Patient: Doctor, it hurts when I do like this.
  Doctor: Well, then don't do that!

Seriously, though: you _have_ been warned that you are going to
overwrite an existing file, you decided to go ahead anyway, and now
you complain?  If I were you, I'd first visit "c:/foo" and would
decide whether to overwrite it depending on what contents I saw.

What happens is this: both "c:/ foo" and "c:/  foo" map to the same
short alias "c:/foo".  So, to solve the name clash, Windows generates
numeric tails, so one of the two files (or both) has its short alias
as "c:/foo~2" or some such.  Which one of the two maps to "c:/foo"
depends on which one was created first.  In your case, it sounds like
it was "c:/  foo"; it could be the other way around.  When you rename
"c:/ foo", it nukes "c:/foo", whose data is the one you referred to as
"c:/  foo".

Windows NT and its descendants, including XP which I run, don't have
this problem, because I think they don't store short aliases on the
disk in the same way as Windows 9x does.

Re: `rename-file' with leading spaces in filename

[martin rudalics]

Thanks for the delightful response.

 > I don't have access to Windows 9x/ME anymore to verify my guess, but I
 > think I know what is the reason: it's the fact that Windows 9x
 > generates a 8+3 alias for every file whose name violates the DOS
 > file-naming constraints.

 > The Windows algorithm for generating the 8+3 alias begins with
 > removing all the whitespace characters from the file name, and then
 > replaces the other invalid characters with `_' and truncates the
 > result to 8+3 limits.  This makes short alias for "c:/ foo" to be --
 > ta-da! -- "c:/foo".  You should be able to verify that by looking at
 > the output of the DIR command built into command.com, which shows the
 > 8+3 alias next to the long file name.

Your guess is precise.  It wouldn't have come into my mind that Windows
does _not_ tildify when there are 8+3 offenders and all of them are
spaces.  The renaming problem occurs when there's a short alias that
does not have a tilde and an offending long name maps to that alias.

By the way look at the dir output below:

FOO                     10  02.01.06   8:16  foo
FOO~1                   13  02.01.06   8:19   foo
FOO~                    13  02.01.06   8:13   foo~
FOO~                    10  02.01.06   7:45  foo~

The funny thing here is that the backups have identic short aliases.
When I `find-file-noselect' " foo~" or "  foo~" Emacs opens the correct
file.  When I `find-file-noselect' the non-existent "foo~" Emacs opens
"  foo~" instead.

 > Windows NT and its descendants, including XP which I run,
 > don't have this problem, because I think they don't store short
 > aliases on the disk in the same way as Windows 9x does.

Maybe because FAT32 records file names in the order they have been
created and NTFS records them alphabetically.  However, I believe that
FindFirstFile may return the short alias regardless of the file system
employed and/or whether Unicode is used.  Hence I wouldn't rely on XPs
never running into a similar problem.

To solve this problem for me I'd probably have to compare `cFileName'
with the name FindFirstFile was looking for.  If these match, the file
should really exist and I'd issue the standard warning.  Otherwise, I'd
temporarily rename the file referenced by `cAlternateFileName', do the
renaming I intended, and re-rename the temporary one.  Essentially what
`sys_rename' does for Windows 95.

Re: `rename-file' with leading spaces in filename

[Eli Zaretskii]

> Date: Mon, 02 Jan 2006 10:48:59 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: emacs-devel@gnu.org
> 
> Your guess is precise.  It wouldn't have come into my mind that Windows
> does _not_ tildify when there are 8+3 offenders and all of them are
> spaces.

I'm not sure, but perhaps this is due to some non-default setting,
like if NameNumericTails is set to zero in the Registry.

> By the way look at the dir output below:
> 
> FOO                     10  02.01.06   8:16  foo
> FOO~1                   13  02.01.06   8:19   foo
> FOO~                    13  02.01.06   8:13   foo~
> FOO~                    10  02.01.06   7:45  foo~
> 
> The funny thing here is that the backups have identic short aliases.

Yes, it's a known bug in the alias generation algorithm.

> When I `find-file-noselect' " foo~" or "  foo~" Emacs opens the correct
> file.  When I `find-file-noselect' the non-existent "foo~" Emacs opens
> "  foo~" instead.

Probably because " foo~" is the first one of these two in the
directory, as the output from DIR suggests.

Re: `rename-file' with leading spaces in filename

[martin rudalics]

 > Yes, it's a known bug in the alias generation algorithm.

Which means that when "foobar" and "foo bar" reside in one and the same
directory and I save a buffer visiting "foobar", I can take for granted
that my backup for "foo bar" is lost irrevocably.

Re: `rename-file' with leading spaces in filename

[Eli Zaretskii]

> Date: Tue, 03 Jan 2006 10:47:16 +0100
> From: martin rudalics <rudalics@gmx.at>
> CC: emacs-devel@gnu.org
> 
>  > Yes, it's a known bug in the alias generation algorithm.
> 
> Which means that when "foobar" and "foo bar" reside in one and the same
> directory and I save a buffer visiting "foobar", I can take for granted
> that my backup for "foo bar" is lost irrevocably.

Did you actually try that?  w32.c:sys_rename jumps through the hoops
to work around the Windows 9X bugs related to aliases.  Perhaps it
fails in this case, but it would be nice if you could step with a
debugger through sys_rename and tell where the logic there fails.

Re: `rename-file' with leading spaces in filename

[martin rudalics]

 > Did you actually try that?  w32.c:sys_rename jumps through the hoops
 > to work around the Windows 9X bugs related to aliases.  Perhaps it
 > fails in this case, but it would be nice if you could step with a
 > debugger through sys_rename and tell where the logic there fails.

I completely fail to understand the logic of this, hence you probably
have to tell me more precisely what to do (I didn't anticipate that this
special part of sys_rename would apply to Windows9_8_ at all).

Suppose my directory "c:/x" contains

FOOBAR~1                11  04.01.06   8:48 foo bar
FOOBAR~                 11  04.01.06   8:47 foo bar~
FOOBAR                  10  04.01.06   8:43 foobar

and I try to save some changes in a buffer visiting "foobar".

With `backup-by-copying' non-nil - my standard setting - saving doesn't
pass through sys_rename.  With `backup-by-copying' nil saving behaves in
a rather unpredictable way, I never bothered to find out why.  Anyway,
from a trace with sys_rename residing at line 2137 of w32.c I got the
following excerpts:


... line 2155:
oldname = 0x22f8af0 "c:/x/foobar"

... line 2178:
o = 0x12ead2d "foobar"
p = 0xd2d675 "foobar"
i = 0
result = 33620403
temp = "c:\\x\\foobar\0\230×Ò\0׺ìÿ°ÖÒ\0\224ÖÒ\0\001\0\0\0\006\0\0\0\004\0\0\200\0\0\0\0\0\0\0\0Ø\032\a\001\0\0\0\0ÿ\v\003Àÿÿÿÿ\037\0\001", '\0' <repeats 25 times>, "\177\003ÿÿ \001ÿÿÿÿÿÿ", '\0' <repeats 14 times>, "ÿÿ", '\0' <repeats 46 
times>, "\200Æ\b@", '\0' <repeats 12 times>, "@\237jïØÎ7@<'\\\017Øìv\207\035@\n\0\0\0\0\0\0\0g5\0\0\207\001\0\0\207\001\0\0\003\003\001\002ã\230\"\002³\001\001\002Ð\001\001\002\220\001\001\002\0\0\0\0\230×Ò\0Ø\032\a\001\177\001\0\0F"...
oldname = 0x12ead28 "c:\\x\\foobar"

... line 2179:
o = 0x12ead2d "foobar"
p = 0xd2d675 "_.foobar.0"
i = 0
result = 33620403
temp = "c:\\x\\_.foobar.0\0׺ìÿ°ÖÒ\0\224ÖÒ\0\001\0\0\0\006\0\0\0\004\0\0\200\0\0\0\0\0\0\0\0Ø\032\a\001\0\0\0\0ÿ\v\003Àÿÿÿÿ\037\0\001", '\0' <repeats 25 times>, "\177\003ÿÿ \001ÿÿÿÿÿÿ", '\0' <repeats 14 times>, "ÿÿ", '\0' <repeats 46 
times>, "\200Æ\b@", '\0' <repeats 12 times>, "@\237jïØÎ7@<'\\\017Øìv\207\035@\n\0\0\0\0\0\0\0g5\0\0\207\001\0\0\207\001\0\0\003\003\001\002ã\230\"\002³\001\001\002Ð\001\001\002\220\001\001\002\0\0\0\0\230×Ò\0Ø\032\a\001\177\001\0\0F"...
oldname = 0x12ead28 "c:\\x\\foobar"

... line 2199:
result = 33620403
temp = "c:\\x\\_.foobar.0\0׺ìÿ°ÖÒ\0\224ÖÒ\0\001\0\0\0\006\0\0\0\004\0\0\200\0\0\0\0\0\0\0\0Ø\032\a\001\0\0\0\0ÿ\v\003Àÿÿÿÿ\037\0\001", '\0' <repeats 25 times>, "\177\003ÿÿ \001ÿÿÿÿÿÿ", '\0' <repeats 14 times>, "ÿÿ", '\0' <repeats 46 
times>, "\200Æ\b@", '\0' <repeats 12 times>, "@\237jïØÎ7@<'\\\017Øìv\207\035@\n\0\0\0\0\0\0\0g5\0\0\207\001\0\0\207\001\0\0\003\003\001\002ã\230\"\002³\001\001\002Ð\001\001\002\220\001\001\002\0\0\0\0\230×Ò\0Ø\032\a\001\177\001\0\0F"...
oldname = 0x12ead28 "c:\\x\\foobar"


Regardless of how `backup-by-copying' is set, "foo bar~" doesn't exist
any more after this:

FOOBAR~1                11  04.01.06   8:48 foo bar
FOOBAR~                 10  04.01.06   8:43 foobar~
FOOBAR                  10  04.01.06   8:56 foobar

Re: `rename-file' with leading spaces in filename

[Richard M. Stallman]

    Suppose I have a file called "c:/foo"

    Doing M-x rename-file RET c:/foo RET c:/ foo

At that point, which relevant files exist?

	   M-x rename-file RET c:/ foo RET c:/foo

    gets me:

    File c:/foo already exists; rename to it anyway? (y or n)

Can you determine which primitive concludes that it exists
and see what arguments it got?

Re: `rename-file' with leading spaces in filename

[martin rudalics]

 >     Suppose I have a file called "c:/foo"

FOO                     10  02.01.06   7:45 foo

where "FOO" is the short filename (aka 8+3 alias) and "foo" is the long
filename.

 >     Doing M-x rename-file RET c:/foo RET c:/ foo
 >
 > At that point, which relevant files exist?

FOO                     10  02.01.06   7:45  foo

The short filename remains unaffected by the renaming operation, the
additional space between "7:45" and "foo" shows that the long filename
was changed.

 >
 > 	   M-x rename-file RET c:/ foo RET c:/foo
 >
 >     gets me:
 >
 >     File c:/foo already exists; rename to it anyway? (y or n)
 >
 > Can you determine which primitive concludes that it exists
 > and see what arguments it got?
 >

The FindFirstFile call in

fh = FindFirstFile (name, &wfd);

around line 2430 in w32.c finds "c:/foo" due to the fact that
FindFirstFile is allowed to return a file whose short filename matches
the parameter "name" (provided - as I suspect - it does not find a file
whose long filename matches "name").

Re: `rename-file' with leading spaces in filename

[Eli Zaretskii]

> Date: Mon, 02 Jan 2006 10:52:53 +0100
> From: martin rudalics <rudalics@gmx.at>
> Cc: emacs-devel@gnu.org
> 
>  > Can you determine which primitive concludes that it exists
>  > and see what arguments it got?
>  >
> 
> The FindFirstFile call in
> 
> fh = FindFirstFile (name, &wfd);
> 
> around line 2430 in w32.c

which is part of the MS-Windows port's emulation of `stat'.

> finds "c:/foo" due to the fact that
> FindFirstFile is allowed to return a file whose short filename matches
> the parameter "name" (provided - as I suspect - it does not find a file
> whose long filename matches "name").

Yes, FindFirstFile is allowed to find short aliases.