bug#76023: Sv: bug#76023: 31.0.50; Observed behavior for substitute-in-file-name different from the doc string

[arthur miller <arthur.miller@live.com>]

[-- Attachment #1: Type: text/plain, Size: 10806 bytes --]

> > From: arthur miller <arthur.miller@live.com>
> > CC: "76023@debbugs.gnu.org" <76023@debbugs.gnu.org>
> > Date: Mon, 3 Feb 2025 15:34:48 +0000
> >
> > > > (substitute-in-file-name "some-string/~$HOME") => some-string/~C:\Users\arthur
> > > >
> > > > However:
> > > >
> > > > (substitute-in-file-name "some-string~/$HOME") => C:\Users\arthur
> > >
> > > Yes.  Emacs on MS-Windows doesn't support the "~foo" notation, which
> >
> > Actually, is seems you do support that notation on Windows :-). It works here:
> >
> > ~/repos/emsrc/emacs/src $ (expand-file-name "~")
> > c:/Users/Arthur
> > ~/repos/emsrc/emacs/src $ (expand-file-name "~arthur")
> > c:/Users/Arthur
>
> Yes, if "foo" is your user name, then we do support "~foo" (because
> it's easy and expected).  But not for any other "foo".

> > ~/repos/emsrc/emacs/src $ (expand-file-name "~/test")
> > c:/Users/Arthur/test
> > ~/repos/emsrc/emacs/src $
> >
> > It seems that you are thinking of the wrong function here, or haven't looked at
> > the implementation. That would be expand-file-name that supports "~foo"
> > notation, not substitute-in-file-name.
>
> The function is different, but the reason is the same: Emacs doesn't
> support ~foo on MS-Windows, even if the user "foo" does exist on the
> system:
>
>   (substitute-in-file-name "some-string/~Administrator")
>    => "some-string/~Administrator"
>   (substitute-in-file-name "some-string/~SYSTEM")
>    => "some-string/~SYSTEM"

Seems like I have taken unfortunate example; since HOME expands into an absolute
file name, and I also tried PATH witch also expands into absolute. I see now:

(substitute-in-file-name "some-string~/$USERDOMAIN") => "some-string~/PASCAL"
(substitute-in-file-name "some-string/~$USERDOMAIN") => "some-string/~PASCAL"
(substitute-in-file-name "some-string//$USERDOMAIN") => "/PASCAL"

> > We are speaking about substitute-in-file-name, which supports "$foo" to
> > substitute an env variable in the string. Furthermore, it also supports "/~" and
> > "//" which means that everything before those two strings should be
> > discarded. As described in the bug report, "~/" does what the doc string
> > states.
>
> Yes.  Once again, Emacs on MS-Windows supports ~ and ~/ and ~user when
> "user" is the name of the current session's user.  In any other case
> the "~" is interpreted literally on MS-Windows, and "~SOMETHING" is
> *not* supposed to cause this function discard everything, because then
> it will be inconsistent with how other primitives treat ~foo on
> MS-Windows.
> > Observe that substitute-in-file-name never calls expand-file-name; so
> > that expansion of tilde you seem to speak about, should never take place there.
>
> I know.  But observe that it calls user_homedir, which in Emacs on
> MS-Windows works as I described above.

Ok, I see where you are doing it; happens because file_name_absolute_p actually
tries to obtain home directory for the token passed in. So this:

(substitute-in-file-name "some-string/~$USERNAME") => "~Arthur"

> > In short: everything is supported, the docs just display a bad string.
>
> No, that's an incorrect conclusion, for the reasons I explained.

Ok, got it now.

> > I think you are confused by my example. I choose "HOME" because the result
> > string is shorter. I could have used "$PATH" in those examples, or "$UserProfile".
>
> Yes, because the name of the current session's user is unlikely to be
> the expansion of $PATH.  But try this:
>
>   (substitute-in-file-name "some-string/~$LOGNAME")
>    => "~eliz"

I get just "some-string/~$LOGNAME". I guess I didn't configured my log.

> (My user name on this system is "eliz".)
>
> So I don't think I'm confused, no.
>
> > > It isn't a typo in the doc string, because on Posix systems Emacs
> > > behaves like the doc string says.  It is a common trait in
> > > documentation Emacs not to describe too many details about the
> > > idiosyncrasies of MS-Windows, especially in dark corners such as this
> > > one.
> >
> > Why does "~/" works as described in the doc string of substitute-in-file-name,
> > but the doc says "/~"?
>
> Both work here:
>
>   (substitute-in-file-name "some-string/~")
>    => "~"
>   (substitute-in-file-name "some-string/~/")
>    => "~/"

Also this works:

(substitute-in-file-name "some-string/~/foo") => "~/foo"

But this wont: (substitute-in-file-name "some-string/~foo")

Because I am not logged in as user "foo" on the system.

This will too clip everything before the substitution

(substitute-in-file-name "some-string~/$HOME") => "C:\\Users\\Arthur"

because the code first expands variables, and than clips the path, resulting
from file_absolute_path_p seing the drive letter, and it will do the same for
any env that starts with a drive letter:

(substitute-in-file-name "some-string~/$pylon")
(substitute-in-file-name "some-string/$pylon")
(substitute-in-file-name "foo/$pylon")

=> "C:\\Program Files\\Basler\\pylon 5\\Development\\"

Also: (substitute-in-file-name "foo/$pylon$JAVA_HOME")

=> "C:\\Program Files\\Java\\jdk-9\\"

Any part of string before a variable that expands into an absolute path name is
discarded:

(substitute-in-file-name "foo/$pylon$JAVA_HOME$OS")

=> "C:\\Program Files\\Java\\jdk-9\\Windows_NT"

Now I understand why ~/$HOME appeared like it"~/" was cut; it was actually
effect of $HOME expanding in an absolute path that caused the string before it to
be discarded. And I thought it was a typo :-).

> > > That shouldn't matter, since Emacs can use both forward and
> > > backslashes in file names.
> >
> > Yes, I understand that, I have seen it. It is not that it matters to Emacs per
> > se, or if the user types at the prompt interactively. It matters if someone
> > passes a result of that function to some other function which is not Emacs
> > built-in namestring manipulation function. Than that person can not assume
> > platform-independent namestrings, which I don't think would be unreasonable to
> > assume. Also, it is not very difficult to implement it either.
>
> What would that person do with this file name?  All the Emacs

I don't know. But I don't think it is neccessary to assume in which context they
would use the function.

> primitives that accept file names support both flavors of slashes.
> Example:
>
>   (file-name-nondirectory "d:\\foo/bar\\baz")
>    => "baz"

I know, I have implemented most of them in CL by now, and discovered they
work so. They also all emit out Emacs canonical namestrings; it is only those
two that don't (if I haven't missed some). It is easy and cheap to make them to
emit path in the same form as the rest. I first thought not to hack them, but I
did since I was anyway looking at the code. In the case of substitute-in-file-name,
it is just to call dostounix_filename on the result:

---
 src/fileio.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/src/fileio.c b/src/fileio.c
index 64d84b9adfe..12a07fc9add 100644
--- a/src/fileio.c
+++ b/src/fileio.c
@@ -2145,6 +2145,10 @@ DEFUN ("substitute-in-file-name", Fsubstitute_in_file_name,
        need to quote some $ to $$ first.  */
     xnm = p;

+#ifdef DOS_NT
+  dostounix_filename (xnm);
+#endif
+
 #ifdef WINDOWSNT
   if (!NILP (Vw32_downcase_file_names))
     {
--
2.48.1

And in file-name-concat is just to check the last written char for an argument:

#ifdef DOS_NT
#define IS_NATIVE_SEP(_c_) ((_c_) == '\\')
#else
#define IS_NATIVE_SEP(_c_) ((_c_) == '/')
#endif
( ... )
  /* Copy over the data. */
  char *p = SSDATA (result);
  for (i = 0; i < eargs; i++)
    {
      Lisp_Object arg = elements[i];
      memcpy (p, SSDATA (arg), SBYTES (arg));
      p += SBYTES (arg);
#ifdef DOS_NT
      if (IS_NATIVE_SEP ( *(p - 1) ))
      *(p - 1) = DIRECTORY_SEP;
#endif
      /* The last element shouldn't have a slash added at the end. */
      if (i < eargs - 1 && !IS_DIRECTORY_SEP (*(p - 1)))
      *p++ = DIRECTORY_SEP;
    }
( ... )

Alterantivel, dostounix_filename on the result can do it too, but this one seems
cheeper.

> So the only case where a Lisp program will fail for such file names is
> if that Lisp program manually analyzes the file name, instead of using
> Emacs primitives.  In which case I'd say the Lisp program has a bug.

Depends on. People are creative when writing programs and puting Emacs to
different use these days. But it is not so much about what is wrong or correct
usage, it is more about the principle of least surprise and having uniform
behavior for the api.

> But we digress, I think?

You can close the bug, I understand why do you say it is not a bug. Thank you
for the patience and helping me to understand it.

Also if you add a short sentence that says "if an expansion results in an
absolute path, part of the file name before the expansion is discarded" it wont
hurt. Or somthing in that line. It is a little bit arcane function that does
some more than described in the doc string.

> > The only two functions that has this different behavior are
> > substitute-in-file-name and concat-path-name. The reason is probably because
> > they don't expand file name as others functions in fileio.c do, but do their own
> > thing with string concatenations.
> >
> > As regression: (file-name-concat "foo" "bar\\" "baz") => "foo/bar\\baz"
>
> And why is that a problem?  (Of course, one should almost never use
> file-name-concat, it exists for a small number of very special cases;
> everything else should use expand-file-name.)

How should I know it is meant to be for a special cases? There is nothing about
special cases in docs. :) You have to document stuff. I write everywhere that
Emacs is *the* best documented Lisp in existence.

To note, expand-file-name always returns absolute file name. If someone
would like to construct a relative path programmatically than perhaps that
function is more apropriate? I don't know if that is a realistic use-case
but I can imagine it would be useful in that scenario.

But reason why I am looking at them, is because I have implemented those in CL.
If I find a bug of course I will report it and give you a fix if I can.

> > > It is a feature, but I don't think we want to mention it, because it's
> > > Windows-specific, and because "~$HOME" makes very little sense on
> > > Windows.
> >
> > I am not sure why "~$HOME" makes sense anywhere? Do you perhaps mean "~$USER"?
> No, I meant ~$HOME.  Since ~ is the same as $HOME, why double it?

Yes, of course, that is what I mean also: why would someone double it = makes
very little sense anywhere, not just on Windows.

[-- Attachment #2: Type: text/html, Size: 47497 bytes --]