unofficial mirror of bug-gnu-emacs@gnu.org 
 help / color / mirror / code / Atom feed
* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
@ 2019-07-03 18:07 Erik Hahn
  2019-07-08 20:59 ` Lars Ingebrigtsen
                   ` (2 more replies)
  0 siblings, 3 replies; 23+ messages in thread
From: Erik Hahn @ 2019-07-03 18:07 UTC (permalink / raw)
  To: 36490


Call (directory-files-recursively DIR ".*") where DIR contains a
subdirectory named "~" but is not the parent of your home directory. The
function will recurse into your home directory. This is probably because
(expand-file-name "~" ...) returns the home directory.


In GNU Emacs 26.1 (build 1, x86_64-pc-linux-gnu, X toolkit)
 of 2018-11-03 built on p5
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
System Description:	Gentoo Base System release 2.6

Recent messages:
Configured using:
 'configure --prefix=/usr --build=x86_64-pc-linux-gnu
 --host=x86_64-pc-linux-gnu --mandir=/usr/share/man
 --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc
 --localstatedir=/var/lib --disable-silent-rules
 --docdir=/usr/share/doc/emacs-26.1-r3
 --htmldir=/usr/share/doc/emacs-26.1-r3/html --libdir=/usr/lib64
 --program-suffix=-emacs-26 --infodir=/usr/share/info/emacs-26
 --localstatedir=/var
 --enable-locallisppath=/etc/emacs:/usr/share/emacs/site-lisp
 --without-compress-install --without-hesiod --without-pop
 --with-file-notification=inotify --enable-acl --with-dbus
 --without-modules --without-gameuser --with-gpm --without-kerberos
 --without-kerberos5 --without-lcms2 --with-xml2 --without-mailutils
 --without-selinux --with-gnutls --with-libsystemd --with-threads
 --without-wide-int --with-zlib --with-sound=alsa --with-x --without-ns
 --without-gconf --without-gsettings --without-toolkit-scroll-bars
 --with-gif --with-jpeg --with-png --with-rsvg --with-tiff --with-xpm
 --without-imagemagick --without-xft --without-cairo --without-libotf
 --without-m17n-flt --with-x-toolkit=lucid --with-xaw3d 'CFLAGS=-O2
 -pipe -march=native' CPPFLAGS= 'LDFLAGS=-Wl,-O1 -Wl,--as-needed''

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG SOUND GPM DBUS NOTIFY ACL GNUTLS
LIBXML2 ZLIB LUCID X11 THREADS LIBSYSTEMD

Important settings:
  value of $LC_MONETARY: en_US.UTF-8
  value of $LC_NUMERIC: en_US.UTF-8
  value of $LC_TIME: en_GB.UTF-8
  value of $LANG: en_GB.utf8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-03 18:07 bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~" Erik Hahn
@ 2019-07-08 20:59 ` Lars Ingebrigtsen
  2019-07-08 21:08 ` Lars Ingebrigtsen
  2019-07-10 12:04 ` Lars Ingebrigtsen
  2 siblings, 0 replies; 23+ messages in thread
From: Lars Ingebrigtsen @ 2019-07-08 20:59 UTC (permalink / raw)
  To: Erik Hahn; +Cc: 36490

Erik Hahn <erik_hahn@gmx.de> writes:

> Call (directory-files-recursively DIR ".*") where DIR contains a
> subdirectory named "~" but is not the parent of your home directory. The
> function will recurse into your home directory. This is probably because
> (expand-file-name "~" ...) returns the home directory.

Oops:

(expand-file-name "~" "/tmp/")
=> "/home/larsi"

Yup; that function should be rewritten to have no instances of
`expand-file-name', which is too DWIM for a function like this.  I'll do
a rewrite...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-03 18:07 bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~" Erik Hahn
  2019-07-08 20:59 ` Lars Ingebrigtsen
@ 2019-07-08 21:08 ` Lars Ingebrigtsen
  2019-07-08 21:30   ` Erik Hahn
  2019-07-09 15:35   ` Eli Zaretskii
  2019-07-10 12:04 ` Lars Ingebrigtsen
  2 siblings, 2 replies; 23+ messages in thread
From: Lars Ingebrigtsen @ 2019-07-08 21:08 UTC (permalink / raw)
  To: Erik Hahn; +Cc: 36490

Actually, the doc string of expand-file-name is either wrong, or the
implementation is.

It says:

---
An initial ‘~/’ expands to your home directory.
An initial ‘~USER/’ expands to USER’s home directory.
---

Assuming the "An initial" refers to the first parameter, then

(expand-file-name "~/" "/tmp/")
=> "/home/larsi/"

works as advertised, but

(expand-file-name "~" "/tmp/")
=> "/home/larsi"

is a different thing: "~" is a perfectly valid file name, so having this
function map that to something else is just...  wrong.

(expand-file-name "~larsi" "/tmp/")
=> "/home/larsi"

is the same: Also wrong and undocumented.

The doc string continues with further confusion:

---
See also the function ‘substitute-in-file-name’.
---

See it for...  what?  For further expansions this function is going to
do?  Fortunately not:

(expand-file-name "$HOME" "/tmp/")
=> "/tmp/$HOME"

So that's probably just meant as "that's also a function that does file
name stuff, but it has nothing to do with this ~ thing we just
discussed"?

So what to do here?  I think the current, undocumented

(expand-file-name "~" "/tmp/")
=> "/home/larsi"

must surely be an error, and that should be fixed instead of the
callers?  Opinions?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no






^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-08 21:08 ` Lars Ingebrigtsen
@ 2019-07-08 21:30   ` Erik Hahn
  2019-07-08 21:56     ` Lars Ingebrigtsen
  2019-07-09 15:39     ` Eli Zaretskii
  2019-07-09 15:35   ` Eli Zaretskii
  1 sibling, 2 replies; 23+ messages in thread
From: Erik Hahn @ 2019-07-08 21:30 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: 36490

On 08/07/19 23:08, Lars Ingebrigtsen wrote:
> The doc string continues with further confusion:
>
> ---
> See also the function ‘substitute-in-file-name’.
> ---
>
> See it for...  what?  For further expansions this function is going to
> do?

It is related in that it also chokes on files named "~":

(substitute-in-file-name "/tmp/~") => "~"

Its doc string is also wrong. It says

 > If ‘/~’ appears, all of FILENAME through that ‘/’ is discarded.

but

(substitute-in-file-name "/tmp/~foo") => "/tmp/~foo"





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-08 21:30   ` Erik Hahn
@ 2019-07-08 21:56     ` Lars Ingebrigtsen
  2019-07-09 15:39     ` Eli Zaretskii
  1 sibling, 0 replies; 23+ messages in thread
From: Lars Ingebrigtsen @ 2019-07-08 21:56 UTC (permalink / raw)
  To: Erik Hahn; +Cc: 36490

Erik Hahn <erik_hahn@gmx.de> writes:

> It is related in that it also chokes on files named "~":
>
> (substitute-in-file-name "/tmp/~") => "~"
>
> Its doc string is also wrong. It says
>
>> If ‘/~’ appears, all of FILENAME through that ‘/’ is discarded.
>
> but
>
> (substitute-in-file-name "/tmp/~foo") => "/tmp/~foo"

Yup, it's only if the user exists:

(substitute-in-file-name "/tmp/~larsi")
=> "~larsi"

Same as with expand-file-name:

(expand-file-name "~larsi" "/tmp/")
=> "/home/larsi"

(expand-file-name "~foo" "/tmp/")
=> "/tmp/~foo"

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-08 21:08 ` Lars Ingebrigtsen
  2019-07-08 21:30   ` Erik Hahn
@ 2019-07-09 15:35   ` Eli Zaretskii
  2019-07-09 15:50     ` Lars Ingebrigtsen
  1 sibling, 1 reply; 23+ messages in thread
From: Eli Zaretskii @ 2019-07-09 15:35 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: 36490, erik_hahn

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Mon, 08 Jul 2019 23:08:46 +0200
> Cc: 36490@debbugs.gnu.org
> 
> Actually, the doc string of expand-file-name is either wrong, or the
> implementation is.
> 
> It says:
> 
> ---
> An initial ‘~/’ expands to your home directory.
> An initial ‘~USER/’ expands to USER’s home directory.
> ---
> 
> Assuming the "An initial" refers to the first parameter, then
> 
> (expand-file-name "~/" "/tmp/")
> => "/home/larsi/"
> 
> works as advertised, but
> 
> (expand-file-name "~" "/tmp/")
> => "/home/larsi"
> 
> is a different thing: "~" is a perfectly valid file name, so having this
> function map that to something else is just...  wrong.

If you want "~" to be interpreted literally, you need to protect it
with "/:".

> (expand-file-name "~larsi" "/tmp/")
> => "/home/larsi"
> 
> is the same: Also wrong and undocumented.

Why would we want to document that?

> The doc string continues with further confusion:
> 
> ---
> See also the function ‘substitute-in-file-name’.
> ---
> 
> See it for...  what?

For expanding environment variables, and for the special effect of
"//" etc.

> So what to do here?  I think the current, undocumented
> 
> (expand-file-name "~" "/tmp/")
> => "/home/larsi"
> 
> must surely be an error, and that should be fixed instead of the
> callers?  Opinions?

It is not an error.





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-08 21:30   ` Erik Hahn
  2019-07-08 21:56     ` Lars Ingebrigtsen
@ 2019-07-09 15:39     ` Eli Zaretskii
  1 sibling, 0 replies; 23+ messages in thread
From: Eli Zaretskii @ 2019-07-09 15:39 UTC (permalink / raw)
  To: Erik Hahn; +Cc: 36490, larsi

> From: Erik Hahn <erik_hahn@gmx.de>
> Date: Mon, 8 Jul 2019 23:30:38 +0200
> Cc: 36490@debbugs.gnu.org
> 
>  > If ‘/~’ appears, all of FILENAME through that ‘/’ is discarded.
> 
> but
> 
> (substitute-in-file-name "/tmp/~foo") => "/tmp/~foo"

Do you have a user named "foo" on that system?  If not, try a real
user name instead of "foo".





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-09 15:35   ` Eli Zaretskii
@ 2019-07-09 15:50     ` Lars Ingebrigtsen
  2019-07-09 16:12       ` Eli Zaretskii
  2019-07-09 16:17       ` Andreas Schwab
  0 siblings, 2 replies; 23+ messages in thread
From: Lars Ingebrigtsen @ 2019-07-09 15:50 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 36490, erik_hahn

Eli Zaretskii <eliz@gnu.org> writes:

>> An initial ‘~/’ expands to your home directory.
>> An initial ‘~USER/’ expands to USER’s home directory.
>> ---
>> 
>> Assuming the "An initial" refers to the first parameter, then

[...]

> If you want "~" to be interpreted literally, you need to protect it
> with "/:".

Sorry; I don't quite follow you here.  The doc string says that "~/" is
interpreted specially.  There's no "/" in "~".  :-)

That 

(expand-file-name "~/" "/tmp/")
=> "/home/larsi/"

is fine, because "~/" isn't a valid file name.  But "~" is, and that's
the problem.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-09 15:50     ` Lars Ingebrigtsen
@ 2019-07-09 16:12       ` Eli Zaretskii
  2019-07-09 16:27         ` Lars Ingebrigtsen
  2019-07-09 18:58         ` Basil L. Contovounesios
  2019-07-09 16:17       ` Andreas Schwab
  1 sibling, 2 replies; 23+ messages in thread
From: Eli Zaretskii @ 2019-07-09 16:12 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: 36490, erik_hahn

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: erik_hahn@gmx.de,  36490@debbugs.gnu.org
> Date: Tue, 09 Jul 2019 17:50:49 +0200
> 
> > If you want "~" to be interpreted literally, you need to protect it
> > with "/:".
> 
> Sorry; I don't quite follow you here.  The doc string says that "~/" is
> interpreted specially.  There's no "/" in "~".  :-)

But it does NOT say that "~" will NOT be interpreted specially.

> That 
> 
> (expand-file-name "~/" "/tmp/")
> => "/home/larsi/"
> 
> is fine, because "~/" isn't a valid file name.  But "~" is, and that's
> the problem.

No, I think the problem is that the caller should have protected the
literal "~", as I said.





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-09 15:50     ` Lars Ingebrigtsen
  2019-07-09 16:12       ` Eli Zaretskii
@ 2019-07-09 16:17       ` Andreas Schwab
  1 sibling, 0 replies; 23+ messages in thread
From: Andreas Schwab @ 2019-07-09 16:17 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: 36490, erik_hahn

On Jul 09 2019, Lars Ingebrigtsen <larsi@gnus.org> wrote:

> is fine, because "~/" isn't a valid file name.

"~/" is a valid file name, that happens to be a directory.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-09 16:12       ` Eli Zaretskii
@ 2019-07-09 16:27         ` Lars Ingebrigtsen
  2019-07-09 16:44           ` Eli Zaretskii
  2019-07-09 18:58         ` Basil L. Contovounesios
  1 sibling, 1 reply; 23+ messages in thread
From: Lars Ingebrigtsen @ 2019-07-09 16:27 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 36490, erik_hahn

Eli Zaretskii <eliz@gnu.org> writes:

>> > If you want "~" to be interpreted literally, you need to protect it
>> > with "/:".
>> 
>> Sorry; I don't quite follow you here.  The doc string says that "~/" is
>> interpreted specially.  There's no "/" in "~".  :-)
>
> But it does NOT say that "~" will NOT be interpreted specially.

I think it's a good idea to specify what strings will be interpreted
specially -- otherwise it's a bit difficult to use that function.

>> is fine, because "~/" isn't a valid file name.  But "~" is, and that's
>> the problem.
>
> No, I think the problem is that the caller should have protected the
> literal "~", as I said.

Any function that iterates over a list of directory files (as returned
by directory-files) would then have to quote whatever it passes to
expand-file-name -- or avoid that function completely.  That seems like
an odd design.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-09 16:27         ` Lars Ingebrigtsen
@ 2019-07-09 16:44           ` Eli Zaretskii
  2019-07-09 16:50             ` Lars Ingebrigtsen
  2019-07-09 16:52             ` Eli Zaretskii
  0 siblings, 2 replies; 23+ messages in thread
From: Eli Zaretskii @ 2019-07-09 16:44 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: 36490, erik_hahn

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: erik_hahn@gmx.de,  36490@debbugs.gnu.org
> Date: Tue, 09 Jul 2019 18:27:28 +0200
> 
> > No, I think the problem is that the caller should have protected the
> > literal "~", as I said.
> 
> Any function that iterates over a list of directory files (as returned
> by directory-files) would then have to quote whatever it passes to
> expand-file-name -- or avoid that function completely.  That seems like
> an odd design.

What happens if a function that iterates over a directory finds a
subdirectory whose name is literally "~"?





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-09 16:44           ` Eli Zaretskii
@ 2019-07-09 16:50             ` Lars Ingebrigtsen
  2019-07-09 16:52             ` Eli Zaretskii
  1 sibling, 0 replies; 23+ messages in thread
From: Lars Ingebrigtsen @ 2019-07-09 16:50 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 36490, erik_hahn

Eli Zaretskii <eliz@gnu.org> writes:

>> Any function that iterates over a list of directory files (as returned
>> by directory-files) would then have to quote whatever it passes to
>> expand-file-name -- or avoid that function completely.  That seems like
>> an odd design.
>
> What happens if a function that iterates over a directory finds a
> subdirectory whose name is literally "~"?

I think I must be missing something, because I don't think anything
special should happen?

(find-file "/tmp/~/foo")

works fine, for instance.  And

(expand-file-name "foo" "/tmp/~")
=> "/tmp/~/foo"

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-09 16:44           ` Eli Zaretskii
  2019-07-09 16:50             ` Lars Ingebrigtsen
@ 2019-07-09 16:52             ` Eli Zaretskii
  2019-07-09 17:00               ` Lars Ingebrigtsen
  1 sibling, 1 reply; 23+ messages in thread
From: Eli Zaretskii @ 2019-07-09 16:52 UTC (permalink / raw)
  To: larsi; +Cc: 36490, erik_hahn

> Date: Tue, 09 Jul 2019 19:44:31 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 36490@debbugs.gnu.org, erik_hahn@gmx.de
> 
> What happens if a function that iterates over a directory finds a
> subdirectory whose name is literally "~"?

Sorry, that's not what I meant to say.  I meant to say how would a
Lisp program know whether (expand-file-name "~/") means the home
directory or a directory whose name is literally "~"?

Btw, stuff like (expand-file-name "foo/~/") already does what you
want, so the problem is only with the leading '~', and can be avoided
if we avoid that situation.  IOW, why should this example:

  (expand-file-name "~" "/tmp/")
     => "/home/larsi"

determine how directory-files-recursively behaves?





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-09 16:52             ` Eli Zaretskii
@ 2019-07-09 17:00               ` Lars Ingebrigtsen
  2019-07-09 17:23                 ` Eli Zaretskii
  0 siblings, 1 reply; 23+ messages in thread
From: Lars Ingebrigtsen @ 2019-07-09 17:00 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 36490, erik_hahn

Eli Zaretskii <eliz@gnu.org> writes:

> Sorry, that's not what I meant to say.  I meant to say how would a
> Lisp program know whether (expand-file-name "~/") means the home
> directory or a directory whose name is literally "~"?

Well, we have documented that in expand-file-name "~/" means the home
directory, and I have no problems with that.

"~/" isn't something you'll ever get from functions like
directory-files, so it's not something you'd feed to expand-file-name in
these situations...

> Btw, stuff like (expand-file-name "foo/~/") already does what you
> want, so the problem is only with the leading '~', and can be avoided
> if we avoid that situation.  IOW, why should this example:
>
>   (expand-file-name "~" "/tmp/")
>      => "/home/larsi"
>
> determine how directory-files-recursively behaves?

expand-file-name's use case is to (basically) concatenate a directory
name and a file name, but it's used instead of concat because nobody
wants to care about whether the directory name has a trailing slash or
not.

(concat "/tmp/" "foo")
=> "/tmp/foo" ; Good

(concat "/tmp" "foo")
=> "/tmpfoo" ; Bad.

(expand-file-name "foo" "/tmp")
=> "/tmp/foo" ; Yay

That's basically the use case for expand-file-name, and using it has
avoided a lot of basic concatenation problems over the years (because
Emacs allows sloppy handling of directory file names in most
situations).

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-09 17:00               ` Lars Ingebrigtsen
@ 2019-07-09 17:23                 ` Eli Zaretskii
  2019-07-09 18:16                   ` Michael Albinus
  2019-07-10 11:55                   ` Lars Ingebrigtsen
  0 siblings, 2 replies; 23+ messages in thread
From: Eli Zaretskii @ 2019-07-09 17:23 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: 36490, erik_hahn

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: 36490@debbugs.gnu.org,  erik_hahn@gmx.de
> Date: Tue, 09 Jul 2019 19:00:13 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Sorry, that's not what I meant to say.  I meant to say how would a
> > Lisp program know whether (expand-file-name "~/") means the home
> > directory or a directory whose name is literally "~"?
> 
> Well, we have documented that in expand-file-name "~/" means the home
> directory, and I have no problems with that.

Documenting a problem doesn't necessarily solve it.  E.g., it is also
documented that you must quote file names with special characters, but
you still raised the objection that the "~" use case makes that "odd".

I'm saying that the "~/" use case is "odd" as well, and for the same
reasons.

> "~/" isn't something you'll ever get from functions like
> directory-files

That's sheer luck, because:

  (file-name-as-directory "~")
    => "~/"

So just running "~" through an innocent API gives you a "magic"
directory name (if you consider "~" not "magic" by itself).  How is
this different from the "odd" use case where one must quote "~" to
avoid its interpretation as the home directory?  Who can guarantee
that some day directory-files-recursively will not want to do
something like the above?  If it does, we will be right back at the
same problem.

I say we should fix this problem in a way that isn't fragile, and
doesn't crucially depend on what the current code does or avoids
doing.

> > Btw, stuff like (expand-file-name "foo/~/") already does what you
> > want, so the problem is only with the leading '~', and can be avoided
> > if we avoid that situation.  IOW, why should this example:
> >
> >   (expand-file-name "~" "/tmp/")
> >      => "/home/larsi"
> >
> > determine how directory-files-recursively behaves?
> 
> expand-file-name's use case is to (basically) concatenate a directory
> name and a file name, but it's used instead of concat because nobody
> wants to care about whether the directory name has a trailing slash or
> not.

Ah, but when the file name begins with a "~", the "concatenation" does
more than what meets the eye.

> That's basically the use case for expand-file-name, and using it has
> avoided a lot of basic concatenation problems over the years (because
> Emacs allows sloppy handling of directory file names in most
> situations).

I think this is a simplification.  It ignores the fact that
expand-file-name interprets ~/, it ignores the fact that it does
arbitrary stuff for "remote" file names, it ignores the fact that on
Windows it prepends the drive letter if there isn't one already, etc.
IOW, expand-file-name is concatenation-like, but it has a few tricks
up its sleeve, and in this case the trick works against us.  We need
to disable that trick to support files and directories whose names
begin with a literal "~".  I see no way around that.





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-09 17:23                 ` Eli Zaretskii
@ 2019-07-09 18:16                   ` Michael Albinus
  2019-07-10 11:55                   ` Lars Ingebrigtsen
  1 sibling, 0 replies; 23+ messages in thread
From: Michael Albinus @ 2019-07-09 18:16 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 36490, Lars Ingebrigtsen, erik_hahn

Eli Zaretskii <eliz@gnu.org> writes:

>> expand-file-name's use case is to (basically) concatenate a directory
>> name and a file name, but it's used instead of concat because nobody
>> wants to care about whether the directory name has a trailing slash or
>> not.
>
> Ah, but when the file name begins with a "~", the "concatenation" does
> more than what meets the eye.
>
>> That's basically the use case for expand-file-name, and using it has
>> avoided a lot of basic concatenation problems over the years (because
>> Emacs allows sloppy handling of directory file names in most
>> situations).
>
> I think this is a simplification.  It ignores the fact that
> expand-file-name interprets ~/, it ignores the fact that it does
> arbitrary stuff for "remote" file names, it ignores the fact that on
> Windows it prepends the drive letter if there isn't one already, etc.
> IOW, expand-file-name is concatenation-like, but it has a few tricks
> up its sleeve, and in this case the trick works against us.  We need
> to disable that trick to support files and directories whose names
> begin with a literal "~".  I see no way around that.

For the records, I second Eli. See also the discussion in bug#16984. And
yes, I believe it makes sense to quote file names (suppress special
meaning of "~") in the loop of directory-files-recursively.

Best regards, Michael.





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-09 16:12       ` Eli Zaretskii
  2019-07-09 16:27         ` Lars Ingebrigtsen
@ 2019-07-09 18:58         ` Basil L. Contovounesios
  2019-07-09 19:19           ` Eli Zaretskii
  1 sibling, 1 reply; 23+ messages in thread
From: Basil L. Contovounesios @ 2019-07-09 18:58 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 36490, Lars Ingebrigtsen, erik_hahn

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Lars Ingebrigtsen <larsi@gnus.org>
>> Cc: erik_hahn@gmx.de,  36490@debbugs.gnu.org
>> Date: Tue, 09 Jul 2019 17:50:49 +0200
>> 
>> > If you want "~" to be interpreted literally, you need to protect it
>> > with "/:".
>> 
>> Sorry; I don't quite follow you here.  The doc string says that "~/" is
>> interpreted specially.  There's no "/" in "~".  :-)
>
> But it does NOT say that "~" will NOT be interpreted specially.

Indeed it explicitly says that "~" will be interpreted specially:

  Second arg DEFAULT-DIRECTORY is directory to start with if NAME is
  relative (does not start with slash or tilde); both the directory name
  and a directory's file name are accepted.

So AIUI (expand-file-name "~") should be equivalent to
(directory-file-name (expand-file-name "~/")).

-- 
Basil





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-09 18:58         ` Basil L. Contovounesios
@ 2019-07-09 19:19           ` Eli Zaretskii
  0 siblings, 0 replies; 23+ messages in thread
From: Eli Zaretskii @ 2019-07-09 19:19 UTC (permalink / raw)
  To: Basil L. Contovounesios; +Cc: 36490, larsi, erik_hahn

> From: "Basil L. Contovounesios" <contovob@tcd.ie>
> Cc: Lars Ingebrigtsen <larsi@gnus.org>,  36490@debbugs.gnu.org,  erik_hahn@gmx.de
> Date: Tue, 09 Jul 2019 19:58:18 +0100
> 
> So AIUI (expand-file-name "~") should be equivalent to
> (directory-file-name (expand-file-name "~/")).

And that's exactly what it does.





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-09 17:23                 ` Eli Zaretskii
  2019-07-09 18:16                   ` Michael Albinus
@ 2019-07-10 11:55                   ` Lars Ingebrigtsen
  2019-07-10 14:56                     ` Eli Zaretskii
  1 sibling, 1 reply; 23+ messages in thread
From: Lars Ingebrigtsen @ 2019-07-10 11:55 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 36490, erik_hahn

Eli Zaretskii <eliz@gnu.org> writes:

>> "~/" isn't something you'll ever get from functions like
>> directory-files
>
> That's sheer luck, because:
>
>   (file-name-as-directory "~")
>     => "~/"
>
> So just running "~" through an innocent API gives you a "magic"
> directory name (if you consider "~" not "magic" by itself).  How is
> this different from the "odd" use case where one must quote "~" to
> avoid its interpretation as the home directory?  Who can guarantee
> that some day directory-files-recursively will not want to do
> something like the above?  If it does, we will be right back at the
> same problem.

Well...  That kinda sounds odd to me.

"~/" is not, and never will be, a valid file name in any OS that Emacs
is going to support from now on.  So having that have a special meaning
in `expand-file-name' is not surprising.  Having "~" do something
special is surprising.

But changing that is probably not going to happen, so how about just
clarifying the documentation in that function to say what "~" means
explicitly instead of the caller having to guess?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-03 18:07 bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~" Erik Hahn
  2019-07-08 20:59 ` Lars Ingebrigtsen
  2019-07-08 21:08 ` Lars Ingebrigtsen
@ 2019-07-10 12:04 ` Lars Ingebrigtsen
  2 siblings, 0 replies; 23+ messages in thread
From: Lars Ingebrigtsen @ 2019-07-10 12:04 UTC (permalink / raw)
  To: Erik Hahn; +Cc: 36490

Erik Hahn <erik_hahn@gmx.de> writes:

> Call (directory-files-recursively DIR ".*") where DIR contains a
> subdirectory named "~" but is not the parent of your home directory. The
> function will recurse into your home directory. This is probably because
> (expand-file-name "~" ...) returns the home directory.

This is now fixed on the trunk.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-10 11:55                   ` Lars Ingebrigtsen
@ 2019-07-10 14:56                     ` Eli Zaretskii
  2019-07-11 15:05                       ` Lars Ingebrigtsen
  0 siblings, 1 reply; 23+ messages in thread
From: Eli Zaretskii @ 2019-07-10 14:56 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: 36490, erik_hahn

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: 36490@debbugs.gnu.org,  erik_hahn@gmx.de
> Date: Wed, 10 Jul 2019 13:55:33 +0200
> 
> >   (file-name-as-directory "~")
> >     => "~/"
> >
> > So just running "~" through an innocent API gives you a "magic"
> > directory name (if you consider "~" not "magic" by itself).  How is
> > this different from the "odd" use case where one must quote "~" to
> > avoid its interpretation as the home directory?  Who can guarantee
> > that some day directory-files-recursively will not want to do
> > something like the above?  If it does, we will be right back at the
> > same problem.
> 
> Well...  That kinda sounds odd to me.
> 
> "~/" is not, and never will be, a valid file name in any OS that Emacs
> is going to support from now on.

I don't think I follow you.  "~" is a perfectly valid name of a
directory, and Emacs does support such names in general.  So I don't
think I understand why you are saying this will not be a valid file
name.  What did I miss?

> But changing that is probably not going to happen, so how about just
> clarifying the documentation in that function to say what "~" means
> explicitly instead of the caller having to guess?

I don't object to documenting this.





^ permalink raw reply	[flat|nested] 23+ messages in thread

* bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~"
  2019-07-10 14:56                     ` Eli Zaretskii
@ 2019-07-11 15:05                       ` Lars Ingebrigtsen
  0 siblings, 0 replies; 23+ messages in thread
From: Lars Ingebrigtsen @ 2019-07-11 15:05 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 36490, erik_hahn

Eli Zaretskii <eliz@gnu.org> writes:

>> "~/" is not, and never will be, a valid file name in any OS that Emacs
>> is going to support from now on.
>
> I don't think I follow you.  "~" is a perfectly valid name of a
> directory, and Emacs does support such names in general.  So I don't
> think I understand why you are saying this will not be a valid file
> name.  What did I miss?

"~" and "~/" are not the same thing in any OS that Emacs is currently
supporting.  The first is a file name (including directories, which are
just files), while the second is a file name with a slash after it.

Inside Emacs, though, things are different.

>> But changing that is probably not going to happen, so how about just
>> clarifying the documentation in that function to say what "~" means
>> explicitly instead of the caller having to guess?
>
> I don't object to documenting this.

OK; I'll do so.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no





^ permalink raw reply	[flat|nested] 23+ messages in thread

end of thread, other threads:[~2019-07-11 15:05 UTC | newest]

Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-03 18:07 bug#36490: 26.1; directory-files-recursively breaks when it encounters a directory named "~" Erik Hahn
2019-07-08 20:59 ` Lars Ingebrigtsen
2019-07-08 21:08 ` Lars Ingebrigtsen
2019-07-08 21:30   ` Erik Hahn
2019-07-08 21:56     ` Lars Ingebrigtsen
2019-07-09 15:39     ` Eli Zaretskii
2019-07-09 15:35   ` Eli Zaretskii
2019-07-09 15:50     ` Lars Ingebrigtsen
2019-07-09 16:12       ` Eli Zaretskii
2019-07-09 16:27         ` Lars Ingebrigtsen
2019-07-09 16:44           ` Eli Zaretskii
2019-07-09 16:50             ` Lars Ingebrigtsen
2019-07-09 16:52             ` Eli Zaretskii
2019-07-09 17:00               ` Lars Ingebrigtsen
2019-07-09 17:23                 ` Eli Zaretskii
2019-07-09 18:16                   ` Michael Albinus
2019-07-10 11:55                   ` Lars Ingebrigtsen
2019-07-10 14:56                     ` Eli Zaretskii
2019-07-11 15:05                       ` Lars Ingebrigtsen
2019-07-09 18:58         ` Basil L. Contovounesios
2019-07-09 19:19           ` Eli Zaretskii
2019-07-09 16:17       ` Andreas Schwab
2019-07-10 12:04 ` Lars Ingebrigtsen

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).