bug#31305: 27.0.50; Symlinks recognized as dirs

bug#31305: 27.0.50; Symlinks recognized as dirs

[Jimmy Yuen Ho Wong]

Reproduction:

0. Get a Mac
1. Download the most recent Emacs 26 nightly from 
https://emacsformacosx.com/builds
2. Symlink a file in a dir to another file in another dir
3. C-x C-f some prefix to the symlink
4. TAB for completion
5. Completion completed the path with an extra /
6. RET
7. Emacs opens up the dir as a new file in a fundamental mode buffer

Expectation:
Completion should not complete the symlink as a dir, opening the dir as
a regular file instead of dired should also not be possible.


In GNU Emacs 27.0.50 (build 1, x86_64-apple-darwin13.4.0, NS 
appkit-1265.21 Version 10.9.5 (Build 13F1911))
of 2018-04-18 built on builder10-9.porkrind.org
Windowing system distributor 'Apple', version 10.3.1561
System Description: Mac OS X 10.13.4

Recent messages:
Warning: arch-dependent data dir 
’/Users/build/workspace/Emacs-Multi-Build/label/mavericks/emacs-source/nextstep/Emacs.app/Contents/MacOS/libexec/’: 
No such file or directory

For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
'configure --with-ns '--enable-locallisppath=/Library/Application
Support/Emacs/${version}/site-lisp:/Library/Application
Support/Emacs/site-lisp' --with-modules'

Configured features:
NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS MODULES THREADS

Important settings:
value of $LANG: en_GB.UTF-8
locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
tooltip-mode: t
global-eldoc-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
tool-bar-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
buffer-read-only: t
line-number-mode: t
transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv dired dired-loaddefs format-spec rfc822 mml
easymenu mml-sec password-cache epa derived epg epg-config gnus-util
rmail rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader cl-loaddefs cl-lib sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils time-date elec-pair
tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type
mwheel term/ns-win ns-win ucs-normalize mule-util term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow isearch timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core term/tty-colors frame cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote kqueue cocoa ns
multi-tty make-network-process emacs)

Memory information:
((conses 16 204043 8971)
(symbols 48 19821 1)
(miscs 40 69 175)
(strings 32 28749 2003)
(string-bytes 1 763036)
(vectors 16 35111)
(vector-slots 8 715296 15296)
(floats 8 47 69)
(intervals 56 206 0)
(buffers 992 12))

bug#31305: 27.0.50; Symlinks recognized as dirs

[Jimmy Yuen Ho Wong]

Correction:

This bug was submitted from a Emacs 27 nightly build, but the same
problem occurs in Emacs 26 RC1 also.

bug#31305: 27.0.50; Symlinks recognized as dirs

[Alan Third]

On Sat, Apr 28, 2018 at 09:44:38PM +0100, Jimmy Yuen Ho Wong wrote:
> Reproduction:
> 
> 0. Get a Mac
> 1. Download the most recent Emacs 26 nightly from
> https://emacsformacosx.com/builds
> 2. Symlink a file in a dir to another file in another dir
> 3. C-x C-f some prefix to the symlink
> 4. TAB for completion
> 5. Completion completed the path with an extra /
> 6. RET
> 7. Emacs opens up the dir as a new file in a fundamental mode buffer
> 
> Expectation:
> Completion should not complete the symlink as a dir, opening the dir as
> a regular file instead of dired should also not be possible.

I could have sworn we fixed this, but I can’t find the previous bug
report.

FWIW, I can’t reproduce it.
-- 
Alan Third

bug#31305: 27.0.50; Symlinks recognized as dirs

[Noam Postavsky]

Alan Third <alan@idiocy.org> writes:

> I could have sworn we fixed this, but I can’t find the previous bug
> report.

Looks similar to Bug#28865 "broken symlink behaviour in read-file-name
on OS X (High Sierra)".

bug#31305: 27.0.50; Symlinks recognized as dirs

[Alan Third]

On Sun, Apr 29, 2018 at 04:25:08PM -0400, Noam Postavsky wrote:
> Alan Third <alan@idiocy.org> writes:
> 
> > I could have sworn we fixed this, but I can’t find the previous bug
> > report.
> 
> Looks similar to Bug#28865 "broken symlink behaviour in read-file-name
> on OS X (High Sierra)".

That’s it exactly. Thanks Noam!

With further testing I’ve discovered that although I can’t replicate
it with my locally built version, I can with an emacsformacosx.com
build.

I guess that means either Paul’s fix doesn’t work on macOS 10.9, or
David needs to update something in his build tree.
-- 
Alan Third

bug#31305: 27.0.50; Symlinks recognized as dirs

[Paul Eggert]

> I guess that means either Paul’s fix doesn’t work on macOS 10.9, or
> David needs to update something in his build tree.


Possibly the Gnulib workaround for the macOS faccessat bug does not work 
in older macOS versions. For example, the Gnulib workaround calls the 
function 'access' on platforms lacking faccessat, and faccessat is 
missing and 'access ("foo/", F_OK)' ignores the trailing slash in older 
macOS versions, then that could explain the problem.

Suppose my guess is right. Then, if this problem occurs because the 
emacsformacosx.com build is for OS X 10.9 or earlier, then a simple fix 
is to have emacsformacosx.com build for OS X 10.10 or later, because 
Emacs can't reasonably support OS versions that Apple itself does not 
support <https://lists.gnu.org/r/emacs-devel/2018-03/msg00798.html>. On 
the other hand, if the problem occurs on OS X 10.10 or later, then 
someone should hack on the Gnulib workaround so as to port the 
workaround to OS X 10.10. I can volunteer to do the hacking in my spare 
time, but I don't have easy access to OS X 10.10 so someone else would 
need to test it.

bug#31305: 27.0.50; Symlinks recognized as dirs

[David Caldwell]

[-- Attachment #1.1: Type: text/plain, Size: 1687 bytes --]

On 5/16/18 11:37 AM, Paul Eggert wrote:
>> I guess that means either Paul’s fix doesn’t work on macOS 10.9, or
>> David needs to update something in his build tree.
> 
> 
> Possibly the Gnulib workaround for the macOS faccessat bug does not work
> in older macOS versions. For example, the Gnulib workaround calls the
> function 'access' on platforms lacking faccessat, and faccessat is
> missing and 'access ("foo/", F_OK)' ignores the trailing slash in older
> macOS versions, then that could explain the problem.
> 
> Suppose my guess is right. Then, if this problem occurs because the
> emacsformacosx.com build is for OS X 10.9 or earlier,

For the record, currently I build on 10.6, 10.8 and 10.9. I'm very very
close to killing the 10.6 build.

> then a simple fix
> is to have emacsformacosx.com build for OS X 10.10 or later,

This is very easy to do, I mostly haven't already because I didn't see
any compelling reason to build on later machines (I'm not aware of any
10.10-10.13 dependent features the same way that say full screen mode
only works in 10.9 and later).

> because
> Emacs can't reasonably support OS versions that Apple itself does not
> support <https://lists.gnu.org/r/emacs-devel/2018-03/msg00798.html>. On
> the other hand, if the problem occurs on OS X 10.10 or later, then
> someone should hack on the Gnulib workaround so as to port the
> workaround to OS X 10.10. I can volunteer to do the hacking in my spare
> time, but I don't have easy access to OS X 10.10 so someone else would
> need to test it.

I have a 10.10 VM and could probably test it. I can't promise I'll be
super available though.

-David




[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#31305: 27.0.50; Symlinks recognized as dirs

[Paul Eggert]

On 05/16/2018 05:29 PM, David Caldwell wrote:
>> then a simple fix
>> is to have emacsformacosx.com build for OS X 10.10 or later,
> This is very easy to do, I mostly haven't already because I didn't see
> any compelling reason to build on later machines (I'm not aware of any
> 10.10-10.13 dependent features the same way that say full screen mode
> only works in 10.9 and later).

Thanks for the info. When you have a moment could you please try 
building on OS X 10.10 and then testing the result? I assume OS X 10.10 
has a file /etc/passwd so you could run something like this:

ln -s /etc/passwd .
src/emacs -Q -batch -eval '(message "%S" (file-name-completion "pass" "."))'

This should output "passwd" (assuming that the build directory has no 
other entry beginning with "pass"), but if the bug I'm hypothesizing is 
triggered, than it will output "passwd/".

bug#31305: 27.0.50; Symlinks recognized as dirs

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 232 bytes --]

One more thing. This reminds me of Bug#30350, so if this new bug still 
occurs under OS X 10.10 please try the attached patch. If this works 
around the OS X 10.10 bug I can polish it up and install it properly 
into Emacs. Thanks.

[-- Attachment #2: macos.diff --]
[-- Type: text/x-patch, Size: 544 bytes --]

diff --git a/src/dired.c b/src/dired.c
index c446223a0b..ed12fce5f8 100644
--- a/src/dired.c
+++ b/src/dired.c
@@ -814,9 +814,9 @@ static bool
 file_name_completion_dirp (int fd, struct dirent *dp, ptrdiff_t len)
 {
   USE_SAFE_ALLOCA;
-  char *subdir_name = SAFE_ALLOCA (len + 2);
+  char *subdir_name = SAFE_ALLOCA (len + 4);
   memcpy (subdir_name, dp->d_name, len);
-  strcpy (subdir_name + len, "/");
+  strcpy (subdir_name + len, "/./");
   bool dirp = faccessat (fd, subdir_name, F_OK, AT_EACCESS) == 0;
   SAFE_FREE ();
   return dirp;

bug#31305: 27.0.50; Symlinks recognized as dirs

[David Caldwell]

[-- Attachment #1.1: Type: text/plain, Size: 1131 bytes --]

On 5/16/18 6:09 PM, Paul Eggert wrote:
> On 05/16/2018 05:29 PM, David Caldwell wrote:
>>> then a simple fix
>>> is to have emacsformacosx.com build for OS X 10.10 or later,
>> This is very easy to do, I mostly haven't already because I didn't see
>> any compelling reason to build on later machines (I'm not aware of any
>> 10.10-10.13 dependent features the same way that say full screen mode
>> only works in 10.9 and later).
> 
> Thanks for the info. When you have a moment could you please try
> building on OS X 10.10 and then testing the result? I assume OS X 10.10
> has a file /etc/passwd so you could run something like this:
> 
> ln -s /etc/passwd .
> src/emacs -Q -batch -eval '(message "%S" (file-name-completion "pass"
> "."))'

Hi Paul,

I built from 60ff8101449eea3a5ca4961299501efd83d011bd.

$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.10.5
BuildVersion:	14F2511
$ src/emacs -Q -batch -eval '(message "%S" (file-name-completion "pass"
"."))'
"passwd"

Since this doesn't seem to match your hypothesis, do you still want me
to try it with the "/./" patch you sent?

-David


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#31305: 27.0.50; Symlinks recognized as dirs

[Paul Eggert]

[-- Attachment #1: Type: text/plain, Size: 707 bytes --]

On 05/18/2018 03:22 AM, David Caldwell wrote:
> $ src/emacs -Q -batch -eval '(message "%S" (file-name-completion "pass"
> "."))'
> "passwd"

Thanks for checking that it works on OS X 10.10.

> Since this doesn't seem to match your hypothesis, do you still want me
> to try it with the "/./" patch you sent?

I wouldn't bother. If the bug is fixed in OS X 10.10 and Apple itself 
doesn't support older systems, then we shouldn't need to worry about 
this bug upstream. I suggest building Emacs for OS X 10.10, recommending 
10.10 (or later) builds to users, and warning that there may be glitches 
if people use builds for older systems. To document this I installed the 
attached into the emacs-26 branch.


[-- Attachment #2: 0001-etc-PROBLEMS-Document-Bug-31305.patch --]
[-- Type: text/x-patch, Size: 1034 bytes --]

From ad74c608988dc855f74e45ac2f2c2e5082019719 Mon Sep 17 00:00:00 2001
From: Paul Eggert <eggert@cs.ucla.edu>
Date: Fri, 18 May 2018 09:24:04 -0700
Subject: [PATCH] * etc/PROBLEMS: Document Bug#31305.

---
 etc/PROBLEMS | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/etc/PROBLEMS b/etc/PROBLEMS
index 1f7fe00bd6..09027332d8 100644
--- a/etc/PROBLEMS
+++ b/etc/PROBLEMS
@@ -3213,6 +3213,15 @@ them to DOS 8+3 limits.  To be useful on NT, the MSDOS port of Emacs
 must be unzipped by a DOS utility, so that long file names are
 properly truncated.
 
+** Apple Macintosh operating systems
+
+*** OS X 10.9 and earlier: symlinks autocomplete as directories
+
+Autocompleting the name of a symbolic link incorrectly appends "/".
+Building and running Emacs on OS X 10.10 (or later) fixes the problem.
+Older operating systems are no longer supported by Apple.
+https://bugs.gnu.org/31305
+
 ** Archaic window managers and toolkits
 
 *** Open Look: Under Open Look, the Emacs window disappears when you type M-q.
-- 
2.17.0

bug#31305: 27.0.50; Symlinks recognized as dirs

[David Caldwell]

[-- Attachment #1.1: Type: text/plain, Size: 1054 bytes --]

On 5/18/18 10:16 AM, Paul Eggert wrote:
> On 05/18/2018 03:22 AM, David Caldwell wrote:
>> $ src/emacs -Q -batch -eval '(message "%S" (file-name-completion "pass"
>> "."))'
>> "passwd"
> 
> Thanks for checking that it works on OS X 10.10.
> 
>> Since this doesn't seem to match your hypothesis, do you still want me
>> to try it with the "/./" patch you sent?
> 
> I wouldn't bother. If the bug is fixed in OS X 10.10 and Apple itself
> doesn't support older systems, then we shouldn't need to worry about
> this bug upstream. I suggest building Emacs for OS X 10.10, recommending
> 10.10 (or later) builds to users, and warning that there may be glitches
> if people use builds for older systems. To document this I installed the
> attached into the emacs-26 branch.

Ok. So I'll start building on at least 10.10. I think Alan made some
changes so that we can actually build on the latest OS and set it so
it'll run on older versions and maybe I start doing that soon since it
should vastly simplify the build process.

-David


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#31305: 27.0.50; Symlinks recognized as dirs

[Paul Eggert]

On 05/18/2018 04:17 PM, David Caldwell wrote:
> I think Alan made some  > changes so that we can actually build on the latest OS and set it so 
 > it'll run on older versions
If that works it'd be great. Although I'd be surprised if it fixes this 
problem, macOS is full of surprises....

bug#31305: 27.0.50; Symlinks recognized as dirs

[Stefan Kangas]

Paul Eggert <eggert@cs.ucla.edu> writes:

> On 05/18/2018 03:22 AM, David Caldwell wrote:
>> $ src/emacs -Q -batch -eval '(message "%S" (file-name-completion "pass"
>> "."))'
>> "passwd"
>
> Thanks for checking that it works on OS X 10.10.
>
>> Since this doesn't seem to match your hypothesis, do you still want me
>> to try it with the "/./" patch you sent?
>
> I wouldn't bother. If the bug is fixed in OS X 10.10 and Apple itself doesn't
> support older systems, then we shouldn't need to worry about this bug
> upstream. I suggest building Emacs for OS X 10.10, recommending 10.10 (or later)
> builds to users, and warning that there may be glitches if people use builds for
> older systems. To document this I installed the attached into the emacs-26
> branch.
>
>>From ad74c608988dc855f74e45ac2f2c2e5082019719 Mon Sep 17 00:00:00 2001
> From: Paul Eggert <eggert@cs.ucla.edu>
> Date: Fri, 18 May 2018 09:24:04 -0700
> Subject: [PATCH] * etc/PROBLEMS: Document Bug#31305.
>
> ---
>  etc/PROBLEMS | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/etc/PROBLEMS b/etc/PROBLEMS
> index 1f7fe00bd6..09027332d8 100644
> --- a/etc/PROBLEMS
> +++ b/etc/PROBLEMS
> @@ -3213,6 +3213,15 @@ them to DOS 8+3 limits.  To be useful on NT, the MSDOS port of Emacs
>  must be unzipped by a DOS utility, so that long file names are
>  properly truncated.
>
> +** Apple Macintosh operating systems
> +
> +*** OS X 10.9 and earlier: symlinks autocomplete as directories
> +
> +Autocompleting the name of a symbolic link incorrectly appends "/".
> +Building and running Emacs on OS X 10.10 (or later) fixes the problem.
> +Older operating systems are no longer supported by Apple.
> +https://bugs.gnu.org/31305
> +
>  ** Archaic window managers and toolkits
>
>  *** Open Look: Under Open Look, the Emacs window disappears when you type M-q.

Let's see if I read this discussion correctly:

1. We don't support macOS older than 10.10.  (Perhaps that should be
   10.11 once 10.15 is released in October?)

2. This bug doesn't manifest itself on macOS 10.10 or later.

So is there anything left to do here?  If the above two points are true,
then I guess this should be closed as wontfix?

Best regards,
Stefan Kangas

bug#31305: 27.0.50; Symlinks recognized as dirs

[Alan Third]

On Sat, Sep 28, 2019 at 08:10:04PM +0200, Stefan Kangas wrote:
> 1. We don't support macOS older than 10.10.  (Perhaps that should be
>    10.11 once 10.15 is released in October?)

As far as I’m aware we support back to 10.6. Apple only support the
three most recent versions, so currently 10.12, 10.13 and 10.14.

There has been some discussion around dropping support for older macOS
versions, but we still have a number of active users and Emacs
developers running 10.6.

-- 
Alan Third

bug#31305: 27.0.50; Symlinks recognized as dirs

[Stefan Kangas]

Alan Third <alan@idiocy.org> writes:

> On Sat, Sep 28, 2019 at 08:10:04PM +0200, Stefan Kangas wrote:
> > 1. We don't support macOS older than 10.10.  (Perhaps that should be
> >    10.11 once 10.15 is released in October?)
>
> As far as I’m aware we support back to 10.6. Apple only support the
> three most recent versions, so currently 10.12, 10.13 and 10.14.
>
> There has been some discussion around dropping support for older macOS
> versions, but we still have a number of active users and Emacs
> developers running 10.6.

Interesting.  In the context of this bug, and given Paul Eggert's
comments earlier, do we still want to fix this?  It's been documented
in etc/PROBLEMS and is fixed by an upgrade to 10.10.

IMHO, this bug is so minor that it's not worth spending time on at
this point.  Does anyone object to closing this as wontfix?  Is that
against our policy since we still technically support 10.6 through
10.9?

Best regards,
Stefan Kangas

bug#31305: 27.0.50; Symlinks recognized as dirs

[Paul Eggert]

On 10/2/19 5:00 AM, Stefan Kangas wrote:
> Does anyone object to closing this as wontfix?

No, and that sounds like the right thing to do. Tagged wontfix and closing.