bug#64102: 28.2; fails to find pathname with a sub dir as symlink and with parent dir ('..')

bug#64102: 28.2; fails to find pathname with a sub dir as symlink and with parent dir ('..')

[Jacob Burckhardt via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Reproduce the bug by running this:

mkdir -p /tmp/usr2/lib/gcc/x86_64-linux-gnu/12
mkdir /tmp/usr2/include
echo symbolic link test > /tmp/usr2/include/test.h
\ln -s usr2/lib /tmp/lib2
cat /tmp/lib2/gcc/x86_64-linux-gnu/12/../../../../include/test.h
emacs -q /tmp/lib2/gcc/x86_64-linux-gnu/12/../../../../include/test.h

Emacs failed to display the content of that file in a buffer.  Since the above cat command shows the contents, Emacs should be able to show it as well.  The following bug is similar and includes some explanations that also apply to my bug.

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=8035

Note that the following code succeeds.  You might consider using code like this to fix the bug:

(find-file (file-truename "/tmp/lib2/gcc/x86_64-linux-gnu/12/../../../../include/test.h"))




In GNU Emacs 28.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.37, cairo version 1.16.0)
 of 2023-05-13, modified by Debian built on x86-ubc-01
Windowing system distributor 'The X.Org Foundation', version 11.0.12101007
System Description: Debian GNU/Linux 12 (bookworm)

Configured using:
 'configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/libexec
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --with-libsystemd --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/28.2/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/28.2/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils
 --with-native-compilation --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/libexec
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --with-libsystemd --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/28.2/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/28.2/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils
 --with-native-compilation --with-cairo --with-x=yes
 --with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g -O2
 -ffile-prefix-map=/build/emacs-mPr7Vr/emacs-28.2+1=. -fstack-protector-strong
 -Wformat -Werror=format-security -Wall' 'CPPFLAGS=-Wdate-time
 -D_FORTIFY_SOURCE=2' LDFLAGS=-Wl,-z,relro'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES
NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF
TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM GTK3 ZLIB

Important settings:
  locale-coding-system: nil

Major mode: Fundamental

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t

Load-path shadows:
/usr/share/emacs/site-lisp/llvm/llvm-mode hides /usr/share/emacs/site-lisp/llvm-11/llvm-mode
/usr/share/emacs/site-lisp/llvm/emacs hides /usr/share/emacs/site-lisp/llvm-11/emacs
/usr/share/emacs/site-lisp/llvm/tablegen-mode hides /usr/share/emacs/site-lisp/llvm-11/tablegen-mode
/usr/share/emacs/site-lisp/vm/vm-license hides /usr/share/emacs/site-lisp/vm.in/vm-license
/usr/share/emacs/site-lisp/vm/vm-user hides /usr/share/emacs/site-lisp/vm.in/vm-user
/usr/share/emacs/site-lisp/vm/vm-macro hides /usr/share/emacs/site-lisp/vm.in/vm-macro
/usr/share/emacs/site-lisp/vm/vm-motion hides /usr/share/emacs/site-lisp/vm.in/vm-motion
/usr/share/emacs/site-lisp/vm/vm-menu hides /usr/share/emacs/site-lisp/vm.in/vm-menu
/usr/share/emacs/site-lisp/vm/vm-w3 hides /usr/share/emacs/site-lisp/vm.in/vm-w3
/usr/share/emacs/site-lisp/vm/vm-thread hides /usr/share/emacs/site-lisp/vm.in/vm-thread
/usr/share/emacs/site-lisp/vm/vm-summary-faces hides /usr/share/emacs/site-lisp/vm.in/vm-summary-faces
/usr/share/emacs/site-lisp/vm/vm-minibuf hides /usr/share/emacs/site-lisp/vm.in/vm-minibuf
/usr/share/emacs/site-lisp/vm/vm-summary hides /usr/share/emacs/site-lisp/vm.in/vm-summary
/usr/share/emacs/site-lisp/vm/vcard hides /usr/share/emacs/site-lisp/vm.in/vcard
/usr/share/emacs/site-lisp/vm/vm-cus-load hides /usr/share/emacs/site-lisp/vm.in/vm-cus-load
/usr/share/emacs/site-lisp/vm/vm-init hides /usr/share/emacs/site-lisp/vm.in/vm-init
/usr/share/emacs/site-lisp/vm/vm-pgg hides /usr/share/emacs/site-lisp/vm.in/vm-pgg
/usr/share/emacs/site-lisp/vm/vm-version hides /usr/share/emacs/site-lisp/vm.in/vm-version
/usr/share/emacs/site-lisp/vm/vm-rfaddons hides /usr/share/emacs/site-lisp/vm.in/vm-rfaddons
/usr/share/emacs/site-lisp/vm/vm-startup hides /usr/share/emacs/site-lisp/vm.in/vm-startup
/usr/share/emacs/site-lisp/vm/vm hides /usr/share/emacs/site-lisp/vm.in/vm
/usr/share/emacs/site-lisp/vm/vm-grepmail hides /usr/share/emacs/site-lisp/vm.in/vm-grepmail
/usr/share/emacs/site-lisp/vm/vm-pcrisis hides /usr/share/emacs/site-lisp/vm.in/vm-pcrisis
/usr/share/emacs/site-lisp/vm/vm-serial hides /usr/share/emacs/site-lisp/vm.in/vm-serial
/usr/share/emacs/site-lisp/vm/vm-undo hides /usr/share/emacs/site-lisp/vm.in/vm-undo
/usr/share/emacs/site-lisp/vm/vm-w3m hides /usr/share/emacs/site-lisp/vm.in/vm-w3m
/usr/share/emacs/site-lisp/vm/vm-pine hides /usr/share/emacs/site-lisp/vm.in/vm-pine
/usr/share/emacs/site-lisp/vm/vm-build hides /usr/share/emacs/site-lisp/vm.in/vm-build
/usr/share/emacs/site-lisp/vm/vm-toolbar hides /usr/share/emacs/site-lisp/vm.in/vm-toolbar
/usr/share/emacs/site-lisp/vm/vm-avirtual hides /usr/share/emacs/site-lisp/vm.in/vm-avirtual
/usr/share/emacs/site-lisp/vm/u-vm-color hides /usr/share/emacs/site-lisp/vm.in/u-vm-color
/usr/share/emacs/site-lisp/vm/vm-mark hides /usr/share/emacs/site-lisp/vm.in/vm-mark
/usr/share/emacs/site-lisp/vm/vm-dired hides /usr/share/emacs/site-lisp/vm.in/vm-dired
/usr/share/emacs/site-lisp/vm/vm-crypto hides /usr/share/emacs/site-lisp/vm.in/vm-crypto
/usr/share/emacs/site-lisp/vm/vm-autoload hides /usr/share/emacs/site-lisp/vm.in/vm-autoload
/usr/share/emacs/site-lisp/vm/vm-page hides /usr/share/emacs/site-lisp/vm.in/vm-page
/usr/share/emacs/site-lisp/vm/vm-biff hides /usr/share/emacs/site-lisp/vm.in/vm-biff
/usr/share/emacs/site-lisp/vm/vm-vars hides /usr/share/emacs/site-lisp/vm.in/vm-vars
/usr/share/emacs/site-lisp/vm/vm-message-history hides /usr/share/emacs/site-lisp/vm.in/vm-message-history
/usr/share/emacs/site-lisp/vm/vm-mime hides /usr/share/emacs/site-lisp/vm.in/vm-mime
/usr/share/emacs/site-lisp/vm/vm-vcard hides /usr/share/emacs/site-lisp/vm.in/vm-vcard
/usr/share/emacs/site-lisp/vm/vm-mouse hides /usr/share/emacs/site-lisp/vm.in/vm-mouse
/usr/share/emacs/site-lisp/vm/vm-message hides /usr/share/emacs/site-lisp/vm.in/vm-message
/usr/share/emacs/site-lisp/vm/vm-delete hides /usr/share/emacs/site-lisp/vm.in/vm-delete
/usr/share/emacs/site-lisp/vm/tapestry hides /usr/share/emacs/site-lisp/vm.in/tapestry
/usr/share/emacs/site-lisp/vm/vm-reply hides /usr/share/emacs/site-lisp/vm.in/vm-reply
/usr/share/emacs/site-lisp/vm/vm-edit hides /usr/share/emacs/site-lisp/vm.in/vm-edit
/usr/share/emacs/site-lisp/vm/vm-digest hides /usr/share/emacs/site-lisp/vm.in/vm-digest
/usr/share/emacs/site-lisp/vm/vm-misc hides /usr/share/emacs/site-lisp/vm.in/vm-misc
/usr/share/emacs/site-lisp/vm/vm-search hides /usr/share/emacs/site-lisp/vm.in/vm-search
/usr/share/emacs/site-lisp/vm/vm-imap hides /usr/share/emacs/site-lisp/vm.in/vm-imap
/usr/share/emacs/site-lisp/vm/vm-sort hides /usr/share/emacs/site-lisp/vm.in/vm-sort
/usr/share/emacs/site-lisp/vm/vm-window hides /usr/share/emacs/site-lisp/vm.in/vm-window
/usr/share/emacs/site-lisp/vm/vm-ps-print hides /usr/share/emacs/site-lisp/vm.in/vm-ps-print
/usr/share/emacs/site-lisp/vm/vm-pop hides /usr/share/emacs/site-lisp/vm.in/vm-pop
/usr/share/emacs/site-lisp/vm/vm-save hides /usr/share/emacs/site-lisp/vm.in/vm-save
/usr/share/emacs/site-lisp/vm/vm-folder hides /usr/share/emacs/site-lisp/vm.in/vm-folder
/usr/share/emacs/site-lisp/vm/vm-virtual hides /usr/share/emacs/site-lisp/vm.in/vm-virtual
/usr/share/emacs/site-lisp/elpa/dash-2.17.0/dash hides /usr/share/emacs/site-lisp/elpa/dash-2.19.1/dash
/usr/share/emacs/site-lisp/elpa/dash-2.17.0/dash-autoloads hides /usr/share/emacs/site-lisp/elpa/dash-2.19.1/dash-autoloads
/usr/share/emacs/site-lisp/elpa/dash-2.17.0/dash-pkg hides /usr/share/emacs/site-lisp/elpa/dash-2.19.1/dash-pkg
/usr/share/emacs/site-lisp/elpa/seq-2.22/seq hides /usr/share/emacs/site-lisp/elpa/seq-2.23/seq
/usr/share/emacs/site-lisp/elpa/seq-2.22/seq-24 hides /usr/share/emacs/site-lisp/elpa/seq-2.23/seq-24
/usr/share/emacs/site-lisp/elpa/seq-2.22/seq-25 hides /usr/share/emacs/site-lisp/elpa/seq-2.23/seq-25
/usr/share/emacs/site-lisp/elpa/seq-2.22/seq-autoloads hides /usr/share/emacs/site-lisp/elpa/seq-2.23/seq-autoloads
/usr/share/emacs/site-lisp/elpa/seq-2.22/seq-pkg hides /usr/share/emacs/site-lisp/elpa/seq-2.23/seq-pkg
/usr/share/emacs/site-lisp/elpa/slime-2.26.1/slime-autoloads hides /usr/share/emacs/site-lisp/elpa/slime-2.27/slime-autoloads
/usr/share/emacs/site-lisp/elpa/slime-2.26.1/slime hides /usr/share/emacs/site-lisp/elpa/slime-2.27/slime
/usr/share/emacs/site-lisp/elpa/slime-2.26.1/slime-tests hides /usr/share/emacs/site-lisp/elpa/slime-2.27/slime-tests
/usr/share/emacs/site-lisp/elpa/slime-2.26.1/slime-pkg hides /usr/share/emacs/site-lisp/elpa/slime-2.27/slime-pkg
/usr/share/emacs/site-lisp/elpa/dash-2.17.0/dash hides /usr/share/emacs/site-lisp/elpa-src/dash-2.19.1/dash
/usr/share/emacs/site-lisp/elpa/dash-2.17.0/dash-autoloads hides /usr/share/emacs/site-lisp/elpa-src/dash-2.19.1/dash-autoloads
/usr/share/emacs/site-lisp/elpa/dash-2.17.0/dash-pkg hides /usr/share/emacs/site-lisp/elpa-src/dash-2.19.1/dash-pkg
/usr/share/emacs/site-lisp/elpa/epl-0.9/epl-pkg hides /usr/share/emacs/site-lisp/elpa-src/epl-0.9/epl-pkg
/usr/share/emacs/site-lisp/elpa/epl-0.9/epl-autoloads hides /usr/share/emacs/site-lisp/elpa-src/epl-0.9/epl-autoloads
/usr/share/emacs/site-lisp/elpa/epl-0.9/epl hides /usr/share/emacs/site-lisp/elpa-src/epl-0.9/epl
/usr/share/emacs/site-lisp/elpa/flycheck-32snapshot/flycheck-ert hides /usr/share/emacs/site-lisp/elpa-src/flycheck-32snapshot/flycheck-ert
/usr/share/emacs/site-lisp/elpa/flycheck-32snapshot/flycheck-autoloads hides /usr/share/emacs/site-lisp/elpa-src/flycheck-32snapshot/flycheck-autoloads
/usr/share/emacs/site-lisp/elpa/flycheck-32snapshot/flycheck hides /usr/share/emacs/site-lisp/elpa-src/flycheck-32snapshot/flycheck
/usr/share/emacs/site-lisp/elpa/flycheck-32snapshot/flycheck-pkg hides /usr/share/emacs/site-lisp/elpa-src/flycheck-32snapshot/flycheck-pkg
/usr/share/emacs/site-lisp/elpa/flycheck-32snapshot/flycheck-buttercup hides /usr/share/emacs/site-lisp/elpa-src/flycheck-32snapshot/flycheck-buttercup
/usr/share/emacs/site-lisp/elpa/let-alist-1.0.6/let-alist-pkg hides /usr/share/emacs/site-lisp/elpa-src/let-alist-1.0.6/let-alist-pkg
/usr/share/emacs/site-lisp/elpa/let-alist-1.0.6/let-alist hides /usr/share/emacs/site-lisp/elpa-src/let-alist-1.0.6/let-alist
/usr/share/emacs/site-lisp/elpa/let-alist-1.0.6/let-alist-autoloads hides /usr/share/emacs/site-lisp/elpa-src/let-alist-1.0.6/let-alist-autoloads
/usr/share/emacs/site-lisp/elpa/pkg-info-0.6/pkg-info-autoloads hides /usr/share/emacs/site-lisp/elpa-src/pkg-info-0.6/pkg-info-autoloads
/usr/share/emacs/site-lisp/elpa/pkg-info-0.6/pkg-info hides /usr/share/emacs/site-lisp/elpa-src/pkg-info-0.6/pkg-info
/usr/share/emacs/site-lisp/elpa/pkg-info-0.6/pkg-info-pkg hides /usr/share/emacs/site-lisp/elpa-src/pkg-info-0.6/pkg-info-pkg
/usr/share/emacs/site-lisp/elpa/seq-2.22/seq hides /usr/share/emacs/site-lisp/elpa-src/seq-2.23/seq
/usr/share/emacs/site-lisp/elpa/seq-2.22/seq-25 hides /usr/share/emacs/site-lisp/elpa-src/seq-2.23/seq-25
/usr/share/emacs/site-lisp/elpa/seq-2.22/seq-24 hides /usr/share/emacs/site-lisp/elpa-src/seq-2.23/seq-24
/usr/share/emacs/site-lisp/elpa/seq-2.22/seq-autoloads hides /usr/share/emacs/site-lisp/elpa-src/seq-2.23/seq-autoloads
/usr/share/emacs/site-lisp/elpa/seq-2.22/seq-pkg hides /usr/share/emacs/site-lisp/elpa-src/seq-2.23/seq-pkg
/usr/share/emacs/site-lisp/elpa/slime-2.26.1/slime-autoloads hides /usr/share/emacs/site-lisp/elpa-src/slime-2.27/slime-autoloads
/usr/share/emacs/site-lisp/elpa/slime-2.26.1/slime-tests hides /usr/share/emacs/site-lisp/elpa-src/slime-2.27/slime-tests
/usr/share/emacs/site-lisp/elpa/slime-2.26.1/slime-pkg hides /usr/share/emacs/site-lisp/elpa-src/slime-2.27/slime-pkg
/usr/share/emacs/site-lisp/elpa/slime-2.26.1/slime hides /usr/share/emacs/site-lisp/elpa-src/slime-2.27/slime
/usr/share/emacs/site-lisp/elpa/seq-2.22/seq hides /usr/share/emacs/28.2/lisp/emacs-lisp/seq
/usr/share/emacs/site-lisp/elpa/let-alist-1.0.6/let-alist hides /usr/share/emacs/28.2/lisp/emacs-lisp/let-alist

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg rfc6068 epg-config gnus-util rmail
rmail-loaddefs auth-source eieio eieio-core eieio-loaddefs
password-cache json map text-property-search time-date mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail comp comp-cstr subr-x rx cl-seq cl-macs cl-extra help-mode seq
seq-25 byte-opt gv warnings bytecomp byte-compile cconv rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils cus-edit pp cus-start cus-load
wid-edit cl-loaddefs cl-lib vm-autoload vm-autoloads vm-version vm-vars
vm-init iso-transl tooltip eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode lisp-mode prog-mode register
page tab-bar menu-bar rfn-eshadow isearch easymenu timer select
scroll-bar mouse jit-lock font-lock syntax font-core term/tty-colors
frame minibuffer cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice button
loaddefs faces cus-face macroexp files window text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit x multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 133934 12639)
 (symbols 48 11016 1)
 (strings 32 31394 3667)
 (string-bytes 1 1130590)
 (vectors 16 19658)
 (vector-slots 8 370387 24015)
 (floats 8 39 31)
 (intervals 56 415 0)
 (buffers 992 14))

bug#64102: 28.2; fails to find pathname with a sub dir as symlink and with parent dir ('..')

[Eli Zaretskii]

> Date: Thu, 15 Jun 2023 22:25:56 +0000
> msip_labels: 
> From:  Jacob Burckhardt via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> Reproduce the bug by running this:
> 
> mkdir -p /tmp/usr2/lib/gcc/x86_64-linux-gnu/12
> mkdir /tmp/usr2/include
> echo symbolic link test > /tmp/usr2/include/test.h
> \ln -s usr2/lib /tmp/lib2
> cat /tmp/lib2/gcc/x86_64-linux-gnu/12/../../../../include/test.h
> emacs -q /tmp/lib2/gcc/x86_64-linux-gnu/12/../../../../include/test.h
> 
> Emacs failed to display the content of that file in a buffer.  Since the above cat command shows the contents, Emacs should be able to show it as well.  The following bug is similar and includes some explanations that also apply to my bug.
> 
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=8035
> 
> Note that the following code succeeds.  You might consider using code like this to fix the bug:
> 
> (find-file (file-truename "/tmp/lib2/gcc/x86_64-linux-gnu/12/../../../../include/test.h"))

This would mean expand-file-name would need to call file-truename to
resolve such tricky symlinks, which I think is not reasonable.  We
never did that, AFAICT, and the code in find-file-noselect that begins
with expand-file-name on the argument FILENAME has been there since
1992.

bug#64102: 28.2; fails to find pathname with a sub dir as symlink and with parent dir ('..')

[Eli Zaretskii]

> Cc: 64102@debbugs.gnu.org
> Date: Fri, 16 Jun 2023 10:11:49 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > Date: Thu, 15 Jun 2023 22:25:56 +0000
> > msip_labels: 
> > From:  Jacob Burckhardt via "Bug reports for GNU Emacs,
> >  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> > 
> > Reproduce the bug by running this:
> > 
> > mkdir -p /tmp/usr2/lib/gcc/x86_64-linux-gnu/12
> > mkdir /tmp/usr2/include
> > echo symbolic link test > /tmp/usr2/include/test.h
> > \ln -s usr2/lib /tmp/lib2
> > cat /tmp/lib2/gcc/x86_64-linux-gnu/12/../../../../include/test.h
> > emacs -q /tmp/lib2/gcc/x86_64-linux-gnu/12/../../../../include/test.h
> > 
> > Emacs failed to display the content of that file in a buffer.  Since the above cat command shows the contents, Emacs should be able to show it as well.  The following bug is similar and includes some explanations that also apply to my bug.
> > 
> > https://debbugs.gnu.org/cgi/bugreport.cgi?bug=8035
> > 
> > Note that the following code succeeds.  You might consider using code like this to fix the bug:
> > 
> > (find-file (file-truename "/tmp/lib2/gcc/x86_64-linux-gnu/12/../../../../include/test.h"))
> 
> This would mean expand-file-name would need to call file-truename to
> resolve such tricky symlinks, which I think is not reasonable.  We
> never did that, AFAICT, and the code in find-file-noselect that begins
> with expand-file-name on the argument FILENAME has been there since
> 1992.

Paul, any comments or ideas?

bug#64102: 28.2; fails to find pathname with a sub dir as symlink and with parent dir ('..')

[Paul Eggert]

On 2023-06-18 01:53, Eli Zaretskii wrote:

> Paul, any comments or ideas?

Emacs's behavior is well-documented and longstanding. It's also wrong, 
in that it disagrees with almost every other program and it therefore 
confuses users, particularly non-experts.

One possible way out would be to add a user-settable flag to select the 
longstanding Emacs behavior, or the behavior of most other programs. We 
could document that the default might change in the future. Of course 
this approach would have its own problems.

bug#64102: 28.2; fails to find pathname with a sub dir as symlink and with parent dir ('..')

[Eli Zaretskii]

> Date: Sun, 18 Jun 2023 11:03:59 -0700
> Cc: jacob@teamconnectusa.com, 64102@debbugs.gnu.org
> From: Paul Eggert <eggert@cs.ucla.edu>
> 
> On 2023-06-18 01:53, Eli Zaretskii wrote:
> 
> > Paul, any comments or ideas?
> 
> Emacs's behavior is well-documented and longstanding. It's also wrong, 
> in that it disagrees with almost every other program and it therefore 
> confuses users, particularly non-experts.
> 
> One possible way out would be to add a user-settable flag to select the 
> longstanding Emacs behavior, or the behavior of most other programs. We 
> could document that the default might change in the future. Of course 
> this approach would have its own problems.

Before we consider how to trigger the new behavior, would you please
describe what kind of changes will be needed for implementing it?

My take from the description of the issue is that we'd need to change
expand-file-name to follow symlinks, i.e. hit the disk, something that
it never did, at least not on Posix platforms.  If this is what is
needed, it would have serious implications, at least performance-wise,
because expand-file-name is called everywhere in Emacs where we deal
with file names.

bug#64102: 28.2; fails to find pathname with a sub dir as symlink and with parent dir ('..')

[Paul Eggert]

On 2023-06-18 11:25, Eli Zaretskii wrote:
>> One possible way out would be to add a user-settable flag to select the
>> longstanding Emacs behavior, or the behavior of most other programs. We
>> could document that the default might change in the future. Of course
>> this approach would have its own problems.
> Before we consider how to trigger the new behavior, would you please
> describe what kind of changes will be needed for implementing it?

That'd be a lot of work, and I'm a bit pressed for time unfortunately. 
Perhaps someone who has more time (or cares more about the issue) could 
carry this torch.

bug#64102: 28.2; fails to find pathname with a sub dir as symlink and with parent dir ('..')

[Al Petrofsky]

[-- Attachment #1: Type: text/plain, Size: 2365 bytes --]

Jacob:
> cat /tmp/lib2/gcc/x86_64-linux-gnu/12/../../../../include/test.h
> emacs -q /tmp/lib2/gcc/x86_64-linux-gnu/12/../../../../include/test.h

> Emacs failed to display the content of that file in a buffer.  Since
> the above cat command shows the contents, Emacs should be able to show
> it as well.

Paul:
> Emacs's behavior is well-documented and longstanding. It's also wrong,
> in that it disagrees with almost every other program and it therefore
> confuses users, particularly non-experts.
>
> One possible way out would be to add a user-settable flag to select the
> longstanding Emacs behavior, or the behavior of most other programs. We
> could document that the default might change in the future. Of course
> this approach would have its own problems.

Eli:
> My take from the description of the issue is that we'd need to change
> expand-file-name to follow symlinks, i.e. hit the disk, something that
> it never did, at least not on Posix platforms.  If this is what is
> needed, it would have serious implications, at least performance-wise,
> because expand-file-name is called everywhere in Emacs where we deal
> with file names.

Perhaps another approach would be that Emacs continues to have its own
idiosyncratic understanding of pathnames internally, but pathnames
that come in from external sources are interpreted correctly according
to the rules of the external environment.

That seems to be the approach that was taken by Lars in 2021 to fix
bug 8035 (referenced by Jacob in the first message of the current
bug).  See the patch that added file-truename calls to compile.el:


http://git.savannah.gnu.org/cgit/emacs.git/commit/lisp/progmodes/compile.el?id=ee2ffd9c9eb33a17307f36ff58caec1ba79878d2

As far as I can tell, that change was made without adding a
user-settable flag to select the old behavior, nor with any
announcement in NEWS or in the manual.  Are any problems known to have
arisen yet?

The current bug could similarly be fixed by making any filename that
is obtained from the argv of emacs or emacsclient be passed through
file-truename.  I don't know how much work that would be to implement,
or how many user-confusing situations would remain and/or be created,
but at least I don't see it being a performance problem, as the cost
of an exec of emacs or emacsclient would likely dwarf the cost of the
truename calls.

[-- Attachment #2: Type: text/html, Size: 2801 bytes --]

bug#64102: 28.2; fails to find pathname with a sub dir as symlink and with parent dir ('..')

[Eli Zaretskii]

> From: Al Petrofsky <al@petrofsky.org>
> Date: Sun, 18 Jun 2023 16:13:16 -0400
> 
> > My take from the description of the issue is that we'd need to change
> > expand-file-name to follow symlinks, i.e. hit the disk, something that
> > it never did, at least not on Posix platforms.  If this is what is
> > needed, it would have serious implications, at least performance-wise,
> > because expand-file-name is called everywhere in Emacs where we deal
> > with file names.
> 
> Perhaps another approach would be that Emacs continues to have its own
> idiosyncratic understanding of pathnames internally, but pathnames
> that come in from external sources are interpreted correctly according
> to the rules of the external environment.

This bug report is against what happens when visiting files given by
their name.  Emacs visits files in umpteen different ways, and they
all need to behave consistently.  Moreover, Emacs often visits files
or inserts their contents internally, for various processing purposes,
such, for example, processing of .mailalias files, save-place files,
etc.  If some of these features resolve symlinks in all the
directories, but others don't, this will be a source of constant user
surprise and complaints.

> That seems to be the approach that was taken by Lars in 2021 to fix
> bug 8035 (referenced by Jacob in the first message of the current
> bug).  See the patch that added file-truename calls to compile.el:
> 
>   
> http://git.savannah.gnu.org/cgit/emacs.git/commit/lisp/progmodes/compile.el?id=ee2ffd9c9eb33a17307f36ff58caec1ba79878d2

That changed a single place where using the resolved names solved a
particular problem in some internal processing step.  What is being
discussed here is a completely different, much more general problem.

> The current bug could similarly be fixed by making any filename that
> is obtained from the argv of emacs or emacsclient be passed through
> file-truename.

That's just two ways Emacs gets a file name that it is asked to visit.
It won't solve the following slight variation of the original report:

  emacs -q
  C-x C-f /tmp/lib2/gcc/x86_64-linux-gnu/12/../../../../include/test.h RET

> I don't know how much work that would be to implement,
> or how many user-confusing situations would remain and/or be created,
> but at least I don't see it being a performance problem, as the cost
> of an exec of emacs or emacsclient would likely dwarf the cost of the
> truename calls.

The performance problem will happen if we make expand-file-name
resolve all the symlinks.  That is the solution I had in mind, and
AFAIU it is the _only_ solution that will produce consistent results.

However, maybe I'm missing some clever alternative.  Which is why I
asked Paul to describe what he had in mind.  But solving it one case
at a time is definitely not TRT.