bug#8035: Processing of .. in a file path after going thru symlink

bug#8035: Processing of .. in a file path after going thru symlink

[spucci]

The behavior is general, but it's a particular problem with next-error: 

When emacs attempts to find a file with "../" as a path component, it
appears to be "smart" about it and simply remove the previous directory path
(e.g., "foo/bar/../x" gets converted to "foo/x").  But if bar is a symlink,
then it doesn't properly find the file.  So in compiler output, which
references such files, the next-error function fails to find the file with
the given name. 

mkdir dest 
mkdir dest/subdir 
mkdir src 
ln -s ../dest/subdir src/subdir 
echo "#error This is an error" > dest/foo.c 

Now M-x compile, and give it cc -c src/subdir/../foo.c 

*compilation* buffer has: 
cc -c src/subdir/../foo.c 
src/subdir/../foo.c:1:2: error: #error This is an error 

and do a next-error: Emacs complains it can't find the file.  And if you try
to find-file that file path (src/subdir/../foo.c) it doesn't work either. 
Nor does "emacsclient src/subdir/../foo.c.  I couldn't find an option to
disable this behavior; it seems like there should be one even if the default
continues to be as it is today.

This is Gnu Emacs 23.2.1 I built myself on MacOS X 10.6.6. 

Thanks, 
  Steve 

-- 
View this message in context: http://old.nabble.com/Processing-of-..-in-a-file-path-after-going-thru-symlink-tp30918394p30918394.html
Sent from the Emacs - Bugs mailing list archive at Nabble.com.

bug#8035: Processing of .. in a file path after going thru symlink

[Glenn Morris]

spucci wrote:

> When emacs attempts to find a file with "../" as a path component, it
> appears to be "smart" about it and simply remove the previous directory path
> (e.g., "foo/bar/../x" gets converted to "foo/x").  But if bar is a symlink,
> then it doesn't properly find the file.

This appears to be a feature of expand-file-name (and possibly other
things). It does seem a little weird that there isn't even an option to
have a more thorough expansion...

bug#8035: Processing of .. in a file path after going thru symlink

[Steve Pucci]

On Feb 19, 2011, at 1:37 PM, Glenn Morris wrote:

> This appears to be a feature of expand-file-name (and possibly other
> things). It does seem a little weird that there isn't even an option to
> have a more thorough expansion...

So I ran an experiment, redefining expand-file-name as follows to skip the ".." processing (except in default-directory) and otherwise do the rest:

(or (fboundp 'save-expand-file-name)
    (fset 'save-expand-file-name (symbol-function 'expand-file-name)))
(defun expand-file-name (NAME &optional DEFAULT-DIRECTORY)
  (cond ((string-match "^/" NAME)
         NAME)
        ((string-match "^\\(~[^/]*\\)\\(.*\\)$" NAME)
         (let ((userdir (match-string 1 NAME))
               (rest (match-string 2 NAME)))
           (concat (save-expand-file-name userdir) rest)))
        (t (concat (save-expand-file-name (if DEFAULT-DIRECTORY
                                              DEFAULT-DIRECTORY
                                            default-directory))
                   NAME))))

While this does the trick (it expands only ~ and relative paths and properly leaves the OP path intact), it fails to fix the problem with next-error.  So I conclude you're right there are other places that do the same thing, apparently.

My workaround now is to wrap my build script in a Perl script which rewrites all "../" paths it finds within its output.  So I'm ok, though I'm surprised this hasn't come up before...

bug#8035: Processing of .. in a file path after going thru symlink

[Steve Pucci]

On Feb 19, 2011, at 6:35 PM, Steve Pucci wrote:

> So I conclude you're right there are other places that do the same thing, apparently.

Well, my experiment didn't intercept the (literally) 98 places that call Fexpand_file_name() directly from the C code, of course...

bug#8035: Processing of .. in a file path after going thru symlink

[Lars Ingebrigtsen]

spucci <slpnabble@blackberry-hill.com> writes:

> When emacs attempts to find a file with "../" as a path component, it
> appears to be "smart" about it and simply remove the previous directory path
> (e.g., "foo/bar/../x" gets converted to "foo/x").  But if bar is a symlink,
> then it doesn't properly find the file.  So in compiler output, which
> references such files, the next-error function fails to find the file with
> the given name. 
>
> mkdir dest 
> mkdir dest/subdir 
> mkdir src 
> ln -s ../dest/subdir src/subdir 
> echo "#error This is an error" > dest/foo.c 
>
> Now M-x compile, and give it cc -c src/subdir/../foo.c 
>
> *compilation* buffer has: 
> cc -c src/subdir/../foo.c 
> src/subdir/../foo.c:1:2: error: #error This is an error 
>
> and do a next-error: Emacs complains it can't find the file. 

This is an even more foundational problem:

(file-exists-p "/tmp/comp/src/subdir/../foo.c")
=> nil
(file-truename "/tmp/comp/src/subdir/../foo.c")
=> "/tmp/comp/dest/foo.c"
(file-exists-p (file-truename "/tmp/comp/src/subdir/../foo.c"))
=> t

And this is because:

static Lisp_Object
check_file_access (Lisp_Object file, Lisp_Object operation, int amode)
{
  file = Fexpand_file_name (file, Qnil);

I'm guessing it's calling expand-file-name here to resolve "~"?  It's
also a micro-optimisation, I guess -- collapsing "parent/.." textually
without checking the file system is very cheap.

But...  this does mean that `file-exists-p' and friends are unreliable
in the presence of symlinks, which is pretty depressing.  I think that
`expand-file-name' call in check_file_access should be changed to
something that just does the "~" expansion.

Any opinions?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#8035: Processing of .. in a file path after going thru symlink

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Thu, 26 Aug 2021 18:29:54 +0200
> Cc: 8035@debbugs.gnu.org
> 
> (file-exists-p "/tmp/comp/src/subdir/../foo.c")
> => nil
> (file-truename "/tmp/comp/src/subdir/../foo.c")
> => "/tmp/comp/dest/foo.c"
> (file-exists-p (file-truename "/tmp/comp/src/subdir/../foo.c"))
> => t
> 
> And this is because:
> 
> static Lisp_Object
> check_file_access (Lisp_Object file, Lisp_Object operation, int amode)
> {
>   file = Fexpand_file_name (file, Qnil);
> 
> I'm guessing it's calling expand-file-name here to resolve "~"?

No, because we always must call expand-file-name before invoking a
libc function that accesses files -- to support file names relative to
their buffer's default-directory.

> I think that `expand-file-name' call in check_file_access should be
> changed to something that just does the "~" expansion.

I don't think we can do that, for the reason explained above.

bug#8035: Processing of .. in a file path after going thru symlink

[Andreas Schwab]

On Aug 26 2021, Lars Ingebrigtsen wrote:

> But...  this does mean that `file-exists-p' and friends are unreliable
> in the presence of symlinks,

This is not true.  It is consistent with all other file handling
functions in Emacs.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#8035: Processing of .. in a file path after going thru symlink

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> I'm guessing it's calling expand-file-name here to resolve "~"?
>
> No, because we always must call expand-file-name before invoking a
> libc function that accesses files -- to support file names relative to
> their buffer's default-directory.

These are absolute file names, though.  I can understand expanding if
they're not.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#8035: Processing of .. in a file path after going thru symlink

[Lars Ingebrigtsen]

Andreas Schwab <schwab@linux-m68k.org> writes:

> On Aug 26 2021, Lars Ingebrigtsen wrote:
>
>> But...  this does mean that `file-exists-p' and friends are unreliable
>> in the presence of symlinks,
>
> This is not true.  It is consistent with all other file handling
> functions in Emacs.

I don't see a contradiction between `file-exists-p' being unreliable
and being consistent with all other file handling functions in Emacs.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#8035: Processing of .. in a file path after going thru symlink

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: slpnabble@blackberry-hill.com,  8035@debbugs.gnu.org
> Date: Thu, 26 Aug 2021 19:03:24 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> I'm guessing it's calling expand-file-name here to resolve "~"?
> >
> > No, because we always must call expand-file-name before invoking a
> > libc function that accesses files -- to support file names relative to
> > their buffer's default-directory.
> 
> These are absolute file names, though.  I can understand expanding if
> they're not.

You asked why expand-file-name is there in general, so I gave a
general explanation.

expand-file-name also normalizes file names, though.  Which is also
needed in some use cases, e.g. when the /foo/bar/ isn't accessible,
but /foo/ is, in which case failing to normalize /foo/bar/../quux
could fail some file I/O function.

bug#8035: Processing of .. in a file path after going thru symlink

[Andreas Schwab]

On Aug 26 2021, Lars Ingebrigtsen wrote:

> Andreas Schwab <schwab@linux-m68k.org> writes:
>
>> On Aug 26 2021, Lars Ingebrigtsen wrote:
>>
>>> But...  this does mean that `file-exists-p' and friends are unreliable
>>> in the presence of symlinks,
>>
>> This is not true.  It is consistent with all other file handling
>> functions in Emacs.
>
> I don't see a contradiction between `file-exists-p' being unreliable
> and being consistent with all other file handling functions in Emacs.

It would be unreliable if it were inconsistent.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#8035: Processing of .. in a file path after going thru symlink

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> expand-file-name also normalizes file names, though.  Which is also
> needed in some use cases, e.g. when the /foo/bar/ isn't accessible,
> but /foo/ is, in which case failing to normalize /foo/bar/../quux
> could fail some file I/O function.

Hm, yes, that's a point...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#8035: Processing of .. in a file path after going thru symlink

[Lars Ingebrigtsen]

spucci <slpnabble@blackberry-hill.com> writes:

> mkdir dest 
> mkdir dest/subdir 
> mkdir src 
> ln -s ../dest/subdir src/subdir 
> echo "#error This is an error" > dest/foo.c 
>
> Now M-x compile, and give it cc -c src/subdir/../foo.c 

This should now be fixed in Emacs 28.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#8035: Processing of .. in a file path after going thru symlink

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: slpnabble@blackberry-hill.com,  8035@debbugs.gnu.org
> Date: Thu, 26 Aug 2021 19:24:28 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > expand-file-name also normalizes file names, though.  Which is also
> > needed in some use cases, e.g. when the /foo/bar/ isn't accessible,
> > but /foo/ is, in which case failing to normalize /foo/bar/../quux
> > could fail some file I/O function.
> 
> Hm, yes, that's a point...

The OP's use case is clearly documented in the ELisp manual:

     Note also that ‘expand-file-name’ does not follow symbolic links at
     any level.  This results in a difference between the way
     ‘file-truename’ and ‘expand-file-name’ treat ‘..’.  Assuming that
     ‘/tmp/bar’ is a symbolic link to the directory ‘/tmp/foo/bar’ we
     get:

          (file-truename "/tmp/bar/../myfile")
               ⇒ "/tmp/foo/myfile"
          (expand-file-name "/tmp/bar/../myfile")
               ⇒ "/tmp/myfile"

     If you may need to follow symbolic links preceding ‘..’, you should
     make sure to call ‘file-truename’ without prior direct or indirect
     calls to ‘expand-file-name’.  *Note Truenames::.

So I think we need to use file-truename if we want to support such
uses of symlinks in "M-x compile" etc.