bug#1135: 23.0.60; (expand-file-name ".." "/") produces surprising results

bug#1135: 23.0.60; (expand-file-name ".." "/") produces surprising results

[Daniel Pittman]

Please write in English if possible, because the Emacs maintainers
usually do not have translators to read other languages for them.

Your bug report will be posted to the emacs-pretest-bug@gnu.org mailing list.

Please describe exactly what actions triggered the bug
and the precise symptoms of the bug:


I use some code that uses (setq dir (expand-file-name ".." dir)) to walk
up a directory tree to the root.  This works fine on the local disk, but
has some surprising results -- infinite loop -- on Tramp files.

I debugged this down to the unexpected (to me) behaviour of

    (expand-file-name ".." "/")
    => "/.."

    (expand-file-name ".." "/..")
    => "/"

This was causing the code, which checked if the current and previous
result of (expand-file-name ".." dir) were the same, to go into an
infinite loop cycling between those two values...


I would have expected that the result was:

    (expand-file-name ".." "/")
    => "/"

Regards,
        Daniel


If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
If you would like to further debug the crash, please read the file
/usr/share/emacs/23.0.60/etc/DEBUG for instructions.


In GNU Emacs 23.0.60.1 (x86_64-pc-linux-gnu, GTK+ Version 2.12.9)
 of 2008-03-20 on yellow, modified by Debian
 (emacs-snapshot package, version 1:20080228-1ubuntu1)
Windowing system distributor `The X.Org Foundation', version 11.0.10400090
configured using `configure  '--build' 'x86_64-linux-gnu' '--host' 'x86_64-linux-gnu' '--prefix=/usr' '--sharedstatedir=/var/lib' '--libexecdir=/usr/lib' '--localstatedir=/var' '--infodir=/usr/share/info' '--mandir=/usr/share/man' '--with-pop=yes' '--enable-locallisppath=/etc/emacs-snapshot:/etc/emacs:/usr/local/share/emacs/23.0.60/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/23.0.60/site-lisp:/usr/share/emacs/site-lisp:/usr/share/emacs/23.0.60/leim' '--with-x=yes' '--with-x-toolkit=gtk' '--enable-font-backend' 'build_alias=x86_64-linux-gnu' 'host_alias=x86_64-linux-gnu' 'CFLAGS=-DDEBIAN -DSITELOAD_PURESIZE_EXTRA=5000 -g -O2' 'LDFLAGS=-g -Wl,--as-needed' 'CPPFLAGS=''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_AU.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default-enable-multibyte-characters: t

Major mode: Emacs-Lisp

Minor modes in effect:
  eldoc-mode: t
  highlight-parentheses-mode: t
  flyspell-mode: t
  shell-dirtrack-mode: t
  show-paren-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
c c c c c d d d d d d d d d c c c c c c c c c c c c 
c c C-x k <return> C-x C-f <up> <return> c c c c c 
c c c c c c c c c c c c c c c c c c c c c c c c c c 
c c c c c c c c c c c c c c c c c c c c c c c c c c 
c c c c c c c c c c c c c c c c c c c c c c c c c c 
c c c c c c c c c c c c c c d d d d d d d d d d d c 
c c c q C-/ C-/ C-M-x C-x k <return> M-: ( e x p a 
n d - f i l e - n a m e SPC " . . " S-SPC " / " ) <return> 
M-: <up> C-e <left> <left> <left> <left> <right> <right> 
. p . <backspace> <backspace> . <return> <tab> C-h 
f e x p a n d - f i l / <tab> <backspace> <tab> <return> 
<switch-frame> SPC n q C-s g n u <down> <down> <down> 
<down> <return> SPC C-k C-k C-k C-k q p p g n SPC SPC 
n n n <switch-frame> <switch-frame> <switch-frame> 
<switch-frame> <switch-frame> <switch-frame> <C-tab> 
C-p C-p C-p C-f C-f C-f C-f C-f C-f C-f C-f C-f C-SPC 
M-f M-f M-f <C-insert> <switch-frame> <switch-frame> 
<switch-frame> <switch-frame> <switch-frame> M-x r 
e p o r t <tab> C-g <switch-frame> M-x C-g M-x r e 
p o r t - e m <tab> b <tab> <return>

Recent messages:
Reading active file from archive via nnfolder...done
Checking new news...done
Retrieving newsgroup: nntp+enki:gmane.emacs.devel...
Fetching headers for nntp+enki:gmane.emacs.devel...done
Suppressing duplicates...done
Scoring...done
Generating summary...done
Mark set
Auto-saving...done
Quit
Quit

bug#1135: 23.0.60; (expand-file-name ".." "/") produces surprising results

[Stefan Monnier]

> I use some code that uses (setq dir (expand-file-name ".." dir)) to walk
> up a directory tree to the root.

Was there any particular reason not to use:

   (setq dir (directory-file-name (file-name-directory dir)))

> I debugged this down to the unexpected (to me) behaviour of

>     (expand-file-name ".." "/")
>     => "/.."

>     (expand-file-name ".." "/..")
>     => "/"

That's a bit problematic, indeed.  The reason for the first behavior is
because there exist(ed) special file systems that were accessible under
/.. (IIRC machines at cs.cmu.edu around early 90s, maybe late 80s, had
such a distributed filesystem that allowed access to other machines via
/../<hostname>/..., maybe there were other such things).  The second is
there because nobody knows what /../.. is supposed to mean in such
a context: if /.. is a mounted filesystem, then /../.. should get you
back to /; then again if /.. is really the Root, then /../.. should just
be /..; or maybe there's a hierarchy of roots so /../.. should just stay
as /../..?

> This was causing the code, which checked if the current and previous
> result of (expand-file-name ".." dir) were the same, to go into an
> infinite loop cycling between those two values...

The more I see such problems the more I think that such code should
check (> (length before) (length after)).  At least, that should
*guarantee* termination, no matter how weird expand-file-name can behave.


        Stefan

bug#1135: 23.0.60; (expand-file-name ".." "/") produces surprising results

[Daniel Pittman]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I use some code that uses (setq dir (expand-file-name ".." dir)) to
>> walk up a directory tree to the root.
>
> Was there any particular reason not to use:
>
>    (setq dir (directory-file-name (file-name-directory dir)))

While applying `file-name-directory' to a directory works fine and makes
sense, it never occurred to me to consider that.

>> I debugged this down to the unexpected (to me) behaviour of
>
>>     (expand-file-name ".." "/")
>>     => "/.."
>
>>     (expand-file-name ".." "/..")
>>     => "/"
>
> That's a bit problematic, indeed.  The reason for the first behavior is
> because there exist(ed) special file systems that were accessible under
> /.. (IIRC machines at cs.cmu.edu around early 90s, maybe late 80s, had
> such a distributed filesystem that allowed access to other machines via
> /../<hostname>/..., maybe there were other such things).  The second is
> there because nobody knows what /../.. is supposed to mean in such
> a context: if /.. is a mounted filesystem, then /../.. should get you
> back to /; then again if /.. is really the Root, then /../.. should just
> be /..; or maybe there's a hierarchy of roots so /../.. should just stay
> as /../..?

*nod*  I would certainly (myself) encourage this to be documented, or to
be added as a machine dependent extension to the code on those
particular systems.

I knew about a whole bunch of odd file-name handling on various systems
through the decades, but this is a new one to me.  I can imagine it
would be rather a shock to, for example, people used only to Windows and
Linux systems. :)


For what it is worth, since I hit this only via Tramp I initially
assumed that it was a bug in the implementation of `expand-file-name' in
that code, probably through an optimization that didn't quite work out.

>> This was causing the code, which checked if the current and previous
>> result of (expand-file-name ".." dir) were the same, to go into an
>> infinite loop cycling between those two values...
>
> The more I see such problems the more I think that such code should
> check (> (length before) (length after)).  At least, that should
> *guarantee* termination, no matter how weird expand-file-name can
> behave.

*nod*  I had assumed that while things like Tramp and, possibly, other
random platforms meant my code couldn't assume that simply walking up to
`(equal dir (expand-file-name "/"))' would assure termination, a
back-and-forth loop would not occur.

Anyway, thank you for both the advice on a better (and terminating)
approach, and your interest in the fault.  


My suggestion would be the documentation approach; in my Emacs the help
reads, with my extension following:

    expand-file-name is a built-in function in `C source code'.

    (expand-file-name name &optional default-directory)

    Convert filename name to absolute, and canonicalize it.
    Second arg default-directory is directory to start with if name is relative
    (does not start with slash or tilde); if default-directory is nil or missing,
    the current buffer's value of `default-directory' is used.
    File name components that are `.' are removed, and
    so are file name components followed by `..', along with the `..' itself;
    note that these simplifications are done without checking the resulting
    file names in the file system.
    An initial `~/' expands to your home directory.
    An initial `~USER/' expands to USER's home directory.
    See also the function `substitute-in-file-name'.

In some cases `expand-file-name' can return surprising results, such as
expanding `..' against `/', which correctly returns `/..'; in most cases
`file-name-directory' and `directory-file-name' are more appropriate for
walking filesystem trees.


Regards,
        Daniel

bug#1135: marked as done (23.0.60; (expand-file-name ".." "/") produces surprising results)

[Emacs bug Tracking System]

[-- Attachment #1: Type: text/plain, Size: 900 bytes --]


Your message dated Thu, 16 Oct 2008 14:01:06 -0400
with message-id <87ljwo1kt9.fsf@cyd.mit.edu>
and subject line Re: bug#1135: 23.0.60; (expand-file-name ".." "/") produces surprising results
has caused the Emacs bug report #1135,
regarding 23.0.60; (expand-file-name ".." "/") produces surprising results
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact don@donarmstrong.com
immediately.)


-- 
1135: http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=1135
Emacs Bug Tracking System
Contact don@donarmstrong.com with problems

[-- Attachment #2: Type: message/rfc822, Size: 6785 bytes --]

From: Daniel Pittman <daniel@rimspace.net>
To: emacs-pretest-bug@gnu.org
Subject: 23.0.60; (expand-file-name ".." "/") produces surprising results
Date: Fri, 10 Oct 2008 14:41:55 +1100
Message-ID: <87k5chce0s.fsf@rimspace.net>


Please write in English if possible, because the Emacs maintainers
usually do not have translators to read other languages for them.

Your bug report will be posted to the emacs-pretest-bug@gnu.org mailing list.

Please describe exactly what actions triggered the bug
and the precise symptoms of the bug:


I use some code that uses (setq dir (expand-file-name ".." dir)) to walk
up a directory tree to the root.  This works fine on the local disk, but
has some surprising results -- infinite loop -- on Tramp files.

I debugged this down to the unexpected (to me) behaviour of

    (expand-file-name ".." "/")
    => "/.."

    (expand-file-name ".." "/..")
    => "/"

This was causing the code, which checked if the current and previous
result of (expand-file-name ".." dir) were the same, to go into an
infinite loop cycling between those two values...


I would have expected that the result was:

    (expand-file-name ".." "/")
    => "/"

Regards,
        Daniel


If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
If you would like to further debug the crash, please read the file
/usr/share/emacs/23.0.60/etc/DEBUG for instructions.


In GNU Emacs 23.0.60.1 (x86_64-pc-linux-gnu, GTK+ Version 2.12.9)
 of 2008-03-20 on yellow, modified by Debian
 (emacs-snapshot package, version 1:20080228-1ubuntu1)
Windowing system distributor `The X.Org Foundation', version 11.0.10400090
configured using `configure  '--build' 'x86_64-linux-gnu' '--host' 'x86_64-linux-gnu' '--prefix=/usr' '--sharedstatedir=/var/lib' '--libexecdir=/usr/lib' '--localstatedir=/var' '--infodir=/usr/share/info' '--mandir=/usr/share/man' '--with-pop=yes' '--enable-locallisppath=/etc/emacs-snapshot:/etc/emacs:/usr/local/share/emacs/23.0.60/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/23.0.60/site-lisp:/usr/share/emacs/site-lisp:/usr/share/emacs/23.0.60/leim' '--with-x=yes' '--with-x-toolkit=gtk' '--enable-font-backend' 'build_alias=x86_64-linux-gnu' 'host_alias=x86_64-linux-gnu' 'CFLAGS=-DDEBIAN -DSITELOAD_PURESIZE_EXTRA=5000 -g -O2' 'LDFLAGS=-g -Wl,--as-needed' 'CPPFLAGS=''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_AU.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default-enable-multibyte-characters: t

Major mode: Emacs-Lisp

Minor modes in effect:
  eldoc-mode: t
  highlight-parentheses-mode: t
  flyspell-mode: t
  shell-dirtrack-mode: t
  show-paren-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
c c c c c d d d d d d d d d c c c c c c c c c c c c 
c c C-x k <return> C-x C-f <up> <return> c c c c c 
c c c c c c c c c c c c c c c c c c c c c c c c c c 
c c c c c c c c c c c c c c c c c c c c c c c c c c 
c c c c c c c c c c c c c c c c c c c c c c c c c c 
c c c c c c c c c c c c c c d d d d d d d d d d d c 
c c c q C-/ C-/ C-M-x C-x k <return> M-: ( e x p a 
n d - f i l e - n a m e SPC " . . " S-SPC " / " ) <return> 
M-: <up> C-e <left> <left> <left> <left> <right> <right> 
. p . <backspace> <backspace> . <return> <tab> C-h 
f e x p a n d - f i l / <tab> <backspace> <tab> <return> 
<switch-frame> SPC n q C-s g n u <down> <down> <down> 
<down> <return> SPC C-k C-k C-k C-k q p p g n SPC SPC 
n n n <switch-frame> <switch-frame> <switch-frame> 
<switch-frame> <switch-frame> <switch-frame> <C-tab> 
C-p C-p C-p C-f C-f C-f C-f C-f C-f C-f C-f C-f C-SPC 
M-f M-f M-f <C-insert> <switch-frame> <switch-frame> 
<switch-frame> <switch-frame> <switch-frame> M-x r 
e p o r t <tab> C-g <switch-frame> M-x C-g M-x r e 
p o r t - e m <tab> b <tab> <return>

Recent messages:
Reading active file from archive via nnfolder...done
Checking new news...done
Retrieving newsgroup: nntp+enki:gmane.emacs.devel...
Fetching headers for nntp+enki:gmane.emacs.devel...done
Suppressing duplicates...done
Scoring...done
Generating summary...done
Mark set
Auto-saving...done
Quit
Quit



[-- Attachment #3: Type: message/rfc822, Size: 1126 bytes --]

From: Chong Yidong <cyd@stupidchicken.com>
To: Daniel Pittman <daniel@rimspace.net>
Cc: 1135-done@emacsbugs.donarmstrong.com
Subject: Re: bug#1135: 23.0.60; (expand-file-name ".." "/") produces surprising results
Date: Thu, 16 Oct 2008 14:01:06 -0400
Message-ID: <87ljwo1kt9.fsf@cyd.mit.edu>

> My suggestion would be the documentation approach

I checked in a slightly different doc fix.  Thanks for reporting the
bug.