bug#9039: 24.0.50; dired "f" fails on directory that begins with a space

bug#9039: 24.0.50; dired "f" fails on directory that begins with a space

[merlyn]

This bug report will be sent to the Bug-GNU-Emacs mailing list
and the GNU bug tracker at debbugs.gnu.org.  Please check that
the From: line contains a valid email address.  After a delay of up
to one day, you should receive an acknowledgement at that address.

Please write in English if possible, as the Emacs maintainers
usually do not have translators for other languages.

Please describe exactly what actions triggered the bug, and
the precise symptoms of the bug.  If you can, give a recipe
starting from `emacs -Q':

Create a directory that begins with space.
Visit the parent directory in dired.
Move to the line that references the space-beginning directory
hit "f"
Emacs fails to descend.

If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
For information about debugging Emacs, please read the file
/Users/merlyn/Applications/Emacs.app/Contents/Resources/etc/DEBUG.


In GNU Emacs 24.0.50.1 (x86_64-apple-darwin10.8.0, NS
apple-appkit-1038.36)
 of 2011-07-09 on Locohost.local
Windowing system distributor `Apple', version 10.3.1038
configured using `configure  '--with-ns' '--without-x' '--without-dbus'
'--prefix=/tmp/emacs''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: nil
  value of $XMODIFIERS: nil
  locale-coding-system: nil
  default enable-multibyte-characters: t

Major mode: Dired by name

Minor modes in effect:
  show-paren-mode: t
  global-auto-revert-mode: t
  display-time-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
C-x C-f ~ <return> f <escape> x r e p o r t - e m a 
c s - b u g <return>

Recent messages:
Loading time...done
Loading autorevert...done
Loading avoid...done
Loading paren...done
dired-get-file-for-visit: File no longer exists; type `g' to update
dired buffer

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr message format-spec rfc822 mml mml-sec
mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mailabbrev mail-utils gmm-utils mailheader
emacsbug dired jka-compr uniquify advice help-fns advice-preload paren
avoid autorevert time cus-start cus-load server objj-mode derived
objc-c-mode cc-mode cc-fonts easymenu cc-guess cc-menus cc-cmds
cc-styles cc-align cc-engine cc-vars cc-defs regexp-opt time-date
tooltip ediff-hook vc-hooks lisp-float-type mwheel ns-win tool-bar dnd
fontset image fringe lisp-mode register page menu-bar rfn-eshadow timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
frame cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook help
simple abbrev minibuffer loaddefs button faces cus-face files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process ns
multi-tty emacs)

bug#9039: 24.0.50; dired "f" fails on directory that begins with a space

[Glenn Morris]

merlyn@stonehenge.com wrote:

> Create a directory that begins with space.
> Visit the parent directory in dired.
> Move to the line that references the space-beginning directory
> hit "f"
> Emacs fails to descend.

What is the value of `dired-use-ls-dired'?
If it is nil, I think this kind of problem is expected.

What is the result of evaluating:

(call-process insert-directory-program nil nil nil "--dired")

> In GNU Emacs 24.0.50.1 (x86_64-apple-darwin10.8.0, NS
> apple-appkit-1038.36)
>  of 2011-07-09 on Locohost.local
> Windowing system distributor `Apple', version 10.3.1038
> configured using `configure  '--with-ns' '--without-x' '--without-dbus'
> '--prefix=/tmp/emacs''

bug#9039: 24.0.50; dired "f" fails on directory that begins with a space

[Randal L. Schwartz]

>>>>> "GM" == Glenn Morris <rgm@gnu.org> writes:


GM> What is the value of `dired-use-ls-dired'?
GM> If it is nil, I think this kind of problem is expected.

nil.  The default.

GM> What is the result of evaluating:

GM> (call-process insert-directory-program nil nil nil "--dired")

1

So, why is the *default* behavior *wrong*?  Can that be fixed?

And my "ls" doesn't have a --dired switch, so I can't change that value.
Must be a GNU-ism.  What can you do with POSIX-based tools?

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.posterous.com/ for Smalltalk discussion

bug#9039: 24.0.50; dired "f" fails on directory that begins with a space

[Glenn Morris]

Randal L. Schwartz wrote:

> GM> What is the value of `dired-use-ls-dired'?
> GM> If it is nil, I think this kind of problem is expected.
>
> nil.  The default.

The default is `unspecified'.

> GM> What is the result of evaluating:
>
> GM> (call-process insert-directory-program nil nil nil "--dired")
>
> 1

OK, your dired doesn't support the --dired option. In this case, dired
is known to not perform as well, especially with regards to file names
with spaces. If you install GNU ls, it will work better.

bug#9039: 24.0.50; dired "f" fails on directory that begins with a space

[Glenn Morris]

Glenn Morris wrote (on Sat, 9 Jul 2011 at 22:42 -0400):

> OK, your dired doesn't support the --dired option.

s/dired/ls

bug#9039: 24.0.50; dired "f" fails on directory that begins with a space

[Randal L. Schwartz]

>>>>> "GM" == Glenn Morris <rgm@gnu.org> writes:

>> nil.  The default.

GM> The default is `unspecified'.

How so?  I have nothing to set it, and it comes up nil.  How is that not
the defaul then?

GM> OK, your dired doesn't support the --dired option. In this case, dired
GM> is known to not perform as well, especially with regards to file names
GM> with spaces. If you install GNU ls, it will work better.

And yet, "find-file" finds the directory with a space just fine, and
even autocompletes it.

Why isn't GNU Emacs using readdir() and stat() internally?  Why is it
depending on an external ls?

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.posterous.com/ for Smalltalk discussion

bug#9039: 24.0.50; dired "f" fails on directory that begins with a space

[Eli Zaretskii]

> From: merlyn@stonehenge.com (Randal L. Schwartz)
> Date: Sat, 09 Jul 2011 19:46:03 -0700
> Cc: 9039@debbugs.gnu.org
> 
> GM> OK, your dired doesn't support the --dired option. In this case, dired
> GM> is known to not perform as well, especially with regards to file names
> GM> with spaces. If you install GNU ls, it will work better.
> 
> And yet, "find-file" finds the directory with a space just fine, and
> even autocompletes it.

find-file does not use `ls'.

> Why isn't GNU Emacs using readdir() and stat() internally?  Why is it
> depending on an external ls?

It was always like that with Dired on Posix platforms.  If you want a
Lisp emulation of `ls', try loading ls-lisp, maybe you will find it
better for your taste.

bug#9039: 24.0.50; dired "f" fails on directory that begins with a space

[Randal L. Schwartz]

>>>>> "Eli" == Eli Zaretskii <eliz@gnu.org> writes:

>> Why isn't GNU Emacs using readdir() and stat() internally?  Why is it
>> depending on an external ls?

Eli> It was always like that with Dired on Posix platforms.  If you want a
Eli> Lisp emulation of `ls', try loading ls-lisp, maybe you will find it
Eli> better for your taste.

As I said on the dev-list:

    By the way, (directory-files-and-attributes "~") does the right thing
    with respect to this directory.  So it still looks like dired is doing
    the Wrong Thing.

Not only would it be faster, it'd be more portable.

So that's the bug here.  Dired should be using the low-level routines,
not forking a lot.

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.posterous.com/ for Smalltalk discussion

bug#9039: 24.0.50; dired "f" fails on directory that begins with a space

[Glenn Morris]

I've improved the manual and the doc of dired-use-ls-dired, as well
as making dired give a message the first time it it used if --dired
is not found to be supported. Personally, I'm inclined to close this
report now.

bug#9039: 24.0.50; dired "f" fails on directory that begins with a space

[Randal L. Schwartz]

>>>>> "GM" == Glenn Morris <rgm@gnu.org> writes:

GM> I've improved the manual and the doc of dired-use-ls-dired, as well
GM> as making dired give a message the first time it it used if --dired
GM> is not found to be supported. Personally, I'm inclined to close this
GM> report now.

What I'm confused about is if setting the other switch always does the
right thing (and faster, without forking), why would dired *ever* use
"ls --dired"?

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.posterous.com/ for Smalltalk discussion

bug#9039: 24.0.50; dired "f" fails on directory that begins with a space

[Glenn Morris]

Randal L. Schwartz wrote:

> What I'm confused about is if setting the other switch always does the
> right thing (and faster, without forking), why would dired *ever* use
> "ls --dired"?

In the doc of dired-use-ls-dired, I mention that ls-lisp does not
support as many options as GNU ls.

bug#9039: 24.0.50; dired "f" fails on directory that begins with a space

[Randal L. Schwartz]

>>>>> "GM" == Glenn Morris <rgm@gnu.org> writes:

GM> Randal L. Schwartz wrote:
>> What I'm confused about is if setting the other switch always does the
>> right thing (and faster, without forking), why would dired *ever* use
>> "ls --dired"?

GM> In the doc of dired-use-ls-dired, I mention that ls-lisp does not
GM> support as many options as GNU ls.

So what features of dired break?  All I see in the docs for that var is:

    dired-use-ls-dired is a variable defined in `dired.el'.
    Its value is unspecified

    Documentation:
    Non-nil means Dired should use "ls --dired".
    The special value of `unspecified' means to check explicitly, and
    save the result in this variable.  This is performed the first
    time `dired-insert-directory' is called.

    You can customize this variable.

Ahh, in ls-lisp.el, I see

    ;; RESTRICTIONS ======================================================

    ;; * A few obscure ls switches are still ignored: see the docstring of
    ;; `insert-directory'.

    ;; TO DO =============================================================

    ;; Complete handling of F switch (if/when possible).

    ;; FJW: May be able to sort much faster by consing the sort key onto
    ;; the front of each list element, sorting and then stripping the key
    ;; off again!

Is that what you're talking about?

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.posterous.com/ for Smalltalk discussion

bug#9039: 24.0.50; dired "f" fails on directory that begins with a space

[Glenn Morris]

Randal L. Schwartz wrote:

> How so?  I have nothing to set it, and it comes up nil.  How is that not
> the defaul then?

Because it's probed for the first time you actually run dired, to see
whether your ls supports --dired. If it does, it gets set to t, if not,
to nil. If you do

C-h v dired-use-ls-dired

this is very obvious:

    dired-use-ls-dired is a variable defined in `dired.el'.
    Its value is t
    Original value was unspecified

    Documentation:
    Non-nil means Dired should use "ls --dired".
    The special value of `unspecified' means to check explicitly, and
    save the result in this variable.  This is performed the first
    time `dired-insert-directory' is called.

> Why isn't GNU Emacs using readdir() and stat() internally?  Why is it
> depending on an external ls?

I don't know. It's a long-standing design decision AFAIK. There
certainly has been no change in this regard for several years, since at
least Emacs 22.1.

bug#9039: 24.0.50; dired "f" fails on directory that begins with a space

[Manuel Giraud]

This issue may be already sorted out. But anyway, I have put these 2
lines in my .emacs and that does the trick:

(require 'ls-lisp)
(setq ls-lisp-use-insert-directory-program nil)

Now, dired will use a lisp implementation of ls (which in fact, uses
directory-files-and-attributes internally). It works for me on OpenBSD.

-- 
Manuel Giraud

bug#9039: 24.0.50; dired "f" fails on directory that begins with a space

[Randal L. Schwartz]

>>>>> "Manuel" == Manuel Giraud <manuel.giraud@univ-nantes.fr> writes:

Manuel> This issue may be already sorted out. But anyway, I have put these 2
Manuel> lines in my .emacs and that does the trick:

Manuel> (require 'ls-lisp)
Manuel> (setq ls-lisp-use-insert-directory-program nil)

Yeah, why isn't this the default?

-- 
Randal L. Schwartz - Stonehenge Consulting Services, Inc. - +1 503 777 0095
<merlyn@stonehenge.com> <URL:http://www.stonehenge.com/merlyn/>
Smalltalk/Perl/Unix consulting, Technical writing, Comedy, etc. etc.
See http://methodsandmessages.posterous.com/ for Smalltalk discussion

bug#9039: 24.0.50; dired "f" fails on directory that begins with a space

[Eli Zaretskii]

> From: merlyn@stonehenge.com (Randal L. Schwartz)
> Cc: eliz@gnu.org,  rgm@gnu.org,  9039@debbugs.gnu.org
> Date: Wed, 13 Jul 2011 11:05:35 -0700
> 
> Manuel> (require 'ls-lisp)
> Manuel> (setq ls-lisp-use-insert-directory-program nil)
> 
> Yeah, why isn't this the default?

Because using `ls' is faster, and because `ls' support options that
ls-lisp.el does not.