bug#22477: 24.5; Dired does not work if file names contain newlines

bug#22477: 24.5; Dired does not work if file names contain newlines

[Aura Kelloniemi]

1. Create a file which contains a newline character in its name.
2. Open the directory where the newly created file resides in dired.
3. Dired buffer is garbled and the cursor is positioned at the end of the
buffer. No files can be visited (whether they contain newlines in their names
or not) (Dired says: "No file on this line").

My ls version is: ls (GNU coreutils) 8.25
In my test directory the output of ls -l --dired looks like this:
  total 8
  -rw------- 1 aura users  2 Jan 27 13:37 ''\''Hello world'\''.txt'
  -rw------- 1 aura users 17 Jan 27 13:47 'Hello'$'\n''World.txt'
//DIRED// 52 77 120 143
//DIRED-OPTIONS// --quoting-style=shell-escape
      





In GNU Emacs 24.5.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.16.6)
 of 2015-09-10 on foutrelis
Configured using:
 `configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-x-toolkit=gtk3 --with-xft 'CFLAGS=-march=x86-64
 -mtune=generic -O2 -pipe -fstack-protector-strong --param=ssp-buffer-size=4'
 CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro'

Important settings:
  value of $LC_COLLATE: C
  value of $LANG: fi_FI.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Dired by name

Minor modes in effect:
  gpm-mouse-mode: t
  minibuffer-electric-default-mode: t
  icomplete-mode: t
  diff-auto-refine-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t

Recent messages:
[deleted, not necessary]

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug sendmail mule-util t-mouse w3m-filter w3m
browse-url doc-view jka-compr dired image-mode timezone w3m-hist w3m-fb
bookmark-w3m w3m-ems w3m-ccl ccl w3m-favicon w3m-image w3m-proc w3m-util
haskell-interactive-mode haskell-presentation-mode haskell-mode
haskell-indentation haskell-sort-imports haskell-lexeme rx
haskell-align-imports haskell-complete-module flymake dabbrev
haskell-font-lock haskell-collapse haskell-process haskell-session
haskell-string haskell-compat noutline outline etags url-util url-parse
auth-source eieio byte-opt bytecomp byte-compile cl-extra cconv eieio-core
gnus-util time-date url-vars json haskell-navigate-imports haskell-compile
haskell-cabal haskell-utils haskell-customize compile comint ansi-color ring
minibuf-eldef ido icomplete company pcase cus-start cus-load notmuch hl-line
notmuch-message notmuch-maildir-fcc notmuch-hello edmacro kmacro wid-edit
notmuch-tree notmuch-parser notmuch-show notmuch-print notmuch-crypto
notmuch-mua notmuch-address notmuch-wash diff-mode easy-mmode coolj
notmuch-query goto-addr thingatpt icalendar diary-lib diary-loaddefs cal-menu
calendar cal-loaddefs notmuch-tag crm notmuch-lib advice notmuch-version cl gv
cl-loaddefs cl-lib message idna format-spec rfc822 mml mailabbrev mail-utils
gmm-utils mailheader mm-view mml-smime mml-sec smime password-cache dig
mm-decode mm-bodies mm-encode mailcap mail-parse rfc2231 rfc2047 rfc2045
ietf-drums mm-util help-fns mail-prsvr dired-single align info easymenu
package epg-config warnings server tooltip electric uniquify ediff-hook
vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list newcomment lisp-mode prog-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese tibetan
thai tai-viet lao korean japanese hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook help simple
abbrev minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule custom
widget hashtable-print-readable backquote make-network-process dbusbind
gfilenotify dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty emacs)

Memory information:
((conses 16 231966 12461)
 (symbols 48 33370 0)
 (miscs 40 44 150)
 (strings 32 62167 11179)
 (string-bytes 1 1719435)
 (vectors 16 24064)
 (vector-slots 8 548856 5180)
 (floats 8 150 299)
 (intervals 56 273 24)
 (buffers 960 14)
 (heap 1024 46043 958))

-- 
Aura

bug#22477: 24.5; Dired does not work if file names contain newlines

[Glenn Morris]

Aura Kelloniemi wrote:

> 1. Create a file which contains a newline character in its name.

If you like to do weird things like that, adding "-b" to
dired-listing-switches helps.

bug#22477: 24.5; Dired does not work if file names contain newlines

[Aura Kelloniemi]

Glenn Morris <rgm@gnu.org> writes:

 > Aura Kelloniemi wrote:

 > > 1. Create a file which contains a newline character in its name.

 > If you like to do weird things like that, adding "-b" to
 > dired-listing-switches helps.

Oh no, I don't like to do such weird things. My point is not to nitpick or
irritate you. I'm just interested about completeness: if Unix supports file
names with control characters (which arguably is a questionable feature), then
Dired (being the best file manager on the planet) should IMHO be able to deal
with them, or at least document the incompatibility clearly.

Also I wonder if a cracker can confuse Dired by creating a file name like this
"Hello World.txt\ndrwx-r--r--  1 user users ..."

-b probably is not portable, because it's not Dired's default. But Dired could
 test for -b just like it tests for --dired. WDired should also support these
 escapes.

I tried to edit with WDired a symbolic link which points to a file containing
 the string "->" which is a real file on my system. (In music files I use ->
 to separate the performer and composition names.) This did not work either.
 WDired failed with error "wdired-get-filename: Wrong type argument:
 integer-or-marker-p, nil".
 
I noticed too, that the -F switch to ls makes Dired fail the same way as file
names with embedded newlines. It was hard to google for this issue, others
have certainly faced it too. But (again IMHO) -F shouldn't break Dired, or at
least there should be another way to have Dired put a slash at the end of
directory names.

I wish you could do something to these issues. If you can't extend Dired's and
WDired's escaping support, you could at least document the issues.

Anyway, thank you for reading my complaints.

-- 
Aura

bug#22477: 24.5; Dired does not work if file names contain newlines

[John Wiegley]

>>>>> Aura Kelloniemi <kaura.dev@sange.fi> writes:

> If Unix supports file names with control characters (which arguably is a
> questionable feature), then Dired (being the best file manager on the
> planet) should IMHO be able to deal with them, or at least document the
> incompatibility clearly.

Sounds like a reasonable request. If we don't have the feature yet, adding a
comment to the dired documentation on the use -b as a workaround would be
good.

-- 
John Wiegley                  GPG fingerprint = 4710 CF98 AF9B 327B B80F
http://newartisans.com                          60E1 46C4 BD1A 7AC1 4BA2

bug#22477: 24.5; Dired does not work if file names contain newlines

[Lars Ingebrigtsen]

Glenn Morris <rgm@gnu.org> writes:

> Aura Kelloniemi wrote:
>
>> 1. Create a file which contains a newline character in its name.
>
> If you like to do weird things like that, adding "-b" to
> dired-listing-switches helps.

The reason characters like that are problematic is because we're using
"ls" and then parsing the output, I guess?  Is there any particular
reason dired is still doing that?  Is the ls-lisp.el implementation
noticeably slower still?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#22477: 24.5; Dired does not work if file names contain newlines

[Andreas Schwab]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> The reason characters like that are problematic is because we're using
> "ls" and then parsing the output, I guess?

dired uses ls --dired which is unambiguous.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#22477: 24.5; Dired does not work if file names contain newlines

[Aura Kelloniemi]

Lars Ingebrigtsen <larsi@gnus.org> writes:

 > Glenn Morris <rgm@gnu.org> writes:

 > > Aura Kelloniemi wrote:
 > >
 > >> 1. Create a file which contains a newline character in its name.
 > >
 > > If you like to do weird things like that, adding "-b" to
 > > dired-listing-switches helps.

 > The reason characters like that are problematic is because we're using
 > "ls" and then parsing the output, I guess?  Is there any particular
 > reason dired is still doing that?  Is the ls-lisp.el implementation
 > noticeably slower still?

According to the Emacs manual, ls-lisp is lacking some features that the ls
binary generally has.

What comes to special characters, ls supports many quoting styles, which Emacs
could implement (at least GNU ls supports them).

But the most portable, secure and configurable solution would be to enhance
(or rewrite) ls-lisp. It could support additional features which cannot be
very well implemented when using read ls, like allowing the user to control
the format of printed directory entries (I personally would like the file
permissions and other attributes to be shown on the right edge of the window
and the file name on the left, not the other way around as with 'ls -l').

-- 
Aura