Newlines in file names

Newlines in file names

[Whitfield Diffie]

Question: How do you translate a filename containing a newline to one containing
          a \n without getting a \\n.

Problem: Create a directory ``test''.

         In the directory create a file with (call-process "touch" nil
nil nil "Icon\n").
         This file's name has five characters of which the last is a
newline. (Such files
         appear in some downloads.)

         This directory now appears as

                /Users/diffie/test:
                total used in directory 0 available 327738716
                drwxr-xr-x   3 diffie  staff  102 Dec 10 11:04 .
                drwxr-xr-x  25 diffie  staff  850 Dec 10 11:03 ..
                -rw-r--r--   1 diffie  staff    0 Dec  9 10:01 Icon

         Place the cursor on the line with the Icon file and type

                <esc>: (file-exists-p (dired get filename))

         The response is ``nil''.

         Make the buffer writable with <ctrl-x><ctrl-q> and edit the
         name by hand to ``Icon\n''.  Type

                <esc>: (file-exists-p (dired-get-filename))

         The response is now ``t''.

         Type g to revert the buffer and the name returns to being ``Icon''.

         Place the cursor on the line with the Icon file again and type

                M+x

         The entire file line disappears leaving only . and ..  in the
directory.

         Type g to revert the buffer and the line reappears but the file
         has not become executable.

         Edit the filename as before and repeat the attempt to make it
executable.

         Now the file line becomes

                -rwxr-xr-x   1 diffie  staff    0 Dec  9 10:01 Icon

         The file has become executable but its name has returned to being
         ``Icon'', lacks the newline character.

Objective: Repair dired so that filenames containing newlines are displayed
           with \n in place of newline characters (and \r in place of carriage
           returns).

           I have tried setting print-escape-newlines to t and recoding
           filename with buffer-file-coding-system, file-name-coding-system,
           and default-file-name-coding-system, without success.


                                             Whit

RE: Newlines in file names

[Doug Lewan]

What version of emacs are you using?

I don't have such problems in version 24, but I do remember dired using a regular expression that wouldn't match such uncommon names in earlier versions.

,Douglas
Douglas Lewan
Shubert Ticketing
(201) 489-8600 ext 224

When I do good, I feel good. When I do bad, I feel bad and that's my religion. - Abraham Lincoln


> -----Original Message-----
> From: help-gnu-emacs-bounces+dougl=shubertticketing.com@gnu.org
> [mailto:help-gnu-emacs-bounces+dougl=shubertticketing.com@gnu.org] On
> Behalf Of Whitfield Diffie
> Sent: Monday, 2012 December 10 14:48
> To: help-gnu-emacs@gnu.org
> Subject: Newlines in file names
> 
> Question: How do you translate a filename containing a newline to one
> containing
>           a \n without getting a \\n.
> 
> Problem: Create a directory ``test''.
> 
>          In the directory create a file with (call-process "touch" nil
> nil nil "Icon\n").
>          This file's name has five characters of which the last is a
> newline. (Such files
>          appear in some downloads.)
> 
>          This directory now appears as
> 
>                 /Users/diffie/test:
>                 total used in directory 0 available 327738716
>                 drwxr-xr-x   3 diffie  staff  102 Dec 10 11:04 .
>                 drwxr-xr-x  25 diffie  staff  850 Dec 10 11:03 ..
>                 -rw-r--r--   1 diffie  staff    0 Dec  9 10:01 Icon
> 
>          Place the cursor on the line with the Icon file and type
> 
>                 <esc>: (file-exists-p (dired get filename))
> 
>          The response is ``nil''.
> 
>          Make the buffer writable with <ctrl-x><ctrl-q> and edit the
>          name by hand to ``Icon\n''.  Type
> 
>                 <esc>: (file-exists-p (dired-get-filename))
> 
>          The response is now ``t''.
> 
>          Type g to revert the buffer and the name returns to being
> ``Icon''.
> 
>          Place the cursor on the line with the Icon file again and type
> 
>                 M+x
> 
>          The entire file line disappears leaving only . and ..  in the
> directory.
> 
>          Type g to revert the buffer and the line reappears but the
> file
>          has not become executable.
> 
>          Edit the filename as before and repeat the attempt to make it
> executable.
> 
>          Now the file line becomes
> 
>                 -rwxr-xr-x   1 diffie  staff    0 Dec  9 10:01 Icon
> 
>          The file has become executable but its name has returned to
> being
>          ``Icon'', lacks the newline character.
> 
> Objective: Repair dired so that filenames containing newlines are
> displayed
>            with \n in place of newline characters (and \r in place of
> carriage
>            returns).
> 
>            I have tried setting print-escape-newlines to t and recoding
>            filename with buffer-file-coding-system, file-name-coding-
> system,
>            and default-file-name-coding-system, without success.
> 
> 
>                                              Whit

Re: Newlines in file names

[Whitfield Diffie]

> What version of emacs are you using?

    22.1.2.  I apologize for not including that; it hit me that I
hadn't just after I had sent the message.

> I don't have such problems in version 24, but I do remember dired using a regular expression that wouldn't match such uncommon names in earlier versions.

    I think you mean a string used in a dired-string-replace-match in
dired-get-filename to remove quotes that come in from ls.  That filter
solves some bad filenames bit I didn't succeed in diddling it to solve
the newline problem.


                                                     Whit

RE: Newlines in file names

[Doug Lewan]

> -----Original Message-----
> From: whitfielddiffie@gmail.com [mailto:whitfielddiffie@gmail.com] On
> Behalf Of Whitfield Diffie
> Sent: Tuesday, 2012 December 11 10:19
> To: Doug Lewan
> Cc: help-gnu-emacs@gnu.org
> Subject: Re: Newlines in file names
> 
> > What version of emacs are you using?
> 
>     22.1.2.  I apologize for not including that; it hit me that I
> hadn't just after I had sent the message.
> 
> > I don't have such problems in version 24, but I do remember dired
> using a regular expression that wouldn't match such uncommon names in
> earlier versions.
> 
>     I think you mean a string used in a dired-string-replace-match in
> dired-get-filename to remove quotes that come in from ls.  That filter
> solves some bad filenames bit I didn't succeed in diddling it to solve
> the newline problem.

That's probably what I meant.

I remember having problems exactly with filenames that ended with newline. I also remember dealing with them by hand every time. Enough to annoy. Infrequent enough not to program against. Yuck.

> 
> 
>                                                      Whit

,Douglas
Douglas Lewan
Shubert Ticketing
(201) 489-8600 ext 224

When I do good, I feel good. When I do bad, I feel bad and that's my religion. - Abraham Lincoln

bug#13165: Fwd: Newlines in file names

[Whitfield Diffie]

    Since this didn't attract much attention as a request for help,
let me try it as a bug report.

    In emacs 22.1.1, if you have a file whose name contains a carriage
return and put the cursor on the line with that file in dired, then
(file-exists-p (dired-get-filename)) is nil.

                                               Whit


---------- Forwarded message ----------
From: Whitfield Diffie <whitfield.diffie@gmail.com>
Date: Mon, Dec 10, 2012 at 11:47 AM
Subject: Newlines in file names
To: help-gnu-emacs@gnu.org


Question: How do you translate a filename containing a newline to one
containing a \n without getting a \\n.

Problem: Create a directory ``test''.

         In the directory create a file with

                 (call-process "touch" nil nil nil "Icon\n").

         This file's name has five characters of which the last is a
         newline. (Such files appear in some downloads.)

         This directory now appears as

                /Users/diffie/test:
                total used in directory 0 available 327738716
                drwxr-xr-x   3 diffie  staff  102 Dec 10 11:04 .
                drwxr-xr-x  25 diffie  staff  850 Dec 10 11:03 ..
                -rw-r--r--   1 diffie  staff    0 Dec  9 10:01 Icon

         Place the cursor on the line with the Icon file and type

                <esc>: (file-exists-p (dired get filename))

         The response is ``nil''.

         Make the buffer writable with <ctrl-x><ctrl-q> and edit the
         name by hand to ``Icon\n''.  Type

                <esc>: (file-exists-p (dired-get-filename))

         The response is now ``t''.

         Type g to revert the buffer and the name returns to being ``Icon''.

         Place the cursor on the line with the Icon file again and type

                M+x

         The entire file line disappears leaving only . and ..  in the
          directory.

         Type g to revert the buffer and the line reappears but the file
         has not become executable.

         Edit the filename as before and repeat the attempt to make it
         executable.

         Now the file line becomes

                -rwxr-xr-x   1 diffie  staff    0 Dec  9 10:01 Icon

         The file has become executable but its name has returned to being
         ``Icon'', lacks the newline character.

Objective: Repair dired so that filenames containing newlines are displayed
           with \n in place of newline characters (and \r in place of carriage
           returns).

           I have tried setting print-escape-newlines to t and recoding
           filename with buffer-file-coding-system, file-name-coding-system,
           and default-file-name-coding-system, without success.


                                             Whit

bug#13165: Fwd: Newlines in file names

[Glenn Morris]

Whitfield Diffie wrote:

>     Since this didn't attract much attention as a request for help,
> let me try it as a bug report.
>
>     In emacs 22.1.1, if you have a file whose name contains a carriage
> return and put the cursor on the line with that file in dired, then
> (file-exists-p (dired-get-filename)) is nil.

It works fine if you add -b to dired-listing-switches.

  emacs-22.1 -Q --eval '(setq dired-listing-switches "-alb")'

This was a suggestion when this was a help request:

http://lists.gnu.org/archive/html/help-gnu-emacs/2012-12/msg00133.html

I see no prospect of Dired handling file names with newlines without the
-b switch in use. Note there are some issues with -b that are fixed in
newer versions of Emacs than the one you are using, eg

http://debbugs.gnu.org/10469
http://debbugs.gnu.org/10596

bug#13165: Fwd: Newlines in file names

[Whitfield Diffie]

>>     In emacs 22.1.1, if you have a file whose name contains a carriage
>> return and put the cursor on the line with that file in dired, then
>> (file-exists-p (dired-get-filename)) is nil.
>
> It works fine if you add -b to dired-listing-switches.

    I find it hard to see (null (file-exists-p (dired-get-filename)))
as anything but a bug; if -b didn't put the filenames in the right
form, dired should have done so.

    I am, however, very grateful to you for setting me on the right
path.  My problem was code in dired-insert-directory that reformats
filenames if it does not find b in the listing-switches.  Your message
put me on the right track and I have solved the problem.  Thank you.

    What I am working with is a substantially revised dired, e.g.,

 /Users/diffie/system/emacs/local/dired-mods:

  dr-x   10  29Oct12 17:10:33  dired-sort-halves.el
  dr-x   16  27Oct12 18:16:51  dired-sorting.el

  -r-- 2006  22Apr09 07:41:09  directory-needs-reversion-p.el
  -r-- 4714   7Feb12 20:19:43  dired-add-entry.el
  -r--  418  20Jun06 22:16:47  dired-add-file.el
  -r--  939  18Mar07 21:06:53  dired-add-zero-in-filename.el
  -r--  594  30Nov11 11:06:45  dired-approximate-position.el
  -r--  231   7Feb12 19:58:01  dired-at-headerline.el

in which there are numerous format switches.  (Anyone who is
interested,  is naturally welcome to it; I haven't compared it with
dired in later versions of Emacs.)

>   emacs-22.1 -Q --eval '(setq dired-listing-switches "-alb")'
> This was a suggestion when this was a help request:

    If it was a response to my help request, it didn't reach me.


> I see no prospect of Dired handling file names with newlines without the
> -b switch in use.

    My dired doesn't depend on ls.  I does use it at present but I
think directory-files would do just as well.


                                       Thanks again,


                                                             Whit

bug#13165: Fwd: Newlines in file names

[Whitfield Diffie]

>     I find it hard to see (null (file-exists-p (dired-get-filename)))
> as anything but a bug; if -b didn't put the filenames in the right
> form, dired should have done so.

    Moby sorry: I meant ``if ls didn't put the filenames in the right form...''


                                    Whit

bug#13165: Fwd: Newlines in file names

[Stefan Monnier]

>> I find it hard to see (null (file-exists-p (dired-get-filename)))
>> as anything but a bug; if ls didn't put the filenames in the right
>> form, dired should have done so.

I don't understand what you're saying here.  How could dired guess what
is the right form if ls doesn't provide it?  Tho, I guess that with
"ls --dired" the extra data returned by ls could arguably provide the
needed info.


        Stefan