bug#699: grep in eshell incorrect

bug#699: grep in eshell incorrect

[xah lee]

The “grep” utility in “eshell” is incorrect. Here's is a  
eshell log that illustrate it:

------------------------------

Welcome to the Emacs shell

~/web/emacs $ which grep
eshell/grep is a compiled Lisp function in `em-unix.el'
~/web/emacs $ ls | grep lin
elisp_link_report.el                        modernization.html
elisp_link_report.el~
modernization_html_mail.html
elisp_link_report.html
modernization_meta_key.html
elisp_make_link.html                        ms_keyboard
emacs_installing_packages.html              wrap-url2.html.gz
emacs_manual_problem.html                   xah_emacs_linkify.el
emacs_n_unicode.html                        xah_emacs_linkify.el~
~/web/emacs $

--------------------------

   Xah
∑ http://xahlee.org/

☄

bug#699: grep in eshell incorrect

[Eli Zaretskii]

> From: xah lee <xah@xahlee.org>
> Date: Tue, 12 Aug 2008 00:01:04 -0700
> Cc: 
> 
> 
> The “grep” utility in “eshell” is incorrect. Here's is a  
> eshell log that illustrate it:
> 
> ------------------------------
> 
> Welcome to the Emacs shell
> 
> ~/web/emacs $ which grep
> eshell/grep is a compiled Lisp function in `em-unix.el'
> ~/web/emacs $ ls | grep lin
> elisp_link_report.el                        modernization.html
> elisp_link_report.el~
> modernization_html_mail.html
> elisp_link_report.html
> modernization_meta_key.html
> elisp_make_link.html                        ms_keyboard
> emacs_installing_packages.html              wrap-url2.html.gz
> emacs_manual_problem.html                   xah_emacs_linkify.el
> emacs_n_unicode.html                        xah_emacs_linkify.el~
> ~/web/emacs $

Would you please care to explain what is wrong with it?  As far as I
see, each line has a "lin" substring somewhere in it (modulo the
evident line wrapping).

bug#699: grep in eshell incorrect

[xah lee]

> Would you please care to explain what is wrong with it?  As far as I
> see, each line has a "lin" substring somewhere in it (modulo the
> evident line wrapping).


Thanks. I never realized that.

Anyway, i thought this is unexpected behavior. For example, on mac os  
x's teminal, it would give this:

[xah@xahg5][Tue Aug 12 11:37:38][~/web/emacs]
ls | grep lin
elisp_link_report.el
elisp_link_report.el~
elisp_link_report.html
elisp_make_link.html
emacs_installing_packages.html
xah_emacs_linkify.el
xah_emacs_linkify.el~
[xah@xahg5][Tue Aug 12 11:37:40][~/web/emacs]


So, possibly the eshell's "ls" command forming multiple columns is a  
bad design. Because after all it is supposed to emulate the unix  
shell behavior and "ls | grep xyz" is common idiom.

   Xah
∑ http://xahlee.org/

☄

On Aug 12, 2008, at 11:27 AM, Eli Zaretskii wrote:

> From: xah lee <xah@xahlee.org>
> Date: Tue, 12 Aug 2008 00:01:04 -0700
> Cc:
>
>
> The “grep” utility in “eshell” is incorrect. Here's is a
> eshell log that illustrate it:
>
> ------------------------------
>
> Welcome to the Emacs shell
>
> ~/web/emacs $ which grep
> eshell/grep is a compiled Lisp function in `em-unix.el'
> ~/web/emacs $ ls | grep lin
> elisp_link_report.el                        modernization.html
> elisp_link_report.el~
> modernization_html_mail.html
> elisp_link_report.html
> modernization_meta_key.html
> elisp_make_link.html                        ms_keyboard
> emacs_installing_packages.html              wrap-url2.html.gz
> emacs_manual_problem.html                   xah_emacs_linkify.el
> emacs_n_unicode.html                        xah_emacs_linkify.el~
> ~/web/emacs $

Would you please care to explain what is wrong with it?  As far as I
see, each line has a "lin" substring somewhere in it (modulo the
evident line wrapping).

☄

bug#699: grep in eshell incorrect

[Eli Zaretskii]

> Cc: 699@emacsbugs.donarmstrong.com,
>  bug-gnu-emacs@gnu.org
> From: xah lee <xah@xahlee.org>
> Date: Tue, 12 Aug 2008 11:40:25 -0700
> 
> So, possibly the eshell's "ls" command forming multiple columns is a  
> bad design. Because after all it is supposed to emulate the unix  
> shell behavior and "ls | grep xyz" is common idiom.

Yes, it _is_ supposed to emulate a Unix shell.  But I just tried that
same pipe both on GNU/Linux and on MS-Windows, and in both cases "ls"
still forms multiple columns even in a pipe.  So I don't see anything
wrong with how Eshell's "ls" works.

bug#699: grep in eshell incorrect

[xah lee]

Ok. I just did ls, and yes it forms multi columns.

However, when i do

ls | grep xyz

it doesn't produce the result eshell would. i.e. the multi column  
produced by ls seems to be a done as display formatting, not as its  
real output. I might be wrong... but i mean the bottom line is that "  
ls | grep some" have different behavior in unix shell and eshell.

   Xah
∑ http://xahlee.org/

☄

On Aug 12, 2008, at 11:54 AM, Eli Zaretskii wrote:

> Cc: 699@emacsbugs.donarmstrong.com,
>  bug-gnu-emacs@gnu.org
> From: xah lee <xah@xahlee.org>
> Date: Tue, 12 Aug 2008 11:40:25 -0700
>
> So, possibly the eshell's "ls" command forming multiple columns is a
> bad design. Because after all it is supposed to emulate the unix
> shell behavior and "ls | grep xyz" is common idiom.

Yes, it _is_ supposed to emulate a Unix shell.  But I just tried that
same pipe both on GNU/Linux and on MS-Windows, and in both cases "ls"
still forms multiple columns even in a pipe.  So I don't see anything
wrong with how Eshell's "ls" works.

☄

bug#699: grep in eshell incorrect

[Eli Zaretskii]

> Cc: 699@emacsbugs.donarmstrong.com,
>  bug-gnu-emacs@gnu.org
> From: xah lee <xah@xahlee.org>
> Date: Tue, 12 Aug 2008 12:03:55 -0700
> 
> Ok. I just did ls, and yes it forms multi columns.
> 
> However, when i do
> 
> ls | grep xyz
> 
> it doesn't produce the result eshell would. i.e. the multi column  
> produced by ls seems to be a done as display formatting, not as its  
> real output.

Sorry, I don't understand: what's the difference between ``display
formatting'' and ``real output''?  You don't see the output of `ls'
_except_ formatted by it.

> I might be wrong... but i mean the bottom line is that "  
> ls | grep some" have different behavior in unix shell and eshell.

And I see the _same_ behavior, modulo the different defaults about how
many columns to put on one line.

bug#699: grep in eshell incorrect

[xah lee]

for example, i want to use "ls | grep lin" to find file names  
containing "lin". In OSX, i do:


[xah@xahg5][Tue Aug 12 12:25:04][~/web/emacs]
ls | grep lin
elisp_link_report.el
elisp_link_report.el~
elisp_link_report.html
elisp_make_link.html
emacs_installing_packages.html
xah_emacs_linkify.el
xah_emacs_linkify.el~
[xah@xahg5][Tue Aug 12 12:25:10][~/web/emacs]

---------------------

in eshell, i get:

Welcome to the Emacs shell

/Users/xah/Documents/temp $ cd ~/web/emacs
~/web/emacs $ ls | grep lin
elisp_link_report.el                        modernization.html
elisp_link_report.el~                       modernization_html_mail.html
elisp_link_report.html                      modernization_meta_key.html
elisp_make_link.html                        ms_keyboard
emacs_installing_packages.html              wrap-url2.html.gz
emacs_manual_problem.html                   xah_emacs_linkify.el
emacs_n_unicode.html                        xah_emacs_linkify.el~
~/web/emacs $

---------------------

   Xah
∑ http://xahlee.org/

☄

On Aug 12, 2008, at 12:19 PM, Eli Zaretskii wrote:

> Cc: 699@emacsbugs.donarmstrong.com,
>  bug-gnu-emacs@gnu.org
> From: xah lee <xah@xahlee.org>
> Date: Tue, 12 Aug 2008 12:03:55 -0700
>
> Ok. I just did ls, and yes it forms multi columns.
>
> However, when i do
>
> ls | grep xyz
>
> it doesn't produce the result eshell would. i.e. the multi column
> produced by ls seems to be a done as display formatting, not as its
> real output.

Sorry, I don't understand: what's the difference between ``display
formatting'' and ``real output''?  You don't see the output of `ls'
_except_ formatted by it.

> I might be wrong... but i mean the bottom line is that "
> ls | grep some" have different behavior in unix shell and eshell.

And I see the _same_ behavior, modulo the different defaults about how
many columns to put on one line.

☄

bug#699: grep in eshell incorrect

[Glenn Morris]

severity 699 minor
stop


Just use 'ls -1' explicitly in eshell pipes for now.


Eli Zaretskii wrote:

> And I see the _same_ behavior, modulo the different defaults about how
> many columns to put on one line.

I do not.

ls --version
ls (GNU coreutils) 5.97

info ls

If standard output is a terminal, the output is in columns (sorted
vertically) and control characters are output as question marks;
otherwise, the output is listed one per line and control characters
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
are output as-is.

Processed: Re: bug#699: grep in eshell incorrect

[Emacs bug Tracking System]

Processing commands for control@emacsbugs.donarmstrong.com:

> severity 699 minor
bug#699: grep in eshell incorrect
Severity set to `minor' from `normal'

> stop
Stopping processing here.

Please contact me if you need assistance.

Don Armstrong
(administrator, Emacs bugs database)

bug#699: grep in eshell incorrect

[Glenn Morris]

Actually, it seems this is easy to fix.
(Except that now eshell/ls behaves differently to normal ls at the end
of pipeline, but that's probably less common.)

*** em-ls.el	7 Jun 2008 02:37:12 -0000	1.33
--- em-ls.el	13 Aug 2008 02:37:48 -0000
***************
*** 635,641 ****
    "Output a list of FILES.
  Each member of FILES is either a string or a cons cell of the form
  \(FILE .  ATTRS)."
!   (if (memq listing-style '(long-listing single-column))
        (eshell-for file files
  	(if file
  	    (eshell-ls-file file size-width copy-fileinfo)))
--- 635,642 ----
    "Output a list of FILES.
  Each member of FILES is either a string or a cons cell of the form
  \(FILE .  ATTRS)."
!   (if (or eshell-in-pipeline-p
! 	  (memq listing-style '(long-listing single-column)))
        (eshell-for file files
  	(if file
  	    (eshell-ls-file file size-width copy-fileinfo)))

bug#699: grep in eshell incorrect

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: 699@emacsbugs.donarmstrong.com,  xah lee <xah@xahlee.org>
> Date: Tue, 12 Aug 2008 15:59:05 -0400
> 
> info ls
> 
> If standard output is a terminal, the output is in columns (sorted
> vertically) and control characters are output as question marks;
> otherwise, the output is listed one per line and control characters
>                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Not with "ls -x".

bug#699: grep in eshell incorrect

[Glenn Morris]

Eli Zaretskii wrote:

>> If standard output is a terminal, the output is in columns (sorted
>> vertically) and control characters are output as question marks;
>> otherwise, the output is listed one per line and control characters
>>                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>
> Not with "ls -x".

Alrighty.

*** em-ls.el	7 Jun 2008 02:37:12 -0000	1.33
--- em-ls.el	13 Aug 2008 03:20:19 -0000
***************
*** 635,641 ****
    "Output a list of FILES.
  Each member of FILES is either a string or a cons cell of the form
  \(FILE .  ATTRS)."
!   (if (memq listing-style '(long-listing single-column))
        (eshell-for file files
  	(if file
  	    (eshell-ls-file file size-width copy-fileinfo)))
--- 635,646 ----
    "Output a list of FILES.
  Each member of FILES is either a string or a cons cell of the form
  \(FILE .  ATTRS)."
!   ;; Mimic behavior of coreutils ls, which lists a single file per
!   ;; line when output is not a tty.  Unless -x was supplied.
!   ;; FIXME we should not be doing this if we are at the _end_ of a pipeline.
!   (if (or (and (not (eq listing-style 'by-lines))
! 	       eshell-in-pipeline-p)
! 	  (memq listing-style '(long-listing single-column)))
        (eshell-for file files
  	(if file
  	    (eshell-ls-file file size-width copy-fileinfo)))

Re: bug#699: grep in eshell incorrect

[Ted Zlatanov]

On Tue, 12 Aug 2008 21:54:29 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 

>> Cc: 699@emacsbugs.donarmstrong.com,
>> bug-gnu-emacs@gnu.org
>> From: xah lee <xah@xahlee.org>
>> Date: Tue, 12 Aug 2008 11:40:25 -0700
>> 
>> So, possibly the eshell's "ls" command forming multiple columns is a  
>> bad design. Because after all it is supposed to emulate the unix  
>> shell behavior and "ls | grep xyz" is common idiom.

EZ> Yes, it _is_ supposed to emulate a Unix shell.  But I just tried that
EZ> same pipe both on GNU/Linux and on MS-Windows, and in both cases "ls"
EZ> still forms multiple columns even in a pipe.  So I don't see anything
EZ> wrong with how Eshell's "ls" works.

My tests show different results.  On my Ubuntu system:

% /bin/ls             
Makefile  README.txt

% /bin/ls |/bin/grep M
Makefile
README.txt

% /bin/ls |/bin/grep E
README.txt

So if it's piped (probably /bin/ls tests if the terminal is not
interactive) it acts as if the -1 option is set.  This is how it's
always worked for me, since I often do `ls -t | head -1` to find the
newest file in a directory.

Ted

bug#699: grep in eshell incorrect

[Eli Zaretskii]

> From: Ted Zlatanov <tzz@lifelogs.com>
> Date: Tue, 12 Aug 2008 15:18:55 -0500
> 
> On Tue, 12 Aug 2008 21:54:29 +0300 Eli Zaretskii <eliz@gnu.org> wrote: 
> 
> >> Cc: 699@emacsbugs.donarmstrong.com,
> >> bug-gnu-emacs@gnu.org
> >> From: xah lee <xah@xahlee.org>
> >> Date: Tue, 12 Aug 2008 11:40:25 -0700
> >> 
> >> So, possibly the eshell's "ls" command forming multiple columns is a  
> >> bad design. Because after all it is supposed to emulate the unix  
> >> shell behavior and "ls | grep xyz" is common idiom.
> 
> EZ> Yes, it _is_ supposed to emulate a Unix shell.  But I just tried that
> EZ> same pipe both on GNU/Linux and on MS-Windows, and in both cases "ls"
> EZ> still forms multiple columns even in a pipe.  So I don't see anything
> EZ> wrong with how Eshell's "ls" works.
> 
> My tests show different results.

Yes, you are right: I forgot that my `ls' is aliased to "ls -x".
Under -x, the pipe doesn't imply -1.

Sorry for the confusion.

bug#699: marked as done (grep in eshell incorrect)

[Emacs bug Tracking System]

[-- Attachment #1: Type: text/plain, Size: 822 bytes --]


Your message dated Fri, 15 Aug 2008 15:16:13 -0400
with message-id <orbpzugkci.fsf@fencepost.gnu.org>
and subject line Re: bug#699: grep in eshell incorrect
has caused the Emacs bug report #699,
regarding grep in eshell incorrect
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact don@donarmstrong.com
immediately.)


-- 
699: http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=699
Emacs Bug Tracking System
Contact don@donarmstrong.com with problems

[-- Attachment #2: Type: message/rfc822, Size: 3167 bytes --]

From: xah lee <xah@xahlee.org>
To: bug-gnu-emacs@gnu.org
Subject: grep in eshell incorrect
Date: Tue, 12 Aug 2008 00:01:04 -0700
Message-ID: <CFFB5D73-86BC-48D3-AACA-FB1FBF31197E@xahlee.org>


The “grep” utility in “eshell” is incorrect. Here's is a  
eshell log that illustrate it:

------------------------------

Welcome to the Emacs shell

~/web/emacs $ which grep
eshell/grep is a compiled Lisp function in `em-unix.el'
~/web/emacs $ ls | grep lin
elisp_link_report.el                        modernization.html
elisp_link_report.el~
modernization_html_mail.html
elisp_link_report.html
modernization_meta_key.html
elisp_make_link.html                        ms_keyboard
emacs_installing_packages.html              wrap-url2.html.gz
emacs_manual_problem.html                   xah_emacs_linkify.el
emacs_n_unicode.html                        xah_emacs_linkify.el~
~/web/emacs $

--------------------------

   Xah
∑ http://xahlee.org/

☄






[-- Attachment #3: Type: message/rfc822, Size: 1866 bytes --]

From: Glenn Morris <rgm@gnu.org>
To: 699-done@emacsbugs.donarmstrong.com
Subject: Re: bug#699: grep in eshell incorrect
Date: Fri, 15 Aug 2008 15:16:13 -0400
Message-ID: <orbpzugkci.fsf@fencepost.gnu.org>


Fixed in 22 and trunk.

2008-08-15  Glenn Morris  <rgm@gnu.org>

    * eshell/em-ls.el (eshell-ls-files): List one per line in
      a pipeline. (Bug#699)