Examining the output of a shell command?

Examining the output of a shell command?

[Raffaele Ricciardi]

I would like to analyse the escape sequences emitted by a shell
command. I have tried with `shell-command` and
`shell-command-to-string`, but escape sequences are stripped away.

Thank you.

Re: Examining the output of a shell command?

[Pascal J. Bourguignon]

Raffaele Ricciardi <rfflrccrd@gmail.com> writes:

> I would like to analyse the escape sequences emitted by a shell
> command. I have tried with `shell-command` and
> `shell-command-to-string`, but escape sequences are stripped away.

Nope. Try a better lie.

(shell-command-to-string "echo $RED hello $NORMAL")
--> "[31m hello [0m
"

-- 
__Pascal Bourguignon__                 http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk

Re: Examining the output of a shell command?

[Dan Espen]

"Pascal J. Bourguignon" <pjb@informatimago.com> writes:

> Raffaele Ricciardi <rfflrccrd@gmail.com> writes:
>
>> I would like to analyse the escape sequences emitted by a shell
>> command. I have tried with `shell-command` and
>> `shell-command-to-string`, but escape sequences are stripped away.
>
> Nope. Try a better lie.
>
> (shell-command-to-string "echo $RED hello $NORMAL")
> --> "[31m hello [0m
> "

Maybe the OP doesn't realize that some programs that output
escape sequences won't do so when run as an Emacs shell command.

/bin/ls being a good example.

In that case, run:

/bin/ls --color

as an Emacs shell command.


-- 
Dan Espen

Re: Examining the output of a shell command?

[Michael Heerdegen]

Dan Espen <despen@verizon.net> writes:

> Maybe the OP doesn't realize that some programs that output
> escape sequences won't do so when run as an Emacs shell command.
>
> /bin/ls being a good example.
>
> In that case, run:
>
> /bin/ls --color
>
> as an Emacs shell command.

I guess programs like "ls" check the TERM environment variable.

Try:

(setenv "TERM" "xterm")

(shell-command-to-string "ls --color")


Michael.

Re: Examining the output of a shell command?

[Raffaele Ricciardi]

On 28/06/15 15:57, Pascal J. Bourguignon wrote:
> (shell-command-to-string "echo $RED hello $NORMAL")
> --> "[31m hello [0m
> "

Here is:

(shell-command-to-string "echo $RED hello $NORMAL")
--> "hello
"

Tried with GNU Emacs 24.5.1:
  - `emacs -Q` (started from a shell where $RED and $NORMAL are defined)
  - M-: (shell-command-to-string "echo $RED hello $NORMAL") RET

Tried with `emacs -nw -Q`, too, in case was a GUI problem.

Re: Examining the output of a shell command?

[Eli Zaretskii]

> From: Raffaele Ricciardi <rfflrccrd@gmail.com>
> Date: Sun, 28 Jun 2015 16:31:29 +0200
> 
> On 28/06/15 15:57, Pascal J. Bourguignon wrote:
> > (shell-command-to-string "echo $RED hello $NORMAL")
> > --> "[31m hello [0m
> > "
> 
> Here is:
> 
> (shell-command-to-string "echo $RED hello $NORMAL")
> --> "hello
> "

Check your shell init files.

Re: Examining the output of a shell command?

[Raffaele Ricciardi]

On 28/06/15 16:55, Eli Zaretskii wrote:
>> (shell-command-to-string "echo $RED hello $NORMAL")
>> --> "hello
>> "
>
> Check your shell init files.

I did that already:

    $ echo $RED
    \[\e[1;31m

Another puzzling behaviour:

    $ echo $RED
    \[\e[1;31m
    $ emacs -nw -Q
    M-: (getenv "RED") RET
    --> nil

What's up?  I am using Bash 4.3.30.

Re: Examining the output of a shell command?

[Eli Zaretskii]

> From: Raffaele Ricciardi <rfflrccrd@gmail.com>
> Date: Sun, 28 Jun 2015 17:03:52 +0200
> 
> On 28/06/15 16:55, Eli Zaretskii wrote:
> >> (shell-command-to-string "echo $RED hello $NORMAL")
> >> --> "hello
> >> "
> >
> > Check your shell init files.
> 
> I did that already:
> 
>     $ echo $RED
>     \[\e[1;31m
> 
> Another puzzling behaviour:
> 
>     $ echo $RED
>     \[\e[1;31m
>     $ emacs -nw -Q
>     M-: (getenv "RED") RET
>     --> nil

Did you "export RED"?

Anyway, what I wanted to point out was that when you invoke the shell
from Emacs, it is a non-interactive session, which might get different
settings.

Re: Examining the output of a shell command?

[Raffaele Ricciardi]

On 28/06/15 17:43, Eli Zaretskii wrote:
>> Another puzzling behaviour:
>>
>>      $ echo $RED
>>      \[\e[1;31m
>>      $ emacs -nw -Q
>>      M-: (getenv "RED") RET
>>      --> nil
>
> Did you "export RED"?

This.  I didn't.

> Anyway, what I wanted to point out was that when you invoke the shell
> from Emacs, it is a non-interactive session, which might get different
> settings.

Right.  I will look into it.

Re: Examining the output of a shell command?

[Pascal J. Bourguignon]

Raffaele Ricciardi <rfflrccrd@gmail.com> writes:

> On 28/06/15 15:57, Pascal J. Bourguignon wrote:
>> (shell-command-to-string "echo $RED hello $NORMAL")
>> --> "[31m hello [0m
>> "
>
> Here is:
>
> (shell-command-to-string "echo $RED hello $NORMAL")
> --> "hello
> "

Of course, you need to define RED and NORMAL in your .bashrc!
You may do so by sourcing: 
http://git.informatimago.com/viewgit/?a=viewblob&p=public/bin&h=5cd80cb6bfe405bcc4f9cf7c4f857845b31d1830&hb=08a0ed1597b44dbb32057924837b1a1aeb4a8d61&f=ansi-codes


-- 
__Pascal Bourguignon__                 http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk

Re: Examining the output of a shell command?

[Dan Espen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Dan Espen <despen@verizon.net> writes:
>
>> Maybe the OP doesn't realize that some programs that output
>> escape sequences won't do so when run as an Emacs shell command.
>>
>> /bin/ls being a good example.
>>
>> In that case, run:
>>
>> /bin/ls --color
>>
>> as an Emacs shell command.
>
> I guess programs like "ls" check the TERM environment variable.
>
> Try:
>
> (setenv "TERM" "xterm")

I doubt $TERM is the key.
/bin/ls knows when it is interactive.

> (shell-command-to-string "ls --color")

This will show escape sequences regardless of $TERM.
It did for me with the Emacs default value "dumb".

-- 
Dan Espen

Re: Examining the output of a shell command?

[Michael Heerdegen]

Dan Espen <despen@verizon.net> writes:

> > (shell-command-to-string "ls --color")
>
> This will show escape sequences regardless of $TERM.
> It did for me with the Emacs default value "dumb".

Mmh, not for me.  I don't know much about this.


Michael.

Re: Examining the output of a shell command?

[Barry Margolin]

In article <mmpa5e$ma0$1@dont-email.me>, Dan Espen <despen@verizon.net> 
wrote:

> Michael Heerdegen <michael_heerdegen@web.de> writes:
> 
> > Dan Espen <despen@verizon.net> writes:
> >
> >> Maybe the OP doesn't realize that some programs that output
> >> escape sequences won't do so when run as an Emacs shell command.
> >>
> >> /bin/ls being a good example.
> >>
> >> In that case, run:
> >>
> >> /bin/ls --color
> >>
> >> as an Emacs shell command.
> >
> > I guess programs like "ls" check the TERM environment variable.
> >
> > Try:
> >
> > (setenv "TERM" "xterm")
> 
> I doubt $TERM is the key.
> /bin/ls knows when it is interactive.

What they often check is whether stdout is connected to a terminal.

> > (shell-command-to-string "ls --color")
> 
> This will show escape sequences regardless of $TERM.
> It did for me with the Emacs default value "dumb".

-- 
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***

Re: Examining the output of a shell command?

[Emanuel Berg]

"Pascal J. Bourguignon" <pjb@informatimago.com>
writes:

> Of course, you need to define RED and NORMAL in your
> .bashrc!

Try this:

    $ export NORMAL="[0m"
    $ emacs -Q --batch --eval '(message (shell-command-to-string "echo $NORMAL"))'

    [0m

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Examining the output of a shell command?

[Pascal J. Bourguignon]

Emanuel Berg <embe8573@student.uu.se> writes:

> "Pascal J. Bourguignon" <pjb@informatimago.com>
> writes:
>
>> Of course, you need to define RED and NORMAL in your
>> .bashrc!
>
> Try this:
>
>     $ export NORMAL="[0m"
>     $ emacs -Q --batch --eval '(message (shell-command-to-string "echo $NORMAL"))'
>
>     [0m

Well you need to insert an escape: 

   export NORMAL="[0m"

Type C-q ESC in emacs, or C-v ESC in the terminal, before [0m.

-- 
__Pascal Bourguignon__                 http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk

Re: Examining the output of a shell command?

[Raffaele Ricciardi]

On 28/06/15 15:57, Pascal J. Bourguignon wrote:
 > (shell-command-to-string "echo $RED hello $NORMAL")
 > --> "[31m hello [0m> "

Hence, how could I capture a colorized output in Emacs as in a Bash
session?

Below is a Bash script that recursively searches "*.el" files for the
string "defun" and highlights matches (the complex `find` call is the
shell command generated by `M-x rgrep` for the same task):

     #!/bin/bash

     export GREP_COLORS="mt=01;31:fn=:ln=:bn=:se=:sl=:cx=:ne"
     export GREP_OPTIONS="--color=auto"
     find . -type d \( -path \*/SCCS -o -path \*/RCS -o -path \*/CVS -o 
-path \*/MCVS -o -path \*/.svn -o -path \*/.git -o -path \*/.hg -o -path 
\*/.bzr -o -path \*/_MTN -o -path \*/_darcs -o -path \*/\{arch\} \) 
-prune -o \! -type d \( -name \*_flymake.\* -o -name .\#\* -o -name \*.o 
-o -name \*\~ -o -name \*.bin -o -name \*.lbin -o -name \*.so -o -name 
\*.a -o -name \*.ln -o -name \*.blg -o -name \*.bbl -o -name \*.elc -o 
-name \*.lof -o -name \*.glo -o -name \*.idx -o -name \*.lot -o -name 
\*.fmt -o -name \*.tfm -o -name \*.class -o -name \*.fas -o -name \*.lib 
-o -name \*.mem -o -name \*.x86f -o -name \*.sparcf -o -name \*.dfsl -o 
-name \*.pfsl -o -name \*.d64fsl -o -name \*.p64fsl -o -name \*.lx64fsl 
-o -name \*.lx32fsl -o -name \*.dx64fsl -o -name \*.dx32fsl -o -name 
\*.fx64fsl -o -name \*.fx32fsl -o -name \*.sx64fsl -o -name \*.sx32fsl 
-o -name \*.wx64fsl -o -name \*.wx32fsl -o -name \*.fasl -o -name 
\*.ufsl -o -name \*.fsl -o -name \*.dxl -o -name \*.lo -o -name \*.la -o 
-name \*.gmo -o -name \*.mo -o -name \*.toc -o -name \*.aux -o -name 
\*.cp -o -name \*.fn -o -name \*.ky -o -name \*.pg -o -name \*.tp -o 
-name \*.vr -o -name \*.cps -o -name \*.fns -o -name \*.kys -o -name 
\*.pgs -o -name \*.tps -o -name \*.vrs -o -name \*.pyc -o -name \*.pyo 
\) -prune -o -type f \( -iname \*.el \) -exec grep -i -n -e defun 
/dev/null {} +

I have tested the above Bash script with `bash --noprofile --norc` as
well, to disable any initialization.  As far as i can tell, Emacs is
running Bash -- `(getenv "SHELL")` returns "/bin/bash" -- without any
initialization scripts -- `(getenv "BASH_ENV")` returns `nil`.

Below is an Emacs Lisp function that returns the output of the same
shell command as a string:

     (defun emacs-grep-test ()
       "Return the output of the `grep` program."
       (let ((process-environment (copy-list process-environment)))
         (setenv "GREP_COLORS" "mt=01;31:fn=:ln=:bn=:se=:sl=:cx=:ne")
         (setenv "GREP_OPTIONS" "--color=auto")
         (shell-command-to-string "find . -type d \\( -path \\*/SCCS -o 
-path \\*/RCS -o -path \\*/CVS -o -path \\*/MCVS -o -path \\*/.svn -o 
-path \\*/.git -o -path \\*/.hg -o -path \\*/.bzr -o -path \\*/_MTN -o 
-path \\*/_darcs -o -path \\*/\\{arch\\} \\) -prune -o \\! -type d \\( 
-name \\*_flymake.\\* -o -name .\\#\\* -o -name \\*.o -o -name \\*\\~ -o 
-name \\*.bin -o -name \\*.lbin -o -name \\*.so -o -name \\*.a -o -name 
\\*.ln -o -name \\*.blg -o -name \\*.bbl -o -name \\*.elc -o -name 
\\*.lof -o -name \\*.glo -o -name \\*.idx -o -name \\*.lot -o -name 
\\*.fmt -o -name \\*.tfm -o -name \\*.class -o -name \\*.fas -o -name 
\\*.lib -o -name \\*.mem -o -name \\*.x86f -o -name \\*.sparcf -o -name 
\\*.dfsl -o -name \\*.pfsl -o -name \\*.d64fsl -o -name \\*.p64fsl -o 
-name \\*.lx64fsl -o -name \\*.lx32fsl -o -name \\*.dx64fsl -o -name 
\\*.dx32fsl -o -name \\*.fx64fsl -o -name \\*.fx32fsl -o -name 
\\*.sx64fsl -o -name \\*.sx32fsl -o -name \\*.wx64fsl -o -name 
\\*.wx32fsl -o -name \\*.fasl -o -name \\*.ufsl -o -name \\*.fsl -o 
-name \\*.dxl -o -name \\*.lo -o -name \\*.la -o -name \\*.gmo -o -name 
\\*.mo -o -name \\*.toc -o -name \\*.aux -o -name \\*.cp -o -name \\*.fn 
-o -name \\*.ky -o -name \\*.pg -o -name \\*.tp -o -name \\*.vr -o -name 
\\*.cps -o -name \\*.fns -o -name \\*.kys -o -name \\*.pgs -o -name 
\\*.tps -o -name \\*.vrs -o -name \\*.pyc -o -name \\*.pyo \\) -prune -o 
-type f \\( -iname \\*.el \\) -exec grep -i -n -e defun /dev/null {} +")))

However, the string returned by `M-: (emacs-grep-test) RET` does not
contain escape sequences to colorize the output, as I would expect.

Re: Examining the output of a shell command?

[Pascal J. Bourguignon]

Raffaele Ricciardi <rfflrccrd@gmail.com> writes:

> On 28/06/15 15:57, Pascal J. Bourguignon wrote:
>> (shell-command-to-string "echo $RED hello $NORMAL")
>> --> "[31m hello [0m> "
>
> Hence, how could I capture a colorized output in Emacs as in a Bash
> session?

This is an entirely different question, isn't it.

In *shell* and other comint based modes, you can activate interpretion
of ANSI escape codes for colorizing with 
M-x ansi-color-for-comint-mode-on RET

You can process the string with ansi-color-apply:

(ansi-color-apply
 (shell-command-to-string "echo $RED hello $YELLOW world $NORMAL"))
--> #(" hello  world 
" 0 7 (font-lock-face (foreground-color . "red")) 7 14 (font-lock-face (foreground-color . "yellow")))

(insert (ansi-color-apply
          (shell-command-to-string "echo $RED hello $YELLOW world $NORMAL")))
 hello  world 
  ^red    ^yellow
--> nil


-- 
__Pascal Bourguignon__                 http://www.informatimago.com/
“The factory of the future will have only two employees, a man and a
dog. The man will be there to feed the dog. The dog will be there to
keep the man from touching the equipment.” -- Carl Bass CEO Autodesk

Re: Examining the output of a shell command?

[Eli Zaretskii]

> From: Raffaele Ricciardi <rfflrccrd@gmail.com>
> Date: Mon, 29 Jun 2015 13:26:38 +0200
> 
> Hence, how could I capture a colorized output in Emacs as in a Bash
> session?
> 
> Below is a Bash script that recursively searches "*.el" files for the
> string "defun" and highlights matches (the complex `find` call is the
> shell command generated by `M-x rgrep` for the same task):
> 
>      #!/bin/bash
> 
>      export GREP_COLORS="mt=01;31:fn=:ln=:bn=:se=:sl=:cx=:ne"
>      export GREP_OPTIONS="--color=auto"
                                    ^^^^
Did you try with --color=always?

Re: Examining the output of a shell command?

[Raffaele Ricciardi]

On 29/06/15 16:51, Eli Zaretskii wrote:
>>       #!/bin/bash
>>
>>       export GREP_COLORS="mt=01;31:fn=:ln=:bn=:se=:sl=:cx=:ne"
>>       export GREP_OPTIONS="--color=auto"
>                                      ^^^^
> Did you try with --color=always?

This.  Thank you.

And yet, in my tests, apparently Emacs was calling `grep` with 
"--color=auto" instead.  But maybe I was missing some other settings.
Never mind.