A better way to run shell cmd?

A better way to run shell cmd?

[Nala Ginrut]

hi folks! I'm on my trip and inconvenient to meet you guys on IRC.
Things gonna be normal next month.

Anyway, there's a problem for you.
I'm trying to write a simple wrapper for "sed" with our popen module:
--------------code------------
(use-modules (ice-9 popen) (rnrs))
(define (sed pattern str)
  (let ((p (open-input-output-pipe (string-append "sed " pattern))))
    (display str p)
    (get-string-all p)))
----------------end-------------

I expect it run like this:
-----------------------
(sed "s:a:b:g" "abcabc")
==> "bbcbbc"
-----------------------

But it halts that I have to interrupt.
Is it accepted?

Unlimited to "sed", I expect any shell cmd could be used in this way.
And I think it's easy to implement it with fork-then-exec. But I think
it's better do it with our (ice-9 popen).

I think the alternative way would be:
------------------code----------------
(define (sed pattern str)
  (let ((p (open-input-output-pipe (string-append "echo -n " str "|sed
" pattern))))
         (get-string-all p)))
-------------------end-----------------
which works but less elegant to the former.

Or anyone provide a better solution for shell cmd?

Re: A better way to run shell cmd?

[Daniel Hartwig]

On 13 June 2012 14:55, Nala Ginrut <nalaginrut@gmail.com> wrote:
> hi folks! I'm on my trip and inconvenient to meet you guys on IRC.
> Things gonna be normal next month.
>
> Anyway, there's a problem for you.
> I'm trying to write a simple wrapper for "sed" with our popen module:
> --------------code------------
> (use-modules (ice-9 popen) (rnrs))
> (define (sed pattern str)
>  (let ((p (open-input-output-pipe (string-append "sed " pattern))))
>    (display str p)
>    (get-string-all p)))
> ----------------end-------------
>
> I expect it run like this:
> -----------------------
> (sed "s:a:b:g" "abcabc")
> ==> "bbcbbc"
> -----------------------
>
> But it halts that I have to interrupt.
> Is it accepted?
>

The sed program may be expecting a newline, or waiting for EOF or a
full buffer before producing it's output.  Some users have had trouble
with this in the past, because you can not send EOF to one end of the
pipe without closing the other.

One way around this to use run-with-pipe from the guile-lib module (os
process).  This returns two separate pipe objects for input and
output, thus you can close/EOF one and continue to use the other.
Many filter programs do not flush their output until their input EOFs,
which is why your solution using 'echo -n' works.

See this thread for a short discussion:

http://lists.gnu.org/archive/html/guile-user/2012-02/msg00066.html

Updating guile to support closing one end of the pipe is long off on my TODO.

Re: A better way to run shell cmd?

[Nala Ginrut]

hi Daniel, thanks for reply!
I tried guile-lib just now, it's nice. I'll use it.

Besides, do you think pipe is the proper way to implement "sed" function?
Or it's better to implement a module with pure Guile?

On Wed, Jun 13, 2012 at 3:19 PM, Daniel Hartwig <mandyke@gmail.com> wrote:
> On 13 June 2012 14:55, Nala Ginrut <nalaginrut@gmail.com> wrote:
>> hi folks! I'm on my trip and inconvenient to meet you guys on IRC.
>> Things gonna be normal next month.
>>
>> Anyway, there's a problem for you.
>> I'm trying to write a simple wrapper for "sed" with our popen module:
>> --------------code------------
>> (use-modules (ice-9 popen) (rnrs))
>> (define (sed pattern str)
>>  (let ((p (open-input-output-pipe (string-append "sed " pattern))))
>>    (display str p)
>>    (get-string-all p)))
>> ----------------end-------------
>>
>> I expect it run like this:
>> -----------------------
>> (sed "s:a:b:g" "abcabc")
>> ==> "bbcbbc"
>> -----------------------
>>
>> But it halts that I have to interrupt.
>> Is it accepted?
>>
>
> The sed program may be expecting a newline, or waiting for EOF or a
> full buffer before producing it's output.  Some users have had trouble
> with this in the past, because you can not send EOF to one end of the
> pipe without closing the other.
>
> One way around this to use run-with-pipe from the guile-lib module (os
> process).  This returns two separate pipe objects for input and
> output, thus you can close/EOF one and continue to use the other.
> Many filter programs do not flush their output until their input EOFs,
> which is why your solution using 'echo -n' works.
>
> See this thread for a short discussion:
>
> http://lists.gnu.org/archive/html/guile-user/2012-02/msg00066.html
>
> Updating guile to support closing one end of the pipe is long off on my TODO.
>

Re: A better way to run shell cmd?

[Daniel Hartwig]

On 13 June 2012 18:02, Nala Ginrut <nalaginrut@gmail.com> wrote:
> hi Daniel, thanks for reply!
> I tried guile-lib just now, it's nice. I'll use it.

Yes, guile-lib has lots of great code in it. :-)

>
> Besides, do you think pipe is the proper way to implement "sed" function?
> Or it's better to implement a module with pure Guile?

This depends on how often you use it, and on how much data.

On the one hand, sed is very powerful, already written, and works
well.  On the other hand, opening the pipe has some overhead, but you
won't notice this unless you are processing lots of very short strings
individually.

The main thing I have used pipes for is compressing data.  This is
done once per file, and the files are relatively large.  In this case
the overhead for creating each pipe is tiny, much less than writing
and debugging my own version of gzip in Scheme.

Regards

Re: A better way to run shell cmd?

[gregory benison]

On Tue, Jun 12, 2012 at 11:55 PM, Nala Ginrut <nalaginrut@gmail.com> wrote:

>
> Anyway, there's a problem for you.
> I'm trying to write a simple wrapper for "sed" with our popen module:

This is exactly the kind of thing SCSH does well:

1> (run/string (sed s:a:b:g)(<< "banana"))
"bbnbnb"

Nevertheless the current state of guile's scsh implementation is,
according to the manual, "bitrotten"


-- 
Greg Benison <gbenison@gmail.com>
[blog] http://gcbenison.wordpress.com
[twitter] @gcbenison