Filter IO through an external command

Filter IO through an external command

[Tristan Colgate]

Hi,

  I just got this working and thought people mind find it useful. It
passes IO through an external command (I'm using it to get svg output
from dot to include inline in html). I tried using
open-input-output-pipe but hit issues.

  It could probably handle errors nicely. Comments very welcome:

(use-modules (ice-9 rdelim))

(define (filter inproc outproc cmd . args)
  "filter the output of inproc via cmd to outproc through an external cmd"
  (let* ((read-pipe (pipe))
         (read-in  (car read-pipe))
         (read-out  (cdr read-pipe))
         (write-pipe (pipe))
         (write-in  (car write-pipe))
         (write-out  (cdr write-pipe))
         (child-pid    (primitive-fork)))
    (if (= child-pid 0)
        (begin
          (close-output-port read-out)
          (close-input-port write-in)
          (dup2 (fileno read-in) 0)
          (dup2 (fileno write-out) 1)
          (apply execlp (append (list cmd) (append (list cmd) args))))
        (begin
          (close-input-port read-in)
          (close-output-port write-out)
          (parameterize ((current-output-port read-out))
            (inproc))
          (close-output-port read-out)
          (parameterize ((current-input-port write-in))
            (outproc))
          (close-input-port write-in)
          (waitpid child-pid 0)))))

; To use
> (filter
    (lambda()
      (display "Hello World!")(newline))
    (lambda()
      (let loop ((line (read-line)))
        (if (not  (eof-object? line))
          (begin
            (write-line line)
            (loop (read-line))))))
    "sed" "s/World/Planet/")
Hello Planet!
$4 = (3892 . 0)
>

-- 
Tristan Colgate-McFarlane
----
  "You can get all your daily vitamins from 52 pints of guiness, and a
glass of milk"

Re: Filter IO through an external command

[Thien-Thi Nguyen]

() Tristan Colgate <tcolgate@gmail.com>
() Thu, 23 Feb 2012 14:26:28 +0000

   Comments very welcome:

     (let* ((read-pipe (pipe))
            (read-in  (car read-pipe))
            (read-out  (cdr read-pipe))

You might try SRFI 1 ‘car+cdr’ for this (and also for the write side)
in conjunction w/ SRFI 11 ‘let-values’, to improve code readability.

       (if (= child-pid 0)

You can use ‘zero?’ here.  Small nit: ‘child-pid’
as a variable name is misleading; better simply ‘pid’.

             (apply execlp (append (list cmd) (append (list cmd) args)))

This can be simplified to ‘(apply execlp cmd cmd args)’,
Also, i would add ‘(exit #f)’ afterwards, to appease Murphy.

Re: Filter IO through an external command

[Thien-Thi Nguyen]

() Tristan Colgate <tcolgate@gmail.com>
() Thu, 23 Feb 2012 14:26:28 +0000

   I tried using open-input-output-pipe but hit issues.

What were the issues?

Re: Filter IO through an external command

[Daniel Hartwig]

On 24 February 2012 01:16, Thien-Thi Nguyen <ttn@gnuvola.org> wrote:
> () Tristan Colgate <tcolgate@gmail.com>
> () Thu, 23 Feb 2012 14:26:28 +0000
>
>   I tried using open-input-output-pipe but hit issues.
>
> What were the issues?
>

Not sure if this is similar to the OP's issue…

Some programs wait until their stdin is closed before producing any
output--gzip is a good example.  open-pipe is insufficient for this
because (close-output-port pipe) closes both ends.

This looks more like a limitation of soft-port than open-pipe:

scheme@(guile-user)> (use-modules (ice-9 popen))
scheme@(guile-user)> (define pipe (open-pipe "gzip" OPEN_BOTH))
scheme@(guile-user)> (display "foo\n" pipe)
scheme@(guile-user)> (force-output pipe)
scheme@(guile-user)> (drain-input pipe)
$7 = ""
scheme@(guile-user)> (close-output-port pipe)
scheme@(guile-user)> (drain-input pipe)
ERROR: In procedure drain-input:
ERROR: In procedure drain-input: Wrong type argument in position 1
(expecting open input port): #<closed: soft 0>

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,q
scheme@(guile-user)> (close-pipe pipe)
$8 = 36096


I ended up using run-with-pipe from guile-lib's (os process) module
which returns separate port objects--similar to the OP's proc..

Re: Filter IO through an external command

[Andy Wingo]

On Fri 24 Feb 2012 04:31, Daniel Hartwig <mandyke@gmail.com> writes:

> I ended up using run-with-pipe from guile-lib's (os process) module
> which returns separate port objects--similar to the OP's proc..

Should we incorporate something like this into Guile?

Want to make a patch?  It could make open-process return the read and
write ports separately.

Andy
-- 
http://wingolog.org/

Re: Filter IO through an external command

[Daniel Hartwig]

On 24 February 2012 18:25, Andy Wingo <wingo@pobox.com> wrote:
> On Fri 24 Feb 2012 04:31, Daniel Hartwig <mandyke@gmail.com> writes:
>
>> I ended up using run-with-pipe from guile-lib's (os process) module
>> which returns separate port objects--similar to the OP's proc..
>
> Should we incorporate something like this into Guile?
>
> Want to make a patch?  It could make open-process return the read and
> write ports separately.
>

Sure.

Since open-process is internal to the popen module, changing that is
no problem.  With regards to the open-pipe procedures would you rather
an API-breaking change to returning the pair of ports, or a new
procedure which returns the pair?

Having separate read and write ports is greatly more useful than
combining them.  It also removes the overhead of passing everything
through a soft port.  Some users may prefer the current convenience of
dealing with a single port, but it does not reflect the underlying
mechanics very well.