how to pipe the output of one process to another using guile ?

how to pipe the output of one process to another using guile ?

[Samuel Barreto]

Hi everyone,

I was wondering how I could replicate the simple

	cat "my-toy-file.txt" | wc -l

using Guile ? I have been reading the POSIX Pipes and Default Ports
Guile manual entry very thoroughly but I could not wrap my head around
it. (I am not familiar with POSIX and all I am afraid …)

Thank you !
Samuel

re: how to pipe the output of one process to another using guile ?

[tantalum]

hi

(ice-9 popen) 
https://www.gnu.org/software/guile/manual/html_node/Pipes.html
is the only module that can create sub-processes with pipe input/output 
and does not use shell. it takes a program name and runs it as a 
sub-process with a pipe connected to its input/output. but its limited 
in that it can not set up multiple processes to be linked directly by 
pipes.
that means a solution using (ice-9 popen) would look like this:
* two open-pipe calls, one for a pipe for reading from the first program 
and another open-pipe call with a pipe for writing to the second program
* then copying the data with procedures like get-bytevector and 
put-bytevector
this means all data goes through the guile process as well as two pipes 
and not from sub-process to sub-process via one pipe

as far as i know, linking two processes, like in your example, is not 
currently cleanly possible without installing a guile library or 
extension.
there is (os process) 
http://www.nongnu.org/guile-lib/doc/ref/os.process/ in guile-lib which 
offers bindings that seem to do exactly what your example implies:

     (tail-call-pipeline ("ls" "/etc") ("grep" "passwd"))

maybe that will work for you.
i am not sure what limitations this has, since i am under the impression 
that there is no other reliable way to create sub-processes in guile 
(without core extensions) other than with (ice-9 popen).

alternatively you can use shell ("system" uses "sh")

     (system "cat my-toy-file.txt | wc -l")

and if you need the result as a string:

     (import (ice-9 popen) (rnrs io ports))

     (define (shell-eval->string command-str)
         (let* ((port (open-pipe command-str OPEN_READ)) (result 
(get-string-all port))) (close-pipe port)
         result))

     (display (shell-eval->string "cat my-toy-file.txt | wc -l"))

lastly, there is a third option, a scheme library and guile extension 
that is part of sph-lib https://github.com/sph-mn/sph-lib .
the name of the relevant library is (sph process create) and it is 
specifically for creating sub-processes and process chains.
to my knowledge it works reliably (i use it frequently), but it requires 
the compilation and installation of a guile extension (a shared 
library).
and it currently lacks proper online documentation. but here is the 
source code with docstrings anyway: 
https://github.com/sph-mn/sph-lib/blob/master/modules/sph/process/create.scm
here is how the example would be implemented using it:

     (import (sph process create))

     (process-chain-finish
         (process-chain #f (current-output-port) (list (list "cat" 
"my-toy-file.txt") (list "wc" "-l"))))

Re: how to pipe the output of one process to another using guile ?

[Samuel Barreto]

Hi,

Well thank you, I did not know about guile-lib (os process) nor sph-mn !
(tail-call-pipeline ...) seems to do the job for me. I'll let you know
what I ended up with !

Thanks for your help :)
Samuel

tantalum writes:

> hi
>
> (ice-9 popen)
> https://www.gnu.org/software/guile/manual/html_node/Pipes.html
> is the only module that can create sub-processes with pipe input/output
> and does not use shell. it takes a program name and runs it as a
> sub-process with a pipe connected to its input/output. but its limited
> in that it can not set up multiple processes to be linked directly by
> pipes.
> that means a solution using (ice-9 popen) would look like this:
> * two open-pipe calls, one for a pipe for reading from the first program
> and another open-pipe call with a pipe for writing to the second program
> * then copying the data with procedures like get-bytevector and
> put-bytevector
> this means all data goes through the guile process as well as two pipes
> and not from sub-process to sub-process via one pipe
>
> as far as i know, linking two processes, like in your example, is not
> currently cleanly possible without installing a guile library or
> extension.
> there is (os process)
> http://www.nongnu.org/guile-lib/doc/ref/os.process/ in guile-lib which
> offers bindings that seem to do exactly what your example implies:
>
>      (tail-call-pipeline ("ls" "/etc") ("grep" "passwd"))
>
> maybe that will work for you.
> i am not sure what limitations this has, since i am under the impression
> that there is no other reliable way to create sub-processes in guile
> (without core extensions) other than with (ice-9 popen).
>
> alternatively you can use shell ("system" uses "sh")
>
>      (system "cat my-toy-file.txt | wc -l")
>
> and if you need the result as a string:
>
>      (import (ice-9 popen) (rnrs io ports))
>
>      (define (shell-eval->string command-str)
>          (let* ((port (open-pipe command-str OPEN_READ)) (result
> (get-string-all port))) (close-pipe port)
>          result))
>
>      (display (shell-eval->string "cat my-toy-file.txt | wc -l"))
>
> lastly, there is a third option, a scheme library and guile extension
> that is part of sph-lib https://github.com/sph-mn/sph-lib .
> the name of the relevant library is (sph process create) and it is
> specifically for creating sub-processes and process chains.
> to my knowledge it works reliably (i use it frequently), but it requires
> the compilation and installation of a guile extension (a shared
> library).
> and it currently lacks proper online documentation. but here is the
> source code with docstrings anyway:
> https://github.com/sph-mn/sph-lib/blob/master/modules/sph/process/create.scm
> here is how the example would be implemented using it:
>
>      (import (sph process create))
>
>      (process-chain-finish
>          (process-chain #f (current-output-port) (list (list "cat"
> "my-toy-file.txt") (list "wc" "-l"))))