What is the output port of `system*'?

What is the output port of `system*'?

[Diogo F. S. Ramos]

The following program doesn't output to a string, which I expected.

--8<---------------cut here---------------start------------->8---
(define foo
  (with-output-to-string
    (lambda ()
      (system* "ls" "/tmp"))))
--8<---------------cut here---------------end--------------->8---

Re: What is the output port of `system*'?

[Taylan Ulrich Bayirli/Kammer]

"Diogo F. S. Ramos" <dfsr@riseup.net> writes:

> The following program doesn't output to a string, which I expected.
>
> (define foo
>   (with-output-to-string
>     (lambda ()
>       (system* "ls" "/tmp"))))

As the manual says about `system*':
> The command is executed using fork and execlp.

That implies certain restrictions.  See (ice-9 popen), documented in
(info "(guile) Pipes").  That also spawns a process but sets its stdout
to a Unix pipe for which it gives you a Guile port, from which you can
drain the output to construct a string.

(Details: `with-output-to-string' could be said to work by setting the
current output port to some special value (a "string port" I guess) that
tells the deepest Guile IO procedures to construct a string instead of
writing to a file descriptor; when you spawn a separate process you
obviously lose this ability, the spawned process simply inherits the
current stdout file descriptor of the Guile process and writes to that.)

By the way, you might be better off doing things directly within Guile
instead of spawning external processes and trying to use their output.
In case you really wanted to do something with the output of ls(1), you
might want to use (ice-9 ftw) instead, but I suppose it was just an
example.

Taylan

Re: What is the output port of `system*'?

[Diogo F. S. Ramos]

> "Diogo F. S. Ramos" <dfsr@riseup.net> writes:
>
>> The following program doesn't output to a string, which I expected.
>>
>> (define foo
>>   (with-output-to-string
>>     (lambda ()
>>       (system* "ls" "/tmp"))))
>
> As the manual says about `system*':
>> The command is executed using fork and execlp.
>
> That implies certain restrictions.  See (ice-9 popen), documented in
> (info "(guile) Pipes").  That also spawns a process but sets its stdout
> to a Unix pipe for which it gives you a Guile port, from which you can
> drain the output to construct a string.

Thanks for the pipe reference.

Indeed, something can probably be written with these pipes to achieve
what my sample program tried to do tho the higher issue remains.

> (Details: `with-output-to-string' could be said to work by setting the
> current output port to some special value (a "string port" I guess) that
> tells the deepest Guile IO procedures to construct a string instead of
> writing to a file descriptor; when you spawn a separate process you
> obviously lose this ability, the spawned process simply inherits the
> current stdout file descriptor of the Guile process and writes to that.)

I'm not sure about the underlying implementation, but I expected all
output to (current-output-port) going to the resulting string, hence my
question.

As an example of this behavior, I point to the following Racket program:

--8<---------------cut here---------------start------------->8---
#lang racket

(define foo
  (with-output-to-string
    (lambda ()
      (system* (find-executable-path "ls") "/tmp"))))
--8<---------------cut here---------------end--------------->8---

Re: What is the output port of `system*'?

[Mark H Weaver]

"Diogo F. S. Ramos" <dfsr@riseup.net> writes:

>> "Diogo F. S. Ramos" <dfsr@riseup.net> writes:
>>
>>> The following program doesn't output to a string, which I expected.
>>>
>>> (define foo
>>>   (with-output-to-string
>>>     (lambda ()
>>>       (system* "ls" "/tmp"))))
>>
>> As the manual says about `system*':
>>> The command is executed using fork and execlp.
>>
>> That implies certain restrictions.  See (ice-9 popen), documented in
>> (info "(guile) Pipes").  That also spawns a process but sets its stdout
>> to a Unix pipe for which it gives you a Guile port, from which you can
>> drain the output to construct a string.
>
> Thanks for the pipe reference.
>
> Indeed, something can probably be written with these pipes to achieve
> what my sample program tried to do tho the higher issue remains.
>
>> (Details: `with-output-to-string' could be said to work by setting the
>> current output port to some special value (a "string port" I guess) that
>> tells the deepest Guile IO procedures to construct a string instead of
>> writing to a file descriptor; when you spawn a separate process you
>> obviously lose this ability, the spawned process simply inherits the
>> current stdout file descriptor of the Guile process and writes to that.)
>
> I'm not sure about the underlying implementation, but I expected all
> output to (current-output-port) going to the resulting string, hence my
> question.

All output to (current-output-port) _does_ go into the resulting string.
However, when you launch a subprocess using 'system*', the output of
that process does _not_ go to (current-output-port).  Instead, it goes
to POSIX file descriptor 1, which is left unchanged by the Scheme
constructs that change (current-output-port).  Ditto for
(current-input-port) and (current-error-port).  There are at least two
reasons for this:

* POSIX file descriptors cannot represent all of the types of ports
  supported by Guile, such as string ports and custom binary ports.

* The current-output-port is a fluid variable, which among other things
  is thread-local.  Obviously, POSIX file descriptors are per-process.

> As an example of this behavior, I point to the following Racket program:
>
> #lang racket
>
> (define foo
>   (with-output-to-string
>     (lambda ()
>       (system* (find-executable-path "ls") "/tmp"))))

I agree that it would be nice to make this work as you expected.  In
order to handle ports that are not simply raw POSIX file descriptors
(e.g. string ports), we'd need to create a pipe behind the scenes, and
arrange for thread(s) in the current process to copy data between the
pipe(s) and the Scheme port.

I'd like to do this at some point, although for the sake of backward
compatibility we'd probably have to leave 'system' and 'system*' as-is
and create a new API that works more intuitively.

      Mark

Re: What is the output port of `system*'?

[Diogo F. S. Ramos]

>> As an example of this behavior, I point to the following Racket program:
>>
>> #lang racket
>>
>> (define foo
>>   (with-output-to-string
>>     (lambda ()
>>       (system* (find-executable-path "ls") "/tmp"))))
>
> I agree that it would be nice to make this work as you expected.  In
> order to handle ports that are not simply raw POSIX file descriptors
> (e.g. string ports), we'd need to create a pipe behind the scenes, and
> arrange for thread(s) in the current process to copy data between the
> pipe(s) and the Scheme port.
>
> I'd like to do this at some point, although for the sake of backward
> compatibility we'd probably have to leave 'system' and 'system*' as-is
> and create a new API that works more intuitively.

If you do, may I suggest taking a look at Racket's Processes?

I'm not aware of things like "custodian" or "synchromizable event" in
Guile, but I've used Racket's Processes successfully in the past.