bug#43364: Possible bug with output redirection

bug#43364: Possible bug with output redirection

[pinoaffe]

Dear guilers,

When using with-output-to-string, the output of external processes
started using system* and the like is not redirected to the temporary
port. As far as I can tell, it just redirects things written/displayed
from within guile.
This seems to be a bug, or if this is intended behaviour it might be
beneficial to document this somewhere.

As an example:
I'd expect snippet [0] to behave like snippet [1] and return "bar",
instead it just returns the empty string.

This probably affects other similar functions.

Pandemically,
pinoaffe

[0]:
(with-output-to-string
 (lambda ()
   (system* "echo" "bar")))

[1]:
(with-output-to-string
 (lambda ()
   (display "bar")))

bug#43364: Possible bug with output redirection

[tomas]

[-- Attachment #1: Type: text/plain, Size: 984 bytes --]

On Sat, Sep 12, 2020 at 10:59:23PM +0200, pinoaffe wrote:
> Dear guilers,
> 
> When using with-output-to-string, the output of external processes
> started using system* and the like is not redirected to the temporary
> port. As far as I can tell, it just redirects things written/displayed
> from within guile.
> This seems to be a bug, or if this is intended behaviour it might be
> beneficial to document this somewhere.

I think this is intentional (or rather: out of Guile's scope). While the
Guile process's output functions are the scope of `with-output-to-string',
the subprocess started with `system' just happens to inherit the standard
output file descriptor, which is an operating system mechanism.

No idea, for example, about what would happen under Windows or other
(more or less) operating systems.

But you are right that it can be a bit confusing. I'm not sure whether a
hint in the doc would be in place.

What do oters think?

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

bug#43364: with-output-to-port works with file ports

[Felix Lechner via Bug reports for GUILE, GNU's Ubiquitous Extension Language]

Hi,

In an interesting (or perhaps maddening) inconsistency,
'with-output-to-port' captures stdout from system* here

(call-with-output-file "/tmp/test.log"
  (lambda (port)
    (with-output-to-port
      port
      (lambda ()
        (system* "mktemp" "-d")))))

but 'with-output-to-string' does not do so here

(with-output-to-string
  (lambda ()
    (system* "mktemp" "-d")))

According to lloda on #guile, system* handles the redirection only
when the current ports are file ports. Thanks for that pointer!

Kind regards
Felix

bug#43364: with-output-to-port works with file ports

[Ricardo Wurmus]

Felix Lechner via "Bug reports for GUILE, GNU's Ubiquitous Extension Language" <bug-guile@gnu.org> writes:

> Hi,
>
> In an interesting (or perhaps maddening) inconsistency,
> 'with-output-to-port' captures stdout from system* here
>
> (call-with-output-file "/tmp/test.log"
>   (lambda (port)
>     (with-output-to-port
>       port
>       (lambda ()
>         (system* "mktemp" "-d")))))
>
> but 'with-output-to-string' does not do so here
>
> (with-output-to-string
>   (lambda ()
>     (system* "mktemp" "-d")))
>
> According to lloda on #guile, system* handles the redirection only
> when the current ports are file ports. Thanks for that pointer!

That’s correct.  I’ve been using the following monstrosity to capture
and process output.  Perhaps someone finds a prettier way?

--8<---------------cut here---------------start------------->8---
(define* (call-with-output-processor command proc #:optional capture-stderr?)
  "Silently execute COMMAND, a list of strings representing an
executable with its arguments, and apply PROC to every line printed to
standard output and, optionally when CAPTURE-STDERR? is #T, standard
error.  Return the exit status of COMMAND."
  ;; We can only capture a program's standard error by parameterizing
  ;; current-error-port to a *file* port before using system* or
  ;; open-pipe*.  The process will write its standard error stream to
  ;; the provided file descriptor.  Meanwhile we read from the file
  ;; descriptor (blocking) for new lines until the process exits.
  (match (socketpair PF_UNIX SOCK_STREAM 0)
    ((in . out)
     (let ((err (if capture-stderr?
                    (dup out)
                    (%make-void-port "w"))))
       (catch #true
         (lambda ()
           (let ((thread
                  (parameterize ((current-error-port err)
                                 (current-output-port out))
                    (call-with-new-thread
                     (lambda ()
                       (let ((status
                              (status:exit-val
                               (apply system* command))))
                         (close-port err)
                         (close-port out)
                         status))))))
             (let loop ()
               (match (read-line in 'concat)
                 ((? eof-object?)
                  (for-each
                   (lambda (port)
                     (false-if-exception (close-port port)))
                   (list err out in))
                  (join-thread thread))
                 (line
                  (proc line)
                  (loop))))))
         (lambda (key . args)
           (for-each
            (lambda (port)
              (false-if-exception (close-port port)))
            (list err out in))
           (apply throw key args)))))))
--8<---------------cut here---------------end--------------->8---


-- 
Ricardo

bug#43364: with-output-to-port works with file ports

[Fabio Natali]

Dear All 👋,

I've been encountering a few times the need for a procedure like
'system' or 'system*' but that might be capable of capturing the output
of the program that's being called.

While this is something that can be solved with a simple 'open-pipe*'
wrapper, the pattern seems relatively frequent in Guix, which is where I
noticed it. I suggested that some variant of such a wrapper could be
added to '(guix build utils)'⁰.

In reply to my post, it was brought to my attention that the problem
might be addressed at the Guile level and pointed me to this bug report.

I thought of sending a quick follow-up here to see if there's any rough
consensus on a possible way of addressing this - or why it might be
difficult or potentially not worth the effort.

Thanks, all best, Fabio.

⁰ https://lists.gnu.org/archive/html/guix-devel/2024-04/msg00199.html


-- 
Fabio Natali
https://fabionatali.com

bug#43364: with-output-to-port works with file ports

[Tony Garnock-Jones]

This is a nice idea. The way Racket does it is to make 
with-output-to-string cooperate with system:

     (with-output-to-string (lambda () (system "date")))

but a quick check shows that Guile does not yet have such cooperation in 
place. Would that be a good place to start looking?

Tony


On 23/04/2024 18:32, Fabio Natali wrote:
> Dear All 👋,
> 
> I've been encountering a few times the need for a procedure like
> 'system' or 'system*' but that might be capable of capturing the output
> of the program that's being called.
> 
> While this is something that can be solved with a simple 'open-pipe*'
> wrapper, the pattern seems relatively frequent in Guix, which is where I
> noticed it. I suggested that some variant of such a wrapper could be
> added to '(guix build utils)'⁰.
> 
> In reply to my post, it was brought to my attention that the problem
> might be addressed at the Guile level and pointed me to this bug report.
> 
> I thought of sending a quick follow-up here to see if there's any rough
> consensus on a possible way of addressing this - or why it might be
> difficult or potentially not worth the effort.
> 
> Thanks, all best, Fabio.
> 
> ⁰ https://lists.gnu.org/archive/html/guix-devel/2024-04/msg00199.html
> 
>