Shell commands with output to string

Shell commands with output to string

[Zelphir Kaltstahl]

Hello Guile users!

How would I run a shell command from inside Guile and get its output as a 
string, instead of the output being outputted directly? (Guile 3.0.8)

So far I have found

~~~~
(system ...)
~~~~

which I tried to use with

~~~~
scheme@(guile-user)> (with-output-to-string
   (system "ls -al"))

;; lots of output immediately shown and not stored in variable

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Wrong type to apply: 0

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,bt
In ice-9/ports.scm:
     476:4  2 (with-output-to-string 0)
While executing meta-command:
In procedure frame-local-ref: Argument 2 out of range: 1
~~~~

But this does not give me a string back.

I also tried with

~~~~
scheme@(guile-user)> (call-with-values (lambda () (system "ls -al"))
... (lambda (exit-code output) output))

;; lots of output immediately shown

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Wrong number of values returned to continuation (expected 2)

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.

scheme@(guile-user) [1]> ,bt
In current input:
     10:29  1 (_)
In ice-9/boot-9.scm:
   1685:16  0 (raise-exception _ #:continuable? _)
~~~~

Is there another function I should be using?

I would like to have the exit code and the output of a command.

Best regards,
Zelphir

-- 
repositories: https://notabug.org/ZelphirKaltstahl

Re: Shell commands with output to string

[Zelphir Kaltstahl]

Corrections below.

On 2/22/22 10:29, Zelphir Kaltstahl wrote:
> Hello Guile users!
>
> How would I run a shell command from inside Guile and get its output as a 
> string, instead of the output being outputted directly? (Guile 3.0.8)
>
> So far I have found
>
> ~~~~
> (system ...)
> ~~~~
>
> which I tried to use with
>
> ~~~~
> scheme@(guile-user)> (with-output-to-string
>   (system "ls -al"))
>
> ;; lots of output immediately shown and not stored in variable
>
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> Wrong type to apply: 0
>
> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
> scheme@(guile-user) [1]> ,bt
> In ice-9/ports.scm:
>     476:4  2 (with-output-to-string 0)
> While executing meta-command:
> In procedure frame-local-ref: Argument 2 out of range: 1
> ~~~~
>
> But this does not give me a string back.
>
> I also tried with
>
> ~~~~
> scheme@(guile-user)> (call-with-values (lambda () (system "ls -al"))
> ... (lambda (exit-code output) output))
>
> ;; lots of output immediately shown
>
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> Wrong number of values returned to continuation (expected 2)
>
> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
>
> scheme@(guile-user) [1]> ,bt
> In current input:
>     10:29  1 (_)
> In ice-9/boot-9.scm:
>   1685:16  0 (raise-exception _ #:continuable? _)
> ~~~~
>
> Is there another function I should be using?
>
> I would like to have the exit code and the output of a command.
>
> Best regards,
> Zelphir

Of course I should use `with-output-to-string` correctly:

~~~~
scheme@(guile-user)> (with-output-to-string
   (lambda () (system "ls -al")))
;; directly outputted stuff
$1 = ""
~~~~

But still not a win.

Regards,
Zelphir

-- 
repositories: https://notabug.org/ZelphirKaltstahl	

Re: Shell commands with output to string

[Alex Sassmannshausen]

[-- Attachment #1: Type: text/plain, Size: 2228 bytes --]

Hi Zelphir,

I think you want to be using the popen / pipe procedures for this. See
https://www.gnu.org/software/guile/docs/docs-2.2/guile-ref/Pipes.html
for the chapter in the manual.

Hope this helps :)

Alex

Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> writes:

> Corrections below.
>
> On 2/22/22 10:29, Zelphir Kaltstahl wrote:
>> Hello Guile users!
>>
>> How would I run a shell command from inside Guile and get its output
>> as a string, instead of the output being outputted directly? (Guile
>> 3.0.8)
>>
>> So far I have found
>>
>> ~~~~
>> (system ...)
>> ~~~~
>>
>> which I tried to use with
>>
>> ~~~~
>> scheme@(guile-user)> (with-output-to-string
>>   (system "ls -al"))
>>
>> ;; lots of output immediately shown and not stored in variable
>>
>> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
>> Wrong type to apply: 0
>>
>> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
>> scheme@(guile-user) [1]> ,bt
>> In ice-9/ports.scm:
>>     476:4  2 (with-output-to-string 0)
>> While executing meta-command:
>> In procedure frame-local-ref: Argument 2 out of range: 1
>> ~~~~
>>
>> But this does not give me a string back.
>>
>> I also tried with
>>
>> ~~~~
>> scheme@(guile-user)> (call-with-values (lambda () (system "ls -al"))
>> ... (lambda (exit-code output) output))
>>
>> ;; lots of output immediately shown
>>
>> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
>> Wrong number of values returned to continuation (expected 2)
>>
>> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
>>
>> scheme@(guile-user) [1]> ,bt
>> In current input:
>>     10:29  1 (_)
>> In ice-9/boot-9.scm:
>>   1685:16  0 (raise-exception _ #:continuable? _)
>> ~~~~
>>
>> Is there another function I should be using?
>>
>> I would like to have the exit code and the output of a command.
>>
>> Best regards,
>> Zelphir
>
> Of course I should use `with-output-to-string` correctly:
>
> ~~~~
> scheme@(guile-user)> (with-output-to-string
>   (lambda () (system "ls -al")))
> ;; directly outputted stuff
> $1 = ""
> ~~~~
>
> But still not a win.
>
> Regards,
> Zelphir


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 272 bytes --]

Re: Shell commands with output to string

[tomas]

[-- Attachment #1: Type: text/plain, Size: 519 bytes --]

On Tue, Feb 22, 2022 at 09:38:35AM +0000, Zelphir Kaltstahl wrote:
> Corrections below.
> 
> On 2/22/22 10:29, Zelphir Kaltstahl wrote:
> > Hello Guile users!
> > 
> > How would I run a shell command from inside Guile and get its output as
> > a string, instead of the output being outputted directly? (Guile 3.0.8)

Sorry for the short answer (bad connectivity).

I think you want something like `popen' (use-modules (ice-9 popen)).

See the ref manual "Guile Modules" > POSIX > Pipes.

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Shell commands with output to string

[post]

Hi,

to throw in an example: I once used a function like the one below to 
handle stdout and stderr from external commands (from 
https://github.com/tdanckaert/jobview/blob/master/jobtools.scm#L38 ).  
Probably far from perfect (my first and only scheme project...), but 
hopefully it gives you an idea.

(define (process-output proc cmd)
   "Runs CMD as an external process, with an input port from which the
process' stdout may be read, and runs the procedure PROC that takes
this input port as a single argument.  Throws an exception 'cmd-failed
if CMD's exit status is non-zero."
   (let* ((err-pipe (pipe))
	 (err-write (cdr err-pipe))
	 (err-read (car err-pipe))
	 (stderr (current-error-port)))
     (with-error-to-port err-write
       (lambda ()
	(let* ((port (open-input-pipe cmd))
	       (ignore (setvbuf port 'block))
	       (result
		(catch #t
		  ;; Catch any exception thrown by applying PROC to
		  ;; the output of CMD: if CMD fails, we check the
		  ;; exit status below; if CMD succeeds, PROC must be
		  ;; able to deal with its output.
		  (lambda () (proc port))
		  (lambda (key . args)
		    (format stderr "Caught exception ~a from ~y~%" key proc))))
	       (status (close-pipe port)))
	  (close-port err-write)
	  (or (zero? status)
	      (throw 'cmd-failed cmd status
		     (get-string-all err-read)))
	  result)))))

Thomas


On 2022-02-22 11:20, Alex Sassmannshausen wrote:
> Hi Zelphir,
> 
> I think you want to be using the popen / pipe procedures for this. See
> https://www.gnu.org/software/guile/docs/docs-2.2/guile-ref/Pipes.html
> for the chapter in the manual.
> 
> Hope this helps :)
> 
> Alex
> 
> Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> writes:
> 
>> Corrections below.
>> 
>> On 2/22/22 10:29, Zelphir Kaltstahl wrote:
>>> Hello Guile users!
>>> 
>>> How would I run a shell command from inside Guile and get its output
>>> as a string, instead of the output being outputted directly? (Guile
>>> 3.0.8)
>>> 
>>> So far I have found
>>> 
>>> ~~~~
>>> (system ...)
>>> ~~~~
>>> 
>>> which I tried to use with
>>> 
>>> ~~~~
>>> scheme@(guile-user)> (with-output-to-string
>>>   (system "ls -al"))
>>> 
>>> ;; lots of output immediately shown and not stored in variable
>>> 
>>> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
>>> Wrong type to apply: 0
>>> 
>>> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to 
>>> continue.
>>> scheme@(guile-user) [1]> ,bt
>>> In ice-9/ports.scm:
>>>     476:4  2 (with-output-to-string 0)
>>> While executing meta-command:
>>> In procedure frame-local-ref: Argument 2 out of range: 1
>>> ~~~~
>>> 
>>> But this does not give me a string back.
>>> 
>>> I also tried with
>>> 
>>> ~~~~
>>> scheme@(guile-user)> (call-with-values (lambda () (system "ls -al"))
>>> ... (lambda (exit-code output) output))
>>> 
>>> ;; lots of output immediately shown
>>> 
>>> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
>>> Wrong number of values returned to continuation (expected 2)
>>> 
>>> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to 
>>> continue.
>>> 
>>> scheme@(guile-user) [1]> ,bt
>>> In current input:
>>>     10:29  1 (_)
>>> In ice-9/boot-9.scm:
>>>   1685:16  0 (raise-exception _ #:continuable? _)
>>> ~~~~
>>> 
>>> Is there another function I should be using?
>>> 
>>> I would like to have the exit code and the output of a command.
>>> 
>>> Best regards,
>>> Zelphir
>> 
>> Of course I should use `with-output-to-string` correctly:
>> 
>> ~~~~
>> scheme@(guile-user)> (with-output-to-string
>>   (lambda () (system "ls -al")))
>> ;; directly outputted stuff
>> $1 = ""
>> ~~~~
>> 
>> But still not a win.
>> 
>> Regards,
>> Zelphir

Re: Shell commands with output to string

[Neil Jerram]

On Tue, 22 Feb 2022 at 10:23, Alex Sassmannshausen
<alex.sassmannshausen@gmail.com> wrote:
>
> Hi Zelphir,
>
> I think you want to be using the popen / pipe procedures for this. See
> https://www.gnu.org/software/guile/docs/docs-2.2/guile-ref/Pipes.html
> for the chapter in the manual.

Another example, for reading transactions out of a Ledger file:

(use-modules (ice-9 popen))

(define (ledger-transactions filename account payee commodity year)
  (let* ((cmd (string-append "ledger -f " filename))
         (cmd-add! (lambda strings (set! cmd (apply string-append cmd
" " strings)))))
    (if payee
        (cmd-add! "-l 'payee=~/" payee "/'"))
    (if year
        (cmd-add! "--begin " (number->string year) " --end "
(number->string (1+ year))))
    (cmd-add! "reg")
    (if account
        (cmd-add! account))
    (cmd-add! "-F '(\"%(format_date(date, \"%Y-%m-%d\"))\" \"%P\" \"%(t)\")\n'")
    (let ((p (open-input-pipe cmd)))
      (let loop ((txs '()))
        (let ((tx (read p)))
          (if (eof-object? tx)
              (reverse! txs)
              (begin
                (if commodity
                    (set-car! (cddr tx) (string-replace-substring
(caddr tx) commodity "")))
                (loop (cons tx txs)))))))))

Re: Shell commands with output to string

[Olivier Dion via General Guile related discussions]

On Tue, 22 Feb 2022, Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> wrote:
> Hello Guile users!
>
> How would I run a shell command from inside Guile and get its output
> as a string, instead of the output being outputted directly? (Guile
> 3.0.8)

I use the following:

(define-module (shell utils)
  #:use-module (ice-9 format)
  #:use-module (ice-9 popen)
  #:use-module (ice-9 textual-ports))

(define (shell% proc fmt . args)
  (let* ((port (open-input-pipe (format #f "~?" fmt args)))
         (output (proc port)))
    (close-pipe port)
    output))

(define-public (shell . args)
  (apply shell% (cons get-string-all args)))

(define-public (shell$ . args)
  (apply shell% (cons get-line args)))

Then
(shell "ls" "-l")

The $ variant is to get a single line in the output.

-- 
Olivier Dion
Polymtl

Re: Shell commands with output to string

[Leo Butler]

Olivier Dion via General Guile related discussions <guile-user@gnu.org>
writes:

> On Tue, 22 Feb 2022, Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> wrote:
>> Hello Guile users!
>>
>> How would I run a shell command from inside Guile and get its output
>> as a string, instead of the output being outputted directly? (Guile
>> 3.0.8)
>
> I use the following:
>
> (define-module (shell utils)
>   #:use-module (ice-9 format)
>   #:use-module (ice-9 popen)
>   #:use-module (ice-9 textual-ports))
>
> (define (shell% proc fmt . args)
>   (let* ((port (open-input-pipe (format #f "~?" fmt args)))
>          (output (proc port)))
>     (close-pipe port)
>     output))

You probably want to inspect the exit value of the shell process, so
that you can handle/throw the error. This is what I use (similar to your
`shell'):

(define* (shell-command-to-string cmd)
  (catch 'shell-command-error
    ;; thunk                                                                                                                                                                                                                        
    (lambda ()
      (let* ((port (open-pipe cmd OPEN_READ))
             (str (read-string port))
             (wtpd (close-pipe port))
             (xval (status:exit-val wtpd)))
        (if (or (eqv? xval #f) (> xval 0)) (throw 'shell-command-error cmd str))
        str))
    ;; handler                                                                                                                                                                                                                      
    (lambda (key cmd str)
      (simple-format #t "ERROR: in command ~a\nstring: ~a\n" cmd str)
      (throw 'error-in-shell-command-to-string cmd str))))

>
> (define-public (shell . args)
>   (apply shell% (cons get-string-all args)))
>
> (define-public (shell$ . args)
>   (apply shell% (cons get-line args)))
>
> Then
> (shell "ls" "-l")
>
> The $ variant is to get a single line in the output.

I wonder why there is no module already in ice-9 which does this stuff?
It seems like a lot of people are re-inventing the wheel.

Leo

Re: Shell commands with output to string

[Olivier Dion via General Guile related discussions]

On Tue, 22 Feb 2022, Leo Butler <leo.butler@umanitoba.ca> wrote:
> Olivier Dion via General Guile related discussions <guile-user@gnu.org>
> writes:
>
>> On Tue, 22 Feb 2022, Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> wrote:
>>> Hello Guile users!
>>>
>>> How would I run a shell command from inside Guile and get its output
>>> as a string, instead of the output being outputted directly? (Guile
>>> 3.0.8)
>>
>> I use the following:
>>
>> (define-module (shell utils)
>>   #:use-module (ice-9 format)
>>   #:use-module (ice-9 popen)
>>   #:use-module (ice-9 textual-ports))
>>
>> (define (shell% proc fmt . args)
>>   (let* ((port (open-input-pipe (format #f "~?" fmt args)))
>>          (output (proc port)))
>>     (close-pipe port)
>>     output))
>
> You probably want to inspect the exit value of the shell process, so
> that you can handle/throw the error. This is what I use (similar to your
> `shell'):

You're absolutely right.  It would also probably a good idea to use
dynamic-wind for close-pipe in case an exception is thrown in
read-string I think.

>
> (define* (shell-command-to-string cmd)
>   (catch 'shell-command-error
>     ;; thunk                                                                                                                                                                                                                        
>     (lambda ()
>       (let* ((port (open-pipe cmd OPEN_READ))
>              (str (read-string port))
>              (wtpd (close-pipe port))
>              (xval (status:exit-val wtpd)))
>         (if (or (eqv? xval #f) (> xval 0)) (throw 'shell-command-error cmd str))
>         str))
>     ;; handler                                                                                                                                                                                                                      
>     (lambda (key cmd str)
>       (simple-format #t "ERROR: in command ~a\nstring: ~a\n" cmd str)
>       (throw 'error-in-shell-command-to-string cmd str))))
>
>>
>> (define-public (shell . args)
>>   (apply shell% (cons get-string-all args)))
>>
>> (define-public (shell$ . args)
>>   (apply shell% (cons get-line args)))
>>
>> Then
>> (shell "ls" "-l")
>>
>> The $ variant is to get a single line in the output.
>
> I wonder why there is no module already in ice-9 which does this
> stuff?  It seems like a lot of people are re-inventing the wheel.

There's ton of missing stuffs in the standard library IMO.  On top of my
head, filesystem paths manipulation (e.g. path-join) is also one that is
probably getting re-invented a lots.

I believe that the successful story of Python is not just about its
pretty syntax, but also dues to its very large standard library.

-- 
Olivier Dion
Polymtl

Re: Shell commands with output to string

[Zelphir Kaltstahl]

Hi Oliver!

On 2/22/22 17:33, Olivier Dion wrote:
> On Tue, 22 Feb 2022, Leo Butler <leo.butler@umanitoba.ca> wrote:
>> Olivier Dion via General Guile related discussions <guile-user@gnu.org>
>> writes:
>>
>>> On Tue, 22 Feb 2022, Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> wrote:
>>>> Hello Guile users!
>>>>
>>>> How would I run a shell command from inside Guile and get its output
>>>> as a string, instead of the output being outputted directly? (Guile
>>>> 3.0.8)
>>> I use the following:
>>>
>>> (define-module (shell utils)
>>>    #:use-module (ice-9 format)
>>>    #:use-module (ice-9 popen)
>>>    #:use-module (ice-9 textual-ports))
>>>
>>> (define (shell% proc fmt . args)
>>>    (let* ((port (open-input-pipe (format #f "~?" fmt args)))
>>>           (output (proc port)))
>>>      (close-pipe port)
>>>      output))
>> You probably want to inspect the exit value of the shell process, so
>> that you can handle/throw the error. This is what I use (similar to your
>> `shell'):
> You're absolutely right.  It would also probably a good idea to use
> dynamic-wind for close-pipe in case an exception is thrown in
> read-string I think.
>
>> (define* (shell-command-to-string cmd)
>>    (catch 'shell-command-error
>>      ;; thunk
>>      (lambda ()
>>        (let* ((port (open-pipe cmd OPEN_READ))
>>               (str (read-string port))
>>               (wtpd (close-pipe port))
>>               (xval (status:exit-val wtpd)))
>>          (if (or (eqv? xval #f) (> xval 0)) (throw 'shell-command-error cmd str))
>>          str))
>>      ;; handler
>>      (lambda (key cmd str)
>>        (simple-format #t "ERROR: in command ~a\nstring: ~a\n" cmd str)
>>        (throw 'error-in-shell-command-to-string cmd str))))
>>
>>> (define-public (shell . args)
>>>    (apply shell% (cons get-string-all args)))
>>>
>>> (define-public (shell$ . args)
>>>    (apply shell% (cons get-line args)))
>>>
>>> Then
>>> (shell "ls" "-l")
>>>
>>> The $ variant is to get a single line in the output.
>> I wonder why there is no module already in ice-9 which does this
>> stuff?  It seems like a lot of people are re-inventing the wheel.
> There's ton of missing stuffs in the standard library IMO.  On top of my
> head, filesystem paths manipulation (e.g. path-join) is also one that is
> probably getting re-invented a lots.

I actually made something for that, trying to copy mostly the Python behavior 
for os.path.join:

https://notabug.org/ZelphirKaltstahl/guile-fslib

Also available as a GNU Guix package, but not updated in a while on Guix. 
Repository contains more up to date version.

> I believe that the successful story of Python is not just about its
> pretty syntax, but also dues to its very large standard library.

I think so too. Although I sometimes have the feeling, that Guile does things in 
a cleaner way, once one figures out how to do them in the first place. One thing 
I really like are the ports. Stuff like call-with-output-string. Takes some 
twisting of the brain, but once one gets it, it becomes very useful and elegant.

But yes, Python is very beginner friendly in terms of batteries included. 
Although I think that its syntax feels a bit ad-hoc. As in "Oh we want some 
syntax for X … lets invent this keyword here." or some new operators or things 
like that. I like Guile syntax (or Scheme in general) much more. However, it is 
difficult to motivate others in a quick demo to learn the language, when you 
cannot take half an hour time to explain, what that for other people weird 
looking syntax is actually really cool.

Regards,
Zelphir

-- 
repositories: https://notabug.org/ZelphirKaltstahl

Re: Shell commands with output to string

[Zelphir Kaltstahl]

Hi all!

On 2/22/22 12:20, Neil Jerram wrote:
> On Tue, 22 Feb 2022 at 10:23, Alex Sassmannshausen
> <alex.sassmannshausen@gmail.com> wrote:
>> Hi Zelphir,
>>
>> I think you want to be using the popen / pipe procedures for this. See
>> https://www.gnu.org/software/guile/docs/docs-2.2/guile-ref/Pipes.html
>> for the chapter in the manual.
> Another example, for reading transactions out of a Ledger file:
>
> (use-modules (ice-9 popen))
>
> (define (ledger-transactions filename account payee commodity year)
>    (let* ((cmd (string-append "ledger -f " filename))
>           (cmd-add! (lambda strings (set! cmd (apply string-append cmd
> " " strings)))))
>      (if payee
>          (cmd-add! "-l 'payee=~/" payee "/'"))
>      (if year
>          (cmd-add! "--begin " (number->string year) " --end "
> (number->string (1+ year))))
>      (cmd-add! "reg")
>      (if account
>          (cmd-add! account))
>      (cmd-add! "-F '(\"%(format_date(date, \"%Y-%m-%d\"))\" \"%P\" \"%(t)\")\n'")
>      (let ((p (open-input-pipe cmd)))
>        (let loop ((txs '()))
>          (let ((tx (read p)))
>            (if (eof-object? tx)
>                (reverse! txs)
>                (begin
>                  (if commodity
>                      (set-car! (cddr tx) (string-replace-substring
> (caddr tx) commodity "")))
>                  (loop (cons tx txs)))))))))

I will look at all the ideas and try to get a good picture of it : )

Thank you all for the advice and examples!

Regards,
Zelphir

-- 
repositories: https://notabug.org/ZelphirKaltstahl

Re: Shell commands with output to string

[Zelphir Kaltstahl]

Hi Alex!

On 2/22/22 11:20, Alex Sassmannshausen wrote:
> Hi Zelphir,
>
> I think you want to be using the popen / pipe procedures for this. See
> https://www.gnu.org/software/guile/docs/docs-2.2/guile-ref/Pipes.html
> for the chapter in the manual.
>
> Hope this helps :)
>
> Alex
>
> Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> writes:
>
>> Corrections below.
>>
>> On 2/22/22 10:29, Zelphir Kaltstahl wrote:
>>> Hello Guile users!
>>>
>>> How would I run a shell command from inside Guile and get its output
>>> as a string, instead of the output being outputted directly? (Guile
>>> 3.0.8)
>>>
>>> So far I have found
>>>
>>> ~~~~
>>> (system ...)
>>> ~~~~
>>>
>>> which I tried to use with
>>>
>>> ~~~~
>>> scheme@(guile-user)> (with-output-to-string
>>>    (system "ls -al"))
>>>
>>> ;; lots of output immediately shown and not stored in variable
>>>
>>> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
>>> Wrong type to apply: 0
>>>
>>> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
>>> scheme@(guile-user) [1]> ,bt
>>> In ice-9/ports.scm:
>>>      476:4  2 (with-output-to-string 0)
>>> While executing meta-command:
>>> In procedure frame-local-ref: Argument 2 out of range: 1
>>> ~~~~
>>>
>>> But this does not give me a string back.
>>>
>>> I also tried with
>>>
>>> ~~~~
>>> scheme@(guile-user)> (call-with-values (lambda () (system "ls -al"))
>>> ... (lambda (exit-code output) output))
>>>
>>> ;; lots of output immediately shown
>>>
>>> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
>>> Wrong number of values returned to continuation (expected 2)
>>>
>>> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
>>>
>>> scheme@(guile-user) [1]> ,bt
>>> In current input:
>>>      10:29  1 (_)
>>> In ice-9/boot-9.scm:
>>>    1685:16  0 (raise-exception _ #:continuable? _)
>>> ~~~~
>>>
>>> Is there another function I should be using?
>>>
>>> I would like to have the exit code and the output of a command.
>>>
>>> Best regards,
>>> Zelphir
>> Of course I should use `with-output-to-string` correctly:
>>
>> ~~~~
>> scheme@(guile-user)> (with-output-to-string
>>    (lambda () (system "ls -al")))
>> ;; directly outputted stuff
>> $1 = ""
>> ~~~~
>>
>> But still not a win.
>>
>> Regards,
>> Zelphir

Thanks for that!

Regards,
Zelphir

-- 
repositories: https://notabug.org/ZelphirKaltstahl

Re: Shell commands with output to string

[Josselin Poiret]

Hello,


post@thomasdanckaert.be writes:

> Hi,
>
> to throw in an example: I once used a function like the one below to 
> handle stdout and stderr from external commands (from 
> https://github.com/tdanckaert/jobview/blob/master/jobtools.scm#L38 ).  
> Probably far from perfect (my first and only scheme project...), but 
> hopefully it gives you an idea.

Just chiming in to say that [1] isn't fixed yet, so you may run into
issues if you try to redirect out and err to the same port.  In Guix, we
use the following workaround for now ([2]):
--8<---------------cut here---------------start------------->8---
  (match-let (((input . output) (pipe)))
    ;; Hack to work around Guile bug 52835
    (define dup-output (duplicate-port output "w"))
    ;; Void pipe, but holds the pid for close-pipe.
    (define dummy-pipe
      (with-input-from-file "/dev/null"
        (lambda ()
          (with-output-to-port output
            (lambda ()
              (with-error-to-port dup-output
                (lambda ()
                  (apply open-pipe* (cons "" command)))))))))
    (close-port output)
    (close-port dup-output)
    (handler input)
    (close-port input)
    (close-pipe dummy-pipe))
--8<---------------cut here---------------end--------------->8---

[1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=52835
[2] https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/installer/utils.scm?id=c0bc08d82c73e464a419f213d5ae5545bc67e2bf#n87

Best,
-- 
Josselin Poiret

Re: Shell commands with output to string

[Olivier Dion via General Guile related discussions]

On Wed, 23 Feb 2022, Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> wrote:

>> There's ton of missing stuffs in the standard library IMO.  On top of my
>> head, filesystem paths manipulation (e.g. path-join) is also one that is
>> probably getting re-invented a lots.
>
> I actually made something for that, trying to copy mostly the Python
> behavior for os.path.join:
>
> https://notabug.org/ZelphirKaltstahl/guile-fslib
>
> Also available as a GNU Guix package, but not updated in a while on
> Guix.  Repository contains more up to date version.

This is great and should be merged into the standard library of Guile.
Maybe not at it's (I have not read everything), but this would be trully
benifical to all Guile users.

By keeping this as an external library such as a Guix package, I fear
we're aiming at the 'nodejs syndrome' where everything -- down to
the simplest hello world -- is a package dependency.

>> I believe that the successful story of Python is not just about its
>> pretty syntax, but also dues to its very large standard library.
>
> I think so too. Although I sometimes have the feeling, that Guile does
> things in a cleaner way, once one figures out how to do them in the
> first place. One thing I really like are the ports. Stuff like
> call-with-output-string. Takes some twisting of the brain, but once
> one gets it, it becomes very useful and elegant.

Of course Scheme does things better than Python.  I wouldn't be hacking
with it if it was not the case ^^.  Not only the syntax, but the
programming paradigms.  You want FP, you got it.  You want OOP, sure.
You prefer procedural, no problem.  With Python it's more like just OOP
although there's been progress on the FP side.

> But yes, Python is very beginner friendly in terms of batteries
> included.  Although I think that its syntax feels a bit ad-hoc. As in
> "Oh we want some syntax for X … lets invent this keyword here. or some
> new operators or things like that."

Yeah they were late to the FP paradigm so they had to invent something
without breaking their OOP echo chamber I guess.  IIRC, there's talks
about adding patterns matching which Lisp has since the begining.  So
yeah very late to the game of FP.

> I like Guile syntax (or Scheme in general) much more. However, it is
> difficult to motivate others in a quick demo to learn the language,
> when you cannot take half an hour time to explain, what that for other
> people weird looking syntax is actually really cool.

I'm not sure if it's only the syntax.  Yes we often see "but the
parentheses burn my eyes" but I believe that things are more complex
than that.  What I think is difficult is to explain to someone why they
should try to learn new languages in the first place.  On top of my
head, people ask themself the following question before learning a new
language:

      - Profit -> Can I easily find a good paying job by developing in
        this language?
      
      - Field -> Can I develop in my field (e.g. video game, mathematic)
        with this language?  Is there a community?

      - Support -> Can I find solutions to my problems easily online?
        Are there libraries for common problems that I could encounter?

      - Prototyping -> How easy is it for me to scratch an idea with
        this language?

and probably other reasons.  Now, try to explain to someone that does
web developpement in Javascript or Python that Scheme is far more
supperior.  That person would just ask: Does Guile has support for
MongoDB?  And proceed to dismiss it.

Of course, some hackers like me just like to learn new stuffs.  It took
me about an one hour into Scheme course of the MIT on youtube to
understand that I just found the holy grail of programming language.
And yet, I have not find the answer to the `profit` question.

-- 
Olivier Dion
Polymtl

Re: Shell commands with output to string

[Blake Shaw]

Olivier Dion via General Guile related discussions <guile-user@gnu.org>
writes:

> On Wed, 23 Feb 2022, Zelphir Kaltstahl <zelphirkaltstahl@posteo.de>
> wrote:
>
>>> There's ton of missing stuffs in the standard library IMO.  On top
>>> of my
>>> head, filesystem paths manipulation (e.g. path-join) is also one
>>> that is
>>> probably getting re-invented a lots.
>>
Nice work Zelphir!
>> I actually made something for that, trying to copy mostly the Python
>> behavior for os.path.join:
>>
>> https://notabug.org/ZelphirKaltstahl/guile-fslib
>>
>> Also available as a GNU Guix package, but not updated in a while on
>> Guix.  Repository contains more up to date version.
>
> This is great and should be merged into the standard library of Guile.
> Maybe not at it's (I have not read everything), but this would be trully
> benifical to all Guile users.
>
I very much disagree. While it's a large implementation of Scheme, Guile
is still Scheme. This is all really simple stuff, and there are many
ways to implement these functions. Do the Guile maintainers really need
to be burdened by adding trivial functionality to the core library?

This is really simple stuff, but it was nevertheless difficult for me to
figure out how to do much of it in Guile. Many things that confused me
to death ended up being a big "oh, its just that??" Which seems to
indicate that the issue is less with the core library lacking
functionality and more with the presentation of Guile itself -- the
docs, which in December we heard a large number of complaints about.

> By keeping this as an external library such as a Guix package, I fear
> we're aiming at the 'nodejs syndrome' where everything -- down to
> the simplest hello world -- is a package dependency.
>
>>> I believe that the successful story of Python is not just about its
>>> pretty syntax, but also dues to its very large standard library.
>>
>> I think so too. Although I sometimes have the feeling, that Guile does
>> things in a cleaner way, once one figures out how to do them in the
>> first place. One thing I really like are the ports. Stuff like
>> call-with-output-string. Takes some twisting of the brain, but once
>> one gets it, it becomes very useful and elegant.
>
I agree, there is just a mental leap getting to there. I think with
examples based on the "expressive graduation" strategy I outlined in my
Guix Days presentation can lessen that load quite a bit, and get folks
moving with the goods they need to build their own tools quickly.
> Of course Scheme does things better than Python.  I wouldn't be hacking
> with it if it was not the case ^^.  Not only the syntax, but the
> programming paradigms.  You want FP, you got it.  You want OOP, sure.
> You prefer procedural, no problem.  With Python it's more like just OOP
> although there's been progress on the FP side.
>
>> But yes, Python is very beginner friendly in terms of batteries
>> included.  Although I think that its syntax feels a bit ad-hoc. As in
>> "Oh we want some syntax for X … lets invent this keyword here. or some
>> new operators or things like that."
>
> Yeah they were late to the FP paradigm so they had to invent something
> without breaking their OOP echo chamber I guess.  IIRC, there's talks
> about adding patterns matching which Lisp has since the begining.  So
> yeah very late to the game of FP.
>
>> I like Guile syntax (or Scheme in general) much more. However, it is
>> difficult to motivate others in a quick demo to learn the language,
>> when you cannot take half an hour time to explain, what that for other
>> people weird looking syntax is actually really cool.
>
> I'm not sure if it's only the syntax.  Yes we often see "but the
> parentheses burn my eyes" but I believe that things are more complex
> than that.  What I think is difficult is to explain to someone why they
> should try to learn new languages in the first place.  On top of my
> head, people ask themself the following question before learning a new
> language:
>
>       - Profit -> Can I easily find a good paying job by developing in
>         this language?
So why are Haskell and Rust some of the most popular languages today
besides the "big names" (which I don't its reasonable to imagine a scheme
implementation suddenly breaking the mold and acceding to)
>
>       - Field -> Can I develop in my field (e.g. video game, mathematic)
>         with this language?  Is there a community?
Yes and yes to those fields, with great tools! (Haunt, recent vector libs)
But the community, like many a scheme, is a bit hermetic, and this is
based in practices like mailing lists, which I don't think can/will
change. We could start a guile-hackers discourse server, or something
else more public-facing, but would any of the current users join?

So then we are the Day Zero guile crew, and thats gonna be rough to get
going.
>
>       - Support -> Can I find solutions to my problems easily online?
>         Are there libraries for common problems that I could encounter?
This! While I think the Guile website is really nice, I think something
like a guile wiki which catalogues everything in one place, with a firm
organizational strategy from the outset, could be massively beneficial.

Finding whats happening in Guile is labyrinth, and I still discover new
stuff every couple weeks. The ecosystem is immensely larger than I
imagined a few months ago. And also, lots (maybe half?) of of it isn't
even in Guix, and I'm talking about recent projects. Meanwhile much of
what is in Guix builds but doesn't work.
>
>       - Prototyping -> How easy is it for me to scratch an idea with
>         this language?
This too! I think that this is what the Guile-Cookbook discussion is
hoping to solve
>
> and probably other reasons.  Now, try to explain to someone that does
> web developpement in Javascript or Python that Scheme is far more
> supperior.  That person would just ask: Does Guile has support for
> MongoDB?  And proceed to dismiss it.
>
> Of course, some hackers like me just like to learn new stuffs.  It took
> me about an one hour into Scheme course of the MIT on youtube to
> understand that I just found the holy grail of programming language.
> And yet, I have not find the answer to the `profit` question.

I agree with much of what you're saying, but I disagree with the idea of
Guile becoming a "batteries included" Scheme loaded with conveniences.
Andy et al have far more important issues to worry about.

On the other hand, I would be down to organize and contribute to an
effort to create a battery pack, based on a study of the utilities
offered by Python. We saw that Guix is picking up in production use
and such a library would be very helpful (and I think Tropin's RDE
might be trying to accomplish something like this)

There is the existing Guile-lib, which seems to have intended something
similar in the past. Let me briefly comment on its TOC:

> (apicheck)
>   Describe and verify library programming interfaces
So it tells you what a library does? Could be cool,
but not what I'm looking for as a newcomer. 
> (config load)
>   Loading configuration files
of what? where am I getting these configuration files from?
> (container async-queue)
>   A thread-safe message queue
Sounds like great, powerful tools... if you have already adjusted to
hacking guile
> (container nodal-tree)
>    A tree consisting of nodes with attributes
Ditto.
> (container delay-tree)
>    A nodal tree with lazily evaluated fields
Ditto.
> (debugging assert)
>    Helpful assert macro
This is something I think newcomers should be exposed to early on,
> (debugging time)
>    A simple macro to time the execution of an expression
Ditto.
>(graph topological-sort)
>    Routines to perform topological sorts
if you came for topological sort, guiles got you covered
>(htmlprag)
>    Neil Van Dyke's permissive ("pragmatic") HTML parser
nice, super useful
>(io string)
>    SLIB's IO routines dealing with strings
Ditto
>(logging logger)
>    A flexible logging system
every guix dream, yet I doubt most know this is here
>(logging port-log)
>    A logger that outputs to a port
ditto
>(logging rotating-log)
>    A logger that rotates its output files
ditto
>(match-bind)
>    Nifty and concise regular expression routines
I wasn looking for such a thing weeks ago... another reason a wiki or
some other quick and simple to edit archive of everything in guile would
be nice
>(math minima)
>    A golden-section minimum finder
sicp vibes over here
>(math primes)
>    Functions related to prime numbers and factorization
ditto
>(os process)
>    Spawning processes and capturing their output
where has this been my whole guile experience?!? too late, I made one
myself (:
> (scheme documentation)
>    Macros to define different kinds of variables with documentation
this sounds great! didn't catch it my first few glances through
>(scheme kwargs)
>    Defining functions with flexible keyword arguments
how is that different from existing kwargs?
>(search basic)
>    Classic search functions
useful but also the kind of thing one typically implements to learn a lang 
>(string completion)
>    Building blocks for tab completion
thats cool! but why didnt they just go ahead and... complete it?
>(string soundex)
>    The SOUNDEX string categorization algorithm
no idea what that is
>(string transform)
>    Beyond SRFI-13
would be nice to have subtext say a bit more
>(string wrap)
>    A versatile string formatter
why do I need more than (format ...)?
>(term ansi-color)
>    Generate ANSI color escape sequences
nice!
>(unit-test)
>    A JUnit-style unit testing framework
nice

As you can see, much of this is stuff that won't come in handy until you
already feel compfortable with guile.

In fact, some of it will just seem like weird alien technology until
you've spent some time with scheme.

So while I agree that an organized effort to roll-out a battery pack
would be hugely beneficial, I think efforts to make the existing
knowledge base of guile more accessible/easier to navigate/less messy
would get folks moving with the large ecosystem faster,
speaking/thinking in guile, and thus would be more rewarding than
porting conventional idioms into a box that makes guile more familiar
for python and JS developers.


-- 
“In girum imus nocte et consumimur igni”

Re: Shell commands with output to string

[Olivier Dion via General Guile related discussions]

On Thu, 24 Feb 2022, Blake Shaw <blake@nonconstructivism.com> wrote:
> Olivier Dion via General Guile related discussions <guile-user@gnu.org>
> writes:
>> This is great and should be merged into the standard library of Guile.
>> Maybe not at it's (I have not read everything), but this would be trully
>> benifical to all Guile users.
>>
> I very much disagree. While it's a large implementation of Scheme,
> Guile is still Scheme. This is all really simple stuff, and there are
> many ways to implement these functions. Do the Guile maintainers
> really need to be burdened by adding trivial functionality to the core
> library?

It it's simple stuff, but as you mentioned, it's difficult to determine
how things are done in Guile as a beginer.  So maybe the problem is not
the lack of functionality, but lack of knowledge/understanding.

I for one am a big advocate of do your own stuff if you can.  That's
what I do for my projects.  But some people prefers to just glue stuffs
together instead and that's okay too.

>>       - Field -> Can I develop in my field (e.g. video game, mathematic)
>>         with this language?  Is there a community?
> Yes and yes to those fields, with great tools! (Haunt, recent vector
> libs) But the community, like many a scheme, is a bit hermetic, and
> this is based in practices like mailing lists, which I don't think
> can/will change. We could start a guile-hackers discourse server, or
> something else more public-facing, but would any of the current users
> join?  So then we are the Day Zero guile crew, and thats gonna be
> rough to get going.

I think that would certainly reach out to more people.  In my experience
-- at least where I live -- people in their mid 20 like me don't use
mailing list or see them as a thing of the past of dead project.

> I agree with much of what you're saying, but I disagree with the idea
> of Guile becoming a "batteries included" Scheme loaded with
> conveniences.  Andy et al have far more important issues to worry
> about.

I don't think that Guile should have a batteries included standard
library like Python does.  Maybe just a few more friendly functions that
are commonly used.  And I certainly don't want to impose any burden on
anyone to implement them!

> On the other hand, I would be down to organize and contribute to an
> effort to create a battery pack, based on a study of the utilities
> offered by Python. We saw that Guix is picking up in production use
> and such a library would be very helpful (and I think Tropin's RDE
> might be trying to accomplish something like this)

I think this would be the best option.  Leaving it to the community.
This would remove the burden on the maintainers while making the library
bigger and bigger while also ensuring the prefer coding style of the
community.  However, we should probably also lookg at what is done on
the Racket side.  They have very nice things like type system and
contract programming!

> There is the existing Guile-lib, which seems to have intended something
> similar in the past. Let me briefly comment on its TOC:

Everytime I've looked into it <https://www.nongnu.org/guile-lib/> I feel
like it was a dead project by the look of the website.  Even though its
last update was in 2021.  The documentation is also lacking in my souvenir.
But something like this is a good start.

>>(os process)
>>    Spawning processes and capturing their output
> where has this been my whole guile experience?!? too late, I made one
> myself (:

We all did I believe ^^

>>(scheme kwargs)
>>    Defining functions with flexible keyword arguments
> how is that different from existing kwargs?

I think this was before curried definitions were introduced?

> As you can see, much of this is stuff that won't come in handy until
> you already feel compfortable with guile.
>
> In fact, some of it will just seem like weird alien technology until
> you've spent some time with scheme.
>
> So while I agree that an organized effort to roll-out a battery pack
> would be hugely beneficial, I think efforts to make the existing
> knowledge base of guile more accessible/easier to navigate/less messy
> would get folks moving with the large ecosystem faster,
> speaking/thinking in guile, and thus would be more rewarding than
> porting conventional idioms into a box that makes guile more familiar
> for python and JS developers.

Refactoring of the documentation I think we all agree on that after your
talk last week and its echo on the ML.  I also believe that things need
to be modernized a bit.  Like your discourse proposition for discussion,
but also things like the website of guile-lib.

-- 
Olivier Dion
Polymtl

Re: Shell commands with output to string

[Zelphir Kaltstahl]

Hello Oliver!

On 2/23/22 15:13, Olivier Dion wrote:
> On Wed, 23 Feb 2022, Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> wrote:
>
>>> There's ton of missing stuffs in the standard library IMO.  On top of my
>>> head, filesystem paths manipulation (e.g. path-join) is also one that is
>>> probably getting re-invented a lots.
>> I actually made something for that, trying to copy mostly the Python
>> behavior for os.path.join:
>>
>> https://notabug.org/ZelphirKaltstahl/guile-fslib
>>
>> Also available as a GNU Guix package, but not updated in a while on
>> Guix.  Repository contains more up to date version.
> This is great and should be merged into the standard library of Guile.
> Maybe not at it's (I have not read everything), but this would be trully
> benifical to all Guile users.
>
> By keeping this as an external library such as a Guix package, I fear
> we're aiming at the 'nodejs syndrome' where everything -- down to
> the simplest hello world -- is a package dependency.

If there is any wish to include it in the standard library, I'd be happy to help 
out in any way. Might need help setting everything up so that stuff is 
acceptable for the standard library, but it would be great to be able to 
contribute in this way. It might allow me to give something back to Guile, for 
all the joy it has given me.

>>> I believe that the successful story of Python is not just about its
>>> pretty syntax, but also dues to its very large standard library.
>> I think so too. Although I sometimes have the feeling, that Guile does
>> things in a cleaner way, once one figures out how to do them in the
>> first place. One thing I really like are the ports. Stuff like
>> call-with-output-string. Takes some twisting of the brain, but once
>> one gets it, it becomes very useful and elegant.
> Of course Scheme does things better than Python.  I wouldn't be hacking
> with it if it was not the case ^^.  Not only the syntax, but the
> programming paradigms.  You want FP, you got it.  You want OOP, sure.
> You prefer procedural, no problem.  With Python it's more like just OOP
> although there's been progress on the FP side.
>
>> But yes, Python is very beginner friendly in terms of batteries
>> included.  Although I think that its syntax feels a bit ad-hoc. As in
>> "Oh we want some syntax for X … lets invent this keyword here. or some
>> new operators or things like that."
> Yeah they were late to the FP paradigm so they had to invent something
> without breaking their OOP echo chamber I guess.  IIRC, there's talks
> about adding patterns matching which Lisp has since the begining.  So
> yeah very late to the game of FP.
>
>> I like Guile syntax (or Scheme in general) much more. However, it is
>> difficult to motivate others in a quick demo to learn the language,
>> when you cannot take half an hour time to explain, what that for other
>> people weird looking syntax is actually really cool.
> I'm not sure if it's only the syntax.  Yes we often see "but the
> parentheses burn my eyes" but I believe that things are more complex
> than that.  What I think is difficult is to explain to someone why they
> should try to learn new languages in the first place.  On top of my
> head, people ask themself the following question before learning a new
> language:
>
>        - Profit -> Can I easily find a good paying job by developing in
>          this language?
>        
>        - Field -> Can I develop in my field (e.g. video game, mathematic)
>          with this language?  Is there a community?
>
>        - Support -> Can I find solutions to my problems easily online?
>          Are there libraries for common problems that I could encounter?
>
>        - Prototyping -> How easy is it for me to scratch an idea with
>          this language?
>
> and probably other reasons.  Now, try to explain to someone that does
> web developpement in Javascript or Python that Scheme is far more
> supperior.  That person would just ask: Does Guile has support for
> MongoDB?  And proceed to dismiss it.

And I myself had some issues trying to use sqlite recently, which I have not yet 
confirmed to not only be silly mistakes I made.

I wish people were looking more at long term consequences of technology choices, 
but of course the questions you list are important considerations for people, 
who are not in it for the learning and tinkering experience.

> Of course, some hackers like me just like to learn new stuffs.  It took
> me about an one hour into Scheme course of the MIT on youtube to
> understand that I just found the holy grail of programming language.
> And yet, I have not find the answer to the `profit` question.

Best regards,
Zelphir

-- 
repositories: https://notabug.org/ZelphirKaltstahl

Re: Shell commands with output to string

[Zelphir Kaltstahl]

Hello Josselin and all!

On 2/23/22 15:01, Josselin Poiret wrote:
> Hello,
>
>
> post@thomasdanckaert.be  writes:
>
>> Hi,
>>
>> to throw in an example: I once used a function like the one below to
>> handle stdout and stderr from external commands (from
>> https://github.com/tdanckaert/jobview/blob/master/jobtools.scm#L38  ).
>> Probably far from perfect (my first and only scheme project...), but
>> hopefully it gives you an idea.
> Just chiming in to say that [1] isn't fixed yet, so you may run into
> issues if you try to redirect out and err to the same port.  In Guix, we
> use the following workaround for now ([2]):
> --8<---------------cut here---------------start------------->8---
>    (match-let (((input . output) (pipe)))
>      ;; Hack to work around Guile bug 52835
>      (define dup-output (duplicate-port output "w"))
>      ;; Void pipe, but holds the pid for close-pipe.
>      (define dummy-pipe
>        (with-input-from-file "/dev/null"
>          (lambda ()
>            (with-output-to-port output
>              (lambda ()
>                (with-error-to-port dup-output
>                  (lambda ()
>                    (apply open-pipe* (cons "" command)))))))))
>      (close-port output)
>      (close-port dup-output)
>      (handler input)
>      (close-port input)
>      (close-pipe dummy-pipe))
> --8<---------------cut here---------------end--------------->8---
>
> [1]https://debbugs.gnu.org/cgi/bugreport.cgi?bug=52835
> [2]https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/installer/utils.scm?id=c0bc08d82c73e464a419f213d5ae5545bc67e2bf#n87
>
> Best,

I have questions regarding this workaround:

Can you explain how and why this works? I have tried to make sense of it and 
here are my notes so far (reference: 
https://notabug.org/ZelphirKaltstahl/guile-examples/src/2dead9f7bb9b40fc26eb490a93e1dc7abca7252c/shell/system-asterisk-stdout-to-stderr-redirection-bug.scm):

~~~~
(match-let (((input . output) (pipe)))
   ;; Hack to work around Guile bug 52835 -- How does
   ;; duplicating the port help? From the docs: "Returns a
   ;; new port which is opened on a duplicate of the file
   ;; descriptor underlying port, with mode string modes as
   ;; for open-file. The two ports will share a file position
   ;; and file status flags. [...]"
   (define dup-output (duplicate-port output "w"))
   ;; Void pipe, but holds the pid for close-pipe.
   (define dummy-pipe
     ;; Set current-input-port to /dev/null. -- What will be
     ;; read from there? Nothing?
     (with-input-from-file "/dev/null"
       (lambda ()
         ;; Set the current-output-port to the one created
         ;; above using (pipe).
         (with-output-to-port output
           (lambda ()
             ;; Set the error port to the duplicated output
             ;; port. This might be the redirection of stderr
             ;; to stdout.
             (with-error-to-port dup-output
               (lambda ()
                 ;; Run open-file*, but why is there an empty
                 ;; string prepended to command? Perhaps to
                 ;; allow using either a list or a string as
                 ;; a command?
                 (apply open-pipe* (cons "" command)))))))))

   (close-port output)
   (close-port dup-output)
   (handler input)
   (close-port input)
   (close-pipe dummy-pipe))
~~~~

My other question is: Do I still need this workaround, if I use the following, 
to run commands? And if so, why? In which cases would my code not do the right 
thing? (reference: 
https://notabug.org/ZelphirKaltstahl/guile-examples/src/2dead9f7bb9b40fc26eb490a93e1dc7abca7252c/shell/example-03-using-popen-get-out-and-error.scm):

~~~~
(import (ice-9 popen)
         (ice-9 textual-ports)
         (ice-9 exceptions)
         (ice-9 receive)
         (ice-9 match))


;; Removed comments to shorting this example. For more
;; explanation see the first example.
(define run-command
   (λ (cmd)
     "Runs CMD as an external process, with an input port
from which the process' stdout may be read."
     (match-let ([(err-read . err-write) (pipe)]
                 [stderr (current-error-port)])
       (with-error-to-port err-write
         (λ ()
           (let* (;; Run the actual command. If an error
                  ;; happens, it should write to the
                  ;; err-write port. Output of the command
                  ;; should be written to an output port,
                  ;; which corresponds to the input-port,
                  ;; which is returned by open-input-pipe.
                  [in-port (open-input-pipe cmd)]
                  ;; Read in block mode.
                  [_ignored (setvbuf in-port 'block)]
                  ;; Get command output and error output.
                  [command-output (get-string-all in-port)]
                  ;; Get the exit code of the command.
                  [exit-code (close-pipe in-port)])
             ;; Close the port, to which the child process
             ;; was to write errors, as the child process has
             ;; finished (either successfully or
             ;; unsuccessfully, but definitely finished).
             (close-port err-write)
             (let (;; Get the error message, if there is any.
                   [error-message (get-string-all err-read)])
               (values exit-code
                       command-output
                       error-message))))))))


(receive (exit-code command-output error-message)
     (let ([command "echo 'bong' 1>&2"])
       (run-command command))
   (display (simple-format #f "exit code: ~a\n" exit-code))
   (unless (string-null? command-output)
     (display (simple-format #f "command-output: \n~a" command-output)))
   (unless (string-null? error-message)
     (display (simple-format #f "error-message: \n~a" error-message))))


(receive (exit-code command-output error-message)
     (let ([command "ls -al"])
       (run-command command))
   (display (simple-format #f "exit code: ~a\n" exit-code))
   (unless (string-null? command-output)
     (display (simple-format #f "command-output: \n~a" command-output)))
   (unless (string-null? error-message)
     (display (simple-format #f "error-message: \n~a" error-message))))


;; Both, output and error:
(receive (exit-code command-output error-message)
     (let ([command "ls -al 2>&1 && echo 'bong' 1>&2"])
       (run-command command))
   (display (simple-format #f "exit code: ~a\n" exit-code))
   (unless (string-null? command-output)
     (display (simple-format #f "command-output: \n~a" command-output)))
   (unless (string-null? error-message)
     (display (simple-format #f "error-message: \n~a" error-message))))
~~~~

Best regards,
Zelphir

-- 
repositories:https://notabug.org/ZelphirKaltstahl

Re: Shell commands with output to string

[Josselin Poiret]

Hello Zelphir,

Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> writes:
> I have questions regarding this workaround:
>
> Can you explain how and why this works? I have tried to make sense of it and 
> here are my notes so far (reference: 
> https://notabug.org/ZelphirKaltstahl/guile-examples/src/2dead9f7bb9b40fc26eb490a93e1dc7abca7252c/shell/system-asterisk-stdout-to-stderr-redirection-bug.scm):

Note that here, I used undocumented behavior of open-pipe*, which can be
understood by inspecting libguile/posix.c (look for piped-process) and
module/ice-9/popen.scm (look for open-pipe*).

> ~~~~
> (match-let (((input . output) (pipe)))
>    ;; Hack to work around Guile bug 52835 -- How does
>    ;; duplicating the port help? From the docs: "Returns a
>    ;; new port which is opened on a duplicate of the file
>    ;; descriptor underlying port, with mode string modes as
>    ;; for open-file. The two ports will share a file position
>    ;; and file status flags. [...]"
>    (define dup-output (duplicate-port output "w"))

The above Guile bug occurs because the current output and error ports
point to the same file descriptor.  Using duplicate-port makes sure that
we get a port with a duplicated file descriptor!

>    ;; Void pipe, but holds the pid for close-pipe.
>    (define dummy-pipe
>      ;; Set current-input-port to /dev/null. -- What will be
>      ;; read from there? Nothing?
>      (with-input-from-file "/dev/null"

Yes, for our use-case we don't need to feed anything to the command, but
it's always possible to hook this up to a pipe if you need to.

>        (lambda ()
>          ;; Set the current-output-port to the one created
>          ;; above using (pipe).
>          (with-output-to-port output
>            (lambda ()
>              ;; Set the error port to the duplicated output
>              ;; port. This might be the redirection of stderr
>              ;; to stdout.
>              (with-error-to-port dup-output

Exactly, this is the redirection.

>                (lambda ()
>                  ;; Run open-file*, but why is there an empty
>                  ;; string prepended to command? Perhaps to
>                  ;; allow using either a list or a string as
>                  ;; a command?
>                  (apply open-pipe* (cons "" command)))))))))

Here's the undefined behavior, the first argument of open-pipe* is a
mode for the pipe it opens, but here we don't want it to open any pipes,
all our default ports are setup so that start_child will set the child's
stdin/out/err to their file descriptors.

>    (close-port output)
>    (close-port dup-output)
>    (handler input)
>    (close-port input)
>    (close-pipe dummy-pipe))
> ~~~~
>
> My other question is: Do I still need this workaround, if I use the following, 
> to run commands? And if so, why? In which cases would my code not do the right 
> thing? (reference: 
> https://notabug.org/ZelphirKaltstahl/guile-examples/src/2dead9f7bb9b40fc26eb490a93e1dc7abca7252c/shell/example-03-using-popen-get-out-and-error.scm):

Looking briefly at your code, I don't think that bug could be affecting
you.  You can have an issue if you're trying to redirect a standard fd
to another standard fd, but if you're using fresh ports it should be ok.

Best,
-- 
Josselin Poiret