Threading / Pipe Macro

Threading / Pipe Macro

[Zelphir Kaltstahl]

Hi Guile Users!

I recently looked at some online course about Elixir and saw an elegant
use of pipes (the operator `|>` in Elixir). Then I remembered, that in
Racket there are so called threading macros, which seem to accomplish
the same:

https://github.com/lexi-lambda/threading/blob/master/threading-lib/threading/main.rkt

I also searched around for tutorials or explanations on how to write
these macros. Sometimes I found excellent documenation in the Chicken
Scheme wiki, so I checked there:

https://wiki.call-cc.org/eggref/5/pipes

However, I would like to use a threading macro or pipes in Guile. I am
seeing these options:

(1) I could start trying to translate the Racket version to Guile, maybe
it would work easily, maybe it uses Racket specific macro stuff, I don't
know. However, I am not sure I would learn how the macros actually work.
Maybe I would.

(2) I could start from zero and try to implement the pipes I saw in the
online course about Elixir.

(3) Maybe something already exists in Guile, that I am unaware of and
could not find through searching. Maybe there are even more names for pipes.

So my questions are:

(1) Does something already exist?

(2) Would translating the Racket version be an easy thing to do, or is
there something in there, that cannot so easily be achieved with
syntax-parse, syntax-case and the likes? (For someone who rarely touches
macros and does not have a lot experience writing them.)

Regards,

Zelphir

Re: Threading / Pipe Macro

[Chris Vine]

On Sun, 7 Jul 2019 12:42:03 +0200
Zelphir Kaltstahl <zelphirkaltstahl@gmail.com> wrote:
> Hi Guile Users!
> 
> I recently looked at some online course about Elixir and saw an elegant
> use of pipes (the operator `|>` in Elixir). Then I remembered, that in
> Racket there are so called threading macros, which seem to accomplish
> the same:
> 
> https://github.com/lexi-lambda/threading/blob/master/threading-lib/threading/main.rkt
> 
> I also searched around for tutorials or explanations on how to write
> these macros. Sometimes I found excellent documenation in the Chicken
> Scheme wiki, so I checked there:
> 
> https://wiki.call-cc.org/eggref/5/pipes
> 
> However, I would like to use a threading macro or pipes in Guile. I am
> seeing these options:
> 
> (1) I could start trying to translate the Racket version to Guile, maybe
> it would work easily, maybe it uses Racket specific macro stuff, I don't
> know. However, I am not sure I would learn how the macros actually work.
> Maybe I would.
> 
> (2) I could start from zero and try to implement the pipes I saw in the
> online course about Elixir.
> 
> (3) Maybe something already exists in Guile, that I am unaware of and
> could not find through searching. Maybe there are even more names for pipes.
> 
> So my questions are:
> 
> (1) Does something already exist?
> 
> (2) Would translating the Racket version be an easy thing to do, or is
> there something in there, that cannot so easily be achieved with
> syntax-parse, syntax-case and the likes? (For someone who rarely touches
> macros and does not have a lot experience writing them.)

I have a pipeline macro which sort-of mimics ML's |> pipeline operator
which I use a lot:

(define-syntax ->
  (lambda (x)
    (syntax-case x ()
      [(k exp0 . exps)
       (let* ([reversed (reverse (cons (syntax->datum #'exp0)
                                       (syntax->datum #'exps)))]
              [out (let loop ([first (car reversed)]
                              [rest (cdr reversed)])
                     (if (null? rest)
                         first
                         (let ([func (car first)]
                               [args (cdr first)])
                           (append `(,func ,@args)
                                   (list (loop (car rest) (cdrrest)))))))])
         (datum->syntax #'k out))])))

Because all the macro does is to rearrange input forms, this is hygienic
without the need to manipulate syntax objects - you can convert to a datum,
rearrange and then convert back to a syntax object again, as above.

The syntax of the -> macro is like this:

(-> (+ 3 4 5)
    (- 18)
    (format #t "~A\n"))

The first expression in the '->' block is evaluated, and its value is
passed as the last (or only) argument of the following expression, and
so on.  Each expression after the first one must be a function application
(the first one can be either a function application or a value).  The
applied function comprising each expression after the first one appears to
be curried, but in fact there is no currying, nor any runtime overhead at
all.  The above example code is expanded into the following scheme form:

(format #t "~A\n" (- 18 (+ 3 4 5)))

You can use it with a monadic bind (or applicative mapping) if you want so
as to behave in a similar way to the >>= operator, but such things have
little use in scheme in my opinion.  (I can however provide a simple
example of that if you think it would be interesting.)

Chris

Re: Threading / Pipe Macro

[Chris Vine]

On Sun, 7 Jul 2019 19:32:59 +0100
Chris Vine <vine35792468@gmail.com> wrote:
[snip]
Text munge by the editor

This:
                                     (list (loop (car rest) (cdrrest)))))))])
                                                            ^^^^^^^^^  
should be:
                                     (list (loop (car rest) (cdr rest)))))))])

Re: Threading / Pipe Macro

[Mark H Weaver]

Hi Chris,

Chris Vine <vine35792468@gmail.com> writes:

> I have a pipeline macro which sort-of mimics ML's |> pipeline operator
> which I use a lot:
>
> (define-syntax ->
>   (lambda (x)
>     (syntax-case x ()
>       [(k exp0 . exps)
>        (let* ([reversed (reverse (cons (syntax->datum #'exp0)
>                                        (syntax->datum #'exps)))]
>               [out (let loop ([first (car reversed)]
>                               [rest (cdr reversed)])
>                      (if (null? rest)
>                          first
>                          (let ([func (car first)]
>                                [args (cdr first)])
>                            (append `(,func ,@args)
>                                    (list (loop (car rest) (cdrrest)))))))])
>          (datum->syntax #'k out))])))
>
> Because all the macro does is to rearrange input forms, this is hygienic
> without the need to manipulate syntax objects - you can convert to a datum,
> rearrange and then convert back to a syntax object again, as above.

This macro is *not* hygienic.  The calls to 'syntax->datum' strip all of
the context information from the syntax objects, and then build a new
expression using raw S-expressions.  The result is essentially the same
as if you used 'define-macro'.  This results various problems.

For example:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (define-syntax ->
  (lambda (x)
    (syntax-case x ()
      [(k exp0 . exps)
       (let* ([reversed (reverse (cons (syntax->datum #'exp0)
                                       (syntax->datum #'exps)))]
              [out (let loop ([first (car reversed)]
                              [rest (cdr reversed)])
                     (if (null? rest)
                         first
                         (let ([func (car first)]
                               [args (cdr first)])
                           (append `(,func ,@args)
                                   (list (loop (car rest) (cdr rest)))))))])
         (datum->syntax #'k out))])))
scheme@(guile-user)> (define t 'global-t)
scheme@(guile-user)> (define-syntax-rule (foo x)
                       (-> x (format #t "[t=~A] ~A\n" t)))
scheme@(guile-user)> (let ((t 'inner-t)) (foo t))
[t=global-t] global-t
$1 = #t
scheme@(guile-user)> 
--8<---------------cut here---------------end--------------->8---

I recommend reformulating the -> macro using 'syntax-rules' as follows:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (define-syntax ->
                       (syntax-rules ()
                         ((-> exp)
                          exp)
                         ((-> exp ... (op args ...))
                          (op args ... (-> exp ...)))))
scheme@(guile-user)> (let ((t 'inner-t)) (foo t))
[t=global-t] inner-t
$8 = #t
scheme@(guile-user)> 
--8<---------------cut here---------------end--------------->8---

This macro is hygienic, and also easier to comprehend (IMO).  Of course,
it could also be implemented using syntax-case.  The key is to always
work with the syntax objects.

Whenever you use 'syntax->datum' on expressions that are not purely
literals, you will be sacrificing hygiene.

      Regards,
        Mark

Re: Threading / Pipe Macro

[Chris Vine]

On Sun, 07 Jul 2019 15:30:36 -0400
Mark H Weaver <mhw@netris.org> wrote:
> Hi Chris,
> 
> Chris Vine <vine35792468@gmail.com> writes:
> 
> > I have a pipeline macro which sort-of mimics ML's |> pipeline operator
> > which I use a lot:
> >
> > (define-syntax ->
> >   (lambda (x)
> >     (syntax-case x ()
> >       [(k exp0 . exps)
> >        (let* ([reversed (reverse (cons (syntax->datum #'exp0)
> >                                        (syntax->datum #'exps)))]
> >               [out (let loop ([first (car reversed)]
> >                               [rest (cdr reversed)])
> >                      (if (null? rest)
> >                          first
> >                          (let ([func (car first)]
> >                                [args (cdr first)])
> >                            (append `(,func ,@args)
> >                                    (list (loop (car rest) (cdrrest)))))))])
> >          (datum->syntax #'k out))])))
> >
> > Because all the macro does is to rearrange input forms, this is hygienic
> > without the need to manipulate syntax objects - you can convert to a datum,
> > rearrange and then convert back to a syntax object again, as above.
> 
> This macro is *not* hygienic.  The calls to 'syntax->datum' strip all of
> the context information from the syntax objects, and then build a new
> expression using raw S-expressions.  The result is essentially the same
> as if you used 'define-macro'.  This results various problems.
> 
> For example:
> 
> --8<---------------cut here---------------start------------->8---
> scheme@(guile-user)> (define-syntax ->
>   (lambda (x)
>     (syntax-case x ()
>       [(k exp0 . exps)
>        (let* ([reversed (reverse (cons (syntax->datum #'exp0)
>                                        (syntax->datum #'exps)))]
>               [out (let loop ([first (car reversed)]
>                               [rest (cdr reversed)])
>                      (if (null? rest)
>                          first
>                          (let ([func (car first)]
>                                [args (cdr first)])
>                            (append `(,func ,@args)
>                                    (list (loop (car rest) (cdr rest)))))))])
>          (datum->syntax #'k out))])))
> scheme@(guile-user)> (define t 'global-t)
> scheme@(guile-user)> (define-syntax-rule (foo x)
>                        (-> x (format #t "[t=~A] ~A\n" t)))
> scheme@(guile-user)> (let ((t 'inner-t)) (foo t))
> [t=global-t] global-t
> $1 = #t
> scheme@(guile-user)> 
> --8<---------------cut here---------------end--------------->8---
> 
> I recommend reformulating the -> macro using 'syntax-rules' as follows:
> 
> --8<---------------cut here---------------start------------->8---
> scheme@(guile-user)> (define-syntax ->
>                        (syntax-rules ()
>                          ((-> exp)
>                           exp)
>                          ((-> exp ... (op args ...))
>                           (op args ... (-> exp ...)))))
> scheme@(guile-user)> (let ((t 'inner-t)) (foo t))
> [t=global-t] inner-t
> $8 = #t
> scheme@(guile-user)> 
> --8<---------------cut here---------------end--------------->8---
> 
> This macro is hygienic, and also easier to comprehend (IMO).  Of course,
> it could also be implemented using syntax-case.  The key is to always
> work with the syntax objects.
> 
> Whenever you use 'syntax->datum' on expressions that are not purely
> literals, you will be sacrificing hygiene.

How strange.  Both your and my macro gives 'global-t' when I test them,
which is the result I would expect.  (Maybe I am missing something here,
but a result of 'inner-t' would seem to me to imply unhygiene.)

However if I change my macro to manipulate syntax objects I do get
'inner-t'

(define-syntax -->
  (lambda (x)
    (syntax-case x ()
      [(_ exp0 exp1 ...)
       (let ([reversed (reverse #'(exp0 exp1 ...))])
	 (with-syntax
	     ([out
	       (let loop ([first (car reversed)]
			  [rest (cdr reversed)])
		 (if (null? rest)
		     first
		     (syntax-case first ()
		       [(func arg0 ...)
			(append #'(func arg0 ...)
				(list (loop (car rest) (cdr rest))))])))])
	   #'out))])))

I need to think more about this and/or reproduce this later.

This is with guile-2.2.6 by the way.

Chris

Re: Threading / Pipe Macro

[Mark H Weaver]

Hi Chris,

Here's a complete, unedited transcript with Guile 2.2.6:

--8<---------------cut here---------------start------------->8---
mhw@jojen ~$ guile
GNU Guile 2.2.6
Copyright (C) 1995-2019 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (define-syntax ->
  (lambda (x)
    (syntax-case x ()
      [(k exp0 . exps)
       (let* ([reversed (reverse (cons (syntax->datum #'exp0)
                                       (syntax->datum #'exps)))]
              [out (let loop ([first (car reversed)]
                              [rest (cdr reversed)])
                     (if (null? rest)
                         first
                         (let ([func (car first)]
                               [args (cdr first)])
                           (append `(,func ,@args)
                                   (list (loop (car rest) (cdr rest)))))))])
         (datum->syntax #'k out))])))
scheme@(guile-user)> (define t 'global-t)
scheme@(guile-user)> (define-syntax-rule (foo x)
                       (-> x (format #t "[t=~A] ~A\n" t)))
scheme@(guile-user)> (let ((t 'inner-t))
                       (foo t))
[t=global-t] global-t
$1 = #t
scheme@(guile-user)> (define-syntax ->
                       (syntax-rules ()
                         ((-> exp)
                          exp)
                         ((-> exp ... (op args ...))
                          (op args ... (-> exp ...)))))
scheme@(guile-user)> (let ((t 'inner-t))
                       (foo t))
[t=global-t] inner-t
$2 = #t
scheme@(guile-user)>
--8<---------------cut here---------------end--------------->8---

Chris Vine <vine35792468@gmail.com> writes:
> How strange.  Both your and my macro gives 'global-t' when I test them,

Can you show me a complete, unedited transcript that demonstrates what
you're seeing?

> which is the result I would expect.  (Maybe I am missing something here,
> but a result of 'inner-t' would seem to me to imply unhygiene.)

(foo EXPR) is supposed to print "[t=global-t] VAL", where VAL is the
result of evaluating EXPR.  With this in mind,

    (let ((t 'inner-t))
      (foo t))

The argument to 'foo' here should refer to the lexical binding of 't',
i.e. the variable with value 'inner-t'.  I'm curious what would make you
expect otherwise.

On the other hand, the reference to 't' in the template of the 'foo'
macro should refer to the toplevel variable 't', because the template
does not appear within the 'let'.

This is a good example of why syntax objects are needed, to distinguish
between these two references to distinct variables named 't'.  When you
convert the references to datums, the distinctions are lost.

     Regards,
       Mark

Re: Threading / Pipe Macro

[Chris Vine]

On Sun, 7 Jul 2019 21:16:13 +0100
Chris Vine <vine35792468@gmail.com> wrote:
> I need to think more about this and/or reproduce this later.
> 
> This is with guile-2.2.6 by the way.

OK, I did set up the test of your macro incorrectly (the one using
syntax-rules): tested properly, as you say it produces

  [t=global-t] inner-t'.

So does my revised syntax-case version which manipulates syntax objects.
I also agree that that is the answer that should be produced.  My
version was hygienic when used in functions, but not when used as a
macro in a macro.

Chris

Re: Threading / Pipe Macro

[Mark H Weaver]

Hi Chris,

Chris Vine <vine35792468@gmail.com> writes:
> My version was hygienic when used in functions, but not when used as a
> macro in a macro.

The word "hygienic" should not be used to describe your original macro.
It would be more accurate to say that although your original version was
unhygienic, in practice it would work in many common cases.  That is
generally the case for unhygienic macros.

     Best,
      Mark

Re: Threading / Pipe Macro

[Chris Vine]

On Sun, 07 Jul 2019 18:24:32 -0400
Mark H Weaver <mhw@netris.org> wrote:
> Hi Chris,
> 
> Here's a complete, unedited transcript with Guile 2.2.6:
> 
> --8<---------------cut here---------------start------------->8---
> mhw@jojen ~$ guile
> GNU Guile 2.2.6
> Copyright (C) 1995-2019 Free Software Foundation, Inc.
> 
> Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
> This program is free software, and you are welcome to redistribute it
> under certain conditions; type `,show c' for details.
> 
> Enter `,help' for help.
> scheme@(guile-user)> (define-syntax ->
>   (lambda (x)
>     (syntax-case x ()
>       [(k exp0 . exps)
>        (let* ([reversed (reverse (cons (syntax->datum #'exp0)
>                                        (syntax->datum #'exps)))]
>               [out (let loop ([first (car reversed)]
>                               [rest (cdr reversed)])
>                      (if (null? rest)
>                          first
>                          (let ([func (car first)]
>                                [args (cdr first)])
>                            (append `(,func ,@args)
>                                    (list (loop (car rest) (cdr rest)))))))])
>          (datum->syntax #'k out))])))
> scheme@(guile-user)> (define t 'global-t)
> scheme@(guile-user)> (define-syntax-rule (foo x)
>                        (-> x (format #t "[t=~A] ~A\n" t)))
> scheme@(guile-user)> (let ((t 'inner-t))
>                        (foo t))
> [t=global-t] global-t
> $1 = #t
> scheme@(guile-user)> (define-syntax ->
>                        (syntax-rules ()
>                          ((-> exp)
>                           exp)
>                          ((-> exp ... (op args ...))
>                           (op args ... (-> exp ...)))))
> scheme@(guile-user)> (let ((t 'inner-t))
>                        (foo t))
> [t=global-t] inner-t
> $2 = #t
> scheme@(guile-user)>
> --8<---------------cut here---------------end--------------->8---
> 
> Chris Vine <vine35792468@gmail.com> writes:
> > How strange.  Both your and my macro gives 'global-t' when I test them,
> 
> Can you show me a complete, unedited transcript that demonstrates what
> you're seeing?
> 
> > which is the result I would expect.  (Maybe I am missing something here,
> > but a result of 'inner-t' would seem to me to imply unhygiene.)
> 
> (foo EXPR) is supposed to print "[t=global-t] VAL", where VAL is the
> result of evaluating EXPR.  With this in mind,
> 
>     (let ((t 'inner-t))
>       (foo t))
> 
> The argument to 'foo' here should refer to the lexical binding of 't',
> i.e. the variable with value 'inner-t'.  I'm curious what would make you
> expect otherwise.
> 
> On the other hand, the reference to 't' in the template of the 'foo'
> macro should refer to the toplevel variable 't', because the template
> does not appear within the 'let'.
> 
> This is a good example of why syntax objects are needed, to distinguish
> between these two references to distinct variables named 't'.  When you
> convert the references to datums, the distinctions are lost.

You will have seen that your post crossed with mine which agreed with
you.

My assumption as it happens was that when all you were doing was
rearranging input forms then it was not possible when using it to have
"two distinct variables named 't'".  I have been proved to be wrong in
the case of your particular example of a macro in a macro.

Anyway, the original poster now has two implementations of a pipeline
operator which do work correctly, one using syntax-rules and one doing
it the harder way with syntax-case.  It had not occurred to me that
syntax-rules could actually do it.

It is a sufficiently useful macro that there seems to me to be a case
for putting it in guile.  As I mentioned, I use it frequently.

Chris

Re: Threading / Pipe Macro

[Erik Edrosa]

On 7/7/19 6:42 AM, Zelphir Kaltstahl wrote:
> 
> (1) Does something already exist?
> 

This project was posted not too long ago on this mailing list.

https://bitbucket.org/bjoli/guile-threading-macros/src/default/

- Erik

Re: Threading / Pipe Macro

[Mark H Weaver]

Hello again,

Chris Vine <vine35792468@gmail.com> writes:
> My version was hygienic when used in functions, but not when used as a
> macro in a macro.

Not to belabor the point, but I wanted to mention that the unhygienic
'->' macro may fail when used in procedures, even when '->' is never
used in the definition of another macro.  Here's an example, where 'cut'
and '->' are used in combination:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use (srfi srfi-26)
scheme@(guile-user)> (define-syntax ->
  (lambda (x)
    (syntax-case x ()
      [(k exp0 . exps)
       (let* ([reversed (reverse (cons (syntax->datum #'exp0)
                                       (syntax->datum #'exps)))]
              [out (let loop ([first (car reversed)]
                              [rest (cdr reversed)])
                     (if (null? rest)
                         first
                         (let ([func (car first)]
                               [args (cdr first)])
                           (append `(,func ,@args)
                                   (list (loop (car rest) (cdr rest)))))))])
         (datum->syntax #'k out))])))
scheme@(guile-user)> (define foo (cut -> <> (format #t "~A\n")))
;;; <stdin>:17:12: warning: possibly unbound variable `t-15fc270a-2d'
scheme@(guile-user)> (foo 4)
<unnamed port>:17:34: In procedure foo:
In procedure module-lookup: Unbound variable: t-15fc270a-2d

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,q
scheme@(guile-user)> ,expand (cut -> <> (format #t "~A\n"))
$1 = (lambda (t-15fc270a-34-1)
  (format #t "~A\n" t-15fc270a-34))
scheme@(guile-user)> (define-syntax ->
                       (syntax-rules ()
                         ((-> exp)
                          exp)
                         ((-> exp ... (op args ...))
                          (op args ... (-> exp ...)))))
scheme@(guile-user)> (define foo (cut -> <> (format #t "~A\n")))
scheme@(guile-user)> (foo 4)
4
$2 = #t
scheme@(guile-user)> ,expand (cut -> <> (format #t "~A\n"))
$3 = (lambda (t-15fc270a-59)
  (format #t "~A\n" t-15fc270a-59))
scheme@(guile-user)> 
--8<---------------cut here---------------end--------------->8---

So, more generally, unhygienic macros may cause problems when they are
used in combination with other macros.  Since macros are so ubiquitous
in Scheme, attempting to avoid such combinations is likely to be
brittle.

      Best,
       Mark

Re: Threading / Pipe Macro

[Linus Björnstam]

I am the author of that if there are any questions. It uses syntax-rules so it should be portable to any r567rs scheme by just replacing the module definition. In r6rs pure also by exporting the <> and <...> syntax, preferably in a way so that both srfi-26 and threading can be used. This does not matter in guile.

-- 
  Linus Björnstam

On Mon, 8 Jul 2019, at 02:49, Erik Edrosa wrote:
> On 7/7/19 6:42 AM, Zelphir Kaltstahl wrote:
> > 
> > (1) Does something already exist?
> > 
> 
> This project was posted not too long ago on this mailing list.
> 
> https://bitbucket.org/bjoli/guile-threading-macros/src/default/
> 
> - Erik
> 
>

Re: Threading / Pipe Macro

[Zelphir Kaltstahl]

Ah thanks! I did not find that before. Good to know where to look for more!

I should link to other places, where one can find macros in my repo.

On 7/8/19 2:49 AM, Erik Edrosa wrote:
> On 7/7/19 6:42 AM, Zelphir Kaltstahl wrote:
>> (1) Does something already exist?
>>
> This project was posted not too long ago on this mailing list.
>
> https://bitbucket.org/bjoli/guile-threading-macros/src/default/
>
> - Erik