Multiple values passed as single argument to procedure

Multiple values passed as single argument to procedure

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1495 bytes --]

Hi,

I've noticed that when one passes multiple values as a single argument
to a procedure, only the first value gets used.  Is this expected?
Here's an example:

--8<---------------cut here---------------start------------->8---
$ guile
GNU Guile 2.2.2
Copyright (C) 1995-2017 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (define (proc x) x)
scheme@(guile-user)> (proc (values 1 2))
$1 = 1
scheme@(guile-user)> 
--8<---------------cut here---------------end--------------->8---

I've looked in a few places to try to figure out the answer to this
question, but I haven't found much.  In addition to the guile-devel and
guile-user email list archives, I've checked the following sections in
the manual:

(guile) Binding Multiple Values
(guile) Lambda
(guile) Fly Evaluation
(guile) Compiled Procedures
(guile) Expression Syntax

However, I did find the following information in R6RS (Section 5.8:
"Multiple return values"), which seems possibly relevant:

"Not all continuations accept any number of values. For example, a
continuation that accepts the argument to a procedure call is guaranteed
to accept exactly one value.  The effect of passing some other number of
values to such a continuation is unspecified."

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Multiple values passed as single argument to procedure

[David Kastrup]

Chris Marusich <cmmarusich@gmail.com> writes:

> I've noticed that when one passes multiple values as a single argument
> to a procedure, only the first value gets used.  Is this expected?

Yes.

> However, I did find the following information in R6RS (Section 5.8:
> "Multiple return values"), which seems possibly relevant:
>
> "Not all continuations accept any number of values. For example, a
> continuation that accepts the argument to a procedure call is guaranteed
> to accept exactly one value.  The effect of passing some other number of
> values to such a continuation is unspecified."

Guile resolves this unspecification by passing the first value.  Here is
a proposal that tries normalizing the special case of passing zero
values as well:

<https://debbugs.gnu.org/cgi/bugreport.cgi?bug=17474>

This "use only the first value in non-multiple-value accepting contexts"
is similar to what Lua uses in the context of "..." argument lists.

-- 
David Kastrup

Re: Multiple values passed as single argument to procedure

[Mark H Weaver]

Chris Marusich <cmmarusich@gmail.com> writes:
> I've noticed that when one passes multiple values as a single argument
> to a procedure, only the first value gets used.  Is this expected?

Yes.  Scheme has no concept of a "multiple values" object that can be
given a single name, or passed as a single argument to a procedure.
Returning multiple values from a procedure is analogous to passing
multiple arguments to a procedure.

Use 'call-with-values', 'let-values', or 'receive' to call a procedure
that returns multiple values (or no values).

If you do not use one of the above forms (or a macro that expands to one
of them) to call a procedure that returns multiple values, then Guile
will discard all but the first result.  Note that this is a
Guile-specific extension.  Other Scheme implementations may behave
differently (e.g. report an error) if multiple values (or no values) are
returned to a procedure call that was not done using one of the forms
listed above.

      Mark


PS: I should mention that Guile does indeed have a "multiple values"
    object at the C level only, to allow C code to return or accept
    multiple values without uglifying the C calling convention for
    Scheme procedures.  This is to work around the fact that C does not
    support returning multiple values from a function.

Re: Multiple values passed as single argument to procedure

[Mark H Weaver]

I wrote:

> Use 'call-with-values', 'let-values', or 'receive' to call a procedure
> that returns multiple values (or no values).
>
> If you do not use one of the above forms (or a macro that expands to one
> of them) to call a procedure that returns multiple values, then Guile
> will discard all but the first result.  Note that this is a
> Guile-specific extension.  Other Scheme implementations may behave
> differently (e.g. report an error) if multiple values (or no values) are
> returned to a procedure call that was not done using one of the forms
> listed above.

I forgot to mention that procedure calls in "tail position" are a
special case known as "tail calls".  In Scheme such a call is
semantically a "GOTO with arguments".  In such cases, the caller
procedure (let's call it FOO) has nothing left to do after its callee
(let's call it BAR) returns, and can simply arrange for BAR to return
its values directly to the caller of FOO.  In such cases, BAR can return
as many values as FOO's caller is prepared to accept.

For example:

  (define (div-and-mod x y)
    (if (negative? y)
        (ceiling/ x y)
        (floor/   x y)))
  
  (define (div x y)
    (let-values (((q r) (div-and-mod x y)))
      q))

Here, both 'floor/' and 'ceiling/' return two values (see the Guile
manual).  Since both of those procedures are called from tail position
in 'div-and-mod', they are tail calls (semantically GOTO with
arguments), and they return their values directly to the caller of
'div-and-mod'.

FYI, the 'div-and-mod' and 'div' defined above are equivalent to the
standard procedures of the same name in R6RS scheme, and to 'euclidean/'
and 'euclidean-quotient' included in core Guile.

Incidentally, the fact that tail calls are semantically GOTOs with
arguments, and that this behavior is guaranteed and not a mere "tail
call optimization", is the reason that we are able to write loops in
Scheme as recursive calls without using unbounded stack space.  In fact,
Scheme has no loop facility in the core language.

      Mark

Re: Multiple values passed as single argument to procedure

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 2796 bytes --]

Hi Mark,

Thank you for the detailed response!  I learn something new every day.

Mark H Weaver <mhw@netris.org> writes:

> Use 'call-with-values', 'let-values', or 'receive' to call a procedure
> that returns multiple values (or no values).
>
> If you do not use one of the above forms (or a macro that expands to one
> of them) to call a procedure that returns multiple values, then Guile
> will discard all but the first result.  Note that this is a
> Guile-specific extension.  Other Scheme implementations may behave
> differently (e.g. report an error) if multiple values (or no values) are
> returned to a procedure call that was not done using one of the forms
> listed above.

I see.  So, this behavior is implementation-specific for Guile scheme.

Is this behavior documented in the Guile reference manual?  I looked,
but I couldn't find information about it.  So, it is not clear to me if
one should rely on this behavior, or if it is likely to change in the
future.  I was hoping to find this behavior documented in either
"(guile) Multiple Values" or somewhere in "(guile) About Procedures".
Perhaps there's a better location.  In any case, I think it would be
helpful if this were documented in the manual.

Here's another question.  I've also noticed that when the 'list'
procedure is composed with a procedure f that returns multiple values,
the list that gets returned when calling the composition differs from
the list that results when "manually" invoking the same composition.  An
example will clarify what I mean:

--8<---------------cut here---------------start------------->8---
$ guile
GNU Guile 2.2.2
Copyright (C) 1995-2017 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (define (f . _) (values 1 2))
scheme@(guile-user)> (f)
$1 = 1
$2 = 2
scheme@(guile-user)> (define g (compose list f))
scheme@(guile-user)> (g)
$3 = (1 2)
scheme@(guile-user)> (list (f))
$4 = (1)
scheme@(guile-user)> 
--8<---------------cut here---------------end--------------->8---

In the above, I was surprised to find that $3 was not the same as $4.
To put this another way, I was surprised to find that the composition
via 'compose' (which returned $3) did not behave the same as the
'manual' composition (which returned $4).  What's going on here?  I
couldn't find the answer by looking at the documentation for the compose
((guile) Higher-Order Functions) or list ((guile) List Constructors)
procedures in the Guile manual.

Thank you for taking the time to help me understand this better.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Multiple values passed as single argument to procedure

[Mark H Weaver]

Hi Chris,

Chris Marusich <cmmarusich@gmail.com> writes:

> Mark H Weaver <mhw@netris.org> writes:
>
>> Use 'call-with-values', 'let-values', or 'receive' to call a procedure
>> that returns multiple values (or no values).
>>
>> If you do not use one of the above forms (or a macro that expands to one
>> of them) to call a procedure that returns multiple values, then Guile
>> will discard all but the first result.  Note that this is a
>> Guile-specific extension.  Other Scheme implementations may behave
>> differently (e.g. report an error) if multiple values (or no values) are
>> returned to a procedure call that was not done using one of the forms
>> listed above.
>
> I see.  So, this behavior is implementation-specific for Guile scheme.
>
> Is this behavior documented in the Guile reference manual?  I looked,
> but I couldn't find information about it.

Indeed, I was not able to find it either.

> So, it is not clear to me if one should rely on this behavior, or if
> it is likely to change in the future.

I would recommend against relying on this behavior, mainly because I
would consider it a bit sloppy.  However, I also think it's very
unlikely that we would ever remove this extension, because I don't
anticipate a compelling reason to remove it, and it would surely break
existing code.

> I was hoping to find this behavior documented in either
> "(guile) Multiple Values" or somewhere in "(guile) About Procedures".
> Perhaps there's a better location.  In any case, I think it would be
> helpful if this were documented in the manual.

Agreed.  "(guile) Multiple Values" is probably the appropriate place.

> Here's another question.  I've also noticed that when the 'list'
> procedure is composed with a procedure f that returns multiple values,
> the list that gets returned when calling the composition differs from
> the list that results when "manually" invoking the same composition.  An
> example will clarify what I mean:
>
> --8<---------------cut here---------------start------------->8---
> $ guile
> GNU Guile 2.2.2
> Copyright (C) 1995-2017 Free Software Foundation, Inc.
>
> Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
> This program is free software, and you are welcome to redistribute it
> under certain conditions; type `,show c' for details.
>
> Enter `,help' for help.
> scheme@(guile-user)> (define (f . _) (values 1 2))
> scheme@(guile-user)> (f)
> $1 = 1
> $2 = 2
> scheme@(guile-user)> (define g (compose list f))
> scheme@(guile-user)> (g)
> $3 = (1 2)
> scheme@(guile-user)> (list (f))
> $4 = (1)
> scheme@(guile-user)> 
> --8<---------------cut here---------------end--------------->8---
>
> In the above, I was surprised to find that $3 was not the same as $4.
> To put this another way, I was surprised to find that the composition
> via 'compose' (which returned $3) did not behave the same as the
> 'manual' composition (which returned $4).  What's going on here?

The problem is that you implemented your 'manual' composition in a way
that allows only one value to pass between the two procedures.  Remember
that when a procedure call is made without 'call-with-values' (or some
macro that uses it), and is not in tail position, then all but the first
return value is discarded.  That's what's happening in your call to 'f'
in (list (f)).  The call (f) is neither in tail position nor called
using 'call-with-values', so only one of its values is kept.

Try this instead:

  (let-values ((vals (f)))
    (apply list vals))

Or, more simply in the case where 'f' takes no arguments:

  (call-with-values f list)

I suppose you are thinking of 'compose' as being implemented like this:

  (define (compose f g)
    (lambda (x)
      (f (g x))))

and that's a fine definition for unary procedures that return a single
argument.

Here's one way to implement 'compose' that supports procedures of
arbitrary arity that return an arbitrary number of values:

  (define (compose f g)
    (lambda args-for-g
      (let-values ((args-for-f (apply g args-for-g)))
        (apply f args-for-f))))

Using 'call-with-values', it looks like this:

  (define (compose f g)
    (lambda args
      (call-with-values (lambda () (apply g args))
        f)))

I should note that the simple unary-only version of 'compose' that I
gave above produces more efficient procedures than the latter two,
because no heap allocation is required during execution of the resulting
procedure.  The more general versions of 'compose' produce procedures
that must allocate lists of arguments 'args', and 'args-for-f' and
'args-for-g' on the GC heap.

The core Guile version of 'compose' is defined in ice-9/boot-9.scm.

      Mark

Re: Multiple values passed as single argument to procedure

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 3222 bytes --]

Hi Mark,

Regarding the Guile-specific behavior where passing multiple values as
an argument to a procedure causes only the first value to be used as the
argument:

> I would recommend against relying on this behavior, mainly because I
> would consider it a bit sloppy.  However, I also think it's very
> unlikely that we would ever remove this extension, because I don't
> anticipate a compelling reason to remove it, and it would surely break
> existing code.
>
>> I was hoping to find this behavior documented in either
>> "(guile) Multiple Values" or somewhere in "(guile) About Procedures".
>> Perhaps there's a better location.  In any case, I think it would be
>> helpful if this were documented in the manual.
>
> Agreed.  "(guile) Multiple Values" is probably the appropriate place.

OK.  I'll submit a patch later this week to update the manual with
information taken from this email thread.  Hopefully that will help
clarify the behavior for others in the future.

Regarding the behavior of composed procedures:

> The problem is that you implemented your 'manual' composition in a way
> that allows only one value to pass between the two procedures.  Remember
> that when a procedure call is made without 'call-with-values' (or some
> macro that uses it), and is not in tail position, then all but the first
> return value is discarded.  That's what's happening in your call to 'f'
> in (list (f)).  The call (f) is neither in tail position nor called
> using 'call-with-values', so only one of its values is kept.

I think I'm missing something here.  In (list (f)), the call to f
certainly looks like it's happening at a position that one might
intuitively call a "tail" position.  So, in this case, what disqualifies
f from being in tail position?  Can you give me an example of a call to
f that would be in tail position, so I can understand the difference?
Sorry if you've already provided such an example; I appreciate your
explanations, and I'm just trying to make sure I fully understand.

> Try this instead:
>
>   (let-values ((vals (f)))
>     (apply list vals))
>
> Or, more simply in the case where 'f' takes no arguments:
>
>   (call-with-values f list)

Am I correct in understanding that in these cases, the two values
returned by the 'f' procedure basically get "converted" into two
separate arguments that get passed to the 'list' procedure, as if I had
invoked (list 1 2)?

> I suppose you are thinking of 'compose' as being implemented like
>this:
>
>   (define (compose f g)
>     (lambda (x)
>       (f (g x))))

Yes, that's what I assumed.  I think I see now why my assumption was
incorrect for the case where g returns multiple values.  I think the
problem here was that I didn't understand how multiple values would be
handled under various situations, even after consulting the Guile
manual.

> The core Guile version of 'compose' is defined in ice-9/boot-9.scm.

Thank you!  It's helpful to see the implementation.  I see that it's
similar to one of the examples you gave for implementing 'compose' in a
way that supports procedures of arbitrary arity that return an arbitrary
number of values.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Multiple values passed as single argument to procedure

[Neil Jerram]

On 12/06/17 09:19, Chris Marusich wrote:
> I think I'm missing something here. In (list (f)), the call to f
> certainly looks like it's happening at a position that one might
> intuitively call a "tail" position.  So, in this case, what disqualifies
> f from being in tail position?  Can you give me an example of a call to
> f that would be in tail position, so I can understand the difference?
> Sorry if you've already provided such an example; I appreciate your
> explanations, and I'm just trying to make sure I fully understand.

Mark will probably have a more precise answer for you, but let me offer 
my understanding too.  In general, in

    ( ... arbitrary code around ...
         (f)
      ... )

the (f) call is in a tail position if _nothing_ else needs to be done, 
to the return value(s) of (f), before returning from that block as a whole.

So, common examples of tail position are

   (begin
      ...
      (f))

and

    (if <condition>
        (f))

The case you mentioned, (list (f)), is probably the simplest example of 
a non-tail position, because something very clearly does need to be done 
to the return value of (f): it needs to be inserted into a newly 
allocated list.

Regards - Neil

Re: Multiple values passed as single argument to procedure

[David Kastrup]

Chris Marusich <cmmarusich@gmail.com> writes:

> I think I'm missing something here.  In (list (f)), the call to f
> certainly looks like it's happening at a position that one might
> intuitively call a "tail" position.

It is, but list does not take multiple values and thus discards
additional values returned by f.  If list were a procedure/continuation
accepting multiple values, this would likely work as you expected.

-- 
David Kastrup

Re: Multiple values passed as single argument to procedure

[Neil Jerram]

On 12/06/17 09:55, Neil Jerram wrote:
> On 12/06/17 09:19, Chris Marusich wrote:
>> I think I'm missing something here. In (list (f)), the call to f
>> certainly looks like it's happening at a position that one might
>> intuitively call a "tail" position.  So, in this case, what disqualifies
>> f from being in tail position?  Can you give me an example of a call to
>> f that would be in tail position, so I can understand the difference?
>> Sorry if you've already provided such an example; I appreciate your
>> explanations, and I'm just trying to make sure I fully understand.
>
> Mark will probably have a more precise answer for you, but let me 
> offer my understanding too.  In general, in
>
>    ( ... arbitrary code around ...
>         (f)
>      ... )
>
> the (f) call is in a tail position if _nothing_ else needs to be done, 
> to the return value(s) of (f), before returning from that block as a 
> whole.
>
> So, common examples of tail position are
>
>   (begin
>      ...
>      (f))
>
> and
>
>    (if <condition>
>        (f))
>
> The case you mentioned, (list (f)), is probably the simplest example 
> of a non-tail position, because something very clearly does need to be 
> done to the return value of (f): it needs to be inserted into a newly 
> allocated list.
>
> Regards - Neil
>
>

Oh, following David's reply, I see that my reply here may have missed 
your point - sorry if so!

(Yes, it is undeniable that '(f)' is in tail position in the expression 
'(f)' :-). )

            Neil

Re: Multiple values passed as single argument to procedure

[Mark H Weaver]

I'm sorry David, but _everything_ that you wrote below is incorrect.

David Kastrup <dak@gnu.org> writes:

> Chris Marusich <cmmarusich@gmail.com> writes:
>
>> I think I'm missing something here.  In (list (f)), the call to f
>> certainly looks like it's happening at a position that one might
>> intuitively call a "tail" position.
>
> It is,

No it isn't.

> but list does not take multiple values

Yes it does.  'list' accepts an arbitrary number of values (arguments).

> and thus discards additional values returned by f.

'list' does not discard anything.  The additional values are discarded
before 'list' is called.

> If list were a procedure/continuation accepting multiple values,

See above.

> would likely work as you expected.

It would not work as Chris expected no matter what procedure was called.

I will try to clarify all of this in another message.

      Mark

Re: Multiple values passed as single argument to procedure

[David Kastrup]

Mark H Weaver <mhw@netris.org> writes:

> I'm sorry David, but _everything_ that you wrote below is incorrect.

Well, let me try again.  It's not all that easy to understand.

> David Kastrup <dak@gnu.org> writes:
>
>> Chris Marusich <cmmarusich@gmail.com> writes:
>>
>>> I think I'm missing something here.  In (list (f)), the call to f
>>> certainly looks like it's happening at a position that one might
>>> intuitively call a "tail" position.
>>
>> It is,
>
> No it isn't.
>
>> but list does not take multiple values
>
> Yes it does.  'list' accepts an arbitrary number of values
> (arguments).

In my book, multiple values/arguments are different things but I admit
that this is a quagmire to me and strictly speaking what I call
"multiple values" (namely the unmitigated return value from calling
values) never occurs in the position of an argument (at least when not
looking at the C API and/or Guile 1.x).

dak@lola:/usr/local/tmp/lilypond$ guile-2.0 
GNU Guile 2.0.13
Copyright (C) 1995-2016 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (list (values 1 2 3))
$1 = (1)
scheme@(guile-user)> 

>> and thus discards additional values returned by f.
>
> 'list' does not discard anything.  The additional values are discarded
> before 'list' is called.

Ok.  Because a function call cannot take multiple values.  It does take
multiple _arguments_.

A function call can appear to return multiple values by using call/cc
(which is essentially what (values ...) does) and the construct
call-with-values can salvage them as long as they are only passed
through tail calls (or rather tail returns):

(call-with-values
  (lambda () (call/cc (lambda (c) (c 1 2 3))))
  list) => (1 2 3)

Basically call-with-values is a mechanism that lets the continuation
delivered by call/cc be a function accepting multiple arguments.  An
ordinary function call doesn't.

> I will try to clarify all of this in another message.

Well, I probably messed up things a whole lot more now.  But at least it
should be an excellent opportunity for correcting the half-truths and
misconceptions that Scheme's procedure/continuation model might inspire
in people (like me) without a firm grasp of its concepts.

-- 
David Kastrup

Re: Multiple values passed as single argument to procedure

[Mark H Weaver]

David Kastrup <dak@gnu.org> writes:

> Mark H Weaver <mhw@netris.org> writes:
>
>> I'm sorry David, but _everything_ that you wrote below is incorrect.
>
> Well, let me try again.  It's not all that easy to understand.

You're obviously quite intelligent and knowledgeable, so it's probably
my failure to explain it well more than anything else.  We can try again
together.

>> David Kastrup <dak@gnu.org> writes:
>>
>>> Chris Marusich <cmmarusich@gmail.com> writes:
>>>
>>>> I think I'm missing something here.  In (list (f)), the call to f
>>>> certainly looks like it's happening at a position that one might
>>>> intuitively call a "tail" position.
>>>
>>> It is,
>>
>> No it isn't.

The reason that (f) is not in tail position with respect to (list (f))
is because there is still something left to do after calling (f) but
before invoking the continuation of (list (f)).  The thing remaining to
do is to apply 'list' to the returned value.

Putting it another way, if (f) evaluates to 5, you cannot proceed by
reducing (list (f)) to 5.  The continuation of (f) cannot be the same as
the continuation of (list (f)), because that would skip the step of
calling 'list'.

* * *

It might be helpful to be more precise about what continuations are.  To
a first approximation, a continuation can be visualized as an expression
with a hole.  For example, in the expression (list (f) (f) (f)), suppose
that the compiler chooses to evaluate the second argument first.  Then
the continuation of the middle (f) is:

  (list (f) <> (f))

where <> is the hole.  When 'f' returns its value (i.e. invokes its
continuation), what remains to be done is this: evaluate the expression
which is formed by putting the return value into the hole.  So, if that
middle (f) returns 5, then we proceed to evaluate:

  (list (f) 5 (f))

If the compiler chooses to evaluate the first argument next, then the
continuation for the first (f) will be:

  (list <> 5 (f))

and so on.  To make this a bit more precise, we could actually use
normal Scheme procedures to model these continuations.  The continuation
above could be written like this:

  (lambda (x)
    (list x 5 (f)))

Supposing that (f) returns 6 this time (maybe it's a counter?), the
continuation of the last (f) will be:

  (lambda (z)
    (list 6 5 z))

In fact, in Scheme we can actually write code in such a way that makes
these continuations explicit.  Here's how we do it: we add an extra
argument to every procedure: its continuation.  Instead of returning
values from procedures, we explicitly call the continuation, passing the
return value as its argument.

If you do this until every procedure call is in tail position, the
resulting program is in "continuation passing style" or CPS.

Here's what (lambda () (list (f 1) (f 2) (f 3))) looks like in CPS,
using the same evaluation order as I chose above:

  (lambda (outer-k)
    (f^ 2 (lambda (y)
            (f^ 1 (lambda (x)
                    (f^ 3 (lambda (z)
                            (list^ x y z outer-k))))))))

Where 'f^' and 'list^' are variants of 'f' and 'list' that accept an
explicit continuation argument.

Continuation passing style makes more things explicit.  It forces us to
explicitly choose an evaluation order, and it entails assigning names to
every intermediate value.  The new variables x, y, and z will hold the
intermediate values returned by (f 1), (f 2), and (f 3), respectively.

So far, I've shown only unary continuations, but when a program is in
continuation passing form, it becomes trivial to "return" an arbitrary
number of values by simply adding more arguments to the continuations.

For example, we could modify the CPS example above to receive two values
from 'f', and to return a list of all six results:

  (lambda (outer-k)
    (f^ 2 (lambda (y1 y2)
            (f^ 1 (lambda (x1 x2)
                    (f^ 3 (lambda (z1 z2)
                            (list^ x1 x2 y1 y2 z1 z2 outer-k))))))))

In CPS, this is a very straightforward, and requires no additional
syntax or primitives.  However, in "direct style" (i.e. the normal way
we write programs) we cannot add multiple return values so easily.  The
difficulty is essentially syntactic in nature.

Here's one way to do the same job as above in direct style, and making
the order of argument evaluation unspecified:

  (lambda ()
    (let-values (((x1 x2) (f 1))
                 ((y1 y2) (f 2))
                 ((z1 z2) (f 3)))
      (list x1 x2 y1 y2 z1 z2)))

compared with the earlier version that keeps only one argument from each
call to 'f'.

  (lambda () (list (f 1) (f 2) (f 3)))

Notice how similar the two CPS examples are, while the corresponding
programs in direct style are strikingly different.

>>> but list does not take multiple values
>>
>> Yes it does.  'list' accepts an arbitrary number of values
>> (arguments).
>
> In my book, multiple values/arguments are different things

As you can see above, in CPS, they are *exactly* the same thing.  And in
fact Guile 2.2 transforms programs into CPS as an integral part of
compilation before code generation.

> but I admit that this is a quagmire to me and strictly speaking what I
> call "multiple values" (namely the unmitigated return value from
> calling values)

Your statement above, where you call "multiple values" "the unmitigated
return value" (singular) reveals your confusion.  I suspect you are
still thinking of multiple value returns as if the values are being
packaged up into a single object which is returned and then taken apart
by the caller.  This is a mistake.  Nothing like this is happening
inside Guile's VM.

I suspect that you do not have any trouble envisioning multiple
arguments being passed *to* a procedure without packaging them into a
single object.  You surely know that in C, the arguments are stored in
separate registers, or placed in separate entries on the stack, before
jumping to a C function.  Something similar can be done to return
multiple values, if the calling convention allows it.  Guile's does.

If you free your mind from the biases of direct style and C calling
conventions, there's a remarkable symmetry between passing arguments
*to* a procedure and returning values *from* a procedure.  The apparent
lack of symmetry is purely a syntactic artifact of the "direct style"
that we generally prefer to work with.

>>> and thus discards additional values returned by f.
>>
>> 'list' does not discard anything.  The additional values are discarded
>> before 'list' is called.
>
> Ok.  Because a function call cannot take multiple values.  It does take
> multiple _arguments_.

Let's take another look at Chris's example:

  (define (f . _) (values 1 2))
  (list (f)) => (1)

If you expected (1 2), then consider this:

  (list (f) (f) (f)) => (1 1 1)

Would you expect (1 2 1 2 1 2)?  If so, I guess you are looking for
splicing behavior in argument lists.  To implement this, we'd need to
convert things to CPS differently.  Instead of:

  (lambda (outer-k)
    (f^ 2 (lambda (y)
            (f^ 1 (lambda (x)
                    (f^ 3 (lambda (z)
                            (list^ x y z outer-k))))))))

You could do something like this:

  (lambda (outer-k)
    (f^ 2 (lambda ys
            (f^ 1 (lambda xs
                    (f^ 3 (lambda zs
                            (append^ xs ys zs
                                     (lambda (args)
                                       (apply^ list^ args outer-k))))))))))

I know that there are some languages that do this kind of splicing, but
in my opinion it's a very bad idea.  In Scheme, when you see the
procedure call:

  (f (x) (y))

You know just from this, without any context, that (x) will be the first
argument and (y) will be the second argument.  This fact is important
for both humans and compilers.  For humans it is important because we
can more easily and reliably reason about the code we are looking at.

For compilers it is important for the same reason, actually.  For
example, if 'f' is known at compile time, and something useful is known
about (y), we can inline the call to 'f', substituting (y) for its
second argument, and use our knowledge of (y) to optimize the result.

However, if we had splicing semantics for procedure calls, and if 'x' is
not known at compile time (it usually isn't), then we would have _no_
idea which argument of 'f' (y) will be bound to, and we can't do much of
anything to optimize this.

So, I suppose you could consider it design choice to insist that in an
arbitrary procedure call

  (f (x) (y))

that we know statically (i.e. at compile time) that no matter how (x)
and (y) behave, two arguments will always be passed in this call, and
the first argument will be (x) and the second argument will be (y).

I should mention that if you consider examples where the procedure being
called is 'list' or '+' or something else which accepts an arbitrary
number of arguments and treats them all interchangably, then the
splicing behavior seems more sensible.  However, such procedures are
quite rare in practice.

The overwhelmingly common case is that a procedure's arguments have very
different roles, where splicing is not at all useful and merely makes it
difficult for humans (and usually impossible for the compiler) to know
which argument a given operand will be assigned to.

Does that make sense?

    Regards,
      Mark

Re: Multiple values passed as single argument to procedure

[Mark H Weaver]

In the discussion about continuation passing style, I forgot to explain
the semantics of when and how Guile discards extra return values.  It's
very simple:

I wrote:
> Here's what (lambda () (list (f 1) (f 2) (f 3))) looks like in CPS,
> using the same evaluation order as I chose above:
>
>   (lambda (outer-k)
>     (f^ 2 (lambda (y)
>             (f^ 1 (lambda (x)
>                     (f^ 3 (lambda (z)
>                             (list^ x y z outer-k))))))))

In the CPS examples, I modelled these normal "unary" continuations as
unary procedures of the form:

  (lambda (x) ...)

In Guile, these "unary" continuations are not truly unary.  Instead,
they can be modeled roughly as procedures of this form:

  (lambda (x . _) ...)

Where '_' does not occur free in the body.  So, to model Guile's
behavior in CPS, the example above becomes:

   (lambda (outer-k)
     (f^ 2 (lambda (y . _)
             (f^ 1 (lambda (x . _)
                     (f^ 3 (lambda (z . _)
                             (list^ x y z outer-k))))))))

       Mark

Re: Multiple values passed as single argument to procedure

[Mark H Weaver]

Mark H Weaver <mhw@netris.org> writes:

> The reason that (f) is not in tail position with respect to (list (f))
> is because there is still something left to do after calling (f) but
> before invoking the continuation of (list (f)).  The thing remaining to
> do is to apply 'list' to the returned value.
>
> Putting it another way, if (f) evaluates to 5, you cannot proceed by
> reducing (list (f)) to 5.  The continuation of (f) cannot be the same as
> the continuation of (list (f)), because that would skip the step of
> calling 'list'.

FYI, the precise rules for what it means to be in "tail position" are
given in R6RS section 11.20 and R7RS section 3.5.  In those documents,
they call the same concept "tail context".  See:

  http://www.r6rs.org/final/html/r6rs/r6rs-Z-H-14.html#node_sec_11.20

       Mark

Re: Multiple values passed as single argument to procedure

[dsmich]

---- Mark H Weaver <mhw@netris.org> wrote: 
> David Kastrup <dak@gnu.org> writes:
> 
> > Mark H Weaver <mhw@netris.org> writes:
> >ument
> >> I'm sorry David, but _everything_ that you wrote below is incorrect.
> >
> > Well, let me try again.  It's not all that easy to understand.
> 
> You're obviously quite intelligent and knowledgeable, so it's probably
> my failure to explain it well more than anything else.  We can try again
> together.

Thanks Mark for that most excellently lucid explanation! I had vague notions before
but I have a much deeper appreciation now.

Back in the day, I used Forth a bit.  Multiple arguments and multiple return values
were pretty much the same thing. (the stack) And it was easy, because of the syntax
of the language.  Yeah, I agree, syntax is the main blocker here.

Thanks again!
  -Dale

Re: Multiple values passed as single argument to procedure

[Alex Vong]

[-- Attachment #1: Type: text/plain, Size: 1409 bytes --]

Here is an explanation of multiple values by Oleg[0] and an
implementation from the Scheme book[1]. These are more advanced
stuff. Honestly, I usually don't understand what Oleg said. But it is
good to know these references. It is like math, one day when you have
enough background and the right perspective, you will understand what's
going on.

[0]: http://okmij.org/ftp/Scheme/misc.html#multiple-value-effect
[1]: http://www.scheme.com/tspl4/control.html#./control:h8

<dsmich@roadrunner.com> writes:

> ---- Mark H Weaver <mhw@netris.org> wrote: 
>> David Kastrup <dak@gnu.org> writes:
>> 
>> > Mark H Weaver <mhw@netris.org> writes:
>> >ument
>> >> I'm sorry David, but _everything_ that you wrote below is incorrect.
>> >
>> > Well, let me try again.  It's not all that easy to understand.
>> 
>> You're obviously quite intelligent and knowledgeable, so it's probably
>> my failure to explain it well more than anything else.  We can try again
>> together.
>
> Thanks Mark for that most excellently lucid explanation! I had vague
> notions before
> but I have a much deeper appreciation now.
>
> Back in the day, I used Forth a bit.  Multiple arguments and multiple
> return values
> were pretty much the same thing. (the stack) And it was easy, because
> of the syntax
> of the language.  Yeah, I agree, syntax is the main blocker here.
>
> Thanks again!
>   -Dale

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Multiple values passed as single argument to procedure

[Chris Marusich]

[-- Attachment #1.1: Type: text/plain, Size: 821 bytes --]


Earlier, I wrote:

> OK.  I'll submit a patch later this week to update the manual with
> information taken from this email thread.  Hopefully that will help
> clarify the behavior for others in the future.

Please find the promised patch attached.  It's still this week, isn't
it? :-)

As a Guile newbie, I found the behavior related to multiple values to be
very confusing at first.  I've tried to explain things in a way that is
accurate, easy for a newbie to understand, and points the curious reader
to appropriate resources for further reading.  Hopefully, this small
contribution will help to clarify the behavior for future Guile newbies.

For reference, this patch is follow-up to the following email thread:

https://lists.gnu.org/archive/html/guile-user/2017-06/msg00043.html

-- 
Chris

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: 0001-doc-Discuss-Guile-specific-behavior-related-to-multi.patch --]
[-- Type: text/x-patch, Size: 5698 bytes --]

From c14d283b5e830eb0f9054e51986545b9ba15f247 Mon Sep 17 00:00:00 2001
From: Chris Marusich <cmmarusich@gmail.com>
Date: Thu, 31 Aug 2017 23:46:25 -0700
Subject: [PATCH] doc: Discuss Guile-specific behavior related to multiple
 values.

* doc/ref/api-control.texi (Multiple Values): Explain (and caution
  against relying upon) Guile-specific behavior when passing multiple
  values as a single argument to a procedure.

* doc/ref/api-procedures.texi (Higher-Order Functions): Explain (and
  caution against relying upon) potentially counter-intuitive behavior
  that can arise when attempting to mimic the behavior of the 'compose'
  procedure by naively chaining together multi-valued procedure calls.
---
 doc/ref/api-control.texi    | 30 +++++++++++++++++++++++++++
 doc/ref/api-procedures.texi | 50 ++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 79 insertions(+), 1 deletion(-)

diff --git a/doc/ref/api-control.texi b/doc/ref/api-control.texi
index 2d696ea89..46da2f17f 100644
--- a/doc/ref/api-control.texi
+++ b/doc/ref/api-control.texi
@@ -902,6 +902,36 @@ procedure body.  @code{call-with-values} combines a procedure returning
 multiple values with a procedure which accepts these values as
 parameters.
 
+You might sometimes be tempted to try to pass multiple values as a
+single argument to a procedure (e.g., when directly passing the output
+of one procedure call into another as input).  This works fine for
+single values, but caution is required when multiple values are
+involved.  The correct way to pass multiple values as arguments to a
+procedure is to use @code{call-with-values} or something else that uses
+it, such as @code{let-values} (@pxref{SRFI-11}) or @code{compose}
+(@pxref{Higher-Order Functions}).  If you attempt to pass multiple
+values as a single argument to a procedure, Guile will only use the
+first value as the argument.  For example, consider the following:
+
+@example
+scheme@@(guile-user)> (values 1 2)
+$1 = 1
+$2 = 2
+scheme@@(guile-user)> (list (values 1 2))
+$3 = (1)
+@end example
+
+Here, the @code{list} procedure is being called with a single argument.
+Therefore, even though the expression @code{(values 1 2)} evaluates to
+multiple values, Guile will only use the first value as the argument, so
+the result is the same as if we had written @code{(list 1)}.
+
+This behavior is specific to the Guile implementation of Scheme.  In
+R6RS, the behavior when attempting to pass multiple values as a single
+argument to a procedure (and in some other situations, too) is
+explicitly left unspecified.  Other implementations might behave
+differently, so you should think twice before relying upon it.
+
 @rnindex values
 @deffn {Scheme Procedure} values arg @dots{}
 @deffnx {C Function} scm_values (args)
diff --git a/doc/ref/api-procedures.texi b/doc/ref/api-procedures.texi
index df24178f9..bf97f7038 100644
--- a/doc/ref/api-procedures.texi
+++ b/doc/ref/api-procedures.texi
@@ -684,7 +684,10 @@ Return a procedure with the same arity as @var{proc} that returns the
 Compose @var{proc1} with the procedures @var{proc2} @dots{}  such that
 the last @var{proc} argument is applied first and @var{proc1} last, and
 return the resulting procedure.  The given procedures must have
-compatible arity.
+compatible arity; the first value returned by procedure @code{N} will be
+passed into procedure @code{N-1} as its first argument, the second value
+returned by procedure @code{N} will be passed into procedure @code{N-1}
+as its second argument, etc.
 
 @lisp
 (procedure? (compose 1+ 1-)) @result{} #t
@@ -695,6 +698,51 @@ compatible arity.
 ((compose zip unzip2) '((1 2) (a b)))
                              @result{} ((1 2) (a b))
 @end lisp
+
+When a procedure other than @var{proc1} returns multiple values, the
+behavior observed when calling the procedures one after another might
+differ from the behavior observed when calling their composition,
+depending on how the procedures are called.  This can lead to
+potentially counter-intuitive results like the following:
+
+@example
+scheme@@(guile-user)> (define (g) (values 1 2))
+scheme@@(guile-user)> (g)
+$1 = 1
+$2 = 2
+scheme@@(guile-user)> (define composition (compose list g))
+scheme@@(guile-user)> (composition)
+$3 = (1 2)
+scheme@@(guile-user)> (list (g))
+$4 = (1)
+scheme@@(guile-user)>
+@end example
+
+At first blush, it might seem intuitive that the expressions
+@code{(composition)} and @code{(list (g))} ought to evaluate to the same
+result.  This intuition would be correct if the procedure @code{g}
+returned only a single value.  However, it returns multiple values, and
+when you try to pass multiple values as a single argument to a
+procedure, Guile will only use the first value as the argument
+(@pxref{Multiple Values}).  Therefore, the result of evaluating
+@code{(list (g))} is the same as if we had written @code{(list 1)}.  It
+is not a correct composition, and you should think twice before relying
+on this Guile-specific behavior.
+
+It is possible to correctly compose procedures like these without using
+@code{compose}, but it requires the use of @code{call-with-values} or
+something else that uses it.  For example, the following correctly
+composes @code{list} with @code{g}:
+
+@example
+scheme@@(guile-user)> (call-with-values g list)
+$5 = (1 2)
+@end example
+
+In fact, this is essentially what the implementation of @code{compose}
+does: it recursively uses @code{call-with-values} to correctly compose
+all the functions.
+
 @end deffn
 
 @deffn {Scheme Procedure} identity x
-- 
2.14.1


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]