Re: continuation passing in Emacs vs. JUST-THIS-ONE

[Tomas Hlavaty <tom@logand.com>]

On Tue 11 Apr 2023 at 16:22, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> It is useful to acknowledge, that there are 3 different use-cases:
>> a) asynchronous processes
>> b) threads
>> c) iter
>
> I can't see how `iter` would be a use case for futures.

It makes it easy to implement async/await without the need for
asynchronous processes or threads.

>> My impression was that futur.el was trying to address a) and b) but
>> now you say it does address a) only.  That is rather limited.
>
> Looking at existing code, iter and threads are virtually never used,
> so from where I stand it seems to cover the 99% cases.

Strange that futur.el is "primarily concerned with making it easier to
write asynchronous code" but limits itself to asynchronous processes
only.

I am also "concerned with making it easier to write asynchronous code"
but I explore various options available in Emacs so that the result
would not end up with some patological flaw due to specifics of
asynchronous processes.

I do not know how useable threads in Emacs are at the moment,
but they are already there and the examples I tried worked well.
If they are not useable now, I hope they will be useable in the future.
(Pun intended:-)

And I do not think I am the only one:

   From: Po Lu <luangruo@yahoo.com>
   Date: Mon, 03 Apr 2023 12:03:17 +0800
   id:87mt3pr4sa.fsf@yahoo.com

   I have not looked carefully at this thread, but I would hope that if
   people are discussing a way to add multiprocessing to Emacs, we
   settle on separate threads of execution executing in parallel, with
   all the interlocking necessary to make that happen, like in most Unix
   thread implementations.

It would be a shame not to consider them.
Especially when that use-case is easy to implement and
works as demonstrated with the promise-pipelining-server3 example.

>>>> No, the iter case does map directly to futures:
>>>>
>>>> (await
>>>>  (async-iter
>>>>    (let ((a (async-iter
>>>>               (message "a1")
>>>>               (await-iter (sleep-iter3 3))
>>>>               (message "a2")
>>>>               1))
>>>>          (b (async-iter
>>>>               (message "b1")
>>>>               (let ((c (async-iter
>>>>                          (message "c1")
>>>>                          (await-iter (sleep-iter3 3))
>>>>                          (message "c2")
>>>>                          2)))
>>>>                 (message "b2")
>>>>                 (+ 3 (await-iter c))))))
>>>>      (+ (await-iter a) (await-iter b)))))
>>>
>>> I must say I don't understand this example: in which sense is it using
>>> "iter"?  I don't see any `iter-yield`.
>>
>> await-iter and async-iter macros are using iter under the hood.
>
> The point of `iter` is to provide something that will iterate through
> a sequence of things.  Here I don't see any form of iteration.

I sent the whole self-contained example.  The iteration happens in the
top-level loop (see await-in-background).

> You seem to use your `iter`s just as (expensive) thunks (futures).

What do you mean?
Are thunks expensive?
More expensive than cl-struct and CLOS in futur.el?
Surely it's the other way round.

In the use-case of iter (no asynchronous processes or threads), iter is
used to do cps rewriting needed by async-iter.

> Maybe what you mean by "iter" is the use of CPS-translation
> (implemented by `generator.el`)?

Yes, iter provides the sequence of steps needed to compute the whole
async-iter expression.  See

   (iter-make (iter-yield (progn ,@body)))

in async-iter macro.

>>> Of course.  You could do something like
>>>
>>>       (futur-let*
>>>           ((a (futur-let* ((_ <- (futur-process-make
>>>                                  :command '("sleep" "9"))))
>>>                  9))
>>>            (b (futur-let* ((_ <- (futur-process-make
>>>                                  :command '("sleep" "8"))))
>>>                  8))
>>>            (a-val <- a)
>>>            (b-val <- b))
>>>         (message "Result = %s" (+ a-val b-val))))
>>
>> So will futur.el take 9sec or 17sec?
>
> 9 secs, of course: the above creates 2 futures and emits the message
> when they're both done.  Since those futures are executed in
> subprocesses, they execute concurrently.

Good.  So I don't understand your remark about parallelism.  Maybe I
should believe you that it would take 9sec, but I would rather verify
that by myself because being "executed in subprocesses" does not
necessarily mean "they execute concurrently".

>>> Similarly the "intended return value" of a process will depend on what
>>> the process does.  In some cases it will be the stdout, but I see no
>>> reason to restrict my fundamental function to such a choice.
>> This overgeneralized thinking is beyond usefulness and harmfully leads
>> to the problem of how to maintain state.
>
> While I do like to over-generalize, in this case, there is no
> generalization involved.  The code is the simple result of a thin
> wrapper around the existing `make-process` to make it obey the
> `futur.el` API.  So if it's overgeneralized, it's not my fault, it's
> `make-process`s :-)

Not really, make-process is general because it covers many use-cases.
Futures are quite specific.  Anything beyond the specifics cause issues.