Re: continuation passing in Emacs vs. JUST-THIS-ONE

[Stefan Monnier <monnier@iro.umontreal.ca>]

> I wrote that incorrectly - I meant that primitive operations would add
> a continuation to the future and return a future for their result.
> Basically, a computation would never block, it would just build
> continuation trees (in the form of futures) and return to the
> top-level.  Although that assumes the system would be able to allocate
> those futures without blocking for GC work.

I think this would end up being extremely inefficient, since for every
tiny operation you'd end up creating a future, linked to the
previous computation (basically a sort of fine-grained dataflow graph).
I'm sure in theory it can be made tolerably efficient, but it'd need
a very different implementation strategy than what we have now.
Furthermore I expect it would lead to surprising semantics when used
with side-effecting operations.

IOW you can probably create a usable system that uses this approach, but
with a different language and a different implementation :-)

`futur.el` instead forces the programmer to be somewhat explicit about
the concurrency points, so they have *some* control over efficiency and
interaction with side-effects.

> At some point in this thread you stated you weren't sure what the
> right semantics are in terms of the information to save, etc.  I posed
> this implicit semantics as a way to think about what "the right thing"
> would be.  Would all operations preserve the same (lisp) machine
> state, or would it differ depending on the nature of the operator?  [
> is the kind of question it might be useful to work out in this thought
> experiment ]

I can't imagine it working sanely if the kind of state that's saved
depends on the operation: the saved-state is basically private to the
continuation, so it might make sense to do it differently depending on
the continuation (tho even that would introduce a lot of complexity),
but not depending on the operation.

The coders will need to know what is saved and what isn't, so the
more complex this rule is, the harder it is to learn to use this
tool correctly.

> The way you've defined future-let, the variable being bound is a
> future because you are constructing it as one, but it is still a
> normal variable.
>
> What if, instead, we define a "futur-abstraction" (lambda/futur (v)
> body ...) in which v is treated as a future by default, and a
> future-conditional form (if-available v ready-expr not-ready-expr)
> with the obvious meaning.  If v appears as the argument to a
> lambda/future function object it will be passed as is.  Otherwise, the
> reference to v would be rewritten as (futur-wait v).  Some syntactic
> sugar (futur-escape v) => (if-available v v) could be used to pass the
> future to arbitrary functions.

Seems complex, and I'm not sure it would buy you anything in practice.

> Otherwise, I'm not sure what the syntax really buys you.

Not very much.
To some extent it simply helps reduce the indentation due to nesting.

> It would be easier if elisp threads were orthogonal to system threads,
> so that any elisp thread could be run on any available system thread.

Currently, only one thread can run ELisp at a time.  Whether that's
implemented using several system threads or not is largely an
internal detail.

> Multiprocessing could be done by creating multiple lisp VMs in a
> process (i.e. lisp VM orthogonal to a physical core),

Yes, "could".
Other than approaches like `async.el` we don't really know how to
implement that, sadly.
[ Many years ago I proposed to rely on `fork`, but it so happens that
  it's not really an option for the w32 builds :-(  ]

> each with their own heap and globals in addition to some shared heap
> with well-defined synchronization.  The "global interpreter lock"
> would become a "lisp machine lock", with (non-preemptive, one-shot
> continuation type) elisp threads being local to the machine.
> That seems to me the simplest way to coherently extend the lisp
> semantics to multi-processing.  The display would presumably have to
> exist in the shared space for anything interesting to happen in terms
> of editing, but buffers could be local to a particular lisp machine.

It's not too hard to come up with a design that makes sense, indeed, the
problem is to actually do the work of bringing the current code to
that design.


        Stefan