Re: Asynchronous event loop brainstorm at FSF 30

[Nala Ginrut <nalaginrut@gmail.com>]

Hi Christopher!
I'm one of guys who's interested in this topic, say, async IO in Guile.

On Sat, 2015-10-03 at 17:29 -0500, Christopher Allan Webber wrote:
>  - This would be like asyncio or node.js, asynchronous but *not* OS
>    thread based (it's too much work to make much of Guile fit around
>    that for now)

Personally, I would like to avoid callback way for asyncio which is the
way in node.js. And I believe we have chances to avoid it in Guile.

>  - If you really need to maximize your multiple cores, you can do
>    multiple processes with message passing anyway

I've learned roughly about how Erlang deals with multicore problem (I
don't know if Erlang has better solution since 2008). I'm not going to
discuss about the design here, but I have to mention that a proper
general coroutine interface in Guile core should provide a flexible way
to let users do there scheduling policy with multicores. Maybe it's too
idealism, but it's no harm to list it before the coding work.

>  - Initially, this would probably providing a general API for
>    coroutines.  Mark thinks delimited continuations would not be as
>    efficient as he'd like, but we think it's okay because we could
>    provide a nice abstraction where maybe something nicer could be
>    swapped out later, so delimited continuations could at least be a
>    starting point.

Yes the current delimited-continuations is not so efficient. I think it
has to be optimized in the future. But I'm still standing on optimistic
side of delimited-continuation based asyncio design. Yes, I know it's
faster (in single core) if we use libev/libuv, which keeps fewer
information/states for each request compared to continuations. But I
don't see it's the best solution for scalability taking advantage of
distributed system.

Maybe I'm a little out of topic, a general async event loop is
unnecessary to be salable.

>  - So what we really need is a nice API for how to do coroutines, write
>    asynchronous code, and work with some event loop with a scheduler
> 
>  - On top of this, "fancier" high level systems like an actor model or
>    something like Sly's functional reactive programming system could be
>    done.

Yes, it's better than playing raw continuations, which is hard to handle
and debug. ;-)

>  - Probably a good way to start on this would be to use libuv (or is it
>    libev?) and prototype this.  It's not clear if that's a good long
>    term approach (eg, I think it doesn't work on the HURD for those who
>    care about that, and Guix certainly does)

Anyway, it's no harm to add an implementation based on libuv/libev, but
I think it's improper to be put in Guile core. Or we'll introduce third
party lib as the prerequisites.
Unfortunately, there's no epoll in GNU/Hurd although it is believed not
so hard to be added based on current `select' in Glibc (different from
Linux, these stuffs are in userland on Hurd). I have discussed it with
Hurd folks, but we still have no coding work for it yet.

Again, it's no harm to add it as a lib, folks need efficient asyncio,
always.

>  - Binary/custom ports could be a nice means of abstraction for this

Years ago, Mark Weave has told me (actually it's a kindly warning) that
we have to handle the ports properly for nonblock asyncio. It could be
more complex if one uses DB in her/his program. It's better if we could
have better abstraction for these ports. But I still have no idea about
it.


Any more comments? I think it's better to discuss it more, and list
more potential issues to be noticed。There're always big problems when you
want to design general interface.