Re: Comments on ‘wip-nio’

[Andy Wingo <wingo@pobox.com>]

Heya :)

On Wed 21 Mar 2012 23:39, ludo@gnu.org (Ludovic Courtès) writes:

> I had a quick look at ‘wip-nio’, and here are initial comments.

Thanks for the feedback!

>   • epoll is Linux-specific.  Any idea how a more multi-platform API
>     could be provided?  I guess libevent could be used, but we don’t
>     want to add one more dependency in 2.0.  How much is (ice-9 nio)
>     dependent on epoll vs. poll, for instance?

It's possible to write a compatibility wrapper.  Note that not all
platforms have poll either; mingw being a notable example.  The reason I
haven't done so yet is that epoll offers a strange interface, I don't
know anything about kqueue yet, and abstraction layers have the
potential to introduce inefficiency.  I'd like to give edge-triggered
epoll a try at some point, for example.  Also the fact that an epoll set
is an fd which itself may be polled allows for nesting schedulers.

In summary, it's not done yet because it's early.  But I have some hopes
that we can produce a portable system, in time.

>   • ‘nio’ doesn’t seem very descriptive to me; ‘nbio’ maybe?

I chose it because of Java's NIO, which introduced non-blocking I/O for
Java.  I'm fine with changing names.

Note that since you reviewed this patch, I split things up a bit more.
(ice-9 nio) holds the nonblocking primitives -- currently read, write,
and accept.  (ice-9 ethreads) builds coroutines on top of that.  I think
I'm going to try to split an (ice-9 eports) out of (ice-9 ethreads)
today.

>   • I agree that it’s an important problem to address, and I like the
>     use of coroutines, and I like that it actually solves the problem
>     for some applications like the web server.
>
>     However, I think the approach has shortcomings: (1) it doesn’t
>     accept other sources of events (timers, mouse clicks, etc.), and
>     (2) it doesn’t allow you to do something while waiting (of course,
>     other threads/waiters get scheduled, but the caller is just
>     waiting, so you can’t for instance update your progress bar while
>     waiting for your bytevector to arrive.)
>
>     Functional reactive programming à la FrTime [0], and synchronous
>     reactive programming à la HipHop [1] seem to be much more generic
>     and expressive.  It would be great if Guile would come with such a
>     framework eventually.  It’s probably more work, though, so it
>     probably makes sense to have an alternative mechanism in the
>     meantime.

I'll look into this.  I've always been meaning to understand FRP, and
the HipHop link looks interesting too.  I'm sure you're right here.

One correction though: you are able to wait for other sources of input.
Consider this definition of `sleep':

    (define (sleep seconds)
      (suspend
       (lambda (ctx thread)
         (add-sleeper! ctx thread seconds))))

For completeness, here's suspend:

    ;; The AFTER-SUSPEND thunk allows the user to suspend the current
    ;; thread, saving its state, and then perform some other nonlocal
    ;; control flow.
    ;;
    (define* (suspend #:optional (after-suspend (lambda (ctx thread) #f)))
      ((abort-to-prompt (econtext-prompt-tag (current-econtext))
                        after-suspend)))

So you see that the procedure passed to suspend gets called from the
dynamic context of the scheduler -- the one that installed the prompt.
Sleeps happen to be handled by the scheduler.  To resume a thread,
because you just got some input, call resume:

    (define* (resume thread thunk #:optional (ctx (ensure-current-econtext)))
      (let* ((cont (ethread-data thread))
             (thunk (lambda () (cont thunk))))
        (schedule-thread! ctx thread thunk)))

This marks the thread as runnable, if it wasn't already.  The thunk that
you pass ends up being invoked in the continuation of the suspend --
check the double-parens above.  In this way you can pass arbitrary
numbers of values, cause exceptions, etc.

The only caveat here is that it doesn't work to resume a thread for a
context that's in an epoll(), because the context won't wake up.  We
need to add a wakeup fd to the epoll, as Haskell's event manager does.

OK, off to hack some more!

Andy
-- 
http://wingolog.org/