Re: anyone define port types?

[Chris Vine <chris@cvine.freeserve.co.uk>]

On Sun, 19 Jun 2016 19:48:03 +0200
Andy Wingo <wingo@pobox.com> wrote:
> On Sun 19 Jun 2016 17:33, Chris Vine <chris@cvine.freeserve.co.uk>
> writes:
> 
> > The answer I have adopted when reading from TCP sockets is to
> > extract individual bytes only from the port into a bytevector using
> > R6RS's get-u8 procedure and (if the port is textual rather than
> > binary) to reconstruct characters from that using
> > bytevector->string at, say, a line end[1].  An EAGAIN/EWOULDBLOCK
> > exception is then just treated as an invitation to return to the
> > prompt, and read state is retained in the bytevector.  
> 
> Yep, makes sense, though it will be really slow of course.  And, you'd
> have to make people rewrite their code to use your I/O primitives
> instead of using read-line from (ice-9 rdelim); a bit of a drag, but
> OK.

It's not excessively slow provided that the port itself (as opposed
to the conversion to unicode characters, which you have to do yourself)
can be buffered. You can loop calls to get-u8 on char-ready? to vacate
the buffers so far as wanted, without returning to the prompt for every
byte.  From that point of view it shouldn't be that much slower than
repeatedly calling read-char on a port with latin-1 encoding on a
normal blocking port.
 
[snip] 
> > I don't think I have got to grips with how to do that with
> > read-waiter, because the read-waiter comprises in effect another
> > loop (in which the main event loop with its own prompts would have
> > to run) until the read request has been satisfied.  I would need to
> > think about it.  Since ethreads use a poll()/epoll() loop,
> > presumably you think it is straightforward enough to integrate the
> > two, even if at present I don't.  
> 
> Here is the code.  First, a helper:
> 
>   ;; The AFTER-SUSPEND thunk allows the user to suspend the current
>   ;; thread, saving its state, and then perform some other nonlocal
>   ;; control flow.
>   ;;
>   (define* (suspend #:optional (after-suspend (lambda (ctx thread)
> #f))) ((abort-to-prompt (econtext-prompt-tag (current-econtext))
>                       after-suspend)))
> 
> Assume there is some current-econtext parameter with a "context" which
> holds the prompt tag associated with the scheduler.  As you see when
> the continuation resumes, it resumes by calling a thunk, allowing
> exceptions to be thrown from the context of the suspend.
> 
> Here's the main loop function, which you could replace with the GLib
> main loop or whatever:
> 
>   (define* (run #:optional (ctx (ensure-current-econtext))
>                 #:key (install-suspendable-ports? #t))
>     (when install-suspendable-ports? (install-suspendable-ports!))
>     (parameterize ((current-econtext ctx)
>                    (current-read-waiter wait-for-readable)
>                    (current-write-waiter wait-for-writable))
>       (let lp ()
>         (run-ethread ctx (next-thread ctx))
>         (lp))))
> 
> Cool.  Now the wait functions:
> 
>   (define (wait-for-readable port)
>     (wait-for-events port (port-read-wait-fd port) (logior EPOLLIN
> EPOLLRDHUP)))
> 
>   (define (wait-for-writable port)
>     (wait-for-events port (port-write-wait-fd port) EPOLLOUT))
> 
> Now the wait-for-events function:
> 
>   (define (wait-for-events port fd events)
>     (handle-events
>      port
>      events
>      (suspend
>       (lambda (ctx thread)
>         ...))))
> 
> Well that's a mess, but the thing to know is that the `suspend' will
> abort to the relevant prompt, and then invoke the thunk that's its
> argument.  Here's `handle-events' and we are done:
> 
>   (define (handle-events port events revents)
>     (unless (zero? (logand revents EPOLLERR))
>       (error "error reading from port" port)))
> 
> But I guess that error could be "reading or writing"; oh well.
> 
> See
> http://git.savannah.gnu.org/cgit/guile.git/tree/module/ice-9/ethreads.scm?h=wip-ethreads&id=253dc1a7114b89351a3aa330caf173b98c5a65dd,
> it's not long but it takes some time to read.  I think I can fairly
> ask that of you though, given your interest in this area :)

OK I am grateful for your patience in explaining this.  I need to think
about it, but while this works where all events come from user-derived
events, I doubt that this would work with guile-gnome and the glib main
loop in the round, because for gtk+ to function the glib main loop must
block within glib's own internal call to the poll() and not within the
guile prompt, or gdk and other events will jam up.  You would probably
need to run the glib main loop in its own (pthread) thread, about which
you have previously given dire warnings so far as guile as it currently
stands is concerned.

As I say, I really need to think more about this and look further at
the code.  I may well be missing something.

> > On a side issue, I am still trying to understand the point of
> > causing guile-2.2's read of a non-blocking C port to block.  The
> > whole point of making a descriptor non-blocking is that that
> > shouldn't happen, and there are circumstances where pealing
> > individual bytes off a non-blocking port as they become available
> > is what you want to do.  It makes guile's select wrapper unusable
> > with TCP sockets on linux.  I understand that suspendable-ports
> > work differently, but that is another matter.  
> 
> I would start by saying that many things are possible including adding
> the ability to enable the error-throwing behavior.  I think your use
> case is a great one and we should support it well.  I think though
> that perhaps you didn't see how the waiter parameters could apply to
> your use case; are things clearer now?  I think that once you allow
> yourself to use port buffers, things will be much easier and much
> more efficient as well.  But, I could be misunderstanding too.
> 
> Guile's I/O in the past was very oriented towards ASCII.  2.0 went a
> long way towards good textual I/O with other encodings, and with 2.2
> we will have good textual non-blocking I/O.  It could be that by
> focusing on this, I have neglected some binary non-blocking I/O use
> cases, though I think that get-bytevector-some, having enabled
> suspendable ports, serves many of these purposes very well.  But if
> there is a need for an additional binary I/O primitive, let's figure
> out what it is and implement it.

This is an issue with R6RS.  In my view it ought to have a
get-bytevector-some! procedure which emulates unix read(), but once you
decide that C ports should not have non-blocking reads, that is an end
to it really.

In my view though, a non-blocking get-bytevector-some! would be really
nice.  It could return the number of bytes extracted, with a return
value of 0 for EAGAIN.

Chris