From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Chris Vine Newsgroups: gmane.lisp.guile.devel,gmane.lisp.guile.user Subject: Re: anyone define port types? Date: Sun, 19 Jun 2016 21:09:12 +0100 Message-ID: <20160619210912.48528ffa@dell.homenet> References: <87y492mnjp.fsf@pobox.com> <87pots9tag.fsf@gnu.org> <87bn37wtf2.fsf@pobox.com> <20160612092513.3eb1c8a3@laptop.homenet> <8760t5mthu.fsf@pobox.com> <20160619163327.36246706@dell.homenet> <8737o9kr3g.fsf@pobox.com> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Trace: ger.gmane.org 1466366988 23575 80.91.229.3 (19 Jun 2016 20:09:48 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 19 Jun 2016 20:09:48 +0000 (UTC) To: guile-user , guile-devel Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Sun Jun 19 22:09:39 2016 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bEj2H-0001Hu-KF for guile-devel@m.gmane.org; Sun, 19 Jun 2016 22:09:37 +0200 Original-Received: from localhost ([::1]:39906 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bEj2D-000817-Al for guile-devel@m.gmane.org; Sun, 19 Jun 2016 16:09:33 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:58213) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bEj25-0007xU-C0 for guile-devel@gnu.org; Sun, 19 Jun 2016 16:09:27 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bEj1z-00061P-He for guile-devel@gnu.org; Sun, 19 Jun 2016 16:09:24 -0400 Original-Received: from avasout07.plus.net ([84.93.230.235]:43800) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bEj1z-00060Q-8Y for guile-devel@gnu.org; Sun, 19 Jun 2016 16:09:19 -0400 Original-Received: from dell.homenet ([87.114.218.169]) by avasout07 with smtp id 8k9C1t0063fstn301k9Ena; Sun, 19 Jun 2016 21:09:15 +0100 X-CM-Score: 0.00 X-CNFS-Analysis: v=2.1 cv=QqujpgGd c=1 sm=1 tr=0 a=4ynVEcF4YjHkqINdvT3UIQ==:117 a=4ynVEcF4YjHkqINdvT3UIQ==:17 a=L9H7d07YOLsA:10 a=9cW_t1CCXrUA:10 a=s5jvgZ67dGcA:10 a=kj9zAlcOel0A:10 a=pD_ry4oyNxEA:10 a=ybZZDoGAAAAA:8 a=Fwo3Pw8wAAAA:8 a=mDV3o1hIAAAA:8 a=hQfocF3yJo61jf2H1mkA:9 a=ccKwqeaiEzpYSDcr:21 a=EP3vqE149MM8ZW0Q:21 a=CjuIK1q_8ugA:10 a=0RhZnL1DYvcuLYC8JZ5M:22 a=sTfVWhjbJpNfc6ZWP1zY:22 a=_FVE-zBwftR9WsbkzFJk:22 Original-Received: from dell.homenet (localhost [127.0.0.1]) by dell.homenet (Postfix) with ESMTP id 87138443386; Sun, 19 Jun 2016 21:09:12 +0100 (BST) In-Reply-To: <8737o9kr3g.fsf@pobox.com> X-Mailer: Claws Mail 3.13.1 (GTK+ 2.24.30; x86_64-unknown-linux-gnu) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x (no timestamps) [generic] X-Received-From: 84.93.230.235 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.org gmane.lisp.guile.devel:18353 gmane.lisp.guile.user:12656 Archived-At: On Sun, 19 Jun 2016 19:48:03 +0200 Andy Wingo wrote: > On Sun 19 Jun 2016 17:33, Chris Vine > writes: > > > The answer I have adopted when reading from TCP sockets is to > > extract individual bytes only from the port into a bytevector using > > R6RS's get-u8 procedure and (if the port is textual rather than > > binary) to reconstruct characters from that using > > bytevector->string at, say, a line end[1]. An EAGAIN/EWOULDBLOCK > > exception is then just treated as an invitation to return to the > > prompt, and read state is retained in the bytevector. > > Yep, makes sense, though it will be really slow of course. And, you'd > have to make people rewrite their code to use your I/O primitives > instead of using read-line from (ice-9 rdelim); a bit of a drag, but > OK. It's not excessively slow provided that the port itself (as opposed to the conversion to unicode characters, which you have to do yourself) can be buffered. You can loop calls to get-u8 on char-ready? to vacate the buffers so far as wanted, without returning to the prompt for every byte. From that point of view it shouldn't be that much slower than repeatedly calling read-char on a port with latin-1 encoding on a normal blocking port. [snip] > > I don't think I have got to grips with how to do that with > > read-waiter, because the read-waiter comprises in effect another > > loop (in which the main event loop with its own prompts would have > > to run) until the read request has been satisfied. I would need to > > think about it. Since ethreads use a poll()/epoll() loop, > > presumably you think it is straightforward enough to integrate the > > two, even if at present I don't. > > Here is the code. First, a helper: > > ;; The AFTER-SUSPEND thunk allows the user to suspend the current > ;; thread, saving its state, and then perform some other nonlocal > ;; control flow. > ;; > (define* (suspend #:optional (after-suspend (lambda (ctx thread) > #f))) ((abort-to-prompt (econtext-prompt-tag (current-econtext)) > after-suspend))) > > Assume there is some current-econtext parameter with a "context" which > holds the prompt tag associated with the scheduler. As you see when > the continuation resumes, it resumes by calling a thunk, allowing > exceptions to be thrown from the context of the suspend. > > Here's the main loop function, which you could replace with the GLib > main loop or whatever: > > (define* (run #:optional (ctx (ensure-current-econtext)) > #:key (install-suspendable-ports? #t)) > (when install-suspendable-ports? (install-suspendable-ports!)) > (parameterize ((current-econtext ctx) > (current-read-waiter wait-for-readable) > (current-write-waiter wait-for-writable)) > (let lp () > (run-ethread ctx (next-thread ctx)) > (lp)))) > > Cool. Now the wait functions: > > (define (wait-for-readable port) > (wait-for-events port (port-read-wait-fd port) (logior EPOLLIN > EPOLLRDHUP))) > > (define (wait-for-writable port) > (wait-for-events port (port-write-wait-fd port) EPOLLOUT)) > > Now the wait-for-events function: > > (define (wait-for-events port fd events) > (handle-events > port > events > (suspend > (lambda (ctx thread) > ...)))) > > Well that's a mess, but the thing to know is that the `suspend' will > abort to the relevant prompt, and then invoke the thunk that's its > argument. Here's `handle-events' and we are done: > > (define (handle-events port events revents) > (unless (zero? (logand revents EPOLLERR)) > (error "error reading from port" port))) > > But I guess that error could be "reading or writing"; oh well. > > See > http://git.savannah.gnu.org/cgit/guile.git/tree/module/ice-9/ethreads.scm?h=wip-ethreads&id=253dc1a7114b89351a3aa330caf173b98c5a65dd, > it's not long but it takes some time to read. I think I can fairly > ask that of you though, given your interest in this area :) OK I am grateful for your patience in explaining this. I need to think about it, but while this works where all events come from user-derived events, I doubt that this would work with guile-gnome and the glib main loop in the round, because for gtk+ to function the glib main loop must block within glib's own internal call to the poll() and not within the guile prompt, or gdk and other events will jam up. You would probably need to run the glib main loop in its own (pthread) thread, about which you have previously given dire warnings so far as guile as it currently stands is concerned. As I say, I really need to think more about this and look further at the code. I may well be missing something. > > On a side issue, I am still trying to understand the point of > > causing guile-2.2's read of a non-blocking C port to block. The > > whole point of making a descriptor non-blocking is that that > > shouldn't happen, and there are circumstances where pealing > > individual bytes off a non-blocking port as they become available > > is what you want to do. It makes guile's select wrapper unusable > > with TCP sockets on linux. I understand that suspendable-ports > > work differently, but that is another matter. > > I would start by saying that many things are possible including adding > the ability to enable the error-throwing behavior. I think your use > case is a great one and we should support it well. I think though > that perhaps you didn't see how the waiter parameters could apply to > your use case; are things clearer now? I think that once you allow > yourself to use port buffers, things will be much easier and much > more efficient as well. But, I could be misunderstanding too. > > Guile's I/O in the past was very oriented towards ASCII. 2.0 went a > long way towards good textual I/O with other encodings, and with 2.2 > we will have good textual non-blocking I/O. It could be that by > focusing on this, I have neglected some binary non-blocking I/O use > cases, though I think that get-bytevector-some, having enabled > suspendable ports, serves many of these purposes very well. But if > there is a need for an additional binary I/O primitive, let's figure > out what it is and implement it. This is an issue with R6RS. In my view it ought to have a get-bytevector-some! procedure which emulates unix read(), but once you decide that C ports should not have non-blocking reads, that is an end to it really. In my view though, a non-blocking get-bytevector-some! would be really nice. It could return the number of bytes extracted, with a return value of 0 for EAGAIN. Chris