From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Andy Wingo Newsgroups: gmane.lisp.guile.devel,gmane.lisp.guile.user Subject: Re: anyone define port types? Date: Sun, 19 Jun 2016 19:48:03 +0200 Message-ID: <8737o9kr3g.fsf@pobox.com> References: <87y492mnjp.fsf@pobox.com> <87pots9tag.fsf@gnu.org> <87bn37wtf2.fsf@pobox.com> <20160612092513.3eb1c8a3@laptop.homenet> <8760t5mthu.fsf@pobox.com> <20160619163327.36246706@dell.homenet> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1466358541 31622 80.91.229.3 (19 Jun 2016 17:49:01 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 19 Jun 2016 17:49:01 +0000 (UTC) Cc: guile-user , guile-devel To: Chris Vine Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Sun Jun 19 19:48:53 2016 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1bEgq2-0006BR-49 for guile-devel@m.gmane.org; Sun, 19 Jun 2016 19:48:50 +0200 Original-Received: from localhost ([::1]:39585 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bEgpy-0006vF-0p for guile-devel@m.gmane.org; Sun, 19 Jun 2016 13:48:46 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:38324) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bEgpY-0006dv-2W for guile-devel@gnu.org; Sun, 19 Jun 2016 13:48:21 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1bEgpT-0000DN-SW for guile-devel@gnu.org; Sun, 19 Jun 2016 13:48:19 -0400 Original-Received: from pb-sasl1.pobox.com ([64.147.108.66]:58010 helo=sasl.smtp.pobox.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1bEgpT-0000DA-Nu; Sun, 19 Jun 2016 13:48:15 -0400 Original-Received: from sasl.smtp.pobox.com (unknown [127.0.0.1]) by pb-sasl1.pobox.com (Postfix) with ESMTP id A79B920956; Sun, 19 Jun 2016 13:48:12 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type; s=sasl; bh=jjHTOcIWgwOfbb5WDmFn4yD3wBE=; b=CS0nOV ZiTOkIIRYxDXdojXE3GefSJesN3KbXi56JJQkqaiRR+R+8yb8olKfImot4j/lOFo rCy3Gia5BEzrULHy4yVebhaHJDXhYH6g7z3qwm4mg1UlCjkFjEcJ3MQX0syTU8lr DaKI0qvfCAlZ8A++Ypaa6aSs2Q1Cxv4y0VUh4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type; q=dns; s=sasl; b=xxNFVgUpiHcijMxlbFmgqRIBh0G5YtrM QKnbkL/c/gK45ERw/NICkZVi1ZsMXVVhgVARIZ7DiDgqpFCb+0x8rxCUCHNoaTw3 E5fA2rnqFKonJIOpjjIwM5NRsyhyKtaRRmVmzPFYL9Q4J0KxCpxlTFywnHQLV2+d C80Tt5bRhYM= Original-Received: from pb-sasl1.nyi.icgroup.com (unknown [127.0.0.1]) by pb-sasl1.pobox.com (Postfix) with ESMTP id 8F88E20954; Sun, 19 Jun 2016 13:48:12 -0400 (EDT) Original-Received: from clucks (unknown [88.160.190.192]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pb-sasl1.pobox.com (Postfix) with ESMTPSA id 8BBC920951; Sun, 19 Jun 2016 13:48:11 -0400 (EDT) In-Reply-To: <20160619163327.36246706@dell.homenet> (Chris Vine's message of "Sun, 19 Jun 2016 16:33:27 +0100") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) X-Pobox-Relay-ID: FDB50DA4-3645-11E6-90C6-C1836462E9F6-02397024!pb-sasl1.pobox.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 64.147.108.66 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.org gmane.lisp.guile.devel:18352 gmane.lisp.guile.user:12653 Archived-At: On Sun 19 Jun 2016 17:33, Chris Vine writes: > The answer I have adopted when reading from TCP sockets is to extract > individual bytes only from the port into a bytevector using R6RS's > get-u8 procedure and (if the port is textual rather than binary) to > reconstruct characters from that using bytevector->string at, say, a > line end[1]. An EAGAIN/EWOULDBLOCK exception is then just treated as > an invitation to return to the prompt, and read state is retained in > the bytevector. Yep, makes sense, though it will be really slow of course. And, you'd have to make people rewrite their code to use your I/O primitives instead of using read-line from (ice-9 rdelim); a bit of a drag, but OK. >> Why not just (install-suspendable-ports!) and >> >> (parameterize ((current-read-waiter my-read-waiter)) ...) >> >> etc? It is entirely possible with Guile 2.1.3 to build an >> asynchronous coroutine-style concurrent system in user-space using >> these primitives. See the wip-ethread branch for an example >> implementation. > > I would want to continue using an external event loop implemented with > poll() or select() and delimited continuations. Yes yes a million times yes! That's what I want too. To that end, the current-read-waiter parameter's value is a function which takes the port being waited on, and when it returns the operation will retry. In wip-ethreads I suspend to a prompt, add the port's fd to an epoll set, and when the fd becomes readable again we resume the continuation, resulting in a return from the current-read-waiter. > I don't think I have got to grips with how to do that with read-waiter, > because the read-waiter comprises in effect another loop (in which the > main event loop with its own prompts would have to run) until the read > request has been satisfied. I would need to think about it. Since > ethreads use a poll()/epoll() loop, presumably you think it is > straightforward enough to integrate the two, even if at present I don't. Here is the code. First, a helper: ;; The AFTER-SUSPEND thunk allows the user to suspend the current ;; thread, saving its state, and then perform some other nonlocal ;; control flow. ;; (define* (suspend #:optional (after-suspend (lambda (ctx thread) #f))) ((abort-to-prompt (econtext-prompt-tag (current-econtext)) after-suspend))) Assume there is some current-econtext parameter with a "context" which holds the prompt tag associated with the scheduler. As you see when the continuation resumes, it resumes by calling a thunk, allowing exceptions to be thrown from the context of the suspend. Here's the main loop function, which you could replace with the GLib main loop or whatever: (define* (run #:optional (ctx (ensure-current-econtext)) #:key (install-suspendable-ports? #t)) (when install-suspendable-ports? (install-suspendable-ports!)) (parameterize ((current-econtext ctx) (current-read-waiter wait-for-readable) (current-write-waiter wait-for-writable)) (let lp () (run-ethread ctx (next-thread ctx)) (lp)))) Cool. Now the wait functions: (define (wait-for-readable port) (wait-for-events port (port-read-wait-fd port) (logior EPOLLIN EPOLLRDHUP))) (define (wait-for-writable port) (wait-for-events port (port-write-wait-fd port) EPOLLOUT)) Now the wait-for-events function: (define (wait-for-events port fd events) (handle-events port events (suspend (lambda (ctx thread) ...)))) Well that's a mess, but the thing to know is that the `suspend' will abort to the relevant prompt, and then invoke the thunk that's its argument. Here's `handle-events' and we are done: (define (handle-events port events revents) (unless (zero? (logand revents EPOLLERR)) (error "error reading from port" port))) But I guess that error could be "reading or writing"; oh well. See http://git.savannah.gnu.org/cgit/guile.git/tree/module/ice-9/ethreads.scm?h=wip-ethreads&id=253dc1a7114b89351a3aa330caf173b98c5a65dd, it's not long but it takes some time to read. I think I can fairly ask that of you though, given your interest in this area :) > On a side issue, I am still trying to understand the point of causing > guile-2.2's read of a non-blocking C port to block. The whole point of > making a descriptor non-blocking is that that shouldn't happen, and > there are circumstances where pealing individual bytes off a > non-blocking port as they become available is what you want to do. It > makes guile's select wrapper unusable with TCP sockets on linux. I > understand that suspendable-ports work differently, but that is another > matter. I would start by saying that many things are possible including adding the ability to enable the error-throwing behavior. I think your use case is a great one and we should support it well. I think though that perhaps you didn't see how the waiter parameters could apply to your use case; are things clearer now? I think that once you allow yourself to use port buffers, things will be much easier and much more efficient as well. But, I could be misunderstanding too. Guile's I/O in the past was very oriented towards ASCII. 2.0 went a long way towards good textual I/O with other encodings, and with 2.2 we will have good textual non-blocking I/O. It could be that by focusing on this, I have neglected some binary non-blocking I/O use cases, though I think that get-bytevector-some, having enabled suspendable ports, serves many of these purposes very well. But if there is a need for an additional binary I/O primitive, let's figure out what it is and implement it. Andy