From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Mark H Weaver Newsgroups: gmane.lisp.guile.devel Subject: Re: [PATCH] Add current-suspendable-io-status parameter Date: Tue, 14 May 2019 16:22:09 -0400 Message-ID: <878sv8kcqb.fsf@netris.org> References: <87pnomjcs9.fsf@netris.org> <87lfzaj6ya.fsf@netris.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="123710"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux) Cc: guile-devel To: Nala Ginrut Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Tue May 14 22:24:16 2019 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hQdyC-000W4F-98 for guile-devel@m.gmane.org; Tue, 14 May 2019 22:24:16 +0200 Original-Received: from localhost ([127.0.0.1]:53735 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQdyB-0002Pd-7o for guile-devel@m.gmane.org; Tue, 14 May 2019 16:24:15 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:34134) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hQdxv-0002PN-Dt for guile-devel@gnu.org; Tue, 14 May 2019 16:24:00 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hQdxu-00012x-3m for guile-devel@gnu.org; Tue, 14 May 2019 16:23:59 -0400 Original-Received: from world.peace.net ([64.112.178.59]:43434) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hQdxu-00012I-0N for guile-devel@gnu.org; Tue, 14 May 2019 16:23:58 -0400 Original-Received: from mhw by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1hQdxs-0005FS-Eh; Tue, 14 May 2019 16:23:56 -0400 In-Reply-To: (Nala Ginrut's message of "Tue, 14 May 2019 12:22:24 +0800") X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 64.112.178.59 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.org gmane.lisp.guile.devel:19914 Archived-At: Hi Nala, Nala Ginrut writes: > On Tue, May 14, 2019 at 7:01 AM Mark H Weaver wrote: >> I guess what you want is the ability to see incremental reports on the >> progress of your large I/O operations. Is that right? If we are going >> to add an API for this, it needs to be reliable, and always give reports >> in terms of the high-level requests that the user gave. > > Yes, that's exactly what I want. We need to get the progress of I/O > operation when it's blocking > so that we can compute a fair priority for the tasks. > >> My preferred approach would be something like this: we could add a >> 'put-bytevector-some' I/O primitive which writes some bytes from a >> bytevector, blocking only as needed to write at least one byte. It >> would return the number of bytes written. This can be used to implement >> an efficient variant of 'put-bytevector' that gives you access to the >> real-time progress information. > > I'm not sure if put-bytevector-some does the work, I'll list my concerns: > > 1. All I/O will be managed by Guile when we enabled suspendable-port. > That is to say, from the users side, users never know their I/O > operations are blocking or not. It's transparent to users. > Guile will guarantee the I/O operations to be finished by managing all > the blocking I/O mechanisms. > Users can only interact with the task with read or write waiter, which > are registered by users themselves. > In this scenario, users are out of control of I/O operations. And they > have no way to get the progress of I/O, since there's > no way to pass this status to the waiter function except for > parameters in my patch. > > 2. suspendable-port module has already provided a bunch of overridden > bytevector-* functions. > However, they're hidden from users. I think it's good since the > purpose of suspendable-port is to abstract all these details from > users. Users only consider the read-waiter and write-waiter for scheduling. > If we provide the low-level bytevector functions to users to let them > do the non-blocking I/O by themselves, just like most C framework > does. Then Guile suspendable-port will lose a critical feature, > although users can still implement asynchronous non-blocking I/O by > themselves with a non-managed approach. Say, do the I/O, check result > by themselves, and do the scheduling. > Personally, I'm fine with this way, since I'm familiar with both ways. > But managed I/O of suspendable-port is a good selling point for many > inexperienced server-side developers, they can use it in Scheme just > like IOCP or AIO. > > Of course, I may misunderstand your mind. > Could you elaborate more about your approach? Here's what I had in mind. I'm not suggesting that you use asynchronous non-blocking I/O. 'put-bytevector-some' would be like 'put-bytevector' except that it can write less than you asked it to. Ideally it would block only as needed to write at least one byte. It would be analogous to POSIX write(2). Here's an untested draft implementation of 'put-bytevector*' which sets the parameter 'current-io-status' to provide the information you wanted, except that here the information will reliably refer to your own buffer. The primitive it is built on 'put-bytevector-some' does not yet exist, but we could add it. Note that 'put-bytevector-some' would still block, and it would be suspendable. --8<---------------cut here---------------start------------->8--- (define* (put-bytevector* port bv #:optional (start 0) (count (bytevector-length bv))) (let loop ((start start) (count count)) (unless (zero? count) (let ((written (parameterize ((current-io-status (cons start count))) (put-bytevector-some port bv start count)))) (loop (+ start written) (- count written)))))) --8<---------------cut here---------------end--------------->8--- We can already do something analogous on the reader side, using either 'get-bytevector-some' or the already-implemented-but-not-yet-pushed 'get-bytevector-some!'. What do you think? Mark