From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Andy Wingo Newsgroups: gmane.lisp.guile.devel Subject: Re: ice-9 async-queue Date: Tue, 07 Feb 2012 09:44:33 +0100 Message-ID: <87ehu7f2xa.fsf@pobox.com> References: <87d39rhp6i.fsf@pobox.com> <87vcnjmuxx.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: dough.gmane.org 1328604293 22709 80.91.229.3 (7 Feb 2012 08:44:53 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Tue, 7 Feb 2012 08:44:53 +0000 (UTC) Cc: guile-devel@gnu.org To: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Tue Feb 07 09:44:52 2012 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([140.186.70.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1RugfT-0005q1-6u for guile-devel@m.gmane.org; Tue, 07 Feb 2012 09:44:51 +0100 Original-Received: from localhost ([::1]:59321 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RugfS-0002Au-Qb for guile-devel@m.gmane.org; Tue, 07 Feb 2012 03:44:50 -0500 Original-Received: from eggs.gnu.org ([140.186.70.92]:47008) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RugfM-0002Am-32 for guile-devel@gnu.org; Tue, 07 Feb 2012 03:44:49 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1RugfG-00050U-FJ for guile-devel@gnu.org; Tue, 07 Feb 2012 03:44:44 -0500 Original-Received: from a-pb-sasl-sd.pobox.com ([74.115.168.62]:51894 helo=sasl.smtp.pobox.com) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1RugfG-00050Q-Ci; Tue, 07 Feb 2012 03:44:38 -0500 Original-Received: from sasl.smtp.pobox.com (unknown [127.0.0.1]) by a-pb-sasl-sd.pobox.com (Postfix) with ESMTP id DFC50694C; Tue, 7 Feb 2012 03:44:37 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type:content-transfer-encoding; s=sasl; bh=WJaQIszV/OJN 2hCuTTxB50N095o=; b=EaDRAS3h3Tr+XxQWj45MyVtAKJKEk4ivzy5mTlbs/AtJ 6lXNKeUyqkeb1WjsBlAsEejsyceUQ3NSvopczBr07GUJDP66E3CcVp1sOrrNOrtR D5FsyQvNW3yuWXFYXOSXqy0uVSQkCzzXI8mWVaRolGPZ2Sw6dchN8/aFUJYUyak= DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=from:to:cc :subject:references:date:in-reply-to:message-id:mime-version :content-type:content-transfer-encoding; q=dns; s=sasl; b=BYewYc 7JhBp1Pv+1Q+nu6nLGcyPbMLPoYjSbP9fmvZ3LOZtl9dEoDzyPPeA/w/HpSS9Kmd Kcxz0ega1lUg1Y45lt55CLMQSvUy1OMXDiYtZa2hgCkcCS6rDVCQ4F9f6k+kEFo9 D7s6yYMIJnqzQlG9Gvm4npDkHkXraZj8XEZAM= Original-Received: from a-pb-sasl-sd.pobox.com (unknown [127.0.0.1]) by a-pb-sasl-sd.pobox.com (Postfix) with ESMTP id D99CE694B; Tue, 7 Feb 2012 03:44:37 -0500 (EST) Original-Received: from badger (unknown [90.164.198.39]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by a-pb-sasl-sd.pobox.com (Postfix) with ESMTPSA id 1A731694A; Tue, 7 Feb 2012 03:44:36 -0500 (EST) In-Reply-To: <87vcnjmuxx.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Mon, 06 Feb 2012 23:57:30 +0100") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.3 (gnu/linux) X-Pobox-Relay-ID: F76F2528-5167-11E1-8146-65B1DE995924-02397024!a-pb-sasl-sd.pobox.com X-detected-operating-system: by eggs.gnu.org: Solaris 10 (beta) X-Received-From: 74.115.168.62 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:13809 Archived-At: On Mon 06 Feb 2012 23:57, ludo@gnu.org (Ludovic Court=C3=A8s) writes: > Andy Wingo skribis: > >> I was thinking of adding the following to Guile, to eventually help make >> the web server a little less terrible. What do you think? > > An =E2=80=9Casynchronous queue=E2=80=9D is a queue of tasks, right? It's a message queue for use in threads organized as producers and consumers. > What kind of tasks would it be: I/O? Computation? Depends on what you want to do. > How does it fit with the web server? The web server is single-threaded and uses blocking IO (though it does poll(2) for keepalive). As such, any slow writer or slow reader can block the process. Using non-blocking I/O is too difficult, for now. So, threads. I'd like to create a pool of threads for I/O. Some threads would pop ports off of the "to-read" queue, read request headers and bodies, then push the requests onto a "to-process" queue. Something (currently the main thread) would process requests, and push them onto the "to-write" queue. IO threads would pop data (or closures) off of the to-write queue, and write them to clients, possibly pushing the ports back on a "to-keepalive" queue, which the poll loop would notice and add those fds back to the poll set. I'd also like to consider creating a separate pool of threads for computation. Obviously the size of these thread pools would be different. We could use futures for that, I suppose, but I'd like to also be able to stop those threads, forcefully if needed, when the web server stops. Andy --=20 http://wingolog.org/