From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Mark H Weaver Newsgroups: gmane.lisp.guile.devel Subject: Re: Delimited continuations to the rescue of futures Date: Sat, 17 Nov 2012 11:56:59 -0500 Message-ID: <87fw48qias.fsf@tines.lan> References: <87ip95unl3.fsf@gnu.org> <87wqxkrggv.fsf@tines.lan> <87haootkdh.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1353171445 30526 80.91.229.3 (17 Nov 2012 16:57:25 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sat, 17 Nov 2012 16:57:25 +0000 (UTC) Cc: guile-devel@gnu.org To: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Sat Nov 17 17:57:34 2012 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1TZli1-0006fd-Iz for guile-devel@m.gmane.org; Sat, 17 Nov 2012 17:57:33 +0100 Original-Received: from localhost ([::1]:60725 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TZlhr-00053U-85 for guile-devel@m.gmane.org; Sat, 17 Nov 2012 11:57:23 -0500 Original-Received: from eggs.gnu.org ([208.118.235.92]:50913) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TZlhl-000534-ML for guile-devel@gnu.org; Sat, 17 Nov 2012 11:57:20 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1TZlhi-0002lM-K4 for guile-devel@gnu.org; Sat, 17 Nov 2012 11:57:17 -0500 Original-Received: from world.peace.net ([96.39.62.75]:43276) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1TZlhi-0002lH-Fg; Sat, 17 Nov 2012 11:57:14 -0500 Original-Received: from 209-6-91-212.c3-0.smr-ubr1.sbo-smr.ma.cable.rcn.com ([209.6.91.212] helo=tines.lan) by world.peace.net with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.72) (envelope-from ) id 1TZlhb-0007X8-4W; Sat, 17 Nov 2012 11:57:07 -0500 In-Reply-To: <87haootkdh.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Sat, 17 Nov 2012 14:43:54 +0100") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.2 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 96.39.62.75 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:15199 Archived-At: Hi Ludovic, ludo@gnu.org (Ludovic Court=C3=A8s) writes: > Mark H Weaver skribis: >> In other words, if you have 4 cores and call 'par-map' on a list of 1000 >> elements, the main thread will only be used to process 1 out of 1000 >> elements, and only 3 cores will be used to process the other 999. >> Is that right? > > No. :-) > > That=E2=80=99s what would happen with current stable-2.0. But in > =E2=80=98wip-nested-futures=E2=80=99, the main thread behaves as a worker= while waiting > for the tail future to complete (which addresses your main concern, I > think, as we had discussed on IRC.) Thus, all cores are always used. Ah, okay, good! :) I guess the one remaining concern I have is that if there are any long-running futures in the process, then any 'touch' could take a very long time to complete, even if the future it is waiting for is a very short job. For example, (par-map - '(1 2 3)) could take several minutes to complete if (par-map process-image list-of-images) is being done in another thread. This doesn't sit well with me. It would be good if we could avoid it, but at the very least we'll have to warn users about this in the documentation. I should mention that this problem already exists in Guile 2.0, and is not related to your recent work. Thanks, Mark