From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Chris Vine Newsgroups: gmane.lisp.guile.user Subject: Re: Limiting parallelism using futures, parallel forms and fibers Date: Fri, 10 Jan 2020 16:08:25 +0000 Message-ID: <20200110160825.ed165619cb590b8755d94f8e@gmail.com> References: <04cb0461-18a1-ef17-4db7-2475c7c806e6@posteo.de> <20200108114402.2cabbd011c52456a73a2fca8@gmail.com> <27b27190-cc9c-f0b8-574f-7ef21bf3f313@posteo.de> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="111788"; mail-complaints-to="usenet@blaine.gmane.org" Cc: Guile User To: Zelphir Kaltstahl Original-X-From: guile-user-bounces+guile-user=m.gmane-mx.org@gnu.org Fri Jan 10 17:11:54 2020 Return-path: Envelope-to: guile-user@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1ipwpj-000Cp0-TM for guile-user@m.gmane-mx.org; Fri, 10 Jan 2020 17:08:24 +0100 Original-Received: from localhost ([::1]:48530 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ipwpi-0006Gt-GF for guile-user@m.gmane-mx.org; Fri, 10 Jan 2020 11:08:22 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:51796) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ipwpV-0006Cs-2a for guile-user@gnu.org; Fri, 10 Jan 2020 11:08:10 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ipwpT-0001wF-RU for guile-user@gnu.org; Fri, 10 Jan 2020 11:08:08 -0500 Original-Received: from mail-wm1-x329.google.com ([2a00:1450:4864:20::329]:55548) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ipwpT-0001to-HP for guile-user@gnu.org; Fri, 10 Jan 2020 11:08:07 -0500 Original-Received: by mail-wm1-x329.google.com with SMTP id q9so2574913wmj.5 for ; Fri, 10 Jan 2020 08:08:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UziVcUU2PckJ2KP9Vuxj4xNT2+A7iGf7M6rMkBGv/no=; b=efnYy/jO+P2V3D3GfqUdnua7tiQHZO7+1xQGw0y6NRcyHzuFATT2nJJuBg8bAIehZp jb1mjIzqbtql1mPXKNFmeTm1DOCyLPtNF+nKXM0GvgEwJneO3FNNIIUuQg5a/KY1L9Pk 8pt9eVrvq2mzaFinSD0yrJ3ClFWkYPnNvGx5qmOiO8zBwgpSteVVLOPSx3Qr6AfrmBmO fyEYHoeyI60taIyaCjxuNIdO3OxB6/T00JGBbEuLdT27o0VDtxPPJ5XBPvWekAPoUQ1S ZfN0Uf8rFbCA8v8UWuol2tQyGmPlb8jpayodrPvhBHbe+YKFwHHPJ0XWRwlXm7XWnkLC ChXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UziVcUU2PckJ2KP9Vuxj4xNT2+A7iGf7M6rMkBGv/no=; b=PiEK0nm7etStu0NiWmGNNdP5McPIxrdlY0PmaormuI8TtvBSHDIVJgkPhOQ2DN+xoq kzh0onFVUBAukiKHvBulldAumKd7eD0NDoj+1oB/MkqlI+2tK3eHrS4G9e0z/GG/i5ay d8uFdjT9vFxmcwVNoXKTwrogaGENBFA/22v6kSf6Ibcd92lACIQ1ZXpIbb5c4mvipu7W b/h1o7lV7/34mp0BtJxYswc3QOH/yZiH1l8ACtqeLQ3Dy2v3r8KbrCpLmUbt/FQHkjJh gaBL5FpUgVbnYVfcA9XoIX/4ti/woIt2jA7267wXskxD5L8vC/aDG/bCs0iUjHsV78q8 wX4w== X-Gm-Message-State: APjAAAUPgxNGMMxzBeKm69U0vWkdb8+q1odhdwIeXniedVxhKOPgIj3P 94avsoDPHBosQ9aHbPG+VCWK48ul X-Google-Smtp-Source: APXvYqzvJ6KfP9HiSMUNFmrqK5pQh5s6Hh5nbnagIwd+dC5KmRImakXBcqDtRd108cDnxpZISdqLnA== X-Received: by 2002:a7b:c084:: with SMTP id r4mr4909710wmh.99.1578672483648; Fri, 10 Jan 2020 08:08:03 -0800 (PST) Original-Received: from bother.homenet ([91.110.243.20]) by smtp.gmail.com with ESMTPSA id v3sm2616976wru.32.2020.01.10.08.08.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Jan 2020 08:08:03 -0800 (PST) Original-Received: from bother.homenet (localhost [127.0.0.1]) by bother.homenet (Postfix) with SMTP id 669F52603FC; Fri, 10 Jan 2020 16:08:25 +0000 (GMT) In-Reply-To: <27b27190-cc9c-f0b8-574f-7ef21bf3f313@posteo.de> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; x86_64-unknown-linux-gnu) X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::329 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane-mx.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.org gmane.lisp.guile.user:16026 Archived-At: On Fri, 10 Jan 2020 01:43:18 +0100 Zelphir Kaltstahl wrote: > Hi Chris! > > On 1/8/20 12:44 PM, Chris Vine wrote: > > On Wed, 8 Jan 2020 08:56:11 +0100 > > Zelphir Kaltstahl wrote: > > [snip] > >> So my questions are: > >> > >> - Is there a default / recommended way to limit parallelism for > >> recursive calls to parallel forms? > >> > >> - Is there a better way than a global counter with locking, to limit the > >> number of futures created during recursive calls? I would dislike very > >> much to have to do something like global state + mutex. > >> > >> - What do you recommend in general to solve this? > > I think you have it wrong, and that futures use a global queue and a > > global set of worker threads. I don't see how futures could work > > without at least a global set of worker threads. Have a look at the > > futures source code. > > So far I did not write any parallel code in my project. I am only > exploring possibilities, finding out what I should be using. I also > could not think of a good way to limit the number of threads to anything > less than the (current-processor-count) value, but thought maybe someone > knows something I did not see or read about Guile's futures yet : ) I still don't understand why you think you want to do that, given the aims you set out in your earlier posts. Why do you want to spawn less than (processor-count - 1) threads for your algorithm, which is what futures would use? But if you really want to do that, and you don't want to use a stand-alone thread pool, the parallelism keyword of fibers seems to do what you want.