From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Mark H Weaver Newsgroups: gmane.lisp.guile.devel Subject: Re: Whats' the proper senario of par-map? (Was Re: bug#13188: par-map causes VM stack overflow) Date: Thu, 28 Mar 2013 14:00:52 -0400 Message-ID: <87ip4b9zfv.fsf@tines.lan> References: <1355559152.27310.5.camel@Renee-desktop.suse> <87y5d8rclr.fsf@gnu.org> <1364439334.2730.41.camel@Renee-desktop.suse> <874nfwazc3.fsf@tines.lan> <87r4izprks.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1364493677 7545 80.91.229.3 (28 Mar 2013 18:01:17 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 28 Mar 2013 18:01:17 +0000 (UTC) Cc: 13188-done@debbugs.gnu.org, guile-devel@gnu.org To: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Thu Mar 28 19:01:41 2013 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1ULH8r-0001Pv-Ob for guile-devel@m.gmane.org; Thu, 28 Mar 2013 19:01:37 +0100 Original-Received: from localhost ([::1]:53948 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ULH8T-0003Ri-ND for guile-devel@m.gmane.org; Thu, 28 Mar 2013 14:01:13 -0400 Original-Received: from eggs.gnu.org ([208.118.235.92]:34167) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ULH8P-0003PE-Bl for guile-devel@gnu.org; Thu, 28 Mar 2013 14:01:10 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ULH8O-0002k1-0w for guile-devel@gnu.org; Thu, 28 Mar 2013 14:01:09 -0400 Original-Received: from world.peace.net ([96.39.62.75]:53066) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ULH8N-0002jv-R7; Thu, 28 Mar 2013 14:01:07 -0400 Original-Received: from 209-6-91-212.c3-0.smr-ubr1.sbo-smr.ma.cable.rcn.com ([209.6.91.212] helo=tines.lan) by world.peace.net with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.72) (envelope-from ) id 1ULH8G-0007Nc-4C; Thu, 28 Mar 2013 14:01:00 -0400 In-Reply-To: <87r4izprks.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Thu, 28 Mar 2013 14:44:03 +0100") User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 96.39.62.75 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:16048 Archived-At: Hi Ludovic, ludo@gnu.org (Ludovic Court=C3=A8s) writes: > Mark H Weaver skribis: > >> It only makes sense to use 'par-map' when the procedure is fairly >> expensive to compute. > > Indeed. > >> There is inevitably a lot of overhead in creating and joining the >> threads. > > We use a thread pool, so there=E2=80=99s no such cost. Sorry, I was using the term 'threads' not in the sense of OS-level threads, but in a more general sense. I should have been more clear. What I meant is that from the user's perspective, threads are being created and joined, and even if you build those using a pool of OS-level threads, this inevitably involves thread synchronization, which is very expensive on modern architectures. So I maintain that there _is_ such a cost, and it can't be avoided. The point I was really trying to make here, in the simplest possible terms, is that it will *never* make sense to replace all uses of 'map' with 'par-map' wherever it is safe to do so. > But there are other costs. When delimited continuations are used, we=E2= =80=99re > on the slow path. Also, Guile=E2=80=99s fat mutexes & co. are terribly > inefficient. And finally, there may be contention on the futexes mutex > (esp. when the computations is too small.) Indeed, I wouldn't be surprised if we could improve this by an order of magnitude. More items for my TODO list :) > So yes, there=E2=80=99s room for improvement. Yet, it should be fruitful, > provided you use it for reasonably long computations, as Mark outlines. Regards, Mark