From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Olivier Dion via "Developers list for Guile, the GNU extensibility library" Newsgroups: gmane.lisp.guile.devel,gmane.lisp.guile.user Subject: Re: map-par slower than map Date: Wed, 12 Oct 2022 17:55:10 -0400 Message-ID: <87bkqg7lmp.fsf@laura> References: Reply-To: Olivier Dion Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="35239"; mail-complaints-to="usenet@ciao.gmane.io" To: Damien Mattei , guile-user , guile-devel Original-X-From: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Wed Oct 12 23:55:42 2022 Return-path: Envelope-to: guile-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oijhV-0008yF-3P for guile-devel@m.gmane-mx.org; Wed, 12 Oct 2022 23:55:41 +0200 Original-Received: from localhost ([::1]:44762 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oijhT-0004WE-M5 for guile-devel@m.gmane-mx.org; Wed, 12 Oct 2022 17:55:39 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:54814) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oijhD-0004Vs-CZ; Wed, 12 Oct 2022 17:55:23 -0400 Original-Received: from smtp.polymtl.ca ([132.207.4.11]:34564) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oijhA-00085h-GD; Wed, 12 Oct 2022 17:55:22 -0400 Original-Received: from localhost (modemcable094.169-200-24.mc.videotron.ca [24.200.169.94]) by smtp.polymtl.ca (8.14.7/8.14.7) with ESMTP id 29CLtAWN012858; Wed, 12 Oct 2022 17:55:15 -0400 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp.polymtl.ca 29CLtAWN012858 In-Reply-To: X-Poly-FromMTA: (modemcable094.169-200-24.mc.videotron.ca [24.200.169.94]) at Wed, 12 Oct 2022 21:55:10 +0000 Received-SPF: pass client-ip=132.207.4.11; envelope-from=olivier.dion@polymtl.ca; helo=smtp.polymtl.ca X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.io gmane.lisp.guile.devel:21415 gmane.lisp.guile.user:18630 Archived-At: On Wed, 12 Oct 2022, Damien Mattei wrote: > Hello, > all is in the title, i test on a approximately 30000 element list , i got > 9s with map and 3min 30s with par-map on exactly the same piece of > code!? I can only speculate here. But trying with a very simple example here: --8<---------------cut here---------------start------------->8--- (use-modules (statprof)) (statprof (lambda () (par-map 1+ (iota 300000)))) --8<---------------cut here---------------end--------------->8--- Performance are terrible. I don't know how par-map is implemented, but if it does 1 element static scheduling -- which it probably does because you pass a linked list and not a vector -- then yeah you can assure that thing will be very slow. You're probably better off with dynamic scheduling with vectors. Here's a quick snippet I made for static scheduling but with vectors. Feel free to roll your own. --8<---------------cut here---------------start------------->8--- (use-modules (srfi srfi-1) (ice-9 threads)) (define* (par-map-vector proc input #:optional (max-thread (current-processor-count))) (let* ((block-size (quotient (vector-length input) max-thread)) (rest (remainder (vector-length input) max-thread)) (output (make-vector (vector-length input) #f))) (when (not (zero? block-size)) (let ((mtx (make-mutex)) (cnd (make-condition-variable)) (n 0)) (fold (lambda (scale output) (begin-thread (let lp ((i 0)) (when (< i block-size) (let ((i (+ i (* scale block-size)))) (vector-set! output i (proc (vector-ref input i)))) (lp (1+ i)))) (with-mutex mtx (set! n (1+ n)) (signal-condition-variable cnd))) output) output (iota max-thread)) (with-mutex mtx (while (not (< n max-thread)) (wait-condition-variable cnd mtx)))) (let ((base (- (vector-length input) rest))) (let lp ((i 0)) (when (< i rest) (let ((i (+ i base))) (vector-set! output i (proc (vector-ref input i)))) (lp (1+ i))))) output)) --8<---------------cut here---------------end--------------->8--- -- Olivier Dion oldiob.dev