From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Mikael Djurfeldt Newsgroups: gmane.lisp.guile.devel,gmane.lisp.guile.user Subject: Re: map-par slower than map Date: Tue, 25 Oct 2022 11:11:14 +0200 Message-ID: References: <5608809c-89a2-118c-5c05-c46ac3a0e21b@posteo.de> Reply-To: mikael@djurfeldt.com Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000e5c36705ebd84af2" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="1588"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Damien Mattei , guile-user , guile-devel To: Zelphir Kaltstahl Original-X-From: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Tue Oct 25 11:15:08 2022 Return-path: Envelope-to: guile-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1onG1b-0000EZ-Mj for guile-devel@m.gmane-mx.org; Tue, 25 Oct 2022 11:15:07 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1onFyi-0007t2-Bl; Tue, 25 Oct 2022 05:12:08 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1onFyI-0006Sx-DR; Tue, 25 Oct 2022 05:11:44 -0400 Original-Received: from mail-ed1-f50.google.com ([209.85.208.50]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1onFy3-0003Fz-Fz; Tue, 25 Oct 2022 05:11:42 -0400 Original-Received: by mail-ed1-f50.google.com with SMTP id x2so7990864edd.2; Tue, 25 Oct 2022 02:11:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:reply-to:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ZsQ0ZcYNxh7BETHXHfp3fsnOexmsZw+7IBo0yk5bNQM=; b=oBWvQvoQpXzcGCWMcopbPIu+a3PkZsqqVtdWvCJL/Ydr+HNwgdwtu6YRqxN4Eu1Wee 10AQt2v+aUmC6E0DeJa8/S7cJZeDOflsS0Gr53Bs4vCU4TmYWGZol5mshhkupCIaxqql Xz7W8KNBM1ANp/Nnw19067T4mbeVz3M8LEDLtGRYLN+MWsn7mnwEL1eaoXy8L/klh0/f /i5wfRcPxm4To2MuQpL1qooFeKkYzgIuymMcYWXM6bc0/7M5sY7AlP6o1DT/Gt9d/2jv LyJEhLHoDAsjEZP8vZgTm6s4MKYLXf7aALWadtPY1MfkIrsyIxrpz5TliSrMQHv1Rv0P xYaA== X-Gm-Message-State: ACrzQf3jRvyx6Foj+fCFVP66yAfGNT5ZkkKz1hgSuj4D0IGFY3zrgF6c /csgrZC1ltzFLhFjC35G6I2cVvopfz+pARSZeBFnB2p1Zzo= X-Google-Smtp-Source: AMsMyM7orZJ8ooSUnzDSn2fI+Naqnzv2suMa/V5GMuJFBgWYmPPqNda+8QkIik09XJruQs/0B6gJY0prlKDqo6/5beQ= X-Received: by 2002:aa7:c61a:0:b0:461:c48d:effe with SMTP id h26-20020aa7c61a000000b00461c48deffemr8763890edq.7.1666689085587; Tue, 25 Oct 2022 02:11:25 -0700 (PDT) In-Reply-To: Received-SPF: pass client-ip=209.85.208.50; envelope-from=mdjurfeldt@gmail.com; helo=mail-ed1-f50.google.com X-Spam_score_int: -15 X-Spam_score: -1.6 X-Spam_bar: - X-Spam_report: (-1.6 / 5.0 requ) BAYES_00=-1.9, FREEMAIL_FORGED_FROMDOMAIN=0.001, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: "guile-devel" Errors-To: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.lisp.guile.devel:21453 gmane.lisp.guile.user:18680 Archived-At: --000000000000e5c36705ebd84af2 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Also, I would believe that any crashes in this context are neither due to the futures implementation nor par-map et al. I would think that crashes are due to the Guile basic thread support itself. On Tue, Oct 25, 2022 at 11:07 AM Mikael Djurfeldt wrote: > A piece of background on par-map: > > When I introduced par-map et al the only ambition was to have simple > language constructs to invoke parallelism. The use case I had in mind was > course grained parallelism where each piece of work is somewhat > substantial. Back then, a thread was launched for each piece of work, > however there was also a thread pool such that not all of the overhead of > launching new threads always was required. > > Since then, par-map has been rewritten (by others) to be based on futures= . > (And now the thread pool is localized in the futures implementation---as > "workers".) Looking in the code now, I think it is fair to say that it is > still intended for coarse grained parallelism. There is some heavy liftin= g > going on with mutexes and condition variables as well as shuffling around > with list pairs. > > So, applying par-map on a huge list with small amount of work per item wa= s > never and still isn't the intended use case. > > It would of course be interesting if the code could be improved to suppor= t > fine grained parallelism. :-) > > Best regards, > Mikael > > On Wed, Oct 12, 2022 at 11:30 PM Zelphir Kaltstahl < > zelphirkaltstahl@posteo.de> wrote: > >> Hi! >> >> On 10/12/22 22:27, Damien Mattei wrote: >> > >> https://github.com/damien-mattei/library-FunctProg/blob/master/guile/log= iki%2B.scm#L1674 >> > >> > i commited the current version of code here with all files but it is >> > huge.... :-/ >> > >> > On Wed, Oct 12, 2022 at 10:20 PM Damien Mattei > > >> > wrote: >> > >> >> Mutex? i do not think code has situation where dead lock could happen= , >> it >> >> is a code about minimalising logic expressions, it uses minterms , >> minterms >> >> set is a set of minterms :like this: >> >> >> >> example: >> >> ((1 1 0) (1 1 1)) will be unified : (1 1 x) >> >> because 0 and 1 are replaced by x >> >> the minterms-set could have thousands of pair (mathematic not lisp) >> >> minterms to unify >> >> if there is more than one x as result there is no need to continue so= i >> >> escape with a continuation: >> >> >> >> minterms-set =3D >> >> { >> >> ((1 0 1 0) (1 1 1 0)) >> >> ((1 0 1 0) (1 1 0 1)) >> >> ((1 0 1 0) (1 0 1 1)) >> >> ((1 0 1 0) (0 1 1 1)) >> >> ((0 1 1 0) (1 1 1 0)) >> >> ((0 1 1 0) (1 1 0 1)) >> >> ((0 1 1 0) (1 0 1 1)) >> >> ((0 1 1 0) (0 1 1 1)) >> >> ((0 1 0 1) (1 1 1 0)) >> >> ((0 1 0 1) (1 1 0 1)) >> >> ((0 1 0 1) (1 0 1 1)) >> >> ((0 1 0 1) (0 1 1 1)) >> >> ((0 0 1 1) (1 1 1 0)) >> >> ((0 0 1 1) (1 1 0 1)) >> >> ((0 0 1 1) (1 0 1 1)) >> >> ((0 0 1 1) (0 1 1 1)) >> >> } >> >> >> >> replace { } by () to have the list, other example at another level : >> >> >> >> minterms-set =3D >> >> { >> >> ((0 x 1 1) (x 1 1 1)) >> >> ((0 x 1 1) (1 x 1 1)) >> >> ((0 x 1 1) (1 1 x 1)) >> >> ((0 x 1 1) (1 1 1 x)) >> >> ((x 0 1 1) (x 1 1 1)) >> >> ((x 0 1 1) (1 x 1 1)) >> >> ((x 0 1 1) (1 1 x 1)) >> >> ((x 0 1 1) (1 1 1 x)) >> >> ((0 1 x 1) (x 1 1 1)) >> >> ((0 1 x 1) (1 x 1 1)) >> >> ((0 1 x 1) (1 1 x 1)) >> >> ((0 1 x 1) (1 1 1 x)) >> >> ((x 1 0 1) (x 1 1 1)) >> >> ((x 1 0 1) (1 x 1 1)) >> >> ((x 1 0 1) (1 1 x 1)) >> >> ((x 1 0 1) (1 1 1 x)) >> >> ((0 1 1 x) (x 1 1 1)) >> >> ((0 1 1 x) (1 x 1 1)) >> >> ((0 1 1 x) (1 1 x 1)) >> >> ((0 1 1 x) (1 1 1 x)) >> >> ((x 1 1 0) (x 1 1 1)) >> >> ((x 1 1 0) (1 x 1 1)) >> >> ((x 1 1 0) (1 1 x 1)) >> >> ((x 1 1 0) (1 1 1 x)) >> >> ((1 0 1 x) (x 1 1 1)) >> >> ((1 0 1 x) (1 x 1 1)) >> >> ((1 0 1 x) (1 1 x 1)) >> >> ((1 0 1 x) (1 1 1 x)) >> >> ((1 x 1 0) (x 1 1 1)) >> >> ((1 x 1 0) (1 x 1 1)) >> >> ((1 x 1 0) (1 1 x 1)) >> >> ((1 x 1 0) (1 1 1 x)) >> >> } >> >> >> >> here we see some minterms are already unified >> >> >> >> it is not easy to read even by me because i wrote the code many >> years ago >> >> and is split in many files, but here it is: >> >> >> >> (par-map function-unify-minterms-list minterms-set) >> >> >> >> {function-unify-minterms-list <+ (=CE=BB (L) (apply >> >> function-unify-two-minterms-and-tag L))} >> >> >> >> (define (unify-two-minterms mt1 mt2) >> >> (function-map-with-escaping-by-kontinuation2 >> >> (macro-function-compare-2-bits-with-continuation) mt1 mt2)) >> >> >> >> ;; (function-map-with-escaping-by-kontinuation2 >> >> (macro-function-compare-2-bits-with-continuation) '(1 1 0 1 0 1 1 0= ) >> '(1 >> >> 1 0 1 1 1 1 1)) >> >> >> >> ;; list1 =3D (1 1 0 1 0 1 1 0) >> >> ;; more-lists =3D ((1 1 0 1 1 1 1 1)) >> >> ;; lists =3D ((1 1 0 1 0 1 1 0) (1 1 0 1 1 1 1 1)) >> >> ;; clozure =3D # >> >> >> >> ;; #f >> >> ;; >> >> ;; (function-map-with-escaping-by-kontinuation2 >> >> (macro-function-compare-2-bits-with-continuation) '(1 1 0 1 0 1 1 >> 0) '(1 >> >> 1 0 1 1 1 1 0)) >> >> >> >> ;; list1 =3D (1 1 0 1 0 1 1 0) >> >> ;; more-lists =3D ((1 1 0 1 1 1 1 0)) >> >> ;; lists =3D ((1 1 0 1 0 1 1 0) (1 1 0 1 1 1 1 0)) >> >> ;; clozure =3D # >> >> >> >> ;; '(1 1 0 1 x 1 1 0) >> >> (define (function-map-with-escaping-by-kontinuation2 clozure list1 . >> >> more-lists) >> >> (call/cc (lambda (kontinuation) >> >> (let ((lists (cons list1 more-lists)) >> >> (funct-continu ;; this function have the kontinuation in his >> environment >> >> (lambda (arg1 . more-args) >> >> (let ((args (cons arg1 more-args))) >> >> (apply clozure kontinuation args))))) ;; a tester: (apply clozure (co= ns >> >> conti args)) >> >> >> >> ;; (newline) >> >> ;; (dv list1) >> >> ;; (dv more-lists) >> >> ;; (dv lists) >> >> ;; (dv clozure) >> >> ;; (newline) >> >> >> >> (apply map funct-continu lists))))) >> >> >> >> (define-syntax macro-function-compare-2-bits-with-continuation ;; >> >> continuation version of macro-compare-2-bits >> >> ;; i need a macro because of external function to the clozure >> >> (syntax-rules () >> >> ((_) (let ((cnt 0)) ;; counter >> >> (lambda (continuation b1 b2) (if (equal? b1 b2) >> >> b1 >> >> (begin >> >> (set! cnt (add1 cnt)) ;; we leave with continuation in case cpt > >> 1, we >> >> can have used a flag too instead of a counter >> >> (when (> cnt 1) (continuation #f)) ;; escaping with the >> continuation >> >> 'x))))))) ;; return x in case of (b1,b2) =3D (O,1) or (1,0) >> >> >> >> what could have caused mutex if in the latter definition above (let >> ((cnt >> >> 0)) ;; counter was defined at top level and shared by all threads!!! >> yes >> >> there could have be some mutex but this is not the case, i think eve= n >> all >> >> function are pure so why is it more slow with // than without? >> >> Damien >> >> >> >> On Wed, Oct 12, 2022 at 8:45 PM Maxime Devos >> >> wrote: >> >> >> >>> On 12-10-2022 19:19, Damien Mattei wrote: >> >>>> Hello, >> >>>> all is in the title, i test on a approximately 30000 element list ,= i >> >>> got >> >>>> 9s with map and 3min 30s with par-map on exactly the same piece of >> >>> code!? >> >>> > [...] >> >>> > >> >>>> translated from Scheme+ to Scheme: >> >>>> (define unified-minterms-set-1 (map function-unify-minterms-list >> >>>> minterms-set)) ;;(par-map function-unify-minterms-list minterms-set= )) >> >>> The definition of 'function-unify-minterms-list' and 'minterms-set' = is >> >>> missing. Without a test case, we can only speculate what's going on= . >> >>> (E.g., maybe it grabs a mutex). >> >>> >> >>> Greetings, >> >>> Maxime. >> I don't want to scare anyone, just maybe warn about parallel map. I once >> tried >> to use Guile's parallel map function for a decision tree implementation >> ( >> https://notabug.org/ZelphirKaltstahl/guile-ml/src/cf666801fea91c9fa8fa29= 0764ff6c60b7f3949d/decision-tree.scm), >> >> where each branch while learning the tree would call parallel map again >> for sub >> branches and so on. Somehow it made Guile crash (I don't have the error >> message >> any longer, but I did post about it on the mailing list back then.). I >> never >> figured out, what went wrong. All I had was pure function calls and math >> inside >> the thing that parallel map was supposed to run. >> >> Ultimately I simply tried other parallelism constructs and when I >> switched to >> using futures instead, everything worked fine, no crashes, no errors. >> >> Since that time, I did not use parallel map and instead used futures. >> Recently I >> made a parallelization thing for solving exercises of Project Euler usin= g >> multiple cores, so that some solutions are calculated faster. Maybe this >> can >> help or can be adapted to another use case: >> >> >> https://notabug.org/ZelphirKaltstahl/guile-project-euler-solutions/src/e= bb19b11b465903105924adb6252f1e2ecf63859/lib/parallelism.scm#L11-L30 >> >> It expects ranges of things, which are called `segments` in the code. >> Usually >> ranges of numbers for Project Euler things. Here is the code to split a >> range >> into segments: >> >> >> https://notabug.org/ZelphirKaltstahl/guile-project-euler-solutions/src/e= bb19b11b465903105924adb6252f1e2ecf63859/lib/segment.scm >> >> (Check any solution using it for an example.) >> >> So this might be a bit too specific for general parallel things, but I >> guess one >> could change the way futures are used in `run-in-parallel`, to fit any >> other >> purpose. >> >> Best regards, >> Zelphir >> >> -- >> repositories: https://notabug.org/ZelphirKaltstahl >> >> >> --000000000000e5c36705ebd84af2 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Also, I would believe that any crashes in this context are= neither due to the futures implementation nor par-map et al. I would think= that crashes are due to the Guile basic thread support itself.
On Tue, = Oct 25, 2022 at 11:07 AM Mikael Djurfeldt <mikael@djurfeldt.com> wrote:
A piece of backgroun= d on par-map:

When I introduced par-map et al the = only ambition was to have simple language constructs to invoke parallelism.= The use case I had in mind was course grained parallelism where each piece= of work is somewhat substantial. Back then, a thread was launched for each= piece of work, however there was also a thread pool such that not all of t= he overhead of launching new threads always was required.

Since then, par-map has been rewritten (by others) to be based on f= utures. (And now the thread pool is localized in the futures implementation= ---as "workers".) Looking in the code now, I think it is fair to = say that it is still intended for coarse grained parallelism. There is some= heavy lifting going on with mutexes and condition variables as well as shu= ffling around with list pairs.

So, applying par-ma= p on a huge list with small amount of work per item was never and still isn= 't the intended use case.

It would of course b= e interesting if the code could be improved to support fine grained paralle= lism. :-)

Best regards,
Mikael
=

= On Wed, Oct 12, 2022 at 11:30 PM Zelphir Kaltstahl <zelphirkaltstahl@posteo.de&= gt; wrote:
Hi!
On 10/12/22 22:27, Damien Mattei wrote:
> https:/= /github.com/damien-mattei/library-FunctProg/blob/master/guile/logiki%2B.scm= #L1674
>
> i commited the current version of code here with all files but it is > huge.... :-/
>
> On Wed, Oct 12, 2022 at 10:20 PM Damien Mattei <damien.mattei@gmail.com> > wrote:
>
>> Mutex? i do not think code has situation where dead lock could hap= pen, it
>> is a code about minimalising logic expressions, it uses minterms ,= minterms
>> set is a set of minterms :like this:
>>
>> example:
>> ((1 1 0) (1 1 1)) will be unified : (1 1 x)
>> because 0 and 1 are replaced by x
>> the minterms-set could have thousands of pair (mathematic not lisp= )
>> minterms to unify
>> if there is more than one x as result there is no need to continue= so i
>> escape with a continuation:
>>
>> minterms-set =3D
>> {
>> ((1 0 1 0) (1 1 1 0))
>> ((1 0 1 0) (1 1 0 1))
>> ((1 0 1 0) (1 0 1 1))
>> ((1 0 1 0) (0 1 1 1))
>> ((0 1 1 0) (1 1 1 0))
>> ((0 1 1 0) (1 1 0 1))
>> ((0 1 1 0) (1 0 1 1))
>> ((0 1 1 0) (0 1 1 1))
>> ((0 1 0 1) (1 1 1 0))
>> ((0 1 0 1) (1 1 0 1))
>> ((0 1 0 1) (1 0 1 1))
>> ((0 1 0 1) (0 1 1 1))
>> ((0 0 1 1) (1 1 1 0))
>> ((0 0 1 1) (1 1 0 1))
>> ((0 0 1 1) (1 0 1 1))
>> ((0 0 1 1) (0 1 1 1))
>> }
>>
>> replace { } by () to have the list, other example at another level= :
>>
>> minterms-set =3D
>> {
>> ((0 x 1 1) (x 1 1 1))
>> ((0 x 1 1) (1 x 1 1))
>> ((0 x 1 1) (1 1 x 1))
>> ((0 x 1 1) (1 1 1 x))
>> ((x 0 1 1) (x 1 1 1))
>> ((x 0 1 1) (1 x 1 1))
>> ((x 0 1 1) (1 1 x 1))
>> ((x 0 1 1) (1 1 1 x))
>> ((0 1 x 1) (x 1 1 1))
>> ((0 1 x 1) (1 x 1 1))
>> ((0 1 x 1) (1 1 x 1))
>> ((0 1 x 1) (1 1 1 x))
>> ((x 1 0 1) (x 1 1 1))
>> ((x 1 0 1) (1 x 1 1))
>> ((x 1 0 1) (1 1 x 1))
>> ((x 1 0 1) (1 1 1 x))
>> ((0 1 1 x) (x 1 1 1))
>> ((0 1 1 x) (1 x 1 1))
>> ((0 1 1 x) (1 1 x 1))
>> ((0 1 1 x) (1 1 1 x))
>> ((x 1 1 0) (x 1 1 1))
>> ((x 1 1 0) (1 x 1 1))
>> ((x 1 1 0) (1 1 x 1))
>> ((x 1 1 0) (1 1 1 x))
>> ((1 0 1 x) (x 1 1 1))
>> ((1 0 1 x) (1 x 1 1))
>> ((1 0 1 x) (1 1 x 1))
>> ((1 0 1 x) (1 1 1 x))
>> ((1 x 1 0) (x 1 1 1))
>> ((1 x 1 0) (1 x 1 1))
>> ((1 x 1 0) (1 1 x 1))
>> ((1 x 1 0) (1 1 1 x))
>> }
>>
>> here we see some minterms are already unified
>>
>>=C2=A0 =C2=A0it is not easy to read even by me because i wrote the = code many years ago
>> and is split in many files, but here it is:
>>
>> (par-map function-unify-minterms-list minterms-set)
>>
>> {function-unify-minterms-list <+ (=CE=BB (L) (apply
>> function-unify-two-minterms-and-tag L))}
>>
>> (define (unify-two-minterms mt1 mt2)
>>=C2=A0 =C2=A0 (function-map-with-escaping-by-kontinuation2
>>=C2=A0 =C2=A0(macro-function-compare-2-bits-with-continuation) mt1 = mt2))
>>
>> ;; (function-map-with-escaping-by-kontinuation2
>> (macro-function-compare-2-bits-with-continuation)=C2=A0 =C2=A0'= ;(1 1 0 1 0 1 1 0) '(1
>> 1 0 1 1 1 1 1))
>>
>> ;; list1 =3D (1 1 0 1 0 1 1 0)
>> ;; more-lists =3D ((1 1 0 1 1 1 1 1))
>> ;; lists =3D ((1 1 0 1 0 1 1 0) (1 1 0 1 1 1 1 1))
>> ;; clozure =3D #<procedure:...gos-DrRacket.scm:195:11>
>>
>> ;; #f
>> ;;
>> ;;=C2=A0 (function-map-with-escaping-by-kontinuation2
>> (macro-function-compare-2-bits-with-continuation)=C2=A0 =C2=A0 = 9;(1 1 0 1 0 1 1 0) '(1
>> 1 0 1 1 1 1 0))
>>
>> ;; list1 =3D (1 1 0 1 0 1 1 0)
>> ;; more-lists =3D ((1 1 0 1 1 1 1 0))
>> ;; lists =3D ((1 1 0 1 0 1 1 0) (1 1 0 1 1 1 1 0))
>> ;; clozure =3D #<procedure:...gos-DrRacket.scm:195:11>
>>
>> ;; '(1 1 0 1 x 1 1 0)
>> (define (function-map-with-escaping-by-kontinuation2 clozure list1= .
>> more-lists)
>>=C2=A0 =C2=A0 (call/cc (lambda (kontinuation)
>>=C2=A0 =C2=A0 =C2=A0 (let ((lists (cons list1 more-lists))
>>=C2=A0 =C2=A0 (funct-continu ;; this function have the kontinuation= in his environment
>>=C2=A0 =C2=A0 =C2=A0(lambda (arg1 . more-args)
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0(let ((args (cons arg1 more-args)))
>> (apply clozure kontinuation args))))) ;; a tester: (apply clozure = (cons
>> conti args))
>>
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0;; (newline)
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0;; (dv list1)
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0;; (dv more-lists)
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0;; (dv lists)
>>=C2=A0 =C2=A0;; (dv clozure)
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0;; (newline)
>>
>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 (apply map funct-continu lists)))))
>>
>> (define-syntax macro-function-compare-2-bits-with-continuation ;;<= br> >> continuation version of macro-compare-2-bits
>>=C2=A0 =C2=A0 ;; i need a macro because of external function to the= clozure
>>=C2=A0 =C2=A0 (syntax-rules ()
>>=C2=A0 =C2=A0 =C2=A0 ((_) (let ((cnt 0)) ;; counter
>>=C2=A0 =C2=A0 (lambda (continuation b1 b2) (if (equal? b1 b2)
>>=C2=A0 =C2=A0b1
>>=C2=A0 =C2=A0(begin
>>=C2=A0 =C2=A0 =C2=A0(set! cnt (add1 cnt)) ;; we leave with continua= tion in case cpt > 1, we
>> can have used a flag too instead of a counter
>>=C2=A0 =C2=A0 =C2=A0(when (> cnt 1) (continuation #f)) ;; escapi= ng with the continuation
>>=C2=A0 =C2=A0 =C2=A0'x))))))) ;; return x in case of (b1,b2) = =3D (O,1) or (1,0)
>>
>> what could have caused mutex if in the latter definition above (le= t ((cnt
>> 0)) ;; counter was defined at top level and shared by all threads!= !! yes
>> there could have be some mutex=C2=A0 but this is not the case, i t= hink even all
>> function are pure so why is it more slow with // than without?
>> Damien
>>
>> On Wed, Oct 12, 2022 at 8:45 PM Maxime Devos <maximedevos@telenet.be> >> wrote:
>>
>>> On 12-10-2022 19:19, Damien Mattei wrote:
>>>> Hello,
>>>> all is in the title, i test on a approximately 30000 eleme= nt list , i
>>> got
>>>> 9s with map and 3min 30s with par-map on exactly the same = piece of
>>> code!?
>>>=C2=A0 =C2=A0> [...]
>>>=C2=A0 =C2=A0>
>>>> translated from Scheme+ to Scheme:
>>>> (define unified-minterms-set-1 (map function-unify-minterm= s-list
>>>> minterms-set)) ;;(par-map function-unify-minterms-list min= terms-set))
>>> The definition of 'function-unify-minterms-list' and &= #39;minterms-set' is
>>> missing.=C2=A0 Without a test case, we can only speculate what= 's going on.
>>> (E.g., maybe it grabs a mutex).
>>>
>>> Greetings,
>>> Maxime.
I don't want to scare anyone, just maybe warn about parallel map. I onc= e tried
to use Guile's parallel map function for a decision tree implementation=
(https://notabug.org/ZelphirKaltstahl/guile-ml/src/cf666801fea91c9f= a8fa290764ff6c60b7f3949d/decision-tree.scm),
where each branch while learning the tree would call parallel map again for= sub
branches and so on. Somehow it made Guile crash (I don't have the error= message
any longer, but I did post about it on the mailing list back then.). I neve= r
figured out, what went wrong. All I had was pure function calls and math in= side
the thing that parallel map was supposed to run.

Ultimately I simply tried other parallelism constructs and when I switched = to
using futures instead, everything worked fine, no crashes, no errors.

Since that time, I did not use parallel map and instead used futures. Recen= tly I
made a parallelization thing for solving exercises of Project Euler using <= br> multiple cores, so that some solutions are calculated faster. Maybe this ca= n
help or can be adapted to another use case:

https://notabug.org/ZelphirKaltstahl= /guile-project-euler-solutions/src/ebb19b11b465903105924adb6252f1e2ecf63859= /lib/parallelism.scm#L11-L30

It expects ranges of things, which are called `segments` in the code. Usual= ly
ranges of numbers for Project Euler things. Here is the code to split a ran= ge
into segments:

https://notabug.org/ZelphirKaltstahl/guile-proje= ct-euler-solutions/src/ebb19b11b465903105924adb6252f1e2ecf63859/lib/segment= .scm

(Check any solution using it for an example.)

So this might be a bit too specific for general parallel things, but I gues= s one
could change the way futures are used in `run-in-parallel`, to fit any othe= r
purpose.

Best regards,
Zelphir

--
repositories: https://notabug.org/ZelphirKaltstahl


--000000000000e5c36705ebd84af2--