From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Damien Mattei Newsgroups: gmane.lisp.guile.user,gmane.lisp.guile.devel Subject: Re: map-par slower than map Date: Fri, 14 Oct 2022 10:21:07 +0200 Message-ID: References: <5608809c-89a2-118c-5c05-c46ac3a0e21b@posteo.de> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="18006"; mail-complaints-to="usenet@ciao.gmane.io" Cc: guile-user , guile-devel To: Zelphir Kaltstahl Original-X-From: guile-user-bounces+guile-user=m.gmane-mx.org@gnu.org Fri Oct 14 10:22:46 2022 Return-path: Envelope-to: guile-user@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ojFxu-0004Wj-9k for guile-user@m.gmane-mx.org; Fri, 14 Oct 2022 10:22:46 +0200 Original-Received: from localhost ([::1]:55020 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ojFxt-0000zs-5I for guile-user@m.gmane-mx.org; Fri, 14 Oct 2022 04:22:45 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:33858) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ojFwh-0000xK-Kj; Fri, 14 Oct 2022 04:21:31 -0400 Original-Received: from mail-ej1-x632.google.com ([2a00:1450:4864:20::632]:37447) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ojFwa-00053l-MD; Fri, 14 Oct 2022 04:21:31 -0400 Original-Received: by mail-ej1-x632.google.com with SMTP id a26so8911194ejc.4; Fri, 14 Oct 2022 01:21:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=sKcAHvIo5nwpZtp2u2G10Wv0fbX0orZLu1KpZfKQbME=; b=NJ+x/FHtADuceSlDrSA59VKta7XLzeVS0SHor7y008WzIU7KYa1eWng3GJ4VMig20i 9MNUwBdTaNhjTFyydTu61j2WMXrh+aKrp4sqv58LAx/9KjI2tcrXeo8KKDbilSWBo1F4 Iy2m4H2oeNHHxSD4jOSoznQTCNLvkNph7B8YKGR7vq2Y+a3zK59KZyFkrvkZ10N6agOt stI4UKR0n6nrJdDx9Xte+EF3AT8/XaQl3s9MO35YRnDPmmd8zsDciKtEKNuRLEB8VHCj WHVyuzCyDurz8pKqAcAcP0dmeDTvs6FYSxj/e37YPmFCUSVNwKUSqFaEkDMTxQCdEoxz pKuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=sKcAHvIo5nwpZtp2u2G10Wv0fbX0orZLu1KpZfKQbME=; b=X92SUx7C9wLBxTp+1B1a1CsuZRmkqK92PXOSUgKCnuEpz15u0QyhkVwSLfV1S0u0l0 TTqe9DtwXo4pLjXi9QO8EZbNtcwzst2wfTpQFUI62WvH88ZG9Al/1OXgG7AZfNUeHufw Vvc8TKcSJ0zxma4r1ynrD5f0KMqVjnhCEJjnUSW/Qr0etjf8w7qOOeKGMFVBUntxy5DX odtTIi4ldvQ+3xgppQnPWQrHRygWH94np3d/msal7I51fzCzQ33DK+3SJRlprF9G86nK PRplw9TgmY6lw29lllxvecc3fqGmImn9NM2f2DcdBhbWqW83iIvaLVg6mpGKzXhvZgK3 K6fw== X-Gm-Message-State: ACrzQf1P7YG49Ik4i210yfmMnri93ADIi2aPTSnCkqTaqPIk/L0Q5k1p 4IVCWinPDs3B+vBBXVseQQrJa3p6FN53zFTPQHI= X-Google-Smtp-Source: AMsMyM7tk9eG0QfXQgyvXaM8KTGYjWXttMhAqZrwDcHyyKVfnbrD6TFxPi3XCuDvOZJTyhJNpcLzMKKf5gygwk12R/k= X-Received: by 2002:a17:906:5d16:b0:787:a9ee:3c9c with SMTP id g22-20020a1709065d1600b00787a9ee3c9cmr2707645ejt.467.1665735679131; Fri, 14 Oct 2022 01:21:19 -0700 (PDT) In-Reply-To: <5608809c-89a2-118c-5c05-c46ac3a0e21b@posteo.de> Received-SPF: pass client-ip=2a00:1450:4864:20::632; envelope-from=damien.mattei@gmail.com; helo=mail-ej1-x632.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane-mx.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.io gmane.lisp.guile.user:18647 gmane.lisp.guile.devel:21435 Archived-At: yes Zelphir i think there is a problem in the threading part of guile, whatever the code is ,it run well the first time, and after is unstable. Long ago at the very begin of Java language on my master project at university i experienced same problem with Java "green" threads, under windows and/or linux ,i can't remember. I'm trying your code and future, which have perheaps also the portability with other schemes, future can provide "light" // , with carefull code it could be ok. in your segment.scm code ,is segment-count possibly replaced by the number of available CPUs or nodes, if i understand well? Regards, Damien On Wed, Oct 12, 2022 at 11:29 PM Zelphir Kaltstahl < zelphirkaltstahl@posteo.de> wrote: > Hi! > > On 10/12/22 22:27, Damien Mattei wrote: > > > https://github.com/damien-mattei/library-FunctProg/blob/master/guile/logi= ki%2B.scm#L1674 > > > > i commited the current version of code here with all files but it is > > huge.... :-/ > > > > On Wed, Oct 12, 2022 at 10:20 PM Damien Mattei > > wrote: > > > >> Mutex? i do not think code has situation where dead lock could happen, > it > >> is a code about minimalising logic expressions, it uses minterms , > minterms > >> set is a set of minterms :like this: > >> > >> example: > >> ((1 1 0) (1 1 1)) will be unified : (1 1 x) > >> because 0 and 1 are replaced by x > >> the minterms-set could have thousands of pair (mathematic not lisp) > >> minterms to unify > >> if there is more than one x as result there is no need to continue so = i > >> escape with a continuation: > >> > >> minterms-set =3D > >> { > >> ((1 0 1 0) (1 1 1 0)) > >> ((1 0 1 0) (1 1 0 1)) > >> ((1 0 1 0) (1 0 1 1)) > >> ((1 0 1 0) (0 1 1 1)) > >> ((0 1 1 0) (1 1 1 0)) > >> ((0 1 1 0) (1 1 0 1)) > >> ((0 1 1 0) (1 0 1 1)) > >> ((0 1 1 0) (0 1 1 1)) > >> ((0 1 0 1) (1 1 1 0)) > >> ((0 1 0 1) (1 1 0 1)) > >> ((0 1 0 1) (1 0 1 1)) > >> ((0 1 0 1) (0 1 1 1)) > >> ((0 0 1 1) (1 1 1 0)) > >> ((0 0 1 1) (1 1 0 1)) > >> ((0 0 1 1) (1 0 1 1)) > >> ((0 0 1 1) (0 1 1 1)) > >> } > >> > >> replace { } by () to have the list, other example at another level : > >> > >> minterms-set =3D > >> { > >> ((0 x 1 1) (x 1 1 1)) > >> ((0 x 1 1) (1 x 1 1)) > >> ((0 x 1 1) (1 1 x 1)) > >> ((0 x 1 1) (1 1 1 x)) > >> ((x 0 1 1) (x 1 1 1)) > >> ((x 0 1 1) (1 x 1 1)) > >> ((x 0 1 1) (1 1 x 1)) > >> ((x 0 1 1) (1 1 1 x)) > >> ((0 1 x 1) (x 1 1 1)) > >> ((0 1 x 1) (1 x 1 1)) > >> ((0 1 x 1) (1 1 x 1)) > >> ((0 1 x 1) (1 1 1 x)) > >> ((x 1 0 1) (x 1 1 1)) > >> ((x 1 0 1) (1 x 1 1)) > >> ((x 1 0 1) (1 1 x 1)) > >> ((x 1 0 1) (1 1 1 x)) > >> ((0 1 1 x) (x 1 1 1)) > >> ((0 1 1 x) (1 x 1 1)) > >> ((0 1 1 x) (1 1 x 1)) > >> ((0 1 1 x) (1 1 1 x)) > >> ((x 1 1 0) (x 1 1 1)) > >> ((x 1 1 0) (1 x 1 1)) > >> ((x 1 1 0) (1 1 x 1)) > >> ((x 1 1 0) (1 1 1 x)) > >> ((1 0 1 x) (x 1 1 1)) > >> ((1 0 1 x) (1 x 1 1)) > >> ((1 0 1 x) (1 1 x 1)) > >> ((1 0 1 x) (1 1 1 x)) > >> ((1 x 1 0) (x 1 1 1)) > >> ((1 x 1 0) (1 x 1 1)) > >> ((1 x 1 0) (1 1 x 1)) > >> ((1 x 1 0) (1 1 1 x)) > >> } > >> > >> here we see some minterms are already unified > >> > >> it is not easy to read even by me because i wrote the code many year= s > ago > >> and is split in many files, but here it is: > >> > >> (par-map function-unify-minterms-list minterms-set) > >> > >> {function-unify-minterms-list <+ (=CE=BB (L) (apply > >> function-unify-two-minterms-and-tag L))} > >> > >> (define (unify-two-minterms mt1 mt2) > >> (function-map-with-escaping-by-kontinuation2 > >> (macro-function-compare-2-bits-with-continuation) mt1 mt2)) > >> > >> ;; (function-map-with-escaping-by-kontinuation2 > >> (macro-function-compare-2-bits-with-continuation) '(1 1 0 1 0 1 1 0) > '(1 > >> 1 0 1 1 1 1 1)) > >> > >> ;; list1 =3D (1 1 0 1 0 1 1 0) > >> ;; more-lists =3D ((1 1 0 1 1 1 1 1)) > >> ;; lists =3D ((1 1 0 1 0 1 1 0) (1 1 0 1 1 1 1 1)) > >> ;; clozure =3D # > >> > >> ;; #f > >> ;; > >> ;; (function-map-with-escaping-by-kontinuation2 > >> (macro-function-compare-2-bits-with-continuation) '(1 1 0 1 0 1 1 0= ) > '(1 > >> 1 0 1 1 1 1 0)) > >> > >> ;; list1 =3D (1 1 0 1 0 1 1 0) > >> ;; more-lists =3D ((1 1 0 1 1 1 1 0)) > >> ;; lists =3D ((1 1 0 1 0 1 1 0) (1 1 0 1 1 1 1 0)) > >> ;; clozure =3D # > >> > >> ;; '(1 1 0 1 x 1 1 0) > >> (define (function-map-with-escaping-by-kontinuation2 clozure list1 . > >> more-lists) > >> (call/cc (lambda (kontinuation) > >> (let ((lists (cons list1 more-lists)) > >> (funct-continu ;; this function have the kontinuation in his > environment > >> (lambda (arg1 . more-args) > >> (let ((args (cons arg1 more-args))) > >> (apply clozure kontinuation args))))) ;; a tester: (apply clozure (con= s > >> conti args)) > >> > >> ;; (newline) > >> ;; (dv list1) > >> ;; (dv more-lists) > >> ;; (dv lists) > >> ;; (dv clozure) > >> ;; (newline) > >> > >> (apply map funct-continu lists))))) > >> > >> (define-syntax macro-function-compare-2-bits-with-continuation ;; > >> continuation version of macro-compare-2-bits > >> ;; i need a macro because of external function to the clozure > >> (syntax-rules () > >> ((_) (let ((cnt 0)) ;; counter > >> (lambda (continuation b1 b2) (if (equal? b1 b2) > >> b1 > >> (begin > >> (set! cnt (add1 cnt)) ;; we leave with continuation in case cpt > > 1, we > >> can have used a flag too instead of a counter > >> (when (> cnt 1) (continuation #f)) ;; escaping with the continuati= on > >> 'x))))))) ;; return x in case of (b1,b2) =3D (O,1) or (1,0) > >> > >> what could have caused mutex if in the latter definition above (let > ((cnt > >> 0)) ;; counter was defined at top level and shared by all threads!!! y= es > >> there could have be some mutex but this is not the case, i think even > all > >> function are pure so why is it more slow with // than without? > >> Damien > >> > >> On Wed, Oct 12, 2022 at 8:45 PM Maxime Devos > >> wrote: > >> > >>> On 12-10-2022 19:19, Damien Mattei wrote: > >>>> Hello, > >>>> all is in the title, i test on a approximately 30000 element list , = i > >>> got > >>>> 9s with map and 3min 30s with par-map on exactly the same piece of > >>> code!? > >>> > [...] > >>> > > >>>> translated from Scheme+ to Scheme: > >>>> (define unified-minterms-set-1 (map function-unify-minterms-list > >>>> minterms-set)) ;;(par-map function-unify-minterms-list minterms-set)= ) > >>> The definition of 'function-unify-minterms-list' and 'minterms-set' i= s > >>> missing. Without a test case, we can only speculate what's going on. > >>> (E.g., maybe it grabs a mutex). > >>> > >>> Greetings, > >>> Maxime. > I don't want to scare anyone, just maybe warn about parallel map. I once > tried > to use Guile's parallel map function for a decision tree implementation > ( > https://notabug.org/ZelphirKaltstahl/guile-ml/src/cf666801fea91c9fa8fa290= 764ff6c60b7f3949d/decision-tree.scm), > > where each branch while learning the tree would call parallel map again > for sub > branches and so on. Somehow it made Guile crash (I don't have the error > message > any longer, but I did post about it on the mailing list back then.). I > never > figured out, what went wrong. All I had was pure function calls and math > inside > the thing that parallel map was supposed to run. > > Ultimately I simply tried other parallelism constructs and when I switche= d > to > using futures instead, everything worked fine, no crashes, no errors. > > Since that time, I did not use parallel map and instead used futures. > Recently I > made a parallelization thing for solving exercises of Project Euler using > multiple cores, so that some solutions are calculated faster. Maybe this > can > help or can be adapted to another use case: > > > https://notabug.org/ZelphirKaltstahl/guile-project-euler-solutions/src/eb= b19b11b465903105924adb6252f1e2ecf63859/lib/parallelism.scm#L11-L30 > > It expects ranges of things, which are called `segments` in the code. > Usually > ranges of numbers for Project Euler things. Here is the code to split a > range > into segments: > > > https://notabug.org/ZelphirKaltstahl/guile-project-euler-solutions/src/eb= b19b11b465903105924adb6252f1e2ecf63859/lib/segment.scm > > (Check any solution using it for an example.) > > So this might be a bit too specific for general parallel things, but I > guess one > could change the way futures are used in `run-in-parallel`, to fit any > other > purpose. > > Best regards, > Zelphir > > -- > repositories: https://notabug.org/ZelphirKaltstahl > >