From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Damien Mattei Newsgroups: gmane.lisp.guile.devel,gmane.lisp.guile.user Subject: Re: map-par slower than map Date: Sun, 23 Oct 2022 03:06:01 +0200 Message-ID: References: <5608809c-89a2-118c-5c05-c46ac3a0e21b@posteo.de> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000f2a75005eba9475c" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="21748"; mail-complaints-to="usenet@ciao.gmane.io" Cc: guile-user , guile-devel To: Zelphir Kaltstahl Original-X-From: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Mon Oct 24 03:10:51 2022 Return-path: Envelope-to: guile-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1omlzO-0005VF-GP for guile-devel@m.gmane-mx.org; Mon, 24 Oct 2022 03:10:51 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1omk2G-0003vN-Tf for guile-devel@m.gmane-mx.org; Sun, 23 Oct 2022 19:05:40 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1omPRX-0007UQ-OW; Sat, 22 Oct 2022 21:06:23 -0400 Original-Received: from mail-ed1-x52e.google.com ([2a00:1450:4864:20::52e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1omPRT-0008Gs-Ma; Sat, 22 Oct 2022 21:06:23 -0400 Original-Received: by mail-ed1-x52e.google.com with SMTP id m15so19102174edb.13; Sat, 22 Oct 2022 18:06:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=xriKesRNV2aCBoA+kmS9Gt3D97Gf59XqsMqjvChe5Aw=; b=m99/i6vqrxBlfmDCmOHmE14+pD+Cbk+/8Q/aAxEz/6DdQXTtpIoIm+21UlX3hjT8tA aKpkfLr9PSVWPMhAXuR9wgHpl5pEdlZKef8HL9rLPCNEabfUygfEgu5ZwRHo9eZj/y60 Uq6F2WQiCZQ6peyNUSC58fsccwo6chALUqP5QccaWYaqeoUP80pm3JAFrXOpoWNnd/KS Fvru1Vgrta42CLdM6tjDpSN3VJzKB6SkYLwjSoLW13aizn/YCPwBpfxTAaKhbohY+/O8 JilyGGSqZOBxUN/16RjgZw0hC+eKoXkNZp1XADhrW+jTS98Z5iG8hWDJ0pvB75DO+Qb9 YDhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=xriKesRNV2aCBoA+kmS9Gt3D97Gf59XqsMqjvChe5Aw=; b=3LpWpWzDGfScg9FcqfBTZv2IZbwQnwDaiNos0HXAvc6hofsVRZFtMAJNc2CaqC+v0p UwB+AoLaVbyxIIRBY0sO1s1fxaxWWD6VmoHNPMQBXVs/texlMYe90p9Ow21maOhkOne7 4Pnk6x+AKvNzuQzYzD8KUH0odF1Oj89yPolorq2jsYkLl1SCxyTQaIElvThA7edjpzQ7 YKz5tD//62KJ+1NAoDFKVHAQNb8iuMMlQS5+V5oN+FML77mIdxo9n40q14QIlkPTIV76 IHe3MIlGYewApfiC4f5kyXMwX4wucj+9c4cnDjDU/xkfv9BzeHwsraP4knA5qpDOtDKP L5yw== X-Gm-Message-State: ACrzQf2uUPTzylCW3R3gM6kpqcsM2gbPZEEoa2rSNtbkeYUCjZWHPa5p s/OfplSF1FVPSNMxY/XZcqrYFIsbYiAU8Ct5nTw= X-Google-Smtp-Source: AMsMyM5W3QsYsAoTOHu6WwoKF9IxX0HvnCzSLxEMy9RLy3P8Bu8U98YvoXIIvzGDxyPzBgq07pMf9YQKqIKV17bcQxo= X-Received: by 2002:a17:907:2d91:b0:78d:8747:71b4 with SMTP id gt17-20020a1709072d9100b0078d874771b4mr21779914ejc.545.1666487172637; Sat, 22 Oct 2022 18:06:12 -0700 (PDT) In-Reply-To: Received-SPF: pass client-ip=2a00:1450:4864:20::52e; envelope-from=damien.mattei@gmail.com; helo=mail-ed1-x52e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane-mx.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.io gmane.lisp.guile.devel:21451 gmane.lisp.guile.user:18670 Archived-At: --000000000000f2a75005eba9475c Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable after intense coding i finally got the good results, my assumption about the global variable hash table was true ,it is incompatible with 'future : the competition for the writing into the hash table breaks the code. If i do writing in hash table out of // region all is ok: a simple function to extract the hash table access: (define (tag-minterms i umt) (when umt {mt1 <+ (first {minterms-vector[i]})} {mt2 <+ (second {minterms-vector[i]})} {minterms-ht[mt1] <- #t} {minterms-ht[mt2] <- #t})) ;; proc to be called with futures (define (proc-unify-minterms-seg seg) {function-unify-minterms-list <+ (=CE=BB (L) (apply function-unify-two-minterms L))} ;;{function-unify-minterms-list <+ (=CE=BB (L) (apply function-unify-two-minterms-and-tag L))} {start <+ (segment-start seg)} {end <+ (segment-end seg)} (for ({i <+ start} {i <=3D end} {i <- {i + 1}}) {mtL <+ {minterms-vector[i]}} (nodebug (dv mtL)) {unified-minterms-vector-1[i] <- (function-unify-minterms-list mtL)})) (declare minterms-vector unified-minterms-vector-1) https://github.com/damien-mattei/library-FunctProg/blob/master/guile/logiki= %2B.scm#L2893 and the call to the code that use hash table is now completely out of the // region: (debug (display-nl "before //")) (run-in-parallel segmts proc-unify-minterms-seg) ;; run the parallel code (debug (display-nl "after //")) (vector-for-each tag-minterms unified-minterms-vector-1) ;; tag the minterms in the hash table with a partitioned vector it works because one cell is not accessed by different threads other stuff like continuation seems to work with 'future, i wanted to rewrite code with call/cc in 'future but it was too long and apparently it works,any way i do not see why call/cc would cause trouble in a thread... i think 'futures are good if you use them carefully like in OpenMP regions... in other languages. the code is very very fast now i reached C15,in a few minutes on an Apple M= 1 scheme@(guile-user)> (current-processor-count) $1 =3D 8 C15 =3D ((B1 =E2=88=A7 B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B7 =E2=88=A7 = B9 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B13) =E2=88=A8 (B1 =E2=88=A7 B3 = =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88= =A8 (B1 =E2=88=A7 B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B7 =E2=88=A7 B9 = =E2=88=A7 B11 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B7 =E2= =88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7= B4 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B1 =E2= =88=A7 B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B9 =E2=88=A7 B11= =E2=88=A7 B13) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B11 =E2= =88=A7 B13) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 = B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B13) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88= =A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B1 =E2=88=A7 B= 3 =E2=88=A7 B5 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B1 = =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B9 =E2=88=A7 = B11 =E2=88=A7 B13) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B10 =E2= =88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7= B6 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B13) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88= =A7 B6 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B= 1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B1 = =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7= B12 =E2=88=A7 B13) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B8 =E2=88=A7 B10 =E2= =88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7= B7 =E2=88=A7 B9 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B13) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88= =A7 B7 =E2=88=A7 B9 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B= 1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B1 = =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7= B13) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88= =A7 B12 =E2=88=A7 B14) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B6= =E2=88=A7 B8 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B6 =E2= =88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88= =A7 B3 =E2=88=A7 B5 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B13) =E2=88=A8 (B2 = =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7= B12 =E2=88=A7 B14) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B8 =E2=88=A7 B10 =E2= =88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7= B7 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88= =A7 B7 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B= 2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B13) =E2=88=A8 (B2 = =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B10 =E2=88=A7= B12 =E2=88=A7 B14) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B11 =E2= =88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7= B7 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88= =A7 B8 =E2=88=A7 B10 =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B= 4 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B4 = =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88= =A7 B14) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B10 =E2=88=A7 B12 =E2= =88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 = B9 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88= =A7 B9 =E2=88=A7 B11 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B2 =E2=88=A7 B= 4 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B6 =E2= =88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B13) =E2=88=A8= (B2 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B14) =E2= =88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B1= 1 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B7 =E2=88=A7 B9 =E2= =88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7= B8 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88= =A7 B8 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B4= =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B6 = =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B13) =E2=88=A8 (B2 =E2= =88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B14)) and it consumes more than 2Gb in memory: PID COMMAND %CPU TIME #TH #WQ #PORT MEM PURG CMPRS PGRP PPID STATE BOOSTS %CPU_ME 15081 guile 442.8 57:24.28 17/8 0 44 *2423M* 0B 118M 15081 10551 running *0[1] 0.0000 i checked a part of the logical computation with Mathematica online and it was ok but i will do a Python program to check the logical expression again and compare speed with Sympy... and for the space complexity i do not think it can be reduced good night all On Sat, Oct 22, 2022 at 6:01 PM Damien Mattei wrote: > just for the fun things, i just find that the parallelized results are > false :-) even for low indices: > the non // result: C5 =3D ((B1 =E2=88=A7 B3) =E2=88=A8 (B2 =E2=88=A7 B3) = =E2=88=A8 (B2 =E2=88=A7 B4)) > is different than the parallelized result below: > C5 =3D ((B1 =E2=88=A7 B3) =E2=88=A8 (B2 =E2=88=A7 B3) =E2=88=A8 (B2 =E2= =88=A7 B4) =E2=88=A8 (B3 =E2=88=A7 B4)) > > i have to check things again ,i think futures are easy to use but limited > in solving acute // problems, such as using a global variable like an has= h > table... > > Damien > > > > On Mon, Oct 17, 2022 at 3:17 PM Damien Mattei > wrote: > >> Hello, >> >> sorry for my late answer ( i wrote the code friday but i could only debu= g >> today, being busy and on travel last week-end) >> >> i modified my code to works with 'futures' , the speed is now incredible >> !!! (and it computes exact) >> the speed seems multiplied by even more than the number of CPUs (6): >> cat /proc/cpuinfo | grep processor >> processor : 0 >> processor : 1 >> processor : 2 >> processor : 3 >> processor : 4 >> processor : 5 >> >> >> https://github.com/damien-mattei/library-FunctProg/blob/master/guile/log= iki%2B.scm#L1900 >> >> (declare minterms-vector unified-minterms-vector-1) >> >> (define (funct-unify-minterms-set-1-unit-future set1 set2) >> >> {function-unify-minterms-list <+ (=CE=BB (L) (apply >> function-unify-two-minterms-and-tag L))} >> >> ;; note : sorting is useless >> >> {minterms-set <+ (product-set-with-set-imperative-sorted set1 set2)} >> ;;(product-set-with-set-imperative set1 set2)} ;;(product-set-with-set s= et1 >> set2)} ;;(associate-set-with-set set1 set2)} ;; set multiplication : cre= ate >> list of pair of minterms >> >> {minterms-vector <- (list->vector minterms-set)} >> >> {minterms-vector-length <+ (vector-length minterms-vector)} >> >> {nb-procs <+ (current-processor-count)} >> >> {segmts <+ (segment 0 {minterms-vector-length - 1} nb-procs)} ;; >> compute the segments >> >> {unified-minterms-vector-1 <- (make-vector minterms-vector-length #f)} >> >> (run-in-parallel segmts proc-unify-minterms-seg) ;; run the parallel >> code >> >> {unified-minterms-set-1 <+ (vector->list unified-minterms-vector-1)} >> >> {unified-minterms-set-2 <+ (filter (=CE=BB (x) x) unified-minterms-set= -1)} >> ;; remove #f results >> >> {unified-minterms-set <+ (remove-duplicates-sorted >> unified-minterms-set-2)} ;; uniq >> >> unified-minterms-set) >> >> i keeped your 'segment' definitions to index an array of data after >> convert it from list to vector (list->vector) which probably consume >> additional time on big data list ( i had more than 1000000 element list >> lengths at some point) >> >> i used a simplified version of run-in parallel (that do not do 'reduce >> after processing data): >> >> >> https://github.com/damien-mattei/library-FunctProg/blob/master/guile/log= iki%2B.scm#L1794 >> >> the computation on a segment is done by those functions: >> >> >> https://github.com/damien-mattei/library-FunctProg/blob/master/guile/log= iki%2B.scm#L1842 >> >> {function-unify-minterms-list <+ (=CE=BB (L) (apply >> function-unify-two-minterms-and-tag L))} >> >> ;; proc to be called with futures >> (define (proc-unify-minterms-seg seg) >> {start <+ (segment-start seg)} >> {end <+ (segment-end seg)} >> (for ({i <+ start} {i <=3D end} {i <- {i + 1}}) >> {mtL <+ {minterms-vector[i]}} >> {unified-minterms-vector-1[i] <- (function-unify-minterms-list >> mtL)})) >> >> >> i use those code in another program symbolically to compute a value name= d >> Cn: >> >> C0 =3D T >> C1 =3D T >> C2 =3D T >> C3 =3D (B1 =E2=88=A8 B2) >> C4 =3D (B2 =E2=88=A8 (B1 =E2=88=A7 B3)) >> C5 =3D ((B1 =E2=88=A7 B3) =E2=88=A8 (B2 =E2=88=A7 B3) =E2=88=A8 (B2 =E2= =88=A7 B4) =E2=88=A8 (B3 =E2=88=A7 B4)) >> C6 =3D ((B1 =E2=88=A7 B3 =E2=88=A7 B5) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88= =A7 B5) =E2=88=A8 (B2 =E2=88=A7 B4) =E2=88=A8 (B3 =E2=88=A7 B4) =E2=88=A8 (= B4 =E2=88=A7 B5)) >> C7 =3D ((B1 =E2=88=A7 B3 =E2=88=A7 B5) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88= =A7 B5) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B6) =E2=88=A8 (B3 =E2=88=A7 B4= =E2=88=A7 B6) =E2=88=A8 >> (B4 =E2=88=A7 B5) =E2=88=A8 (B5 =E2=88=A7 B6)) >> C8 =3D ((B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7) =E2=88=A8 (B2 =E2=88= =A7 B3 =E2=88=A7 B5 =E2=88=A7 B7) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B6) = =E2=88=A8 (B3 =E2=88=A7 >> B4 =E2=88=A7 B6) =E2=88=A8 (B4 =E2=88=A7 B5 =E2=88=A7 B7) =E2=88=A8 (B5 = =E2=88=A7 B6) =E2=88=A8 (B6 =E2=88=A7 B7)) >> C9 =3D ((B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7) =E2=88=A8 (B2 =E2=88= =A7 B3 =E2=88=A7 B5 =E2=88=A7 B7) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B6 = =E2=88=A7 B8) =E2=88=A8 >> (B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8) =E2=88=A8 (B4 =E2=88=A7 B5 = =E2=88=A7 B7) =E2=88=A8 (B5 =E2=88=A7 B6 =E2=88=A7 B8) =E2=88=A8 (B6 =E2=88= =A7 B7) =E2=88=A8 (B7 =E2=88=A7 >> B8)) >> >> from C1 to C9 used to be fast,less than a minute for the whole with or >> without // >> >> C10 =3D ((B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88= =A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 (B2 = =E2=88=A7 B4 =E2=88=A7 >> B6 =E2=88=A7 B8) =E2=88=A8 (B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8) = =E2=88=A8 (B4 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 (B5 =E2=88= =A7 B6 =E2=88=A7 B8) =E2=88=A8 (B6 >> =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 (B7 =E2=88=A7 B8) =E2=88=A8 (B8 =E2= =88=A7 B9)) >> >> C10 takes a few minutes >> >> but C11 used to take one day before // with 'future' and i got now the >> result during less than one hour!!! >> >> C11 =3D ((B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88= =A8 (B10 =E2=88=A7 B2 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8) =E2=88=A8 (B1= 0 =E2=88=A7 B3 =E2=88=A7 >> B4 =E2=88=A7 B6 =E2=88=A7 B8) =E2=88=A8 (B10 =E2=88=A7 B5 =E2=88=A7 B6 = =E2=88=A7 B8) =E2=88=A8 (B10 =E2=88=A7 B7 =E2=88=A7 B8) =E2=88=A8 (B10 =E2= =88=A7 B9) =E2=88=A8 (B2 =E2=88=A7 >> B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 (B4 =E2=88=A7 B5 = =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 (B6 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88= =A8 (B8 =E2=88=A7 B9)) >> >> i never got C12 in the past ,even with 7 days of computation! and i got >> it today during the lunchbreak !!! >> >> C12 =3D ((B1 =E2=88=A7 B11 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88= =A7 B9) =E2=88=A8 (B10 =E2=88=A7 B11) =E2=88=A8 (B10 =E2=88=A7 B2 =E2=88=A7= B4 =E2=88=A7 B6 >> =E2=88=A7 B8) =E2=88=A8 (B10 =E2=88=A7 B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2= =88=A7 B8) =E2=88=A8 (B10 =E2=88=A7 B5 =E2=88=A7 B6 =E2=88=A7 B8) =E2=88=A8= (B10 =E2=88=A7 B7 =E2=88=A7 B8) >> =E2=88=A8 (B10 =E2=88=A7 B9) =E2=88=A8 (B11 =E2=88=A7 B2 =E2=88=A7 B3 = =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 (B11 =E2=88=A7 B4 =E2=88= =A7 B5 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 >> (B11 =E2=88=A7 B6 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 (B11 =E2=88=A7 B8= =E2=88=A7 B9)) >> >> (note that a wise people can find a general formula for Cn based on Cn-1 >> but that is another (mathematical) story....) >> >> i'm computing C13 but i see that it consumes 12gb out of 16gb of my >> system ! and stopped on the linux box : >> GC Warning: Repeated allocation of very large block (appr. size >> 510935040): >> May lead to memory leak and poor performance >> Processus arr=C3=AAt=C3=A9 >> but still running on the Mac laptop...ok will see that but i think that >> the data set is now huge and there is noting that can be done with that.= .. >> >> note that there is still an hash table used in >> function-unify-two-minterms-and-tag and i will use another data structur= e >> and algo to avoid that, i feared that the concurrent access to hash tabl= e >> can cause the guile 'future' to crash or fails or to slow down the proce= ss >> but not! result is incredibly fast.Also i use continuation and i read it >> can cause problem with 'future' i will improve that too... >> >> I will see where i can improve the algo and data structure to optimize >> again but this is already really good. >> >> Thanks >> >> Damien >> >> >> On Fri, Oct 14, 2022 at 10:39 AM Zelphir Kaltstahl < >> zelphirkaltstahl@posteo.de> wrote: >> >>> Hello Damien, >>> On 10/14/22 10:21, Damien Mattei wrote: >>> >>> yes Zelphir i think there is a problem in the threading part of guile, >>> whatever the code is ,it run well the first time, and after is unstable= . >>> Long ago at the very begin of Java language on my master project at >>> university i experienced same problem with Java "green" threads, under >>> windows and/or linux ,i can't remember. >>> >>> I'm trying your code and future, which have perheaps also the >>> portability with other schemes, future can provide "light" // , with >>> carefull code it could be ok. >>> >>> in your segment.scm code ,is segment-count possibly replaced by the >>> number of available CPUs or nodes, if i understand well? >>> >>> Regards, >>> Damien >>> >>> Correct. For each segment one future is used. >>> >>> However, there are scenarios, where one would rather want to interleave >>> the numbers of the whole range, to have a "fairer" workload per future,= for >>> example: >>> >>> (1 2 3 4 5 6 7 8 9) >>> >>> -> (1 4 7), (2 5 8), (3 6 9) >>> >>> instead of >>> >>> -> (1 2 3) (4 5 6) (7 8 9) >>> >>> (I seem to remember this to be the case for Mandelbrot set calculations= , >>> but might be wrong.) >>> >>> But that is all a matter of forming some segments and what a segment is= , >>> not really part of the logic of creating futures. So one could create t= hen >>> in any way that fits ones use-case. I have not generalized that segment >>> logic that far, but it is doable. >>> >>> Anyway, if anyone more knowledgeable could chime in on what the >>> performance differences between futures and parallel map are, that woul= d be >>> great! Or pointing out some flaws that this kind of future usage might >>> have. Or when the point is reached to switch to guile-fibers library. >>> >>> Regards, >>> Zelphir >>> >>> -- >>> repositories: https://notabug.org/ZelphirKaltstahl >>> >>> --000000000000f2a75005eba9475c Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
aft= er intense coding i finally got the good results,
my assumption about the global variable = hash table was true ,it is incompatible with 'future : the competition = for the writing into the hash table breaks the code.

If i do writing in hash table out of // region = all is ok:

=
a simple funct= ion to extract the hash table access:

(define (tag-minterms i umt)
=C2=A0 (when umt
=C2=A0=C2= =A0=C2=A0=C2=A0 {mt1 <+ (first {minterms-vector[i]})}
=C2=A0=C2=A0=C2= =A0=C2=A0 {mt2 <+ (second {minterms-vector[i]})}
=C2=A0=C2=A0=C2=A0= =C2=A0 {minterms-ht[mt1] <- #t}
=C2=A0=C2=A0=C2=A0=C2=A0 {minterms-ht= [mt2] <- #t}))



;; proc to be called with f= utures
(define (proc-unify-minterms-seg seg)

=C2=A0 {function-uni= fy-minterms-list <+ (=CE=BB (L) (apply function-unify-two-minterms L))}<= br>=C2=A0 ;;{function-unify-minterms-list <+ (=CE=BB (L) (apply function= -unify-two-minterms-and-tag L))}
=C2=A0
=C2=A0
=C2=A0 {start <= ;+ (segment-start seg)}
=C2=A0 {end <+ (segment-end seg)}
=C2=A0 (= for ({i <+ start} {i <=3D end} {i <- {i + 1}})
=C2=A0 =C2=A0 = =C2=A0 =C2=A0{mtL <+ {minterms-vector[i]}}
=C2=A0 =C2=A0 =C2=A0 =C2= =A0(nodebug
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (dv m= tL))
=C2=A0 =C2=A0 =C2=A0 =C2=A0{unified-minterms-vector-1[i] <- (fun= ction-unify-minterms-list mtL)}))


(declare minterms-vector unifi= ed-minterms-vector-1)

https://github.com/damien-mattei/library-FunctProg/b= lob/master/guile/logiki%2B.scm#L2893

and the call to the code that use hash table is now complet= ely out of the // region:

=C2=A0(debug
=C2=A0 =C2=A0(display-nl "before //"))
=C2=A0=
=C2=A0 (run-in-parallel segmts proc-unify-minterms-seg) ;; run the par= allel code

=C2=A0 (debug
=C2=A0 =C2=A0(display-nl "after //&= quot;))

=C2=A0 (vector-for-each tag-minterms unified-minterms-vector= -1) ;; tag the minterms in the hash table

with a partitioned vector it works because one cell is not= accessed by different threads

other stuff like continuation seems to work with 'future, i = wanted to rewrite code with call/cc in 'future but it was too long and = apparently it works,any way i do not see why call/cc would cause trouble in= a thread...

i think= 'futures are good if you use them carefully like in OpenMP regions... = in other languages.

t= he code is very very fast now i reached C15,in a few minutes on an Apple M1=
scheme@(guile-= user)> =C2=A0(current-processor-count)
$1 =3D 8

C15 =3D ((B1 =E2=88=A7 B3 =E2=88=A7 B4 =E2=88= =A7 B6 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B13)= =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B7 =E2=88= =A7 B9 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B1 =E2=88=A7 B= 3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88= =A7 B12 =E2=88=A7 B14) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B4 =E2=88=A7 B6= =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B1 =E2= =88=A7 B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B9 =E2=88=A7 B11= =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B4 =E2= =88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 = (B1 =E2=88=A7 B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B10 =E2= =88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B4 =E2=88=A7= B6 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B13) =E2=88=A8 (B1 = =E2=88=A7 B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7= B12 =E2=88=A7 B14) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B6 = =E2=88=A7 B8 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88= =A8 (B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B9 = =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88= =A7 B6 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B= 1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88= =A7 B12 =E2=88=A7 B13) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B6= =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B1 =E2= =88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B1= 1 =E2=88=A7 B13) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2= =88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B13) =E2=88=A8 (B1 =E2=88= =A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 = =E2=88=A7 B14) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88= =A7 B9 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B13) =E2=88=A8 (B1 =E2=88=A7 B= 3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88= =A7 B14) =E2=88=A8 (B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9 = =E2=88=A7 B11 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B1 =E2=88=A7 B3 =E2= =88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 = (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B9 =E2=88= =A7 B11 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B= 5 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B13) =E2= =88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B1= 0 =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2= =88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B13) =E2=88=A8= (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B10 =E2= =88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7= B7 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B2 = =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7= B12 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 = =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B2 =E2= =88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B10 =E2=88=A7 B1= 2 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2= =88=A7 B9 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B2 =E2=88= =A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B12 = =E2=88=A7 B14) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88= =A7 B9 =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B5= =E2=88=A7 B7 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B11 =E2=88=A7 B13) =E2= =88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B8 =E2=88=A7 B1= 0 =E2=88=A7 B12 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B5 =E2= =88=A7 B7 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8= (B2 =E2=88=A7 B4 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B10 =E2= =88=A7 B12 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B5 =E2=88=A7= B7 =E2=88=A7 B9 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B2 = =E2=88=A7 B4 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7= B12 =E2=88=A7 B14) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B5 =E2=88=A7 B7 = =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88= =A7 B6 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B13)= =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88= =A7 B10 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B= 6 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B12 =E2=88=A7 B14) =E2= =88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B7 =E2=88=A7 B9 =E2=88=A7 B1= 1 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8 =E2= =88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B12 =E2=88=A7 B14) =E2=88=A8 (B2 =E2=88= =A7 B4 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B9 =E2=88=A7 B11 =E2=88=A7 B13) = =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88= =A7 B11 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8= =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B13) =E2=88=A8 (B2 =E2=88=A7 B4 =E2= =88=A7 B6 =E2=88=A7 B8 =E2=88=A7 B10 =E2=88=A7 B12 =E2=88=A7 B14))

and it consumes more than 2Gb in = memory:


PID =C2=A0 = =C2=A0COMMAND =C2=A0 =C2=A0 =C2=A0%CPU =C2=A0TIME =C2=A0 =C2=A0 #TH =C2=A0 = #WQ =C2=A0#PORT MEM =C2=A0 =C2=A0PURG =C2=A0 CMPRS =C2=A0PGRP =C2=A0PPID = =C2=A0STATE =C2=A0 =C2=A0BOOSTS =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0%CPU_ME15081 =C2=A0guile =C2=A0 =C2=A0 =C2=A0 =C2=A0442.8 57:24.28 17/8 =C2=A00 = =C2=A0 =C2=A044 =C2=A0 =C2=A02423M =C2=A00B =C2=A0 =C2=A0 118M =C2= =A0 15081 10551 running =C2=A0*0[1] =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0.00= 00

i checked a part of th= e logical computation with Mathematica online and it was ok but i will do a= Python program to check the logical expression again and compare speed wit= h Sympy...

=
and for the sp= ace complexity i do not think it can be reduced

good night all

On Sat, Oct 22, 2022 at= 6:01 PM Damien Mattei <damie= n.mattei@gmail.com> wrote:
just for the fun things, i just find that t= he parallelized results are false :-) even for low indices:
the non // result: C5 =3D ((B1= =E2=88=A7 B3) =E2=88=A8 (B2 =E2=88=A7 B3) =E2=88=A8 (B2 =E2=88=A7 B4))
is different than = the parallelized result below:
C5 =3D ((B1 =E2=88=A7 B3) =E2=88=A8 (B2 =E2=88=A7 B3) =E2= =88=A8 (B2 =E2=88=A7 B4) =E2=88=A8 (B3 =E2=88=A7 B4))

i have to check things again ,i think futures = are easy to use but limited in solving acute // problems, such as using a g= lobal variable like an hash table...

=
Dam= ien



On Mon, Oct 17, 2022 at 3:17 PM Damien Mattei <damien.mattei@gmail.com> wrot= e:
Hello,

sorry for my late answer ( = i wrote the code friday but i could only debug today, being busy and on tra= vel last week-end)

i modified my code to works with 'futures'= , the speed is now incredible !!! (and it computes exact)
the speed seems multiplied by even more than the nu= mber of CPUs (6):
cat /proc/cpuinfo =C2= =A0| grep processor
processor : 0
processor : 1
processor : 2
p= rocessor : 3
processor : 4
processor : 5

(declare minterms-vector unified-mint= erms-vector-1)

(define (funct-unify-minterms-set-1-unit-future set1 set2)=

=C2=A0 {function-unify-minterms-list <+ (=CE=BB (L) (apply funct= ion-unify-two-minterms-and-tag L))}

=C2=A0 ;; note : sorting is usel= ess

=C2=A0 {minterms-set <+ (product-set-with-set-imperative-sort= ed set1 set2)} ;;(product-set-with-set-imperative set1 set2)} ;;(product-se= t-with-set set1 set2)} ;;(associate-set-with-set set1 set2)} ;; set multipl= ication : create list of pair of minterms

=C2=A0 {minterms-vector &l= t;- (list->vector minterms-set)}

=C2=A0 {minterms-vector-length &= lt;+ (vector-length minterms-vector)}

=C2=A0 {nb-procs <+ (curren= t-processor-count)}
=C2=A0
=C2=A0 {segmts <+ (segment 0 {minterms= -vector-length - 1} nb-procs)} ;; compute the segments

=C2=A0 {unifi= ed-minterms-vector-1 <- (make-vector minterms-vector-length #f)}
=C2= =A0
=C2=A0 (run-in-parallel segmts proc-unify-minterms-seg) ;; run the p= arallel code

=C2=A0 {unified-minterms-set-1 <+ (vector->list u= nified-minterms-vector-1)}
=C2=A0
=C2=A0 {unified-minterms-set-2 <= ;+ (filter (=CE=BB (x) x) unified-minterms-set-1)} ;; remove #f results
=
=C2=A0 {unified-minterms-set <+ (remove-duplicates-sorted unified-mi= nterms-set-2)} ;; uniq
=C2=A0 =C2=A0 =C2=A0
=C2=A0 unified-minterms-= set)

i keeped your 'segment' definitions to index an array of= data after convert it from list to vector (list->vector) which probably= consume additional time on big data list ( i had more than 1000000 element= list lengths at some point)

=
i used a simplified version of run-in parall= el (that do not do 'reduce after processing data):


the computation on a segment = is done by those functions:

<= div style=3D"font-size:large">h= ttps://github.com/damien-mattei/library-FunctProg/blob/master/guile/logiki%= 2B.scm#L1842

{function-unify-minterms-list <+ (=CE=BB (L) (apply function-unify-two-minterms-and-tag L))}

;; proc to be call= ed with futures
(define (proc-unify-minterms-seg seg)
=C2=A0 {start &= lt;+ (segment-start seg)}
=C2=A0 {end <+ (segment-end seg)}
=C2=A0= (for ({i <+ start} {i <=3D end} {i <- {i + 1}})
=C2=A0 =C2=A0 = =C2=A0 =C2=A0{mtL <+ {minterms-vector[i]}}
=C2=A0 =C2=A0 =C2=A0 =C2= =A0{unified-minterms-vector-1[i] <- (function-unify-minterms-list mtL)})= )


i use those code in another pr= ogram symbolically to compute a value named Cn:

C0 =3D T
C1 =3D T
C= 2 =3D T
C3 =3D (B1 =E2=88=A8 B2)
C4 =3D (B2 =E2=88=A8 (B1 =E2=88=A7 B= 3))
C5 =3D ((B1 =E2=88=A7 B3) =E2=88=A8 (B2 =E2=88=A7 B3) =E2=88=A8 (B2 = =E2=88=A7 B4) =E2=88=A8 (B3 =E2=88=A7 B4))
C6 =3D ((B1 =E2=88=A7 B3 =E2= =88=A7 B5) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5) =E2=88=A8 (B2 =E2=88=A7= B4) =E2=88=A8 (B3 =E2=88=A7 B4) =E2=88=A8 (B4 =E2=88=A7 B5))
C7 =3D ((B= 1 =E2=88=A7 B3 =E2=88=A7 B5) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5) =E2= =88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B6) =E2=88=A8 (B3 =E2=88=A7 B4 =E2=88=A7 = B6) =E2=88=A8 (B4 =E2=88=A7 B5) =E2=88=A8 (B5 =E2=88=A7 B6))
C8 =3D ((B1= =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88= =A7 B5 =E2=88=A7 B7) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B6) =E2=88=A8 (B3= =E2=88=A7 B4 =E2=88=A7 B6) =E2=88=A8 (B4 =E2=88=A7 B5 =E2=88=A7 B7) =E2=88= =A8 (B5 =E2=88=A7 B6) =E2=88=A8 (B6 =E2=88=A7 B7))
C9 =3D ((B1 =E2=88=A7= B3 =E2=88=A7 B5 =E2=88=A7 B7) =E2=88=A8 (B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2= =88=A7 B7) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8) =E2=88=A8 = (B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8) =E2=88=A8 (B4 =E2=88=A7 B5 =E2= =88=A7 B7) =E2=88=A8 (B5 =E2=88=A7 B6 =E2=88=A7 B8) =E2=88=A8 (B6 =E2=88=A7= B7) =E2=88=A8 (B7 =E2=88=A7 B8))

<= /div>
from C1 to C9 used to be fast,less than= =C2=A0 a minute for the whole with or without //

C10 =3D ((B1 =E2=88= =A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 (B2 =E2=88=A7 B3 = =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 (B2 =E2=88=A7 B4 =E2=88= =A7 B6 =E2=88=A7 B8) =E2=88=A8 (B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8) = =E2=88=A8 (B4 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 (B5 =E2=88= =A7 B6 =E2=88=A7 B8) =E2=88=A8 (B6 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 (B7= =E2=88=A7 B8) =E2=88=A8 (B8 =E2=88=A7 B9))

C10 takes a few minutes <= br>

but C11 used to take one day before // with 'future' and i go= t now the result during less than one hour!!!

C11 =3D ((B1 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88= =A7 B9) =E2=88=A8 (B10 =E2=88=A7 B2 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8)= =E2=88=A8 (B10 =E2=88=A7 B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8) =E2=88= =A8 (B10 =E2=88=A7 B5 =E2=88=A7 B6 =E2=88=A7 B8) =E2=88=A8 (B10 =E2=88=A7 B= 7 =E2=88=A7 B8) =E2=88=A8 (B10 =E2=88=A7 B9) =E2=88=A8 (B2 =E2=88=A7 B3 =E2= =88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 (B4 =E2=88=A7 B5 =E2=88=A7 B= 7 =E2=88=A7 B9) =E2=88=A8 (B6 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 (B8 =E2= =88=A7 B9))

i never got C12 in the past ,even with 7 days of computation!= and i got it today during the lunchbreak !!!

C12 =3D ((B1 =E2=88=A7 = B11 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 (B10 =E2= =88=A7 B11) =E2=88=A8 (B10 =E2=88=A7 B2 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7= B8) =E2=88=A8 (B10 =E2=88=A7 B3 =E2=88=A7 B4 =E2=88=A7 B6 =E2=88=A7 B8) = =E2=88=A8 (B10 =E2=88=A7 B5 =E2=88=A7 B6 =E2=88=A7 B8) =E2=88=A8 (B10 =E2= =88=A7 B7 =E2=88=A7 B8) =E2=88=A8 (B10 =E2=88=A7 B9) =E2=88=A8 (B11 =E2=88= =A7 B2 =E2=88=A7 B3 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 (B11 = =E2=88=A7 B4 =E2=88=A7 B5 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 (B11 =E2=88= =A7 B6 =E2=88=A7 B7 =E2=88=A7 B9) =E2=88=A8 (B11 =E2=88=A7 B8 =E2=88=A7 B9)= )

(note that a wise people can find a general formula for Cn based on Cn-= 1 but that is another (mathematical) story....)

i'm computing C13= but i see that it consumes 12gb out of 16gb of my system ! and stopped on = the linux box :
GC Warning: Repeated al= location of very large block (appr. size 510935040):
May lead to memory= leak and poor performance
Processus arr=C3=AAt=C3=A9
but still running on the Mac laptop...ok will see that bu= t i think that the data set is now huge and there is noting that can be don= e with that...

note that there is still an hash table used in function-un= ify-two-minterms-and-tag and i will use another data structure and algo to = avoid that, i feared that the concurrent access to hash table can cause the= guile 'future' to crash or fails or to slow down the process but n= ot! result is incredibly fast.Also i use continuation and i read it can cau= se problem with 'future' i will improve that too...

I will se= e where i can improve the algo and data structure to optimize again but thi= s is already really good.

Thanks

=
Damien


On Fri, Oct 14, 2022 at 10:39 AM Zelphir Kaltstahl <= zelphirkalt= stahl@posteo.de> wrote:
=20 =20 =20

Hello Damien,

On 10/14/22 10:21, Damien Mattei wrote:
=20
yes Zelphir i think there is a problem in the threading part of guile, whatever the code is ,it run well the first time, and after is unstable. Long ago at the very begin of Java language on my master project at university i experienced same problem with Java "green" threads, under windows and/or linux ,i can= 't remember.

I'm trying your code and future, which have perheaps also the portability with other schemes, future can provide "light" // , wit= h carefull code it could be ok.

in your segment.scm code ,is segment-count possibly replaced by the number of available CPUs or nodes, if i understand well?

Regards,
Damien

Correct. For each segment one future is used.

However, there are scenarios, where one would rather want to interleave the numbers of the whole range, to have a "fairer&quo= t; workload per future, for example:

(1 2 3 4 5 6 7 8 9)

-> (1 4 7), (2 5 8), (3 6 9)

instead of

-> (1 2 3) (4 5 6) (7 8 9)

(I seem to remember this to be the case for Mandelbrot set calculations, but might be wrong.)

But that is all a matter of forming some segments and what a segment is, not really part of the logic of creating futures. So one could create then in any way that fits ones use-case. I have not generalized that segment logic that far, but it is doable.

Anyway, if anyone more knowledgeable could chime in on what the performance differences between futures and parallel map are, that would be great! Or pointing out some flaws that this kind of future usage might have. Or when the point is reached to switch to guile-fibers library.

Regards,
Zelphir

--=20
repositories: https://notabug.org/ZelphirKaltstahl
--000000000000f2a75005eba9475c--