From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Damien Mattei Newsgroups: gmane.lisp.guile.user,gmane.lisp.guile.devel Subject: Re: map-par slower than map Date: Thu, 13 Oct 2022 09:40:28 +0200 Message-ID: References: <87bkqg7lmp.fsf@laura> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="1899"; mail-complaints-to="usenet@ciao.gmane.io" Cc: guile-user , guile-devel Original-X-From: guile-user-bounces+guile-user=m.gmane-mx.org@gnu.org Thu Oct 13 10:13:26 2022 Return-path: Envelope-to: guile-user@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1oitLK-0000Jh-Ot for guile-user@m.gmane-mx.org; Thu, 13 Oct 2022 10:13:26 +0200 Original-Received: from localhost ([::1]:37032 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oitLJ-0003lF-MM for guile-user@m.gmane-mx.org; Thu, 13 Oct 2022 04:13:25 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:54156) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oisph-00084x-Az for guile-user@gnu.org; Thu, 13 Oct 2022 03:40:45 -0400 Original-Received: from mail-ed1-x532.google.com ([2a00:1450:4864:20::532]:45770) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oispe-0005ab-Sy for guile-user@gnu.org; Thu, 13 Oct 2022 03:40:45 -0400 Original-Received: by mail-ed1-x532.google.com with SMTP id a67so1396218edf.12 for ; Thu, 13 Oct 2022 00:40:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=lhu2nQat5GcrXIpRNyv+WAXoJIQV2qP7uDbRGtU7txk=; b=UzkVwHXzKfDEOh2xcTEEONCcklue0wwWmbX2TNTrPTuFvyP8uOyvBsNWfSJocXlYKZ 3mIivxwXTmPNWd/Hnpb0BU0mE3xDA5NNI8cl4TN4IN4dIYAeXGG6tJlTjnc8QDzpxRpe gkYRBj8fZ6GaX059YABO0rGcZyHCFeSRZMVWH1L3SoEyIYpYqQTrDQWVUU93CFEjl35d PD37ZoT0f3vj1oCOiBD88uDhi4SSLS4MiZZCX4uQ9p2GX1XgQZA6KeGGDXdNBLdGYMI7 wYvBd/LzRHTS0RKfdxCuQ2kEVII883AnuItEYCOkEd7Bt2ttHRwsiZ91AjoU1K0WAWOZ L2qg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=lhu2nQat5GcrXIpRNyv+WAXoJIQV2qP7uDbRGtU7txk=; b=wBeUeZYhXgwkS9wEG8ICcP7qnXtkhx3dvaqWm4l13jZymyxLY8YnlhxiKnu+CYDAxr d7wJB7bVp0OkSBkzDRwl5h4Ida7YGuezZck+JPkehniFhn0uEYWPB1w094JR++Z4wlrI do7pui8Nen6jAbjfFGcHW4kuRVxM35ZSDXU90/Yl537Pr26Bt0xHdyZbXskLQ60L9qns S8bbaqFSCOJ9Exmdhaa6CYllmuGIHQZuZioPm5AMQtVXI3+XN08EO7qMFE+Xx3btYVfq 9NLYx8oRFhlDF1GXWGpNB20BvOaF93lKMqcvE/NPMIRJ75SCalqiaJuB9qFNMVQc1TMm bfJg== X-Gm-Message-State: ACrzQf2LmxzCM8//H7s+Ll4v3Hneti0bhSVrhE1YeMHPokB3hn8IEPoF MiR7bzWXEx4dbyxv6MCvrac+ev0OEYiI4xG1Uue2kFS2 X-Received: by 2002:a05:6402:524d:b0:459:3619:9cfa with SMTP id t13-20020a056402524d00b0045936199cfamt20249442edd.227.1665646839572; Thu, 13 Oct 2022 00:40:39 -0700 (PDT) In-Reply-To: <87bkqg7lmp.fsf@laura> Received-SPF: pass client-ip=2a00:1450:4864:20::532; envelope-from=damien.mattei@gmail.com; helo=mail-ed1-x532.google.com X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, MISSING_HEADERS=1.021, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane-mx.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.io gmane.lisp.guile.user:18632 gmane.lisp.guile.devel:21420 Archived-At: ok , i think the problem comes both from my code and from guile parmap so. Obviously parmap could be slower on other codes because of the nature of list i think, it is hard to split a list in sublist and send them to thread and after redo a single list, i better use vector. As mentioned and forget by me, i apologize, i use an hash table which is a global variable and mutex can be set on it , no dead lock , here but it slow down the code than it is dead for speed , but not locked. The examples given are good but i do not want to write long and specific code for //, for // must a ssimple as OpenMP directives (on arrays) not be a pain in the ass like things i already did at work with GPUs in C or Fortran,that's nightmare and i do not want to have this in functional programming. ok i will try to modify the code but i do not think there is easy solution to replace the hash table, any structure i would use would be global and the problem (this function is not pure, it does a side-effect),at the end i do not think this code could be // but i'm not a specialist of //. I think this is a parallelization problem, how can we deal with a global variable such as an hash table? On Wed, Oct 12, 2022 at 11:55 PM Olivier Dion wrote: > On Wed, 12 Oct 2022, Damien Mattei wrote: > > Hello, > > all is in the title, i test on a approximately 30000 element list , i got > > 9s with map and 3min 30s with par-map on exactly the same piece of > > code!? > > I can only speculate here. But trying with a very simple example here: > --8<---------------cut here---------------start------------->8--- > (use-modules (statprof)) > (statprof (lambda () (par-map 1+ (iota 300000)))) > --8<---------------cut here---------------end--------------->8--- > > Performance are terrible. I don't know how par-map is implemented, but > if it does 1 element static scheduling -- which it probably does because > you pass a linked list and not a vector -- then yeah you can assure that > thing will be very slow. > > You're probably better off with dynamic scheduling with vectors. Here's > a quick snippet I made for static scheduling but with vectors. Feel > free to roll your own. > > --8<---------------cut here---------------start------------->8--- > (use-modules > (srfi srfi-1) > (ice-9 threads)) > > (define* (par-map-vector proc input > #:optional > (max-thread (current-processor-count))) > > (let* ((block-size (quotient (vector-length input) max-thread)) > (rest (remainder (vector-length input) max-thread)) > (output (make-vector (vector-length input) #f))) > (when (not (zero? block-size)) > (let ((mtx (make-mutex)) > (cnd (make-condition-variable)) > (n 0)) > (fold > (lambda (scale output) > (begin-thread > (let lp ((i 0)) > (when (< i block-size) > (let ((i (+ i (* scale block-size)))) > (vector-set! output i (proc (vector-ref input i)))) > (lp (1+ i)))) > (with-mutex mtx > (set! n (1+ n)) > (signal-condition-variable cnd))) > output) > output > (iota max-thread)) > (with-mutex mtx > (while (not (< n max-thread)) > (wait-condition-variable cnd mtx)))) > (let ((base (- (vector-length input) rest))) > (let lp ((i 0)) > (when (< i rest) > (let ((i (+ i base))) > (vector-set! output i (proc (vector-ref input i)))) > (lp (1+ i))))) > output)) > --8<---------------cut here---------------end--------------->8--- > > -- > Olivier Dion > oldiob.dev >