From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Linas Vepstas Newsgroups: gmane.lisp.guile.devel Subject: Re: guile-2.9.1 impressions Date: Sun, 26 May 2019 23:40:30 -0500 Message-ID: References: <877eahotro.fsf@pobox.com> Reply-To: linasvepstas@gmail.com Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000ad24310589d72623" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="94589"; mail-complaints-to="usenet@blaine.gmane.org" Cc: Guile Development To: Andy Wingo Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Mon May 27 06:41:08 2019 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hV7Ra-000OSS-Ad for guile-devel@m.gmane.org; Mon, 27 May 2019 06:41:06 +0200 Original-Received: from localhost ([127.0.0.1]:39908 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hV7RZ-0006WY-5R for guile-devel@m.gmane.org; Mon, 27 May 2019 00:41:05 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:51578) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hV7RH-0006WT-E5 for guile-devel@gnu.org; Mon, 27 May 2019 00:40:48 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hV7RF-0005nz-Tq for guile-devel@gnu.org; Mon, 27 May 2019 00:40:47 -0400 Original-Received: from mail-lj1-x233.google.com ([2a00:1450:4864:20::233]:42278) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hV7RF-0005nQ-I4 for guile-devel@gnu.org; Mon, 27 May 2019 00:40:45 -0400 Original-Received: by mail-lj1-x233.google.com with SMTP id 188so13412464ljf.9 for ; Sun, 26 May 2019 21:40:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to:cc; bh=zp4fTgOelbZ4xInDdUcDs/dnGRl0FK2EW9il7U9SkXw=; b=DjZdElrRBqqyOPWy7QJ68MhkRM/Vlk78BNpuWx9DN21w2/8k1m3Benk6c0EMqNKYL0 2+Im8ec14PGe7jkZXJGfrlv79cyGzStGcMUYyLeJOVxYbK8QZ/zt58LVyL3lxsIQCDPb WUUH+ntbLmdcjwExqYYqLSkRakHLd/F4DJsNPVyt5SyRmGtWpiKQ4o9qPPwaL1w6aIQ1 jQOsaseJLXh8Mj3fYthBD3GkAeca+KVy94rBkV78azvrgfKJpSUuOxnYJAC3LldC6EL9 7OBAHuRQW3fuB+7MPYnu0v7CXahA3xn2WSXE1EcJ3LPakg3pMc0dqZPT3nLqIdegfhks 5b5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=zp4fTgOelbZ4xInDdUcDs/dnGRl0FK2EW9il7U9SkXw=; b=SoIdgaruKu+OZAaBQBraa2uc9CaHACGEd3ronkPSa7Oiy5kDZVmO2clalL6x877Z+4 XC3ntVdOcR19NNHDHqrm9OMg1Vec5RfWzkmDm+fVGZnEZZcuGtwLGFNcWwIEDcg+cQxW o1K5SVWBzIt6CIIdhTZhrkCcSnU+qLWhIAtT8xRVD9EtO1mqUHbMJqJxlAvRE6EWxYnf tGSsHRZN4hxktsxuOpUzqDRxWErkn6ywy4r0hZXTtSmffafuYNTTJj5RDxmjQnDn8haJ TIDY5fT3xW6eiavOP9OYFgMBUFLj6JHikF9AzwcrSlRPit9Ref79xKsq8LoSXTV24yMB lKyg== X-Gm-Message-State: APjAAAWDlEg3tOhTlLgbt3InpKlydy5U02urv4WcG6LadTcGefjwiaD/ O3QFj+rV8B+i51/3FTBVDqNq4tR9NfGnzTekD1M= X-Google-Smtp-Source: APXvYqyjj1EFjeOzCagN9eh+2Kc9qKth619TwoxwgrfFI2jXtTPJXba051BRwbLHxvk38QwBZTjqJ/Rhg3xcyd6qnxA= X-Received: by 2002:a2e:8556:: with SMTP id u22mr7019969ljj.70.1558932043407; Sun, 26 May 2019 21:40:43 -0700 (PDT) In-Reply-To: X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::233 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.org gmane.lisp.guile.devel:19930 Archived-At: --000000000000ad24310589d72623 Content-Type: text/plain; charset="UTF-8" Andy, I want to temporarily, maybe permanently retract my last email. I found a bug(let) in the intialization of my unit test that appears to maybe account for all observed crashes, maybe. I'm re-verifying on multiple machines but this will take 24-48 hours. Basically, it looks like guile-2.9.2 is crash free (for the last 6 hours) on one os/compiler/glibc combination, I have to cross-check on another. --linas On Sun, May 26, 2019 at 8:40 AM Linas Vepstas wrote: > Hi Andy, > > Thanks! I just tried out the master branch of guile in git (the one > tagged v2.9.2). It now passes all of my unit tests. So that's good! ... > More or less -- there's still some infrequent multi-threading bug(s). Let > me describe. > > My unit test just transitions C->guile->C and returns, in rapid > succession, in 20 threads. So, in pseudocode: > > SCM do_stuff(SCM a, SCM b) { > scm_to_utf8_string(a); > scm_to_int(b); > ... minor stuff... return scm_from_int(...); > } > scm_c_define_gsubr("do-things",2,0,0, do_stuff); > > void thing_doer(int thread_id) { > for (i=0; i=15000; i++) > char str[100]; > sprintf(str, "(do-things foo %d)", i); > scm_c_catch(scm_eval_string, str); > } > > main () { > for (int i=1; i<15; i++) // start 15 threads > std::thread(&thing_doer, i); > } > > I'm guessing the above code spends maybe 90% of its time bouncing between > guile and C. The string "(do-things foo 42)" changes each time in the loop, > so, not sure how the compile vs. interpret tradeoff is done. Either way, > its relatively trivial. Likewise, the do_stuff() C routine is fairly thin; > after decoding it's args, it doesn't do all that much (sub-microsecond of > computing). Based on old, old measurements, scm_eval_string really is the > primary CPU consumer, in the 20-microseconds range. Launching 15 threads > means that this thing is racing as fast as it can. > > Anyway, with guile-2.9.2, the above crashes after about 10-15 minutes, > either with memory corruption, or with segfault. I worried, so I retested > with guile-2.2.4 ... which also crashes, but much much less frequently: > seven times in 44 hours wall-clock time (so once ever 6 hours). Which is > still more than desired, but...OK. > > So where's the crash? No clue. Could be my code, could be guile. Since > there's a big difference between guile-2.2 and guile-2.9, I'm ready to > blame guile. I did try to run it with `valgrind --tool helgrind` and got an > ocean of complaints about guile GC, which are probably harmless. I haven't > tried to dig deeper yet. > > -- Linas > > > On Thu, May 23, 2019 at 2:28 PM Andy Wingo wrote: > >> Hi! >> >> On Thu 06 Dec 2018 06:21, Linas Vepstas writes: >> >> > After sending the email below, I scanned the guile-devel archives, >> > and I see Thomas Morley talking about Lilypond performance. >> > The example program he offers up caught my eye: nested deep >> > in a loop is this: >> > >> > (eval-string "'(a b c)") >> >> In this case I believe Guile 2.9 / 3 should be significantly faster than >> 2.2, because `eval' is compiled to native code rather than bytecode. My >> measurements showed it to be on par with the hand-optimized C >> implementation from 1.8 and before. Depends of course on how much the >> expander is part of your workload, there are differences relative to >> Guile 1.8. Anyway, thanks for the note and I just wanted to mention >> this point. >> >> Regarding Scheme -> C++ transitions, there is the possibility that this >> too could be much faster with Guile 2.9.x given that these calls are now >> JIT-compiled instead of interpreted. We'll have to see. >> >> Cheers, >> >> Andy >> > > > -- > cassette tapes - analog TV - film cameras - you > -- cassette tapes - analog TV - film cameras - you --000000000000ad24310589d72623 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Andy,

I want to temporarily,= maybe permanently retract my last email. I found a bug(let) in the intiali= zation of my unit test that appears to maybe account for all observed crash= es, maybe. I'm re-verifying on multiple machines but this will take 24-= 48 hours.=C2=A0 Basically, it looks like guile-2.9.2 is crash free (for the= last 6 hours) on one os/compiler/glibc combination, I have to cross-check = on another.

--linas

On Sun, May 26, 2019= at 8:40 AM Linas Vepstas <lin= asvepstas@gmail.com> wrote:
Hi Andy,

Thanks!= =C2=A0 I just tried out the master branch of guile in git (the one tagged v= 2.9.2). It now passes all of my unit tests.=C2=A0 So that's good! ... M= ore or less -- there's still some infrequent multi-threading bug(s).=C2= =A0 Let me describe.=C2=A0

My unit test just = transitions C->guile->C and returns, in rapid succession, in 20 threa= ds. So, in pseudocode:

SCM do_stuff(SCM a, SC= M b) {
=C2=A0 =C2=A0 =C2=A0 scm_to_utf8_string(a);
=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 scm_to_int(b);
=C2=A0=C2=A0=C2=A0=C2= =A0 ... minor stuff... return scm_from_int(...);
}
scm_= c_define_gsubr("do-things",2,0,0, do_stuff);

=
void thing_doer(int thread_id) {
=C2=A0=C2=A0=C2=A0=C2=A0 fo= r (i=3D0; i=3D15000; i++)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 char str[100];
=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 sprintf(str, "(do-things fo= o %d)", i);
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 scm_c_catch(scm_eval_string,=C2=A0 str);
}
=

main () {
=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 for (int i=3D1; i<15; i++)=C2=A0=C2=A0 // start 15 threads
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = std::thread(&thing_doer, i);
}

I= 'm guessing the above code spends maybe 90% of its time bouncing betwee= n guile and C. The string "(do-things foo 42)" changes each time = in the loop, so, not sure how the compile vs. interpret tradeoff is done.= =C2=A0 Either way, its relatively trivial. Likewise, the do_stuff() C routi= ne is fairly thin; after decoding it's args, it doesn't do all that= much (sub-microsecond of computing).=C2=A0 Based on old, old measurements,= scm_eval_string really is the primary CPU consumer, in the 20-microseconds= range. Launching 15 threads means that this thing is racing as fast as it = can.

Anyway, with guile-2.9.2, the above crashes a= fter about 10-15 minutes, either with memory corruption, or with segfault.= =C2=A0 I worried, so I retested with guile-2.2.4 ... which also crashes, bu= t much much less frequently: seven times in 44 hours wall-clock time (so on= ce ever 6 hours).=C2=A0 Which is still more than desired, but...OK.

So where's the crash? No clue. Could be my code, coul= d be guile. Since there's a big difference between guile-2.2 and guile-= 2.9, I'm ready to blame guile. I did try to run it with `valgrind --too= l helgrind` and got an ocean of complaints about guile GC, which are probab= ly harmless. I haven't tried to dig deeper yet.

-- Linas


<= div class=3D"gmail_quote">
On Thu, May= 23, 2019 at 2:28 PM Andy Wingo <wingo@pobox.com> wrote:
Hi!

On Thu 06 Dec 2018 06:21, Linas Vepstas <linasvepstas@gmail.com> writes:

> After sending the email below, I scanned the guile-devel archives,
> and I see Thomas Morley talking about Lilypond performance.
> The example program he offers up caught my eye: nested deep
> in a loop is this:
>
> (eval-string "'(a b c)")

In this case I believe Guile 2.9 / 3 should be significantly faster than 2.2, because `eval' is compiled to native code rather than bytecode.=C2= =A0 My
measurements showed it to be on par with the hand-optimized C
implementation from 1.8 and before.=C2=A0 Depends of course on how much the=
expander is part of your workload, there are differences relative to
Guile 1.8.=C2=A0 Anyway, thanks for the note and I just wanted to mention this point.

Regarding Scheme -> C++ transitions, there is the possibility that this<= br> too could be much faster with Guile 2.9.x given that these calls are now JIT-compiled instead of interpreted.=C2=A0 We'll have to see.

Cheers,

Andy


--
cassette tapes = - analog TV - film cameras - you


--
cassette tapes - analog TV - film cameras = - you
--000000000000ad24310589d72623--