From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Linas Vepstas Newsgroups: gmane.lisp.guile.devel Subject: Re: guile-2.9.1 impressions Date: Sun, 26 May 2019 08:40:59 -0500 Message-ID: References: <877eahotro.fsf@pobox.com> Reply-To: linasvepstas@gmail.com Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="000000000000bfb5fb0589ca95c3" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="64934"; mail-complaints-to="usenet@blaine.gmane.org" Cc: Guile Development To: Andy Wingo Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Sun May 26 15:50:17 2019 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hUtXV-000Gls-AY for guile-devel@m.gmane.org; Sun, 26 May 2019 15:50:17 +0200 Original-Received: from localhost ([127.0.0.1]:55612 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hUtXT-00084d-Us for guile-devel@m.gmane.org; Sun, 26 May 2019 09:50:16 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:49865) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hUtXJ-00081i-1a for guile-devel@gnu.org; Sun, 26 May 2019 09:50:06 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hUtOm-0008Ud-L7 for guile-devel@gnu.org; Sun, 26 May 2019 09:41:18 -0400 Original-Received: from mail-lf1-x132.google.com ([2a00:1450:4864:20::132]:40688) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hUtOm-0008NO-A1 for guile-devel@gnu.org; Sun, 26 May 2019 09:41:16 -0400 Original-Received: by mail-lf1-x132.google.com with SMTP id h13so10216811lfc.7 for ; Sun, 26 May 2019 06:41:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:reply-to:from:date:message-id :subject:to:cc; bh=2h9TSxfw6xXlpszwNILiyOxFPn+NF7eqpaxwKzqTuPo=; b=Jhskwrv/uPIgBpw9d+vQroL31gGa77fuirCMQUpnj9ryJCKloPW3nll9tLL6VIsdKE E/on7ipcpTgifqC3eSnm2HyqeuQ0QYqCKA2seqduifW81W17Fqh8wz88wn7+fymDok7A V+ktRDUqIGumacNOa2myA/urS61omCBO3GuKlJLa0Yt0OjQgVR21zZAclx9gHqj3/OnJ fRo9Qlo3Xn+TdpCYcVmz550BzVywGgs0EwZD/epWdzQxQ3hc2/ONHWEoH+gVaX95lQHc GDS+9/3bAmgorv77jFWJI+zuaI8VzYmdAqR5U8bXfiOVUk73U1Z8M3LS/DDabhps4E8i q9RA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=2h9TSxfw6xXlpszwNILiyOxFPn+NF7eqpaxwKzqTuPo=; b=dxSlYAhwxiMsW6qEZu2fgw1loHn9i9mGf29miCVIVMzQrrIANqb7mxQjMbWMq4ixSd JUGqVBb9yUaFoeWbIbUXPT333dqxeDsslRAzW7ZN0TLSaTMEHXkRMG+4I4Np9VVxwIZC CvID8D/3R5tu7BZ1IdlzRu5gqVPeMPdOrKKALTWeRIhM9h+k8vcKOmhTNRd5onWlKT2U eVhw+omQKMkstYoJf7mQn4EcBhij+W/d0g+H6pGziMyYQEp79+lEnsR+vf5hQ20tTH9z Vg+ZlcEWJKMGm56GBZZV8R3DYzhmikjb/kptC1soo1fyyUdZeOx4HmAnmgkzU/JKJLCt HBpQ== X-Gm-Message-State: APjAAAUga0p1BKYwRFzkNKz9r3aRBlBSsk20hu4sKV/Eid+jUGTevOXJ DdgxNoEriPQO2hkzyq0kd8uxOry7bosU4BB/JxqKgVNwevQ= X-Google-Smtp-Source: APXvYqyF6vOTupSCSg7W8/nqy4qkc9+oP+V9xx9JCTLaW+NaHD/5ObeDJ2YSMD3rfZir1vWb8DpUuWSr1cRyTiOZc4g= X-Received: by 2002:ac2:4209:: with SMTP id y9mr17065039lfh.83.1558878072319; Sun, 26 May 2019 06:41:12 -0700 (PDT) In-Reply-To: <877eahotro.fsf@pobox.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::132 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.org gmane.lisp.guile.devel:19929 Archived-At: --000000000000bfb5fb0589ca95c3 Content-Type: text/plain; charset="UTF-8" Hi Andy, Thanks! I just tried out the master branch of guile in git (the one tagged v2.9.2). It now passes all of my unit tests. So that's good! ... More or less -- there's still some infrequent multi-threading bug(s). Let me describe. My unit test just transitions C->guile->C and returns, in rapid succession, in 20 threads. So, in pseudocode: SCM do_stuff(SCM a, SCM b) { scm_to_utf8_string(a); scm_to_int(b); ... minor stuff... return scm_from_int(...); } scm_c_define_gsubr("do-things",2,0,0, do_stuff); void thing_doer(int thread_id) { for (i=0; i=15000; i++) char str[100]; sprintf(str, "(do-things foo %d)", i); scm_c_catch(scm_eval_string, str); } main () { for (int i=1; i<15; i++) // start 15 threads std::thread(&thing_doer, i); } I'm guessing the above code spends maybe 90% of its time bouncing between guile and C. The string "(do-things foo 42)" changes each time in the loop, so, not sure how the compile vs. interpret tradeoff is done. Either way, its relatively trivial. Likewise, the do_stuff() C routine is fairly thin; after decoding it's args, it doesn't do all that much (sub-microsecond of computing). Based on old, old measurements, scm_eval_string really is the primary CPU consumer, in the 20-microseconds range. Launching 15 threads means that this thing is racing as fast as it can. Anyway, with guile-2.9.2, the above crashes after about 10-15 minutes, either with memory corruption, or with segfault. I worried, so I retested with guile-2.2.4 ... which also crashes, but much much less frequently: seven times in 44 hours wall-clock time (so once ever 6 hours). Which is still more than desired, but...OK. So where's the crash? No clue. Could be my code, could be guile. Since there's a big difference between guile-2.2 and guile-2.9, I'm ready to blame guile. I did try to run it with `valgrind --tool helgrind` and got an ocean of complaints about guile GC, which are probably harmless. I haven't tried to dig deeper yet. -- Linas On Thu, May 23, 2019 at 2:28 PM Andy Wingo wrote: > Hi! > > On Thu 06 Dec 2018 06:21, Linas Vepstas writes: > > > After sending the email below, I scanned the guile-devel archives, > > and I see Thomas Morley talking about Lilypond performance. > > The example program he offers up caught my eye: nested deep > > in a loop is this: > > > > (eval-string "'(a b c)") > > In this case I believe Guile 2.9 / 3 should be significantly faster than > 2.2, because `eval' is compiled to native code rather than bytecode. My > measurements showed it to be on par with the hand-optimized C > implementation from 1.8 and before. Depends of course on how much the > expander is part of your workload, there are differences relative to > Guile 1.8. Anyway, thanks for the note and I just wanted to mention > this point. > > Regarding Scheme -> C++ transitions, there is the possibility that this > too could be much faster with Guile 2.9.x given that these calls are now > JIT-compiled instead of interpreted. We'll have to see. > > Cheers, > > Andy > -- cassette tapes - analog TV - film cameras - you --000000000000bfb5fb0589ca95c3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Andy,

Thanks!=C2=A0 I just tried= out the master branch of guile in git (the one tagged v2.9.2). It now pass= es all of my unit tests.=C2=A0 So that's good! ... More or less -- ther= e's still some infrequent multi-threading bug(s).=C2=A0 Let me describe= .=C2=A0

My unit test just transitions C->g= uile->C and returns, in rapid succession, in 20 threads. So, in pseudoco= de:

SCM do_stuff(SCM a, SCM b) {
= =C2=A0 =C2=A0 =C2=A0 scm_to_utf8_string(a);
=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 scm_to_int(b);
=C2=A0=C2=A0=C2=A0=C2=A0 ... minor stuff= ... return scm_from_int(...);
}
scm_c_define_gsubr(&quo= t;do-things",2,0,0, do_stuff);

void thing_doe= r(int thread_id) {
=C2=A0=C2=A0=C2=A0=C2=A0 for (i=3D0; i=3D15000= ; i++)
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 char str[100];
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0 sprintf(str, "(do-things foo %d)", i);
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 s= cm_c_catch(scm_eval_string,=C2=A0 str);
}

main () {
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 for (int i=3D1= ; i<15; i++)=C2=A0=C2=A0 // start 15 threads
=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 std::thread(&thi= ng_doer, i);
}

I'm guessing the = above code spends maybe 90% of its time bouncing between guile and C. The s= tring "(do-things foo 42)" changes each time in the loop, so, not= sure how the compile vs. interpret tradeoff is done.=C2=A0 Either way, its= relatively trivial. Likewise, the do_stuff() C routine is fairly thin; aft= er decoding it's args, it doesn't do all that much (sub-microsecond= of computing).=C2=A0 Based on old, old measurements, scm_eval_string reall= y is the primary CPU consumer, in the 20-microseconds range. Launching 15 t= hreads means that this thing is racing as fast as it can.

Anyway, with guile-2.9.2, the above crashes after about 10-15 minut= es, either with memory corruption, or with segfault.=C2=A0 I worried, so I = retested with guile-2.2.4 ... which also crashes, but much much less freque= ntly: seven times in 44 hours wall-clock time (so once ever 6 hours).=C2=A0= Which is still more than desired, but...OK.

So wh= ere's the crash? No clue. Could be my code, could be guile. Since there= 's a big difference between guile-2.2 and guile-2.9, I'm ready to b= lame guile. I did try to run it with `valgrind --tool helgrind` and got an = ocean of complaints about guile GC, which are probably harmless. I haven= 9;t tried to dig deeper yet.

-- Linas


On Thu, May 23, 2019 at 2:28 PM An= dy Wingo <wingo@pobox.com> wro= te:
Hi!

On Thu 06 Dec 2018 06:21, Linas Vepstas <linasvepstas@gmail.com> writes:

> After sending the email below, I scanned the guile-devel archives,
> and I see Thomas Morley talking about Lilypond performance.
> The example program he offers up caught my eye: nested deep
> in a loop is this:
>
> (eval-string "'(a b c)")

In this case I believe Guile 2.9 / 3 should be significantly faster than 2.2, because `eval' is compiled to native code rather than bytecode.=C2= =A0 My
measurements showed it to be on par with the hand-optimized C
implementation from 1.8 and before.=C2=A0 Depends of course on how much the=
expander is part of your workload, there are differences relative to
Guile 1.8.=C2=A0 Anyway, thanks for the note and I just wanted to mention this point.

Regarding Scheme -> C++ transitions, there is the possibility that this<= br> too could be much faster with Guile 2.9.x given that these calls are now JIT-compiled instead of interpreted.=C2=A0 We'll have to see.

Cheers,

Andy


--
cassette tapes - analog TV - film cameras = - you
--000000000000bfb5fb0589ca95c3--