Guile benchmark

Guile benchmark

[Rchar]

Hello,
I wanted to compare Guile scheme to other scheme implementations and I found this:https://ecraven.github.io/r7rs-benchmarks/benchmark.html

Is Guile slow or fast, comparing to others?
Is Guile scheme slows down entire GuixSD (If Guile speeds up, GuixSD also speeds up)?
Recently I found scheme compared to C, is it even possible?

Please help answering these questions, and share your opinion.



Regards
rchar

sidhu1fveeouoe

Re: Guile benchmark

[Mike Gran]

On Thursday, January 26, 2017 7:31 AM, Rchar <rchar@protonmail.com> wrote:


> Is Guile slow or fast, comparing to others?

Guile is about average compared to the others. But it depends
on the specific task. It is usually not the fastest or slowest.

> Recently I found scheme compared to C, is it even possible?


Not really.  A Guile program should be coded to take advantage
of Guile.   A C program should be coded to take advantage of C.
The programs should have a very different structure.

If you coded a Guile program to have the same structure and algorithms
as a C program, the C program would be faster.

Mike

Re: Guile benchmark

[Linas Vepstas]

On Thu, Jan 26, 2017 at 3:56 PM, Mike Gran <spk121@yahoo.com> wrote:

>
> On Thursday, January 26, 2017 7:31 AM, Rchar <rchar@protonmail.com> wrote:
>
>
>> Is Guile slow or fast, comparing to others?
>
> Guile is about average compared to the others. But it depends
> on the specific task. It is usually not the fastest or slowest.

FWIW, I have C++ code that has bindings both to guile and to python,
via cython.  The guile bindings were faster than python, which then
caused the cython maintainer to enter a nuclear arms race, to improve
performance, doing things like caching values in various places to improve
performance.  But then, one can do the same in guile, and it becomes a
game of who can be more clever in optimizing.

I do have one complaint: guile has a fairly hefty overhead in entering
and exiting: I forget now, and the measurements were done on a
circa-2011 PC, but I was getting something like 7K guile enter-exits
per second, which limits the speed at which you can cross the C/guile
boundary.   You can avoid this cost by entering guile and then caching,
but this requires some C-coding cleverness that naive users won't do.
(I'm currently getting maybe 500K/sec or 1M/sec guile-C crossings, but
only if I do this caching trick).

--linas

Re: Guile benchmark

[Ludovic Courtès]

Hi,

Rchar <rchar@protonmail.com> skribis:

> Is Guile scheme slows down entire GuixSD (If Guile speeds up, GuixSD also speeds up)?

GuixSD is a GNU/Linux distro and even if some components are written in
Guile (such as the Shepherd and some low-level helpers), most of it is
written in C.

Guile’s speed has an influence on the speed of ‘guix’ commands though.

Ludo’.

Re: Guile benchmark

[Andy Wingo]

Hi,

On Thu 26 Jan 2017 09:39, Rchar <rchar@protonmail.com> writes:

> I wanted to compare Guile scheme to other scheme implementations and I found this:https://ecraven.github.io/r7rs-benchmarks/benchmark.html
>
> Is Guile slow or fast, comparing to others?

Besides what Mike said, there are two kinds of Schemes in that
benchmark: the ones that compile to native machine code (ahead of time,
either with their own compilers or by emitting C, or just-in-time), and
the ones that run some kind of bytecode interpreter.

Schemes that compile to native code go faster.  Guile compiles to
bytecode right now, so it's generally (though not always!) slower than
the native-compiling schemes.  But compared to the bytecode-interpreter
schemes it's pretty fast.

One day we'll have a native compiler and we can start playing benchmark
games with the big kids :)

Andy

Re: Guile benchmark

[Chris Vine]

On Mon, 27 Feb 2017 21:00:54 +0100
Andy Wingo <wingo@pobox.com> wrote:
> Hi,
> 
> On Thu 26 Jan 2017 09:39, Rchar <rchar@protonmail.com> writes:
> 
> > I wanted to compare Guile scheme to other scheme implementations
> > and I found
> > this:https://ecraven.github.io/r7rs-benchmarks/benchmark.html
> >
> > Is Guile slow or fast, comparing to others?  
> 
> Besides what Mike said, there are two kinds of Schemes in that
> benchmark: the ones that compile to native machine code (ahead of
> time, either with their own compilers or by emitting C, or
> just-in-time), and the ones that run some kind of bytecode
> interpreter.
> 
> Schemes that compile to native code go faster.  Guile compiles to
> bytecode right now, so it's generally (though not always!) slower than
> the native-compiling schemes.  But compared to the
> bytecode-interpreter schemes it's pretty fast.
> 
> One day we'll have a native compiler and we can start playing
> benchmark games with the big kids :)

On reading this, out of interest I wrote a very simple program solving
primes, using the basic 'seive odd divisors to square root' algorithm.
I tested guile-2.0, guile-2.2, chicken and chez scheme on it.

guile-2.0 was a little slower than chicken with chicken compiled to C,
but guile-2.2 on that test took about 75% of the time of chicken, and
about 50% of the time of guile-2.0.  chez scheme was fastest of all,
taking about 50% of the time of chicken.  OK, chicken may not be the
fastest of "compile to C" schemes.

Having said that, all of them were considerably slower than the same
code written in native C++, probably because the compiler optimizes
better (it was about 5 times faster than chez scheme).  The performance
of clang and gcc was pretty much the same.

Solving primes is probably not particularly representative and I didn't
spend any more time with other benchmarks, so take it as you will. The
performance of guile-2.2 at run time was encouraging however.

Chris

Re: Guile benchmark

[Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1407 bytes --]


Chris Vine <chris@cvine.freeserve.co.uk> writes:

> On Mon, 27 Feb 2017 21:00:54 +0100
> Andy Wingo <wingo@pobox.com> wrote:
>> Hi,
>> 
>> On Thu 26 Jan 2017 09:39, Rchar <rchar@protonmail.com> writes:
>> > https://ecraven.github.io/r7rs-benchmarks/benchmark.html
>> > Is Guile slow or fast, comparing to others?  
>> 
>> Schemes that compile to native code go faster.  Guile compiles to
>> bytecode right now, so it's generally (though not always!) slower than
>> the native-compiling schemes.  But compared to the
>> bytecode-interpreter schemes it's pretty fast.
>
> On reading this, out of interest I wrote a very simple program solving
> primes, using the basic 'seive odd divisors to square root' algorithm.
> I tested guile-2.0, guile-2.2, chicken and chez scheme on it.
>
> guile-2.0 was a little slower than chicken with chicken compiled to C,
> but guile-2.2 on that test took about 75% of the time of chicken, and
> about 50% of the time of guile-2.0.  chez scheme was fastest of all,
> taking about 50% of the time of chicken.  OK, chicken may not be the
> fastest of "compile to C" schemes.

Do I read this correctly that Chez took only about 30% less time than
Guile 2.2?

Could you try stalin, too? (Chez wins 28 comparisons, Stalin 14, so that
would be an obvious target)

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]

Re: Guile benchmark

[Chris Vine]

On Tue, 28 Feb 2017 11:28:32 +0100
Arne Babenhauserheide <arne_bab@web.de> wrote:
> Chris Vine <chris@cvine.freeserve.co.uk> writes:
> 
> > On Mon, 27 Feb 2017 21:00:54 +0100
> > Andy Wingo <wingo@pobox.com> wrote:  
> >> Hi,
> >> 
> >> On Thu 26 Jan 2017 09:39, Rchar <rchar@protonmail.com> writes:  
> >> > https://ecraven.github.io/r7rs-benchmarks/benchmark.html
> >> > Is Guile slow or fast, comparing to others?    
> >> 
> >> Schemes that compile to native code go faster.  Guile compiles to
> >> bytecode right now, so it's generally (though not always!) slower
> >> than the native-compiling schemes.  But compared to the
> >> bytecode-interpreter schemes it's pretty fast.  
> >
> > On reading this, out of interest I wrote a very simple program
> > solving primes, using the basic 'seive odd divisors to square root'
> > algorithm. I tested guile-2.0, guile-2.2, chicken and chez scheme
> > on it.
> >
> > guile-2.0 was a little slower than chicken with chicken compiled to
> > C, but guile-2.2 on that test took about 75% of the time of
> > chicken, and about 50% of the time of guile-2.0.  chez scheme was
> > fastest of all, taking about 50% of the time of chicken.  OK,
> > chicken may not be the fastest of "compile to C" schemes.  
> 
> Do I read this correctly that Chez took only about 30% less time than
> Guile 2.2?
> 
> Could you try stalin, too? (Chez wins 28 comparisons, Stalin 14, so
> that would be an obvious target)

I feared that my enumeration might confuse.

No, on my Haswell laptop running 64-bit linux, solving the 40,000th
prime (479909) is 1.26 times faster in chez scheme than in guile-2.1.7,
on the simple algorithm. (0.509 seconds versus 0.624 seconds, on a
i7-4712HQ CPU @ 2.30GHz.)  This is only the time taken in the body of
the calculation.  Start-up times and printing times and so forth are
ignored.

I have just tried out the same code on an older Sandybridge desktop
@3.30 GHz running 32-bit linux which I can access remotely, and that had
chez scheme 1.7 times faster than guile-2.1.7 so it looks to be
somewhat CPU and/or cache and/or pointer-width sensitive. Anyway, it is
just one benchmark.

I think I am about schemed out so installing Stalin is probably not an
option.  Like you, I am told it is fast.  Guile's selling point IMO is
its libraries and its FFI.  Guile 2.2 certainly seems adequately fast
at run time.  Possibly compile times may be an issue, I don't know.

Chris

Re: Guile benchmark

[Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1352 bytes --]


Chris Vine <chris@cvine.freeserve.co.uk> writes:
> No, on my Haswell laptop running 64-bit linux, solving the 40,000th
> prime (479909) is 1.26 times faster in chez scheme than in guile-2.1.7,
> on the simple algorithm. (0.509 seconds versus 0.624 seconds, on a
> i7-4712HQ CPU @ 2.30GHz.)  This is only the time taken in the body of
> the calculation.  Start-up times and printing times and so forth are
> ignored.

Ah, ok. Thank you!

> I have just tried out the same code on an older Sandybridge desktop
> @3.30 GHz running 32-bit linux which I can access remotely, and that had
> chez scheme 1.7 times faster than guile-2.1.7 so it looks to be
> somewhat CPU and/or cache and/or pointer-width sensitive. Anyway, it is
> just one benchmark.
>
> I think I am about schemed out so installing Stalin is probably not an
> option.  Like you, I am told it is fast.  Guile's selling point IMO is
> its libraries and its FFI.  Guile 2.2 certainly seems adequately fast
> at run time.  Possibly compile times may be an issue, I don't know.

Factor 10 between C++ and pure Scheme is pretty awesome. Implementing
algorithms in pure Python carries a penalty of easily factor 100 (though
delegating anything expensive to builtin operators is fast).

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 800 bytes --]