further guile-vm integration

further guile-vm integration

[Andy Wingo]

Hello all,

The `vm' branch in the guile repo is coming along nicely. It fully
self-compiles, passes test suites, has a really useful repl, etc etc.
There are some problems with call/cc that will be fixed in the future,
but it's mostly correct.

So where to from here? I was making a long list, and ended up deciding
that the first priority is to integrate the VM code into libguile
itself, and thus into the standard `guile' binary. (Currently it builds
a guile-vm binary that links to libguile-vm, which links to libguile.)

The reasons for this, from Guile's perspective, are twofold:

  1. Make module compilation and load-compiled work out "of the box".

  This means that you should be able to make .go files for any module,
  normally as a part of package installation, and they should be
  loadable by guile without any problems or extra configuration.

  2. To improve the load time of Guile itself.

  Loading compiled modules is *much* faster than loading source modules
  -- it's a simple mmap-and-go approach. Compiling boot-9.scm is going
  to be great. Using psyntax is going to have no cost penalties. Having
  guile's modules be compiled will also lead to runtime speed
  improvements in Guile itself.

So I'm going to work on moving the source files over to the libguile/
directory, and to start thinking about the modules. I like that
guile-vm's modules are in a separate namespace (well two, actually --
(system ...) and (language ...)), but I do not intend to move ice-9
things into that hierarchy, to avoid breaking existing code.

User-level code should see no changes, other than speed improvements :)

Let me know if you have thoughts about this plan! My hope would be that
once there are no or very few and solvable regressions, we could merge
this to master and call it 1.10 or 2.0.

Andy
-- 
http://wingolog.org/

Re: further guile-vm integration

[Han-Wen Nienhuys]

Andy Wingo escreveu:
> Let me know if you have thoughts about this plan! My hope would be that
> once there are no or very few and solvable regressions, we could merge
> this to master and call it 1.10 or 2.0.

I'm always for faster release cycles, but wouldn't it be good to push out
1.10 now, and merge the VM (which is a large change) afterwards?

-- 
 Han-Wen Nienhuys - hanwen@xs4all.nl - http://www.xs4all.nl/~hanwen

Re: further guile-vm integration

[Ludovic Courtès]

Hi,

Han-Wen Nienhuys <hanwen@xs4all.nl> writes:

> I'm always for faster release cycles, but wouldn't it be good to push out
> 1.10 now, and merge the VM (which is a large change) afterwards?

I think there are still a lot of small, incremental improvements to be
made in 1.8.  That is admittedly less fun than playing with the VM or
other large changes, but I think it's important, too.

Thanks,
Ludovic.

Re: further guile-vm integration

[Ludovic Courtès]

Hello!

Andy Wingo <wingo@pobox.com> writes:

> The `vm' branch in the guile repo is coming along nicely. It fully
> self-compiles, passes test suites, has a really useful repl, etc etc.
> There are some problems with call/cc that will be fixed in the future,
> but it's mostly correct.

Thanks for the great news!  :-)

>   1. Make module compilation and load-compiled work out "of the box".
>
>   This means that you should be able to make .go files for any module,
>   normally as a part of package installation, and they should be
>   loadable by guile without any problems or extra configuration.

Actually `.go' loading support has been in `boot-9.scm' "forever".  ;-)

Besides, do you have an automatic compilation approach in mind (à la
Python) or something explicit, possibly with source/binary time-stamp
comparison?

The latter looks more "user-friendly" to me but wouldn't be as
transparent.

> So I'm going to work on moving the source files over to the libguile/
> directory, and to start thinking about the modules. I like that
> guile-vm's modules are in a separate namespace (well two, actually --
> (system ...) and (language ...)), but I do not intend to move ice-9
> things into that hierarchy, to avoid breaking existing code.

Agreed.

> Let me know if you have thoughts about this plan! My hope would be that
> once there are no or very few and solvable regressions, we could merge
> this to master and call it 1.10 or 2.0.

Did you benchmark the thing?  I remember speed improvements were not all
that clear back when I played with Guile-VM.

Besides, before making a release, it may be worth thinking about
potential GO format changes that we'd possibly want to make in the
future.  One big thing I had in mind was the use of word-aligned
bytecodes, which should improve performance on several architectures
(accessing data that's not aligned on a machine-word boundary is costly
on RISC architectures, and you end up loading a full machine word
anyway; on Intel, it appears to improve cache performance [0]).  Using
only fixed-size bytecodes might even be an option, as it would simplify
the bytecode interpreter, potentially making it faster as well.

Thanks,
Ludo'.

[0] http://softwarecommunity.intel.com/Wiki/Performanceoptimization/502.htm

Re: further guile-vm integration

[Andy Wingo]

Hi,

On Fri 22 Aug 2008 12:23, ludo@gnu.org (Ludovic Courtès) writes:

> Besides, do you have an automatic compilation approach in mind (à la
> Python) or something explicit, possibly with source/binary time-stamp
> comparison?

I don't know really. Currently it's explicit. But automatic could be
interesting; you have to check timestamps anyway to see if the compiled
file is actually older than the source file, in which case you want to
warn and load the source file instead. But in general I would follow
emacs' lead in this regard.

> Did you benchmark the thing?  I remember speed improvements were not all
> that clear back when I played with Guile-VM.

I've been benchmarking, yes, and the advantages are now pretty clear, at
runtime, and the startup advantages are excellent. But I have yet to
compile boot-9.scm.

> Besides, before making a release, it may be worth thinking about
> potential GO format changes that we'd possibly want to make in the
> future.

Sure, definitely.

> One big thing I had in mind was the use of word-aligned
> bytecodes, which should improve performance on several architectures

Sure, would be a great thing to try.

Andy
-- 
http://wingolog.org/