let's bytecode it!

let's bytecode it!

[Andy Wingo]

Hello,

So long in the past, it seemed that Keisuke's Guile-VM would mature and
replace the existing scm_ceval(). Everyone was down with it: Marius,
Mikael, etc. But it seems that there were some lingering issues, and it
didn't really settle down into Guile proper.

Then a couple yeats ago (!), Ludovic proposed reviving guile-vm, and
started hacking on it in some arch archive somewheres. But again, time
was lacking (I assume) and I can't even find that archive any more.
Ludovic?

I think that a bytecode compiler and VM makes a lot of sense for Guile.
I don't see native compilation ever being a primary target of Guile,
given that our strength is our ubiquity and how we play nice with C, but
a VM would finally bring Guile's speed up to be comparable with e.g.
python.

Also it would be fun to hack!

So how about making guile-vm in core guile a goal for 1.10? I guess the
way to start is for Ludovic to import his repo into git and expose it
somewhere, and perhaps merge it into Guile proper on a branch. Then we
can test it out, see what's going on, hack hack hack&stuff, and maybe
merge it to master at some point.

Thoughts? Neil, Ludo? Other hackers that have always wanted to play
with a compiler (me, me!) ? :-)

Regards,

Andy
-- 
http://wingolog.org/

Re: let's bytecode it!

[Julian Graham]

I'd be down!  This sounds like a great project.


On Thu, Apr 24, 2008 at 3:39 PM, Andy Wingo <wingo@pobox.com> wrote:
> Hello,
>
>  So long in the past, it seemed that Keisuke's Guile-VM would mature and
>  replace the existing scm_ceval(). Everyone was down with it: Marius,
>  Mikael, etc. But it seems that there were some lingering issues, and it
>  didn't really settle down into Guile proper.
>
>  Then a couple yeats ago (!), Ludovic proposed reviving guile-vm, and
>  started hacking on it in some arch archive somewheres. But again, time
>  was lacking (I assume) and I can't even find that archive any more.
>  Ludovic?
>
>  I think that a bytecode compiler and VM makes a lot of sense for Guile.
>  I don't see native compilation ever being a primary target of Guile,
>  given that our strength is our ubiquity and how we play nice with C, but
>  a VM would finally bring Guile's speed up to be comparable with e.g.
>  python.
>
>  Also it would be fun to hack!
>
>  So how about making guile-vm in core guile a goal for 1.10? I guess the
>  way to start is for Ludovic to import his repo into git and expose it
>  somewhere, and perhaps merge it into Guile proper on a branch. Then we
>  can test it out, see what's going on, hack hack hack&stuff, and maybe
>  merge it to master at some point.
>
>  Thoughts? Neil, Ludo? Other hackers that have always wanted to play
>  with a compiler (me, me!) ? :-)

Re: let's bytecode it!

[Ludovic Courtès]

Hey!

Andy Wingo <wingo@pobox.com> writes:

> Then a couple yeats ago (!), Ludovic proposed reviving guile-vm, and
> started hacking on it in some arch archive somewheres. But again, time
> was lacking (I assume) and I can't even find that archive any more.
> Ludovic?

Wwow, great idea!  :-)

Indeed, time was lacking, and it wasn't all that clear to me how it
should deal with macros.

I don't have my old Arch archive at hand right now, but I'll convert it
to Git tonight and put it on-line somewhere.

Guile-VM is written as an independent project currently, so I don't
think it would fit well into core Guile, and I'm actually not sure it'd
be a good idea to put it there, at least for now.

Anyway, I'll let you know when it's on-line!

Thanks,
Ludo'.

Re: let's bytecode it!

[Mikael Djurfeldt]

2008/4/25 Ludovic Courtès <ludo@gnu.org>:
>  Guile-VM is written as an independent project currently, so I don't
>  think it would fit well into core Guile, and I'm actually not sure it'd
>  be a good idea to put it there, at least for now.

[Jumping in again although I shouldn't since I don't normally follow
activity on the list.]

I might be missing something, but I think Andy's idea of putting
guile-vm in the core is obviously the right thing to do.  In fact, as
soon as the vm is fully functional and stable (which may take time, I
grant you that) I think it should replace the current evaluator except
possibly for debugging.  (Currently, the evaluator code is compiled
twice---once for generating the sluggish version of the evaluator, and
yet another time to produce the even more sluggish debugging
evaluator.  A vm could replace the sluggish evaluator.  The debugging
evaluator could be kept for supporting backtraces and the debugger.)
I wanted to do this from the first instance when I tried out Keisuke's
vm, but could never find time to do it.  Even though other properties
of Guile *should* be reasons for people to use Guile, I'm personally
convinced that people take the slugishness as a sign of poor code and
think this is the major reason why Guile hasn't been adopted to an
extent several magnitudes more than currently.

Another note: The author of QScheme (which has a very efficient
byte-code interpreter) once teamed up with us with the aim to combine
ideas from QScheme and Keisuke's vm to implement the Guile bytecode
interpreter, but this never got started for real.  Maybe it's worth
looking at QScheme?

Also, if I moved a bytecode interpreter into the core, I would
probably look at the possibility to place GOOPS method dispatch as a
central mechanism---maybe this could be the only form of type
dispatch?  I would try to remove type checking in all Guile primivites
and move the responsibility for this to this core dispatch mechanism.
In order not to loose backward compatibility for those using the Guile
API from C in their applications, one could have type-dispatching
glue-code with the old C API names.

But if this seems too futuristic, just replacing the sluggish
evaluator with the vm would give Guile a major boost.  Be sure to test
it against code using threads, though...

Re: let's bytecode it!

[Ludovic Courtès]

Hi Mikael,

"Mikael Djurfeldt" <mikael@djurfeldt.com> writes:

> [Jumping in again although I shouldn't since I don't normally follow
> activity on the list.]

Well, it's good to have insight from those who where there at the time,
as Andy pointed out.  ;-)

> I might be missing something, but I think Andy's idea of putting
> guile-vm in the core is obviously the right thing to do.

Guile-VM has a compiler, which compiles s-exps to GHIL, then to GLIL,
then to actual bytecode.  At each stage, it could implement smart
optimizations (although it doesn't currently ;-)).

Conversely, the interpreter has to be kept simple and above all, it has
to do as many things as possible lazily (like variable lookup,
"memoization", etc.).

Running a compile-run cycle each time the user wants to interpret code
is going to be pretty slow (at least the startup time), because you'd
end up going through a potentially slow compile cycle for each sexp.

And IMO we don't want to completely do away with the interpreter in
Guile, because an interpreter has some practical virtues too.  What I'd
envision is to have an interpreter in core Guile, and a compiler/VM *in
addition* (which could eventually be part of core Guile, but may not be
a replacement for the interpreter).

> Even though other properties of Guile *should* be reasons for people
> to use Guile, I'm personally convinced that people take the
> slugishness as a sign of poor code and think this is the major reason
> why Guile hasn't been adopted to an extent several magnitudes more
> than currently.

Yes, and to be honest, many schemers make fun of this:

  A "pure" interpreter, on the other hand, does none of this business
  prior to evaluation (it just interprets source-level s- expressions;
  plain and simple).  I'm actually surprised to see this used in
  anything other than a homework assignment for junior-level course on
  programming languages.

  http://lists.r6rs.org/pipermail/r6rs-discuss/2007-March/001819.html

Not to mention what those who actually looked at the C code say...  ;-)

> But if this seems too futuristic, just replacing the sluggish
> evaluator with the vm would give Guile a major boost.  Be sure to test
> it against code using threads, though...

For the interpreter itself, if we are to keep it in addition to the
compiler/VM, I had in mind an incremental rewriting in a subset of
Scheme then compiled to C, which would make it more tractable, and
possibly allow us to optimize it more easily.  But this too is a bit
futuristic...

Thanks,
Ludovic.

Reviving Guile-VM again

[Ludovic Courtès]

Hello,

I imported the Guile-VM stuff into Git (i.e., CVS + Arch history) and
it's available as follows:

  git-clone http://www.fdn.fr/~lcourtes/software/guile-vm.git

(I actually created a `guile-vm' project at repo.or.cz but my SSH key
there is obsolete and it cannot be updated automatically.  I'll have to
wait until the admins update it...)

Beware: I certainly left it in some random state.  I'll be happy to
receive patches.  :-)

Thanks,
Ludovic.

Re: let's bytecode it!

[Neil Jerram]

Andy Wingo <wingo@pobox.com> writes:

> Thoughts? Neil, Ludo? Other hackers that have always wanted to play
> with a compiler (me, me!) ? :-)

I'm not up to speed with any of the technical detail yet, but in
principle I'm all for this.

       Neil