vm status update, a.k.a. yowsers batman it's february already

vm status update, a.k.a. yowsers batman it's february already

[Andy Wingo]

Greets greets,

An update from the wilds of the vm branch is overdue. So here we go:

 * Opcodes are now numbered statically in the source. This should make
   it easier to maintain bytecode compatibility in the future.

 * The VM now has two new languages:

   - Bytecode, which is just object code as u8vectors; and

   - Assembly, which is between bytecode and GLIL.

   The differences may be seen thusly (hand-pretty-printed):

     scheme@(guile-user)> (compile '(car '(a . b)) #:to 'glil)
     #<glil (program 0 0 0 0 ()
                 (const (a . b))
                 (call car 1)
                 (call return 0))>
     scheme@(guile-user)> (compile '(car '(a . b)) #:to 'assembly)
     (load-program 0 0 0 0 () 13 #f
        (load-symbol "a")
        (load-symbol "b")
        (cons)
        (car)
        (return))
     scheme@(guile-user)> (compile '(car '(a . b)) #:to 'bytecode)
     #u8(0 0 0 0 13 0 0 0 0 0 0 0 63 0 0 1 97 63 0 0 1 98 90 91 48)
     scheme@(guile-user)> (compile '(car '(a . b)) #:to 'objcode)
     #<objcode b728e450>

 * As you can see, the bytecode header is quite long -- it's 12 bytes
   before we get to the meat of the program. (That's 4 for arity, 4 for
   length, and 4 for meta-length -- more on that in a minute). But this
   is OK, because normally this is read-only code mmapped directly from
   disk.
 
 * Originally, when loading programs with meta-data (such as source
   information), we had to load all of that metadata up along with the
   program -- symbols, vectors, conses, etc. So then we hid that loading
   behind a thunk, so we just had to cons up a thunk -- but still that
   was 8 words (4 for the program and 4 for the object code).

   So instead now we just stick the meta-thunk after the main program
   text, and load it only when objcode-meta (or program-meta) is called.
   Voici source information without cost! I stole this trick from the
   Self compiler.

 * Just as we have a tower compilers (and thus languages), we now have a
   tower of /decompilers/. Currently I've only implemented
   value->objcode (only valid for values of type program or objcode),
   objcode->bytecode, and bytecode->assembly, but it's possible to
   implement passes decompiling all the way back to Scheme.

   Or JavaScript! That's the crazy thing: since multiple languages exist
   on top of one substrate, decompilers allow us to do language
   translation -- what Guile originally wanted to do, but as artifact
   rather than as mechanism.

 * Because we put the 4-byte lengths in the objcode directly, and mmap
   that data, bytecode is now endian-specific. Specifically, it's all
   little-endian right now. I know, I know. Worse, it's not aligned. But
   provisions are there to make it aligned and native endian.

***

So, what's up?

Well, things are good. Load time is slightly faster, though we still can
be significantly faster. We cons less than the evaluator. Things are
looking good, and improvable.

I have to fix the endianness/alignment bits.

There are two main regressions. One is a simple bug: backtraces aren't
working right unless you have VM code. I think it's a simple problem
with stack cutting, I have to poke it a bit.

Secondly, GOOPS loads *really slowly*, because of the dynamic
recompilation things that I thought were so clever. I don't know exactly
what to do yet -- profile and see, I guess. I think this is my first
priority right now.

As far as improvements go, there's a laundry list:

  * I'm going to try bytecodes being uint32's instead of uint8's. We'll
    see what the performance impacts are.

  * I'm going to see about coalescing object tables into one vector per
    compilation unit, e.g. file. This should result in faster startup
    time.

  * GOOPS needs some love. I think polymorphic inline caches are the way
    to go, and might be a first way to test out a native-code generator
    (for the cache stubs).

  * It would be nice to have syncase in by default, though perhaps we
    should leave this for R6RS.

  * Decompilers to GLIL, GHIL, and Scheme would be *sweet*.

  * I think there's something publishable in all of this language tower
    business, but I'd need a convincing second high-level language. I
    think JavaScript is the right one. We need to write a compiler to
    GHIL, and probably extend the VM slightly.

    With luck, I'd like to present this fall at the SFP -- any takers
    for help? We could present together :)

Well, that's what's on my mind for now. I'll work at updating the docs
soon, funny to have them bitrot so quickly.

Cheers,

Andy
-- 
http://wingolog.org/

Re: vm status update, a.k.a. yowsers batman it's february already

[Ludovic Courtès]

Hello Andy!

Thanks for all the good news!

Andy Wingo <wingo@pobox.com> writes:

>   * I think there's something publishable in all of this language tower
>     business, but I'd need a convincing second high-level language. I
>     think JavaScript is the right one. We need to write a compiler to
>     GHIL, and probably extend the VM slightly.

Hop [0] has `scm2js', a Scheme to Javascript compiler [1].  There might
be useful ideas or code to borrow from that.

Cheers,
Ludo'.

[0] http://hop.inria.fr/
[1] http://hop.inria.fr/usr/local/share/hop/weblets/home/articles/scheme2js/article.html

Re: vm status update, a.k.a. yowsers batman it's february already

[Neil Jerram]

Andy Wingo <wingo@pobox.com> writes:

>  * Just as we have a tower compilers (and thus languages), we now have a
>    tower of /decompilers/. Currently I've only implemented
>    value->objcode (only valid for values of type program or objcode),
>    objcode->bytecode, and bytecode->assembly, but it's possible to
>    implement passes decompiling all the way back to Scheme.
>
>    Or JavaScript! That's the crazy thing: since multiple languages exist
>    on top of one substrate, decompilers allow us to do language
>    translation -- what Guile originally wanted to do, but as artifact
>    rather than as mechanism.

That does sound interesting.  But it sounds more like something
educational to me, rather than a solution to the original translation
desire.  For example, it would allow us to show people the equivalent
of a given Scheme program in their own preferred language - which
would ultimately be helpful for teaching people Scheme.  And by adding
more sophisticated decompilation passes it might allow us to
demonstrate how a given program can be written (through provable
transformations) in several different ways, and consider which ways
are conventionally considered to be more elegant.

But is that the kind of thing that you had in mind, or have I missed
your point?

Regards,
        Neil

Re: vm status update, a.k.a. yowsers batman it's february already

[Andy Wingo]

On Wed 04 Feb 2009 00:34, Neil Jerram <neil@ossau.uklinux.net> writes:

> [Translation via decompilation] sounds interesting. But it sounds more
> like something educational to me, rather than a solution to the
> original translation desire.

You're right. But it's neat, though!

> Is that the kind of thing that you had in mind, or have I missed your
> point?

You've got it :)

I think that selfishly, I need to implement another language if I'd like
to present something about "guile's multilingual framework", or else I'd
be laughed out of the room. Decompilers aren't necessary but they are
interesting. But the compilers are the important part.

Cheers,

Andy
-- 
http://wingolog.org/

Re: vm status update, a.k.a. yowsers batman it's february already

[Neil Jerram]

Andy Wingo <wingo@pobox.com> writes:

>> Is that the kind of thing that you had in mind, or have I missed your
>> point?
>
> You've got it :)

Hooray!

> I think that selfishly, I need to implement another language if I'd like
> to present something about "guile's multilingual framework", or else I'd
> be laughed out of the room. Decompilers aren't necessary but they are
> interesting. But the compilers are the important part.

Lua was recently mentioned as a relatively small (and so easy)
language to implement...

     Neil