native compilers

native compilers

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 1927 bytes --]

Dear guilers,

I've now coded two version of native compilers for x86-64 linux compiling
either
the old guile-2.0 VM language or guile-2.2 RTL VM language.

I've coded all but the halt rtl instructions and tested most of them by
handwriting rtl instructions.
The guile-2.0 VM is mostly covered and is in  a playable state
There will be more to do especially to test and be able to run call/cc and
prompts in this newly
created language.

I've tested the code and it looks like RTL code has faster function
dispatch then old VM and that
the native one is on par with RTL function dispatch. On the other hand the
meat inside the functions
will run maybe 3x faster than RTL VM code. I added in inline assembler VM
instruction that communicates
to the native compiler to output pure assembler code which means that some
parts can be as fast as ity possibly
can. and you will gain maybe another 5x because of using machine registers
and machine data-structures like int and
doubles.

How to play with these
You may download https://gitorious.org/aschm
There is a RTL branch targeting guile-2.2 wip-rtl
And the master if for guile-2.0

Go the C code parts and make sure that you can compile them.
Look in the guile directory there are some files that needs to be merged to
the same file in the main guile repo
or wip-rtl repo. then compile the pathced guile.

look for test directories to see how to compile, for the rtl branch look in
rtl.scm

Play and have fun.

I feel that I would like to wait for other people to make sure scheme
programs can be compiled to the rtl vm
and also discuss how to proceed and how to solve some technical issues that
may remains
before continuing this work.

A crazy idea I had was to allow for people to code inline C-code in stead
of inline assembly or maybe some schemier
version of C code in order to be more productive for these tasks. But I
really don't know yet how to approach this.

/Stefan

[-- Attachment #2: Type: text/html, Size: 2120 bytes --]

Re: native compilers

[Andy Wingo]

On Sat 22 Sep 2012 23:28, Stefan Israelsson Tampe <stefan.itampe@gmail.com> writes:

> I've now coded two version of native compilers for x86-64 linux
> compiling either the old guile-2.0 VM language or guile-2.2 RTL VM
> language.

This is pretty crazy and wonderful stuff, Stefan.

> https://gitorious.org/aschm 

https://gitorious.org/aschm/aschm/blobs/rtl/module/native/vm/inst.scm
looks really promising.

I guess I need to consolidate the RTL branch now, and we need to make
sure that we can plug in a JIT.  I don't want to incorporate all of this
code at once, so ideally we can make it so that you can load your code
as a module and Guile will have the needed hooks to run JITted code if
it is there.

Excellent hacking!

Andy
-- 
http://wingolog.org/

Re: native compilers

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 2447 bytes --]

Yeah, this is pretty crazy stuff. But crazy fun stuff!

1. Anyway I think that I could just do away with two table lookups to reach
both
c-function pointers and the goto gosub stuff generated from assembler.

2. A think more work is needed of the extra layer to allow for correct
continuation
properties.

3. Would be good if we could just pass a pointer to the rtl-vm local data
in stead of copying
them into a third vm.

4. Good register handling is needed, right now this is the most simplest
approach you can have
towards native / JIT compiling e.g. take the rtl code and transform it
directly to rtl code. you do get
faster code but it would be nice to actually take advantage of hardware
regiosters. And for that
we would need ideally to compile from something else than rtl bytecode.
Anyway I could imagine an
intrmediate state of guile where we took this simplistic approach to
compiling - it should at least
be very fast to compile the code and if we look at it as some speedier VM
it's actually not a bad idea
appart from beeing hard to maintain with respect to hardware architecture.

5. To simplify further we could design most of the meaty stuff in C-code
and restrict the target registers
for this code and hook everything together with some basic assembler. Then
I don't expect the assembler
work to be that hefty.

6. As Noha later on in the list point out, there is already JIT engines out
there that can work with this without the
extra overhead. But I'm so in love with assembler in scheme that I couldn't
resist hacking this together.

Regards
Stefan



On Mon, Jan 21, 2013 at 5:53 PM, Andy Wingo <wingo@pobox.com> wrote:

> On Sat 22 Sep 2012 23:28, Stefan Israelsson Tampe <stefan.itampe@gmail.com>
> writes:
>
> > I've now coded two version of native compilers for x86-64 linux
> > compiling either the old guile-2.0 VM language or guile-2.2 RTL VM
> > language.
>
> This is pretty crazy and wonderful stuff, Stefan.
>
> > https://gitorious.org/aschm
>
> https://gitorious.org/aschm/aschm/blobs/rtl/module/native/vm/inst.scm
> looks really promising.
>
> I guess I need to consolidate the RTL branch now, and we need to make
> sure that we can plug in a JIT.  I don't want to incorporate all of this
> code at once, so ideally we can make it so that you can load your code
> as a module and Guile will have the needed hooks to run JITted code if
> it is there.
>
> Excellent hacking!
>
> Andy
> --
> http://wingolog.org/
>

[-- Attachment #2: Type: text/html, Size: 3686 bytes --]

Re: native compilers

[Stefan Israelsson Tampe]

[-- Attachment #1: Type: text/plain, Size: 569 bytes --]

I guess I need to consolidate the RTL branch now, and we need to make
> sure that we can plug in a JIT.  I don't want to incorporate all of this
> code at once, so ideally we can make it so that you can load your code
> as a module and Guile will have the needed hooks to run JITted code if
> it is there.
>

Yes, We might want to use another asssember, e.g. If we wouldl ike to use
a more advenced one with more features it should be possible to hook it in.
In the end we just will support some smaller subset of the instruction set
which
we ship with guile.

/Stefan

[-- Attachment #2: Type: text/html, Size: 913 bytes --]