wip-rtl status

wip-rtl status

[Andy Wingo]

Hi,

A brief status update on wip-rtl.  If this means nothing to you, just
skip this mail; at some point in the future there will be more
information.

I started to look at static constant allocation.  In order to do so I
needed an object format that could link together different sections with
different permissions and alignments, so I imported an old ELF branch I
had hanging around, and adapted it to work with wip-rtl.  Now the
interface is that you make an assembler, emit some programs, then link,
resulting in a list of ELF objects.  You then link-elf to produce a
bytevector, which can be written to disk, then loaded with
load-thunk-from-disk from (system vm objcode).  You can also load via
load-thunk-from-memory, which takes a bytevector.  In that case you
might want to skip alignment and permissions and just operate on
read-write memory, so there is the #:page-aligned? kwarg to link-elf for
that situation.

Instructions that take constants take them literally.  The various
emit-foo procedures will add the constant to a table if necessary, and
link-objects will serialize a constant table if necessary.  Not sure if
this is being too clever or not.

Currently there are a couple of broken bits.  One is that we don't
currently write an init thunk, to fix up the car and cdr links in static
pairs, for example.  Also a lot of it is untested.  But we needed the
loader facility so that we could actually write tests, so hopefully
things will come together.

Anyway, the hack is ongoing, if a bit sporadic.  Hopefully we'll be able
to start compiling some Scheme shortly.

Cheers,

Andy
-- 
http://wingolog.org/

Re: wip-rtl status

[Ludovic Courtès]

Hi Andy,

Woow, this is dense.  ;-)

Andy Wingo <wingo@pobox.com> skribis:

> I started to look at static constant allocation.  In order to do so I
> needed an object format that could link together different sections with
> different permissions and alignments, so I imported an old ELF branch I
> had hanging around, and adapted it to work with wip-rtl.  Now the
> interface is that you make an assembler, emit some programs, then link,
> resulting in a list of ELF objects.  You then link-elf to produce a
> bytevector, which can be written to disk, then loaded with
> load-thunk-from-disk from (system vm objcode).  You can also load via
> load-thunk-from-memory, which takes a bytevector.  In that case you
> might want to skip alignment and permissions and just operate on
> read-write memory, so there is the #:page-aligned? kwarg to link-elf for
> that situation.

Sounds cool!

Forget it if it’s too late, but would it make sense to first work on ELF
as the container format and merge that in ‘master’, then merge wip-rtl
on top?  It would be easier to digest.  ;-)

> Instructions that take constants take them literally.  The various
> emit-foo procedures will add the constant to a table if necessary, and
> link-objects will serialize a constant table if necessary.  Not sure if
> this is being too clever or not.

I suppose the result is the same, whether the constant table is built at
a level equivalent to GLIL or at some lower level.  So it may be more a
question of which one is more easily or elegantly implemented?

> Currently there are a couple of broken bits.  One is that we don't
> currently write an init thunk, to fix up the car and cdr links in static
> pairs, for example.

Static pairs, as constant pairs stored on-disk?

Keep up the good hack, and let us know how it goes!  :-)

Ludo’.

Re: wip-rtl status

[Andy Wingo]

Hi!

On Mon 04 Jun 2012 00:30, ludo@gnu.org (Ludovic Courtès) writes:

> Woow, this is dense.  ;-)

Yeah, sorry about that ;)  It has been difficult to get to a good
stopping point, and without that, I don't know how to organize my
thoughts :)

> Andy Wingo <wingo@pobox.com> skribis:
>
> Forget it if it’s too late, but would it make sense to first work on ELF
> as the container format and merge that in ‘master’, then merge wip-rtl
> on top?  It would be easier to digest.  ;-)

That's possible to do.  I don't think I would merge the bits that
statically allocate constants though, as it would entail changes to the
old VM, and it's probably not worth it.  So I would just throw our
existing bytecode into one section and link that to ELF, and use the new
loaders to load it.  Could be a good idea.


>> Instructions that take constants take them literally.  The various
>> emit-foo procedures will add the constant to a table if necessary, and
>> link-objects will serialize a constant table if necessary.  Not sure if
>> this is being too clever or not.
>
> I suppose the result is the same, whether the constant table is built at
> a level equivalent to GLIL or at some lower level.  So it may be more a
> question of which one is more easily or elegantly implemented?

Yeah, that's basically it.

>> Currently there are a couple of broken bits.  One is that we don't
>> currently write an init thunk, to fix up the car and cdr links in static
>> pairs, for example.
>
> Static pairs, as constant pairs stored on-disk?

Yes.  Consider a constant like '(1 . 2).  The car and the cdr are both
immediates, so you just allocate some space in the image for two words,
aligned on an 8-byte boundary, and you write the `object-address' of the
immediates directly in the words.

However something like '(1 . (2 . ())) has two pairs: the tail which has
two immediates, and the head that has an immediate in the car but a
pointer in the cdr.  In that case you need to patch up the cdr to point
to the tail, after you load the .go (or .so?) file.

The RTL text is position-independent, and the ELF linker tries to
separate read-only data from read/write data, so as to maximize sharing
among different processes.  But PIC data does mean that there will be
some relocations to initialize the internal pointers of static
constants.  That's what the init thunk does (or will do).

I could have emitted relocations into the ELF and had the loader handle
them, but this way we get to do our own kind of initialization, like
initializing immutable values as in string->symbol and string->number.

Andy
-- 
http://wingolog.org/

Re: wip-rtl status

[Ludovic Courtès]

Hi,

Andy Wingo <wingo@pobox.com> skribis:

> However something like '(1 . (2 . ())) has two pairs: the tail which has
> two immediates, and the head that has an immediate in the car but a
> pointer in the cdr.  In that case you need to patch up the cdr to point
> to the tail, after you load the .go (or .so?) file.

Actually, it could typically be patched by the loader, just like what
happens with the variables emitted by ‘SCM_IMMUTABLE_STRING’ and such, no?

(Hmm, perhaps that’s what you meant?)

Thanks,
Ludo’.

Re: wip-rtl status

[Andy Wingo]

On Mon 04 Jun 2012 23:12, ludo@gnu.org (Ludovic Courtès) writes:

> Andy Wingo <wingo@pobox.com> skribis:
>
>> However something like '(1 . (2 . ())) has two pairs: the tail which has
>> two immediates, and the head that has an immediate in the car but a
>> pointer in the cdr.  In that case you need to patch up the cdr to point
>> to the tail, after you load the .go (or .so?) file.
>
> Actually, it could typically be patched by the loader, just like what
> happens with the variables emitted by ‘SCM_IMMUTABLE_STRING’ and such, no?
>
> (Hmm, perhaps that’s what you meant?)

Yes, that is what I meant.  Either you emit relocations into the ELF
file, and rely on the dynamic loader to interpret them, or you compile a
procedure to do the relocations and call it as an init function.
Wip-rtl does the latter, because that way we can also do one-time
initializations like string->symbol, string->number, etc.

Andy
-- 
http://wingolog.org/