Re: Lightning Bindings

[Noah Lavine <noah549@gmail.com>]

Hi Ludo,

I didn't realize guile-lightning existed! It looks like that project already has most of the code for Lightning bindings, so it might be better to try to update it to work with Guile 2.0. I also saw your idea for JIT, which I could work on as well. However, all three projects use different ideas of how Lightning should connect to Guile, so before I code more I would like to talk about which would be better.

Here is my understanding of the three approaches:

The approach in my project was to make machine code a Guile datatype, which you could allocate with a special init function and write to with writing functions which are just Guile versions of the Lightning macros. It could be called as a function through the dynamic FFI.

The approach in the other guile-lightning project is to represent the Lightning code as a Guile list which mirrors the Lightning virtual instruction set. When a list is completely built, it would then be passed to a special function (written in C) to assemble it. It also has some infrastructure for labels and a special method of calling these functions, neither of which I understand yet.

The approach in your plan for JIT, as I understand it, is to implement this completely in the C layer. The machine code would be stored as part of the representation of a procedure, and would be invisible from the Scheme side.

(I should also point out that my plan for compilation was to first start generating machine code with as few inlined instructions as possible, which would just call VM functions to do its work. This was also your plan, and I believe also the plan of the earlier guile-lightning project.)

It is not clear to me which one of these is the best way, or even if there is a best way.

The reason I did not use the approach of the other guile-lightning, to make a list and then assemble it, was that it seemed inelegant and possibly slow to have to iterate through instructions twice whenever I compiled something, first to generate the list and then to compile it. However, I doubt it would be very slow, and thinking about it now it might even be faster if the iteration programs became smaller and fit in cache.

As for doing it all in C, I am concerned about this because if there were bindings available in Scheme, then it might be possible to write a nice compiler in Scheme someday, which would do clever things like inlining and interprocedural optimization. (Or, more easily, persuade the MIT Scheme or Bigloo people to donate their compilers.) Writing it in C could make that more difficult - but if it also made Guile programs faster right now, then it might be worth doing anyway.

What do you think of this? What way should I try to implement this?

Noah