From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Noah Lavine Newsgroups: gmane.lisp.guile.devel Subject: Re: Lightning Bindings Date: Tue, 1 Jun 2010 10:55:06 -0400 Message-ID: References: NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 X-Trace: dough.gmane.org 1275404142 29722 80.91.229.12 (1 Jun 2010 14:55:42 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Tue, 1 Jun 2010 14:55:42 +0000 (UTC) Cc: guile-devel@gnu.org To: Andy Wingo Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Tue Jun 01 16:55:37 2010 connect(): No such file or directory Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1OJSsM-0006Kb-HR for guile-devel@m.gmane.org; Tue, 01 Jun 2010 16:55:31 +0200 Original-Received: from localhost ([127.0.0.1]:43762 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OJSsL-0008H7-Mm for guile-devel@m.gmane.org; Tue, 01 Jun 2010 10:55:29 -0400 Original-Received: from [140.186.70.92] (port=52534 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OJSs3-0008DL-RS for guile-devel@gnu.org; Tue, 01 Jun 2010 10:55:15 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OJSrz-0001yt-So for guile-devel@gnu.org; Tue, 01 Jun 2010 10:55:11 -0400 Original-Received: from mail-gw0-f41.google.com ([74.125.83.41]:37268) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OJSrz-0001yh-LF for guile-devel@gnu.org; Tue, 01 Jun 2010 10:55:07 -0400 Original-Received: by gwb15 with SMTP id 15so913375gwb.0 for ; Tue, 01 Jun 2010 07:55:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type; bh=eJYuv1bvD2ddwy4e7cnZ9OHR4Yh7iP1uxyIZapu6qY0=; b=Dx6qwC2qhllU7Hy3/NkMb1Buogz/w088rX1e7YdNJQBf+ZEJYh+gk+j1UYhiP8GClJ qSuq9W+FIM498NJVdbSrbReFcDYGfj///QxsHsdj9DggFpNYoTaHdZCEqrlbK50TEo1H 9G/zn+kMQ7Z1SRwsqmJcpOhOJM5HkGDQGrUgQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=nHHPqiYYMWR29ZZEIfGV+fhmcdpDFHwcSC6ZIsqYWC0p5+Tef826oGVYCh72+EJ/sV 8XlQgUklTEXPNjmUyF5Nek5TLgCigfR6G4U+pP8KXE3AXDgixl0d/AgGWxHtHBj2zMJS Fuj2DpPTSIyMD+eTaK3W+SqWEuthannHBFgT8= Original-Received: by 10.231.147.18 with SMTP id j18mr7974731ibv.12.1275404106631; Tue, 01 Jun 2010 07:55:06 -0700 (PDT) Original-Received: by 10.231.32.13 with HTTP; Tue, 1 Jun 2010 07:55:06 -0700 (PDT) In-Reply-To: X-Google-Sender-Auth: 6rmh-aDYJAbSABuztBt9-ukDIv4 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:10411 Archived-At: On Tue, Jun 1, 2010 at 5:06 AM, Andy Wingo wrote: > Hi Noah, Hi Andy! > I feel quite strongly that the calling convention for native code should > be the same as that of bytecode, and that it should use the same stack. > This way we don't have to maintain separate stack walkers or debuggers > or the like. Also this way we get proper tail calls and multiple value > handling as well. That makes a lot of sense. > I tend to think that Ludovic's proposal, or something like it, is the > most practical means to get a portable JIT compiler going; though I > don't really know. You may be right; although I think actually with the FFI, it would be just as easy to get it going in Scheme. According to the docs, right now Scheme code can access any function in the Guile binary - including the functions that implement the VM instructions. I was planning to just use that capability to generate machine code that called those C functions. I just thought of another thing that would be good to think about. I think we might ultimately not want to be generating machine code for each procedure, because that would make it impossible to do tracing optimizations like current JavaScript engines (if I understand the proposal right), and also because that might lead to generating a lot of machine code for non-bottleneck points just to get at one bottleneck. I don't think this is something we should try to implement now, but it'd be nice if the code generation infrastructure was flexible enough that it could be added on later. That also raises another point I had forgotten about. Your earlier thread asked how to get native code generation for Guile in such a way that it didn't make Guile too big to understand. I think a good choice would be to *not* put code generation in the core of Guile. Instead, make a module you can load that will generate native code. Maybe it would offer an alternate language, "compiled-guile", or something. It doesn't even have to be distributed with the Guile libraries, although it probably would be. This has two advantages - first, it makes core Guile small. Perhaps people who do embedded systems would appreciate that, or who want to embed Guile in their programs and care more about small code size than incredible speed. Second, it gives you the chance to have multiple native code backends at once. For instance, a JIT and an AOT backend. All you really need for this is a way for core Guile to call native code, which you now have with the FFI. > In the end though Guile needs to be generating native code > ahead-of-time, I think. Doing that portably is hard; we'll need to > leverage some other project. I think that project should be GCC. I was > talking to Dodji Seketeli the other day, a GCC dev, and he says that GCC > maintainers are willing to allow other programs to use it for code > generation, but they don't know yet what abstractions they need to > provide. Oh, that's excellent! When I was looking at options earlier, I thought that GCC would be the ideal, but it would be too hard to use their stuff. If they're willing to work on this, then probably that would be best. > I would be very interested in your observations, yes :) Well, here are some thoughts on existing work that we might be able to reuse: Nanojit - from Mozilla and Adobe. C++ library for code generation. Has optional optimization passes, but could be used without them. Probably good, but extremely poorly documented. SpiderMonkey - the JavaScript runtime from Firefox, written in C++. In order to be useful, you'd probably have to take out most of their primitive operations and replace them with your own. You'd also have to understand their garbage collection and figure out how it interfaced with Guile's. The upside would be potentially extremely fast code generation, since you'd get the tracing JIT and the fast native code generation. The downside would be a lot of complexity. Has an LGPL license (among others). GNU Lightning - a very simple native code generator, written in C. The upside would be its extreme simplicity. The downside would be the apparently total lack of optimization. GCC - the ideal. It has the best AOT compilation of everything listed here, but it'd need work to use it. I think the biggest obstacle is that GCC assumes everything is written to and from a file. You'd need to change that to use it as a JIT backend. (The biggest issue is that the compiler and assembler are only connected through files.) Other than that, though, it shouldn't be too bad to wrap its C interfaces with the FFI. (Possible idea: get the GCC developers working with us by offering to replace GCC MELT with Guile if they'll make nicer backend interfaces. Everyone would win.) PLT Scheme - contains MzScheme, an LGPL-licensed Scheme interpreter with a JIT backend. Their backend is just a hacked version of Lightning, so it might not be much faster than what we would have with our own Lightning backend, but it could be possible to reuse their code. Unfortunately, the code looks pretty complicated. MIT/GNU Scheme or Bigloo Scheme - two GPL-licensed Scheme implementations with native code generators. MIT Scheme is quite similar to Guile, so it wouldn't be too terrible to implement it as a language on top of Guile and get their code that way (also Edwin, their Emacs clone written in Scheme, which might be neat). The project is also so inactive that they might be willing to merge with Guile, which could be good. Bigloo Scheme might be harder to interface with, but also has an AOT code generation backend that can rival GCC, so that could ultimately have impressive results. > So first we should probably implement a native code compiler directly > ourselves, for one architecture, and see how that experience changes our > internal language barriers and then go to GCC with a set of > requirements. That makes a lot of sense. > (Since I know the question will come up, possibilities are numerous -- > basically we want to emit something on the RTL level, I think, though > I'm sure there will be much back-and-forth here; and licensing-wise, it > might involve a build server (already planned), or a plugin, or a > library. It will take a couple years I think.) Yes, the license is one thing I'm unsure about. There is a lot of good GPL-licensed code for doing what we want to do, and it would be nice to be able to reuse it. Thanks! Noah