From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Andy Wingo Newsgroups: gmane.lisp.guile.devel Subject: a plan for native compilation Date: Fri, 16 Apr 2010 13:09:52 +0200 Message-ID: NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: dough.gmane.org 1271416093 22161 80.91.229.12 (16 Apr 2010 11:08:13 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Fri, 16 Apr 2010 11:08:13 +0000 (UTC) To: guile-devel Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Fri Apr 16 13:08:06 2010 connect(): No such file or directory Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1O2jP2-0003Cf-5p for guile-devel@m.gmane.org; Fri, 16 Apr 2010 13:08:04 +0200 Original-Received: from localhost ([127.0.0.1]:44748 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1O2jP1-0007ir-LC for guile-devel@m.gmane.org; Fri, 16 Apr 2010 07:08:03 -0400 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1O2jOy-0007iR-H2 for guile-devel@gnu.org; Fri, 16 Apr 2010 07:08:00 -0400 Original-Received: from [140.186.70.92] (port=46817 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1O2jOu-0007i5-PE for guile-devel@gnu.org; Fri, 16 Apr 2010 07:07:59 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1O2jOs-0005IY-A6 for guile-devel@gnu.org; Fri, 16 Apr 2010 07:07:56 -0400 Original-Received: from a-pb-sasl-quonix.pobox.com ([208.72.237.25]:45169 helo=sasl.smtp.pobox.com) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1O2jOs-0005IP-5C for guile-devel@gnu.org; Fri, 16 Apr 2010 07:07:54 -0400 Original-Received: from sasl.smtp.pobox.com (unknown [127.0.0.1]) by a-pb-sasl-quonix.pobox.com (Postfix) with ESMTP id 4F5A21439 for ; Fri, 16 Apr 2010 07:07:53 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to :subject:date:message-id:mime-version:content-type; s=sasl; bh=N 3p84v6gY4elee1pGx3AGoYEPNc=; b=MYj4vfy9duSy9vNXtR7OQet2c0tB1P2j8 je1/SuR9Qgv/lfW++4oYMNgRN0DcEJfZ9a/AYgnVB1iLvEAgHHWxNpDALszi6aFt qTz4c5XWItSCTIfvYAghNDP38hTSgKgxrzChejP1g1YxioOypviyAQj42OI/yjYA Z2CiB9Rzxw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=from:to:subject :date:message-id:mime-version:content-type; q=dns; s=sasl; b=K60 P3NyAH4gWgX4v4/ZIXGJIvATsHs/SGP2FcmvogsFe/76ukvOYcYD/5zYqVW/uLcB JqVniONT4guk9r0AfAWV1t0VUQQTpgWuMj15P5L9L7kmOw3MnN4XSyCsFTNZN93T Zcyvh4cuOXJ8sB6CnTmrWeQX8+MR1UHNPiVBH3Dw= Original-Received: from a-pb-sasl-quonix. (unknown [127.0.0.1]) by a-pb-sasl-quonix.pobox.com (Postfix) with ESMTP id 4C5661438 for ; Fri, 16 Apr 2010 07:07:53 -0400 (EDT) Original-Received: from unquote (unknown [83.202.39.53]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by a-pb-sasl-quonix.pobox.com (Postfix) with ESMTPSA id C87501437 for ; Fri, 16 Apr 2010 07:07:52 -0400 (EDT) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.92 (gnu/linux) X-Pobox-Relay-ID: 4D9A36EC-4948-11DF-B11F-D033EE7EF46B-02397024!a-pb-sasl-quonix.pobox.com X-detected-operating-system: by eggs.gnu.org: Solaris 10 (beta) X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.lisp.guile.devel:10234 Archived-At: Hi, I've been thinking some about native compilation, and how to do ahead-of-time and just-in-time compilation, and not totally explode Guile's mental footprint -- the number of cases and things that one has to keep in mind when debugging Guile code. Currently, Guile has a compiler to a custom virtual machine, and the associated toolchain: assemblers and disassemblers, stack walkers, the debugger, etc. One can get the source location of a particular instruction pointer, for example. It's all mostly understandable, and mostly uniform across platforms (modulo word size and byte order, and perhaps we should make that uniform as well, especially if we can do native compilation.) So, my thought is to extend procedures with an additional pointer, a pointer to a "native" code structure. The native code could be written out ahead-of-time, or compiled at runtime. But procedures would still have bytecode, for various purposes for example to enable code coverage via the next-instruction hook, and in the JIT case, because only some procedures will be native-compiled. We keep the same stack representation, so stack walkers and the debugger still work. Some local variables can be allocated into registers, but procedure args are still passed and returned on the stack. Though the procedure's arity and other metadata would be the same, the local variable allocations and source locations would differ, so we would need some additional debugger support, but we can work on that when the time comes. All Scheme procedures, bytecode and native, will run inside the VM. If a bytecode procedure calls a native procedure, the machine registers are saved, and some machine-specific stub transfers control to the native code. Native code calling native code uses the same stack as the VM, though it has its own conventions over what registers to save; and native code calling bytecode prepares the Scheme stack, then restores the VM-saved machine registers. In this way we can do incremental experimentation with compiled procedures, compiling at runtime, or ahead-of-time. Now, what technology to choose for the compiler itself? Dunno. For a JIT, it would be useful to use something portable, and perhaps do the JIT compilation on the bytecode itself, without more source information. It would not produce the fastest code, but it would run fast. AIUI the hotspot compiler actually does an SSA transformation of Java bytecode, then works on that. I'm not particularly interested in something like that; I'm more interested in something direct and fast, and obviously correct and understandable by our debugging infrastructure. I think we can produce better native code ahead-of-time coming from the tree-il layer directly. I feel like eventually we'll need to replace GLIL with something else, but I don't really know; we'll find out in the future I guess. But I do want to do ahead-of-time compilation, because I want Guile programs to start up very quickly and not consume much memory. Anyway, just some thoughts here. I'm not going to focus on native compilation in the coming months, as there are other things to do, but this is how I think it should be done :-) Cheers, Andy -- http://wingolog.org/