From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Andy Wingo Newsgroups: gmane.lisp.guile.devel Subject: guile 3 update, june 2018 edition Date: Fri, 29 Jun 2018 10:13:56 +0200 Message-ID: <871scq2env.fsf@pobox.com> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: blaine.gmane.org 1530259930 4401 195.159.176.226 (29 Jun 2018 08:12:10 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Fri, 29 Jun 2018 08:12:10 +0000 (UTC) User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) To: guile-devel@gnu.org Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Fri Jun 29 10:12:06 2018 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1fYoVh-000137-HV for guile-devel@m.gmane.org; Fri, 29 Jun 2018 10:12:05 +0200 Original-Received: from localhost ([::1]:40562 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYoXo-0006cw-VA for guile-devel@m.gmane.org; Fri, 29 Jun 2018 04:14:16 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:56035) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fYoXj-0006cp-Jh for guile-devel@gnu.org; Fri, 29 Jun 2018 04:14:13 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fYoXe-0003ZK-JG for guile-devel@gnu.org; Fri, 29 Jun 2018 04:14:11 -0400 Original-Received: from pb-sasl2.pobox.com ([64.147.108.67]:63145 helo=sasl.smtp.pobox.com) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1fYoXe-0003Xc-CF for guile-devel@gnu.org; Fri, 29 Jun 2018 04:14:06 -0400 Original-Received: from sasl.smtp.pobox.com (unknown [127.0.0.1]) by pb-sasl2.pobox.com (Postfix) with ESMTP id CCFEECA082 for ; Fri, 29 Jun 2018 04:14:05 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=from:to :subject:date:message-id:mime-version:content-type; s=sasl; bh=e WJ7tYzYeKzIPFaDNdyC9htlI54=; b=lPU0wNj/wgK7df64Y1g5p3752aHmzeFLU eaeCIegP0onyHm0FPU5pQ5rGeRNqeIIMNygXwUexQcKN0Mjl/7Z+YiFgqeQsWvfb hIhP2AMMoV3ik3uBaFXLc0sJYX8veBeYgCnMOw8C7L7pLiE/PEg7KLcTu0oPX+lo 6RNxZnptbU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=pobox.com; h=from:to:subject :date:message-id:mime-version:content-type; q=dns; s=sasl; b=cfK rDuiFF6+8ip/MaIDACWse5IWaK3LMMm8o19+BRoRuyOSC+e5Sy0EMyo+2pib9pQ+ tgQBWfq0X5EqvneC0xbLEE6/rER0rXwxHEXv5376cnLXMWfnSp8E+cyJsCJ32vDG unpQbjllxlIyWjTnCr9A6qgO0ew5svlHRnSMBMfA= Original-Received: from pb-sasl2.nyi.icgroup.com (unknown [127.0.0.1]) by pb-sasl2.pobox.com (Postfix) with ESMTP id C5518CA081 for ; Fri, 29 Jun 2018 04:14:05 -0400 (EDT) Original-Received: from sparrow (unknown [88.160.190.192]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pb-sasl2.pobox.com (Postfix) with ESMTPSA id E532BCA07E for ; Fri, 29 Jun 2018 04:14:04 -0400 (EDT) X-Pobox-Relay-ID: 638A804E-7B74-11E8-82F7-4B4D894C8D7C-02397024!pb-sasl2.pobox.com X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 64.147.108.67 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.org gmane.lisp.guile.devel:19563 Archived-At: Hi, Just wanted to give an update on Guile 3 developments. Last note was here: https://lists.gnu.org/archive/html/guile-devel/2018-04/msg00004.html The news is that the VM has been completely converted over to call out to the Guile runtime through an "intrinsics" vtable. For some intrinsics, the compiler will emit specialized call-intrinsic opcodes. (There's one of these opcodes for each intrinsic function type.) For others that are a bit more specialized, like the intrinsic used in call-with-prompt, the VM calls out directly to the intrinsic. The upshot is that we're now ready to do JIT compilation. JIT-compiled code will use the intrinsics vtable to embed references to runtime routines. In some future, AOT-compiled code can keep the intrinsics vtable in a register, and call indirectly through that register. My current plan is that the frame overhead will still be two slots: the saved previous FP, and the saved return address. Right now the return address is always a bytecode address. In the future it will be bytecode or native code. Guile will keep a runtime routine marking regions of native code so it can know if it needs to if an RA is bytecode or native code, for debugging reasons; but in most operation, Guile won't need to know. The interpreter will tier up to JIT code through an adapter frame that will do impedance matching over virtual<->physical addresses. To tier down to the interpreter (e.g. when JIT code calls interpreted code), the JIT will simply return to the interpreter, which will pick up state from the virtual IP, SP, and FP saved in the VM state. We do walk the stack from Scheme sometimes, notably when making a backtrace. So, we'll make the runtime translate the JIT return addresses to virtual return addresses in the frame API. To Scheme, it will be as if all things were interpreted. This strategy relies on the JIT being a simple code generator, not an optimizer -- the state of the stack whether JIT or interpreted is the same. We can consider relaxing this in the future. My current problem is knowing when a callee has JIT code. Say you're in JITted function F which calls G. Can you directly jump to G's native code, or is G not compiled yet and you need to use the interpreter? I haven't solved this yet. "Known calls" that use call-label and similar can of course eagerly ensure their callees are JIT-compiled, at compilation time. Unknown calls are the problem. I don't know whether to consider reserving another word in scm_tc7_program objects for JIT code. I have avoided JIT overhead elsewhere and would like to do so here as well! For actual JIT code generation, I think my current plan is to import a copy of GNU lightning into Guile's source, using git-subtree merges. Lightning is fine for our purposes as we only need code generation, not optimization, and it supports lots of architectures: ARM, MIPS, PPC, SPARC, x86 / x86-64, IA64, HPPA, AArch64, S390, and Alpha. Lightning will be built statically into libguile. This has the advantage that we always know the version being used, and we are able to extend lightning without waiting for distros to pick up a new version. Already we will need to extend it to support atomic ops. Subtree merges should allow us to pick up upstream improvements without too much pain. This strategy also allows us to drop lightning in the future if that's the right thing. Basically from the user POV it should be transparent. The whole thing will be behind an --enable-jit / --disable-jit configure option. When it is working we can consider enabling shared lightning usage. Happy hacking, Andy