From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id CJ1bDgxaCmZ5cwAA62LTzQ:P1 (envelope-from ) for ; Mon, 01 Apr 2024 08:54:04 +0200 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id CJ1bDgxaCmZ5cwAA62LTzQ (envelope-from ) for ; Mon, 01 Apr 2024 08:54:04 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=philipmcgrath.com header.s=fm2 header.b=nzbvqnCH; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm2 header.b="R XO9jKM"; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1711954443; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=lOkWb2f2yUR/ntzXVHHd/jU0QFI6mOAOFhreib1oE1E=; b=DmXc2sfGR7ISZteJTerc8NBpeYh7y7d53JpME4RhgZ/mu8AetLSr+f0qMvd9k7+OUETSj8 GC8Zydl5ceXXVuOwKRihPD5Ls0knOB10MufmqaWATFveQyqpMO/4jEg7wAWPIoC1soVGFK F7UWa8a+jfY1/Ck355LBgO1arF6zBnHWJZ76KVA1/CHUHlpoSbbApkTvSDwviYfSq3qCL+ iuKA9PLghTro7S9TsK+S5ZZCknsEyrrWaqt0dWpePcfum4vQRP7kfxMkCW6ikgV6dQdD9I mOvR4UAz4xW2WdeGPQ/3Cuyc4nJsssIyaeEy9xQXMhHkpqPb8qdSNnDtrhCAKg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=philipmcgrath.com header.s=fm2 header.b=nzbvqnCH; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm2 header.b="R XO9jKM"; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Seal: i=1; s=key1; d=yhetil.org; t=1711954443; a=rsa-sha256; cv=none; b=EvKKyVtTyQRuufJmHFriuHUTkGwFWEnrD3BJarwwI5/HIDsHgQQwIkhj7Cr/dhxysZj0+P SV4q16TvRrRYVwGm2LsyxA0QcpPpZghKMcBT7JtVWAEzQmL1+yN3z6UomX27tNhoVc5v58 6aF47cRrhcJMBKL3aL/MZXWIoXvHzuVNefGPHk3RUcBss2EQe9V8di+PPUgaqa6jHRKp9D 2vjV8pV8Rk4OBCglEwk0QWWxCCvtoXDJPaWA8ROD4iTlp/q7lABzUB6XRD8QO8y6+ESqzB /TnbqOpYvpxBC2XIvv8E1QV9IbA/mjhJi2RDEWvkQ1TlgQM9ZaEL0wgeGEIC0A== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id BF77922E54 for ; Mon, 1 Apr 2024 08:54:03 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rrBYQ-00048A-1S; Mon, 01 Apr 2024 02:54:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rrBYN-00047w-OE for guix-patches@gnu.org; Mon, 01 Apr 2024 02:54:00 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rrBYM-00085V-Nu for guix-patches@gnu.org; Mon, 01 Apr 2024 02:53:59 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rrBYP-0003S9-MA; Mon, 01 Apr 2024 02:54:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#70065] [PATCH v2 8/8] gnu: chez-scheme: Bootstrap from source. Resent-From: Philip McGrath Original-Sender: "Debbugs-submit" Resent-CC: philip@philipmcgrath.com, guix-patches@gnu.org Resent-Date: Mon, 01 Apr 2024 06:54:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 70065 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 70065@debbugs.gnu.org Cc: Katherine Cox-Buday , Liliana Marie Prikler , Andrew Tropin , Philip McGrath , Skyler Ferris , Philip McGrath X-Debbugs-Original-Xcc: Philip McGrath Received: via spool by 70065-submit@debbugs.gnu.org id=B70065.171195439113209 (code B ref 70065); Mon, 01 Apr 2024 06:54:01 +0000 Received: (at 70065) by debbugs.gnu.org; 1 Apr 2024 06:53:11 +0000 Received: from localhost ([127.0.0.1]:49404 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rrBXa-0003Qy-77 for submit@debbugs.gnu.org; Mon, 01 Apr 2024 02:53:11 -0400 Received: from fhigh3-smtp.messagingengine.com ([103.168.172.154]:53913) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rrBXY-0003Po-1j for 70065@debbugs.gnu.org; Mon, 01 Apr 2024 02:53:08 -0400 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailfhigh.nyi.internal (Postfix) with ESMTP id E35F91140128; Mon, 1 Apr 2024 02:52:59 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Mon, 01 Apr 2024 02:52:59 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= philipmcgrath.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1711954379; x=1712040779; bh=lOkWb2f2yU R/ntzXVHHd/jU0QFI6mOAOFhreib1oE1E=; b=nzbvqnCHIGZUaRTOJvvutxEGR+ rsm4svciv3tS6b2XJdcAWNqdBQKQxlwmLCLfG1+0FMOijJTwEi2Wv2FlP621ns5j f4dUagFp/jrFoHDlxKI0i84S/lq3F4WCTIJp3fiR/Q2a+I2utCsqKiCcnoAw/xLg dsXhUGbI6dzYfjv49P9maVMqqDD1INhLZoPjXO/SD9SzOsc/TZSInDxDlrBUUfhm EuakAwQ/FAiDCVwJUMnGmNDwSWtZHuQ7NqqBDfx4OWgPtOlS0Y5iMjHXdVT/MxoW HkBJV3/xQ/eblgK7a7fhCRnCxIlacvV3ypmRrjxgLNYlWm0lFNUISOB/xNAw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1711954379; x= 1712040779; bh=lOkWb2f2yUR/ntzXVHHd/jU0QFI6mOAOFhreib1oE1E=; b=R XO9jKMYmie895DIHdPVD5bz6gvHg+xHZ76aYHHtfnpqfskTGFeWR/ZX9P/sVX/ld DJ2bMT2FFlVUJKxFWEZWcub8T5xH2emf84Eiwi8kMUc5B7l7AROuhqRNUtN4RlAg RBJgmXZ4OYurxy3n775lU6eBsh161mTY6SyJi9dgCeIFqZyFaScJYb+EsUlXZPAy rtIIOFHtC3yXzVScTywcpms96eiHYucxrFyIFvoze4A8Cf2oUe9belL4HtLVJpwv bx9NV8xoGz8yplKMtNXrmmehUW/MroaUMhF+BwzA8iZ83mTl4Qh0TZbT4u3Zd9AO cDItvK+1+pl4PjRb74AUQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledruddvledguddugecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enogfuuhhsphgvtghtffhomhgrihhnucdlgeelmdenucfjughrpefhvfevufffkffojghf gggtgfesthekredtredtjeenucfhrhhomheprfhhihhlihhpucfotgfirhgrthhhuceoph hhihhlihhpsehphhhilhhiphhmtghgrhgrthhhrdgtohhmqeenucggtffrrghtthgvrhhn peefjefhtdelffevteelgfeljeeuieduuddvvdduhfeukeetjefhjeeileeuheejvdenuc ffohhmrghinheprhgrtghkvghtqdhlrghnghdrohhrghdpghhithhhuhgsrdhiohdpughi shgtohhurhhsvgdrghhrohhuphenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmh epmhgrihhlfhhrohhmpehphhhilhhiphesphhhihhlihhpmhgtghhrrghthhdrtghomh X-ME-Proxy: Feedback-ID: i2b1146f3:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 1 Apr 2024 02:52:59 -0400 (EDT) From: Philip McGrath Date: Mon, 1 Apr 2024 02:51:13 -0400 Message-ID: <577e7e3c1b367fc752c399d2a7ff140e8c258b71.1711953507.git.philip@philipmcgrath.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Queue-Id: BF77922E54 X-Spam-Score: -2.95 X-Migadu-Spam-Score: -2.95 X-Migadu-Scanner: mx10.migadu.com X-TUID: w3P3TL62D89j * gnu/packages/chez.scm (chez-scheme)[source]: Also unbundle pre-built boot files. (chez-scheme-bootstrap-bootfiles): Stop inheriting from 'chez-scheme'. [native-inputs]: Add 'zuo', 'chez-nanopass-bootstrap', and either 'chez-scheme-for-racket' or, for cross builds, 'chez-scheme'. [build-system]: Change to gnu-build-system. [arguments]: Change to run 'configure' and then bootstrap via `make re.boot` or, for cross builds, `make cross.boot` [synopsis, description]: Update accordingly. (chez-scheme-for-racket-bootstrap-bootfiles)[synopsis]: Write "boot files" as two words in prose. [description]: Likewise. * gnu/packages/racket.scm: Update comments. Change-Id: I55d9b57090ecaca0aa2ab19c2c3ebae2243a9173 --- gnu/packages/chez.scm | 123 +++++++++++++++++++++++++++++++--------- gnu/packages/racket.scm | 24 +++++--- 2 files changed, 112 insertions(+), 35 deletions(-) diff --git a/gnu/packages/chez.scm b/gnu/packages/chez.scm index 4d3f79d9a7..db582fa821 100644 --- a/gnu/packages/chez.scm +++ b/gnu/packages/chez.scm @@ -57,15 +57,54 @@ (define-module (gnu packages chez) ;; Commentary: ;; ;; The bootstrapping paths for Chez Scheme and Racket are closely -;; entwined. Racket CS (the default Racket implementation) is based on (a fork -;; of) Chez Scheme. Racket's variant of Chez Scheme shares sources for -;; nanopass and stex with upstream Chez Scheme. +;; entwined. See "Bootstrapping Racket" in the commentary on "racket.scm" for +;; details on the Racket portion of Chez Scheme's bootstrapping path. ;; -;; Racket's variant of Chez Scheme can be bootstrapped by an older Racket -;; implementation, Racket BC, which can be bootstrapped from C. Porting that -;; code to work with upstream Chez Scheme (or finding an old version that -;; does) is our best hope for some day bootstrapping upstream Chez Scheme from -;; source. +;; Chez Scheme is a self-hosting compiler. A small kernel implemented in C +;; loads "boot files" (in a custom object file format) compiled from the parts +;; of the system implemented in Chez Scheme. (While Chez Scheme generates +;; native machine code, it implements its own linker and loader.) +;; +;; As of Chez Scheme 10.0.0 (and the pre-release versions that preceded it on +;; the Racket branch), there are several ways to obtain boot files: +;; +;; 1. The Racket package "cs-bootstrap" (in the "racket/src/rktboot/" +;; directory of the main Racket Git repository) simulates enough of Chez +;; Scheme to load the Chez Scheme compiler purely from source into Racket +;; and apply the compiler to itself, producing the needed boot files +;; (albeit very slowly). +;; Any variant of Racket since version 7.1 can run the simulation. Using +;; the older Racket BC implementation, which does not depend on Chez +;; Scheme, breaks the dependency cycle. +;; However, the simulation relies on implementation details of Chez +;; Scheme, so a given version of Chez Scheme can only be bootstrapped by +;; the corresponding version of the "cs-bootstrap" package. +;; +;; 2. The Chez Scheme makefile provides a "re.boot" target for bootstrapping +;; via a different version of Chez Scheme (9.5.4 or later). +;; This path manages potential differences in implementation details +;; across Chez Scheme versions using a strategy similar to "cs-bootstrap", +;; but the compatibility shim is maintained with the Chez Scheme source +;; code (in "s/reboot.ss"). Also, it's faster, since less indirection is +;; needed. +;; +;; 3. For cross-compilation, or with an extremely similar Chez Scheme, the +;; makefile provides "cross.boot" and related targets. +;; +;; 4. The Chez Scheme Git repository includes pre-built "pb" (portable +;; bytecode) boot files, which can be used for bootstrapping on any +;; platform, but these binary files are removed from the source Guix uses. +;; +;; Concretely, we use racket-vm-bc to bootstrap chez-scheme-for-racket, which +;; we then use to bootstrap both chez-scheme and racket-vm-cs. +;; +;; In principle, it would be possible instead to use chez-scheme to bootstrap +;; chez-scheme-for-racket. However, since Racket is ultimately used for +;; bootstrapping, chez-scheme would still need to be rebuilt when Racket +;; changes, whereas treating chez-scheme as a leaf avoids having to rebuild +;; Racket when upstream Chez Scheme changes. Furthermore, since "cs-bootstrap" +;; is developed in the Racket source repository, we don't have to look for the +;; version of "cs-bootstrap" compatible with the upstream Chez Scheme release. ;; ;; Code: @@ -479,7 +518,8 @@ (define-public chez-scheme (for-each (lambda (dir) (when (directory-exists? dir) (delete-file-recursively dir))) - '("lz4" + '("boot" + "lz4" "nanopass" "stex" "zlib" @@ -575,10 +615,10 @@ (define-public chez-scheme-for-racket-bootstrap-bootfiles "/opt/racket-vm/bin/racket") "../rktboot/main.rkt")))))))) (home-page "https://pkgs.racket-lang.org/package/cs-bootstrap") - (synopsis "Chez Scheme bootfiles bootstrapped by Racket") - (description "Chez Scheme is a self-hosting compiler: building it -requires ``bootfiles'' containing the Scheme-implemented portions compiled for -the current platform. (Chez can then cross-compile bootfiles for all other + (synopsis "Chez Scheme boot files bootstrapped by Racket") + (description "Chez Scheme is a self-hosting compiler: building it requires +``boot files'' containing the Scheme-implemented portions compiled for the +current platform. (Chez can then cross-compile boot files for all other supported platforms.) The Racket package @code{cs-bootstrap} (part of the main Racket Git @@ -591,28 +631,57 @@ (define-public chez-scheme-for-racket-bootstrap-bootfiles (define-public chez-scheme-bootstrap-bootfiles (package - (inherit chez-scheme) (name "chez-scheme-bootstrap-bootfiles") - (inputs '()) - (native-inputs '()) + (version (package-version chez-scheme)) + (source (package-source chez-scheme)) + (native-inputs (list chez-nanopass-bootstrap + (if (%current-target-system) + chez-scheme + chez-scheme-for-racket) + zuo)) (outputs '("out")) - (build-system copy-build-system) - ;; TODO: cross compilation + (build-system gnu-build-system) (arguments - (list #:install-plan - #~`(("boot/" "lib/chez-scheme-bootfiles")))) - (synopsis "Chez Scheme bootfiles (binary seed)") + (list + #:configure-flags + #~`("--force" ; don't complain about missing bootfiles + "ZLIB=-lz" "LZ4=-llz4" "STEXLIB=/GuixNotUsingStex" ; ignore submods + "ZUO=zuo" + ;; could skip -m= for non-cross non-pbarch builds + #$(string-append "-m=" (or (nix-system->native-chez-machine-type) + (nix-system->pbarch-machine-type)))) + #:make-flags + #~(list (string-append "SCHEME=" + (search-input-file %build-inputs "/bin/scheme")) + #$(if (%current-target-system) + "cross.boot" + "re.boot")) + #:phases + #~(modify-phases %standard-phases + (add-after 'unpack 'unpack-nanopass + #$unpack-nanopass) + (replace 'configure + #$chez-configure) + (delete 'check) + (replace 'install + (lambda args + (mkdir-p (string-append #$output "/lib")) + (copy-recursively + "boot" + (string-append #$output "/lib/chez-scheme-bootfiles"))))))) + (home-page "https://cisco.github.io/ChezScheme/") + (synopsis "Bootstrapped Chez Scheme boot files") (description "Chez Scheme is a self-hosting compiler: building it requires -``bootfiles'' containing the Scheme-implemented portions compiled for the +``boot files'' containing the Scheme-implemented portions compiled for the current platform. (Chez can then cross-compile bootfiles for all other supported platforms.) -This package provides bootstrap bootfiles for upstream Chez Scheme. -Currently, it simply packages the binaries checked in to the upstream -repository. Hopefully we can eventually adapt Racket's @code{cs-bootstrap} to -work with upstream Chez Scheme so that we can bootstrap these files from -source."))) +This package provides boot files for the released version of Chez Scheme +bootstrapped by @code{chez-scheme-for-racket}. Chez Scheme 9.5.4 or any later +version can be used for bootstrapping. Guix ultimately uses the Racket package +@code{cs-bootstrap} to bootstrap its initial version of Chez Scheme.") + (license asl2.0))) ;; ;; Chez's bootstrap dependencies: diff --git a/gnu/packages/racket.scm b/gnu/packages/racket.scm index f50f6f3eab..2b20353159 100644 --- a/gnu/packages/racket.scm +++ b/gnu/packages/racket.scm @@ -72,6 +72,7 @@ (define-module (gnu packages racket) ;; ├── bc/ ;; ├── cs/ ;; ├── ChezScheme/ +;; ├── rktboot/ ;; ├── zuo/ ;; └── ... ;; @@ -149,14 +150,12 @@ (define-module (gnu packages racket) ;; - Racket BC [3M] needs an existing Racket to run "xform", ;; which transforms its own C source code to add additional annotations ;; for the precise garbage collector. -;; - Racket CS needs (bootfiles for) Racket's fork of Chez Scheme. -;; It also needs an existing Racket to compile Racket-implemented +;; - Racket CS needs (boot files for) the corresponding version of Chez +;; Scheme. It also needs an existing Racket to compile Racket-implemented ;; parts of the runtime system to R6RS libraries. -;; - Chez Scheme also needs bootfiles for itself, but Racket can simulate -;; enough of Chez Scheme to load Racket's fork of the Chez Scheme compiler -;; purely from source into Racket and apply the compiler to itself, -;; producing the needed bootfiles (albeit very slowly). -;; Any variant of Racket since version 7.1 can run the simulation. +;; - Chez Scheme also needs boot files for itself, but Racket BC can +;; bootstrap these using the code in "racket/src/rktboot/". +;; See the commentary in "chez.scm" for further details ;; ;; So, we build CGC to build 3M to build bootfiles and CS. ;; @@ -164,6 +163,14 @@ (define-module (gnu packages racket) ;; often use "BC" to mean "3M", consistent with `(banner)` and the ;; suffixes used on executables when more than one variant co-exists.) ;; +;; Since the pre-releases for Chez Scheme 10.0.0, all of Racket's changes have +;; been merged upstream, and development will be kept in sync going +;; forward. However, there is no plan to align the Chez Scheme and Racket +;; release cycles. For the near fulture, a given released version of Racket +;; will continue to depend on a specific pre-release version of Chez Scheme as +;; part of Racket CS's "ABI". See upstream discussion at +;; . +;; ;; One remaining bootstrapping limitation is that Racket's reader, module ;; system, and macro expander are implemented in Racket. For Racket CS, ;; they are compiled to R6RS libraries as discussed above. This note from the @@ -189,7 +196,8 @@ (define-module (gnu packages racket) ;; Zuo is notably *not* a problem for bootstrapping. The implementation is a ;; single hand-written C file designed to build with just `cc -o zuo zuo.c`, ;; even with very old or limited compilers. (We use the Autoconf support for -;; convienience.) +;; convienience.) As of Zuo 1.8, Zuo has tagged releases in its own repository +;; independent of the Racket release cycle. ;; ;; CODE: -- 2.41.0