From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms8.migadu.com with LMTPS id 6HEvA51PBmb1gQEA62LTzQ:P1 (envelope-from ) for ; Fri, 29 Mar 2024 06:20:29 +0100 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id 6HEvA51PBmb1gQEA62LTzQ (envelope-from ) for ; Fri, 29 Mar 2024 06:20:29 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=philipmcgrath.com header.s=fm2 header.b=LihfTXCa; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm2 header.b="C EMtTFC"; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1711689628; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=yq81QbPyW8nLfJOKbwkZ0m0ui5wMDlMKyykyox9aK0c=; b=fCiHLCGqaoyT49ehWzBJrmkRxcH5nQd46W3SmKLZ4/eF6/8OgpSk/HqopFeQqgyuT6pAmy 5O+8wEF/eNMmXWxSptGO+453QqjbJSlJq37MexczAAZrBXcrk6MlK96EOg3fL1eALYXnmP 8ljZzhCsWxf8ENaGqVQpkgsVHoHghOJUjV2qM+7gNW2Il96gwWnJffJ9+GfMQVdE4GsLpG 8ctYJt1T+Usl1BhF/TG0t5eOR1amsx+bqhoi9BQw2XN79GG3KK4JikeAg2kwJtbc5LvkkS F87qlTwaapwe6116ZJEVFbQvLa5Wi2rLmg/wAb3qgRraeAo8xhbhbeZXlkfsqw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=philipmcgrath.com header.s=fm2 header.b=LihfTXCa; dkim=fail ("headers rsa verify failed") header.d=messagingengine.com header.s=fm2 header.b="C EMtTFC"; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Seal: i=1; s=key1; d=yhetil.org; t=1711689628; a=rsa-sha256; cv=none; b=AgEetPo3pRk1rGflQr6oXK5xnFg0kbZ4VB1MbEJ2pqGSNl+3jYg7OurDxB6vGu9Ed9TDGu ujtPIkTI2clzYFujPEQ/iUPNNoF1ZSawflRizgwYh9ZSQslkoSsZEtRafW2bjybB1Wqv+m 50n5oS8XG3gfeBz4NJKB67dYjyVVinwbsBjL8Zs39LcLlYOLb070BQZ9v0M0sT7kIBiB0v zUlZyQMeyUgBRT8l29ess1odNdy0nwt2GrasS2hJ3TLTcPZihf6pOk6iRmvJnf3FJkN7W8 JQCWWQjJjdRm3JXZhZVATS0b58jxaEsc+/QX6DaTk+wkebGqEbF5+j9uK8iXzQ== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id D6256C583 for ; Fri, 29 Mar 2024 06:20:27 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rq4f9-0003OV-Pg; Fri, 29 Mar 2024 01:20:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rq4ep-0003JO-7x for guix-patches@gnu.org; Fri, 29 Mar 2024 01:20:03 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rq4eo-00067b-Ty for guix-patches@gnu.org; Fri, 29 Mar 2024 01:20:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1rq4eq-0004zq-02; Fri, 29 Mar 2024 01:20:04 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#70065] [PATCH 6/6] gnu: chez-scheme: Bootstrap from source. Resent-From: Philip McGrath Original-Sender: "Debbugs-submit" Resent-CC: philip@philipmcgrath.com, guix-patches@gnu.org Resent-Date: Fri, 29 Mar 2024 05:20:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 70065 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 70065@debbugs.gnu.org Cc: Philip McGrath X-Debbugs-Original-Xcc: Philip McGrath Received: via spool by 70065-submit@debbugs.gnu.org id=B70065.171168956719114 (code B ref 70065); Fri, 29 Mar 2024 05:20:03 +0000 Received: (at 70065) by debbugs.gnu.org; 29 Mar 2024 05:19:27 +0000 Received: from localhost ([127.0.0.1]:41492 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rq4eE-0004yB-MU for submit@debbugs.gnu.org; Fri, 29 Mar 2024 01:19:27 -0400 Received: from wfout2-smtp.messagingengine.com ([64.147.123.145]:57777) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rq4e3-0004wr-BV for 70065@debbugs.gnu.org; Fri, 29 Mar 2024 01:19:16 -0400 Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfout.west.internal (Postfix) with ESMTP id 8B5A71C000A5 for <70065@debbugs.gnu.org>; Fri, 29 Mar 2024 01:19:08 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Fri, 29 Mar 2024 01:19:08 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= philipmcgrath.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to; s=fm2; t=1711689548; x=1711775948; bh=yq81QbPyW8nLfJOKbwkZ0 m0ui5wMDlMKyykyox9aK0c=; b=LihfTXCaRsJSQ7RUdQzS+EIY14PoLigyehupJ t3Dg02EPx1CllamBmI+22sqmQ4ZWuie90pgfWoghhHnMayEOeqmaUpb6YR0gr7eF YqqTcLRTwUa7B7fdVQUR1LCoqaBmaBE8PKy/AtC+VJejv5ZvgDoMhtCqvly8I9NN 3YzM+aRXU1P+0tj/JXogLqkvI97z3ptBxnDAS0jZnasSuOdzCt1+SD+14TBpUhjS wrQ8i5fF60WgwLa0OB3kSn7HmG7i+wn/VoWkoDQm5yANc1m/NKVc1i9EW07XeGHK m/4zlXGU07VCNFiiks6PCbCgdmt2dpGil1IKo6Q8tIWYNJjMg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1711689548; x= 1711775948; bh=yq81QbPyW8nLfJOKbwkZ0m0ui5wMDlMKyykyox9aK0c=; b=C EMtTFCY8elzHffX254B+2SHKnqLZCF6jzTFzKn9nZr0jp1hcBgjZ0uZ436BPc5do Ghw7CxTHx9PsWO1DUTVlrGV24goc+rak0BhQbwL74qL7SctGBiFENanH90xkes5S sfp53XFZVNRNoJQ+WSPzt7D3mwS23eq8pTKsmdNv1jr8Woks4bDTcBUPgpB5yBED 7bbkaZsLlR3KnS2CjkVNpvB3LcQ5BRVae2mAI+KWzzHnm4cyA7H8YrTU15dKOB0f yxVPKA+Asd+ZC9pDILKG6gfLgpKxOsEivlURNKt7F38x3bqbxRAORzIV6KIjUuay wMT8xcLfZQgOZbLQCwxCg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledruddvtddgfeeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucgoufhushhpvggtthffohhmrghinhculdegledmne cujfgurhephffvufffkffojghfgggtgfesthekredtredtjeenucfhrhhomheprfhhihhl ihhpucfotgfirhgrthhhuceophhhihhlihhpsehphhhilhhiphhmtghgrhgrthhhrdgtoh hmqeenucggtffrrghtthgvrhhnpeeggeeuvdeuieffgeeigfegkefgfffgjeegleeghfeh fffgkeeuueffhffgieehieenucffohhmrghinheprhgrtghkvghtqdhlrghnghdrohhrgh dpghhithhhuhgsrdhiohdpughishgtohhurhhsvgdrghhrohhuphenucevlhhushhtvghr ufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehphhhilhhiphesphhhihhlih hpmhgtghhrrghthhdrtghomh X-ME-Proxy: Feedback-ID: i2b1146f3:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for <70065@debbugs.gnu.org>; Fri, 29 Mar 2024 01:19:07 -0400 (EDT) From: Philip McGrath Date: Fri, 29 Mar 2024 01:18:55 -0400 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Queue-Id: D6256C583 X-Spam-Score: -4.45 X-Migadu-Spam-Score: -4.45 X-Migadu-Scanner: mx10.migadu.com X-TUID: KGeCx9Dlalvv * gnu/packages/chez.scm (chez-scheme)[source]: Also unbundle pre-built boot files. (chez-scheme-bootstrap-bootfiles): Stop inheriting from 'chez-scheme'. [native-inputs]: Add 'zuo', 'chez-nanopass-bootstrap', and either 'chez-scheme-for-racket' or, for cross builds, 'chez-scheme'. [build-system]: Change to gnu-build-system. [arguments]: Change to run 'configure' and then bootstrap via `make re.boot` or, for cross builds, `make cross.boot` [synopsis, description]: Update accordingly. (chez-scheme-for-racket-bootstrap-bootfiles)[synopsis]: Write "boot files" as two words in prose. [description]: Likewise. * gnu/packages/racket.scm: Update comments. Change-Id: I55d9b57090ecaca0aa2ab19c2c3ebae2243a9173 --- gnu/packages/chez.scm | 123 +++++++++++++++++++++++++++++++--------- gnu/packages/racket.scm | 24 +++++--- 2 files changed, 112 insertions(+), 35 deletions(-) diff --git a/gnu/packages/chez.scm b/gnu/packages/chez.scm index 3f9bdbfbfc..11de3b9523 100644 --- a/gnu/packages/chez.scm +++ b/gnu/packages/chez.scm @@ -57,15 +57,54 @@ (define-module (gnu packages chez) ;; Commentary: ;; ;; The bootstrapping paths for Chez Scheme and Racket are closely -;; entwined. Racket CS (the default Racket implementation) is based on (a fork -;; of) Chez Scheme. Racket's variant of Chez Scheme shares sources for -;; nanopass and stex with upstream Chez Scheme. +;; entwined. See "Bootstrapping Racket" in the commentary on "racket.scm" for +;; details on the Racket portion of Chez Scheme's bootstrapping path. ;; -;; Racket's variant of Chez Scheme can be bootstrapped by an older Racket -;; implementation, Racket BC, which can be bootstrapped from C. Porting that -;; code to work with upstream Chez Scheme (or finding an old version that -;; does) is our best hope for some day bootstrapping upstream Chez Scheme from -;; source. +;; Chez Scheme is a self-hosting compiler. A small kernel implemented in C +;; loads "boot files" (in a custom object file format) compiled from the parts +;; of the system implemented in Chez Scheme. (While Chez Scheme generates +;; native machine code, it implements its own linker and loader.) +;; +;; As of Chez Scheme 10.0.0 (and the pre-release versions that preceded it on +;; the Racket branch), there are several ways to obtain boot files: +;; +;; 1. The Racket package "cs-bootstrap" (in the "racket/src/rktboot/" +;; directory of the main Racket Git repository) simulates enough of Chez +;; Scheme to load the Chez Scheme compiler purely from source into Racket +;; and apply the compiler to itself, producing the needed boot files +;; (albeit very slowly). +;; Any variant of Racket since version 7.1 can run the simulation. Using +;; the older Racket BC implementation, which does not depend on Chez +;; Scheme, breaks the dependency cycle. +;; However, the simulation relies on implementation details of Chez +;; Scheme, so a given version of Chez Scheme can only be bootstrapped by +;; the corresponding version of the "cs-bootstrap" package. +;; +;; 2. The Chez Scheme makefile provides a "re.boot" target for bootstrapping +;; via a different version of Chez Scheme (9.5.4 or later). +;; This path manages potential differences in implementation details +;; across Chez Scheme versions using a strategy similar to "cs-bootstrap", +;; but the compatibility shim is maintained with the Chez Scheme source +;; code (in "s/reboot.ss"). Also, it's faster, since less indirection is +;; needed. +;; +;; 3. For cross-compilation, or with an extremely similar Chez Scheme, the +;; makefile provides "cross.boot" and related targets. +;; +;; 4. The Chez Scheme Git repository includes pre-built "pb" (portable +;; bytecode) boot files, which can be used for bootstrapping on any +;; platform, but these binary files are removed from the source Guix uses. +;; +;; Concretely, we use racket-vm-bc to bootstrap chez-scheme-for-racket, which +;; we then use to bootstrap both chez-scheme and racket-vm-cs. +;; +;; In principle, it would be possible instead to use chez-scheme to bootstrap +;; chez-scheme-for-racket. However, since Racket is ultimately used for +;; bootstrapping, chez-scheme would still need to be rebuilt when Racket +;; changes, whereas treating chez-scheme as a leaf avoids having to rebuild +;; Racket when upstream Chez Scheme changes. Furthermore, since "cs-bootstrap" +;; is developed in the Racket source repository, we don't have to look for the +;; version of "cs-bootstrap" compatible with the upstream Chez Scheme release. ;; ;; Code: @@ -476,7 +515,8 @@ (define-public chez-scheme (for-each (lambda (dir) (when (directory-exists? dir) (delete-file-recursively dir))) - '("lz4" + '("boot" + "lz4" "nanopass" "stex" "zlib" @@ -572,10 +612,10 @@ (define-public chez-scheme-for-racket-bootstrap-bootfiles "/opt/racket-vm/bin/racket") "../rktboot/main.rkt")))))))) (home-page "https://pkgs.racket-lang.org/package/cs-bootstrap") - (synopsis "Chez Scheme bootfiles bootstrapped by Racket") - (description "Chez Scheme is a self-hosting compiler: building it -requires ``bootfiles'' containing the Scheme-implemented portions compiled for -the current platform. (Chez can then cross-compile bootfiles for all other + (synopsis "Chez Scheme boot files bootstrapped by Racket") + (description "Chez Scheme is a self-hosting compiler: building it requires +``boot files'' containing the Scheme-implemented portions compiled for the +current platform. (Chez can then cross-compile boot files for all other supported platforms.) The Racket package @code{cs-bootstrap} (part of the main Racket Git @@ -588,28 +628,57 @@ (define-public chez-scheme-for-racket-bootstrap-bootfiles (define-public chez-scheme-bootstrap-bootfiles (package - (inherit chez-scheme) (name "chez-scheme-bootstrap-bootfiles") - (inputs '()) - (native-inputs '()) + (version (package-version chez-scheme)) + (source (package-source chez-scheme)) + (native-inputs (list chez-nanopass-bootstrap + (if (%current-target-system) + chez-scheme + chez-scheme-for-racket) + zuo)) (outputs '("out")) - (build-system copy-build-system) - ;; TODO: cross compilation + (build-system gnu-build-system) (arguments - (list #:install-plan - #~`(("boot/" "lib/chez-scheme-bootfiles")))) - (synopsis "Chez Scheme bootfiles (binary seed)") + (list + #:configure-flags + #~`("--force" ; don't complain about missing bootfiles + "ZLIB=-lz" "LZ4=-llz4" "STEXLIB=/GuixNotUsingStex" ; ignore submods + "ZUO=zuo" + ;; could skip -m= for non-cross non-pbarch builds + #$(string-append "-m=" (or (nix-system->native-chez-machine-type) + (nix-system->pbarch-machine-type)))) + #:make-flags + #~(list (string-append "SCHEME=" + (search-input-file %build-inputs "/bin/scheme")) + #$(if (%current-target-system) + "cross.boot" + "re.boot")) + #:phases + #~(modify-phases %standard-phases + (add-after 'unpack 'unpack-nanopass + #$unpack-nanopass) + (replace 'configure + #$chez-configure) + (delete 'check) + (replace 'install + (lambda args + (mkdir-p (string-append #$output "/lib")) + (copy-recursively + "boot" + (string-append #$output "/lib/chez-scheme-bootfiles"))))))) + (home-page "https://cisco.github.io/ChezScheme/") + (synopsis "Bootstrapped Chez Scheme boot files") (description "Chez Scheme is a self-hosting compiler: building it requires -``bootfiles'' containing the Scheme-implemented portions compiled for the +``boot files'' containing the Scheme-implemented portions compiled for the current platform. (Chez can then cross-compile bootfiles for all other supported platforms.) -This package provides bootstrap bootfiles for upstream Chez Scheme. -Currently, it simply packages the binaries checked in to the upstream -repository. Hopefully we can eventually adapt Racket's @code{cs-bootstrap} to -work with upstream Chez Scheme so that we can bootstrap these files from -source."))) +This package provides boot files for the released version of Chez Scheme +bootstrapped by @code{chez-scheme-for-racket}. Chez Scheme 9.5.4 or any later +version can be used for bootstrapping. Guix ultimately uses the Racket package +@code{cs-bootstrap} to bootstrap its initial version of Chez Scheme.") + (license asl2.0))) ;; ;; Chez's bootstrap dependencies: diff --git a/gnu/packages/racket.scm b/gnu/packages/racket.scm index f50f6f3eab..2b20353159 100644 --- a/gnu/packages/racket.scm +++ b/gnu/packages/racket.scm @@ -72,6 +72,7 @@ (define-module (gnu packages racket) ;; ├── bc/ ;; ├── cs/ ;; ├── ChezScheme/ +;; ├── rktboot/ ;; ├── zuo/ ;; └── ... ;; @@ -149,14 +150,12 @@ (define-module (gnu packages racket) ;; - Racket BC [3M] needs an existing Racket to run "xform", ;; which transforms its own C source code to add additional annotations ;; for the precise garbage collector. -;; - Racket CS needs (bootfiles for) Racket's fork of Chez Scheme. -;; It also needs an existing Racket to compile Racket-implemented +;; - Racket CS needs (boot files for) the corresponding version of Chez +;; Scheme. It also needs an existing Racket to compile Racket-implemented ;; parts of the runtime system to R6RS libraries. -;; - Chez Scheme also needs bootfiles for itself, but Racket can simulate -;; enough of Chez Scheme to load Racket's fork of the Chez Scheme compiler -;; purely from source into Racket and apply the compiler to itself, -;; producing the needed bootfiles (albeit very slowly). -;; Any variant of Racket since version 7.1 can run the simulation. +;; - Chez Scheme also needs boot files for itself, but Racket BC can +;; bootstrap these using the code in "racket/src/rktboot/". +;; See the commentary in "chez.scm" for further details ;; ;; So, we build CGC to build 3M to build bootfiles and CS. ;; @@ -164,6 +163,14 @@ (define-module (gnu packages racket) ;; often use "BC" to mean "3M", consistent with `(banner)` and the ;; suffixes used on executables when more than one variant co-exists.) ;; +;; Since the pre-releases for Chez Scheme 10.0.0, all of Racket's changes have +;; been merged upstream, and development will be kept in sync going +;; forward. However, there is no plan to align the Chez Scheme and Racket +;; release cycles. For the near fulture, a given released version of Racket +;; will continue to depend on a specific pre-release version of Chez Scheme as +;; part of Racket CS's "ABI". See upstream discussion at +;; . +;; ;; One remaining bootstrapping limitation is that Racket's reader, module ;; system, and macro expander are implemented in Racket. For Racket CS, ;; they are compiled to R6RS libraries as discussed above. This note from the @@ -189,7 +196,8 @@ (define-module (gnu packages racket) ;; Zuo is notably *not* a problem for bootstrapping. The implementation is a ;; single hand-written C file designed to build with just `cc -o zuo zuo.c`, ;; even with very old or limited compilers. (We use the Autoconf support for -;; convienience.) +;; convienience.) As of Zuo 1.8, Zuo has tagged releases in its own repository +;; independent of the Racket release cycle. ;; ;; CODE: -- 2.41.0