From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id aO9JMYhfDWJ3UAEAgWs5BA (envelope-from ) for ; Wed, 16 Feb 2022 21:33:12 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id SNi6LohfDWL3LQAA9RJhRA (envelope-from ) for ; Wed, 16 Feb 2022 21:33:12 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 30CE74712D for ; Wed, 16 Feb 2022 21:33:12 +0100 (CET) Received: from localhost ([::1]:32922 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nKQz8-00013L-OB for larch@yhetil.org; Wed, 16 Feb 2022 15:33:10 -0500 Received: from eggs.gnu.org ([209.51.188.92]:59306) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nKQT4-0003Ld-JO for guix-patches@gnu.org; Wed, 16 Feb 2022 15:00:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:55652) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nKQT4-00088g-3N for guix-patches@gnu.org; Wed, 16 Feb 2022 15:00:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nKQT3-0001qp-WA for guix-patches@gnu.org; Wed, 16 Feb 2022 15:00:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#53878] [PATCH 08/11] gnu: Add chez-scheme-for-racket. Resent-From: Philip McGrath Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 16 Feb 2022 20:00:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 53878 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Liliana Marie Prikler , 53878@debbugs.gnu.org Received: via spool by 53878-submit@debbugs.gnu.org id=B53878.16450415437020 (code B ref 53878); Wed, 16 Feb 2022 20:00:01 +0000 Received: (at 53878) by debbugs.gnu.org; 16 Feb 2022 19:59:03 +0000 Received: from localhost ([127.0.0.1]:49549 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nKQS6-0001p9-LQ for submit@debbugs.gnu.org; Wed, 16 Feb 2022 14:59:03 -0500 Received: from mail-qv1-f50.google.com ([209.85.219.50]:41694) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nKQS4-0001oe-B6 for 53878@debbugs.gnu.org; Wed, 16 Feb 2022 14:59:01 -0500 Received: by mail-qv1-f50.google.com with SMTP id x3so4016336qvd.8 for <53878@debbugs.gnu.org>; Wed, 16 Feb 2022 11:59:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=philipmcgrath.com; s=google; h=message-id:date:mime-version:user-agent:from:subject:to:references :content-language:in-reply-to:content-transfer-encoding; bh=D0Qnabzo3meV014AhERnKC4+t6IBAVfftZ9KphyrR/M=; b=UHwn9Tzf6CGrmbe1euNV62igp4GAqsnWXTM7LLcdMj88hYaQ5i4UG2xw1AJdUCfAET iFQH3xivjuCNX07F7valTtHAgE3fFKv/yQTrG1Td6ax7lq6O9L0jtSye8vvChqahtqTS 8r1NTNQrjq3UfMrKUVlrfuwalkq2aObDHj+zXCgQo/oh6kY7ma62MgKYZ6JQlpo8AaK1 acKiafNUGiNZtkly7d+JyME8kg3rxOugcqlD/nDXY2E4ikG6zx3d+tgab9GQXz6IU8I6 6cxEmO8hOzMz6RPs0uD6kzEjixA4Iq3dF4S3vIzThRWs8dwr5G7+e7bMQu84D3gCL1v2 LyHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:from :subject:to:references:content-language:in-reply-to :content-transfer-encoding; bh=D0Qnabzo3meV014AhERnKC4+t6IBAVfftZ9KphyrR/M=; b=pB2TC/uyAja8lCTTKg7eI9bgZr13xSpYY31Mfce1LAKr0mzeN0VxKv9L7hdtHTFGiY QspVSEflXE90MDzJeHQNaa62uv/WhCTo0E2AS4TgQaXQUVGRxXbtaLHIJhTThBlcYS8e fdQSPl0w/xGCYd59MBj2Rz/zJPqDESn8TiG4mQM9rHVFnjPfHLVBW3Z+27NS9n1sLKzD gm4war8crFAOsXR5XCnaUnPNiVa+UovJ6LNHkPVZM4GWZVO9i/FHhIwDzaY9gX6HhCFL eEO70Ljuq9dC0HdgDLswum1uYItiBdWVOoOUwG2n3IfecZi3a033mZLAbv1kbn1c6VtJ C6JQ== X-Gm-Message-State: AOAM533hKlCVX5O8sej/z5YWcGjSqJIrEqWmpHb3BhQCxGbuxxcD8WIo m0zu7VXN8pX5k4XEz26Twv/IYtckNTKFKcWdiVw= X-Google-Smtp-Source: ABdhPJyczKFk/SqNAfFWKpDzdgn44yF9JVLdvBd7MXInxRZTGOit6RKKVi/NspggAJmQBMvEPu0jdw== X-Received: by 2002:ac8:5c50:0:b0:2d5:c5f5:93de with SMTP id j16-20020ac85c50000000b002d5c5f593demr3246819qtj.303.1645041534408; Wed, 16 Feb 2022 11:58:54 -0800 (PST) Received: from [192.168.45.36] (c-73-125-98-51.hsd1.fl.comcast.net. [73.125.98.51]) by smtp.gmail.com with ESMTPSA id 84sm2306460qkk.32.2022.02.16.11.58.53 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 16 Feb 2022 11:58:54 -0800 (PST) Message-ID: <58a2566e-370c-9c9c-0284-023808a093ce@philipmcgrath.com> Date: Wed, 16 Feb 2022 14:58:53 -0500 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.5.0 From: Philip McGrath References: <20220213215127.218952-1-philip@philipmcgrath.com> <20220213215127.218952-9-philip@philipmcgrath.com> Content-Language: en-US In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1645043592; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=D0Qnabzo3meV014AhERnKC4+t6IBAVfftZ9KphyrR/M=; b=s1LS5cGxOuSm0WpVUdgCYOA2nm/Omm6LRWg4wXRnUY34xX5JWFlkZIXO2lgCn2ttW0rulo QRPY6ny9WLnC/xUnky8G3V4vhxLsTTXgba9rgdVQbUzfQnvUYnOWllU5OnZj5urnOLeVoZ nA0o9L7bknf9OQZDVeyNQLorhyBigca2l5NgkodNCHNQjNqWMtvfacDoxG8vMdCDa91AgE YgOURI6pw8EnZayXIM4DSALnS/qAToUWS4Cc5vkfoBgQ2oLvTgC3MoCCAcVjVQ+kuevv4+ 4owi7w+dU0LM/22xSqz6CTSAW10AXJtgAj2iUmEuLyiC8HHnSUFsH4rQ3K4osw== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1645043592; a=rsa-sha256; cv=none; b=bPrUeJt2fpDZr05tv0KemRxWzeSlyr4+DYUOVBRbiR2fucVcDoNXr4NxOME9ypT8hxJJtx CEXx3sB15uNz0TQr7YnrwXBC8ISJ5lX/1VJcA+DYnIv/704kT8PaNrwrSWmsjDywqWAFJo mxl57l+445H+N8hhi8HcumdyxG9NYTDmbLtBqsIFKawaUyM0kN9Yj7mnvR0XMZYkyQzIKn h1cvE1JbyoKOS68WeMNmDDLIoqv+Ns4OdRcb7ZS2GRQChoIFxqnJYLeOsIEjIaAuOW/1oR z2R0xixCw0xKrYcIJh0OW/fyz02DxEicH1VfzzS4duE39o0YwPBO8CjQZnXUBw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=philipmcgrath.com header.s=google header.b=UHwn9Tzf; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -2.13 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=philipmcgrath.com header.s=google header.b=UHwn9Tzf; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 30CE74712D X-Spam-Score: -2.13 X-Migadu-Scanner: scn1.migadu.com X-TUID: tVTE9/5F0f/1 Hi, On 2/14/22 10:18, Liliana Marie Prikler wrote: > Hi, > > Am Sonntag, dem 13.02.2022 um 16:51 -0500 schrieb Philip McGrath: >> [...] >> * gnu/packages/chez-and-racket-bootstrap.scm (racket-vm-cgc): >> (racket-vm-bc): >> (racket-vm-cs): >> (chez-scheme-for-racket-bootstrap-bootfiles): >> (chez-scheme-for-racket): New variables. > One patch per package is probably better. That's fine; I'll change it. (I thought that's what I'd suggested in .) > >> [...] One way of thinking about the >> +;; bounary between the Racket VM and Racket programs is that the VM >> implements > boundary. Thanks, I'll fix it. > >> +;; the primitives accessed by the 'ffi/unsafe/vm' library. Another >> perspective >> +;; is that DrRacket's ``Open defining file''/``Jump to definition'' >> features >> +;; can navigate into Racket programs, including into the >> implementation of >> +;; 'racket/base', but can not jump into the implementation of the >> Racket VM >> +;; itself. A third, related perspective is that Racket code is >> usually >> +;; installed with source files alongside compiled code (though this >> is not >> +;; mandatory), whereas the Racket VM is installed only in compiled >> form. >> [...] >> +;; output. The function 'racket-vm-for-system' returns the >> recomended Racket >> +;; VM package for a given system. > This is a very long comment. Consider how much of it is actually > necessary and how much of it is not (do I really need to know about the > capabilities of DrRacket for instance?) I guess it would be ok to cut the explanation of the distinction between VM primitives and the ``built in'' collections, since the bottom line is that the truly primitive primitives aren't useful without the main collections. I'm not sure what else to cut that wouldn't leave room for confusion. >> +;; (Note: since the CGC variant is basically only for bootstrapping, >> we >> +;; often use "BC" to mean "3M", consistent with `(banner)` and the >> +;; suffixes used on executables when more than one variant co- >> exists.) >> +;; >> +;; One remaining bootstrapping limitation is that Racket's reader, >> module >> +;; system, and macro expander are implemented in Racket. For Racket >> CS, >> +;; they are compiled to R6RS libraries as discussed above. This note >> from the >> +;; README file applies to all such subsystems: >> +;; >> +;;     The Racket version must be practically the same as the >> current Racket >> +;;     verson, although it can be the Racket BC implementation >> (instead of >> +;;     the Racket CS implementation). >> +;; >> +;;     Unlike Chez Scheme boot files, the files generated in >> "schemified" >> +;;     are human-readable and -editable Scheme code. That provides a >> way >> +;;     out of bootstrapping black holes, even without BC. >> +;; >> +;; However, other Racket subsystems implemented in Racket for Racket >> CS >> +;; use older C implementations for Racket BC, whereas the reader, >> expander, >> +;; and module system were completely replaced with the Racket >> implementation >> +;; >> +;; For Racket BC, the compiled "linklet" s-expressions (primitive >> modules) >> +;; are embeded in C as a static string constant. Eventually, they >> are further >> +;; compiled by the C-implemented Racket BC bytecode and JIT >> compilers. >> +;; (On platforms where Racket BC's JIT is not supported, yet another >> compiler >> +;; instead compiles the linklets to C code, but this is not a >> bootstrapping >> +;; issue.) >> +;; > Again, you want to be brief and understandable. What does this mean in > practise? Do we have racket bootstrapped yet or is there still some > magic hidden within? This is just the comment that is currently at the top of "gnu/packages/racket.scm", but moved to this file because this is now going to be where the bootstrapping happens. Are there specific thing you want to cut? On the current state of bootstrapping, almost everything is bootstrapped from C, but the "expander" subsystem (which includes the reader and the module system) is not currently bootstrappable, though it is readily auditable. I made another attempt at an explanation in this email: https://lists.gnu.org/archive/html/guix-devel/2021-08/msg00103.html I'd welcome suggestions to improve the explanation! >>  (define unbundle-chez-submodules >>    #~(begin >>        (use-modules (guix build utils)) >>        (for-each (lambda (dir) >> -                (when (directory-exists? dir) >> -                  (delete-file-recursively dir))) >> -              '("stex" >> -                "nanopass" >> -                "lz4" >> -                "zlib")))) >> +                  (when (directory-exists? dir) >> +                    (delete-file-recursively dir))) >> +                '("stex" >> +                  "nanopass" >> +                  "lz4" >> +                  "zlib")))) > As in one of your previous patches, you're mixing cosmetic changes with > non-cosmetic ones. This one could be prevented by correctly indenting > it in the patch that introduces it. Sorry, I missed this in a previous round of indentation fixing. >> +;; >> +;; Racket VM: >> +;; >> + >> +(define (racket-vm-common-configure-flags) >> +  ;; under a lambda extraction to avoid evaluating bash-minimal too >> early >> +  #~`(,@(cond >> +         ((false-if-exception >> +           (search-input-file %build-inputs "/bin/libtool")) >> +          => (lambda (libtool) >> +               (list (string-append "--enable-lt=" libtool)))) >> +         (else >> +          '())) >> +      ,@(cond >> +         ((false-if-exception >> +           (search-input-file %build-inputs "/opt/racket- >> vm/bin/racket")) > Did we have /opt/racket... before? We should probably avoid such > paths. We did not have "opt/racket-vm/" before---adding it was sort of the point of this patch series. Is the reason to avoid it a dislike for "opt", or something else? An ``in place'' build of Racket is not meant to be unpacked directly into some PREFIX where it will coexist with other software. The build is vaguely FHS-like in that e.g. it includes a "bin" directory, but it also e.g. has a "collects" directory at the top level, and it puts other things in paths like "etc/config.rktd" rather than "etc/racket/config.rktd". Subdirectories of "opt" often have that sort of layout, so it seemed like a reasonable place to put it. I considered just embracing Guix not being tied to FHS and putting it directly in the store output, but in build-side code it turned out to be useful to be able to use `search-input-{file,directory}` without potentially confusing the in-place VM with an intermediate, potentially tethered layer. If the question is, ``why do we want an in-place build?'', I can go into as much depth as you want, but it makes the build-side code easier to reason about, it's more compatible with Racket tools e.g. for cross-compilation, and it should help to reduce closure sizes by letting us build packages with non-tethered intermediate layers. >> +          => (lambda (racket) >> +               (list (string-append "--enable-racket=" racket)))) >> +         (else >> +          '())) >> +      ,(string-append "CPPFLAGS=-DGUIX_RKTIO_PATCH_BIN_SH=" >> +                      #$(file-append bash-minimal "/bin/sh")) >> +      "--disable-strip" >> +      "--enable-origtree") > For the record, why do you need double quoting here? Would ungexp- > splicing extract this too soon? I'm not 100% sure I'm following your question correctly, but yes, there were problems with referencing `bash-minimal` too early. (An alternative I considered was to add these arguments in a wrapper around gnu-build-system's configure phase, so that #:configure-flags would be only for arguments that variants want to override, or plausibly might.) > >> +(define-public racket-vm-cgc >> +  ;; Eventually, it may make sense for some vm packages to not be >> hidden, >> +  ;; but this one is especially likely to remain hidden. >> +  (hidden-package >> +   (package >> +     (name "racket-vm-cgc") >> +     (version "8.4") >> +     ;; ^ Remember to also update the version of >> +     ;;   chez-scheme-for-racket-bootstrap-bootfiles >> +     (source >> +      (origin >> +        (method git-fetch) >> +        (uri (git-reference >> +              (url "https://github.com/racket/racket") >> +              (commit (string-append "v" version)))) >> +        (sha256 >> +         (base32 >> "1vpl66gdgc8rnldmn8rmb7ar9l057jqjvgpfn29k57i3c5skr8s6")) >> +        (file-name (git-file-name "racket" version)) >> +        (patches (search-patches "racket-minimal-sh-via-rktio.patch" >> +                                 ;; Remove by Racket 8.5: >> +                                 "racket-enable-scheme- >> backport.patch")) >> +        (modules '((guix build utils))) >> +        (snippet >> +         #~(begin >> +             ;; Unbundle Chez submodules. >> +             (with-directory-excursion "racket/src/ChezScheme" >> +               #$unbundle-chez-submodules) >> +             ;; Unbundle libffi. >> +             (delete-file-recursively >> "racket/src/bc/foreign/libffi"))))) >> +     (inputs (list ncurses ;; <- common to all variants (for >> #%terminal) >> +                   bash-minimal ;; <- common to all variants (for >> `system`) >> +                   libffi)) ;; <- only for BC variants >> +     (native-inputs (list libtool)) ;; <- only for BC variants >> +     (outputs '("out" "debug")) >> +     (build-system gnu-build-system) >> +     (arguments >> +      (list >> +       #:configure-flags >> +       #~(cons "--enable-cgcdefault" >> +               #$(racket-vm-common-configure-flags)) >> +       ;; Tests are in packages like racket-test-core and >> +       ;; main-distribution-test that aren't part of the main >> +       ;; distribution. >> +       #:tests? #f >> +       ;; Upstream recommends #:out-of-source?, and it does >> +       ;; help with debugging, but it confuses `install-license- >> files`. >> +       #:modules '((ice-9 match) >> +                   (ice-9 regex) >> +                   (guix build gnu-build-system) >> +                   (guix build utils)) >> +       #:strip-directories #~'("opt/racket-vm/bin" >> +                               "opt/racket-vm/lib") >> +       #:phases >> +       #~(let () >> +           (define* ((wrap-racket-vm-outputs phase) . args) >> +             (apply >> +              phase >> +              (let loop ((args args)) >> +                (match args >> +                  ((#:outputs outputs . args) >> +                   `(#:outputs >> +                     ,(let loop ((outputs outputs)) >> +                        (match outputs >> +                          ((("out" . out) . outputs) >> +                           `(("out" . ,(string-append out >> "/opt/racket-vm/")) >> +                             ,@outputs)) >> +                          ((other . outputs) >> +                           (cons other (loop outputs))))) >> +                     ,@args)) >> +                  ((arg . args) >> +                   (cons arg (loop args))))))) > Why? > Why what? Why 'wrap-racket-vm-outputs'? The wrapped phases don't have keywords like #:strip-directories, so adjusting their #:output argument is the only way to tell them where to find the files they need to operate on. -Philip