bug#48682: recursive import between (gnu packages chez) and (gnu packages racket)

[Philip McGrath <philip@philipmcgrath.com>]

Hi, I've been working on Racket packaging. I haven't looked into this 
much yet---hopefully I can tomorrow---but here are a few quick thoughts.

On 5/26/21 2:54 PM, raingloom wrote:
 > I also haven't found any other mention of chez-scheme in racket.scm
 > other than the line that imports it, which is weird, given that Racket

This was my doing. When I added `(gnu packages racket)` in [1], I 
adapted the patch from a racket.scm I'd been using to experiment ([2], 
also some branches of [3]): the changes that actually use (gnu packages 
chez) aren't ready to go yet, but apparently I left the import line in.


On 5/26/21 2:54 PM, raingloom wrote:
 > other than the line that imports it, which is weird, given that Racket
 > is now built on Chez, so I'd expect it to use it as an input. I guess
 > it's a bundled version? In any case I don't think I should just remove
 > the import, because it will be needed eventually, so this issue needs
 > to be fixed by then.

Racket uses a fork of Chez Scheme: I described the situation in [4]. 
Racket won't have upstream Chez as an input in the foreseeable, but, 
from a packaging perspective, building Racket's Chez fork uses many of 
the same pieces as building upstream Chez. I think it may make sense to 
refactor out some of the common infrastructure, so both `(gnu packages 
racket)` and `(gnu packages chez)` could depend on some low-level 
module, but I'm not sure what the Guix-preferred way to organize things 
would be. (I'm mostly a Racketeer using Racket packaging to learn more 
about Guix.)

Here are some ways `(gnu packages chez)` and `(gnu packages racket)` are 
intertwined, in no particular order:

   - `racket-minimal` should use the `nanopass` and `stex` origins
      as `chez-scheme` does. (But it would be easiest to do this
      once I've changed `racket-minimal` to build from the Git source,
      rather than the bootstrapped tarball: I hope that will be soon.)
      Some more of my not-quite-current thoughts on that at [5].

   - The `chez-scheme` phases `unpack-nanopass+stex`, `configure`,
     `prepare-stex`, and `install-doc` should be shared with Racket.
     I think it would be better to put them in a build-side module and
     actually share them, rather than to do tricky things to extract
     their s-expression representation from
     `(package-arguments chez-scheme)`. On the other hand, I think a
     build system would be overkill: it would only build vanilla Chez
     and Racket-flavored Chez.

   - Racket-flavored Chez has added some backends that vanilla Chez
     doesn't support (and can't be readily ported, unless upstream Chez
     eventually adopts Racket's changes to generating backends).
     In particular, Racket-flavored Chez adds support for threading on
     ARMv6 and support for aarch64 (which vanilla Chez doesn't support at
     all). I haven't thought at all deeply about this, but it might make
     sense for the default `chez-scheme` package on those architectures
     to be Racket-flavored Chez.

   - We may in fact want to use Racket to bootstrap vanilla Chez.

     Chez has the usual problem with self-hosting compilers that you
     need the old version to compile the new version. Specifically, you
     need "bootfiles", which encapsulate the Scheme-implemented portion
     of Chez, compiled to machine code. Thus, (a) they are platform-
     specific and (b) they need to agree precisely with the C-implemented
     part of Chez on the layout of datatypes and such.

     The vanilla Chez Git repository [6] has bootfiles for i686, x86_64,
     and (non-threaded) ARMv6. Once you've done a native build for one of
     those platforms, you can use it to cross-compile for any platform
     Chez supports (ppc32, various BSDs, etc.). But these are binary
     blobs. From a "trusting trust" perspective, it's especially striking
     if you consider that Chez Scheme was non-free software from 1984 to
     2016, and the first libre release likewise needed the bootfiles of
     its ancestors. (However, building bootfiles is reproducible:
     Chez in fact builds them twice and errors if they aren't
     byte-for-byte identical.)

     But Racket is able to simulate enough of Chez to (slowly)
     compile the Chez compiler and generate bootfiles, providing a path
     to Chez from just a C compiler. Racket does its whole bootstrapping
     process regularly in CI, and I'm working on getting the Guix package
     to do likewise.

     Unfortunately, the Racket fork has diverged enough from vanilla Chez
     that the current Racket "cs-bootstrap" package [7] can't build
     vanilla Chez, but I hope that the solution is just a matter of
     walking through the Git history to find the right older version of
     the bootstrapping package, before the Racket fork's `#!base-rtd`
     gained a vector of ancestors rather than a parent and a few such
     things. But I'd like to be able to build Racket packages with Guix
     before I try that :)

Ok, that got a bit long.

I don't know where the cycle came from with `emacs-geiser-racket`, but I 
think it would be reasonable either to do some refactoring to `(gnu 
packages chez)` and `(gnu packages racket)`, or to just remove the 
import for now to un-break things and figure out the rest later.

-Philip

[1]: https://issues.guix.gnu.org/47829
[2]: https://gitlab.com/philip1/guix-racket-experiment
[3]: https://gitlab.com/philip1/guix-patches/
[4]: https://issues.guix.gnu.org/46865#3
[5]: https://issues.guix.gnu.org/47153#2
[6]: https://github.com/cisco/ChezScheme
[7]: 
https://github.com/racket/racket/tree/master/racket/src/ChezScheme/rktboot