Re: How did you handle making a GNU/Linux distribution?

[Christine Lemmer-Webber <cwebber@dustycloud.org>]

Philip McGrath <philip@philipmcgrath.com> writes:

> On 9/12/21 11:09 PM, Christine Lemmer-Webber wrote:
>> Philip McGrath <philip@philipmcgrath.com> writes:
>>> Christine Lemmer-Webber had floated the idea at some point of trying
>>> to integrate Racket and Guile.
>>>
>>> IIRC, I think what she's had in mind was trying to make a Guile
>>> backend for Racket along the lines of the Chez Scheme backend (or the
>>> BC backend, or experimental backends like Pycket).
>> Yes that's what I had in mind :)
>
> A few stray thoughts, all with the caveat that I haven't actually
> tried any of this ...
>
>>> there are two things that have struck me
>>> as downsides:
>>>
>>>   1. Flatt et al. say in "Rebuilding Racket on Chez Scheme (Experience
>>>      Report)" (§6, p. 13) that, "If our task were to compile Racket to an
>>>      existing target, then we would not have achieved such a high degree
>>>      of compatibility. … we have taken the liberty of modifying Chez
>>>      Scheme to make it an easier target for Racket."
>>>
>>>      https://www.cs.utah.edu/plt/publications/icfp19-fddkmstz.pdf
>>>
>>>      Presumably a Racket-on-Guile project would face the same trade-off,
>>>      where modifications to Guild, if Racket CS is a guide, could require
>>>      hard work over many years, and lesser compatibility would make the
>>>      result less useful.
>> At one point when I spoke to Matthew, he was very optimistic that
>> the
>> work on porting on top of Chez would open Racket to running on top of
>> many other backends.  But yes... since there have been so many "custom"
>> modifications to Chez, it's easier to be skeptical about that these
>> days...
>
> I think there a few possible senses for what "running on top of many
> other backends" could mean. My impression overall is that it has
> gotten easier, but may not necessarily be easy.
>
> The most clearly demonstrated is that it seems to be easier to port
> Chez Scheme to new architectures than to port Racket BC's VM. Racket
> CS supports the Apple M1 chip natively (and hopefully will support the 
> Linux kernel on the M1 when that stabilizes), which Racket BC does
> not. Racket CS also fully supports some platforms (ARM in general,
> IIRC) on which Racket BC lacks support for futures, places, or the
> JIT. The most promising route to Racket on WASM also seems to be
> adding a WASM backend to the Chez Scheme compiler. (In fairness, there
> are also some architectures, like ppc64le, to which no one has ported
> Racket CS yet, for which Racket BC can offer at least some support via
> C.)

Who will get to WASM first, Racket or Guile?  :)

> More generally, the "linklet" abstraction[1] seems to provide a much
> more clear boundary between the responsibilities of a backend 
> implementation and the common frontend than existed before Racket
> 7.0. For example, the backend doesn't have to deal with macro
> expansion or even shadowing of variables: it just need to have some
> way to instantiate linklets (likely with further backend-specific
> compilation) and to provide a whole bunch of primitives. I believe
> I've heard Sam Tobin-Hochstadt say that linklets have made it possible
> for Pycket (the experimental Racket implementation in Reticulated
> Python) to run most Racket code.
>
> [1]: https://docs.racket-lang.org/reference/linklets.html
>
> Another benefit of linklets is that they've defined a clear path for
> implementing parts of Racket in Racket itself, like the regexp 
> implementation on Racket CS. This seems like it could help a lot with
> supplying that very large number of primitives.
>
> So I expect it would be entirely possible to implement a linklet-based
> Racket backend in Guile. I do suspect that getting production-quality 
> performance and compatibility could be a lot of work. But my bigger
> concern is ...
>
>>>   2. As you probably know, Racket programs can't generally use
>>>      Chez Scheme implemented libraries, because Chez Scheme effectively
>>>      is the "unsafe" layer of the Racket VM. For example, not all Racket
>>>      procedures are Chez Scheme procedures, and Racket's continuations
>>>      wrap Chez Scheme's to implement delimited and composable control,
>>>      threads, parameters, full continuation marks, etc.
>>>
>>>      For Racket CS, this isn't a great loss (there aren't so many
>>>      Chez-specific libraries, and portable libraries can run in Racket's
>>>      R6RS language), but, for a hypothetical Racket-on-Guile,
>>>      bidirectional interoperability would be a big attraction: imagine
>>>      Guix, Shepherd, Mcron, syntax/parse, racket/contract, and more all
>>>      working together.
>
> I don't think a Racket-on-Guile that worked like Racket-on-Chez would
> achieve the most interesting benefit (IMO) of bringing the two
> languages together: safe, bidirectional interoperability.
>
> Probably there are many ways to achieve that goal. In principle, one
> could write a Racket-on-Guile implementation where arbitrary Guile
> could run without breaking Racket's invariants, but I expect that
> would make the task even harder: presumably that's why Racket-on-Chez
> doesn't work that way.
>
> But if linklets are as good as an abstraction as they seem to be for
> compiling code in the presence of modules and macros---all of the 
> complications of Scheme---then a more exciting direction, it seems to
> me, would be to adapt Guile's compiler tower[2] to, at some layer, 
> produce linklets. If the existing Guile backend (which does have some
> strengths) became a linklet engine, it could run linklets generated by 
> the Racket front-end. Linklets from the Guile frontend could likewise
> run on other Racket backends: Racket-and-Guile-on-Chez, Pycket, or any 
> exciting backend someone creates in the future.
>
> [2]: https://www.gnu.org/software/guile/manual/html_node/Compiler-Tower.html
>
> It would probably still be a fair bit of work, particularly in
> figuring out exactly how the semantics of Guile modules correspond to
> Racket modules and in making interoperability convenient. I have no
> particular plans to work on this myself. But I think it would be cool
> :)
>
> -Philip

Lots of intersesting stuff in here.  The compiler tower direction is
indeed something I've hoped could be explored.  But it's also something
not on my agenda immediately either. :)