[bug#57050] [PATCH v2 05/13] gnu: racket: Update to 8.6.

["Philip McGrath" <philip@philipmcgrath.com>]

Hi,

On Thu, Aug 11, 2022, at 7:44 AM, Liliana Marie Prikler wrote:
> Am Donnerstag, dem 11.08.2022 um 07:08 -0400 schrieb Philip McGrath:
>> Also, update 'chez-scheme-for-racket' to 9.5.9.2.
> As with Zuo, can this be split into a separate patch?  If not, why does
> it get mentioned here as something noteworthy rather than just being
> part of the ChangeLog?
>

This absolutely should not be split into more patches. While I split out Zuo as you requested, I continue to believe that it was better as a single patch.

More broadly, I hope to put together a 'racket-build-system' before the 8.7 release. Whenever that happens, it will mean turning the inputs of the 'racket' and 'racket-minimal' packages into, last I counted, 203 additional packages. I think it would be bad for everyone—patch writers, patch reviewers, users, and people reading the history later—to break Racket releases into a deluge of little patches, many of which would leave the Racket universe as a whole in states no one had ever built or tested, because Racket has combinations of technical and social mechanisms to make certain potentially problematic states difficult to fall into.

A Racket release is an update to a family of packages that are developed and released together. Like a TeX Live update (cf. ee25e3fcab9d2e24c2826b771b52d797c152193b) or the KDE Frameworks libraries (cf. 9d3965edca29f80667374da45214cc6f22a85be4), the contents of a Racket release should be updated together. I wouldn't insist on one commit in all imaginable circumstances, but I think it should be the norm. That's all the more true of Racket components that share a canonical Git repository, for which Racket tools take steps to warn you if not fail if you have mismatched versions.

I also want to emphasize that splitting up patches is not free. Splitting/rebasing v2 of this series took hours of work. Even specifically in splitting off Zuo, I made a (trivial) error in my first version of this particular patch and had to amend the commit and rebuild. Splitting this any further would get farther away both from how I actually wrote and tested this patch series over the past four months and from the way Racket is developed and released. It would take a non-negligible amount of effort, all to produce a result that I believe would be worse.

I mention the version number for 'chez-scheme-for-racket' because it shows up relatively prominently in Guix tooling and even e.g. in the path of the package documentation. I don't know why it would be a problem to do so, but I would vastly prefer to remove the mention than to split the patch, if it really has to be a choice. The Racket release notes don't mention the corresponding version of Chez Scheme.

>> [...]
>> @@ -448,18 +456,52 @@ (define-public chez-scheme-for-racket
>>         (delete "libx11" "util-linux:lib")))
>>      (native-inputs
>>       (modify-inputs (package-native-inputs chez-scheme)
>> +       (append zuo)
>>         (replace "chez-scheme-bootstrap-bootfiles"
>>           chez-scheme-for-racket-bootstrap-bootfiles)))
> Prefer prepend over append.
>

Why?

>>      (arguments
>>       (substitute-keyword-arguments (package-arguments chez-scheme)
>> +       ((#:out-of-source? _ #f)
>> +        #t)
>> +       ((#:tests? _ #t)
>> +        ;; FIXME: There have been some flaky test failures. Some have
>> been
>> +        ;; fixed upstream post-release but have proven non-trivial to
>> +        ;; backport; at least one issue remains. Re-enable tests once
>> +        ;; https://github.com/racket/racket/issues/4359 is fixed.
>> +        #f)
> Rather than skipping tests altogether, skip just the flaky ones.
>

If I knew how to do that, I absolutely would. Well, short of making the `process` function silently fail to run `/bin/sh` again, maybe. If you look at the linked issue, you'll see that I've been chasing down failures for the last month. For a while I had cherry-picked 153ff9acb7ad63717a50bb26cd5aaa053870c666, which fixed the only issue with the implementation (a race condition in one mode of the garbage collector), but it didn't seem worth carrying the patch when things were still failing. The other issues seem to be problems with running the test suite. Probably they appeared now because we hadn't actually been running as much of it as we seemed to be. So far, I haven't had failures building with current master, but there aren't any commits touching anything obviously related, and Unicode 14 and grapheme cluster support have landed since the release branch, so it wouldn't be reasonable to just sweep in all the changes indiscriminately. Further, it's at least possible that I may just have been winning the race recently and that the actual problem might still be there. The Chez test suite takes an hour to run (maybe more), and I haven't been able to reproduce the failures outside of Guix, so it's not exactly rapid iteration. All in all, I don't know any change to make other than turning off tests that I can feel confident will work reliably for Guix users.

For what it's worth, difficulty running the Chez tests is not unique to Guix. In the course of adding Zuo, Matthew Flatt commented, "After this conversion, I was able to run the Chez Scheme test suite on Windows for the first time; it must have been possible before, but I never quite got the right tools installed in the right way to make it work." [1] AFAICT Debian skips the Chez test suite unless it is specifically requested [2]: I don't know why, maybe just because it takes so long.

[1]: https://github.com/racket/racket/pull/4179#issuecomment-1094137092
[2]: https://salsa.debian.org/scheme-team/chezscheme/-/blob/fbfa9c35db184f5b22ac6c0058d754e1a45f5f68/debian/rules#L66-69

>> @@ -130,12 +132,12 @@ (define-module (gnu packages racket)
>>  ;; output. The function 'racket-vm-for-system' returns the
>> recomended Racket
>>  ;; VM package for a given system.
>>  ;;
>> -;; The file 'racket.scm' builds on these packages to define 'racket-
>> minimal'
>> -;; and 'racket' packages. These use Racket's support for ``layered
>> -;; installations'', which allow an immutable base layer to be
>> extended with
>> -;; additional packages. They use the layer configuration directly
>> provide
>> -;; ready-to-install FHS-like trees, rather than relying on the built
>> in
>> -;; ``Unix-style install'' mechanism.
>> +;; We then define the packages 'racket-minimal' and
>> +;; 'racket'. These use Racket's support for ``layered
>> installations'', which
>> +;; allow an immutable base layer to be extended with additional
>> packages.
>> +;; They use the layer configuration directly provide ready-to-
>> install FHS-like
>> +;; trees, rather than relying on the built in ``Unix-style install''
>> +;; mechanism.
>>  ;;
>>  ;; Bootstrapping Racket:
>>  ;; ---------------------
> This is a leftover from 8.6, but do the "FHS-like" installations
> actually adhere to the FHS or just to the bit that says "opt means we
> can do whatever we want 😎️"?
>

The 'racket' and 'racket-minimal' packages follow FHS. (Pedantically, because compiled Racket code may or may not be architecture specific, and used to always be architecture-independent, some configuration modes put things in "share" that belong in "lib", but the current upstream "Unix-style" defaults and out packages do not do that.)

Here's a comparison of the hierarchy of our non-VM Racket packages vs. an in-place Racket installation, with the bracketed numbers indicating how the two match up:

/gnu/store/xyz-racket-8.5/
├── bin/ [1]
├── etc/
│   └── racket/
│       └── config.rktd [2]
├── lib/
│   └── racket/ [3]
│       ├── links.rktd
│       ├── mans.rktd
│       └── pkgs/ [4]
└── share/
    ├── applications/ [5]
    │       └── drracket.desktop
    ├── doc/
    │       └── racket/ [6]
    │           └── index.html
    ├── man/ [7]
    │       └── man1/
    │           └── racket.1.gz
    └── racket/ [8]
             └── infocache.rktd

/gnu/store/xyz-racket-vm-cs-8.5/opt/racket-vm/
├── bin/ [1]
├── collects/ [would be share/racket/collects, but not duplicated in layers]
├── doc/ [6]
├── etc/
│   └── config.rktd [2]
├── include/
│   ├── chezscheme.h
│   ├── racketcsboot.h
│   └── racketcs.h
├── lib/ [3]
│   ├── libracketcs.a
│   ├── petite.boot
│   ├── racket.boot
│   └── scheme.boot
├── man/ [7]
│   └── man1/
└── share/ [8]
    ├── applications/ [5]
    └── pkgs/ [4]


>>  ;;
>> -;; Code:
>> +;; Zuo is notably *not* a problem for bootstrapping. The
>> implementation is a
>> +;; single hand-written C file designed to build with just `cc -o zuo
>> zuo.c`,
>> +;; even with very old or limited compilers. (We use the Autoconf
>> support for
>> +;; convienience.)
>> +;;
>> +;; CODE:
> Is that something that needs be mentioned?
>

Some people have asked whether the Zuo implementation does or should share things with Racket and/or Chez Scheme. It seemed worth noting that it intentionally does not, for the sake of bootstrapping.

-Philip