[bug#62231] Chez Scheme for Racket build on aarch64 (patch attached)

[bug#62231] Chez Scheme for Racket build on aarch64 (patch attached)

[Tim Johann]

[-- Attachment #1.1: Type: text/plain, Size: 4542 bytes --]

Hi Guix Team,

tl;dr;

The recipe for package 'chez-scheme-for-racket-bootstrap-bootfiles' (a
dependency for racket-8.*) needs to use the --machine flag to build on aarch64
(and possibly also ppc32, the other non-x86 architectures supported by the
Racket fork of Chez Scheme).  I therefore suggest the attached patch to make
building and using Racket possible on aarch64 Guix.

End of tl;dr;

A few weeks ago I wanted to use Racket on Guix on aarch64, but was made aware,
that Racket was not available as a substitute and it did not build.  On the
other hand, recent versions of Racket were available on Raspberry Pi OS which is
explicitly running on aarch64.  The machine I am running Guix on actually is a
Raspberry Pi 4B.

So, I dug deeper to find that the problem is that the package
  chez-scheme-for-racket-bootstrap-bootfiles
fails with the error
,----
|   string-append: contract violation
|   expected: string?
|   given: #f
|   argument position: 1st
|   other arguments...:
|    ".def"
|   context...:
|    /tmp/guix-build-chez-scheme-for-racket-bootstrap-bootfiles-9.9.9-pre-release.14.drv-0/source/racket/src/rktboot/machine-def.rkt:6:0: open-file-with-machine.def-redirect
| error: in phase 'build': uncaught exception:
| %exception #<&invoke-error program: "/gnu/store/0n4skakcg05bbq9sq7g00j2zm25xh4wa-racket-vm-bc-8.8/opt/racket-vm/bin/racket" arguments: ("../rktboot/main.rkt") exit-status: 1 term-signal: #f stop-signal: #f> 
| phase `build' failed after 14.3 seconds
| command "/gnu/store/0n4skakcg05bbq9sq7g00j2zm25xh4wa-racket-vm-bc-8.8/opt/racket-vm/bin/racket" "../rktboot/main.rkt" failed with status 1
| builder for `/gnu/store/d04mbj02vzjv0ljcqy2lxs33scz7kjz0-chez-scheme-for-racket-bootstrap-bootfiles-9.9.9-pre-release.14.drv' failed with exit code 1
| @ build-failed /gnu/store/d04mbj02vzjv0ljcqy2lxs33scz7kjz0-chez-scheme-for-racket-bootstrap-bootfiles-9.9.9-pre-release.14.drv - 1 builder for `/gnu/store/d04mbj02vzjv0ljcqy2lxs33scz7kjz0-chez-scheme-for-racket-bootstrap-bootfiles-9.9.9-pre-release.14.drv' failed with exit code 1
`----
(see, e.g. the build [442314 on ci.guix.gnu.org]).

The Racket fork of Chez is natively supported on aarch64, which is supported by
comments in module (gnu packages chez), so I went on to find that machine
architecture is communicated in the build process by setting the environment
variable MACH.  I haven't looked to verify this, but it seems that this
environment variable is normally set to a guessed value by the zuo build process
(zuo for 'make' in Mandarin?).

The Guix build recipe for chez-scheme-for-racket-bootstrap-bootfiles, though, is
not configuring the whole downloaded racket source package but takes the
shortcut of just building ChezScheme in the subdirectory
  racket/src/ChezScheme
using the Racket utility
  ../rktboot/main.rkt
This script would actually accept the flag '--machine' and would set the
environment variable MACH to the value of the argument following the flag.

The problem is that in the case of aarch64 the recipe would not set the flag.
In the recipe for chez-scheme-for-racket-bootstrap-bootfiles in module (gnu
packages chez), we find the lines
,----
| #~(invoke
|    (search-input-file (or native-inputs inputs)
|                       "/opt/racket-vm/bin/racket")
|    "../rktboot/main.rkt"
|    #$@(if (racket-cs-native-supported-system?)
|           #~()
|           (let ((m (nix-system->pbarch-machine-type)))
|             #~("--machine" #$m)))))))))))))
`----
and, since `(racket-cs-native-supported-system?)' returns `tarm64le', the first
branch of the if expression is taken, ommitting the flag.

I suggest to change the above code to the following:
,----
| #~(invoke
|    (search-input-file (or native-inputs inputs)
|                       "/opt/racket-vm/bin/racket")
|    "../rktboot/main.rkt"
|    #$@(let ((m (or (racket-cs-native-supported-system?)
|                    (nix-system->pbarch-machine-type))))
|         #~("--machine" #$m))))))))))))
`----
which sets the flag for natively supported architectures and those supported by
portable bytecode.

The package (and subsequently Racket) builds on aarch64, the build on x86_64
seems to be unaffected.

It is a simple change which would make a huge difference for Guix users who want
to be able to play with Racket on an aarch64 machine.

Cheers

Tim

Lá Fhéile Pádraig sona dhaoibh!


[442314 on ci.guix.gnu.org] <https://ci.guix.gnu.org/build/442314/log/raw>

[-- Attachment #1.2: Type: text/html, Size: 8874 bytes --]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: chez-racket-aarch64.patch --]
[-- Type: text/x-diff, Size: 1484 bytes --]

commit f8d7b06a92b3839da1908837ac81ab6295626cf5
Author: Tim Johann <t1m@phrogstar.de>
Date:   Thu Mar 16 18:53:13 2023 +0100

    gnu: chez-scheme-for-racket-bootstrap-bootfiles: change for aarch64.
    
    * gnu/packages/chez.scm (chez-scheme-for-racket-bootstrap-bootfiles):
    change for build on aarch64, making racket available on aarch64.
    [arguments] use --machine even when architecture is supported by Racket's
    fork of ChezScheme.

diff --git a/gnu/packages/chez.scm b/gnu/packages/chez.scm
index c6420a980e..bc709917c2 100644
--- a/gnu/packages/chez.scm
+++ b/gnu/packages/chez.scm
@@ -692,10 +692,9 @@ (define-public chez-scheme-for-racket-bootstrap-bootfiles
                          (search-input-file (or native-inputs inputs)
                                             "/opt/racket-vm/bin/racket")
                          "../rktboot/main.rkt"
-                         #$@(if (racket-cs-native-supported-system?)
-                                #~()
-                                (let ((m (nix-system->pbarch-machine-type)))
-                                  #~("--machine" #$m)))))))))))))
+                         #$@(let ((m (or (racket-cs-native-supported-system?)
+                                         (nix-system->pbarch-machine-type))))
+                              #~("--machine" #$m))))))))))))
     (supported-systems
      (package-supported-systems chez-scheme-for-racket))
     (home-page "https://github.com/racket/ChezScheme")

[bug#62231] Chez Scheme for Racket build on aarch64 (patch attached)

[Philip McGrath]

[-- Attachment #1: Type: text/plain, Size: 3392 bytes --]

Hi Tim,

On Thursday, March 16, 2023 7:23:46 PM EDT Tim Johann wrote:
> A few weeks ago I wanted to use Racket on Guix on aarch64, but was made
> aware, that Racket was not available as a substitute and it did not build. 
> On the other hand, recent versions of Racket were available on Raspberry Pi
> OS which is explicitly running on aarch64.  The machine I am running Guix
> on actually is a Raspberry Pi 4B.
> 

Thanks again for your investigation of this problem! As I commented on the 
upstream issue at <https://github.com/racket/racket/issues/3948>, I still 
think this is an upstream bug (and your analysis helped a great deal in finding 
the root cause), but the approach in your patch is probably the easiest way to 
fix things for Guix users for now. I'd just ask that you add a comment 
referencing the upstream issue and with some explanation, because it's not my 
preferred long-term approach.

> 
> [...]
> 
> using the Racket utility
>   ../rktboot/main.rkt
> This script would actually accept the flag '--machine' and would set the
> environment variable MACH to the value of the argument following the flag.
> 
> The problem is that in the case of aarch64 the recipe would not set the
> flag. In the recipe for chez-scheme-for-racket-bootstrap-bootfiles in
> module (gnu packages chez), we find the lines
> ,----
> 
> | #~(invoke
> | 
> |    (search-input-file (or native-inputs inputs)
> |    
> |                       "/opt/racket-vm/bin/racket")
> |    
> |    "../rktboot/main.rkt"
> |    #$@(if (racket-cs-native-supported-system?)
> |    
> |           #~()
> |           (let ((m (nix-system->pbarch-machine-type)))
> |           
> |             #~("--machine" #$m)))))))))))))
> 
> `----
> and, since `(racket-cs-native-supported-system?)' returns `tarm64le', the
> first branch of the if expression is taken, ommitting the flag.
> 

The "rktboot" scripts (which simulate enough of Chez Scheme to bootstrap its 
bootfiles) are supposed to be able to infer right machine type for the current 
platform, at least if the platform is natively supported. The problem is that 
the upstream code to doesn't actually work on architectures other than i386 
and x86_64. 

> I suggest to change the above code to the following:
> ,----
> 
> | #~(invoke
> | 
> |    (search-input-file (or native-inputs inputs)
> |    
> |                       "/opt/racket-vm/bin/racket")
> |    
> |    "../rktboot/main.rkt"
> |    #$@(let ((m (or (racket-cs-native-supported-system?)
> |    
> |                    (nix-system->pbarch-machine-type))))
> |         
> |         #~("--machine" #$m))))))))))))
> 
> `----
> which sets the flag for natively supported architectures and those supported
> by portable bytecode.

This is a good work-around. The reason I'd like to avoid it in the long run is 
that, ideally, I'd like to get us out of the business of managing Chez machine 
types and the translation from normal architecture and OS strings. I think 
we'll at least be able to eliminate `%chez-features-table` once the ongoing 
work to reunify upstream Chez Scheme with Racket's branch is complete. 
Reducing the number of places where we have to use these machine-type-
producing functions should help advance that goal, so I'd prefer to have a 
comment here reminding us that we can reconsider "--machine" once <https://
github.com/racket/racket/issues/3948> is fixed.

-Philip

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#62231] Chez Scheme for Racket build on aarch64 (patch attached)

[Tim Johann]

[-- Attachment #1: Type: text/plain, Size: 128 bytes --]

Hi Philip,

Happy to be of help.

Thanks for the pointers, here's the amended patch.  I hope it is to your liking.

cheers

Tim

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: chez-racket-aarch64-rev1.patch --]
[-- Type: text/x-diff, Size: 2110 bytes --]

commit e3c514db745d48e5ef7f7abb7b45037341298b17
Author: Tim Johann <t1m@phrogstar.de>
Date:   Fri Mar 24 21:09:52 2023 +0100

    gnu: chez-scheme-for-racket-bootstrap-bootfiles: change for aarch64.
    
    * gnu/packages/chez.scm (chez-scheme-for-racket-bootstrap-bootfiles):
      Temporary change for build on aarch64, making racket available on aarch64.
      Architecture autodetect in rktboot only addresses x86 archs, so far.
      This work-around sets the architecture, looked up in translation table,
      explicitly.  The long term solution should be to fix rktboot to include
      other natively supported architectures in the autodetect mechanism.
        cf. https://github.com/racket/racket/issues/3948
      [arguments] use --machine even when architecture is supported by Racket's
      fork of ChezScheme.

diff --git a/gnu/packages/chez.scm b/gnu/packages/chez.scm
index c6420a980e..ab6eaba6bd 100644
--- a/gnu/packages/chez.scm
+++ b/gnu/packages/chez.scm
@@ -692,10 +692,12 @@ (define-public chez-scheme-for-racket-bootstrap-bootfiles
                          (search-input-file (or native-inputs inputs)
                                             "/opt/racket-vm/bin/racket")
                          "../rktboot/main.rkt"
-                         #$@(if (racket-cs-native-supported-system?)
-                                #~()
-                                (let ((m (nix-system->pbarch-machine-type)))
-                                  #~("--machine" #$m)))))))))))))
+                         ;; Temporary handling of builds on non-x86 architectures,
+                         ;; see https://github.com/racket/racket/issues/3948
+                         ;; Autodetect in rktboot only addresses x86 archs, so far.
+                         #$@(let ((m (or (racket-cs-native-supported-system?)
+                                         (nix-system->pbarch-machine-type))))
+                              #~("--machine" #$m))))))))))))
     (supported-systems
      (package-supported-systems chez-scheme-for-racket))
     (home-page "https://github.com/racket/ChezScheme")

[bug#62231] Chez Scheme for Racket build on aarch64 (patch attached)

[Philip McGrath]

[-- Attachment #1.1: Type: text/plain, Size: 538 bytes --]

On Friday, March 24, 2023 4:03:19 PM EDT Tim Johann wrote:
> Hi Philip,
> 
> Happy to be of help.
> 
> Thanks for the pointers, here's the amended patch.  I hope it is to your
> liking.
> 

Thanks, this addresses my concerns!

I'm not a Guix committer, so I'm cc'ing a couple of people who have applied 
recent patches to this file. They may want to review the commit message for 
style, also.

I expect the underlying issue would also affect riscv64, but I hit some network 
trouble when trying to look for a build log just now.

-Philip

[-- Attachment #1.2: chez-racket-aarch64-rev1.patch --]
[-- Type: text/x-patch, Size: 2110 bytes --]

commit e3c514db745d48e5ef7f7abb7b45037341298b17
Author: Tim Johann <t1m@phrogstar.de>
Date:   Fri Mar 24 21:09:52 2023 +0100

    gnu: chez-scheme-for-racket-bootstrap-bootfiles: change for aarch64.
    
    * gnu/packages/chez.scm (chez-scheme-for-racket-bootstrap-bootfiles):
      Temporary change for build on aarch64, making racket available on aarch64.
      Architecture autodetect in rktboot only addresses x86 archs, so far.
      This work-around sets the architecture, looked up in translation table,
      explicitly.  The long term solution should be to fix rktboot to include
      other natively supported architectures in the autodetect mechanism.
        cf. https://github.com/racket/racket/issues/3948
      [arguments] use --machine even when architecture is supported by Racket's
      fork of ChezScheme.

diff --git a/gnu/packages/chez.scm b/gnu/packages/chez.scm
index c6420a980e..ab6eaba6bd 100644
--- a/gnu/packages/chez.scm
+++ b/gnu/packages/chez.scm
@@ -692,10 +692,12 @@ (define-public chez-scheme-for-racket-bootstrap-bootfiles
                          (search-input-file (or native-inputs inputs)
                                             "/opt/racket-vm/bin/racket")
                          "../rktboot/main.rkt"
-                         #$@(if (racket-cs-native-supported-system?)
-                                #~()
-                                (let ((m (nix-system->pbarch-machine-type)))
-                                  #~("--machine" #$m)))))))))))))
+                         ;; Temporary handling of builds on non-x86 architectures,
+                         ;; see https://github.com/racket/racket/issues/3948
+                         ;; Autodetect in rktboot only addresses x86 archs, so far.
+                         #$@(let ((m (or (racket-cs-native-supported-system?)
+                                         (nix-system->pbarch-machine-type))))
+                              #~("--machine" #$m))))))))))))
     (supported-systems
      (package-supported-systems chez-scheme-for-racket))
     (home-page "https://github.com/racket/ChezScheme")

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#62231: Chez Scheme for Racket build on aarch64 (patch attached)

[Ludovic Courtès]

Hi,

Tim Johann <t1m@phrogstar.de> skribis:

> commit e3c514db745d48e5ef7f7abb7b45037341298b17
> Author: Tim Johann <t1m@phrogstar.de>
> Date:   Fri Mar 24 21:09:52 2023 +0100
>
>     gnu: chez-scheme-for-racket-bootstrap-bootfiles: change for aarch64.
>     
>     * gnu/packages/chez.scm (chez-scheme-for-racket-bootstrap-bootfiles):
>       Temporary change for build on aarch64, making racket available on aarch64.
>       Architecture autodetect in rktboot only addresses x86 archs, so far.
>       This work-around sets the architecture, looked up in translation table,
>       explicitly.  The long term solution should be to fix rktboot to include
>       other natively supported architectures in the autodetect mechanism.
>         cf. https://github.com/racket/racket/issues/3948
>       [arguments] use --machine even when architecture is supported by Racket's
>       fork of ChezScheme.

Applied!

Thank you Tim, and thanks Philip for reviewing.

Tim: please use ‘git format-patch’ in the future; the format here cannot
be ingested as-is by ‘git am’.

Ludo’.