[bug#46865] [PATCH] gnu: racket: update to 8.0.

[bug#46865] [PATCH] gnu: racket: update to 8.0.

[Philip McGrath]

* gnu/packages/scheme.scm (racket, racket-minimal): Update to 8.0.
---
 gnu/packages/scheme.scm | 98 +++++++++++++++++++++++++++++------------
 1 file changed, 71 insertions(+), 27 deletions(-)

diff --git a/gnu/packages/scheme.scm b/gnu/packages/scheme.scm
index 6c30ed8bf6..10be0aa28a 100644
--- a/gnu/packages/scheme.scm
+++ b/gnu/packages/scheme.scm
@@ -398,22 +398,26 @@ implementation techniques and as an expository tool.")
 (define-public racket
   (package
     (name "racket")
-    (version "7.9")            ; note: remember to also update racket-minimal!
+    (version "8.0")            ; note: remember to also update racket-minimal!
     (source (origin
               (method url-fetch)
-              (uri (list (string-append "http://mirror.racket-lang.org/installers/"
-                                        version "/racket-" version "-src.tgz")
+              (uri (list (string-append "https://mirror.racket-lang.org/installers/"
+                                        version "/racket-src.tgz")
+                         ;; this mirror seems to have broken HTTPS:
                          (string-append
                           "http://mirror.informatik.uni-tuebingen.de/mirror/racket/"
-                          version "/racket-" version "-src.tgz")))
+                          version "/racket-src.tgz")))
               (sha256
                (base32
-                "0gmp2ahmfd97nn9bwpfx9lznjmjkd042slnrrbdmyh59cqh98y2m"))
+                "047wpjblfzmf1msz7snrp2c2h0zxyzlmbsqr9bwsyvz3frcg0888"))
               (patches (search-patches
                         "racket-store-checksum-override.patch"))))
     (build-system gnu-build-system)
     (arguments
-     '(#:phases
+     '(#:configure-flags
+       '("--enable-libz"
+         "--enable-liblz4")
+       #:phases
        (modify-phases %standard-phases
          (add-before 'configure 'pre-configure-minimal
            (lambda* (#:key inputs #:allow-other-keys)
@@ -493,13 +497,45 @@ implementation techniques and as an expository tool.")
            (lambda _
              (substitute* "collects/racket/system.rkt"
                (("/bin/sh") (which "sh")))
-             #t)))
+             #t))
+         (add-after 'patch-/bin/sh 'patch-chez-configure
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (substitute* "src/cs/c/Makefile.in"
+               (("/bin/sh") (which "sh")))
+             ;; TODO: Racket CS uses a fork of Chez Scheme.
+             ;; Most of this is copy-pasted from the "chez.scm",
+             ;; but maybe there's a way to reuse more directly.
+             (with-directory-excursion "src/ChezScheme"
+               (substitute* (find-files "mats" "Mf-.*")
+                 (("^[[:space:]]+(cc ) *") "\tgcc "))
+               (substitute*
+                   (find-files "." (string-append
+                                    "("
+                                    "Mf-[a-zA-Z0-9.]+"
+                                    "|Makefile[a-zA-Z0-9.]*"
+                                    "|checkin"
+                                    "|stex\\.stex"
+                                    "|newrelease"
+                                    "|workarea"
+                                    "|unix\\.ms"
+                                    "|^6\\.ms"
+                                    ;;"|[a-zA-Z0-9.]+\\.ms" ; guile can't read
+                                    ")"))
+                 (("/bin/rm") (which "rm"))
+                 (("/bin/ln") (which "ln"))
+                 (("/bin/cp") (which "cp"))
+                 (("/bin/echo") (which "echo")))
+               (substitute* "makefiles/installsh"
+                 (("/bin/true") (which "true")))))))
        ;; XXX: how to run them?
        #:tests? #f))
     (inputs
-     `(("libffi" ,libffi)
-       ;; Hardcode dynamically loaded libraries for better functionality.
+     `(;; Hardcode dynamically loaded libraries for better functionality.
        ;; sqlite and libraries for `racket/draw' are needed to build the doc.
+       ("zlib" ,zlib)
+       ("zlib:static" ,zlib "static")
+       ("lz4" ,lz4)
+       ("lz4:static" ,lz4 "static")
        ("cairo" ,cairo)
        ("fontconfig" ,fontconfig)
        ("glib" ,glib)
@@ -519,10 +555,14 @@ implementation techniques and as an expository tool.")
     (home-page "https://racket-lang.org")
     (synopsis "Implementation of Scheme and related languages")
     (description
-     "Racket is an implementation of the Scheme programming language (R5RS and
-R6RS) and related languages, such as Typed Racket.  It features a compiler and
-a virtual machine with just-in-time native compilation, as well as a large set
-of libraries.")
+     "Racket is a general-purpose programming language in the Scheme family,
+with a large set of libraries and a compiler based on Chez Scheme.  Racket is
+also a platform for language-oriented programming, from small domain-specific
+languages to complete language implementations.
+
+The main Racket distribution comes with many bundled packages, including
+the DrRacket IDE, libraries for GUI and web programming, and implementations
+of languages such as Typed Racket, R5RS and R6RS Scheme, and Datalog.")
     ;; https://download.racket-lang.org/license.html
     (license (list lgpl3+ asl2.0 expat))))
 
@@ -533,14 +573,15 @@ of libraries.")
     (version (package-version racket))
     (source (origin
               (method url-fetch)
-              (uri (list (string-append "http://mirror.racket-lang.org/installers/"
-                                        version "/racket-minimal-" version "-src.tgz")
+              (uri (list (string-append "https://mirror.racket-lang.org/installers/"
+                                        version "/racket-minimal-src.tgz")
+                         ;; this mirror seems to have broken HTTPS:
                          (string-append
                           "http://mirror.informatik.uni-tuebingen.de/mirror/racket/"
-                          version "/racket-minimal-" version "-src.tgz")))
+                          version "/racket-minimal-src.tgz")))
               (sha256
                (base32
-                "0yc5zkpq1bavj64h67pllw6mfjhmdp65fgdpyqcaan3syy6b5cia"))
+                "0mwyffw4gcci8wmzxa3j28h03h0gsz55aard8qrk3lri8r2xyg21"))
               (patches (search-patches
                         "racket-store-checksum-override.patch"))))
     (synopsis "Racket without bundled packages such as Dr. Racket")
@@ -551,18 +592,21 @@ of libraries.")
            ;; Delete fix that applies to files not included in the minimal package.
            (delete 'pre-configure)))))
     (inputs
-     `(("libffi" ,libffi)
-       ("openssl" ,openssl)
-       ("sqlite" ,sqlite)))
+     `(("openssl" ,openssl)
+       ("sqlite" ,sqlite)
+       ("zlib" ,zlib)
+       ("zlib:static" ,zlib "static")
+       ("lz4" ,lz4)
+       ("lz4:static" ,lz4 "static")))
     (description
-     "Racket is an implementation of the Scheme programming language (R5RS and
-R6RS) and related languages, such as Typed Racket.  It features a compiler and
-a virtual machine with just-in-time native compilation, as well as a large set
-of libraries.
+     "Racket is a general-purpose programming language in the Scheme family,
+with a large set of libraries and a compiler based on Chez Scheme.  Racket is
+also a platform for language-oriented programming, from small domain-specific
+languages to complete language implementations.
 
-In this minimal package, the essential package racket-libs is included, as
-well as libraries that live in collections.  In particular, @command{raco} and
-the @code{pkg} library are still bundled.")))
+The ``minimal Racket'' distribution includes just enough of Racket for you to
+use @command{raco pkg} to install more.  Bundled packages, such as the
+Dr. Racket IDE, are not included.")))
 
 (define-public gambit-c
   (package
-- 
2.21.1 (Apple Git-122.3)

[bug#46865] [PATCH] gnu: racket: update to 8.0.

[Philip McGrath]

I changed the description primarily because, as of Racket 8.0, it is no 
longer accurate to say, "It features a compiler and a virtual machine 
with just-in-time native compilation"—the default implementation is now 
a native code compiler based on Chez Scheme.

But there's certainly still room for improvement. In particular, I see 
that Nix is using the description:

 > Racket is a full-spectrum programming language. It goes beyond
 > Lisp and Scheme with dialects that support objects, types,
 > laziness, and more. Racket enables programmers to link
 > components written in different dialects, and it empowers
 > programmers to create new, project-specific dialects. Racket's
 > libraries support applications from web servers and databases to
 > GUIs and charts.

The Racket project has used descriptions along those lines in other 
places before.

I left the synopsis, "Implementation of Scheme and related languages", 
alone, but that's also debatable. From a Racketeer's perspective,
Racket contains a Scheme implementation, but it is not primarily a 
Scheme implementation. Nix uses the short description, "A programmable 
programming language".

bug#46865: [PATCH] gnu: racket: update to 8.0.

[Ludovic Courtès]

Hi Philip,

Philip McGrath <philip@philipmcgrath.com> skribis:

> * gnu/packages/scheme.scm (racket, racket-minimal): Update to 8.0.

I applied it and tweak the commit log to mention the fields that were
changed.  The new synopsis and description LGTM!

Did you try unbundling Chez?  It would be great if we could do that.
(In Guix we unbundle software as much as possible, notably for security
reasons.)

Thanks!

Ludo’.

[bug#46865] [PATCH] gnu: racket: update to 8.0.

[Philip McGrath]

Hi!

On 3/5/21 6:49 AM, Ludovic Courtès wrote:
> I applied it and tweak the commit log to mention the fields that were
> changed.  The new synopsis and description LGTM!

That's great!

> Did you try unbundling Chez?  It would be great if we could do that.
> (In Guix we unbundle software as much as possible, notably for security
> reasons.)

The Racket fork of Chez Scheme has diverged significantly from upstream.
Racket regularly syncs with upstream Chez and contributes patches back, 
but they are likely to coexist as two different "flavors" of Chez for 
the foreseeable future. There's a detailed (though almost two-year-old) 
review of the differences in their ICFP report[1]. In essence, some of 
Racket's "changes to Chez Scheme are either controversial or heavyweight 
compared to the expected benefit for applications other than Racket". 
You can see how one controversial pull request[2] became a Flatt and 
Dybvig 2020 PLDI paper.[3] At this point, for example, Racket's Chez has 
gained an Aarch64 backend, floating-point unboxing, and a 
type-reconstruction compiler pass.

So, no, I don't think there's a prospect of unbundling Chez.

On the other hand, since sending this patch, I saw that much of the work 
Guix and Nix had to do with around absolute paths could be better 
addressed upstream. Racket has merged my fixes for most of them[4] (and 
Matthew solved another in C [5]), which means we can greatly simplify 
the package definition. I hope to get those fixes into upstream Chez, too.

I do think I should be able to unbundle Nanopass, as the Chez Guix 
package does.

Just now I've been working on bootstrapping. Chez has the usual problem 
that you need the old version of Chez (specifically "bootfiles") to 
compile the new version. Racket's fork added an architecture-independent 
backend: those bootfiles are in the source distribution, which are how 
the Guix package is building now. But Racket is also able to simulate 
enough of Chez to (slowly) compile the Chez compiler, providing a path 
to Chez from just a C compiler. Racket does its whole bootstrapping 
process regularly in CI, and I'm working on getting the Guix package to 
do likewise.

(I knew from previous discussions that Racket on Guix could benefit from 
some attention, so I'm hoping to use it as an opportunity to learn more 
about Guix.)

-Philip

[1]: https://www.cs.utah.edu/plt/publications/icfp19-fddkmstz.pdf
[2]: https://github.com/cisco/ChezScheme/pull/336
[3]: https://www.cs.utah.edu/plt/publications/pldi20-fd.pdf
[4]: https://github.com/racket/racket/pull/3710
[5]: 
https://github.com/mflatt/racket/commit/7319a22ed605fb3110b98f6c9aa1e8ed3aa03794

[bug#46865] [PATCH] gnu: racket: update to 8.0.

[Christopher Lemmer Webber]

Ludovic Courtès writes:

> Hi Philip,
>
> Philip McGrath <philip@philipmcgrath.com> skribis:
>
>> * gnu/packages/scheme.scm (racket, racket-minimal): Update to 8.0.
>
> I applied it and tweak the commit log to mention the fields that were
> changed.  The new synopsis and description LGTM!
>
> Did you try unbundling Chez?  It would be great if we could do that.
> (In Guix we unbundle software as much as possible, notably for security
> reasons.)
>
> Thanks!
>
> Ludo’.

My understanding is that Racket is shipping a modified Chez, which could
make that difficult (but the goal is for the Chez it ships to be
less-modified-over-time).

[bug#46865] [PATCH] gnu: racket: update to 8.0.

[Jack Hill]

[-- Attachment #1: Type: text/plain, Size: 703 bytes --]

Philip,

On Fri, 5 Mar 2021, Philip McGrath wrote:

> The Racket fork of Chez Scheme has diverged significantly from upstream.

[…]

> [1]: https://www.cs.utah.edu/plt/publications/icfp19-fddkmstz.pdf
> [2]: https://github.com/cisco/ChezScheme/pull/336
> [3]: https://www.cs.utah.edu/plt/publications/pldi20-fd.pdf
> [4]: https://github.com/racket/racket/pull/3710
> [5]: https://github.com/mflatt/racket/commit/7319a22ed605fb3110b98f6c9aa1e8ed3aa03794

Thanks for pulling all this information together and writing a nice 
summer. It definitely helped me better understand what was going on with 
Racket CS. I'm excited to hear about that possibility of a full-source 
bootstrap as well.

Thanks!
Jack

[bug#46865] [PATCH] gnu: racket: update to 8.0.

[Ludovic Courtès]

Hi,

Philip McGrath <philip@philipmcgrath.com> skribis:

>> Did you try unbundling Chez?  It would be great if we could do that.
>> (In Guix we unbundle software as much as possible, notably for security
>> reasons.)
>
> The Racket fork of Chez Scheme has diverged significantly from upstream.
> Racket regularly syncs with upstream Chez and contributes patches
> back, but they are likely to coexist as two different "flavors" of
> Chez for the foreseeable future. There's a detailed (though almost
> two-year-old) review of the differences in their ICFP report[1]. In
> essence, some of Racket's "changes to Chez Scheme are either
> controversial or heavyweight compared to the expected benefit for
> applications other than Racket". You can see how one controversial
> pull request[2] became a Flatt and Dybvig 2020 PLDI paper.[3] At this
> point, for example, Racket's Chez has gained an Aarch64 backend,
> floating-point unboxing, and a type-reconstruction compiler pass.
>
> So, no, I don't think there's a prospect of unbundling Chez.

OK, that makes sense; thanks for explaining.

> On the other hand, since sending this patch, I saw that much of the
> work Guix and Nix had to do with around absolute paths could be better 
> addressed upstream. Racket has merged my fixes for most of them[4]
> (and Matthew solved another in C [5]), which means we can greatly
> simplify the package definition. I hope to get those fixes into
> upstream Chez, too.

Nice!

> I do think I should be able to unbundle Nanopass, as the Chez Guix
> package does.

Cool, that’d be a welcome improvement.

> Just now I've been working on bootstrapping. Chez has the usual
> problem that you need the old version of Chez (specifically
> "bootfiles") to compile the new version. Racket's fork added an
> architecture-independent backend: those bootfiles are in the source
> distribution, which are how the Guix package is building now. But
> Racket is also able to simulate enough of Chez to (slowly) compile the
> Chez compiler, providing a path to Chez from just a C compiler. Racket
> does its whole bootstrapping process regularly in CI, and I'm working
> on getting the Guix package to do likewise.

Awesome.

> (I knew from previous discussions that Racket on Guix could benefit
> from some attention, so I'm hoping to use it as an opportunity to
> learn more about Guix.)

Yup!  There have been discussions about having a raco importer (for the
“guix import” command), but I think the effort stalled.  It’d go a long
way bringing the Racket/Guix/Guile communities together, and it can be
hacked in a language all three communities love.  :-)

Thanks for your work!

Ludo’.