Racket will move on top of Chez soon

Racket will move on top of Chez soon

[Christopher Lemmer Webber]

Just a heads up that right now you *can* run Racket-on-Chez, but soon
Racket-on-Chez will be the "default"... maybe it's a good idea to see
how hard it will be to make a racket-on-chez package variant.

I'm interested in looking at that but it'll probably have to be a while
before I can do so... if someone does so before me, I'll be interested
in beta testing at least.

But also leaving this here as a self-reminder so we aren't surprised
when it becomes a more important thing to deal with :)

Re: Racket will move on top of Chez soon

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 305 bytes --]

Indeed.  I gave it a shot some months back... and failed :p

If I recall correctly, the Chez source must be dropped at the right
spot.

I don't have much time to work on this at the moment but if someone else
gives it a shot I'd be happy to help! :)

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 511 bytes --]

Re: Racket will move on top of Chez soon

[Bonface M. K.]

[-- Attachment #1: Type: text/plain, Size: 902 bytes --]

Christopher Lemmer Webber <cwebber@dustycloud.org>
writes:

> Just a heads up that right now you *can* run Racket-on-Chez, but soon
> Racket-on-Chez will be the "default"... maybe it's a good idea to see
> how hard it will be to make a racket-on-chez package variant.
>
> I'm interested in looking at that but it'll probably have to be a while
> before I can do so... if someone does so before me, I'll be interested
> in beta testing at least.
>
> But also leaving this here as a self-reminder so we aren't surprised
> when it becomes a more important thing to deal with :)
>
>

Also, since we are talking about Racket, what
happened to having a racket build system?

-- 
Bonface M. K. <https://www.bonfacemunyoki.com> 
Chief Emacs Mchochezi / Rieng ya software sare
Curator of <https://upbookclub.com> / Twitter: @BonfaceKilz
GPG Key: D4F09EB110177E03C28E2FE1F5BBAE1E0392253F

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 869 bytes --]

Racket packages / build system

[Christopher Lemmer Webber]

Bonface M. K. writes:

> Christopher Lemmer Webber <cwebber@dustycloud.org> writes:
>
>> Just a heads up that right now you *can* run Racket-on-Chez, but soon
>> Racket-on-Chez will be the "default"... maybe it's a good idea to see
>> how hard it will be to make a racket-on-chez package variant.
>>
>> I'm interested in looking at that but it'll probably have to be a while
>> before I can do so... if someone does so before me, I'll be interested
>> in beta testing at least.
>>
>> But also leaving this here as a self-reminder so we aren't surprised
>> when it becomes a more important thing to deal with :)
>>
>>
>
> Also, since we are talking about Racket, what
> happened to having a racket build system?

There's interest in it, and Dimos made interesting progress towards
figuring out some of the key problems... there's been interest beyond
that but sadly it seems like organizing the energy to work on it hasn't
happened for whatever reason yet.

I've offered to throw money at the problem if someone's willing to take
it on btw... not much, but some money.  I've talked to a couple of
people about that but it stalled in each iteration.... I don't think
it's impossible but I guess it's one of those tasks that for whatever
reason seems difficult to get going on because there are some
complexities around the way Racket builds "collections" that eg don't
seem to show up in Python.  Anyway that's not a judgement because
despite wanting it fairly badly clearly I haven't made progress on it.

I have the notes that Dimos wrote up not long ago in case anyone is
interested.  Dimos, do you mind if I post them to the list?

 - Chris

Re: Racket packages / build system

[Bonface M. K.]

[-- Attachment #1: Type: text/plain, Size: 2346 bytes --]

Christopher Lemmer Webber <cwebber@dustycloud.org>
writes:

> Bonface M. K. writes:
>
>> Christopher Lemmer Webber <cwebber@dustycloud.org> writes:
>>
>>> Just a heads up that right now you *can* run Racket-on-Chez, but soon
>>> Racket-on-Chez will be the "default"... maybe it's a good idea to see
>>> how hard it will be to make a racket-on-chez package variant.
>>>
>>> I'm interested in looking at that but it'll probably have to be a while
>>> before I can do so... if someone does so before me, I'll be interested
>>> in beta testing at least.
>>>
>>> But also leaving this here as a self-reminder so we aren't surprised
>>> when it becomes a more important thing to deal with :)
>>>
>>>
>>
>> Also, since we are talking about Racket, what
>> happened to having a racket build system?
>
> There's interest in it, and Dimos made interesting progress towards
> figuring out some of the key problems... there's been interest beyond
> that but sadly it seems like organizing the energy to work on it hasn't
> happened for whatever reason yet.
>

Can I volunteer on this task? There's some work in
my team done in Racket; and it would be of great
interest to us to have a working Racket build
system. I can set apart some time to work on
this. I'd ask for alot of guidance/ review on this
though :)

> I've offered to throw money at the problem if someone's willing to take
> it on btw... not much, but some money.  I've talked to a couple of
> people about that but it stalled in each iteration.... I don't think
> it's impossible but I guess it's one of those tasks that for whatever
> reason seems difficult to get going on because there are some
> complexities around the way Racket builds "collections" that eg don't
> seem to show up in Python.  Anyway that's not a judgement because
> despite wanting it fairly badly clearly I haven't made progress on it.
>
> I have the notes that Dimos wrote up not long ago in case anyone is
> interested.  Dimos, do you mind if I post them to the list?
>
Please do share the notes! I can try to hack
something up \m/\m/.

>  - Chris

-- 
Bonface M. K. <https://www.bonfacemunyoki.com> 
Chief Emacs Mchochezi / Rieng ya software sare
Curator of <https://upbookclub.com> / Twitter: @BonfaceKilz
GPG Key: D4F09EB110177E03C28E2FE1F5BBAE1E0392253F

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 869 bytes --]

Re: Racket packages / build system

[Christopher Lemmer Webber]

[-- Attachment #1: Type: text/plain, Size: 2591 bytes --]

Bonface M. K. writes:

> Christopher Lemmer Webber <cwebber@dustycloud.org>
> writes:
>
>> Bonface M. K. writes:
>>
>>> Christopher Lemmer Webber <cwebber@dustycloud.org> writes:
>>>
>>>> Just a heads up that right now you *can* run Racket-on-Chez, but soon
>>>> Racket-on-Chez will be the "default"... maybe it's a good idea to see
>>>> how hard it will be to make a racket-on-chez package variant.
>>>>
>>>> I'm interested in looking at that but it'll probably have to be a while
>>>> before I can do so... if someone does so before me, I'll be interested
>>>> in beta testing at least.
>>>>
>>>> But also leaving this here as a self-reminder so we aren't surprised
>>>> when it becomes a more important thing to deal with :)
>>>>
>>>>
>>>
>>> Also, since we are talking about Racket, what
>>> happened to having a racket build system?
>>
>> There's interest in it, and Dimos made interesting progress towards
>> figuring out some of the key problems... there's been interest beyond
>> that but sadly it seems like organizing the energy to work on it hasn't
>> happened for whatever reason yet.
>>
>
> Can I volunteer on this task? There's some work in
> my team done in Racket; and it would be of great
> interest to us to have a working Racket build
> system. I can set apart some time to work on
> this. I'd ask for alot of guidance/ review on this
> though :)

YES!  Please do.  Let's talk.  You can ping me on IRC also, dustyweb on
freenode.

>> I've offered to throw money at the problem if someone's willing to take
>> it on btw... not much, but some money.  I've talked to a couple of
>> people about that but it stalled in each iteration.... I don't think
>> it's impossible but I guess it's one of those tasks that for whatever
>> reason seems difficult to get going on because there are some
>> complexities around the way Racket builds "collections" that eg don't
>> seem to show up in Python.  Anyway that's not a judgement because
>> despite wanting it fairly badly clearly I haven't made progress on it.
>>
>> I have the notes that Dimos wrote up not long ago in case anyone is
>> interested.  Dimos, do you mind if I post them to the list?
>>
> Please do share the notes! I can try to hack
> something up \m/\m/.

I looked at the email that Dimos sent to me (also the email I had was
apparently not the most recent email that they are using, corrected in
the addressing this time), and they had asked me if they should post it
to the mailing list, so I think that's consent to post it since they
expressed consideration in doing so in our exchange... so I'm attaching
it.


[-- Attachment #2: Dimos' notes on Racket on Guix stuff --]
[-- Type: text/plain, Size: 5943 bytes --]

* Basics of Racket packaging system

Racket provides three abstractions of how to reuse and move around modules of code. These are:
** Libraries

A library in Racket is a single file module that can be used in other files. Libraries that are
serving some higher purpose are organised together in Collections.

Documentation can be found [[https://docs.racket-lang.org/reference/collects.html][here]].

** Collections

Collections are a number of libraries bundled together. They can be added to the system through
packages. Racket gets informed for presence of collections through collection link files.

In the filesystem collections are directories that include library files. The default path they are
stored is #<path:/usr/share/racket/collects>. The corresponding link file is located at
#<path:/usr/share/racket/links.rktd> and includes a list as illustrated below:

#+begin_src racket
((root "pkgs/racket-lib")
 (root "pkgs/main-distribution")
 (root "pkgs/2d"))
#+end_src

More specifics about link files and their structure can be found [[https://docs.racket-lang.org/reference/collects.html#%28tech._collection._links._file%29][here]]. More links can be added to
arbitary directories through 'raco link'. This file informs racket where a collection resides in the
#<path:/usr/share/racket> directory.

** Packages

Packages in Racket are the abstractions used to share and move around modules of code. The include a
number of libraries in a collection, or more collections. They are the means through which
dependencies are defined. The main entry for managing them is 'raco pkg'.

The tool for installing packages is 'raco pkg install' that takes as an argument a package source
(name of package in a catalog, directory, tar or zip file etc). How it will act is defined by the
"info.rkt" file included in the package source. 

Documentation for [[https://docs.racket-lang.org/pkg/cmdline.html#%28part._raco-pkg-install%29]['raco pkg install']] and [[https://docs.racket-lang.org/pkg/metadata.html]["info.rkt"]].

Racket can be configured for where to install and search for packages through a confing file
#<path:/etc/usr/racket/config.rktd>. The documentation for the configurations can be found [[https://docs.racket-lang.org/raco/config-file.html?q=raco][here]].

* Racket build system

I will get into the thoughts that I have for the various phases of the build procedure:

** unpack phase
Here normal procedures form 'gnu-build-system' work just fine. The only exception is the handling of
.plt files, which can be done as the .gem files are handled in 'ruby-build-system'.

** bootstrap/build/install phase
Since building and installing is done with 'raco pkg install', it makes sense to have a single phase
dealing with this. The problems that exist here are:
1. <#path:/etc/racket/config.rktd> needs to be updated with all the the places it needs to look for
   the packages. These include the following:
   1. lib-search-dirs
   2. lib-dir
   3. bin-dir
   4. links-search-files
   5. pkgs-search-dirs
   6. collects-search-dirs
   7. doc-search-dirs
      
   This needs to occur two times. First time to build the package, including just its inputs.  Then
   after/while installing the package, we need to create a new config.rktd that includes the proper
   places for all formerly and newly installed packages.

2. Racket packages have circular dependencies.

3. Racket tries to rebuild the dependencies for the package it installs, even when not needed, based
   on timestamps. There exists an environmental variable "PLT_COMPILED_FILE_CHECK", that in
   documentation says if it's set on exist this won't happen. This doesn't seem to work as intended.

4. Documentation is created by mutating the docs structure and adding new links to new nodes of documentation.

For the *first issue*, Claes Wallin that worked on racket2nix, in fact recreates the racket structure
in the outputs folder by copying in the /usr/share/collects, and /lib/racket folders, and creating a
symlink of /usr/share/racket/include. He then creates uses the store provided binaries of
racket(racket, raco, gracket) with -G flag to set the config directory in the outputs and -X flat to
set the collects directory. This way the packages are built and installed in the racket
configuration we created at the outputs. Then the config.rktd needs to be generated. Claes Wallin
does this with a racket script, which we could use, but could also just use a guile function.
I have implemented this, and works as expected, in most cases that don't meet the following issues.

The *second issue* is resolved by the importer in racket2nix. The derivations are expanded to
include circular dependencies and while building the offending packages, dummy inputs are
created. We can do the same using some DFS guile package. Are there any standard graph libraries in
guile? 

The *third issue* is more complicated. Racket will try to to recompile dependencies based on
timestamps. This creates issues during the build. In racket2nix there is created a separate
environment where all dependencies are writable. It is an ugly workaround, but seems to work.

As for the *fourth issue* I didn't research the racket documentation system a lot, especially since
it's packages are the of the main offenders in circulars dependencies.

Binaries of racket packages are placed in "launchers" in the racket configuration folder, so
symlinks to '/bin' shall be created.

** testing phase

It's mostly trivial with 'raco test'.

* Racket importer
Racket packages are defined with "info.rkt" files. Since it's sexp-based it's trivial to parse them
and create package-definitons in guile. There exists some edge-cases like 'implies' that change the
way the packages are upgraded, but in general it pretty simple.

Here is also the place where maybe we could solve the thing with circular dependencies. Is it
possible to create custom fields for inputs in guix derivations?

Re: Racket packages / build system

[Dimos Dimakakos]

> I looked at the email that Dimos sent to me (also the email I had was
> apparently not the most recent email that they are using, corrected in
> the addressing this time), and they had asked me if they should post it
> to the mailing list, so I think that's consent to post it since they
> expressed consideration in doing so in our exchange... so I'm attaching
> it.

Sure, my notes were meant to be public, so thanks for sharing them
here. I don't follow the guix mailing list a lot these days, or I would
have done so myself.

Bonface, if you think that I could help with anything, or need any
clarification on my notes please ask! 

Re: Racket packages / build system

[Bonface M. K.]

[-- Attachment #1: Type: text/plain, Size: 1618 bytes --]

Christopher Lemmer Webber <cwebber@dustycloud.org>
writes:

[...]

>> Can I volunteer on this task? There's some work in
>> my team done in Racket; and it would be of great
>> interest to us to have a working Racket build
>> system. I can set apart some time to work on
>> this. I'd ask for alot of guidance/ review on this
>> though :)
>
> YES!  Please do.  Let's talk.  You can ping me on IRC also, dustyweb on
> freenode.
>

Thanks! I'm called bonz060 on FreeNode; though nowadays I've
grown to be an e-mail person.

[...]

>> Please do share the notes! I can try to hack
>> something up \m/\m/.
>
> I looked at the email that Dimos sent to me (also the email I had was
> apparently not the most recent email that they are using, corrected in
> the addressing this time), and they had asked me if they should post it
> to the mailing list, so I think that's consent to post it since they
> expressed consideration in doing so in our exchange... so I'm attaching
> it.
>

[...]

Thanks for the notes. I've skimmed through them
and they seem sensible. I'll look at how other
build systems are written as a first step, then
get my hands wet.

Do I open an issue for this---I can't see anything
that tracks this even in archived issues--- then
send patches there? Or do I send patches off the
list and then submit the final drafts(if we get
there) on the list later?

-- 
Bonface M. K. <https://www.bonfacemunyoki.com>
Chief Emacs Bazu / Rieng ya software sare
Curator: <https://upbookclub.com> / Twitter: @BonfaceKilz
GPG Key: D4F09EB110177E03C28E2FE1F5BBAE1E0392253F

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 869 bytes --]

Re: Racket packages / build system

[Christopher Lemmer Webber]

Bonface M. K. writes:

> Christopher Lemmer Webber <cwebber@dustycloud.org>
> writes:
>
> [...]
>
>>> Can I volunteer on this task? There's some work in
>>> my team done in Racket; and it would be of great
>>> interest to us to have a working Racket build
>>> system. I can set apart some time to work on
>>> this. I'd ask for alot of guidance/ review on this
>>> though :)
>>
>> YES!  Please do.  Let's talk.  You can ping me on IRC also, dustyweb on
>> freenode.
>>
>
> Thanks! I'm called bonz060 on FreeNode; though nowadays I've
> grown to be an e-mail person.
>
> [...]
>
>>> Please do share the notes! I can try to hack
>>> something up \m/\m/.
>>
>> I looked at the email that Dimos sent to me (also the email I had was
>> apparently not the most recent email that they are using, corrected in
>> the addressing this time), and they had asked me if they should post it
>> to the mailing list, so I think that's consent to post it since they
>> expressed consideration in doing so in our exchange... so I'm attaching
>> it.
>>
>
> [...]
>
> Thanks for the notes. I've skimmed through them
> and they seem sensible. I'll look at how other
> build systems are written as a first step, then
> get my hands wet.
>
> Do I open an issue for this---I can't see anything
> that tracks this even in archived issues--- then
> send patches there? Or do I send patches off the
> list and then submit the final drafts(if we get
> there) on the list later?

That sounds like a great plan!

Re: Racket packages / build system

[Ludovic Courtès]

Hi!

"Bonface M. K." <bonfacemunyoki@gmail.com> skribis:

> Thanks for the notes. I've skimmed through them
> and they seem sensible. I'll look at how other
> build systems are written as a first step, then
> get my hands wet.

Would be great to see that happen!  There’s also a Chicken build system
under review currently:

  https://issues.guix.gnu.org/43976

Also, you’ll probably want ‘guix import raco’ at some point; that may be
quite easy to implement if Racket provides a JSON API for its packages.
(take a look at the other importers).

Ludo’.

Re: Racket packages / build system

[Bonface M. K.]

[-- Attachment #1: Type: text/plain, Size: 893 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> Hi!
>
> "Bonface M. K." <bonfacemunyoki@gmail.com> skribis:
>
>> Thanks for the notes. I've skimmed through them
>> and they seem sensible. I'll look at how other
>> build systems are written as a first step, then
>> get my hands wet.
>
> Would be great to see that happen!  There’s also a Chicken build system
> under review currently:
>
>   https://issues.guix.gnu.org/43976
>
> Also, you’ll probably want ‘guix import raco’ at some point; that may be
> quite easy to implement if Racket provides a JSON API for its packages.
> (take a look at the other importers).
>

Thanks for the share!

> Ludo’.

-- 
Bonface M. K. <https://www.bonfacemunyoki.com>
Chief Emacs Bazu / Rieng ya software sare
Mchochezi of: <https://upbookclub.com> / Twitter: @BonfaceKilz
GPG Key: D4F09EB110177E03C28E2FE1F5BBAE1E0392253F

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 869 bytes --]

Re: Racket packages / build system

[Stephen Paul Weber]

[-- Attachment #1: Type: text/plain, Size: 562 bytes --]

>Thanks for the notes. I've skimmed through them
>and they seem sensible. I'll look at how other
>build systems are written as a first step, then
>get my hands wet.
>
>Do I open an issue for this

I've recently become interested in this as well.  I pinged cwebber asking for
the best place to start and got shown this thread.  Have you made any progress
in thinking about how to go about this?  Did an issue get created (I don't see
one)?  Any place I can jump in to help out?  Just don't want to start mucking
about and duplicating effort if it's not needed :)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Racket packages / build system

[Bonface M. K.]

[-- Attachment #1.1: Type: text/plain, Size: 4048 bytes --]

Christopher Lemmer Webber <cwebber@dustycloud.org>
writes:

> Bonface M. K. writes:
>

[...]

> I have the notes that Dimos wrote up not long ago in case anyone is
> interested.  Dimos, do you mind if I post them to the list?
>
>  - Chris

Hi! I've been trying to hack around the racket
build system(see attached) for some time now;
mostly using raco... The biggest problem I've
faced so far is that AFAICT, when you use raco to
install packages, racket updates some files from
where it's called(in guix's case store this is the
store where racket is in)--- and I don't think you
are allowed to do this. I've tried doing "raco
install <zipfile>", and also just doing "raco
install" from inside the directory.

The command for building:

--8<---------------cut here---------------start------------->8---
env GUIX_PACKAGE_PATH="/home/bonface/projects/guix-bioinformatics:/home/bonface/projects/guix-past/modules" ./pre-inst-env guix build racket-hello-racket -K
--8<---------------cut here---------------end--------------->8---

with the error:

--8<---------------cut here---------------start------------->8---
starting phase `install'
make-directory: cannot make directory
  path: /homeless-shelter/
  system error: Permission denied; errno=13
  context...:
   /gnu/store/4f148vh30qmrdl6apq1ff6yqb7kl8xlm-racket-minimal-7.8/share/racket/collects/racket/file.rkt:114:0: make-directory*
   [repeats 1 more time]
   /gnu/store/4f148vh30qmrdl6apq1ff6yqb7kl8xlm-racket-minimal-7.8/share/racket/collects/pkg/private/lock.rkt:26:0: with-pkg-lock*
   /gnu/store/4f148vh30qmrdl6apq1ff6yqb7kl8xlm-racket-minimal-7.8/share/racket/collects/pkg/main.rkt:216:16
   (submod "/gnu/store/4f148vh30qmrdl6apq1ff6yqb7kl8xlm-racket-minimal-7.8/share/racket/collects/pkg/main.rkt" main): [running body]
   temp35_0
   for-loop
   run-module-instance!
   for-loop
   [repeats 1 more time]
   run-module-instance!
   "/gnu/store/4f148vh30qmrdl6apq1ff6yqb7kl8xlm-racket-minimal-7.8/share/racket/collects/raco/raco.rkt": [running body]
   temp35_0
   for-loop
   run-module-instance!
   "/gnu/store/4f148vh30qmrdl6apq1ff6yqb7kl8xlm-racket-minimal-7.8/share/racket/collects/raco/main.rkt": [running body]
   ...
Inferred package name from given `--clone' path
  package: source
  given path: /tmp/guix-build-racket-hello-racket-0.0.1.drv-0/source
command "raco" "pkg" "install" "--no-cache" "--no-setup" "--ignore-checksums" "--clone" "/tmp/guix-build-racket-hello-racket-0.0.1.drv-0/source" failed with status 1
builder for `/gnu/store/filph2d8m7k1rq6rpglwx1y082ris6g0-racket-hello-racket-0.0.1.drv' failed with exit code 1
build of /gnu/store/filph2d8m7k1rq6rpglwx1y082ris6g0-racket-hello-racket-0.0.1.drv failed
View build log at '/var/log/guix/drvs/fi/lph2d8m7k1rq6rpglwx1y082ris6g0-racket-hello-racket-0.0.1.drv.bz2'.
guix build: error: build of `/gnu/store/filph2d8m7k1rq6rpglwx1y082ris6g0-racket-hello-racket-0.0.1.drv' failed

--8<---------------cut here---------------end--------------->8---

And when troubleshooting:

--8<---------------cut here---------------start------------->8---
cd /tmp/guix-build-racket-hello-racket-0.0.1.drv-0

/home/bonface/guix/pre-inst-env guix environment \
--no-grafts -C racket-hello-racket --ad-hoc strace gdb

source ./environment-variables

$GUIX_ENVIRONMENT/bin/raco pkg install --no-cache \
--no-setup --ignore-checksums

--8<---------------cut here---------------end--------------->8---

Which works just fine since I'm updating files
inside $GUIX_ENVIRONMENT. Right now I'm looking
for ideas to experiment with to try to overcome
this, and the low hanging fruit is to successfully
build a hello-racket package with zero deps and no
tests.

To simply put it, AFAIU updating a package would
require racket to update it's references(either
links, and other references that I won't go into),
hence creating some form of "global state";
thereby if you use raco, every package updated
would lead to some update with racket's search
paths or dirs somewhere. Any ideas to overcome
this wall? (or anything I've got wrong somewhere?)



[-- Attachment #1.2: /build/racket-build-system --]
[-- Type: text/plain, Size: 2438 bytes --]

;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2020 Bonface Munyoki Kilyungi <bonfacemunyoki@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.

(define-module (guix build racket-build-system)
  #:use-module ((guix build gnu-build-system) #:prefix gnu:)
  #:use-module (guix build union)
  #:use-module (guix build utils)
  #:use-module (srfi srfi-1)
  #:use-module (srfi srfi-26)
  #:export (%standard-phases
            racket-build))

;; Commentary:
;;
;; Builder-side code of the racket package build procedure.
;;
;; Code:

(define (racket-package? name)
  (string-prefix? "racket-" name))


;; Currently not working
(define* (check #:key tests? #:allow-other-keys)
  "Run tests for the racket package"
  (if tests?
      (invoke "raco" "test")
      (format #t "test suite not run~%"))
  #t)

(define (call-raco-pkg command params)
  (apply invoke "raco" "pkg" command params))

;; TODO: Find a work around to make this work without modifying where racket
;; store
(define* (install #:key outputs #:allow-other-keys)
  "Install the racket pkg"
  (let ((out (assoc-ref outputs "out")))
    (call-raco-pkg "install"
                   `("--no-cache" "--no-setup"
                     "--ignore-checksums" "--clone" ,(getcwd)))))

(define %standard-phases
  (modify-phases gnu:%standard-phases
    (delete 'bootstrap)
    (delete 'configure)
    (delete 'patch-generated-file-shebangs)
    (delete 'build)
    (replace 'install install)))


(define* (racket-build #:key inputs (phases %standard-phases)
                       #:allow-other-keys #:rest args)
  "Build the given racket package, applying all of PHASES in order."
  (apply gnu:gnu-build #:inputs inputs #:phases phases args))


[-- Attachment #1.3: build-system/racket.scm --]
[-- Type: text/plain, Size: 5047 bytes --]

;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2020 Bonface Munyoki Kilyungi <bonfacemunyoki@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
;;; GNU Guix is free software; you can redistribute it and/or modify it
;;; under the terms of the GNU General Public License as published by
;;; the Free Software Foundation; either version 3 of the License, or (at
;;; your option) any later version.
;;;
;;; GNU Guix is distributed in the hope that it will be useful, but
;;; WITHOUT ANY WARRANTY; without even the implied warranty of
;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;; GNU General Public License for more details.
;;;
;;; You should have received a copy of the GNU General Public License
;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.

(define-module (guix build-system racket)
  #:use-module (guix utils)
  #:use-module (guix derivations)
  #:use-module (guix search-paths)
  #:use-module (guix build-system)
  #:use-module (guix build-system gnu)
  #:use-module (guix packages)
  #:use-module (ice-9 match)
  #:export (%racket-build-system-modules
            racket-build
            racket-build-system))

;; Commentary:

;; Builder-side code of the standard Racket package build procedure
;;
;;
;; Background about the Racket Installation methods

(define %racket-build-system-modules
  ;; Build-side modules imported and used by default.
  `((guix build racket-build-system)
    (guix build union)
    ,@%gnu-build-system-modules))

(define (default-racket)
  (let ((scheme (resolve-interface '(gnu packages scheme))))
    (module-ref scheme 'racket-minimal)))

(define* (lower name
                #:key source inputs native-inputs outputs system target
                (racket (default-racket))
                #:allow-other-keys
                #:rest arguments)
  "Return a bag for NAME."
  (define private-keywords
    '(#:source #:target #:racket #:inputs #:native-inputs))

  (and (not target)
       (bag
         (name name)
         (system system)
         (host-inputs `(,@(if source
                              `(("source" ,source))
                              '())
                        ,@inputs
                        ;; Keep the standard inputs of 'gnu-build-system'.
                        ,@(standard-packages)))
         (build-inputs `(("racket" ,racket)
                         ,@native-inputs))
         (outputs outputs)
         (build racket-build)
         (arguments (strip-keyword-arguments private-keywords arguments)))))

(define* (racket-build store name inputs
                   #:key
                   (phases '(@ (guix build racket-build-system)
                               %standard-phases))
                   (outputs '("out"))
                   (search-paths '())
                   (unpack-path "")
                   (build-flags ''())
                   (tests? #t)
                   (system (%current-system))
                   (guile #f)
                   (imported-modules %racket-build-system-modules)
                   (modules '((guix build racket-build-system)
                              (guix build union)
                              (guix build utils))))
  (define builder
    `(begin
       (use-modules ,@modules)
       (racket-build #:name ,name
                 #:source ,(match (assoc-ref inputs "source")
                             (((? derivation? source))
                              (derivation->output-path source))
                             ((source)
                              source)
                             (source
                              source))
                 #:system ,system
                 #:phases ,phases
                 #:outputs %outputs
                 #:search-paths ',(map search-path-specification->sexp
                                       search-paths)
                 #:unpack-path ,unpack-path
                 #:build-flags ,build-flags
                 #:tests? ,tests?
                 #:inputs %build-inputs)))

  (define guile-for-build
    (match guile
      ((? package?)
       (package-derivation store guile system #:graft? #f))
      (#f                                         ; the default
       (let* ((distro (resolve-interface '(gnu packages commencement)))
              (guile  (module-ref distro 'guile-final)))
         (package-derivation store guile system
                             #:graft? #f)))))

  (build-expression->derivation store name builder
                                #:inputs inputs
                                #:system system
                                #:modules imported-modules
                                #:outputs outputs
                                #:guile-for-build guile-for-build))
(define racket-build-system
  (build-system
    (name 'racket)
    (description
     "Build system for Racket programs")
    (lower lower)))

[-- Attachment #1.4: Type: text/plain, Size: 213 bytes --]


-- 
Bonface M. K. <https://www.bonfacemunyoki.com>
Chief Emacs Bazu / Rieng ya software sare
Mchochezi of: <https://upbookclub.com> / Twitter: @BonfaceKilz
GPG Key: D4F09EB110177E03C28E2FE1F5BBAE1E0392253F

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 869 bytes --]

Re: Racket packages / build system

[Dimos Dimakakos]

Bonface M. K. writes:

> To simply put it, AFAIU updating a package would
> require racket to update it's references(either
> links, and other references that I won't go into),
> hence creating some form of "global state";
> thereby if you use raco, every package updated
> would lead to some update with racket's search
> paths or dirs somewhere. Any ideas to overcome
> this wall? (or anything I've got wrong somewhere?)

This was one of the main problems that I also encountered when working
on this. racket2nix solves this by generating a temporary environment
(by coping most of the racket folders and the deps needed as writable
folders) where it installs with raco and then tries to update the global
state of racket.

To be honest this solution is kinda hacky and also slow, but I couldn't
think of another one at the time I tried to work on the issue. It's a
reality that the racket install system is quite stateful and also many
operations seem to try to touch files. Installing with raco for example
will try to recompile the dependencies of the new package and other such
examples.

Anyway, I hope you can find a way to move this forward!

Re: Racket packages / build system

[Christopher Lemmer Webber]

Dimos Dimakakos writes:

> Bonface M. K. writes:
>
>> To simply put it, AFAIU updating a package would
>> require racket to update it's references(either
>> links, and other references that I won't go into),
>> hence creating some form of "global state";
>> thereby if you use raco, every package updated
>> would lead to some update with racket's search
>> paths or dirs somewhere. Any ideas to overcome
>> this wall? (or anything I've got wrong somewhere?)
>
> This was one of the main problems that I also encountered when working
> on this. racket2nix solves this by generating a temporary environment
> (by coping most of the racket folders and the deps needed as writable
> folders) where it installs with raco and then tries to update the global
> state of racket.
>
> To be honest this solution is kinda hacky and also slow, but I couldn't
> think of another one at the time I tried to work on the issue. It's a
> reality that the racket install system is quite stateful and also many
> operations seem to try to touch files. Installing with raco for example
> will try to recompile the dependencies of the new package and other such
> examples.
>
> Anyway, I hope you can find a way to move this forward!

I wonder if it would be a good idea to copy many of the points from this
email and the parent to racket-users or racket-dev and see if someone
more familiar with the structure of the system can provide guidance from
there?

If we have to go the racket2nix route, it would be better than nothing I
guess.

Another possible route: don't use the Racket installer tooling.
Instead, read the info.rkt file of the package to understand what raco
*probably would* do, and then do it in a more Guix way instead.

What do you think of that route?

Re: Racket packages / build system

[Bonface M. K.]

[-- Attachment #1: Type: text/plain, Size: 2822 bytes --]


Christopher Lemmer Webber <cwebber@dustycloud.org>
writes:

> Dimos Dimakakos writes:
>
>> Bonface M. K. writes:
>>
>>> To simply put it, AFAIU updating a package would
>>> require racket to update it's references(either
>>> links, and other references that I won't go into),
>>> hence creating some form of "global state";
>>> thereby if you use raco, every package updated
>>> would lead to some update with racket's search
>>> paths or dirs somewhere. Any ideas to overcome
>>> this wall? (or anything I've got wrong somewhere?)
>>
>> This was one of the main problems that I also encountered when working
>> on this. racket2nix solves this by generating a temporary environment
>> (by coping most of the racket folders and the deps needed as writable
>> folders) where it installs with raco and then tries to update the global
>> state of racket.
>>
>> To be honest this solution is kinda hacky and also slow, but I couldn't
>> think of another one at the time I tried to work on the issue. It's a
>> reality that the racket install system is quite stateful and also many
>> operations seem to try to touch files. Installing with raco for example
>> will try to recompile the dependencies of the new package and other such
>> examples.
>>
>> Anyway, I hope you can find a way to move this forward!
>
> I wonder if it would be a good idea to copy many of the points from this
> email and the parent to racket-users or racket-dev and see if someone
> more familiar with the structure of the system can provide guidance from
> there?
>

This is a good idea IMHO. I'll go ahead and do
this. Perhaps there's something more important
we've missed or aren't seeing.

> If we have to go the racket2nix route, it would be better than nothing I
> guess.
>

Yeah. I'm considering going though this route as a
last resort. I don't understand the nix DSL
syntax(though it reads alot like Haskell!).

> Another possible route: don't use the Racket installer tooling.
> Instead, read the info.rkt file of the package to understand what raco
> *probably would* do, and then do it in a more Guix way instead.
>
> What do you think of that route?

I've considered doing this... studying raco's
source and seeing how it actually does and sets up
things. I'd rather do this than the above, but it
would take more time and would lead to alot more
boiler plate I think... I'm not entirely sure
about how to work around the global state
though...

First, let's consult with the racket-devel and
racket-user ML and see what those communities have
to suggest...

-- 
Bonface M. K. <https://www.bonfacemunyoki.com>
Chief Emacs Bazu / Rieng ya software sare
Mchochezi of: <https://upbookclub.com> / Twitter: @BonfaceKilz
GPG Key: D4F09EB110177E03C28E2FE1F5BBAE1E0392253F

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 869 bytes --]

Re: Racket packages / build system

[Christopher Lemmer Webber]

Bonface M. K. writes:

> Christopher Lemmer Webber <cwebber@dustycloud.org>
> writes:
>
>> Dimos Dimakakos writes:
>>
>>> Bonface M. K. writes:
>>>
>>>> To simply put it, AFAIU updating a package would
>>>> require racket to update it's references(either
>>>> links, and other references that I won't go into),
>>>> hence creating some form of "global state";
>>>> thereby if you use raco, every package updated
>>>> would lead to some update with racket's search
>>>> paths or dirs somewhere. Any ideas to overcome
>>>> this wall? (or anything I've got wrong somewhere?)
>>>
>>> This was one of the main problems that I also encountered when working
>>> on this. racket2nix solves this by generating a temporary environment
>>> (by coping most of the racket folders and the deps needed as writable
>>> folders) where it installs with raco and then tries to update the global
>>> state of racket.
>>>
>>> To be honest this solution is kinda hacky and also slow, but I couldn't
>>> think of another one at the time I tried to work on the issue. It's a
>>> reality that the racket install system is quite stateful and also many
>>> operations seem to try to touch files. Installing with raco for example
>>> will try to recompile the dependencies of the new package and other such
>>> examples.
>>>
>>> Anyway, I hope you can find a way to move this forward!
>>
>> I wonder if it would be a good idea to copy many of the points from this
>> email and the parent to racket-users or racket-dev and see if someone
>> more familiar with the structure of the system can provide guidance from
>> there?
>>
>
> This is a good idea IMHO. I'll go ahead and do
> this. Perhaps there's something more important
> we've missed or aren't seeing.
>
>> If we have to go the racket2nix route, it would be better than nothing I
>> guess.
>>
>
> Yeah. I'm considering going though this route as a
> last resort. I don't understand the nix DSL
> syntax(though it reads alot like Haskell!).
>
>> Another possible route: don't use the Racket installer tooling.
>> Instead, read the info.rkt file of the package to understand what raco
>> *probably would* do, and then do it in a more Guix way instead.
>>
>> What do you think of that route?
>
> I've considered doing this... studying raco's source and seeing how it
> actually does and sets up things. I'd rather do this than the above,
> but it would take more time and would lead to alot more boiler plate I
> think... I'm not entirely sure about how to work around the global
> state though...

Regarding the boilerplate, not sure it needs to from a
package-definitions perspective... if the info.rkt can be read in the
general case, this could be the racket-build-system that does most of
the work (probably even by reading the very same info.rkt) rather than
it being output'ed from an importer definition.

> First, let's consult with the racket-devel and racket-user ML and see
> what those communities have to suggest...

Yes, good idea.

Re: Racket packages / build system

[Bonface M. K.]

[-- Attachment #1: Type: text/plain, Size: 1106 bytes --]

Christopher Lemmer Webber <cwebber@dustycloud.org>
writes:

[...]

>> I've considered doing this... studying raco's source and seeing how it
>> actually does and sets up things. I'd rather do this than the above,
>> but it would take more time and would lead to alot more boiler plate I
>> think... I'm not entirely sure about how to work around the global
>> state though...
>
> Regarding the boilerplate, not sure it needs to from a
> package-definitions perspective... if the info.rkt can be read in the
> general case, this could be the racket-build-system that does most of
> the work (probably even by reading the very same info.rkt) rather than
> it being output'ed from an importer definition.
>

I'll try exploring this.

>> First, let's consult with the racket-devel and racket-user ML and see
>> what those communities have to suggest...
>
> Yes, good idea.

-- 
Bonface M. K. <https://www.bonfacemunyoki.com>
Chief Emacs Bazu / Rieng ya software sare
Mchochezi of: <https://upbookclub.com> / Twitter: @BonfaceKilz
GPG Key: D4F09EB110177E03C28E2FE1F5BBAE1E0392253F

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 869 bytes --]

Re: Racket packages / build system

[Bonface M. K.]

[-- Attachment #1: Type: text/plain, Size: 1955 bytes --]


Hi racket-dev and racket-users!

I'm trying to create a build system for Racket in
Guix(a long-standing feature for some of us in the
Guix community). It appears that Racket, vis-a-vis
raco, maintains some "global" state whereby
package-installs leads to the update of some
files, links, search-paths and other refs(hence
the statefulness) that I won't go into.

There's a similar project, racket2nix(please see:
https://github.com/fractalide/racket2nix) which
overcomes this by generating a temporary
environment(by copying most of the racket folders
and the deps needed as writable folders) where it
installs with raco and then tries to update
racket's global state. This feels slow and
hacky. To summarize, the default racket install
system is quite stateful(!) and also many
operations seem to try to touch files.

I'm considering studying the racket2nix more and
going that route as a last resort if I don't find
a work-around for the aforementioned state issues.

Another possibility(though I don't know how
feasible that would be) would be to figure out how
raco works(by studying it's source) and do the
same thing in a Guix-y way. I however, don't know
yet how I would overcome the state issue :(

Racket-dev or racket-users, is there some
important thing I'm missing?  Is there some (maybe
undocumented, new or WIP) way of installing racket
packages without maintaining state?

Any pointers/ suggestions are gladly welcome. I'm
invested of having a way to package racket
packages in Guix; and I believe it's a big step
forward for Racket in increasing it's reach in the
Guix community :)

PS: Full e-mail thread here:
https://lists.gnu.org/archive/html/guix-devel/2020-10/msg00298.html

-- 
Bonface M. K. <https://www.bonfacemunyoki.com>
Chief Emacs Bazu / Rieng ya software sare
Mchochezi of: <https://upbookclub.com> / Twitter: @BonfaceKilz
GPG Key: D4F09EB110177E03C28E2FE1F5BBAE1E0392253F

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 869 bytes --]