* Basics of Racket packaging system Racket provides three abstractions of how to reuse and move around modules of code. These are: ** Libraries A library in Racket is a single file module that can be used in other files. Libraries that are serving some higher purpose are organised together in Collections. Documentation can be found [[https://docs.racket-lang.org/reference/collects.html][here]]. ** Collections Collections are a number of libraries bundled together. They can be added to the system through packages. Racket gets informed for presence of collections through collection link files. In the filesystem collections are directories that include library files. The default path they are stored is #. The corresponding link file is located at # and includes a list as illustrated below: #+begin_src racket ((root "pkgs/racket-lib") (root "pkgs/main-distribution") (root "pkgs/2d")) #+end_src More specifics about link files and their structure can be found [[https://docs.racket-lang.org/reference/collects.html#%28tech._collection._links._file%29][here]]. More links can be added to arbitary directories through 'raco link'. This file informs racket where a collection resides in the # directory. ** Packages Packages in Racket are the abstractions used to share and move around modules of code. The include a number of libraries in a collection, or more collections. They are the means through which dependencies are defined. The main entry for managing them is 'raco pkg'. The tool for installing packages is 'raco pkg install' that takes as an argument a package source (name of package in a catalog, directory, tar or zip file etc). How it will act is defined by the "info.rkt" file included in the package source. Documentation for [[https://docs.racket-lang.org/pkg/cmdline.html#%28part._raco-pkg-install%29]['raco pkg install']] and [[https://docs.racket-lang.org/pkg/metadata.html]["info.rkt"]]. Racket can be configured for where to install and search for packages through a confing file #. The documentation for the configurations can be found [[https://docs.racket-lang.org/raco/config-file.html?q=raco][here]]. * Racket build system I will get into the thoughts that I have for the various phases of the build procedure: ** unpack phase Here normal procedures form 'gnu-build-system' work just fine. The only exception is the handling of .plt files, which can be done as the .gem files are handled in 'ruby-build-system'. ** bootstrap/build/install phase Since building and installing is done with 'raco pkg install', it makes sense to have a single phase dealing with this. The problems that exist here are: 1. <#path:/etc/racket/config.rktd> needs to be updated with all the the places it needs to look for the packages. These include the following: 1. lib-search-dirs 2. lib-dir 3. bin-dir 4. links-search-files 5. pkgs-search-dirs 6. collects-search-dirs 7. doc-search-dirs This needs to occur two times. First time to build the package, including just its inputs. Then after/while installing the package, we need to create a new config.rktd that includes the proper places for all formerly and newly installed packages. 2. Racket packages have circular dependencies. 3. Racket tries to rebuild the dependencies for the package it installs, even when not needed, based on timestamps. There exists an environmental variable "PLT_COMPILED_FILE_CHECK", that in documentation says if it's set on exist this won't happen. This doesn't seem to work as intended. 4. Documentation is created by mutating the docs structure and adding new links to new nodes of documentation. For the *first issue*, Claes Wallin that worked on racket2nix, in fact recreates the racket structure in the outputs folder by copying in the /usr/share/collects, and /lib/racket folders, and creating a symlink of /usr/share/racket/include. He then creates uses the store provided binaries of racket(racket, raco, gracket) with -G flag to set the config directory in the outputs and -X flat to set the collects directory. This way the packages are built and installed in the racket configuration we created at the outputs. Then the config.rktd needs to be generated. Claes Wallin does this with a racket script, which we could use, but could also just use a guile function. I have implemented this, and works as expected, in most cases that don't meet the following issues. The *second issue* is resolved by the importer in racket2nix. The derivations are expanded to include circular dependencies and while building the offending packages, dummy inputs are created. We can do the same using some DFS guile package. Are there any standard graph libraries in guile? The *third issue* is more complicated. Racket will try to to recompile dependencies based on timestamps. This creates issues during the build. In racket2nix there is created a separate environment where all dependencies are writable. It is an ugly workaround, but seems to work. As for the *fourth issue* I didn't research the racket documentation system a lot, especially since it's packages are the of the main offenders in circulars dependencies. Binaries of racket packages are placed in "launchers" in the racket configuration folder, so symlinks to '/bin' shall be created. ** testing phase It's mostly trivial with 'raco test'. * Racket importer Racket packages are defined with "info.rkt" files. Since it's sexp-based it's trivial to parse them and create package-definitons in guile. There exists some edge-cases like 'implies' that change the way the packages are upgraded, but in general it pretty simple. Here is also the place where maybe we could solve the thing with circular dependencies. Is it possible to create custom fields for inputs in guix derivations?