From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id aLkoOEjHjV9iegAA0tVLHw (envelope-from ) for ; Mon, 19 Oct 2020 17:05:12 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id yAD0M0jHjV+oOgAAbx9fmQ (envelope-from ) for ; Mon, 19 Oct 2020 17:05:12 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 7D90F9400D3 for ; Mon, 19 Oct 2020 17:05:12 +0000 (UTC) Received: from localhost ([::1]:56694 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kUYat-0005wd-GX for larch@yhetil.org; Mon, 19 Oct 2020 13:05:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40554) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kUYaj-0005wP-LJ for guix-devel@gnu.org; Mon, 19 Oct 2020 13:05:01 -0400 Received: from dustycloud.org ([2600:3c02::f03c:91ff:feae:cb51]:33854) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kUYah-0001SL-7r for guix-devel@gnu.org; Mon, 19 Oct 2020 13:05:01 -0400 Received: from twig (localhost [127.0.0.1]) by dustycloud.org (Postfix) with ESMTPS id 5FFDE26616; Mon, 19 Oct 2020 13:04:33 -0400 (EDT) References: <87ft6c1ypz.fsf@dustycloud.org> <865z77smyg.fsf@gmail.com> <87a6wizuic.fsf@dustycloud.org> <868sc27cix.fsf@gmail.com> User-agent: mu4e 1.4.12; emacs 26.3 From: Christopher Lemmer Webber To: "Bonface M. K." Subject: Re: Racket packages / build system In-reply-to: <868sc27cix.fsf@gmail.com> Date: Mon, 19 Oct 2020 13:04:03 -0400 Message-ID: <87v9f6w3jw.fsf@dustycloud.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=2600:3c02::f03c:91ff:feae:cb51; envelope-from=cwebber@dustycloud.org; helo=dustycloud.org X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org, Dimos Dimakakos Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Spam-Score: -0.01 X-TUID: d6dH2d2jnVcu --=-=-= Content-Type: text/plain Bonface M. K. writes: > Christopher Lemmer Webber > writes: > >> Bonface M. K. writes: >> >>> Christopher Lemmer Webber writes: >>> >>>> Just a heads up that right now you *can* run Racket-on-Chez, but soon >>>> Racket-on-Chez will be the "default"... maybe it's a good idea to see >>>> how hard it will be to make a racket-on-chez package variant. >>>> >>>> I'm interested in looking at that but it'll probably have to be a while >>>> before I can do so... if someone does so before me, I'll be interested >>>> in beta testing at least. >>>> >>>> But also leaving this here as a self-reminder so we aren't surprised >>>> when it becomes a more important thing to deal with :) >>>> >>>> >>> >>> Also, since we are talking about Racket, what >>> happened to having a racket build system? >> >> There's interest in it, and Dimos made interesting progress towards >> figuring out some of the key problems... there's been interest beyond >> that but sadly it seems like organizing the energy to work on it hasn't >> happened for whatever reason yet. >> > > Can I volunteer on this task? There's some work in > my team done in Racket; and it would be of great > interest to us to have a working Racket build > system. I can set apart some time to work on > this. I'd ask for alot of guidance/ review on this > though :) YES! Please do. Let's talk. You can ping me on IRC also, dustyweb on freenode. >> I've offered to throw money at the problem if someone's willing to take >> it on btw... not much, but some money. I've talked to a couple of >> people about that but it stalled in each iteration.... I don't think >> it's impossible but I guess it's one of those tasks that for whatever >> reason seems difficult to get going on because there are some >> complexities around the way Racket builds "collections" that eg don't >> seem to show up in Python. Anyway that's not a judgement because >> despite wanting it fairly badly clearly I haven't made progress on it. >> >> I have the notes that Dimos wrote up not long ago in case anyone is >> interested. Dimos, do you mind if I post them to the list? >> > Please do share the notes! I can try to hack > something up \m/\m/. I looked at the email that Dimos sent to me (also the email I had was apparently not the most recent email that they are using, corrected in the addressing this time), and they had asked me if they should post it to the mailing list, so I think that's consent to post it since they expressed consideration in doing so in our exchange... so I'm attaching it. --=-=-= Content-Type: text/plain Content-Disposition: inline; filename=guix-racket-dimos-dimakakos.org.txt Content-Description: Dimos' notes on Racket on Guix stuff * Basics of Racket packaging system Racket provides three abstractions of how to reuse and move around modules of code. These are: ** Libraries A library in Racket is a single file module that can be used in other files. Libraries that are serving some higher purpose are organised together in Collections. Documentation can be found [[https://docs.racket-lang.org/reference/collects.html][here]]. ** Collections Collections are a number of libraries bundled together. They can be added to the system through packages. Racket gets informed for presence of collections through collection link files. In the filesystem collections are directories that include library files. The default path they are stored is #. The corresponding link file is located at # and includes a list as illustrated below: #+begin_src racket ((root "pkgs/racket-lib") (root "pkgs/main-distribution") (root "pkgs/2d")) #+end_src More specifics about link files and their structure can be found [[https://docs.racket-lang.org/reference/collects.html#%28tech._collection._links._file%29][here]]. More links can be added to arbitary directories through 'raco link'. This file informs racket where a collection resides in the # directory. ** Packages Packages in Racket are the abstractions used to share and move around modules of code. The include a number of libraries in a collection, or more collections. They are the means through which dependencies are defined. The main entry for managing them is 'raco pkg'. The tool for installing packages is 'raco pkg install' that takes as an argument a package source (name of package in a catalog, directory, tar or zip file etc). How it will act is defined by the "info.rkt" file included in the package source. Documentation for [[https://docs.racket-lang.org/pkg/cmdline.html#%28part._raco-pkg-install%29]['raco pkg install']] and [[https://docs.racket-lang.org/pkg/metadata.html]["info.rkt"]]. Racket can be configured for where to install and search for packages through a confing file #. The documentation for the configurations can be found [[https://docs.racket-lang.org/raco/config-file.html?q=raco][here]]. * Racket build system I will get into the thoughts that I have for the various phases of the build procedure: ** unpack phase Here normal procedures form 'gnu-build-system' work just fine. The only exception is the handling of .plt files, which can be done as the .gem files are handled in 'ruby-build-system'. ** bootstrap/build/install phase Since building and installing is done with 'raco pkg install', it makes sense to have a single phase dealing with this. The problems that exist here are: 1. <#path:/etc/racket/config.rktd> needs to be updated with all the the places it needs to look for the packages. These include the following: 1. lib-search-dirs 2. lib-dir 3. bin-dir 4. links-search-files 5. pkgs-search-dirs 6. collects-search-dirs 7. doc-search-dirs This needs to occur two times. First time to build the package, including just its inputs. Then after/while installing the package, we need to create a new config.rktd that includes the proper places for all formerly and newly installed packages. 2. Racket packages have circular dependencies. 3. Racket tries to rebuild the dependencies for the package it installs, even when not needed, based on timestamps. There exists an environmental variable "PLT_COMPILED_FILE_CHECK", that in documentation says if it's set on exist this won't happen. This doesn't seem to work as intended. 4. Documentation is created by mutating the docs structure and adding new links to new nodes of documentation. For the *first issue*, Claes Wallin that worked on racket2nix, in fact recreates the racket structure in the outputs folder by copying in the /usr/share/collects, and /lib/racket folders, and creating a symlink of /usr/share/racket/include. He then creates uses the store provided binaries of racket(racket, raco, gracket) with -G flag to set the config directory in the outputs and -X flat to set the collects directory. This way the packages are built and installed in the racket configuration we created at the outputs. Then the config.rktd needs to be generated. Claes Wallin does this with a racket script, which we could use, but could also just use a guile function. I have implemented this, and works as expected, in most cases that don't meet the following issues. The *second issue* is resolved by the importer in racket2nix. The derivations are expanded to include circular dependencies and while building the offending packages, dummy inputs are created. We can do the same using some DFS guile package. Are there any standard graph libraries in guile? The *third issue* is more complicated. Racket will try to to recompile dependencies based on timestamps. This creates issues during the build. In racket2nix there is created a separate environment where all dependencies are writable. It is an ugly workaround, but seems to work. As for the *fourth issue* I didn't research the racket documentation system a lot, especially since it's packages are the of the main offenders in circulars dependencies. Binaries of racket packages are placed in "launchers" in the racket configuration folder, so symlinks to '/bin' shall be created. ** testing phase It's mostly trivial with 'raco test'. * Racket importer Racket packages are defined with "info.rkt" files. Since it's sexp-based it's trivial to parse them and create package-definitons in guile. There exists some edge-cases like 'implies' that change the way the packages are upgraded, but in general it pretty simple. Here is also the place where maybe we could solve the thing with circular dependencies. Is it possible to create custom fields for inputs in guix derivations? --=-=-=--