From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id oAtyBhMblWMLYwEAbAwnHQ (envelope-from ) for ; Sun, 11 Dec 2022 00:49:39 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id wNriBBMblWN9JgAAG6o9tA (envelope-from ) for ; Sun, 11 Dec 2022 00:49:39 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 42BD6DE81 for ; Sun, 11 Dec 2022 00:49:37 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p49aZ-00063L-3U; Sat, 10 Dec 2022 18:49:03 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p49aX-00063B-Cq for guix-devel@gnu.org; Sat, 10 Dec 2022 18:49:01 -0500 Received: from mout02.posteo.de ([185.67.36.66]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p49aS-0002pJ-Rx for guix-devel@gnu.org; Sat, 10 Dec 2022 18:49:01 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id E00F5240101 for ; Sun, 11 Dec 2022 00:48:52 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.de; s=2017; t=1670716132; bh=z0GoGi+lagQIzYWspo6Ip6z1C2b9+JLx+1Lqk/t/nW4=; h=From:To:Cc:Subject:Date:From; b=NvvDr4vVx1Br0W12o5RGftuFhJHcmx8knMCGHA4t6xAt099S20kWB6p7ySh0LprTd 88xUAMIcVVtSY5+hr6n8HxG2Rs5XVcDjyvNSynGX5cYJSzYAAU+Qpz8FyG7Yj5e/BZ /mXMqpTkjVA81xbD/HgIWXFHe4Exo2LPgxF3hx0vieDWDmntBFfYki5z7Tb9BA8IxF FP0BccOvk8tiv3gmwDaxRh2Ri0hlaO1fWckQVOSFe7iKl+wm7o/wUZU4pYj+NSTmLT v0xGuvTR8SNfn3JtVRTmtjbSnGDrHcWe5nKJT3BVLvBxQUMcn88uTpXbeVOnlZgwuf BcBpIBJFmBlWg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4NV4NM0Pnwz6tsX; Sun, 11 Dec 2022 00:48:51 +0100 (CET) References: From: Mekeor Melire To: "(" Cc: guix-devel@gnu.org Subject: Re: Dissecting Guix -- blog post series Date: Sat, 10 Dec 2022 21:25:49 +0000 In-reply-to: Message-ID: <87k02ystuz.fsf@posteo.de> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Received-SPF: pass client-ip=185.67.36.66; envelope-from=mekeor@posteo.de; helo=mout02.posteo.de X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, WEIRD_QUOTING=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1670716178; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=p4L0cSp2VxolBjkgxMgRHabXRUtMJJLI6EBbnkZWjPE=; b=ENpB2vcp/JL7I2+09WDACezV6PVryWB9CF3bfIS4BJEdk8zmYM6neCo/AD4D/Txoh6Pwxe n+8LaaD8BfKEqCcaxkgbnsrn+oZOl6XIbDc8s9kWGJr61D/V+1eAQMfsT1DQfwH9q5lrX0 2ZwGLM/JZ+LEdQnR8gt5ramvhyB02wzIdQKEFU3P17DGvlE10p+7oYT8Y76SfdlU2h9tQ+ QGywIR77YnvgO4sL4VGaEBIzEo0g/Wd1pIBH1TQChOfiwvXh1l/0GrTLQmdCc+VoDtRxPX XNpGJ9unBKnYGwKdFql6ebof23GrmmhubnxZOaSMG6LQoC/ZPCk23wWQ2TCyQw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=posteo.de header.s=2017 header.b=NvvDr4vV; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.de ARC-Seal: i=1; s=key1; d=yhetil.org; t=1670716178; a=rsa-sha256; cv=none; b=YhWj5joKxlh36l8himHcuUYhzJbMYhagntHC4MlHmPcgqvzcuF7X1CAOHx6fh/7epXVojl HDj7X6br/j7lDhbAgJSACaTYIWzA59PgjbSMYrUs2ELrK2cXZcASVbaozIx8HFSH6DhPZn Qqyp3aocY+ZJ+cYqXCIqq0K5vyIjijWDJQPDJ3golyNnrwoQSRM7Pg06iyV9dKnUSrhEFs IrhKAST7609FpyWHrqonFWahbHyWlyTzctMl29POgYbMoTwEFDJ2mKDYGHjHoJBKdcam2+ 7ULmqoKhTQfwXFpmwlWb5Y/iIADkBomsxuIv6ynawLjHNko8ZeIxhkoqO5oLSg== X-Migadu-Spam-Score: -7.61 X-Spam-Score: -7.61 X-Migadu-Queue-Id: 42BD6DE81 X-Migadu-Scanner: scn0.migadu.com Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=posteo.de header.s=2017 header.b=NvvDr4vV; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org"; dmarc=pass (policy=none) header.from=posteo.de X-TUID: vcXzqU5Ipy/H --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable 2022-12-08 18:24 paren@disroot.org: > Some of you may have seen on IRC that I've been writing a post for the Gu= ix blog that I hope will form the first part of a series. This series aims = to dissect the internals of Guix, from bottom to top. Great, I'm looking forward to read it! Especially, personally, I'm eager fo= r the rather fundamental concepts of Guix (and Nix). > Perhaps they could go in the cookbook once the series is done? Yes, I personally think the content should be published at two places at th= e same time: OTOH, it should either be incorporated into either the manual = or the cookbook; and OTOH, it should be published as a blog-post. The blog-= post should also link to the respective section of the manual or cookbook i= n a preamble. > * Dissecting Guix, Part 1: Derivations > * Dissecting Guix, Part 2: The Humble G-Expression > * Dissecting Guix, Part 3: Packages > * Dissecting Guix, Part 4: Monads > * Dissecting Guix, Part 5: Profiles and Search Paths > * Dissecting Guix, Part 6: Goings-On in the Build Container > * Dissecting Guix, Part 7: Record Types > * Dissecting Guix, Part 8: Substitutes and Grafts > * Dissecting Guix, Part 9: Cross-Compilation I think it'd also be fine to come up with the titles during the process of = writing as sometimes that process itself is insightful. E.g. I could imagin= e parts 2 and 4 to collapse. Maybe, maybe not, you'll see. > * Dissecting Guix, Part 10: Services > > Walks you through the process of creating a service, and thouroughly expl= ains system configuration. How'd this part differ from section "12.18 Defining Services" of the manual? > * Dissecting Guix, Part 11: Home Services > * Dissecting Guix, Part 12: Writing a Subcommand > * Dissecting Guix, Part 13: Lending a Hand > > How to edit the Guix source code and submit patches to be reviewed by the= lovely Guix community! How'd this part differ from sections 22 and "22.6 Submitting Patches" from = the manual? Now, as for the actual article. Firstly, I added some comments below. Secon= dly, I created a "patch" suggesting some changes. By the way, the text does not seem to strictly fill columns at a certain li= ne width. So I did not bother to fill columns in the "patch". > title: Dissecting Guix, Part 1: Derivations and Derivation > date: TBC > author: ( > tags: Dissecting Guix, Functional package management, Programming interfa= ces, Scheme API > --- > To a new user, Guix's functional architecture can seem quite alien, and > possibly offputting. With a combination of extensive `#guix`-querying, > determined manual-reading, and plenty of source-perusing, they may > eventually figure out how everything fits together by themselves, but this > can be frustrating and often takes a fairly long time. > > However, once you peel back the layers, the "Nix way" is actually rather > elegant, if perhaps not as simple as the mutable, imperative style > implemented by the likes of [`dpkg`](https://wiki.debian.org/dpkg) and, > [`pacman`](https://wiki.archlinux.org/title/pacman). This series of blog > posts will cover basic Guix concepts, taking a "ground-up" approach by > dealing with lower-level concepts first, and hopefully make those months = of > information-gathering unnecessary. > > Before we dig in to Guix-specific concepts, we'll need to learn about one > inherited from [Nix](https://nixos.org), the original functional package > manager and the inspiration for Guix; the idea of a _derivation_ and its > corresponding _store items_. > > These concepts were originally described by Eelco Dolstra, the author of = Nix, > in their [PhD thesis](https://edolstra.github.io/pubs/phd-thesis.pdf); see > _=C2=A7 2.1 The Nix store_ and _=C2=A7 2.4 Store Derivations_. > > # Store Items > > As you almost certainly know, everything that Guix builds is stored in the > _store_, which is almost always the `/gnu/store` directory. It's the job= of > the [`guix-daemon`](https://guix.gnu.org/manual/en/html_node/Invoking-gui= x_002ddaemon.html) > to manage the store and build things. If you run > [`guix build PKG`](https://guix.gnu.org/manual/en/html_node/Invoking-guix= -build.html), > `PKG` will be built or downloaded from a substitute server if available, = and > a path to an item in the store will be displayed. > > ``` > $ guix build irssi > /gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3 > ``` > > This item contains the final result of building [`irssi`](https://irssi.o= rg). > Let's peek inside: > > ``` > $ ls $(guix build irssi) > bin/ etc/ include/ lib/ share/ > $ ls $(guix build irssi)/bin > irssi* > ``` > > `irssi` is quite a simple package. What about a more complex one, like > [`glib`](https://docs.gtk.org/glib)? > > ``` > $ guix build glib > /gnu/store/bx8qq76idlmjrlqf1faslsq6zjc6f426-glib-2.73.3-bin > /gnu/store/j65bhqwr7qq7l77nj0ahmk1f1ilnjr3a-glib-2.73.3-debug > /gnu/store/3pn4ll6qakgfvfpc4mw89qrrbsgj3jf3-glib-2.73.3-doc > /gnu/store/dvsk6x7d26nmwsqhnzws4iirb6dhhr1d-glib-2.73.3 > /gnu/store/4c8ycz501n2d0xdi4blahvnbjhd5hpa8-glib-2.73.3-static > ``` > > `glib` produces five `/gnu/store` items, because it's possible for a pack= age > to produce multiple [outputs](https://guix.gnu.org/manual/en/html_node/Pa= ckages-with-Multiple-Outputs.html). > Each output can be referred to separately, by prefixing a package's name = with > `:OUTPUT` where supported. For example, this > [`guix install`](https://guix.gnu.org/manual/en/html_node/Invoking-guix-p= ackage.html) > invocation will add `glib`'s `bin` output to your profile: > > ``` > $ guix install glib:bin > ``` > > The default output is `out`, so when you pass `glib` by itself to that > command, it will actually install `glib:out` to the profile. > > `guix build` also provides the `--source` flag, which produces the store > item corresponding to the given package's downloaded source code. > > ``` > $ guix build --source irssi > /gnu/store/cflbi4nbak0v9xbyc43lamzl4a539hhb-irssi-1.4.3.tar.xz > $ guix build --source glib > /gnu/store/d22wzjq3xm3q8hwnhbgk2xd3ph7lb6ay-glib-2.73.3.tar.xz > ``` > > But how does Guix know how to build these store outputs in the first plac= e? > That's where derivations come in. > > # `.drv` Files > > You've probably seen these being printed by the Guix CLI now and again. > Derivations, represented in the daemon's eyes by `.drv` files, contain > instructions for building store items. We can retrieve the paths of > these `.drv` files with the `guix build --derivations` command: > > ``` > $ guix build --derivations irssi > /gnu/store/zcgmhac8r4kdj2s6bcvcmhh4k35qvihx-irssi-1.4.3.drv > ``` > > `guix build` can actually also accept derivation paths as an argument, in > lieu of a package, like so: > > ``` > $ guix build /gnu/store/zcgmhac8r4kdj2s6bcvcmhh4k35qvihx-irssi-1.4.3.drv > /gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3 > ``` > > Let's look inside this derivation file. > > ``` > Derive([("out","/gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3",= "","")],[("/gnu/store/9mv9xg4kyj4h1cvsgrw7b9x34y8yppph-glib-2.70.2.drv",["o= ut"]),("/gnu/store/baqpbl4wck7nkxrbyc9nlhma7kq5dyfl-guile-2.0.14.drv",["out= "]),("/gnu/store/bfirgq65ndhf63nn4q6vlkbha9zd931q-openssl-1.1.1l.drv",["out= "]),("/gnu/store/gjwpqzvfhz13shix6a6cs2hjc18pj7wy-module-import-compiled.dr= v",["out"]),("/gnu/store/ij8651x4yh53hhcn6qw2644nhh2s8kcn-glib-2.70.2.drv",= ["out"]),("/gnu/store/jg2vv6yc2yqzi3qzs82dxvqmi5k21lhy-irssi-1.4.3.drv",["o= ut"]),("/gnu/store/qggpjl9g6ic3cq09qrwkm0dfsdjf7pyr-glibc-utf8-locales-2.33= .drv",["out"]),("/gnu/store/zafabw13yyhz93jwrcz7axak1kn1f2cx-openssl-1.1.1s= .drv",["out"])],["/gnu/store/af18nrrsk98c5a71h3fifnxg1zi5mx7y-module-import= ","/gnu/store/qnrwmby5cwqdqxyiv1ga6azvakmdvgl7-irssi-1.4.3-builder"],"x86_6= 4-linux","/gnu/store/hnr4r2d0h0xarx52i6jq9gvsrlc3q81a-guile-2.0.14/bin/guil= e",["--no-auto-compile","-L","/gnu/store/af18nrrsk98c5a71h3fifnxg1zi5mx7y-m= odule-import","-C","/gnu/store/6rkkvvb7pl1l9ng8vvywvwf357vhm3va-module-impo= rt-compiled","/gnu/store/qnrwmby5cwqdqxyiv1ga6azvakmdvgl7-irssi-1.4.3-build= er"],[("allowSubstitutes","0"),("guix properties","((type . graft) (graft (= count . 2)))"),("out","/gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.= 4.3"),("preferLocalBuild","1")]) > ``` > > It's... not exactly human-readable. We could try to format it and break > it down, but it'd still be pretty hard to understand, since `.drv` files > contain no labels for the "arguments" or any other human-readable indicat= ors. > Instead, we're going to explore derivations in a Guile REPL. > > # Exploring Guix Interactively > > Before we continue, we'll want to start a REPL, so that we can try out > the Guix Guile API interactively. To run a REPL in the terminal, simply > [call `guix repl`](https://guix.gnu.org/manual/devel/en/html_node/Using-G= uix-Interactively.html). > > If you're using Emacs, you can instead install [Geiser](https://nongnu.or= g/geiser), > which provides a comfortable Emacs UI for various Lisp REPLs, invoke > `guix repl --listen=3Dtcp:37146 &`, and type `M-x geiser-connect RET RET = RET` to > connect to the running Guile instance. This approach did not work nicely for me. In particular, the last command m= ade Emacs freeze (even with and 'emacs -Q'). And then, when evaluating the = first code block below, i.e. the module-import, I got the warning "Unknown = meta command: geiser-no-values". That's why, in the "patch", I suggest to s= imply run 'M-x geiser RET' which works just fine. (I guess this is due to the fact that "Geiser invokes geiser-guile-binary w= ith certain flags": .) > There are a few Guix modules we'll need. Run this Scheme code to import > them: > > ```scheme > (use-modules (guix derivations) > (guix gexp) > (guix packages) > (guix store) > (gnu packages glib) > (gnu packages irc)) > ``` > > We now have access to the store, G-expression, package, and derivation > APIs, along with the `irssi` and `glib` `` objects. > > # Creating a `` > > The Guix API for derivations revolves around the `` record, > which is the Scheme representation of that whole block of text surrounded= by > `Derive(...)`. If we look in `guix/derivations.scm`, we can see that it's > defined like this: > > ```scheme > (define-immutable-record-type > (make-derivation outputs inputs sources system builder args env-vars > file-name) > derivation? > (outputs derivation-outputs) ; list of name/ p= airs > (inputs derivation-inputs) ; list of > (sources derivation-sources) ; list of store paths > (system derivation-system) ; string > (builder derivation-builder) ; store path > (args derivation-builder-arguments) ; list of strings > (env-vars derivation-builder-environment-vars) ; list of name/value pa= irs > (file-name derivation-file-name)) ; the .drv file name > ``` > > With the exception of `file-name`, each of those fields corresponds to > an "argument" in the `Derive(...)` form. Before we can examine them, > though, we need to figure out how to _lower_ that `irssi` `` > object into a derivation. > > The procedure we use to turn a high-level object like `` into a > derivation is called `lower-object`; more on that in a future post. > However, this doesn't produce a derivation: > > ```scheme > (pk (lower-object irssi)) > ;;; (#) > ``` > > `pk` is an abbreviation for the procedure `peek`, which takes the given > object, writes a representation of it to the output, and returns it. > It's especially handy when you want to view an intermediate value in a > complex expression. > > The returned object is a procedure that needs to be evaluated in the > context of a store connection. We do this by first using `with-store` > to connect to the store and bind the connection to a name, then wrapping > the `lower-object` call with `run-with-store`: > > ```scheme > (define irssi-drv > (pk (with-store %store > (run-with-store %store > (lower-object irssi))))) > ;;; (# /gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3 7fe1902b6= 140>) > > (define glib-drv > (pk (with-store %store > (run-with-store %store > (lower-object glib))))) > ;;; (# /gnu/store/lp7k9ygvpwxgxjvmf8bix8d2aar0azr7-glib-2.70.2-bin /gnu/= store/22mkp8cr6rxg6w8br9q8dbymf51b44m8-glib-2.70.2-debug /gnu/store/a6qb5ar= vir4vm1zlkp4chnl7d8qzzd7x-glib-2.70.2 /gnu/store/y4ak268dcdwkc6lmqfk9g1dgk2= jr9i34-glib-2.70.2-static 7fe17ca13b90>) > ``` > > And we have liftoff! Now we've got two `` records to play > with. > > # Exploring `` I suggest to use subheadings within this chapter because otherwise readers = loose the overview. > The first "argument" in the `.drv` file is `outputs`, which tells the > Guix daemon about the outputs that this build can produce: > > ```scheme > (define irssi-outputs > (pk (derivation-outputs irssi-drv))) > ;;; ((("out" . #< path: "/gnu/store/v5pd69j3hjs1fck4b5= p9hd91wc8yf5qx-irssi-1.4.3" hash-algo: #f hash: #f recursive?: #f>))) > > (pk (assoc-ref irssi-outputs "out")) > > (define glib-outputs > (pk (derivation-outputs glib-drv))) > ;;; ((("bin" . #< path: "/gnu/store/lp7k9ygvpwxgxjvmf8= bix8d2aar0azr7-glib-2.70.2-bin" hash-algo: #f hash: #f recursive?: #f>) ("d= ebug" . #< path: "/gnu/store/22mkp8cr6rxg6w8br9q8dbymf51= b44m8-glib-2.70.2-debug" hash-algo: #f hash: #f recursive?: #f>) ("out" . #= < path: "/gnu/store/a6qb5arvir4vm1zlkp4chnl7d8qzzd7x-gli= b-2.70.2" hash-algo: #f hash: #f recursive?: #f>) ("static" . #< path: "/gnu/store/y4ak268dcdwkc6lmqfk9g1dgk2jr9i34-glib-2.70.2-sta= tic" hash-algo: #f hash: #f recursive?: #f>))) > > (pk (assoc-ref glib-outputs "bin")) > ;;; (#< path: "/gnu/store/lp7k9ygvpwxgxjvmf8bix8d2aar0= azr7-glib-2.70.2-bin" hash-algo: #f hash: #f recursive?: #f>) > ``` > > It's a simple association list mapping output names to `` > records, and it's equivalent to the first "argument" in the `.drv` file: > > ``` > [ ("out", "/gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3", "", = "") > ] > ``` > > The `hash-algo` and `hash` fields are for storing the content hash and the > algorithm used with that hash for what we term a _fixed-output derivation= _, > which is essentially a derivation where we know what the hash of the cont= ent > will be in advance. For instance, `origin`s produce fixed-output derivat= ions: > > ```scheme > (define irssi-src-drv > (pk (with-store %store > (run-with-store %store > (lower-object (package-source irssi)))))) > ;;; (# /gnu/store/cflbi4nbak0v9xbyc43lamzl4a539hhb-irssi-1.4.3.ta= r.xz 7fe17b3c8d70>) > > (define irssi-src-outputs > (pk (derivation-outputs irssi-src-drv))) > ;;; ((("out" . #< path: "/gnu/store/cflbi4nbak0v9xbyc4= 3lamzl4a539hhb-irssi-1.4.3.tar.xz" hash-algo: sha256 hash: #vu8(185 63 113 = 82 35 163 34 230 127 66 182 26 8 165 18 174 41 227 75 212 165 61 127 34 55 = 102 102 10 170 90 4 52) recursive?: #f>))) > > (pk (assoc-ref irssi-src-outputs "out")) > ;;; (#< path: "/gnu/store/cflbi4nbak0v9xbyc43lamzl4a53= 9hhb-irssi-1.4.3.tar.xz" hash-algo: sha256 hash: #vu8(185 63 113 82 35 163 = 34 230 127 66 182 26 8 165 18 174 41 227 75 212 165 61 127 34 55 102 102 10= 170 90 4 52) recursive?: #f>) > ``` > > Note how the `hash` and `hash-algo` now have values. > > Perceptive readers may note that the `` has four field= s, > whereas the tuple in the `.drv` file only has three (minus the label). If > we read the source for `write-derivation`, we can see that the `recursive= ?` > field is serialised by prefixing the `hash-algo` with `r:` if it's true: > > ```scheme > ;;; guix/derivations.scm:630:2 > > (define (write-output output port) > (match output > ((name . ($ path hash-algo hash recursive?)) > (write-tuple (list name path > (if hash-algo > (string-append (if recursive? "r:" "") > (symbol->string hash-algo)) > "") > (or (and=3D> hash bytevector->base16-string) > "")) > write-escaped-string > port)))) > ``` > > The purpose of `recursive?` is difficult to explain, and is out of scope = for > this post. If the purpose is out of scope, then we should not dive in that deeply. Esp= ecially, I'd suggest skip the code snippet. > The next field is `inputs`, which corresponds to, you guessed it, the > second pseudo-"argument" in the `.drv` file format: > > ``` > [ ("/gnu/store/9mv9xg4kyj4h1cvsgrw7b9x34y8yppph-glib-2.70.2.drv", ["out"]= ), > ("/gnu/store/baqpbl4wck7nkxrbyc9nlhma7kq5dyfl-guile-2.0.14.drv", ["out"= ]), > ("/gnu/store/bfirgq65ndhf63nn4q6vlkbha9zd931q-openssl-1.1.1l.drv", ["ou= t"]), > ("/gnu/store/gjwpqzvfhz13shix6a6cs2hjc18pj7wy-module-import-compiled.dr= v", ["out"]), > ("/gnu/store/ij8651x4yh53hhcn6qw2644nhh2s8kcn-glib-2.70.2.drv", ["out"]= ), > ("/gnu/store/jg2vv6yc2yqzi3qzs82dxvqmi5k21lhy-irssi-1.4.3.drv", ["out"]= ), > ("/gnu/store/qggpjl9g6ic3cq09qrwkm0dfsdjf7pyr-glibc-utf8-locales-2.33.d= rv", ["out"]), > ("/gnu/store/zafabw13yyhz93jwrcz7axak1kn1f2cx-openssl-1.1.1s.drv", ["ou= t"]) > ] > ``` > > Here, each tuple specifies a derivation that needs to be built before this > derivation can be built, and the outputs of the derivation that the build > process of this derivation uses. Let's grab us the Scheme equivalent: > > ```scheme > (define irssi-inputs > (pk (derivation-inputs irssi-drv))) > ;;; [a fairly large amount of output] > > (pk (car irssi-inputs)) > ;;; (#< drv: # /gnu/store/2jj2mxn6wfrcw7i85nywk71mmqbn= hzps-glib-2.70.2 7fe1902b6640> sub-derivations: ("out")>) > ``` > > Unlike `derivation-outputs`, `derivation-inputs` maps 1:1 to the `.drv` > form; the `drv` field is a `` to be built, and the > `sub-derivations` field is a list of outputs. > > The other pseudo-"arguments" are pretty simple; none of them involve new Here you write "pretty simple". Later you also write "self-explanatory" and= "obviously". I suggest to let the reader decide what's simple. > records. The third is `derivation-sources`, which contains a list of > all store items used in the build which aren't themselves built using > derivations, whereas `derivation-inputs` contains the dependencies > which are. > > This list usually just contains the path to the Guile _build script_ that > realises the store items when run, which we'll examine in a later post, a= nd > the path to a directory containing extra modules to add to the build scri= pt's > `%load-path`, called `/gnu/store/...-module-import`. > > The next "argument" is self-explanatory: `derivation-system`, which speci= fies > the Nix system we're building for. Next is `derivation-builder`, pointin= g to > the `guile` executable that runs the script; and the second-to-last is > `derivation-args`, which is a list of arguments to pass to `derivation-bu= ilder`. > Note how we use `-L` and `-C` to extend the Guile `%load-path` and > `%load-compiled-path` to include the `module-import` and `module-import-c= ompiled` > directories: > > ```scheme > (pk (derivation-system irssi-drv)) > ;;; ("x86_64-linux") > > (pk (derivation-builder irrsi-drv)) > ;;; ("/gnu/store/hnr4r2d0h0xarx52i6jq9gvsrlc3q81a-guile-2.0.14/bin/guile") > > (pk (derivation-builder-arguments irrsi-drv)) > ;;; (("--no-auto-compile" "-L" "/gnu/store/af18nrrsk98c5a71h3fifnxg1zi5mx= 7y-module-import" "-C" "/gnu/store/6rkkvvb7pl1l9ng8vvywvwf357vhm3va-module-= import-compiled" "/gnu/store/qnrwmby5cwqdqxyiv1ga6azvakmdvgl7-irssi-1.4.3-b= uilder")) > ``` > > The final "argument" contains a list of environment variables to set befo= re > we start the build process: > > ```scheme > (pk (derivation-builder-environment-vars irssi-drv)) > ;;; ((("allowSubstitutes" . "0") ("guix properties" . "((type . graft) (g= raft (count . 2)))") ("out" . "/gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-= irssi-1.4.3") ("preferLocalBuild" . "1"))) > ``` > > Obviously, the last record field, `derivation-file-name`, simply allows y= ou to Before this point, the record fields have been called '"argument"'s all the= time. I think it's not nice style to carry on a quoted term through many p= aragraphs like this. Let's simply write "record field" all the time, instea= d. > retrieve the path to the `.drv` file in Scheme, and so isn't > represented in a serialised derivation. Speaking of serialisation, to > convert between the `.drv` text format and the Scheme `` > record, you can use `write-derivation`, `read-derivation`, and > `read-derivation-from-file`. I think an example for invoking read-derivation-from-file would round up th= is tutorial really nicely because it'd close the circle between .drv-files = and -objects. > # Conclusion > > Derivations are one of Guix's most important concepts, but are fairly eas= y to > understand once you get past the obtuse `.drv` file format. They provide= the > Guix daemon with the initial instructions that it uses to build store ite= ms > like packages, origins, and other file-likes such as `computed-file` and > `local-file`, which will be discussed in a future post! Here, in the conclusion, IMHO, there could be another brief listing of all = fields of a derivation. > #### About GNU Guix > > [GNU Guix](https://guix.gnu.org) is a transactional package manager and > an advanced distribution of the GNU system that [respects user > freedom](https://www.gnu.org/distros/free-system-distribution-guidelines.= html). > Guix can be used on top of any system running the Hurd or the Linux > kernel, or it can be used as a standalone operating system distribution > for i686, x86_64, ARMv7, AArch64 and POWER9 machines. > > In addition to standard package management features, Guix supports > transactional upgrades and roll-backs, unprivileged package management, > per-user profiles, and garbage collection. When used as a standalone > GNU/Linux distribution, Guix offers a declarative, stateless approach to > operating system configuration management. Guix is highly customizable > and hackable through [Guile](https://www.gnu.org/software/guile) > programming interfaces and extensions to the > [Scheme](http://schemers.org) language. --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-many-different-rather-small-changes.patch >From 2abcfaff2f3203d5e568044beffbb964684b99a0 Mon Sep 17 00:00:00 2001 From: user <> Date: Sun, 11 Dec 2022 00:44:07 +0100 Subject: [PATCH] many different, rather small changes --- one.md | 52 +++++++++++++++++++++++++++++++++------------------- 1 file changed, 33 insertions(+), 19 deletions(-) diff --git a/one.md b/one.md index cb2c9d9..ccd0397 100644 --- a/one.md +++ b/one.md @@ -4,7 +4,7 @@ author: ( tags: Dissecting Guix, Functional package management, Programming interfaces, Scheme API --- To a new user, Guix's functional architecture can seem quite alien, and -possibly offputting. With a combination of extensive `#guix`-querying, +possibly be off-putting. With a combination of extensive `#guix`-querying, determined manual-reading, and plenty of source-perusing, they may eventually figure out how everything fits together by themselves, but this can be frustrating and often takes a fairly long time. @@ -118,7 +118,7 @@ Derive([("out","/gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3","","")] It's... not exactly human-readable. We could try to format it and break it down, but it'd still be pretty hard to understand, since `.drv` files -contain no labels for the "arguments" or any other human-readable indicators. +contain no labels for their fields or any other human-readable indicators. Instead, we're going to explore derivations in a Guile REPL. # Exploring Guix Interactively @@ -128,9 +128,8 @@ the Guix Guile API interactively. To run a REPL in the terminal, simply [call `guix repl`](https://guix.gnu.org/manual/devel/en/html_node/Using-Guix-Interactively.html). If you're using Emacs, you can instead install [Geiser](https://nongnu.org/geiser), -which provides a comfortable Emacs UI for various Lisp REPLs, invoke -`guix repl --listen=tcp:37146 &`, and type `M-x geiser-connect RET RET RET` to -connect to the running Guile instance. +which provides a comfortable Emacs UI for various Lisp REPLs, run +`M-x geiser RET`. There are a few Guix modules we'll need. Run this Scheme code to import them: @@ -149,7 +148,7 @@ APIs, along with the `irssi` and `glib` `` objects. # Creating a `` -The Guix API for derivations revolves around the `` record, +The Guix API for derivations revolves around the [`` record](https://guix.gnu.org/en/manual/en/html_node/Derivations.html), which is the Scheme representation of that whole block of text surrounded by `Derive(...)`. If we look in `guix/derivations.scm`, we can see that it's defined like this: @@ -170,7 +169,7 @@ defined like this: ``` With the exception of `file-name`, each of those fields corresponds to -an "argument" in the `Derive(...)` form. Before we can examine them, +a field in the `Derive(...)` form. Before we can examine them, though, we need to figure out how to _lower_ that `irssi` `` object into a derivation. @@ -183,7 +182,7 @@ However, this doesn't produce a derivation: ;;; (#) ``` -`pk` is an abbreviation for the procedure `peek`, which takes the given +`pk` is an abbreviation for the Guile procedure `peek`, which takes the given object, writes a representation of it to the output, and returns it. It's especially handy when you want to view an intermediate value in a complex expression. @@ -212,7 +211,9 @@ with. # Exploring `` -The first "argument" in the `.drv` file is `outputs`, which tells the +## Outputs + +The first derivation record field in the `.drv` file is `outputs`, which tells the Guix daemon about the outputs that this build can produce: ```scheme @@ -221,6 +222,7 @@ Guix daemon about the outputs that this build can produce: ;;; ((("out" . #< path: "/gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3" hash-algo: #f hash: #f recursive?: #f>))) (pk (assoc-ref irssi-outputs "out")) +;;; (#< path: "/gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.1" hash-algo: #f hash: #f recursive?: #f>) (define glib-outputs (pk (derivation-outputs glib-drv))) @@ -231,7 +233,7 @@ Guix daemon about the outputs that this build can produce: ``` It's a simple association list mapping output names to `` -records, and it's equivalent to the first "argument" in the `.drv` file: +records, and it's equivalent to the first field in the `.drv` file: ``` [ ("out", "/gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3", "", "") @@ -260,7 +262,7 @@ will be in advance. For instance, `origin`s produce fixed-output derivations: Note how the `hash` and `hash-algo` now have values. -Perceptive readers may note that the `` has four fields, +You might have noted that the `` has four fields, whereas the tuple in the `.drv` file only has three (minus the label). If we read the source for `write-derivation`, we can see that the `recursive?` field is serialised by prefixing the `hash-algo` with `r:` if it's true: @@ -285,8 +287,10 @@ field is serialised by prefixing the `hash-algo` with `r:` if it's true: The purpose of `recursive?` is difficult to explain, and is out of scope for this post. +## Inputs + The next field is `inputs`, which corresponds to, you guessed it, the -second pseudo-"argument" in the `.drv` file format: +second field in the `.drv` file format: ``` [ ("/gnu/store/9mv9xg4kyj4h1cvsgrw7b9x34y8yppph-glib-2.70.2.drv", ["out"]), @@ -317,8 +321,9 @@ Unlike `derivation-outputs`, `derivation-inputs` maps 1:1 to the `.drv` form; the `drv` field is a `` to be built, and the `sub-derivations` field is a list of outputs. -The other pseudo-"arguments" are pretty simple; none of them involve new -records. The third is `derivation-sources`, which contains a list of all +## Sources + +The third is `derivation-sources`, which contains a list of all store items used in the build which aren't themselves built using derivations, whereas `derivation-inputs` contains the dependencies which are. @@ -328,8 +333,10 @@ realises the store items when run, which we'll examine in a later post, and the path to a directory containing extra modules to add to the build script's `%load-path`, called `/gnu/store/...-module-import`. -The next "argument" is self-explanatory: `derivation-system`, which specifies -the Nix system we're building for. Next is `derivation-builder`, pointing to +## System, Builder and Arguments + +The next record field is `derivation-system`, which specifies +the Nix system we're building for. Then, there is `derivation-builder`, pointing to the `guile` executable that runs the script; and the second-to-last is `derivation-args`, which is a list of arguments to pass to `derivation-builder`. Note how we use `-L` and `-C` to extend the Guile `%load-path` and @@ -347,7 +354,7 @@ directories: ;;; (("--no-auto-compile" "-L" "/gnu/store/af18nrrsk98c5a71h3fifnxg1zi5mx7y-module-import" "-C" "/gnu/store/6rkkvvb7pl1l9ng8vvywvwf357vhm3va-module-import-compiled" "/gnu/store/qnrwmby5cwqdqxyiv1ga6azvakmdvgl7-irssi-1.4.3-builder")) ``` -The final "argument" contains a list of environment variables to set before +The final record field contains a list of environment variables to set before we start the build process: ```scheme @@ -355,9 +362,16 @@ we start the build process: ;;; ((("allowSubstitutes" . "0") ("guix properties" . "((type . graft) (graft (count . 2)))") ("out" . "/gnu/store/v5pd69j3hjs1fck4b5p9hd91wc8yf5qx-irssi-1.4.3") ("preferLocalBuild" . "1"))) ``` -Obviously, the last record field, `derivation-file-name`, simply allows you to +## File Name + +The last record field, `derivation-file-name`, simply allows you to retrieve the path to the `.drv` file in Scheme, and so isn't represented -in a serialised derivation. Speaking of serialisation, to convert between the +in a serialised derivation. But how to do the inverse? How to read a +derivation from a path? + +# Reading a derivation from file + +Speaking of serialisation, to convert between the `.drv` text format and the Scheme `` record, you can use `write-derivation`, `read-derivation`, and `read-derivation-from-file`. -- 2.38.1 --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0000-initialize-markdown.patch --=-=-=--