unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
blob 32f5d51c1507de7a14f4eda10734c0a8eb3d281d 28170 bytes (raw)
name: website/posts/dissecting-guix-3-gexps.md 	 # note: path name is non-authoritative(*)

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
 
title: Dissecting Guix, Part 3: G-Expressions
date: TBC
author: (
tags: Dissecting Guix, Functional package management, Programming interfaces, Scheme API
---
Welcome back to [Dissecting Guix](https://guix.gnu.org/en/blog/tags/dissecting-guix)!
Last time, we discussed [monads](https://guix.gnu.org/en/blog/2023/dissecting-guix-part-2-the-store-monad),
the functional programming idiom used by Guix to thread a store connection
through a series of store-related operations.

Today, we'll be talking about a concept rather more specific to Guix:
_g-expressions_.  Being an implementation of the Scheme language, Guile is built
around [_s-expressions_](https://en.wikipedia.org/wiki/S-expression), which can
represent, as the saying goes, _code as data_, thanks to the simple structure of
Scheme forms.

As Guix's package recipes are written in Scheme, it naturally needs some way to
represent code that is to be run only when the package is built.  Additionally,
there needs to be some way to reference dependencies and retrieve output paths;
otherwise, you wouldn't be able to, for instance, create a phase to install a
file in the output directory.

So, how do we implement this "deferred" code?  Well, initially Guix used plain
old s-expressions for this purpose.

# Once Upon a Time

Let's say we want to create a store item that's just a symlink to the
`bin/irssi` file of the `irssi` package.  How would we do that with an
s-expression?  Well, the s-expression itself, which we call the _builder_, is
fairly simple:

```scheme
(define sexp-builder
  `(let* ((out (assoc-ref %outputs "out"))
          (irssi (assoc-ref %build-inputs "irssi"))
          (bin/irssi (string-append irssi "/bin/irssi")))
     (symlink bin/irssi out)))
```

If you aren't familliar with the "quoting" syntax used to create s-expressions,
I strongly recommend that you read the excellent Scheme Primer; specifically,
section 7, [_Lists and
"cons"_](https://spritely.institute/static/papers/scheme-primer.html#scheme-lists-and-cons)
and section 11, [_On the extensibility of Scheme (and Lisps in
general)_](https://spritely.institute/static/papers/scheme-primer.html#scheme-extensibility)

The `%outputs` and `%build-inputs` variables are bound within builder scripts to
_association lists_, which are lists of pairs that act like key/value stores,
for instance:

```scheme
'(("foo" . "bar")
  ("floob" . "blarb")
  ("fvoolag" . "bvarlag"))
```

To retrieve values from association lists, which are often referred to as
_alists_, we use the `assoc-ref` procedure:

```scheme
(assoc-ref '(("boing" . "bouncy")
             ("floing" . "flouncy"))
           "boing")
⇒ "bouncy"
```

`%outputs`, as the name might suggest, maps derivation output names to the paths
of their respective store items, the default output being `out`, and
`%build-inputs` maps inputs labels to their store items.

The builder is the easy part; we now need to turn it into a derivation and tell
it what `"irssi"` actually refers to.  For this, we use the
`build-expression->derivation` procedure from `(guix derivations)`:

```scheme
(use-modules (guix derivations)
             (guix packages)
             (guix store)
             (gnu packages guile)
             (gnu packages irc))

(with-store store
  (let ((guile-3.0-drv (package-derivation store guile-3.0))
        (irssi-drv (package-derivation store irssi)))
    (build-expression->derivation store "irssi-symlink" sexp-builder
      #:guile-for-build guile-3.0-drv
      #:inputs `(("irssi" ,irssi-drv)))))
⇒ #<derivation /gnu/store/…-irssi-symlink.drv => /gnu/store/…-irssi-symlink …>
```

There are several things to note here:

- The inputs _must_ all be derivations, so we need to first convert the packages
  using `package-derivation`.
- We need to explicitly set `#:guile-for-build`; there's no default value.
- The `build-expression->derivation` and `package-derivation` procedures are
  _not_ monadic, so we need to explicitly pass them the store connection.

The shortcomings of using s-expressions in this way are numerous: we have to
convert everything to a derivation before using it, and _inputs are not an
inherent aspect of the builder_.  G-expressions were designed to overcome these
issues.

# Premortem Examination

A gexp is fundumentally a record of type `<gexp>`, which is, naturally, defined
in `(guix gexp)`.  The two most important fields of this record type, out of a
total of five, are `proc` and `references`; the former is a procedure that
returns the equivalent sexp, the latter a list containing everything from the
"outside world" that's used by the gexp.

When we want to turn the gexp into something that we can actually run as code,
we combine these two fields by first building any gexp inputs that can become
derivations (leaving alone those that cannot, such as and then passing the built
`references` as the arguments of `proc`.

Here's an example gexp that is essentially equivalent to our `sexp-builder`:

```scheme
(use-modules (guix gexp))

(define gexp-builder
  #~(symlink #$(file-append irssi "/bin/irssi")
             #$output))
```

`gexp-builder` is far more concise than `sexp-builder`; let's examine the syntax
and the `<gexp>` object we've created.  To make a gexp, we use the `#~` syntax,
equivalent to the `gexp` macro, rather than the `quasiquote` backtick used to
create sexps.

When we want to embed values from outside as references, we use `#$`, or
`ungexp`, which is, in appearance if not function, equivalent to `unquote`
(`,`).  `ungexp` can accept any of four reference types:

- Sexps (strings, lists, etc), which will be embedded literally.
- Other gexps, embedded literally.
- Expressions returning any sort of object that can be lowered into a
  derivation, such as `<package>`, embedding that object's `out` store item; if
  the expression is specifically a symbol bound to a buildable object, you can
  optionally follow it with a colon and an alternative output name, so
  `package:lib` is permitted, but `(get-package):lib` isn't.
- The symbol `output`, embedding an output path.  Like symbols bound to
  buildable objects, this can be followed by a colon and the output name that
  should be used rather than the default `out`.

All these reference types will be represented by `<gexp-input>` records in the
`references` field, except for the last kind, which will become `<gexp-output>`
records.  To give an example of each type of reference (with the return value
output formatted for easier reading):

```scheme
(use-modules (gnu packages glib))

#~(list #$"foobar"                         ;s-expression
        #$#~(string-append "foo" "bar")    ;g-expression
        #$(file-append irssi "/bin/irssi") ;buildable object (expression)
        #$glib:bin                         ;buildable object (symbol)
        #$output:out)                      ;output
⇒ #<gexp (list #<gexp-input "foobar":out>
               #<gexp-input #<gexp (string-append "foo" "bar") …>:out>
               #<gexp-input #<file-append #<package irssi@1.4.3 …> "/bin/irssi">:out>
               #<gexp-input #<package glib@2.70.2 …>:bin>
               #<gexp-output out>) …>
```

Note the use of `file-append` in both the previous example and `gexp-builder`;
this procedure produces a `<file-append>` object that builds its first argument
and is embedded as the concatenation of the first argument's output path and the
second argument, which should be a string.  For instance,
`(file-append irssi "/bin/irssi")` builds `irssi` and expands to
`/gnu/store/…-irssi/bin/irssi`, rather than the `/gnu/store/…-irssi` that the
package alone would be embedded as.

So, now that we have a gexp, how do we turn it into a derivation?  This process
is known as _lowering_; it entails the use of the aptly-named `lower-gexp`
monadic procedure to combine `proc` and `references` and produce a
`<lowered-gexp>` record, which acts as a sort of intermediate representation
between gexps and derivations.  We can piece apart this lowered form to get a
sense of what the final derivation's builder script would look like:

```scheme
(define lowered-gexp-builder
  (with-store store
    (run-with-store store
      (lower-gexp gexp-builder))))

(lowered-gexp-sexp lowered-gexp-builder)
⇒ (symlink
   "/gnu/store/…-irssi-1.4.3/bin/irssi"
   ((@ (guile) getenv) "out"))
```

And there you have it: a s-expression compiled from a g-expression, ready to be
written into a builder script file in the store.  So, how exactly do you turn
this into said derivation?

Well, it turns out that there isn't an interface for turning lowered gexps into
derivations, only one for turning regular gexps into derivations that first uses
`lower-gexp`, then implements the aforementioned conversion internally, rather
than outsourcing it to some other procedure, so that's what we'll use.

Unsurprisingly, that procedure is called `gexp->derivation`, and unlike its sexp
equivalent, it's monadic.  (`build-expression->derivation` and other deprecated
procedures were in Guix since before the monads system existed.)

```scheme
(with-store store
  (run-with-store store
    (gexp->derivation "irssi-symlink" gexp-builder)))
⇒ #<derivation /gnu/store/…-irssi-symlink.drv => /gnu/store/…-irssi-symlink …>
```

Finally, we have a gexp-based equivalent to the derivation we earlier created
with `build-expression->derivation`!  Here's the code we used for the sexp
version in full:

```scheme
(define sexp-builder
  `(let* ((out (assoc-ref %outputs "out"))
          (irssi (assoc-ref %build-inputs "irssi"))
          (bin/irssi (string-append irssi "/bin/irssi")))
     (symlink bin/irssi out)))

(with-store store
  (let ((guile-3.0-drv (package-derivation store guile-3.0))
        (irssi-drv (package-derivation store irssi)))
    (build-expression->derivation store "irssi-symlink" sexp-builder
      #:guile-for-build guile-3.0-drv
      #:inputs `(("irssi" ,irssi-drv)))))
```

And here's the gexp equivalent:

```scheme
(define gexp-builder
  #~(symlink #$(file-append irssi "/bin/irssi")
             #$output))

(with-store store
  (run-with-store store
    (gexp->derivation "irssi-symlink" gexp-builder)))
```

That's a lot of complexity abstracted away!  For more complex packages and
services, especially, gexps are a lifesaver; you can refer to the output paths
of inputs just as easily as you would a string constant.  You do, however, have
to watch out for situations where `ungexp-native`, written as `#+`, would be
preferable over regular `ungexp`, and that's something we'll discuss later.

A brief digression before we continue: if you'd like to look inside a `<gexp>`
record, but you'd rather not build anything, you can use the
`gexp->approximate-sexp` procedure, which replaces all references with dummy
values:

```scheme
(gexp->approximate-sexp gexp-builder)
⇒ (symlink (*approximate*) (*approximate*))
```

# The Lowerable-Object Hardware Shop

We've seen two examples already of records we can turn into derivations, which
are generally referred to as _lowerable objects_ or _file-like objects_:

- `<package>`, a Guix package.
- `<file-append>`, which wraps another lowerable object and appends a string to
  the embedded output path when ungexped.

There are many more available to us.  Recall from the previous post,
[_The Store Monad_](https://guix.gnu.org/en/blog/2023/dissecting-guix-part-2-the-store-monad),
that Guix provides the two monadic procedures `text-file` and `interned-file`,
which can be used, respectively, to put arbitrary text or files from the
filesystem in the store, returning the path to the created item.

This doesn't work so well with gexps, though; you'd have to wrap each ungexped
use of either of them with `(with-store store (run-with-store store …))`, which
would be quite tedious.  Thankfully, `(guix gexp)` provides the `plain-file` and
`local-file` procedures, which return equivalent lowerable objects.  This code
example builds a directory containing symlinks to files greeting the world:

```scheme
(use-modules (guix monads)
             (ice-9 ftw)
             (ice-9 textual-ports))

(define (build-derivation monadic-drv)
  (with-store store
    (run-with-store store
      (mlet* %store-monad ((drv monadic-drv))
        (mbegin %store-monad
          ;; BUILT-DERIVATIONS is the monadic version of BUILD-DERIVATIONS.
          (built-derivations (list drv))
          (return (derivation-output-path
                   (assoc-ref (derivation-outputs drv) "out"))))))))
                   
(define world-greeting-output
  (build-derivation
   (gexp->derivation "world-greeting"
     #~(begin
         (mkdir #$output)
         (symlink #$(plain-file "hi-world"
                      "Hi, world!")
                  (string-append #$output "/hi"))
         (symlink #$(plain-file "hello-world"
                      "Hello, world!")
                  (string-append #$output "/hello"))
         (symlink #$(plain-file "greetings-world"
                      "Greetings, world!")
                  (string-append #$output "/greetings"))))))

;; We turn the list into multiple values using (APPLY VALUES …).
(apply values
       (map (lambda (file-path)
              (let* ((path (string-append world-greeting-output "/" file-path))
                     (contents (call-with-input-file path get-string-all)))
                (list path contents)))
            ;; SCANDIR from (ICE-9 FTW) returns the list of all files in a
            ;; directory (including ``.'' and ``..'', so we remove them with the
            ;; second argument, SELECT?, which specifies a predicate).
            (scandir world-greeting-output
                     (lambda (path)
                       (not (or (string=? path ".")
                                (string=? path "..")))))))
⇒ ("/gnu/store/…-world-greeting/greetings" "Greetings, world!")
⇒ ("/gnu/store/…-world-greeting/hello" "Hello, world!")
⇒ ("/gnu/store/…-world-greeting/hi" "Hi, world!")
```

Note that we define a procedure for building the output; we will need to build
more derivations in a very similar fashion later, so it helps to have this to
reuse instead of copying the code in `world-greeting-output`.

There are many other useful lowerable objects available as part of the gexp
library.  These include `computed-file`, which accepts a gexp that builds
the output file, `program-file`, which creates an executable Scheme script in
the store using a gexp, and `mixed-text-file`, which allows you to, well, mix
text and lowerable objects; it creates a file from the concatenation of a
sequence of strings and file-likes.  The
[G-Expressions](https://guix.gnu.org/manual/en/html_node/G_002dExpressions.html)
manual page has more details.

So, you may be wondering, at this point: there's so many lowerable objects
included with the gexps library, surely there must be a way to define more?
Naturally, there is; this is Scheme, after all!  We simply need to acquaint
ourselves with the `define-gexp-compiler` macro.

The most basic usage of `define-gexp-compiler` essentially creates a procedure
that takes as arguments a record to lower, the host system, and the target
system, and returns a derivation or store item as a monadic value in
`%store-monad`.

Let's try implementing a lowerable object representing a file that greets the
world.  First, we'll define the record type:

```scheme
(use-modules (srfi srfi-9))

(define-record-type <greeting-file>
  (greeting-file greeting)
  greeting?
  (greeting greeting-file-greeting))
```

Now we use `define-gexp-compiler` like so; note how we can use `lower-object`
to compile down any sort of lowerable object into the equivalent store item or
derivation; essentially, `lower-object` is just the procedure for applying the
right gexp compiler to an object:

```scheme
(use-modules (ice-9 i18n))

(define-gexp-compiler (greeting-file-compiler
                       (greeting-file <greeting-file>)
                       system target)
  (lower-object
   (let ((greeting (greeting-file-greeting greeting-file)))
     (plain-file (string-append greeting "-greeting")
       (string-append (string-locale-titlecase greeting) ", world!")))))
```

Let's try it out now.  Here's how we could rewrite our greetings directory
example from before using `<greeting-file>`:

```scheme
(define world-greeting-2-output
  (build-derivation
   (gexp->derivation "world-greeting-2"
     #~(begin
         (mkdir #$output)
         (symlink #$(greeting-file "hi")
                  (string-append #$output "/hi"))
         (symlink #$(greeting-file "hello")
                  (string-append #$output "/hello"))
         (symlink #$(greeting-file "greetings")
                  (string-append #$output "/greetings"))))))

(apply values
       (map (lambda (file-path)
              (let* ((path (string-append world-greeting-2-output
                                          "/" file-path))
                     (contents (call-with-input-file path get-string-all)))
                (list path contents)))
            (scandir world-greeting-2-output
                     (lambda (path)
                       (not (or (string=? path ".")
                                (string=? path "..")))))))
⇒ ("/gnu/store/…-world-greeting-2/greetings" "Greetings, world!")
⇒ ("/gnu/store/…-world-greeting-2/hello" "Hello, world!")
⇒ ("/gnu/store/…-world-greeting-2/hi" "Hi, world!")
```

Now, this is probably not worth a whole new gexp compiler.  How about something
a bit more complex?  Sharp-eyed readers who are trying all this in the REPL may
have noticed the following output when they used `define-gexp-compiler`
(formatted for ease of reading):

```scheme
⇒ #<<gexp-compiler>
    type: #<record-type <greeting-file>>
    lower: #<procedure … (greeting-file system target)>
    expand: #<procedure default-expander (thing obj output)>>
```

Now, the purpose of `type` and `lower` is self-explanatory, but what's this
`expand` procedure here?  Well, if you recall `file-append`, you may realise
that the text produced by a gexp compiler for embedding into a gexp doesn't
necessarily have to be the exact output path of the produced derivation.

There turns out to be another way to write a `define-gexp-compiler` form that
allows you to specify _both_ the lowering procedure, which produces the
derivation or store item, and the expanding procedure, which produces the text.

Let's make another record; this one will let us build a store item containing a
`bin` directory with multiple scripts inside, and expand to the full path to
that script.

```scheme
(define-record-type <script-directory>
  (script-directory scripts)
  script-directory?
  (scripts script-directory-scripts))
```

Here's how we define both a compiler and expander for our new record:

```scheme
(define-gexp-compiler script-directory-compiler <script-directory>
  compiler => (lambda (obj system target)
                (gexp->derivation "script-directory"
                  #~(let ((bindir (string-append #$output "/bin")))
                      (mkdir #$output)
                      (mkdir bindir)
                      (for-each
                       (lambda (pair)
                         (let* ((name (car pair))
                                (path (cdr pair))
                                (bin-path (string-append bindir "/" name)))
                           (symlink path bin-path)))
                       #$(script-directory-scripts obj)))))
  expander => (lambda (obj drv output)
                (string-append output "/bin/" (script-directory-name obj))))
```

Let's try this out now:

```scheme
(use-modules (gnu packages vim))

(define script-directory-output
  (build-derivation
   (lower-object
    (script-directory
     #~'(("irc" . #$(file-append irssi "/bin/irssi"))
         ("editor" . #$(file-append neovim "/bin/nvim")))))))

(scandir (string-append script-directory-output "/bin"))
⇒ ("." ".." "editor" "irc")
```

Who knows why you'd want to do this, but it certainly works!  We've looked at
why we need gexps, how they work, and how to extend them, and we've now only got
two more advanced features to cover: cross-build support, and modules.

# Importing External Modules

Let's try using one of the helpful procedures from the `(guix build utils)`
module in a gexp.

```scheme
(define silly-directory-output
  (build-derivation
   (gexp->derivation "silly-directory"
     #~(begin
         (use-modules (guix build utils))
         (mkdir-p (string-append #$output "/what/a/silly/directory"))))))
```

Looks fine, right?  We've even got a `use-modules` in th--

```Scheme
ERROR:
  1. &store-protocol-error:
      message: "build of `/gnu/store/…-silly-directory.drv' failed"
      status: 100
```

OUTRAGEOUS.  Fortunately, there's an explanation to be found in the Guix build
log directory, `/var/log/guix/drvs`; locate the file using the first two
characters of the store hash as the subdirectory, and the rest as the file name,
and remember to use `zcat` or `zless`, as the logs are gzipped:

```scheme
Backtrace:
           9 (primitive-load "/gnu/store/…")
In ice-9/eval.scm:
   721:20  8 (primitive-eval (begin (use-modules (guix build #)) (?)))
In ice-9/psyntax.scm:
  1230:36  7 (expand-top-sequence ((begin (use-modules (guix ?)) #)) ?)
  1090:25  6 (parse _ (("placeholder" placeholder)) ((top) #(# # ?)) ?)
  1222:19  5 (parse _ (("placeholder" placeholder)) ((top) #(# # ?)) ?)
   259:10  4 (parse _ (("placeholder" placeholder)) (()) _ c&e (eval) ?)
In ice-9/boot-9.scm:
  3927:20  3 (process-use-modules _)
   222:17  2 (map1 (((guix build utils))))
  3928:31  1 (_ ((guix build utils)))
   3329:6  0 (resolve-interface (guix build utils) #:select _ #:hide ?)

ice-9/boot-9.scm:3329:6: In procedure resolve-interface:
no code for module (guix build utils)
```

It turns out `use-modules` can't actually find `(guix build utils)` at all.
There's no typo; it's just that to ensure the build is isolated, Guix builds
`module-import` and `module-importe-compiled` directories, and sets the
_Guile module path_ within the build environment to contain said directories,
along with those containing the Guile standard library modules.

So, what to do?  Turns out one of the fields in `<gexp>` is `modules`, which,
funnily enough, contains the names of the modules which will be used to build
the aforementioned directories.  To add to this field, we use the
`with-imported-modules` macro.  (`gexp->derivation` _does_ provide a `modules`
parameter, but `with-imported-modules` lets you add the required modules
directly to the gexp value, rather than later on.)

```scheme
(define silly-directory-output
  (build-derivation
   (gexp->derivation "silly-directory"
     (with-imported-modules '((guix build utils))
       #~(begin
           (use-modules (guix build utils))
           (mkdir-p (string-append #$output "/what/a/silly/directory")))))))
           
silly-directory-output
⇒ "/gnu/store/…-silly-directory"
```

It works, yay.  It's worth noting that while passing just the list of modules to
`with-imported-modules` works in this case, this is only because
`(guix build utils)` has no dependencies on other Guix modules.  Were we to try
adding, say, `(guix build emacs-build-system)`, we'd need to use the
`source-module-closure` procedure to add its dependencies to the list:

```scheme
(source-module-closure '((guix build emacs-build-system)))
⇒ ((guix build emacs-build-system)
   (guix build gnu-build-system)
   (guix build utils)
   (guix build gremlin)
   (guix elf)
   (guix build emacs-utils))
```

Here's another scenario: what if we want to use a module not from Guix or Guile
but a third-party library?  In this example, we'll use [guile-json
](https://github.com/aconchillo/guile-json), a library for converting between
S-expressions and [JavaScript Object Notation](https://json.org).

We can't just `with-imported-modules` its modules, since it's not part of Guix,
so `<gexp>` provides another field for this purpose: `extensions`.  Each of
these extensions is a lowerable object that produces a Guile package directory;
so usually a package.  Let's try it out.

```scheme
(use-modules (gnu packages guile))

(define helpful-guide-output
  (build-derivation
   (gexp->derivation "json-file"
     (with-extensions (list guile-json-4)
       #~(begin
           (use-modules (json))
           (mkdir #$output)
           (call-with-output-file (string-append #$output "/helpful-guide.json")
             (lambda (port)
               (scm->json '((truth . "Guix is the best!")
                            (lies . "Guix isn't the best!"))
                          port))))))))

(call-with-input-file
    (string-append helpful-guide-output "/helpful-guide.json")
  get-string-all)
⇒ "{\"truth\":\"Guix is the best!\",\"lies\":\"Guix isn't the best!\"}"
```

Amen to that, `helpful-guide.json`.  Before we continue on to cross-compilation,
there's one last feature of `with-imported-modules` you should note.  We can
add modules to a gexp by name, but we can also create entirely new ones with
lowerable objects, like this pattern, which is used in several places in the
Guix source code:

```scheme
(with-imported-modules `(((guix config) => ,(make-config.scm))
                         …)
  …)
```

In case you're wondering, `make-config.scm` is found in `(guix self)` and
returns a lowerable object that compiles to a version of the `(guix config)`
module, which contains constants usually substituted into the source code at
compile time.

# Native Ungexp

There is another piece of syntax we can use with gexps, and it's called
`ungexp-native`.  This helps us distinguish between native inputs and regular
host-built inputs in cross-compilation situations.  We'll cover
cross-compilation in detail at a later date, but the gist of it is that it
allows you to compile a derivation for one architecture X, the target, using a
machine of architecture Y, the host, and Guix has excellent support for it.

If we cross-compile a gexp G that _non-natively_ ungexps L1, a lowerable object,
from architecture Y to architecture X, both G and L1 will be compiled for
architecture X.  However, if G _natively_ ungexps L1, G will be compiled for X
and L1 for Y.

Essentially, we use `ungexp-native` in situations where there would be no
difference between compiling on different architectures (for instance, if `L1`
were a `plain-file`), or where using L1 built for X would actually _break_ G
(for instance, if `L1` corresponds to a compiled executable that needs to be run
during the build; the executable would fail to run on Y if it was built for X.)

The `ungexp-native` macro naturally has a corresponding reader syntax, `#+`, and
there's also `ungexp-native-splicing`, which is written as `#+@`.  These two
pieces of syntax are used in the same way as their regular counterparts.

# Conclusion

Mastering gexps is essential to understanding Guix's inner workings, so the aim
of this blog post is to be as thorough as possible.  However, if you still find
yourself with questions, please don't hesitate to stop by at the IRC channel
`#guix:libera.chat` and mailing list `help-guix@gnu.org`; we'll be glad to
assist you!

#### About GNU Guix

[GNU Guix](https://guix.gnu.org) is a transactional package manager and
an advanced distribution of the GNU system that [respects user
freedom](https://www.gnu.org/distros/free-system-distribution-guidelines.html).
Guix can be used on top of any system running the Hurd or the Linux
kernel, or it can be used as a standalone operating system distribution
for i686, x86_64, ARMv7, AArch64 and POWER9 machines.

In addition to standard package management features, Guix supports
transactional upgrades and roll-backs, unprivileged package management,
per-user profiles, and garbage collection.  When used as a standalone
GNU/Linux distribution, Guix offers a declarative, stateless approach to
operating system configuration management.  Guix is highly customizable
and hackable through [Guile](https://www.gnu.org/software/guile)
programming interfaces and extensions to the
[Scheme](http://schemers.org) language.

debug log:

solving 32f5d51 ...
found 32f5d51 in https://yhetil.org/guix-patches/20230321205749.4974-1-paren@disroot.org/

applying [1/1] https://yhetil.org/guix-patches/20230321205749.4974-1-paren@disroot.org/
diff --git a/website/posts/dissecting-guix-3-gexps.md b/website/posts/dissecting-guix-3-gexps.md
new file mode 100644
index 0000000..32f5d51

1:303: trailing whitespace.
                   
1:561: trailing whitespace.
           
Checking patch website/posts/dissecting-guix-3-gexps.md...
Applied patch website/posts/dissecting-guix-3-gexps.md cleanly.
warning: 2 lines add whitespace errors.

index at:
100644 32f5d51c1507de7a14f4eda10734c0a8eb3d281d	website/posts/dissecting-guix-3-gexps.md

(*) Git path names are given by the tree(s) the blob belongs to.
    Blobs themselves have no identifier aside from the hash of its contents.^

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).