Help with G-expression

Help with G-expression

[zimoun]

Hi,

Trying to make a Guix script for extracting BigBlueButton videos – the
ones from Café Guix for instance – I am very puzzled by G-expression.

Well, I am trying to adapt this one [1].  But I am failing…

The manual has this example [1],

--8<---------------cut here---------------start------------->8---
(with-imported-modules '((guix build utils))  ;import it
  (computed-file "empty-tree"
                 #~(begin
                     ;; Put it in scope.
                     (use-modules (guix build utils))

                     ;; Happily use its 'mkdir-p' procedure.
                     (mkdir-p (string-append #$output "/a/b/c")))))
--8<---------------cut here---------------end--------------->8---

Which I turn into a function, and from “guix repl”

--8<---------------cut here---------------start------------->8---
scheme@(guix-user)> (define (work)
  (with-imported-modules '((guix build utils))  ;import it
    (computed-file "empty-tree"
                   #~(begin
                       ;; Put it in scope.
                       (use-modules (guix build utils))

                       ;; Happily use its 'mkdir-p' procedure.
                       (mkdir-p (string-append #$output "/a/b/c"))))))

scheme@(guix-user)> ,build (work)
$1 = "/gnu/store/d8psk7dsczqj65sc8bqigpi6ffhd0wvn-empty-tree"
--8<---------------cut here---------------end--------------->8---

where “tree /gnu/store/d8psk7dsczqj65sc8bqigpi6ffhd0wvn-empty-tree” shows the tree a/b/c.  So far, so
good.

If instead, I just add the module (guix profiles), then it fails.

--8<---------------cut here---------------start------------->8---
scheme@(guix-user)> (define (fail)
  (with-imported-modules '((guix build utils)
                           (guix profiles))  ;import it
    (computed-file "empty-tree"
                   #~(begin
                       ;; Put it in scope.
                       (use-modules (guix build utils))

                       ;; Happily use its 'mkdir-p' procedure.
                       (mkdir-p (string-append #$output "/a/b/c"))))))
scheme@(guix-user)> ,build (fail)
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
building /gnu/store/g1jbxhhz4b7zcyixhy4vw13brn2fncia-module-import-compiled.drv...
 50% [####################################                                     ]builder for `/gnu/store/g1jbxhhz4b7zcyixhy4vw13brn2fncia-module-import-compiled.drv' failed with exit code 1
build of /gnu/store/g1jbxhhz4b7zcyixhy4vw13brn2fncia-module-import-compiled.drv failed
View build log at '/var/log/guix/drvs/g1/jbxhhz4b7zcyixhy4vw13brn2fncia-module-import-compiled.drv.gz'.
cannot build derivation `/gnu/store/ln5yb7lvdknr3azc5zh2m3239adgmsjj-empty-tree.drv': 1 dependencies couldn't be built
While executing meta-command:
ERROR:
  1. &store-protocol-error:
      message: "build of `/gnu/store/ln5yb7lvdknr3azc5zh2m3239adgmsjj-empty-tree.drv' failed"
      status: 100
--8<---------------cut here---------------end--------------->8---

Well, the message is not really helping. :-)

Then, because I am not sure to fully understand the difference, let add
’source-module-closure’.

--8<---------------cut here---------------start------------->8---
scheme@(guix-user)> ,use(guix modules)
scheme@(guix-user)> (define (fail-again)
  (with-imported-modules (source-module-closure
                          '((guix build utils)
                            (guix profiles)))  ;import it
    (computed-file "empty-tree"
                   #~(begin
                       ;; Put it in scope.
                       (use-modules (guix build utils))

                       ;; Happily use its 'mkdir-p' procedure.
                       (mkdir-p (string-append #$output "/a/b/c"))))))
scheme@(guix-user)> ,build (fail-again)
While executing meta-command:
Throw to key `match-error' with args `("match" "no matching pattern" (#:declarative? #f #:export (%guix-package-name %guix-version %guix-bug-report-address %guix-home-page-url %channel-metadata %system %store-directory %state-directory %store-database-directory %config-directory %gzip %bzip2 %xz)))'.
--8<---------------cut here---------------end--------------->8---

Ah, another message.  Not helping either. :-)


What do I miss or am doing wrong?


1: <https://gitlab.inria.fr/guix-hpc/website/-/blob/6977da4618814c790e767618da5cf9ec2cab0742/doc/atelier-reproductibilit%C3%A9/render-videos.scm>
2: https://guix.gnu.org/manual/devel/en/guix.html#Build-Utilities

Cheers,
simon

Re: Help with G-expression

[(]

[-- Attachment #1: Type: text/plain, Size: 196 bytes --]

On Thu Dec 1, 2022 at 5:34 PM GMT, zimoun wrote:
> View build log at '/var/log/guix/drvs/g1/jbxhhz4b7zcyixhy4vw13brn2fncia-module-import-compiled.drv.gz'.

What's in this file? :)

    -- (

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: Help with G-expression

[zimoun]

Hi,

On Thu, 01 Dec 2022 at 18:21, "(" <paren@disroot.org> wrote:
> On Thu Dec 1, 2022 at 5:34 PM GMT, zimoun wrote:
>> View build log at '/var/log/guix/drvs/g1/jbxhhz4b7zcyixhy4vw13brn2fncia-module-import-compiled.drv.gz'.
>
> What's in this file? :)

--8<---------------cut here---------------start------------->8---
Backtrace:
          16 (primitive-load "/gnu/store/hld85lyaq7k2h4qzbmlgiflypcx?")
In ice-9/eval.scm:
    619:8 15 (_ #f)
In srfi/srfi-1.scm:
   460:18 14 (fold #<procedure 7ffff4c04900 at ice-9/eval.scm:336:1?> ?)
   460:18 13 (fold #<procedure 7ffff4c04540 at ice-9/eval.scm:336:1?> ?)
In ice-9/eval.scm:
    619:8 12 (_ #(#(#<directory (guix build utils) 7ffff5ef7320>) # ?))
In ice-9/boot-9.scm:
   2835:4 11 (save-module-excursion #<procedure 7ffff4aa2f00 at ice-?>)
In unknown file:
          10 (primitive-load "./guix/profiles.scm")
In ice-9/eval.scm:
   721:20  9 (primitive-eval (define-module (guix profiles) # (# ?) ?))
In ice-9/psyntax.scm:
  1230:36  8 (expand-top-sequence ((define-module (guix #) # (?) ?)) ?)
  1222:19  7 (parse _ (("placeholder" placeholder)) ((top) #(# # ?)) ?)
   259:10  6 (parse _ (("placeholder" placeholder)) (()) _ c&e (eval) ?)
In ice-9/eval.scm:
   293:34  5 (_ #<directory (guix build utils) 7ffff5ef7320>)
In ice-9/boot-9.scm:
   3409:4  4 (define-module* _ #:filename _ #:pure _ #:version _ # _ ?)
  2594:24  3 (call-with-deferred-observers #<procedure 7ffff4bc95f0 ?>)
  3422:24  2 (_)
   222:17  1 (map1 (((guix config) #:select (%state-directory)) (?) ?))
   3329:6  0 (resolve-interface (guix config) #:select _ #:hide _ # _ ?)

ice-9/boot-9.scm:3329:6: In procedure resolve-interface:
no code for module (guix config)
--8<---------------cut here---------------end--------------->8---

Then,

    (guix utils)
    (guix memoization)
    (guix profiling)
    (guix build syscalls)

and I stopped try-error the game before completing. :-)


Cheers,
simon

Re: Help with G-expression

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> no code for module (guix config)

(guix config) is added by using ‘make-config.scm’; there’s an example in
(gnu services base).

For the rest, make sure to use ‘source-module-closure’ or you’ll miss
modules (info "(guix) G-Expressions").

HTH!

Ludo’.

Re: Help with G-expression

[zimoun]

Hi Ludo,

On Fri, 02 Dec 2022 at 10:09, Ludovic Courtès <ludo@gnu.org> wrote:
> zimoun <zimon.toutoune@gmail.com> skribis:
>
>> no code for module (guix config)
>
> (guix config) is added by using ‘make-config.scm’; there’s an example in
> (gnu services base).
>
> For the rest, make sure to use ‘source-module-closure’ or you’ll miss
> modules (info "(guix) G-Expressions").

I have read this section. :-) Something is not clear for me.

Note that ’make-config.scm’ does not appear in the manual.

The only difference between the previous ’work’ example and all the
’fail’ as ’fail-again-not-fixed’ is only the part ’with-imported-modules’.

Well, I am somehow surprised how difficult is to find the correct.

--8<---------------cut here---------------start------------->8---
scheme@(guix-user)> (define (fail-again-not-fixed)
  (with-imported-modules (source-module-closure
                          '((guix build utils)
                            (guix config)
                            (guix profiles)))  ;import it
    (computed-file "empty-tree"
                   #~(begin
                       ;; Put it in scope.
                       (use-modules (guix build utils))

                       ;; Happily use its 'mkdir-p' procedure.
                       (mkdir-p (string-append #$output "/a/b/c"))))))
scheme@(guix-user)> ,build (fail-again-not-fixed)
While executing meta-command:
Throw to key `match-error' with args `("match" "no matching pattern" (#:declarative? #f #:export (%guix-package-name %guix-version %guix-bug-report-address %guix-home-page-url %channel-metadata %system %store-directory %state-directory %store-database-directory %config-directory %gzip %bzip2 %xz)))'.
--8<---------------cut here---------------end--------------->8---

Here it is the same error as ’fail-again’.  And I do not understand the
magic invocation.  Well, what I tried next:

--8<---------------cut here---------------start------------->8---
scheme@(guix-user)> (define (fail-again-bis)
  (with-imported-modules `(((guix config) => ,(make-config.scm))
                           ,@(source-module-closure '((guix build utils)
                                                      (guix profiles))
                                                    #:select? not-config?))
    (computed-file "empty-tree"
                   #~(begin
                       ;; Put it in scope.
                       (use-modules (guix build utils))

                       ;; Happily use its 'mkdir-p' procedure.
                       (mkdir-p (string-append #$output "/a/b/c"))))))
;;; <unknown-location>: warning: possibly unbound variable `make-config.scm'
;;; <unknown-location>: warning: possibly unbound variable `not-config?'
scheme@(guix-user)> ,build (fail-again-bis)
While executing meta-command:
error: make-config.scm: unbound variable
scheme@(guix-user)> ,use(guix self)
scheme@(guix-user)> ,build (fail-again-bis)
While executing meta-command:
error: not-config?: unbound variable
scheme@(guix-user)> (define not-config?
  ;; Select (guix …) and (gnu …) modules, except (guix config).
  (match-lambda
    (('guix 'config) #f)
    (('guix rest ...) #t)
    (('gnu rest ...) #t)
    (rest #f)))
scheme@(guix-user)> ,build (fail-again-bis)
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
substitute: updating substitutes from 'https://bordeaux.guix.gnu.org'... 100.0%
building /gnu/store/xjvjrkl3559jrlm0bnvdj29c838y81x3-config.scm.drv...
building /gnu/store/slwgyy4vnxprz0cj73vyq6psr7bqmbwz-module-import.drv...
building /gnu/store/cfjpfzph97dsishvr7g1bqxqw5kynns8-module-import-compiled.drv...
 20% [#############################                                                                                                                         ]builder for `/gnu/store/cfjpfzph97dsishvr7g1bqxqw5kynns8-module-import-compiled.drv' failed with exit code 1
build of /gnu/store/cfjpfzph97dsishvr7g1bqxqw5kynns8-module-import-compiled.drv failed
View build log at '/var/log/guix/drvs/cf/jpfzph97dsishvr7g1bqxqw5kynns8-module-import-compiled.drv.gz'.
cannot build derivation `/gnu/store/pr3qqw26jilyhyn650i3w7z13i7k6p40-empty-tree.drv': 1 dependencies couldn't be built
While executing meta-command:
ERROR:
  1. &store-protocol-error:
      message: "build of `/gnu/store/pr3qqw26jilyhyn650i3w7z13i7k6p40-empty-tree.drv' failed"
      status: 100
--8<---------------cut here---------------end--------------->8---
      
Below the log file – missing (gcrypt hash).

Well, I give up because the try-error does not explain me how it
works. :-)

Any pointers?

Cheers,
simon

--8<---------------cut here---------------start------------->8---
[ 1/56] Loading './guix/base16.scm'...
[ 2/56] Loading './guix/base32.scm'...
[ 3/56] Loading './guix/base64.scm'...
[ 4/56] Loading './guix/build/syscalls.scm'...
[ 5/56] Loading './guix/build/utils.scm'...
[ 6/56] Loading './guix/build-system.scm'...
[ 7/56] Loading './guix/colors.scm'...
[ 8/56] Loading './guix/combinators.scm'...
[ 9/56] Loading './guix/config.scm'...
[10/56] Loading './guix/deprecation.scm'...
[11/56] Loading './guix/derivations.scm'...
;;; Failed to autoload make-zlib-input-port in (zlib):
;;; no code for module (zlib)
;;; Failed to autoload make-zlib-input-port in (zlib):
;;; no code for module (zlib)
;;; Failed to autoload make-zlib-output-port in (zlib):
;;; no code for module (zlib)
;;; Failed to autoload make-zlib-output-port in (zlib):
;;; no code for module (zlib)
Backtrace:
In ice-9/boot-9.scm:
  3556:26 19 (_)
In unknown file:
          18 (primitive-load-path "guix/store" #<procedure 7fffed89b?>)
In ice-9/eval.scm:
   721:20 17 (primitive-eval _)
In ice-9/psyntax.scm:
  1230:36 16 (expand-top-sequence _ _ _ #f _ _ _)
  1222:19 15 (parse _ (("placeholder" placeholder)) ((top) #(# # ?)) ?)
   259:10 14 (parse _ (("placeholder" placeholder)) (()) _ c&e (eval) ?)
In ice-9/eval.scm:
   293:34 13 (_ #<module (#{ g170}#) 7fffee23baa0>)
In ice-9/boot-9.scm:
   3409:4 12 (define-module* _ #:filename _ #:pure _ #:version _ # _ ?)
  3422:24 11 (_)
   222:29 10 (map1 _)
   222:29  9 (map1 _)
   222:29  8 (map1 _)
   222:29  7 (map1 _)
   222:29  6 (map1 _)
   222:29  5 (map1 (((guix monads)) ((guix records)) ((guix #)) (#) ?))
   222:29  4 (map1 (((guix records)) ((guix base16)) ((guix #)) (#) ?))
   222:29  3 (map1 (((guix base16)) ((guix base32)) ((gcrypt #)) # ?))
   222:29  2 (map1 (((guix base32)) ((gcrypt hash)) ((guix #)) (#) ?))
   222:17  1 (map1 (((gcrypt hash)) ((guix profiling)) ((rnrs #)) # ?))
   3329:6  0 (resolve-interface (gcrypt hash) #:select _ #:hide _ # _ ?)

ice-9/boot-9.scm:3329:6: In procedure resolve-interface:
no code for module (gcrypt hash)
--8<---------------cut here---------------end--------------->8---

Re: Help with G-expression

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> scheme@(guix-user)> ,build (fail-again-not-fixed)
> While executing meta-command:
> Throw to key `match-error' with args `("match" "no matching pattern" (#:declarative? #f #:export (%guix-package-name %guix-version %guix-bug-report-address %guix-home-page-url %channel-metadata %system %store-directory %state-directory %store-database-directory %config-directory %gzip %bzip2 %xz)))'.

Oops, that one is a new bug due to
54003af85cc5b689bd328b30617c93ed2f5fd647.  I’ll look into it.

> Below the log file – missing (gcrypt hash).

So (with-extensions (list guile-gcrypt) …), as in ‘video-ges-project’ in
‘render-videos.scm’¹.

In essence, the gexp you want to run must specify all the Guile modules
and extensions it depends on; this is what ‘with-imported-modules’ and
‘with-extensions’ let you express.

HTH,
Ludo’.

¹ https://gitlab.inria.fr/guix-hpc/website/-/blob/6977da4618814c790e767618da5cf9ec2cab0742/doc/atelier-reproductibilit%C3%A9/render-videos.scm

Re: Help with G-expression

[zimoun]

Hi Ludo,

Thanks for your patience. :-)

On Fri, 02 Dec 2022 at 22:30, Ludovic Courtès <ludo@gnu.org> wrote:

> Oops, that one is a new bug due to
> 54003af85cc5b689bd328b30617c93ed2f5fd647.  I’ll look into it.

Ah, a good news. :-)

>> Below the log file – missing (gcrypt hash).
>
> So (with-extensions (list guile-gcrypt) …), as in ‘video-ges-project’ in
> ‘render-videos.scm’¹.
>
> In essence, the gexp you want to run must specify all the Guile modules
> and extensions it depends on; this is what ‘with-imported-modules’ and
> ‘with-extensions’ let you express.

Not here.  What I running is this:

--8<---------------cut here---------------start------------->8---
(with-imported-modules '((guix build utils))  ;import it
  (computed-file "empty-tree"
                 #~(begin
                     ;; Put it in scope.
                     (use-modules (guix build utils))

                     ;; Happily use its 'mkdir-p' procedure.
                     (mkdir-p (string-append #$output "/a/b/c")))))
--8<---------------cut here---------------end--------------->8---

And it perfectly works. :-)  As explained in the very first email in
this thread.

The issue is when I am adding (guix profiles) and nothing more to the
previous perfectly working ’computed-file’.  How can I guess that this
module (guix profiles) requires (gcrypt hash) as extension?  Just to be
loaded since it is not used.

Sorry I am missing the obvious.

Cheers,
simon

Re: Help with G-expression

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> On Fri, 02 Dec 2022 at 22:30, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> Oops, that one is a new bug due to
>> 54003af85cc5b689bd328b30617c93ed2f5fd647.  I’ll look into it.
>
> Ah, a good news. :-)

Fixed in 9672db290a35f59113682a7f0c17c050a4ae2578.

[…]

> How can I guess that this
> module (guix profiles) requires (gcrypt hash) as extension?  Just to be
> loaded since it is not used.

You cannot guess it.  Guile-Gcrypt here is an indirect dependency of
(guix profiles).  All you can do is notice that it is needed and add a
‘with-extensions’ form.

It’s annoying, but I don’t see how this could be avoided.

HTH!

Ludo’.

Re: Help with G-expression

[zimoun]

Hi Ludo,

Thanks for the help.


On Sat, 03 Dec 2022 at 18:43, Ludovic Courtès <ludo@gnu.org> wrote:

> Fixed in 9672db290a35f59113682a7f0c17c050a4ae2578.

Well, the main issue was that I hit this bug. :-)

I tried to see if this trivial example is covered by the test suite but
I have not found it.  Something like,

--8<---------------cut here---------------start------------->8---
(define (something)
  (with-extensions (list guile-gcrypt)
    (with-imported-modules (source-module-closure
                            '((guix build utils)
                              (guix profiles))) ;import it
      (computed-file "empty-tree"
                     #~(begin
                         ;; Put it in scope.
                         (use-modules (guix build utils))

                         ;; Happily use its 'mkdir-p' procedure.
                         (mkdir-p (string-append #$output "/a/b/c")))))))
,build (something)
--8<---------------cut here---------------end--------------->8---

where ’something’ would be a better name and ’,build’ the real function
call.  Well, maybe it could help to detect some regression. :-)  As
people tell: one bug closed, one test added. ;-)

Although it requires to talk with the daemon and I do not remember if it
is a good idea to add such test.


> It’s annoying, but I don’t see how this could be avoided.

Well, the error message is not super clear…

--8<---------------cut here---------------start------------->8---
building /gnu/store/rh1ii1cjlc0jhkz2aspxcw1mdkr7b2nf-module-import-compiled.drv...
 20% [#####################                                                                                         ]builder for `/gnu/store/rh1ii1cjlc0jhkz2aspxcw1mdkr7b2nf-module-import-compiled.drv' failed with exit code 1
build of /gnu/store/rh1ii1cjlc0jhkz2aspxcw1mdkr7b2nf-module-import-compiled.drv failed
View build log at '/var/log/guix/drvs/rh/1ii1cjlc0jhkz2aspxcw1mdkr7b2nf-module-import-compiled.drv.gz'.
cannot build derivation `/gnu/store/cznhm1wzdv60jwi7fz96cilq925j54i1-empty-tree.drv': 1 dependencies couldn't be built
While executing meta-command:
ERROR:
  1. &store-protocol-error:
      message: "build of `/gnu/store/cznhm1wzdv60jwi7fz96cilq925j54i1-empty-tree.drv' failed"
      status: 100
--8<---------------cut here---------------end--------------->8---

…but the log somehow is:

--8<---------------cut here---------------start------------->8---
[ 1/56] Loading './guix/base16.scm'...

[...]

[11/56] Loading './guix/derivations.scm'...
;;; Failed to autoload make-zlib-input-port in (zlib):
;;; no code for module (zlib)
;;; Failed to autoload make-zlib-input-port in (zlib):
;;; no code for module (zlib)
;;; Failed to autoload make-zlib-output-port in (zlib):
;;; no code for module (zlib)
;;; Failed to autoload make-zlib-output-port in (zlib):
;;; no code for module (zlib)
Backtrace:

[...]

   222:17  1 (map1 (((gcrypt hash)) ((guix profiling)) ((rnrs #)) # ?))
   3329:6  0 (resolve-interface (gcrypt hash) #:select _ #:hide _ # _ ?)

ice-9/boot-9.scm:3329:6: In procedure resolve-interface:
no code for module (gcrypt hash)
--8<---------------cut here---------------end--------------->8---

Well, it seems reasonable, IMHO, to assume that an user diving into such
internal would be able to parse this information.


Cheers,
simon