[bug#67489] [PATCH] monad-repl: Add REPL commands to inspect package arguments.

[bug#67489] [PATCH] monad-repl: Add REPL commands to inspect package arguments.

[Ludovic Courtès]

From: Ludovic Courtès <ludovic.courtes@inria.fr>

* guix/monad-repl.scm (keyword-argument-value, package-argument-command):
New procedures.
(phases, configure-flags, make-flags): New REPL commands.
* doc/guix.texi (package Reference): Link to “Using Guix Interactively”.
(Defining Package Variants): Add “Tips” quotation.
(Build Phases): Add “Tip” quotation.
(Using Guix Interactively): Document the new REPL commands.

Change-Id: I7049c1d8aa9241e07d7c921aa396e578a1b4ef16
---
 doc/guix.texi       | 61 +++++++++++++++++++++++++++++++++++++
 guix/monad-repl.scm | 74 ++++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 131 insertions(+), 4 deletions(-)

Hello!

Someone recently suggested we should have better debugging facilities
for packagers, in particular when dealing with package variants,
inheritance, and so on, which make it harder to understand what’s
going on.

This change partially addresses that problem: new REPL commands
let you inspect the code of #:phases, #:configure-flags, and #:make-flags.

--8<---------------cut here---------------start------------->8---
scheme@(guix-user)> ,use(guix)
scheme@(guix-user)> ,use(gnu packages mpi)
scheme@(guix-user)> ,use(gnu packages guile)
scheme@(guix-user)> ,phases guile-3.0
$1 = (modify-phases (modify-phases %standard-phases
                      (add-before 'configure 'pre-configure
                        (lambda* (#:key inputs #:allow-other-keys)
                          (let ((bash (assoc-ref inputs "bash")))
                            (substitute* "module/ice-9/popen.scm"
                              (("/bin/sh")
                               (string-append bash "/bin/bash"))) #t))))
       (add-before 'check 'disable-stack-overflow-test
         (lambda _
           (substitute* "test-suite/standalone/test-stack-overflow"
             (("!#")
              "!#\n(exit 77)\n")))))
scheme@(guix-user)> ,configure-flags openmpi
$2 = `("--enable-mpi-ext=affinity" "--with-sge"
       "--enable-memchecker"
       "--with-valgrind"
       "--with-hwloc=external"
       "--with-libevent"
       "--enable-mpirun-prefix-by-default"
       "--enable-openib-control-hdr-padding"
       "--enable-openib-dynamic-sl"
       "--enable-openib-udcm"
       "--enable-openib-rdmacm"
       "--enable-openib-rdmacm-ibaddr"
       "--with-pmix=internal"
       ,(string-append "--with-pmi="
         "/gnu/store/h3vmqdgfnfqyk84vc1p3z57dqhrccx7b-slurm-23.02.6"))
--8<---------------cut here---------------end--------------->8---

Thoughts?

Ludo’.

diff --git a/doc/guix.texi b/doc/guix.texi
index 1fd2e21608..7dde9b727b 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -7975,6 +7975,10 @@ package Reference
 @code{#:phases}.  The @code{#:phases} keyword in particular lets you
 modify the set of build phases for your package (@pxref{Build Phases}).
 
+The REPL has dedicated commands to interactively inspect values of some
+of these arguments, as a convenient debugging aid (@pxref{Using Guix
+Interactively}).
+
 @quotation Compatibility Note
 Until version 1.3.0, the @code{arguments} field would typically use
 @code{quote} (@code{'}) or @code{quasiquote} (@code{`}) and no
@@ -8774,6 +8778,23 @@ Defining Package Variants
 applied to implicit inputs as well.
 @end deffn
 
+@quotation Tips
+Understanding what a variant really looks like can be difficult as one
+starts combining the tools shown above.  There are several ways to
+inspect a package before attempting to build it that can prove handy:
+
+@itemize
+@item
+You can inspect the package interactively at the REPL, for instance to
+view its inputs, the code of its build phases, or its configure flags
+(@pxref{Using Guix Interactively}).
+
+@item
+When rewriting dependencies, @command{guix graph} can often help
+visualize the changes that are made (@pxref{Invoking guix graph}).
+@end itemize
+@end quotation
+
 @node Writing Manifests
 @section Writing Manifests
 
@@ -10585,6 +10606,11 @@ Build Phases
 the helpers used by this phase, and for more examples of
 @code{modify-phases}.
 
+@quotation Tip
+You can inspect the code associated with a package's @code{#:phases}
+argument interactively, at the REPL (@pxref{Using Guix Interactively}).
+@end quotation
+
 @cindex code staging
 @cindex staging, of code
 Keep in mind that build phases are code evaluated at the time the
@@ -12763,6 +12789,30 @@ Using Guix Interactively
 $5 = ("." ".." "egrep" "fgrep" "grep")
 @end example
 
+As a packager, you may be willing to inspect the build phases or flags
+of a given package; this is particularly useful when relying a lot on
+inheritance to define package variants (@pxref{Defining Package
+Variants}) or when package arguments are a result of some computation,
+both of which can make it harder to foresee what ends up in the package
+arguments.  Additional commands let you inspect those package arguments:
+
+@example
+scheme@@(guix-user)> ,phases grep
+$1 = (modify-phases %standard-phases
+       (add-after 'install 'fix-egrep-and-fgrep
+         (lambda* (#:key outputs #:allow-other-keys)
+           (let* ((out (assoc-ref outputs "out"))
+                  (bin (string-append out "/bin")))
+             (substitute* (list (string-append bin "/egrep")
+                                (string-append bin "/fgrep"))
+               (("^exec grep")
+                (string-append "exec " bin "/grep")))))))
+scheme@@(guix-user)> ,configure-flags findutils
+$2 = (list "--localstatedir=/var")
+scheme@@(guix-user)> ,make-flags binutils
+$3 = '("MAKEINFO=true")
+@end example
+
 At a lower-level, a useful command is @code{lower}: it takes a file-like
 object and ``lowers'' it into a derivation (@pxref{Derivations}) or a
 store file:
@@ -12794,6 +12844,17 @@ Using Guix Interactively
 shows build events only, and higher levels print build logs.
 @end deffn
 
+@deffn {REPL command} phases @var{package}
+@deffnx {REPL command} configure-flags @var{package}
+@deffnx {REPL command} make-flags @var{package}
+These REPL commands return the value of one element of the
+@code{arguments} field of @var{package} (@pxref{package Reference}): the
+first one show the staged code associated with @code{#:phases}
+(@pxref{Build Phases}), the second shows the code for
+@code{#:configure-flags}, and @code{,make-flags} returns the code for
+@code{#:make-flags}.
+@end deffn
+
 @deffn {REPL command} run-in-store @var{exp}
 Run @var{exp}, a monadic expression, through the store monad.
 @xref{The Store Monad}, for more information.
diff --git a/guix/monad-repl.scm b/guix/monad-repl.scm
index 8a6053edd5..d6b39112b7 100644
--- a/guix/monad-repl.scm
+++ b/guix/monad-repl.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2014, 2015, 2016, 2022 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2014-2016, 2022-2023 Ludovic Courtès <ludo@gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -21,13 +21,15 @@ (define-module (guix monad-repl)
   #:use-module (guix monads)
   #:use-module (guix utils)
   #:use-module (guix packages)
+  #:autoload   (guix build-system) (bag)
   #:use-module (guix status)
-  #:autoload   (guix gexp) (lower-object)
+  #:autoload   (guix gexp) (gexp gexp? lower-gexp lowered-gexp-sexp lower-object)
   #:use-module ((guix derivations)
                 #:select (derivation?
                           derivation->output-paths built-derivations))
+  #:autoload   (guix read-print) (pretty-print-with-comments)
   #:use-module (ice-9 match)
-  #:use-module (ice-9 pretty-print)
+  #:autoload   (ice-9 pretty-print) (pretty-print)
   #:use-module (system repl repl)
   #:use-module (system repl common)
   #:use-module (system repl command)
@@ -138,4 +140,68 @@ (define-meta-command ((enter-store-monad guix) repl)
       (repl-option-set! new 'interp #t)
       (run-repl new))))
 
-;;; monad-repl.scm ends here
+\f
+;;;
+;;; Viewing package arguments.
+;;;
+
+(define (keyword-argument-value args keyword default)
+  "Return the value associated with KEYWORD in ARGS, a keyword/value sequence,
+or DEFAULT if KEYWORD is missing from ARGS."
+  (let loop ((args args))
+    (match args
+      (()
+       default)
+      ((kw value rest ...)
+       (if (eq? kw keyword)
+           value
+           (loop rest))))))
+
+(define (package-argument-command repl form keyword default)
+  "Implement a command that display KEYWORD, a keyword such as #:phases, in
+the arguments of the package FORM evaluates to.  Return DEFAULT is KEYWORD is
+missing from those arguments."
+  (match (repl-eval repl form)
+    ((? package? package)
+     (let* ((bag* (bag
+                    (inherit (package->bag package))
+                    (build (lambda* (name inputs #:rest args)
+                             (with-monad %store-monad
+                               (return (keyword-argument-value args keyword
+                                                               default))))))))
+       (define phases
+         (parameterize ((%graft? #f))
+           (with-store store
+             (set-build-options store
+                                #:print-build-trace #t
+                                #:print-extended-build-trace? #t
+                                #:multiplexed-build-output? #t)
+             (run-with-store store
+               (mlet %store-monad ((exp (bag->derivation bag*)))
+                 (if (gexp? exp)
+                     (mlet %store-monad ((gexp (lower-gexp exp)))
+                       (return (lowered-gexp-sexp gexp)))
+                     (return exp)))))))
+
+       (run-hook before-print-hook phases)
+       (let ((column (port-column (current-output-port))))
+         (pretty-print-with-comments (current-output-port) phases
+                                     #:indent column)
+         (newline (current-output-port)))))
+    (_
+     (format #t ";; ERROR: This command only accepts package records.~%"))))
+
+(define-meta-command ((phases guix) repl (form))
+  "phases
+Return the build phases of the package defined by FORM."
+  (package-argument-command repl form #:phases #~%standard-phases))
+
+(define-meta-command ((configure-flags guix) repl (form))
+  "configure-flags
+Return the configure flags of the package defined by FORM."
+  (package-argument-command repl form #:configure-flags #~'()))
+
+(define-meta-command ((make-flags guix) repl (form))
+  "make-flags
+Return the make flags of the package defined by FORM."
+  (package-argument-command repl form #:make-flags #~'()))

base-commit: f3173bb901c0eee6c71d066cffab115d27052fc1
-- 
2.41.0

[bug#67489] [PATCH] monad-repl: Add REPL commands to inspect package arguments.

[Simon Tournier]

Hi,

TLDR: All LGTM. :-)

Minor comments.


On Mon, 27 Nov 2023 at 18:19, Ludovic Courtès <ludo@gnu.org> wrote:

> Someone recently suggested we should have better debugging facilities
> for packagers, in particular when dealing with package variants,
> inheritance, and so on, which make it harder to understand what’s
> going on.

From a quick test:

--8<---------------cut here---------------start------------->8---
scheme@(guix-user)> ,phases cava
$1 = (modify-phases %standard-phases
       (replace 'bootstrap
         (lambda _
           (setenv "HOME"
                   (getcwd))
           (invoke "sh" "autogen.sh")))
       (add-before 'build 'make-cava-ldflags
         (lambda _
           (mkdir-p (string-append ((@ (guile) getenv)
                                    "out") "/lib"))))
       (add-after 'install 'data
         (lambda _
           (for-each (lambda (file)
                       (install-file file
                                     (string-append ((@ (guile) getenv)
                                                     "out")
                                                    "/share/doc/examples")))
                     (find-files "example_files")))))
scheme@(guix-user)> ,configure-flags cava
$2 = (list (string-append "PREFIX="
                          ((@ (guile) getenv)
                           "out"))
           (string-append "FONT_DIR="
                          ((@ (guile) getenv)
                           "out") "/share/consolefonts"))
--8<---------------cut here---------------end--------------->8---

Cool!  Very helpful.  Much better than, something like:

--8<---------------cut here---------------start------------->8---
scheme@(guix-user)> ,pp (cdr (memq #:phases (package-arguments cava)))
$3 = (#<gexp (modify-phases %standard-phases (replace (quote bootstrap) (lambda _ (setenv "HOME" (getcwd)) (invoke "sh" "autogen.sh"))) (add-before (quote build) (quote make-cava-ldflags) (lambda _ (mkdir-p (string-append #<gexp-output out> "/lib")))) (add-after (quote install) (quote data) (lambda _ (for-each (lambda (file) (install-file file (string-append #<gexp-output out> "/share/doc/examples"))) (find-files "example_files"))))) gnu/packages/audio.scm:5161:11 7f7258403a50>)
--8<---------------cut here---------------end--------------->8---

Without drifting, why not the meta-command ,arguments?  For example, the
package could also have other specific keywords that maybe I would like
to inspect from the REPL.  For example, some packages have their own
keywords as #:julia-package-name.

Well, package-arguments partially does the job:

--8<---------------cut here---------------start------------->8---
scheme@(guix-user)> ,pp (package-arguments julia-uris)
$5 = (#:julia-package-name
 "URIs"
 #:julia-package-uuid
 "5c2747f8-b7ea-4ff2-ba2e-563bfd36b1d4"
 #:phases
 #<gexp (modify-phases %standard-phases (add-before (quote check) (quote change-dir) (lambda* (#:key source outputs #:allow-other-keys) (let ((out (assoc-ref outputs "out"))) (chdir (string-append out "/share/julia/loadpath/URIs/test")))))) gnu/packages/julia-xyz.scm:6482:6 7f725a441120>)
--8<---------------cut here---------------end--------------->8---

Other said, it could be nice to:

 1. pretty-print package-arguments by default,
 2. and compile the G-expressions.

Hence ,arguments. :-)


> +As a packager, you may be willing to inspect the build phases or flags
> +of a given package; this is particularly useful when relying a lot on
> +inheritance to define package variants (@pxref{Defining Package
> +Variants}) or when package arguments are a result of some computation,
> +both of which can make it harder to foresee what ends up in the package
> +arguments.  Additional commands let you inspect those package arguments:
> +
> +@example
> +scheme@@(guix-user)> ,phases grep
> +$1 = (modify-phases %standard-phases
> +       (add-after 'install 'fix-egrep-and-fgrep
> +         (lambda* (#:key outputs #:allow-other-keys)
> +           (let* ((out (assoc-ref outputs "out"))
> +                  (bin (string-append out "/bin")))
> +             (substitute* (list (string-append bin "/egrep")
> +                                (string-append bin "/fgrep"))
> +               (("^exec grep")
> +                (string-append "exec " bin "/grep")))))))
> +scheme@@(guix-user)> ,configure-flags findutils
> +$2 = (list "--localstatedir=/var")
> +scheme@@(guix-user)> ,make-flags binutils
> +$3 = '("MAKEINFO=true")
> +@end example
> +

Here all is clear.  What can be confusing is the expansion of #$output
as ((@ (guile) getenv) "out").  Maybe it could be worth to document this
under G-expression section.  Or provide a similar example using
G-expression under the section G-expression.

Well, that’s unrelated to this patch and could be done separately. :-)



>  ;;; GNU Guix --- Functional package management for GNU
> -;;; Copyright © 2014, 2015, 2016, 2022 Ludovic Courtès <ludo@gnu.org>
> +;;; Copyright © 2014-2016, 2022-2023 Ludovic Courtès <ludo@gnu.org>
>  ;;;
>  ;;; This file is part of GNU Guix.
>  ;;;
> @@ -21,13 +21,15 @@ (define-module (guix monad-repl)
>    #:use-module (guix monads)
>    #:use-module (guix utils)
>    #:use-module (guix packages)
> +  #:autoload   (guix build-system) (bag)
>    #:use-module (guix status)
> -  #:autoload   (guix gexp) (lower-object)
> +  #:autoload   (guix gexp) (gexp gexp? lower-gexp lowered-gexp-sexp lower-object)
>    #:use-module ((guix derivations)
>                  #:select (derivation?
>                            derivation->output-paths built-derivations))
> +  #:autoload   (guix read-print) (pretty-print-with-comments)
>    #:use-module (ice-9 match)
> -  #:use-module (ice-9 pretty-print)
> +  #:autoload   (ice-9 pretty-print) (pretty-print)

Out of curiosity, what is the advantage for #:autoload versus
#:use-module + #:select?


> +(define (keyword-argument-value args keyword default)
> +  "Return the value associated with KEYWORD in ARGS, a keyword/value sequence,
> +or DEFAULT if KEYWORD is missing from ARGS."
> +  (let loop ((args args))
> +    (match args
> +      (()
> +       default)
> +      ((kw value rest ...)
> +       (if (eq? kw keyword)
> +           value
> +           (loop rest))))))

    ( Aside.  Each time, I am surprised that common helpers are not
       Guile builtins, here for extracting keyword value.  It would help
       Guile to provide such builtin, by default or via some ice-9
       module.

     My 2 cents. ;-) )

Cheers,
simon

[bug#67489] [PATCH] monad-repl: Add REPL commands to inspect package arguments.

[Ludovic Courtès]

Hello!

Simon Tournier <zimon.toutoune@gmail.com> skribis:

> TLDR: All LGTM. :-)

Thanks.  :-)

> scheme@(guix-user)> ,pp (cdr (memq #:phases (package-arguments cava)))
> $3 = (#<gexp (modify-phases %standard-phases (replace (quote bootstrap) (lambda _ (setenv "HOME" (getcwd)) (invoke "sh" "autogen.sh"))) (add-before (quote build) (quote make-cava-ldflags) (lambda _ (mkdir-p (string-append #<gexp-output out> "/lib")))) (add-after (quote install) (quote data) (lambda _ (for-each (lambda (file) (install-file file (string-append #<gexp-output out> "/share/doc/examples"))) (find-files "example_files"))))) gnu/packages/audio.scm:5161:11 7f7258403a50>)
>
>
> Without drifting, why not the meta-command ,arguments? 

One can already do “,pp (package-arguments package)”, so I wanted to
have something short and to-the-point that would improve on that.

In particular, we know by convention that certain arguments can be
interpreted as code (gexps or sexps) so we can process them do display
them in a nice way (lowering gexps, applying the pretty printer).  We
cannot necessarily do that for other arguments where there’s no
established convention.

These REPL commands are really tailored for packagers.

> Here all is clear.  What can be confusing is the expansion of #$output
> as ((@ (guile) getenv) "out").  Maybe it could be worth to document this
> under G-expression section.  Or provide a similar example using
> G-expression under the section G-expression.
>
> Well, that’s unrelated to this patch and could be done separately. :-)

Yeah, we could/should document the implementation of #$output.

> Out of curiosity, what is the advantage for #:autoload versus
> #:use-module + #:select?

#:autoload allows modules to be loaded lazily, the first time one of the
selected symbols is used.  It’s a way to reduce startup time (equivalent
to Emacs autoloads).

>> +(define (keyword-argument-value args keyword default)
>> +  "Return the value associated with KEYWORD in ARGS, a keyword/value sequence,
>> +or DEFAULT if KEYWORD is missing from ARGS."
>> +  (let loop ((args args))
>> +    (match args
>> +      (()
>> +       default)
>> +      ((kw value rest ...)
>> +       (if (eq? kw keyword)
>> +           value
>> +           (loop rest))))))
>
>     ( Aside.  Each time, I am surprised that common helpers are not
>        Guile builtins, here for extracting keyword value.  It would help
>        Guile to provide such builtin, by default or via some ice-9
>        module.

Yeah, I was disappointed too.  :-)  (ice-9 optargs) has helpers that
weren’t a good match.

Thanks for your feedback!

Ludo’.

[bug#67489] [PATCH] monad-repl: Add REPL commands to inspect package arguments.

[Simon Tournier]

Re,

On Tue, 28 Nov 2023 at 17:20, Ludovic Courtès <ludo@gnu.org> wrote:

> > Without drifting, why not the meta-command ,arguments?
>
> One can already do “,pp (package-arguments package)”, so I wanted to
> have something short and to-the-point that would improve on that.
>
> In particular, we know by convention that certain arguments can be
> interpreted as code (gexps or sexps) so we can process them do display
> them in a nice way (lowering gexps, applying the pretty printer).  We
> cannot necessarily do that for other arguments where there’s no
> established convention.

Unrelated to this patch series, maybe an opportunity for establishing
some of these conventions, no?

BTW, because this potential meta ,arguments targets package developer,
I would find nice to apply what it is done with ,phases to
'package-arguments'.  As I showed in my previous message, “,pp
(package-arguments package)” is not really practical.  Well, anyway...
let as a exercise for the curious reader. ;-)

Cheers,
simon

bug#67489: [PATCH] monad-repl: Add REPL commands to inspect package arguments.

[Ludovic Courtès]

Hi,

Simon Tournier <zimon.toutoune@gmail.com> skribis:

> TLDR: All LGTM. :-)

Pushed as 3178b1a442c2f2eeeec1ff73bb852d2837b8aa3d!

Ludo’.