[bug#74425] [PATCH] ui: Include channels in load path before searching for commands.

[bug#74425] [PATCH] ui: Include channels in load path before searching for commands.

[Carlo Zancanaro]

* guix/ui.scm (extension-directories): Add channel directories to load path,
compiled load path, and returned list of directories.

Co-authored-by: Ludovic Courtès <ludo@gnu.org>
Change-Id: I3063cbf7164a065b2c6c2a5c6473df79ce8cbe9b
---

This patch allows extensions in channels to be found by the Guix
CLI. Most notably, this means that a script can define commands by
defining an appropriate (guix scripts X) module, with a guix-X
function.

This is a slight modification to a diff that Ludovic sent through IRC.

I've tested that it works when I use "guix time-machine". Guix is able
to find a command that I define in a channel.

I haven't specifically tested that it finds commands when run through
"guix pull", but I have tested that this change does not break "guix
pull".

 guix/ui.scm | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/guix/ui.scm b/guix/ui.scm
index 447550635c..a702b6f3f8 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -19,6 +19,7 @@
 ;;; Copyright © 2018 Steve Sprang <scs@stevesprang.com>
 ;;; Copyright © 2022 Taiju HIGASHI <higashi@taiju.info>
 ;;; Copyright © 2022 Liliana Marie Prikler <liliana.prikler@gmail.com>
+;;; Copyright © 2024 Carlo Zancanaro <carlo@zancanaro.id.au>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -73,6 +74,7 @@ (define-module (guix ui)                       ;import in user interfaces only
   #:use-module (srfi srfi-31)
   #:use-module (srfi srfi-34)
   #:use-module (srfi srfi-35)
+  #:use-module (srfi srfi-71)
   #:autoload   (ice-9 ftw)  (scandir)
   #:use-module (ice-9 match)
   #:use-module (ice-9 format)
@@ -2192,9 +2194,16 @@ (define* (command-files #:optional directory)
 
 (define (extension-directories)
   "Return the list of directories containing Guix extensions."
-  (filter file-exists?
-          (parse-path
-           (getenv "GUIX_EXTENSIONS_PATH"))))
+  (define package-path-entries
+    ;; We need to resolve this lazily, because even using an #:autoload is too
+    ;; much and breaks compilation during "guix pull".
+    (module-ref (resolve-module `(guix describe))
+                (symbol-append 'package-path-entries)))
+  (let ((scm-path go-path (package-path-entries)))
+    (set! %load-path (append %load-path scm-path))
+    (set! %load-compiled-path (append %load-compiled-path go-path))
+    (filter file-exists?
+            (parse-path (getenv "GUIX_EXTENSIONS_PATH") scm-path))))
 
 (define (commands)
   "Return the list of commands, alphabetically sorted."

base-commit: 23cbbe6860782c5d4a0ba599ea1cda0642e91661
-- 
2.46.0

[bug#74425] [PATCH] ui: Include channels in load path before searching for commands.

[Ludovic Courtès]

Hello,

Carlo Zancanaro <carlo@zancanaro.id.au> skribis:

> * guix/ui.scm (extension-directories): Add channel directories to load path,
> compiled load path, and returned list of directories.
>
> Co-authored-by: Ludovic Courtès <ludo@gnu.org>
> Change-Id: I3063cbf7164a065b2c6c2a5c6473df79ce8cbe9b

Nice, thanks for looking into it!

>  (define (extension-directories)
>    "Return the list of directories containing Guix extensions."
> -  (filter file-exists?
> -          (parse-path
> -           (getenv "GUIX_EXTENSIONS_PATH"))))
> +  (define package-path-entries
> +    ;; We need to resolve this lazily, because even using an #:autoload is too
> +    ;; much and breaks compilation during "guix pull".
> +    (module-ref (resolve-module `(guix describe))
> +                (symbol-append 'package-path-entries)))
> +  (let ((scm-path go-path (package-path-entries)))
> +    (set! %load-path (append %load-path scm-path))
> +    (set! %load-compiled-path (append %load-compiled-path go-path))
> +    (filter file-exists?
> +            (parse-path (getenv "GUIX_EXTENSIONS_PATH") scm-path))))

I think you could do:

  #:autoload (guix describe) (package-path-entries)

instead of doing the trick above.

Two things:

  1. The ‘%load-path’ and ‘%load-compiled-path’ modifications can now be
     removed from (gnu packages), after checking that GUIX_PACKAGE_PATH
     still takes precedence.

  2. The performance impact of this change must be tested, in particular
     the startup time of ‘guix’ commands since they’ll now be loading
     (guix channels) & co. unconditionally.

     One way to do that might be:

       PROFILE=$(guix time-machine -q --url=/path/to/checkout)
       guix shell time -- time $PROFILE/bin/guix build --help
       strace -c $PROFILE/bin/guix build --help
     
Could you give it a try?

Ludo’.

[bug#74425] [PATCH] ui: Include channels in load path before searching for commands.

[Carlo Zancanaro]

On Thu, Nov 21 2024, Ludovic Courtès wrote:
> I think you could do:
>
>   #:autoload (guix describe) (package-path-entries)
>
> instead of doing the trick above.

I tried that, and it breaks "guix pull" (as mentioned in the comment in
the code). I don't have the specific error easily accessible, but I
think it was "no code for module: (git)". I assume it was an issue with
how build-program in (build-self) puts things together, but I wasn't
able to investigate further than that.

>   1. The ‘%load-path’ and ‘%load-compiled-path’ modifications can now be
>      removed from (gnu packages), after checking that GUIX_PACKAGE_PATH
>      still takes precedence.

With this change, (extension-directories) does not include the paths in
GUIX_PACKAGE_PATH. Perhaps it should, but I wasn't sure given the
distinction between GUIX_PACKAGE_PATH and GUIX_EXTENSIONS_PATH.

Do we want to run commands from GUIX_PACKAGE_PATH? Or load packages from
GUIX_EXTENSIONS_PATH? At the moment I think the code says "no" to both.

My other concern with removing the code in (gnu packages) is that it may
break code that is using Guix as a library. That is, something like
this:

--8<---------------cut here---------------start------------->8---
guix shell guix guile -- guile -c '(use-modules (gnu packages)) (pk (find-packages-by-name "guile"))'
--8<---------------cut here---------------end--------------->8---

might not add the right things to the load path.

That said, the current patch will add duplicate entries to the load path
when using the Guix CLI, which also isn't good. Perhaps this needs to be
factored out into a single place where we can keep track of whether
we've already added to the load paths.

>   2. The performance impact of this change must be tested, in particular
>      the startup time of ‘guix’ commands since they’ll now be loading
>      (guix channels) & co. unconditionally.
>
>      One way to do that might be:
>
>        PROFILE=$(guix time-machine -q --url=/path/to/checkout)
>        guix shell time -- time $PROFILE/bin/guix build --help
>        strace -c $PROFILE/bin/guix build --help
>      
> Could you give it a try?

Sure!

With the change in my patch, I'm seeing:

strace: number of calls and errors is roughly the same both with and
without my patch.

time: all the numbers look pretty similar with and without my patch.

I haven't done any statistical analysis of the various values recorded
by strace and time, but the numbers don't vary *too* much between runs,
so I'm fairly confident this isn't having too much of an effect.

I also tested leaving off the "-q" in the time machine call (so my other
channels were also included), but this didn't make any meaningful
difference to the numbers when running "guix build --help".

Carlo