Extending Guix without using the Guile load path

Extending Guix without using the Guile load path

[Ricardo Wurmus]

Hi Guix,

I think it’s a bit difficult to install the Guix Workflow Language at
this point and I’d like to change that.

Currently, new sub-commands for Guix are looked up by module name on the
Guile load path.  When installing the “gwl” package, though, the Guile
load path is not automatically altered, so users need to set it up by
themselves.  The load path is only altered automatically when users
install the “guile” package.  This is not a good recommendation because
users may have Guile 2.2 in their profile, and not Guile 3.0 or whatever
version may be needed by the extension.

I wonder if we can make this a little nicer by letting Guix look for
sub-command scripts in directories that are listed in an environment
variable, such as GUIX_EXTENSIONS_PATH.  The “guix” package would set
this search path and packages wanting to provide a sub-command (such as
“guix workflow” or “guix home”) would arrange to have their scripts
placed in that sub-directory of their outputs.

What do you think?

--
Ricardo

Re: Extending Guix without using the Guile load path

[Gábor Boskovits]

[-- Attachment #1: Type: text/plain, Size: 1459 bytes --]

This looks interesting.

Ricardo Wurmus <rekado@elephly.net> ezt írta (időpont: 2020. febr. 19., Sze
16:37):

> Hi Guix,
>
> I think it’s a bit difficult to install the Guix Workflow Language at
> this point and I’d like to change that.
>
> Currently, new sub-commands for Guix are looked up by module name on the
> Guile load path.  When installing the “gwl” package, though, the Guile
> load path is not automatically altered, so users need to set it up by
> themselves.  The load path is only altered automatically when users
> install the “guile” package.  This is not a good recommendation because
> users may have Guile 2.2 in their profile, and not Guile 3.0 or whatever
> version may be needed by the extension.
>
> I wonder if we can make this a little nicer by letting Guix look for
> sub-command scripts in directories that are listed in an environment
> variable, such as GUIX_EXTENSIONS_PATH.  The “guix” package would set
> this search path and packages wanting to provide a sub-command (such as
> “guix workflow” or “guix home”) would arrange to have their scripts
> placed in that sub-directory of their outputs.
>

I have encountered this with guix home, and one of my first setups was to
adjust the path.


> What do you think?
>

I believe this is a good idea. I also think there is an open bug already
for the home thing.


> --
> Ricardo
>

Best regards,
g_bor

>
>

[-- Attachment #2: Type: text/html, Size: 2479 bytes --]

Re: Extending Guix without using the Guile load path

[Julien Lepiller]

Le 19 février 2020 14:46:55 GMT-05:00, "Gábor Boskovits" <boskovits@gmail.com> a écrit :
>This looks interesting.
>
>Ricardo Wurmus <rekado@elephly.net> ezt írta (időpont: 2020. febr. 19.,
>Sze
>16:37):
>
>> Hi Guix,
>>
>> I think it’s a bit difficult to install the Guix Workflow Language at
>> this point and I’d like to change that.
>>
>> Currently, new sub-commands for Guix are looked up by module name on
>the
>> Guile load path.  When installing the “gwl” package, though, the
>Guile
>> load path is not automatically altered, so users need to set it up by
>> themselves.  The load path is only altered automatically when users
>> install the “guile” package.  This is not a good recommendation
>because
>> users may have Guile 2.2 in their profile, and not Guile 3.0 or
>whatever
>> version may be needed by the extension.
>>
>> I wonder if we can make this a little nicer by letting Guix look for
>> sub-command scripts in directories that are listed in an environment
>> variable, such as GUIX_EXTENSIONS_PATH.  The “guix” package would set
>> this search path and packages wanting to provide a sub-command (such
>as
>> “guix workflow” or “guix home”) would arrange to have their scripts
>> placed in that sub-directory of their outputs.
>>
>
>I have encountered this with guix home, and one of my first setups was
>to
>adjust the path.
>
>
>> What do you think?
>>
>
>I believe this is a good idea. I also think there is an open bug
>already
>for the home thing.

I have a workaround in the bash-home-type, but of course it's not at all convenient. Fixing this would help a lot :)

>
>
>> --
>> Ricardo
>>
>
>Best regards,
>g_bor
>
>>
>>

Re: Extending Guix without using the Guile load path

[Ludovic Courtès]

Hello!

Ricardo Wurmus <rekado@elephly.net> skribis:

> I think it’s a bit difficult to install the Guix Workflow Language at
> this point and I’d like to change that.
>
> Currently, new sub-commands for Guix are looked up by module name on the
> Guile load path.  When installing the “gwl” package, though, the Guile
> load path is not automatically altered, so users need to set it up by
> themselves.  The load path is only altered automatically when users
> install the “guile” package.  This is not a good recommendation because
> users may have Guile 2.2 in their profile, and not Guile 3.0 or whatever
> version may be needed by the extension.
>
> I wonder if we can make this a little nicer by letting Guix look for
> sub-command scripts in directories that are listed in an environment
> variable, such as GUIX_EXTENSIONS_PATH.  The “guix” package would set
> this search path and packages wanting to provide a sub-command (such as
> “guix workflow” or “guix home”) would arrange to have their scripts
> placed in that sub-directory of their outputs.
>
> What do you think?

GUIX_EXTENSIONS_PATH sounds like a good idea.  I suppose it could be
implemented pretty much like GUIX_PACKAGE_PATH?

That would also allow us to consider Guix Home a package rather than a
channel, like you did for GWL.

(Though eventually, I do think something like Guix Home should be part
of Guix itself.  :-))

Ludo’.

Re: Extending Guix without using the Guile load path

[Joshua Branson]

Ludovic Courtès <ludo@gnu.org> writes:

>
> (Though eventually, I do think something like Guix Home should be part
> of Guix itself.  :-))

May I ask is Guix Home similar to systemd homed?  Or do they have
slightly different objectives?

-- 
Joshua Branson
Sent from Emacs and Gnus

Re: Extending Guix without using the Guile load path

[Julien Lepiller]

Le 17 mars 2020 14:32:59 GMT-04:00, Joshua Branson <jbranso@dismail.de> a écrit :
>Ludovic Courtès <ludo@gnu.org> writes:
>
>>
>> (Though eventually, I do think something like Guix Home should be
>part
>> of Guix itself.  :-))
>
>May I ask is Guix Home similar to systemd homed?  Or do they have
>slightly different objectives?

From what I understand, homed makes your home an archive (a single file that is transparently mounted) that can be used on different machines. I'm not sure what this is trying to address, but it sounds very much like using a mutable container in terms of benefits.

The home manager doesn't take care of any data in your home, only your configuration. In fact, you need a separate directory for your data so the home manager can take over your home. It's the guix way of creating your home configuration. It makes it portable too, since a single file is required to reproduce the same home environment (excluding your data). In a way, this is similar to how guix is a solution to containers' drawbacks :)

However, just like guix can produce docker containers, in theory I think building a homed archive is perfectly possible. Both approaches are not incompatible.

Even though it's technically nice and intellectually satisfiying, it is still very alpha and limited. Don't expect much of it in practice!

Re: Extending Guix without using the Guile load path

[Ricardo Wurmus]

[-- Attachment #1: Type: text/plain, Size: 2847 bytes --]


Ludovic Courtès <ludo@gnu.org> writes:

> Hello!
>
> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>> I think it’s a bit difficult to install the Guix Workflow Language at
>> this point and I’d like to change that.
>>
>> Currently, new sub-commands for Guix are looked up by module name on the
>> Guile load path.  When installing the “gwl” package, though, the Guile
>> load path is not automatically altered, so users need to set it up by
>> themselves.  The load path is only altered automatically when users
>> install the “guile” package.  This is not a good recommendation because
>> users may have Guile 2.2 in their profile, and not Guile 3.0 or whatever
>> version may be needed by the extension.
>>
>> I wonder if we can make this a little nicer by letting Guix look for
>> sub-command scripts in directories that are listed in an environment
>> variable, such as GUIX_EXTENSIONS_PATH.  The “guix” package would set
>> this search path and packages wanting to provide a sub-command (such as
>> “guix workflow” or “guix home”) would arrange to have their scripts
>> placed in that sub-directory of their outputs.
>>
>> What do you think?
>
> GUIX_EXTENSIONS_PATH sounds like a good idea.  I suppose it could be
> implemented pretty much like GUIX_PACKAGE_PATH?
>
> That would also allow us to consider Guix Home a package rather than a
> channel, like you did for GWL.

Below is a draft that adds Guile modules from GUIX_EXTENSIONS_PATH to
the %load-path and %load-compiled-path.

I think this implementation is not good, but I’d like to provoke some
comments about the following thoughts:

* what happens to the Guile dependencies of an extension?  Those would
  not be added to the load path.  Should the extension take care of this
  by manually augmenting the load path?

* The draft simply uses the same directories that GUILE_LOAD_PATH and
  GUILE_LOAD_COMPILED_PATH use.  Is this a bad idea?  Would it not be
  better to have a new directory prefix (such as “lib/guix/extensions”)?

* The search path on the “guix” package does not distinguish between
  compiled modules and source modules; it simply looks for all the
  conventional directories and puts them on the GUIX_EXTENSIONS_PATH,
  while (guix ui) adds them to both %load-path and %load-compiled-path.

All these points could be solved by only requiring the *entry points* to
be on the GUIX_EXTENSIONS_PATH.  Say for example that we have the
“workflows” extension (provided by the GWL), which is a Guile entry
point script at “libexec/guix/workflows.scm”.  That script would augment
the %load-path and %load-compiled-path with all its modules (in the
conventional Guile directories) and load the extension from those
modules.

What do you think?

-- 
Ricardo



[-- Attachment #2: 0001-WIP-GUIX_EXTENSIONS_PATH.patch --]
[-- Type: text/x-patch, Size: 3411 bytes --]

From a6bdaee8a8cf403ea0cc76426e595afc36c9f559 Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <rekado@elephly.net>
Date: Sat, 31 Oct 2020 23:42:12 +0100
Subject: [PATCH] WIP: GUIX_EXTENSIONS_PATH

---
 gnu/packages/package-management.scm | 14 +++++++++++++-
 guix/ui.scm                         | 11 ++++++++++-
 2 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm
index 1c02c342b1..b6c03acf15 100644
--- a/gnu/packages/package-management.scm
+++ b/gnu/packages/package-management.scm
@@ -400,6 +400,12 @@ $(prefix)/etc/init.d\n")))
          ("guile-git" ,guile-git)
          ("guile-zlib" ,guile-zlib)
          ("guile-lzlib" ,guile-lzlib)))
+      (native-search-paths
+       (list (search-path-specification
+              (variable "GUIX_EXTENSIONS_PATH")
+              (files '("share/guile/site/3.0"
+                       "lib/guile/3.0/site-ccache"
+                       "share/guile/site/3.0")))))
 
       (home-page "https://www.gnu.org/software/guix/")
       (synopsis "Functional package manager for installed software packages and versions")
@@ -496,7 +502,13 @@ the Nix package manager.")
        ("guile-json" ,guile2.2-json)
        ("guile-sqlite3" ,guile2.2-sqlite3)
        ("guile-ssh" ,guile2.2-ssh)
-       ("guile-git" ,guile2.2-git)))))
+       ("guile-git" ,guile2.2-git)))
+    (native-search-paths
+     (list (search-path-specification
+            (variable "GUIX_EXTENSIONS_PATH")
+            (files '("share/guile/site/2.2"
+                     "lib/guile/2.2/site-ccache"
+                     "share/guile/site/2.2")))))))
 
 (define-public guile3.0-guix
   (deprecated-package "guile3.0-guix" guix))
diff --git a/guix/ui.scm b/guix/ui.scm
index 8d7bc238bc..8dbcbe0fd6 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -10,7 +10,7 @@
 ;;; Copyright © 2016 Roel Janssen <roel@gnu.org>
 ;;; Copyright © 2016 Benz Schenk <benz.schenk@uzh.ch>
 ;;; Copyright © 2018 Kyle Meyer <kyle@kyleam.com>
-;;; Copyright © 2018 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2018, 2020 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2019 Chris Marusich <cmmarusich@gmail.com>
 ;;; Copyright © 2019 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2019 Simon Tournier <zimon.toutoune@gmail.com>
@@ -61,6 +61,7 @@
                         ;; Avoid "overrides core binding" warning.
                         delete))
   #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-2)
   #:use-module (srfi srfi-9 gnu)
   #:use-module (srfi srfi-11)
   #:use-module (srfi srfi-19)
@@ -2125,6 +2126,14 @@ Unlike 'guix-main', this procedure assumes that locale, i18n support,
 and signal handling have already been set up."
   (define option? (cut string-prefix? "-" <>))
 
+  (and-let* ((path (getenv "GUIX_EXTENSIONS_PATH"))
+             (not-colon (char-set-complement (char-set #\:)))
+             (extensions (string-tokenize path not-colon)))
+    (set! %load-path
+          (append extensions %load-path))
+    (set! %load-compiled-path
+          (append extensions %load-compiled-path)))
+
   ;; The default %LOAD-EXTENSIONS includes the empty string, which doubles the
   ;; number of 'stat' calls per entry in %LOAD-PATH.  Shamelessly remove it.
   (set! %load-extensions '(".scm"))
-- 
2.28.0

Re: Extending Guix without using the Guile load path

[Ricardo Wurmus]

[-- Attachment #1: Type: text/plain, Size: 2847 bytes --]


Ludovic Courtès <ludo@gnu.org> writes:

> Hello!
>
> Ricardo Wurmus <rekado@elephly.net> skribis:
>
>> I think it’s a bit difficult to install the Guix Workflow Language at
>> this point and I’d like to change that.
>>
>> Currently, new sub-commands for Guix are looked up by module name on the
>> Guile load path.  When installing the “gwl” package, though, the Guile
>> load path is not automatically altered, so users need to set it up by
>> themselves.  The load path is only altered automatically when users
>> install the “guile” package.  This is not a good recommendation because
>> users may have Guile 2.2 in their profile, and not Guile 3.0 or whatever
>> version may be needed by the extension.
>>
>> I wonder if we can make this a little nicer by letting Guix look for
>> sub-command scripts in directories that are listed in an environment
>> variable, such as GUIX_EXTENSIONS_PATH.  The “guix” package would set
>> this search path and packages wanting to provide a sub-command (such as
>> “guix workflow” or “guix home”) would arrange to have their scripts
>> placed in that sub-directory of their outputs.
>>
>> What do you think?
>
> GUIX_EXTENSIONS_PATH sounds like a good idea.  I suppose it could be
> implemented pretty much like GUIX_PACKAGE_PATH?
>
> That would also allow us to consider Guix Home a package rather than a
> channel, like you did for GWL.

Below is a draft that adds Guile modules from GUIX_EXTENSIONS_PATH to
the %load-path and %load-compiled-path.

I think this implementation is not good, but I’d like to provoke some
comments about the following thoughts:

* what happens to the Guile dependencies of an extension?  Those would
  not be added to the load path.  Should the extension take care of this
  by manually augmenting the load path?

* The draft simply uses the same directories that GUILE_LOAD_PATH and
  GUILE_LOAD_COMPILED_PATH use.  Is this a bad idea?  Would it not be
  better to have a new directory prefix (such as “lib/guix/extensions”)?

* The search path on the “guix” package does not distinguish between
  compiled modules and source modules; it simply looks for all the
  conventional directories and puts them on the GUIX_EXTENSIONS_PATH,
  while (guix ui) adds them to both %load-path and %load-compiled-path.

All these points could be solved by only requiring the *entry points* to
be on the GUIX_EXTENSIONS_PATH.  Say for example that we have the
“workflows” extension (provided by the GWL), which is a Guile entry
point script at “libexec/guix/workflows.scm”.  That script would augment
the %load-path and %load-compiled-path with all its modules (in the
conventional Guile directories) and load the extension from those
modules.

What do you think?

-- 
Ricardo



[-- Attachment #2: 0001-WIP-GUIX_EXTENSIONS_PATH.patch --]
[-- Type: text/x-patch, Size: 3411 bytes --]

From a6bdaee8a8cf403ea0cc76426e595afc36c9f559 Mon Sep 17 00:00:00 2001
From: Ricardo Wurmus <rekado@elephly.net>
Date: Sat, 31 Oct 2020 23:42:12 +0100
Subject: [PATCH] WIP: GUIX_EXTENSIONS_PATH

---
 gnu/packages/package-management.scm | 14 +++++++++++++-
 guix/ui.scm                         | 11 ++++++++++-
 2 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm
index 1c02c342b1..b6c03acf15 100644
--- a/gnu/packages/package-management.scm
+++ b/gnu/packages/package-management.scm
@@ -400,6 +400,12 @@ $(prefix)/etc/init.d\n")))
          ("guile-git" ,guile-git)
          ("guile-zlib" ,guile-zlib)
          ("guile-lzlib" ,guile-lzlib)))
+      (native-search-paths
+       (list (search-path-specification
+              (variable "GUIX_EXTENSIONS_PATH")
+              (files '("share/guile/site/3.0"
+                       "lib/guile/3.0/site-ccache"
+                       "share/guile/site/3.0")))))
 
       (home-page "https://www.gnu.org/software/guix/")
       (synopsis "Functional package manager for installed software packages and versions")
@@ -496,7 +502,13 @@ the Nix package manager.")
        ("guile-json" ,guile2.2-json)
        ("guile-sqlite3" ,guile2.2-sqlite3)
        ("guile-ssh" ,guile2.2-ssh)
-       ("guile-git" ,guile2.2-git)))))
+       ("guile-git" ,guile2.2-git)))
+    (native-search-paths
+     (list (search-path-specification
+            (variable "GUIX_EXTENSIONS_PATH")
+            (files '("share/guile/site/2.2"
+                     "lib/guile/2.2/site-ccache"
+                     "share/guile/site/2.2")))))))
 
 (define-public guile3.0-guix
   (deprecated-package "guile3.0-guix" guix))
diff --git a/guix/ui.scm b/guix/ui.scm
index 8d7bc238bc..8dbcbe0fd6 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -10,7 +10,7 @@
 ;;; Copyright © 2016 Roel Janssen <roel@gnu.org>
 ;;; Copyright © 2016 Benz Schenk <benz.schenk@uzh.ch>
 ;;; Copyright © 2018 Kyle Meyer <kyle@kyleam.com>
-;;; Copyright © 2018 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2018, 2020 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2019 Chris Marusich <cmmarusich@gmail.com>
 ;;; Copyright © 2019 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2019 Simon Tournier <zimon.toutoune@gmail.com>
@@ -61,6 +61,7 @@
                         ;; Avoid "overrides core binding" warning.
                         delete))
   #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-2)
   #:use-module (srfi srfi-9 gnu)
   #:use-module (srfi srfi-11)
   #:use-module (srfi srfi-19)
@@ -2125,6 +2126,14 @@ Unlike 'guix-main', this procedure assumes that locale, i18n support,
 and signal handling have already been set up."
   (define option? (cut string-prefix? "-" <>))
 
+  (and-let* ((path (getenv "GUIX_EXTENSIONS_PATH"))
+             (not-colon (char-set-complement (char-set #\:)))
+             (extensions (string-tokenize path not-colon)))
+    (set! %load-path
+          (append extensions %load-path))
+    (set! %load-compiled-path
+          (append extensions %load-compiled-path)))
+
   ;; The default %LOAD-EXTENSIONS includes the empty string, which doubles the
   ;; number of 'stat' calls per entry in %LOAD-PATH.  Shamelessly remove it.
   (set! %load-extensions '(".scm"))
-- 
2.28.0

Re: Extending Guix without using the Guile load path

[Ludovic Courtès]

Hi!

Ricardo Wurmus <rekado@elephly.net> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Hello!
>>
>> Ricardo Wurmus <rekado@elephly.net> skribis:
>>
>>> I think it’s a bit difficult to install the Guix Workflow Language at
>>> this point and I’d like to change that.
>>>
>>> Currently, new sub-commands for Guix are looked up by module name on the
>>> Guile load path.  When installing the “gwl” package, though, the Guile
>>> load path is not automatically altered, so users need to set it up by
>>> themselves.  The load path is only altered automatically when users
>>> install the “guile” package.  This is not a good recommendation because
>>> users may have Guile 2.2 in their profile, and not Guile 3.0 or whatever
>>> version may be needed by the extension.

[...]

>> GUIX_EXTENSIONS_PATH sounds like a good idea.  I suppose it could be
>> implemented pretty much like GUIX_PACKAGE_PATH?
>>
>> That would also allow us to consider Guix Home a package rather than a
>> channel, like you did for GWL.
>
> Below is a draft that adds Guile modules from GUIX_EXTENSIONS_PATH to
> the %load-path and %load-compiled-path.
>
> I think this implementation is not good, but I’d like to provoke some
> comments about the following thoughts:
>
> * what happens to the Guile dependencies of an extension?  Those would
>   not be added to the load path.  Should the extension take care of this
>   by manually augmenting the load path?

Hmm that doesn’t sound great.

> * The draft simply uses the same directories that GUILE_LOAD_PATH and
>   GUILE_LOAD_COMPILED_PATH use.  Is this a bad idea?  Would it not be
>   better to have a new directory prefix (such as “lib/guix/extensions”)?

On one hand, if it’s a different search path, you’d rather use a
different directory like lib/guix/extensions.  OTOH, all this is regular
Guile code and it’d be ridiculous to be unable to just have it on the
Guile load path.

> * The search path on the “guix” package does not distinguish between
>   compiled modules and source modules; it simply looks for all the
>   conventional directories and puts them on the GUIX_EXTENSIONS_PATH,
>   while (guix ui) adds them to both %load-path and %load-compiled-path.

Traditionally distros distinguish between arch-dependent and
arch-independent files, and that would prevent that.

(Thinking out loud.)

What if an extension could instead be a package installed next to Guix
and its channels in ~/.config/guix/current, and we use
‘package-path-entries’ as is done in (gnu packages) to augment
‘%load-path’ and ‘%load-compiled-path’?

Thanks,
Ludo’.

Re: Extending Guix without using the Guile load path

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> What if an extension could instead be a package installed next to Guix
> and its channels in ~/.config/guix/current, and we use
> ‘package-path-entries’ as is done in (gnu packages) to augment
> ‘%load-path’ and ‘%load-compiled-path’?

I’m not sure if that would complicate installation too much.  Installing
a package in the “guix pull” profile suggests to me that it is installed
when “guix pull” is run, and not by “guix package -p
~/.config/guix/current -i my-extension”.

I suppose it would help with propagated inputs, because they would also
end up in the “guix pull” profile, but perhaps that’s not really
desirable as an extension probably should not be able to accidentally
break Guix just because it uses a different version of a Guile library
that Guix also happens to use.

Keeping the directory tree of the extension separate from the “guix
pull” profile would ensure that the core features of Guix keep working,
no matter how badly broken an extension might be.

So I’m thinking that we should instead define GUIX_EXTENSIONS_PATH as a
list of directories containing merely the *entry points* for additional
Guix commands.  For example, the GWL would provide a script “workflow”
in a directory $prefix/share/guix/extensions/ (or whatever), and that
script is a wrapped executable that sets its own %load-path and
%load-compiled-path as needed (and determined at build time).  All Guix
does is search for matching executables on GUIX_EXTENSIONS_PATH when
asked for an unknown command.

This is disappointingly simple, but I think it’s one of the safest
things to do.

An incidental side effect of doing things this way is that extensions
could, in theory, be written in languages other than Guile (stretching
the meaning of “extension” to its limits).

-- 
Ricardo

Re: Extending Guix without using the Guile load path

[Ludovic Courtès]

Hi,

Ricardo Wurmus <rekado@elephly.net> skribis:

> Keeping the directory tree of the extension separate from the “guix
> pull” profile would ensure that the core features of Guix keep working,
> no matter how badly broken an extension might be.

Yeah.

> So I’m thinking that we should instead define GUIX_EXTENSIONS_PATH as a
> list of directories containing merely the *entry points* for additional
> Guix commands.  For example, the GWL would provide a script “workflow”
> in a directory $prefix/share/guix/extensions/ (or whatever), and that
> script is a wrapped executable that sets its own %load-path and
> %load-compiled-path as needed (and determined at build time).  All Guix
> does is search for matching executables on GUIX_EXTENSIONS_PATH when
> asked for an unknown command.
>
> This is disappointingly simple, but I think it’s one of the safest
> things to do.

Yes, sounds like a good idea.

> An incidental side effect of doing things this way is that extensions
> could, in theory, be written in languages other than Guile (stretching
> the meaning of “extension” to its limits).

Oooh, this is eviiiil!  :-)

Ludo’.

[PATCH] Discover extensions via GUIX_EXTENSIONS_PATH.

[Ricardo Wurmus]

* guix/scripts.scm (%command-categories): Add extension category.
* guix/ui.scm (command-files): Accept an optional directory argument.
(extension-directories): New procedure.
(commands): Use it.
(show-guix-help): Hide empty categories.
(run-guix-command): Try loading an extension if there is no Guix command.
---
 guix/scripts.scm |  4 +++-
 guix/ui.scm      | 60 +++++++++++++++++++++++++++++++++++-------------
 2 files changed, 47 insertions(+), 17 deletions(-)

diff --git a/guix/scripts.scm b/guix/scripts.scm
index 9792aaebe9..34cba35401 100644
--- a/guix/scripts.scm
+++ b/guix/scripts.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2014 Deck Pickard <deck.r.pickard@gmail.com>
 ;;; Copyright © 2015, 2016 Alex Kost <alezost@gmail.com>
 ;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
+;;; Copyright © 2021 Ricardo Wurmus <rekado@elephly.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -86,7 +87,8 @@
   (development (G_ "software development commands"))
   (packaging   (G_ "packaging commands"))
   (plumbing    (G_ "plumbing commands"))
-  (internal    (G_ "internal commands")))
+  (internal    (G_ "internal commands"))
+  (extension   (G_ "extension commands")))
 
 (define-syntax define-command
   (syntax-rules (category synopsis)
diff --git a/guix/ui.scm b/guix/ui.scm
index 0a1c9bd615..2ecfb53c7b 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -2046,24 +2046,36 @@ contain a 'define-command' form."
           (_
            (loop)))))))
 
-(define (command-files)
+(define* (command-files #:optional directory)
   "Return the list of source files that define Guix sub-commands."
-  (define directory
-    (and=> (search-path %load-path "guix.scm")
-           (compose (cut string-append <> "/guix/scripts")
-                    dirname)))
+  (define directory*
+    (or directory
+        (and=> (search-path %load-path "guix.scm")
+               (compose (cut string-append <> "/guix/scripts")
+                        dirname))))
 
   (define dot-scm?
     (cut string-suffix? ".scm" <>))
 
-  (if directory
-      (map (cut string-append directory "/" <>)
-           (scandir directory dot-scm?))
+  (if directory*
+      (map (cut string-append directory* "/" <>)
+           (scandir directory* dot-scm?))
       '()))
 
+(define (extension-directories)
+  "Return the list of directories containing Guix extensions."
+  (filter-map (lambda (directory)
+                (let ((scripts (string-append directory "/guix/scripts")))
+                  (and (file-exists? scripts) scripts)))
+              (parse-path
+               (getenv "GUIX_EXTENSIONS_PATH"))))
+
 (define (commands)
   "Return the list of commands, alphabetically sorted."
-  (filter-map source-file-command (command-files)))
+  (filter-map source-file-command
+              (append (command-files)
+                      (append-map command-files
+                                  (extension-directories)))))
 
 (define (show-guix-help)
   (define (internal? command)
@@ -2098,9 +2110,14 @@ Run COMMAND with ARGS.\n"))
                 (('internal . _)
                  #t)                              ;hide internal commands
                 ((category . synopsis)
-                 (format #t "~%  ~a~%" (G_ synopsis))
-                 (display-commands (filter (category-predicate category)
-                                           commands))))
+                 (let ((relevant-commands (filter (category-predicate category)
+                                                  commands)))
+                   ;; Only print categories that contain commands.
+                   (match relevant-commands
+                     ((one . more)
+                      (format #t "~%  ~a~%" (G_ synopsis))
+                      (display-commands relevant-commands))
+                     (_ #f)))))
               categories))
   (show-bug-report-information))
 
@@ -2111,10 +2128,21 @@ found."
     (catch 'misc-error
       (lambda ()
         (resolve-interface `(guix scripts ,command)))
-      (lambda -
-        (format (current-error-port)
-                (G_ "guix: ~a: command not found~%") command)
-        (show-guix-usage))))
+      (lambda _
+        ;; Check if there is a matching extension.
+        (catch 'misc-error
+          (lambda ()
+            (match (search-path (extension-directories)
+                                (format #f "~a.scm" command))
+              (file
+               (load file)
+               (resolve-interface `(guix scripts ,command)))
+              (_
+               (throw 'misc-error))))
+          (lambda _
+            (format (current-error-port)
+                    (G_ "guix: ~a: command not found~%") command)
+            (show-guix-usage))))))
 
   (let ((command-main (module-ref module
                                   (symbol-append 'guix- command))))
-- 
2.29.2