bug#42688: Running a script with `guix repl` doesn't "see" additional channels using (%package-module-path)

bug#42688: Running a script with `guix repl` doesn't "see" additional channels using (%package-module-path)

[pkill9]

Running the following in `guix repl` returns additional channels:
```
itsme@antelope ~> guix repl
GNU Guile 3.0.4
Copyright (C) 1995-2020 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guix-user)> (use-modules (gnu packages))
scheme@(guix-user)> (%package-module-path)
$1 =
(("/gnu/store/kds0mq06qpin125gkikwzdm6mjfwjffc-guix-module-union/share/guile/site/3.0"
. "gnu/packages")
"/gnu/store/96pa4rc57zgqf36y2kv8z20p2jvlgypq-pkill9-free-channel-dependency/share/guile/site/3.0"
"/gnu/store/3ilx18ywdm6xk9f5l1mznrn45vcbncsq-pkill9-free/share/guile/site/3.0")
scheme@(guix-user)>
```

But running the following in "test.scm" with `guix repl /tmp/test.scm
doesn't return additional channels:
```
(use-modules (gnu packages))
(display (%package-module-path))
```

```
((/gnu/store/kds0mq06qpin125gkikwzdm6mjfwjffc-guix-module-union/share/guile/site/3.0
. gnu/packages))
```

fold-available-packages uses this to search for packages, which I am
using for a script. As a result, the script doesn't know about packages
from the additional channels.

bug#42688: Running a script with `guix repl` doesn't "see" additional channels using (%package-module-path)

[zimoun]

Dear,

Thank you for the report.  Well, it seems similar to #37399
<https://issues.guix.gnu.org/issue/37399>.


On Mon, 03 Aug 2020 at 04:33, pkill9 <pkill9@runbox.com> wrote:

> fold-available-packages uses this to search for packages, which I am
> using for a script. As a result, the script doesn't know about packages
> from the additional channels.

What happens if you try the “trick” using GUILE_LOAD_PATH?
Or the option ’-L’?

(Well, maybe a solution for the mean time if you do not have too much
channels).


All the best,
simon

bug#42688: Running a script with `guix repl` doesn't "see" additional channels using (%package-module-path)

[Leo Prikler]

Hi Guix,

I've finally figured out, what causes this issue.

Guix repl uses the following code to call scripts:
```
    (unless (null? script)
      ;; Run script
      (save-module-excursion
       (lambda ()
         (set-program-arguments script)
         (set-user-module)
         (load-in-vicinity "." (car script)))))
```

But `guix describe` (which is used to initialize %package-module-path)
has the following:

```
(define current-profile
  (mlambda ()
    "Return the profile (created by 'guix pull') the calling process
lives in,
or #f if this is not applicable."
    (match (command-line)
      ((program . _)
       (and (string-suffix? "/bin/guix" program)
           [...])))))

(define current-profile-entries [...])
(define current-channel-entries [...])
(define package-path-entries [...])
```

Each of these procedures depends on the previous, building up a chain
that fails exactly in the case where we (set-program-arguments [...])
with a script other than the current channel's guix (which is probably
the way you'd want to use `guix repl`).  

There are some ways of resolving this.  One would be to access earlier
versions of "command-line" – it does resolve to a fluid, but that fluid
itself is not exposed to Guile.  Perhaps there might be some FFI magic
to access it.

You could also set up your script to fake being a Guix command by
setting the command line to be (cons*
"$HOME/.config/guix/current/bin/guix" "repl" (command-line)), i.e.
reconstructing the way your script has been invoked.  This would
obviously break if you were to call it with a different Guix, also
you'd have to resolve $HOME instead of writing it like that, but you'd
have access to your channels.

On the other hand, we could patch `guix repl` to initialize %package-
module-path earlier (still leaving `guix describe` broken) or somehow
try to work around that issue in `guix describe`.

Regards,
Leo

bug#42688: Running a script with `guix repl` doesn't "see" additional channels using (%package-module-path)

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 1999 bytes --]

Hi Leo,

Leo Prikler <leo.prikler@edu.uni-graz.at> skribis:

> I've finally figured out, what causes this issue.
>
> Guix repl uses the following code to call scripts:
> ```
>     (unless (null? script)
>       ;; Run script
>       (save-module-excursion
>        (lambda ()
>          (set-program-arguments script)
>          (set-user-module)
>          (load-in-vicinity "." (car script)))))
> ```
>
> But `guix describe` (which is used to initialize %package-module-path)
> has the following:
>
> ```
> (define current-profile
>   (mlambda ()
>     "Return the profile (created by 'guix pull') the calling process
> lives in,
> or #f if this is not applicable."
>     (match (command-line)
>       ((program . _)
>        (and (string-suffix? "/bin/guix" program)
>            [...])))))
>
> (define current-profile-entries [...])
> (define current-channel-entries [...])
> (define package-path-entries [...])
> ```
>
> Each of these procedures depends on the previous, building up a chain
> that fails exactly in the case where we (set-program-arguments [...])
> with a script other than the current channel's guix (which is probably
> the way you'd want to use `guix repl`).  

Good catch!

> There are some ways of resolving this.  One would be to access earlier
> versions of "command-line" – it does resolve to a fluid, but that fluid
> itself is not exposed to Guile.  Perhaps there might be some FFI magic
> to access it.

‘scm_program_arguments_fluid’ is marked as SCM_INTERNAL, so it’s really
inaccessible.

However, perhaps we could save the initial value of (program-arguments)
in (guix ui) and use that in (guix describe)?

> On the other hand, we could patch `guix repl` to initialize %package-
> module-path earlier (still leaving `guix describe` broken) or somehow
> try to work around that issue in `guix describe`.

Initializing (%package-module-path) earlier sounds like a good idea too,
maybe like this:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 743 bytes --]

diff --git a/guix/scripts/repl.scm b/guix/scripts/repl.scm
index 7d4e474e92..b672489ed6 100644
--- a/guix/scripts/repl.scm
+++ b/guix/scripts/repl.scm
@@ -22,6 +22,7 @@
   #:use-module (guix ui)
   #:use-module (guix scripts)
   #:use-module (guix repl)
+  #:autoload   (gnu packages) (%package-module-path)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
   #:use-module (srfi srfi-37)
@@ -173,6 +174,10 @@ call THUNK."
   (with-error-handling
 
     (unless (null? script)
+      ;; Before running SCRIPT, initialize %PACKAGE-MODULE-PATH so that it
+      ;; contains the user's channels (the statement triggers an autoload).
+      (%package-module-path)
+
       ;; Run script
       (save-module-excursion
        (lambda ()

[-- Attachment #3: Type: text/plain, Size: 28 bytes --]


?

Thanks!

Ludo’.

bug#42688: Running a script with `guix repl` doesn't "see" additional channels using (%package-module-path)

[Leo Prikler]

Hi Ludo,
Am Donnerstag, den 17.09.2020, 17:31 +0200 schrieb Ludovic Courtès:
> Hi Leo,
> 
> [...]
> 
> ‘scm_program_arguments_fluid’ is marked as SCM_INTERNAL, so it’s
> really
> inaccessible.
Thought so.

> However, perhaps we could save the initial value of (program-
> arguments)
> in (guix ui) and use that in (guix describe)?
I'd personally put it in (guix describe) and use the same autoload
trick, that you've now used for %package-module-path (or a dedicated
save-...-excursion).  (guix ui) has a heavy closure for (guix describe)
to pull.

> > On the other hand, we could patch `guix repl` to initialize
> > %package-
> > module-path earlier (still leaving `guix describe` broken) or
> > somehow
> > try to work around that issue in `guix describe`.
> 
> Initializing (%package-module-path) earlier sounds like a good idea
> too,
> maybe like this:
> 
> [...] 
> 
I haven't tested that yet (pre-inst-env makes it so Guix doesn't have
any channels anyway), but yeah, something like that would have been my
idea.

Regards,
Leo

bug#42688: Running a script with `guix repl` doesn't "see" additional channels using (%package-module-path)

[Ludovic Courtès]

Hi,

Leo Prikler <leo.prikler@student.tugraz.at> skribis:

> Am Donnerstag, den 17.09.2020, 17:31 +0200 schrieb Ludovic Courtès:
>> Hi Leo,
>> 
>> [...]
>> 
>> ‘scm_program_arguments_fluid’ is marked as SCM_INTERNAL, so it’s
>> really
>> inaccessible.
> Thought so.
>
>> However, perhaps we could save the initial value of (program-
>> arguments)
>> in (guix ui) and use that in (guix describe)?
> I'd personally put it in (guix describe) and use the same autoload
> trick, that you've now used for %package-module-path (or a dedicated
> save-...-excursion).

In general, (guix …) module should not depend on (gnu …) modules, which
rules out this option.

> (guix ui) has a heavy closure for (guix describe) to pull.

Every (guix scripts …) module depends on (guix ui) via the ‘guix’
command.  (Probably something we could improve, but that’s the way it
is.)

Now, I realize my proposal was misguided because (guix describe) should
remain “UI-free” so to speak.  Hmm…

>> > On the other hand, we could patch `guix repl` to initialize
>> > %package-
>> > module-path earlier (still leaving `guix describe` broken) or
>> > somehow
>> > try to work around that issue in `guix describe`.
>> 
>> Initializing (%package-module-path) earlier sounds like a good idea
>> too,
>> maybe like this:
>> 
>> [...] 
>> 
> I haven't tested that yet (pre-inst-env makes it so Guix doesn't have
> any channels anyway), but yeah, something like that would have been my
> idea.

Alright, I’ll give it a spin.

Thank you!

Ludo’.

bug#42688: Running a script with `guix repl` doesn't "see" additional channels using (%package-module-path)

[Leo Prikler]

Hi,
Am Donnerstag, den 17.09.2020, 21:10 +0200 schrieb Ludovic Courtès:
> Hi,
> 
> Leo Prikler <leo.prikler@student.tugraz.at> skribis:
> 
> > Am Donnerstag, den 17.09.2020, 17:31 +0200 schrieb Ludovic Courtès:
> > > Hi Leo,
> > > 
> > > [...]
> > > 
> > > ‘scm_program_arguments_fluid’ is marked as SCM_INTERNAL, so it’s
> > > really
> > > inaccessible.
> > Thought so.
> > 
> > > However, perhaps we could save the initial value of (program-
> > > arguments)
> > > in (guix ui) and use that in (guix describe)?
> > I'd personally put it in (guix describe) and use the same autoload
> > trick, that you've now used for %package-module-path (or a
> > dedicated
> > save-...-excursion).
> 
> In general, (guix …) module should not depend on (gnu …) modules,
> which
> rules out this option.
Sure, but program-arguments are not defined in (gnu …) and it is a
(guix scripts …) that eventually pulls in %package-module-path. 
Therefore defining %guix-initial-program-arguments (or whatever it will
be called in the end) in (guix describe) still seems like an option to
me. 

> > (guix ui) has a heavy closure for (guix describe) to pull.
> 
> Every (guix scripts …) module depends on (guix ui) via the ‘guix’
> command.  (Probably something we could improve, but that’s the way it
> is.)
> 
> Now, I realize my proposal was misguided because (guix describe)
> should
> remain “UI-free” so to speak.  Hmm…
With that however, I am no longer so sure.  The initial program
arguments are part of the UI, but at the same time, that would make it
not UI-free to begin with.  Kinda strengthens the argument, that it
should be made a fluid/parameter/what have you, that gets initialized
with program-arguments at some point.

Regards,
Leo

bug#42688: Running a script with `guix repl` doesn't "see" additional channels using (%package-module-path)

[Ludovic Courtès]

Hello,

Leo Prikler <leo.prikler@student.tugraz.at> skribis:

> With that however, I am no longer so sure.  The initial program
> arguments are part of the UI, but at the same time, that would make it
> not UI-free to begin with.  Kinda strengthens the argument, that it
> should be made a fluid/parameter/what have you, that gets initialized
> with program-arguments at some point.

Alright.  I went with something along these lines in commit
1b179d7876f19f04009a2f9e248ac10711f4c660.

I tested that it works as intended with ‘guix pull --url=$PWD’ and
running a script from there that accesses modules of a secondary
channel.

Thank you!

Ludo’.