Appropriate way to use Guix API from a Guile program

Appropriate way to use Guix API from a Guile program

[sirgazil]

Hi,

I use Guix's GNU system. 

I had written a Guile script that called Guix procedures to manipulate profiles. I put it in a Guix package in a custom channel and installed it. It used to work, but now it fails with an error I can also reproduce when trying to do something similar from a Guile REPL. For instance:

★★★★★★★★★★★★★★★★★★★★
$ mkdir profile-x
$ guile
GNU Guile 3.0.2
[...]
scheme@(guile-user)> (use-modules (guix scripts package))
scheme@(guile-user)> (guix-package "-m" "manifest.scm" "-p" "profile-x/profile-x")
error: glibc-bootstrap-system-2.2.5.patch: patch not found
★★★★★★★★★★★★★★★★★★★★

This error is familiar. I got it in a slightly different context before when running the guix command directly from a terminal (https://lists.gnu.org/archive/html/help-guix/2020-05/msg00045.html).

In the context of Guile scripting, how do I know which Guix is being imported? 

Also, I didn't include guix as an input in the Guix package of the script (though it worked). Thinking that it could make a difference, I tried adding guix as an input, pulled the new package definition, upgraded the package, ran the script, but got the same error.


---
https://sirgazil.bitbucket.io/

Re: Appropriate way to use Guix API from a Guile program

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1731 bytes --]

On Wed, May 20, 2020 at 04:38:02PM -0500, sirgazil wrote:
> Hi,
> 
> I use Guix's GNU system. 
> 
> I had written a Guile script that called Guix procedures to manipulate profiles. I put it in a Guix package in a custom channel and installed it. It used to work, but now it fails with an error I can also reproduce when trying to do something similar from a Guile REPL. For instance:
> 
> ★★★★★★★★★★★★★★★★★★★★
> $ mkdir profile-x
> $ guile
> GNU Guile 3.0.2
> [...]
> scheme@(guile-user)> (use-modules (guix scripts package))
> scheme@(guile-user)> (guix-package "-m" "manifest.scm" "-p" "profile-x/profile-x")
> error: glibc-bootstrap-system-2.2.5.patch: patch not found
> ★★★★★★★★★★★★★★★★★★★★
> 
> This error is familiar. I got it in a slightly different context before when running the guix command directly from a terminal (https://lists.gnu.org/archive/html/help-guix/2020-05/msg00045.html).
> 
> In the context of Guile scripting, how do I know which Guix is being imported? 
> 
> Also, I didn't include guix as an input in the Guix package of the script (though it worked). Thinking that it could make a difference, I tried adding guix as an input, pulled the new package definition, upgraded the package, ran the script, but got the same error.

if you have a local checkout of Guix then I'd suggest using
'./pre-inst-env guile'. There's something about the core-updates merge
that caused some problems.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Appropriate way to use Guix API from a Guile program

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1401 bytes --]

sirgazil <sirgazil@zoho.com> writes:

> Hi,
>
> I use Guix's GNU system. 
>
> I had written a Guile script that called Guix procedures to manipulate profiles. I put it in a Guix package in a custom channel and installed it. It used to work, but now it fails with an error I can also reproduce when trying to do something similar from a Guile REPL. For instance:
>
> ★★★★★★★★★★★★★★★★★★★★
> $ mkdir profile-x
> $ guile
> GNU Guile 3.0.2
> [...]
> scheme@(guile-user)> (use-modules (guix scripts package))
> scheme@(guile-user)> (guix-package "-m" "manifest.scm" "-p" "profile-x/profile-x")
> error: glibc-bootstrap-system-2.2.5.patch: patch not found
> ★★★★★★★★★★★★★★★★★★★★

There are a couple of things going on here.  Calling the Guix APIs from
Guile will use the 'system-installed' Guix, unless you have fiddled with
GUILE_LOAD_PATH.

You can use 'guix repl' to use the _current_ Guix instead (the one
obtained with 'guix pull'), where this problem has been fixed.

The system-installed Guix in turn fails because of a bug that was fixed
in 179e6c524ae4957e6ace83f72a5651767f786ca4, and ultimately
977f478090fd96e73463efb67c56fb32f28828bd which updates the 'guix'
package to include the fix.  If you reconfigure on a recent Guix, the
missing patch error should go away.

Hope this helps!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Appropriate way to use Guix API from a Guile program

[sirgazil]

 ---- On Thu, 21 May 2020 07:44:58 -0500 Marius Bakke <mbakke@fastmail.com> wrote ----
 > sirgazil <sirgazil@zoho.com> writes:
 > 
 > > Hi,
 > >
 > > I use Guix's GNU system. 
 > >
 > > I had written a Guile script that called Guix procedures to manipulate profiles. I put it in a Guix package in a custom channel and installed it. It used to work, but now it fails with an error I can also reproduce when trying to do something similar from a Guile REPL. For instance:
 > >
 > > ★★★★★★★★★★★★★★★★★★★★
 > > $ mkdir profile-x
 > > $ guile
 > > GNU Guile 3.0.2
 > > [...]
 > > scheme@(guile-user)> (use-modules (guix scripts package))
 > > scheme@(guile-user)> (guix-package "-m" "manifest.scm" "-p" "profile-x/profile-x")
 > > error: glibc-bootstrap-system-2.2.5.patch: patch not found
 > > ★★★★★★★★★★★★★★★★★★★★
 > 
 > There are a couple of things going on here.  Calling the Guix APIs from
 > Guile will use the 'system-installed' Guix, unless you have fiddled with
 > GUILE_LOAD_PATH.

Ok. I haven't touched GUILE_LOAD_PATH.


 > You can use 'guix repl' to use the _current_ Guix instead (the one
 > obtained with 'guix pull'), where this problem has been fixed.
 > 
 > The system-installed Guix in turn fails because of a bug that was fixed
 > in 179e6c524ae4957e6ace83f72a5651767f786ca4, and ultimately
 > 977f478090fd96e73463efb67c56fb32f28828bd which updates the 'guix'
 > package to include the fix.  If you reconfigure on a recent Guix, the
 > missing patch error should go away.

Oh, ok. I'll reconfigure then.

Thanks Marius and Efraim.

Re: Appropriate way to use Guix API from a Guile program

[sirgazil]

The error is gone after reconfiguring, but I still have a question about using Guix API from a Guile script.

 ---- On Thu, 21 May 2020 07:44:58 -0500 Marius Bakke <mbakke@fastmail.com> wrote ----
 > sirgazil <sirgazil@zoho.com> writes:
 > 
 > > Hi,
 > >
 > > I use Guix's GNU system. 
 > >
 > > I had written a Guile script that called Guix procedures to manipulate profiles. I put it in a Guix package in a custom channel and installed it. It used to work, but now it fails with an error I can also reproduce when trying to do something similar from a Guile REPL. For instance:
 > >
 > > ★★★★★★★★★★★★★★★★★★★★
 > > $ mkdir profile-x
 > > $ guile
 > > GNU Guile 3.0.2
 > > [...]
 > > scheme@(guile-user)> (use-modules (guix scripts package))
 > > scheme@(guile-user)> (guix-package "-m" "manifest.scm" "-p" "profile-x/profile-x")
 > > error: glibc-bootstrap-system-2.2.5.patch: patch not found
 > > ★★★★★★★★★★★★★★★★★★★★
 > 
 > There are a couple of things going on here.  Calling the Guix APIs from
 > Guile will use the 'system-installed' Guix, unless you have fiddled with
 > GUILE_LOAD_PATH.

Hmm, and is there any way for a Guile program installed as a Guix package to use the modules of the Guix obtained by the user with "guix pull" and also being able to take advantage of user-defined channels? Or is this just not possible at all?

Because I just tried creating a profile using my script and a manifest that includes one package from a custom channel of mine, and, as expected, I get an error:

★★★★★★★★★★★★★★★★★★★★
error: guile-glab: unknown package
error: failed to load 'requirements/guix.scm':
gnu/packages.scm:523:4: In procedure specification->package+output:
Throw to key `quit' with args `(1)'.
★★★★★★★★★★★★★★★★★★★★

Re: Appropriate way to use Guix API from a Guile program

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 412 bytes --]

sirgazil <sirgazil@zoho.com> writes:

> Hmm, and is there any way for a Guile program installed as a Guix package to use the modules of the Guix obtained by the user with "guix pull" and also being able to take advantage of user-defined channels? Or is this just not possible at all?

You need to add $HOME/.config/guix/current/share/guile/site/3.0 on
GUILE_LOAD_PATH to access channel code from Guile programs.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Appropriate way to use Guix API from a Guile program

[sirgazil]

 ---- On Thu, 21 May 2020 15:52:59 -0500 Marius Bakke <mbakke@fastmail.com> wrote ----
 > sirgazil <sirgazil@zoho.com> writes:
 > 
 > > Hmm, and is there any way for a Guile program installed as a Guix package to use the modules of the Guix obtained by the user with "guix pull" and also being able to take advantage of user-defined channels? Or is this just not possible at all?
 > 
 > You need to add $HOME/.config/guix/current/share/guile/site/3.0 on
 > GUILE_LOAD_PATH to access channel code from Guile programs.


Thanks again, Marius.

Re: Appropriate way to use Guix API from a Guile program

[zimoun]

Dear sirgazil,

On Thu, 21 May 2020 at 22:53, Marius Bakke <mbakke@fastmail.com> wrote:
> sirgazil <sirgazil@zoho.com> writes:

> > Hmm, and is there any way for a Guile program installed as a Guix package to use the modules of the Guix obtained by the user with "guix pull" and also being able to take advantage of user-defined channels? Or is this just not possible at all?
>
> You need to add $HOME/.config/guix/current/share/guile/site/3.0 on
> GUILE_LOAD_PATH to access channel code from Guile programs.

If you feel in an experimental mood, you can try the Konrad's patch
[1].  It sets up the PATH accordingly and hides all the internals.
Well, "guix repl" launches a REPL with the current Guix modules
available and the patch extends that to scripts: "guix repl foo.scm".
Note that '--load-path' is supported so you can extend by your own
modules too.  Last, command-line arguments passed to the scripts are
not supported (yet).

Well, if it is a script without command-line argument, then "guix repl
foo.scm" is the thing that you would want, I guess.
If it is a script with command-line argument, then extend
"GUILE_LOAD_PATH" by '~/.config/guix/current/share/guile/site/3.0' and
run "guile foo.scm arg1 arg2" is the thing that you would want, I
guess.

Be careful to not mix the modules from the package named 'guix' --
modules installed with "guix install guix" -- and the modules of the
current Guix -- modules under
~/.config/guix/current/share/guile/site/3.0.  I did the mistake once.
:-)

[1] http://issues.guix.gnu.org/41253


Hope that helps,
simon

Re: Appropriate way to use Guix API from a Guile program

[Konrad Hinsen]

zimoun <zimon.toutoune@gmail.com> writes:

> If you feel in an experimental mood, you can try the Konrad's patch
> [1].  It sets up the PATH accordingly and hides all the internals.

In particular it hides implementation details that are likely to
change. A look at the path in

  ~/.config/guix/current/share/guile/site/3.0

should be sufficient to suggest that not very long ago, this was

  ~/.config/guix/current/share/guile/site/2.2

and scripts have no way to figure out which one of the two is
appropriate on a given system.

> Well, if it is a script without command-line argument, then "guix repl
> foo.scm" is the thing that you would want, I guess.
> If it is a script with command-line argument, then extend
> "GUILE_LOAD_PATH" by '~/.config/guix/current/share/guile/site/3.0' and
> run "guile foo.scm arg1 arg2" is the thing that you would want, I
> guess.

Thanks for the reminder that my "guix repl" patch needs more work - it
should accept command line arguments for the script. Something like

   guix repl foo.scm –- [ARGS]

should be easily doable, and fits in with existing conventions.

Cheers,
  Konrad

Re: Appropriate way to use Guix API from a Guile program

[zimoun]

Hi Konrad,

On Fri, 22 May 2020 at 09:43, Konrad Hinsen <konrad.hinsen@fastmail.net> wrote:

> Thanks for the reminder that my "guix repl" patch needs more work - it
> should accept command line arguments for the script. Something like
>
>    guix repl foo.scm –- [ARGS]
>
> should be easily doable, and fits in with existing conventions.

It could be really cool! :-)
Aside the use cases like './show-dependencies.scm' from your blog
entry [1], it could be also a good way to easily experiment with new
CLI.

[1] https://guix.gnu.org/blog/2020/reproducible-computations-with-guix/

All the best,
simon

Re: Appropriate way to use Guix API from a Guile program

[Konrad Hinsen]

Hi everyone,

zimoun <zimon.toutoune@gmail.com> writes:

>> Thanks for the reminder that my "guix repl" patch needs more work - it
>> should accept command line arguments for the script. Something like
>>
>>    guix repl foo.scm –- [ARGS]
>>
>> should be easily doable, and fits in with existing conventions.
>
> It could be really cool! :-)

Unfortunately, it doesn't look as straightforward as I thought.
A standard Guile script retrieves command line arguments using
(command-line). "guix repl" should ideally manipulate the return
value of (command-line) to make things just work, but I haven't
found a way to do this.

The other solution is to define a command-line interface specifically
for scripts run using "guix repl", which looks like a lot of effort
for a less-than-optimal result.


Alex Sassmannshausen <alex.sassmannshausen@gmail.com> writes:

> I can't help but think that it would be really useful to have a way to
> inject these load paths into the general user environment, in a way
> similar to `guix package --search-paths', so that the currently pulled
> guix can be just another guile library to be used on the system like
> any other.

I don't see any other way to do this than to shell out to "guix", parse
the return value, and modify load-path. Certainly doable, but I am not
sure it's worth the effort.

The problem with Guix is that are so many of it. Most users have a
single Guix version that they update using "guix pull", but others also
use development branches compiled locally and invoked via the preᵢnstₑnv
script. In the end it's the user who picks the right Guix for the job at
the command line. Any Guile script referring to Guix should offer the
same flexibility.

sirgazil <sirgazil@zoho.com> writes:

> And if you wanted to run the script without typing "guix repl", what would be
> the first line of the script. This?
>
> #!~/.config/guix/current/bin/guix repl
> !#

I'd go for 

> #!/bin/env guix repl
> !#

or something like that. I suspect it requires some more incantation
magic to work.

Cheers,
  Konrad

Re: Appropriate way to use Guix API from a Guile program

[Konrad Hinsen]

Konrad Hinsen <konrad.hinsen@fastmail.net> writes:

> Unfortunately, it doesn't look as straightforward as I thought.
> A standard Guile script retrieves command line arguments using
> (command-line). "guix repl" should ideally manipulate the return
> value of (command-line) to make things just work, but I haven't
> found a way to do this.

Reading the Guile manual helps, of course, once you know where to look.
It's (set-program-arguments ...), which starts to make sense once you
figure out that (program-arguments) is a synonym for (command-line).
I'll see if I can do something with that.

Cheers,
  Konrad

Re: Appropriate way to use Guix API from a Guile program

[Alex Sassmannshausen]

[-- Attachment #1: Type: text/plain, Size: 1123 bytes --]

Hello,

On Fri, 2020-05-22 at 09:43 +0200, Konrad Hinsen wrote:
> zimoun <zimon.toutoune@gmail.com> writes:
> 
> > If you feel in an experimental mood, you can try the Konrad's patch
> > [1].  It sets up the PATH accordingly and hides all the internals.
> 
> In particular it hides implementation details that are likely to
> change. A look at the path in
> 
>   ~/.config/guix/current/share/guile/site/3.0
> 
> should be sufficient to suggest that not very long ago, this was
> 
>   ~/.config/guix/current/share/guile/site/2.2
> 
> and scripts have no way to figure out which one of the two is
> appropriate on a given system.

I agree with all of this, and no doubt the ability to start Guile
scripts through Guix as ad-hoc ways to run scripts is useful.

I can't help but think that it would be really useful to have a way to
inject these load paths into the general user environment, in a way
similar to `guix package --search-paths', so that the currently pulled
guix can be just another guile library to be used on the system like
any other.

Would there be downsides to this?

Alex


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 273 bytes --]

Re: Appropriate way to use Guix API from a Guile program

[sirgazil]

 ---- On Fri, 22 May 2020 02:43:21 -0500 Konrad Hinsen <konrad.hinsen@fastmail.net> wrote ----
 > zimoun <zimon.toutoune@gmail.com> writes:
 > 
 > > If you feel in an experimental mood, you can try the Konrad's patch
 > > [1].  It sets up the PATH accordingly and hides all the internals.
 > 
 > In particular it hides implementation details that are likely to
 > change. A look at the path in
 > 
 >   ~/.config/guix/current/share/guile/site/3.0
 > 
 > should be sufficient to suggest that not very long ago, this was
 > 
 >   ~/.config/guix/current/share/guile/site/2.2
 > 
 > and scripts have no way to figure out which one of the two is
 > appropriate on a given system.
 > 
 > > Well, if it is a script without command-line argument, then "guix repl
 > > foo.scm" is the thing that you would want, I guess.
 > > If it is a script with command-line argument, then extend
 > > "GUILE_LOAD_PATH" by '~/.config/guix/current/share/guile/site/3.0' and
 > > run "guile foo.scm arg1 arg2" is the thing that you would want, I
 > > guess.
 > 
 > Thanks for the reminder that my "guix repl" patch needs more work - it
 > should accept command line arguments for the script. Something like
 > 
 >    guix repl foo.scm –- [ARGS]
 > 
 > should be easily doable, and fits in with existing conventions.

And if you wanted to run the script without typing "guix repl", what would be the first line of the script. This?

#!~/.config/guix/current/bin/guix repl
!#

Re: Appropriate way to use Guix API from a Guile program

[sirgazil]

 ---- On Thu, 21 May 2020 18:02:29 -0500 zimoun <zimon.toutoune@gmail.com> wrote ----
 > Dear sirgazil,
 > 
 > On Thu, 21 May 2020 at 22:53, Marius Bakke <mbakke@fastmail.com> wrote:
 > > sirgazil <sirgazil@zoho.com> writes:
 > 
 > > > Hmm, and is there any way for a Guile program installed as a Guix package to use the modules of the Guix obtained by the user with "guix pull" and also being able to take advantage of user-defined channels? Or is this just not possible at all?
 > >
 > > You need to add $HOME/.config/guix/current/share/guile/site/3.0 on
 > > GUILE_LOAD_PATH to access channel code from Guile programs.
 > 
 > If you feel in an experimental mood, you can try the Konrad's patch
 > [1].  It sets up the PATH accordingly and hides all the internals.
 > Well, "guix repl" launches a REPL with the current Guix modules
 > available and the patch extends that to scripts: "guix repl foo.scm".
 > Note that '--load-path' is supported so you can extend by your own
 > modules too.  Last, command-line arguments passed to the scripts are
 > not supported (yet).
 > 
 > Well, if it is a script without command-line argument, then "guix repl
 > foo.scm" is the thing that you would want, I guess.
 > If it is a script with command-line argument, then extend
 > "GUILE_LOAD_PATH" by '~/.config/guix/current/share/guile/site/3.0' and
 > run "guile foo.scm arg1 arg2" is the thing that you would want, I
 > guess.

Yes, the script accepts arguments.

 
 > Be careful to not mix the modules from the package named 'guix' --
 > modules installed with "guix install guix" -- and the modules of the
 > current Guix -- modules under
 > ~/.config/guix/current/share/guile/site/3.0.  I did the mistake once.
 > :-)

Been there, done that :)

Thanks, simon.