Adding a subcommand "load-profile"

Adding a subcommand "load-profile"

[Roel Janssen]

Dear Guix,

Years ago we implemented GNU Guix on the high-performance computing cluster in
Utrecht.  One of the things we added was a wrapper around the "guix" command
(called "guixr") to enable communication between the guix-daemon (on one node),
and the client-side "guix" command.  (We actually copied the great "guixr"
script from Ricardo at the time.)

Lots of improvements have been made for the HPC use-case that the need for the
"guixr" wrapper script is no longer needed.   Except for one thing.

We added a subcommand in the "guixr" script called "load-profile".  It allows a
user to do the following:
--------------------------------------------------------------------------------
$ guixr package -i ... -p /my/profile
$ guixr load-profile /my/profile
[env]$
  # ... A new shell is spawned here.
  #     Inside this shell only the environment variables in
  #     /my/profile/etc/profile are set ...
[env]$ exit
  # Return to the normal shell state 
--------------------------------------------------------------------------------

The code of "guixr" is available at [1].

I sometimes wish I had this command available in "guix" itself.  So I'd like to
implement the "load-profile" subcommand in Scheme, so that it can be part of
Guix.

Would there be any interest from others to have this as well? And also, the
shell implementation heavy relies on Bash.  What other shells should I attempt
to implement?

If there is interest in having this as a "load-profile" subcommand, I will post
an initial implementation to the mailing list ASAP.

Thanks all!

Kind regards,
Roel Janssen

[1] 
https://github.com/UMCUGenetics/guix-additions/blob/master/umcu/packages/guix.scm#L191-L339

Re: Adding a subcommand "load-profile"

[zimoun]

Dear Roel,

On Tue, 28 Apr 2020 at 17:43, Roel Janssen <roel@gnu.org> wrote:

> --------------------------------------------------------------------------------
> $ guixr package -i ... -p /my/profile
> $ guixr load-profile /my/profile
> [env]$
>   # ... A new shell is spawned here.
>   #     Inside this shell only the environment variables in
>   #     /my/profile/etc/profile are set ...
> [env]$ exit
>   # Return to the normal shell state
> --------------------------------------------------------------------------------

[...]

> Would there be any interest from others to have this as well? And also, the
> shell implementation heavy relies on Bash.  What other shells should I attempt
> to implement?

It would be cool!
However, if I remember correctly the previous discussion on such
topic, the issue was to respect the user's shell ($SHELL). This
argument is mitigated by the fact that "guix environment" already uses
Bash to spawn the new shell, if I understand correctly.

Well, I find annoying to not able to "go in" and "go out" in one
profile and I prefer having a Bash specific implementation than
nothing.
So, thank you for the initiative. :-)


> If there is interest in having this as a "load-profile" subcommand, I will post
> an initial implementation to the mailing list ASAP.

Instead of another subcommand, I would suggest to add an option to
"guix environment".
Then an another further step should maybe combine "--load-profile" and
"--ad-hoc" in order to create a temporary "augmented" profile.


All the best,
simon

Re: Adding a subcommand "load-profile"

[sirgazil]

 ---- On Tue, 28 Apr 2020 22:42:53 +0000 Roel Janssen <roel@gnu.org> wrote ----
 > Dear Guix,
 > 
 > Years ago we implemented GNU Guix on the high-performance computing cluster in
 > Utrecht.  One of the things we added was a wrapper around the "guix" command
 > (called "guixr") to enable communication between the guix-daemon (on one node),
 > and the client-side "guix" command.  (We actually copied the great "guixr"
 > script from Ricardo at the time.)
 > 
 > Lots of improvements have been made for the HPC use-case that the need for the
 > "guixr" wrapper script is no longer needed.   Except for one thing.
 > 
 > We added a subcommand in the "guixr" script called "load-profile".  It allows a
 > user to do the following:
 > --------------------------------------------------------------------------------
 > $ guixr package -i ... -p /my/profile
 > $ guixr load-profile /my/profile
 > [env]$
 >   # ... A new shell is spawned here.
 >   #     Inside this shell only the environment variables in
 >   #     /my/profile/etc/profile are set ...
 > [env]$ exit
 >   # Return to the normal shell state 
 > --------------------------------------------------------------------------------
 > 
 > The code of "guixr" is available at [1].
 > 
 > I sometimes wish I had this command available in "guix" itself.  So I'd like to
 > implement the "load-profile" subcommand in Scheme, so that it can be part of
 > Guix.
 > 
 > Would there be any interest from others to have this as well? And also, the
 > shell implementation heavy relies on Bash.  What other shells should I attempt
 > to implement?

Personally, I think managing Guix profiles is currently difficult, and I would like to have commands like this one to make it easier.

Actually, I've been exploring using profiles to create development environments for my own projects (using information in the Cookbook), but it turned out to be too much manual work, so I've been prototyping a Guile-based CLI for my personal needs (https://gitlab.com/sirgazil/guix-entorno). 

I would like these kinds of things to be possible with Guix commands instead, though.

Re: Adding a subcommand "load-profile"

[Roel Janssen]

Dear Simon,

Thank you for your ideas!

On Tue, 2020-04-28 at 18:54 +0200, zimoun wrote:
> 
> [...]
> 
> > Would there be any interest from others to have this as well? And also, the
> > shell implementation heavy relies on Bash.  What other shells should I
> > attempt
> > to implement?
> 
> It would be cool!
> However, if I remember correctly the previous discussion on such
> topic, the issue was to respect the user's shell ($SHELL). This
> argument is mitigated by the fact that "guix environment" already uses
> Bash to spawn the new shell, if I understand correctly.

I think we can respect the user's shell for the implementation of "load-
profile".  We could use Guile's "getenv" and "setenv" to prepare the shell
environment in Guile, and then spawn the $SHELL in such a way that it doesn't
load the user's default configuration.

> Well, I find annoying to not able to "go in" and "go out" in one
> profile and I prefer having a Bash specific implementation than
> nothing.
> So, thank you for the initiative. :-)
> 
> 
> > If there is interest in having this as a "load-profile" subcommand, I will
> > post
> > an initial implementation to the mailing list ASAP.
> 
> Instead of another subcommand, I would suggest to add an option to
> "guix environment".
> Then an another further step should maybe combine "--load-profile" and
> "--ad-hoc" in order to create a temporary "augmented" profile.

I would strongly prefer to keep it backwards-compatible for our local HPC users.
Also, the "environment" command generates a new profile, whereas the proposed
"load-profile" merely applies the environment variables of an existing profile
to a newly spawned shell.  I think the way they work differs enough to warrant a
separate subcommand.

Kind regards,
Roel Janssen

Re: Adding a subcommand "load-profile"

[zimoun]

Dear Roel,

On Wed, 29 Apr 2020 at 14:46, Roel Janssen <roel@gnu.org> wrote:

> > > If there is interest in having this as a "load-profile" subcommand, I will
> > > post
> > > an initial implementation to the mailing list ASAP.
> >
> > Instead of another subcommand, I would suggest to add an option to
> > "guix environment".
> > Then an another further step should maybe combine "--load-profile" and
> > "--ad-hoc" in order to create a temporary "augmented" profile.
>
> I would strongly prefer to keep it backwards-compatible for our local HPC users.
> Also, the "environment" command generates a new profile, whereas the proposed
> "load-profile" merely applies the environment variables of an existing profile
> to a newly spawned shell.  I think the way they work differs enough to warrant a
> separate subcommand.

I understand that you would like to keep backward-compatibility with
your current tools and ease the switch for your users. But you could
do so by replacing all the shell code in the 'guixr load-profile' by
"guix environment --load-profile". :-)

On one hand, I am not convinced by adding another subcommand as I said
above. :-)
On the other hand, this new subcommand loading a profile could be the
starting point of the "guix develop" subcommand that was mentioned
some time ago.

https://lists.gnu.org/archive/html/guix-devel/2019-10/msg00613.html
https://lists.gnu.org/archive/html/guix-devel/2019-11/msg00019.html


All the best,
simon

Re: Adding a subcommand "load-profile"

[Roel Janssen]

On Wed, 2020-04-29 at 15:48 +0200, zimoun wrote:
> Dear Roel,
> 
> On Wed, 29 Apr 2020 at 14:46, Roel Janssen <roel@gnu.org> wrote:
> 
> > > > If there is interest in having this as a "load-profile" subcommand, I
> > > > will
> > > > post
> > > > an initial implementation to the mailing list ASAP.
> > > 
> > > Instead of another subcommand, I would suggest to add an option to
> > > "guix environment".
> > > Then an another further step should maybe combine "--load-profile" and
> > > "--ad-hoc" in order to create a temporary "augmented" profile.
> > 
> > I would strongly prefer to keep it backwards-compatible for our local HPC
> > users.
> > Also, the "environment" command generates a new profile, whereas the
> > proposed
> > "load-profile" merely applies the environment variables of an existing
> > profile
> > to a newly spawned shell.  I think the way they work differs enough to
> > warrant a
> > separate subcommand.
> 
> I understand that you would like to keep backward-compatibility with
> your current tools and ease the switch for your users. But you could
> do so by replacing all the shell code in the 'guixr load-profile' by
> "guix environment --load-profile". :-)

Well, I want to get rid of all the shell code.  Not replace it with less shell
code. :)

And cluttering everyone's environment on our cluster with a bash function that
figures out whether to emit "guix environment --load-profile" or another guix
command seems far from optimal too.

Kind regards,
Roel Janssen

Re: Adding a subcommand "load-profile"

[Ludovic Courtès]

Hi,

sirgazil <sirgazil@zoho.com> skribis:

>  ---- On Tue, 28 Apr 2020 22:42:53 +0000 Roel Janssen <roel@gnu.org> wrote ----

[...]

>  > We added a subcommand in the "guixr" script called "load-profile".  It allows a
>  > user to do the following:
>  > --------------------------------------------------------------------------------
>  > $ guixr package -i ... -p /my/profile
>  > $ guixr load-profile /my/profile
>  > [env]$
>  >   # ... A new shell is spawned here.
>  >   #     Inside this shell only the environment variables in
>  >   #     /my/profile/etc/profile are set ...
>  > [env]$ exit
>  >   # Return to the normal shell state 
>  > --------------------------------------------------------------------------------

[...]

> Personally, I think managing Guix profiles is currently difficult, and I would like to have commands like this one to make it easier.

I’m interested in your feedback then.

To me, there are already several ways to do that, notably:

  1. guix environment (but it’s slower)

  2. sh
     . /my/profile/etc/profile.sh
     exit

  3. sh
     eval `guix package -p /my/profile --search-paths`
     exit

Options #2 and #3 are more verbose, and less obvious to someone not
familiar with the shell.

Still, such commands are targeting shell users, which makes me wonder
how far we should go.  It’s an honest question: I’m torn between the
will to provide a simple CLI, and the need to avoid proliferation of
commands all more or less equivalent.

Thoughts?

Ludo’.

Re: Adding a subcommand "load-profile"

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 2113 bytes --]

On 2020-05-03, Ludovic Courtès wrote:
> sirgazil <sirgazil@zoho.com> skribis:
>>  ---- On Tue, 28 Apr 2020 22:42:53 +0000 Roel Janssen <roel@gnu.org> wrote ----
>>  > We added a subcommand in the "guixr" script called "load-profile".  It allows a
>>  > user to do the following:
>>  > --------------------------------------------------------------------------------
>>  > $ guixr package -i ... -p /my/profile
>>  > $ guixr load-profile /my/profile
>>  > [env]$
>>  >   # ... A new shell is spawned here.
>>  >   #     Inside this shell only the environment variables in
>>  >   #     /my/profile/etc/profile are set ...
>>  > [env]$ exit
>>  >   # Return to the normal shell state 
>>  > --------------------------------------------------------------------------------
>
> [...]
>
>> Personally, I think managing Guix profiles is currently difficult, and I would like to have commands like this one to make it easier.
>
> I’m interested in your feedback then.
>
> To me, there are already several ways to do that, notably:
>
>   1. guix environment (but it’s slower)
>
>   2. sh
>      . /my/profile/etc/profile.sh
>      exit
>
>   3. sh
>      eval `guix package -p /my/profile --search-paths`
>      exit
>
> Options #2 and #3 are more verbose, and less obvious to someone not
> familiar with the shell.

The only way i understand how to trivially get a precise environment is
"guix environment --pure --ad-hoc package1, package2, ... packageN"

Don't options #2 and #3 also include everything from the default profile
and system profile?

I like the ability to maintain separate profiles and run "guix upgrade
-p my/profile" on each profile, but I haven't figured out how to get
something equivalent to "guix environment --pure --profile my/profile".

I find this useful when updating diffoscope or reprotest packages, which
tend to have a lot of suggested optional packages that aren't really
hard dependencies.

This looks "load-profile" subcommand seems to attempting to meet that
use-case, if I'm reading it correctly?


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Adding a subcommand "load-profile"

[zimoun]

Hi Ludo,

On Sun, 3 May 2020 at 22:31, Ludovic Courtès <ludo@gnu.org> wrote:

>   1. guix environment (but it’s slower)
>
>   2. sh
>      . /my/profile/etc/profile.sh
>      exit

Personally, time to time I use (thanks Pjotr):

   env -i $(which bash) --login --noprofile --norc
   . /my/profile/etc/profile
   exit

because 'sh' is roughly bare and typing /my/long/path/to/profile
without completion, history, move backward, etc. is not practical.

Note: it is written in the Cookbook. Bottom of [1].
[1] https://guix.gnu.org/cookbook/en/html_node/Basic-setup-with-manifests.html#Basic-setup-with-manifests


>   3. sh
>      eval `guix package -p /my/profile --search-paths`
>      exit
>
> Options #2 and #3 are more verbose, and less obvious to someone not
> familiar with the shell.
>
> Still, such commands are targeting shell users, which makes me wonder
> how far we should go.  It’s an honest question: I’m torn between the
> will to provide a simple CLI, and the need to avoid proliferation of
> commands all more or less equivalent.

Well, part of the subcommand "environment" could be done with "guix
package -p" and then #3 but it is more more (more!) convenient to have
a full subcommand: easy to use day-to-day and easy to explain, etc..

However, I am not in favour for adding another subcommands for only
the use case of loading a profile.  (Note that the point of Roel is
legitimate: minimize the burden of their users already typing "guixr
load-profile".)
From my point of view, instead of a subcommand, it should be an
option: guix environment --load-profile or in short -P.

And if another subcommand is added, it should be the more ambitious
"profile manager", e.g., "guix profile", as it has been partly
discussed, for examples in threads starting there:

https://lists.gnu.org/archive/html/guix-devel/2019-10/msg00565.html
https://lists.gnu.org/archive/html/guix-devel/2019-12/msg00358.html


Last, I prefer another subcommand (load-profile) than nothing.
Because it is requested since a long time.



All the best,
simon

Re: Adding a subcommand "load-profile"

[zimoun]

Dear Vagrant,

On Sun, 3 May 2020 at 22:57, Vagrant Cascadian <vagrant@debian.org> wrote:

> I like the ability to maintain separate profiles and run "guix upgrade
> -p my/profile" on each profile, but I haven't figured out how to get
> something equivalent to "guix environment --pure --profile my/profile".

   env -i $(which bash) --login --noprofile --norc
   . /my/profile/etc/profile

Does it fit your needs?


All the best,
simon

Re: Adding a subcommand "load-profile"

[Ludovic Courtès]

Hi,

Vagrant Cascadian <vagrant@debian.org> skribis:

>> I’m interested in your feedback then.
>>
>> To me, there are already several ways to do that, notably:
>>
>>   1. guix environment (but it’s slower)
>>
>>   2. sh
>>      . /my/profile/etc/profile.sh
>>      exit
>>
>>   3. sh
>>      eval `guix package -p /my/profile --search-paths`
>>      exit
>>
>> Options #2 and #3 are more verbose, and less obvious to someone not
>> familiar with the shell.
>
> The only way i understand how to trivially get a precise environment is
> "guix environment --pure --ad-hoc package1, package2, ... packageN"
>
> Don't options #2 and #3 also include everything from the default profile
> and system profile?

Option #2 yes, option #3 no: there’s ‘--search-paths=exact’ (the
default), ‘--search-paths=prefix’, and ‘--search-paths=suffix’.

Ludo’.