Permanently available environments

Permanently available environments

[Konrad Hinsen]

Hi Guix,

I would like to migrate more of my software use to Guix-based
containers, but I haven't yet found a way to handle them that fully
suits my needs. The root issue is the volatility of environments, and I
wonder if I am missing some feature to handle them better. Here is my
current reasoning:

1. Containers can only be generated using "guix shell" or the older
   "guix environment". There is no way to generate a container based
   on a profile. Correct me if I am wrong!

2. I have been using a plain "guix shell" for a while, but my
   environments break too often after a "guix pull" to continue this
   way. Most of my containerized environments contain no
   security-critical software, so I'd be happy not to update them
   very often (or not at all). That would be trivial with profiles,
   but... point 1.

3. There's the –root option to "guix shell" to protect my environment
   from the garbage collector. But there is no way to say "use the
   environment pointed to by that root, no matter when and how it was
   created". I have to specify the packages with respect to the current
   Guix version (last "pull"). So if I want to protect against updates,
   I have to use "guix time-machine" with "guix shell". Which is fine
   in theory.

4. In practice, I often work with a bad or non-existing network
   connection, so I must be sure to have all my packages in the store.
   And if I use "time-machine", I must also keep the required Guix
   version locally available. But there is no option for that in
   "time-machine". The Guix versions it downloads are garbage-collected
   after a while. So I can find myself in the situation of having all
   the packages for my environment in the store, but unable to access
   it without a network connection, because "time-machine" first needs
   to fetch an old Guix version again.

Did I overlook anything?

Cheers,
  Konrad

Re: Permanently available environments

[Ricardo Wurmus]

Konrad Hinsen <konrad.hinsen@fastmail.net> writes:

> Hi Guix,
>
> I would like to migrate more of my software use to Guix-based
> containers, but I haven't yet found a way to handle them that fully
> suits my needs. The root issue is the volatility of environments, and I
> wonder if I am missing some feature to handle them better. Here is my
> current reasoning:
>
> 1. Containers can only be generated using "guix shell" or the older
>    "guix environment". There is no way to generate a container based
>    on a profile. Correct me if I am wrong!

Not sure if that’s what you mean, but you can use “guix shell” and “guix
environment” with “-p”:

    -p, --profile=PATH     create environment from profile at PATH

> 2. I have been using a plain "guix shell" for a while, but my
>    environments break too often after a "guix pull" to continue this
>    way. Most of my containerized environments contain no
>    security-critical software, so I'd be happy not to update them
>    very often (or not at all). That would be trivial with profiles,
>    but... point 1.

You can also use --root.

> 3. There's the –root option to "guix shell" to protect my environment
>    from the garbage collector. But there is no way to say "use the
>    environment pointed to by that root, no matter when and how it was
>    created".

Oh.

You can.  With “-p”.

I have to specify the packages with respect to the current

> 4. In practice, I often work with a bad or non-existing network
>    connection, so I must be sure to have all my packages in the store.
>    And if I use "time-machine", I must also keep the required Guix
>    version locally available. But there is no option for that in
>    "time-machine". The Guix versions it downloads are garbage-collected
>    after a while. So I can find myself in the situation of having all
>    the packages for my environment in the store, but unable to access
>    it without a network connection, because "time-machine" first needs
>    to fetch an old Guix version again.

I’m not using time-machine often enough to know what to suggest here.

-- 
Ricardo

Re: Permanently available environments

[Guillaume Le Vaillant]

[-- Attachment #1: Type: text/plain, Size: 2276 bytes --]

Konrad Hinsen <konrad.hinsen@fastmail.net> skribis:

> Hi Guix,
>
> I would like to migrate more of my software use to Guix-based
> containers, but I haven't yet found a way to handle them that fully
> suits my needs. The root issue is the volatility of environments, and I
> wonder if I am missing some feature to handle them better. Here is my
> current reasoning:
>
> 1. Containers can only be generated using "guix shell" or the older
>    "guix environment". There is no way to generate a container based
>    on a profile. Correct me if I am wrong!
>
> 2. I have been using a plain "guix shell" for a while, but my
>    environments break too often after a "guix pull" to continue this
>    way. Most of my containerized environments contain no
>    security-critical software, so I'd be happy not to update them
>    very often (or not at all). That would be trivial with profiles,
>    but... point 1.
>
> 3. There's the –root option to "guix shell" to protect my environment
>    from the garbage collector. But there is no way to say "use the
>    environment pointed to by that root, no matter when and how it was
>    created". I have to specify the packages with respect to the current
>    Guix version (last "pull"). So if I want to protect against updates,
>    I have to use "guix time-machine" with "guix shell". Which is fine
>    in theory.
>
> 4. In practice, I often work with a bad or non-existing network
>    connection, so I must be sure to have all my packages in the store.
>    And if I use "time-machine", I must also keep the required Guix
>    version locally available. But there is no option for that in
>    "time-machine". The Guix versions it downloads are garbage-collected
>    after a while. So I can find myself in the situation of having all
>    the packages for my environment in the store, but unable to access
>    it without a network connection, because "time-machine" first needs
>    to fetch an old Guix version again.
>
> Did I overlook anything?
>
> Cheers,
>   Konrad

Hi,

Wouldn't creating a profile with something like

  guix package -m path/to/manifest.scm -p path/to/profile

and then loading it with

  guix shell -q -C -p path/to/profile

do what you want?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

Re: Permanently available environments

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 1025 bytes --]

On 2022-01-28, Konrad Hinsen wrote:
> 4. In practice, I often work with a bad or non-existing network
>    connection, so I must be sure to have all my packages in the store.
>    And if I use "time-machine", I must also keep the required Guix
>    version locally available. But there is no option for that in
>    "time-machine". The Guix versions it downloads are garbage-collected
>    after a while. So I can find myself in the situation of having all
>    the packages for my environment in the store, but unable to access
>    it without a network connection, because "time-machine" first needs
>    to fetch an old Guix version again.

As I understand it, time-machine is basically a wrapper around "guix
pull" and the command you want to execute ... so you could use:

  guix pull --profile=path/to/a/profile --commit=...

And then call the guix from that profile:

  path/to/a/profile/bin/guix shell --container --mainifest=...


Maybe there is some more elegant way, but that should work...


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Permanently available environments

[Konrad Hinsen]

Konrad Hinsen <konrad.hinsen@fastmail.net> writes:

> 1. Containers can only be generated using "guix shell" or the older
>    "guix environment". There is no way to generate a container based
>    on a profile. Correct me if I am wrong!

Thanks to Ricardo and Guillaume, who pointed out the –profile option to
"guix shell", which does exactly what I need. I missed it because it is
not mentioned in the manual. I just submitted a patch.

> 4. In practice, I often work with a bad or non-existing network
>    connection, so I must be sure to have all my packages in the store.
>    And if I use "time-machine", I must also keep the required Guix
>    version locally available. But there is no option for that in
>    "time-machine". The Guix versions it downloads are garbage-collected
>    after a while. So I can find myself in the situation of having all
>    the packages for my environment in the store, but unable to access
>    it without a network connection, because "time-machine" first needs
>    to fetch an old Guix version again.

Thanks to Vagrant for pointing out that "guix pull" has a –profile
option as well (which I wasn't aware of). This can indeed serve as a
workaround, although it looks a bit clumsy.

Cheers,
  Konrad.

Re: Permanently available environments

[Ludovic Courtès]

Hi Konrad,

Time has passed…

Konrad Hinsen <konrad.hinsen@fastmail.net> skribis:

> Konrad Hinsen <konrad.hinsen@fastmail.net> writes:
>
>> 1. Containers can only be generated using "guix shell" or the older
>>    "guix environment". There is no way to generate a container based
>>    on a profile. Correct me if I am wrong!
>
> Thanks to Ricardo and Guillaume, who pointed out the –profile option to
> "guix shell", which does exactly what I need. I missed it because it is
> not mentioned in the manual. I just submitted a patch.

Another option is to rely on the ‘time-machine’ and ‘shell’ LRU caches.
That is, you can run:

  guix time-machine -C channels.scm -- shell -m manifest.scm

Both tools cache their result, with a GC root (similar to what you would
do with ‘--root’), and reuse it (similar to ‘--profile’).  These two GC
roots may be removed if you’ve not touched them for some time, at which
point ‘guix gc’ can potentially remove these profiles.

Would it work for you?

I’m more comfortable recommending this than asking people to manually
manage cached profiles and GC roots.

Thanks,
Ludo’.

Re: Permanently available environments

[Konrad Hinsen]

Hi Ludo,

> Another option is to rely on the ‘time-machine’ and ‘shell’ LRU caches.
> That is, you can run:

That's what I did in the past, until the first time I hit an expired
cache. In conformance with Murphy's law, I was in a train, with a bad
Internet-by-phone connection, and Guix started downloading texlive.

So...

> Would it work for you?

No. I tend to keep project-specific environments unchanged for long
times, longer than Guix keeps its caches.

Cheers,
  Konrad