bug#47846: Feature Request: Add ability to disable having cache or generations

bug#47846: Feature Request: Add ability to disable having cache or generations

[bo0od]

Hi There,

Current situation with the guix distro upgrade is:(as i understand)

A) User Packages: whenever there is an upgrade to package A version 1 to 
new Version lets call it A version 2 , So the process is ADD A2 → SWITCH 
to A2 → Cache A1 and so on.

B) System Packages: Same process but it will be saved through generations

This causes unpleasant actions to some users:

- Bloating the disk size
- Having old unnecessary files/packages
- Questionable security of the saved old versions. As it depend if they 
have access to suid or not (i didnt investigate this, but if they have 
then thats big problem but this is not the ticket to discuss it)

I know someone would jump in and say but roll back is great feature and 
its useful and....i know that but like i said might be not suiting all 
users (specially with limited space).

Current manual solution is to delete this extra mess using 2 commands:

guix gc -d 1s && sudo guix system delete-generation

This should be run whenever there is no space left, Or to get rid of the 
old stuff

My suggestion is to have the ability to make Guix automatically just 
having the latest up to date packages without extra consumed storage (no 
cache no generation no nothing more than having the latest packages 
available in the distro).

So the process is ADD A2 → SWITCH to A2 → Delete A1 , Or Download A2 → 
Replace over A1 and so on.


ThX!

bug#47846: Feature Request: Add ability to disable having cache or generations

[Leo Famulari]

On Sat, Apr 17, 2021 at 06:29:56PM +0000, bo0od wrote:
> - Questionable security of the saved old versions. As it depend if they have
> access to suid or not (i didnt investigate this, but if they have then thats
> big problem but this is not the ticket to discuss it)

They do not.

bug#47846: Feature Request: Add ability to disable having cache or generations

[Leo Prikler]

Hi,
Am Samstag, den 17.04.2021, 18:29 +0000 schrieb bo0od:
> Hi There,
> 
> Current situation with the guix distro upgrade is:(as i understand)
> 
> A) User Packages: whenever there is an upgrade to package A version 1
> to 
> new Version lets call it A version 2 , So the process is ADD A2 →
> SWITCH 
> to A2 → Cache A1 and so on.
> 
> B) System Packages: Same process but it will be saved through
> generations
There is no active caching going on.  Besides potentially building
software, the process of "upgrading" one generation of your Guix
profile or system is simply the act of letting a symbolic link point
elsewhere.  Nothing more, nothing less.  Each generation is itself a
"root" in GC terms from the moment it is built.

> This causes unpleasant actions to some users:
> 
> - Bloating the disk size
That's debatable.  Now, yes, it is no secret, that Guix uses more disk
space than your traditional software, as keeps copies of your old data
around, but on a desktop with 500MB storage, you can keep several
months of that around if you want to.  Things might be a bit different
on smartphones and embedded systems, which may want to GC more often,
but it's not like minimal setups are impossible.
> - Having old unnecessary files/packages
Which is bad how?
> - Questionable security of the saved old versions. As it depend if
> they 
> have access to suid or not (i didnt investigate this, but if they
> have 
> then thats big problem but this is not the ticket to discuss it)
You would have to explicitly run those old, insecure versions, for them
to be an attack surface, which I'd hazard you won't unless you're still
actively using them anyway.  Note that for the case, that the mere
existence of those is a threat, you must assume your attacker to have
arbitrary shell code execution already.

> I know someone would jump in and say but roll back is great feature
> and 
> its useful and....i know that but like i said might be not suiting
> all 
> users (specially with limited space).
Because it is.  There are things larger than package generations.  My
current profile weighs 8.5GB according to du, much of which can be
shared between generations.  A typical anime episode encoded with x264
at 1080p weighs 1GB or more.  So one season of your favourite show is
literally more data than all of your software.

> Current manual solution is to delete this extra mess using 2
> commands:
> 
> guix gc -d 1s && sudo guix system delete-generation
> 
> This should be run whenever there is no space left, Or to get rid of
> the 
> old stuff
Just FYI deleting all that so often only puts unnecessary stress on
your disk, because native inputs will have to be redownloaded and
you're not even freeing up that much space.

> My suggestion is to have the ability to make Guix automatically just 
> having the latest up to date packages without extra consumed storage
> (no 
> cache no generation no nothing more than having the latest packages 
> available in the distro).
That's not very functional.  Again, you're putting more stress on your
hardware by actively asking it to remove stuff.

Regards,
Leo

bug#47846: Feature Request: Add ability to disable having cache or generations

[bo0od]

 > There is no active caching going on.

Not sure what do you mean by this.

 > but on a desktop with 500MB storage, you can keep several
months of that around if you want to.

Im using 20GB+9GB swap, its nightmare you cant just upgrade without each 
and everytime delete cache. So no, Sorry The statement isnt accurate 
about 500MB. (my personal experience, not someone telling me nor 
guessing things)

 > Which is bad how?

Imagine i upgraded to FF version 79, but as well i have 78.9.2,78.9.0... 
These are wasted software we are not hunting deer and keeping trophies, 
  Dont get me wrong roll back is great/usable but not for 
everyone/everytime case.

 > You would have to explicitly run those old, insecure versions, for 
them to be an attack surface[...]

True, Already answered by Leo Famulari.

 >Just FYI deleting all that so often only puts unnecessary stress on
your disk, because native inputs will have to be redownloaded and
you're not even freeing up that much space.

There is no way i can upgrade without using them.

 > That's not very functional.  Again, you're putting more stress on your
hardware by actively asking it to remove stuff.

If you mean by the method of removing, Thats not my job to know what is 
the best method to be used, There are main distros like 
debian,fedora..etc devs can look at them and see how they can 
adopt/merge some methods.

Leo Prikler:
> Hi,
> Am Samstag, den 17.04.2021, 18:29 +0000 schrieb bo0od:
>> Hi There,
>>
>> Current situation with the guix distro upgrade is:(as i understand)
>>
>> A) User Packages: whenever there is an upgrade to package A version 1
>> to
>> new Version lets call it A version 2 , So the process is ADD A2 →
>> SWITCH
>> to A2 → Cache A1 and so on.
>>
>> B) System Packages: Same process but it will be saved through
>> generations
> There is no active caching going on.  Besides potentially building
> software, the process of "upgrading" one generation of your Guix
> profile or system is simply the act of letting a symbolic link point
> elsewhere.  Nothing more, nothing less.  Each generation is itself a
> "root" in GC terms from the moment it is built.
> 
>> This causes unpleasant actions to some users:
>>
>> - Bloating the disk size
> That's debatable.  Now, yes, it is no secret, that Guix uses more disk
> space than your traditional software, as keeps copies of your old data
> around, but on a desktop with 500MB storage, you can keep several
> months of that around if you want to.  Things might be a bit different
> on smartphones and embedded systems, which may want to GC more often,
> but it's not like minimal setups are impossible.
>> - Having old unnecessary files/packages
> Which is bad how?
>> - Questionable security of the saved old versions. As it depend if
>> they
>> have access to suid or not (i didnt investigate this, but if they
>> have
>> then thats big problem but this is not the ticket to discuss it)
> You would have to explicitly run those old, insecure versions, for them
> to be an attack surface, which I'd hazard you won't unless you're still
> actively using them anyway.  Note that for the case, that the mere
> existence of those is a threat, you must assume your attacker to have
> arbitrary shell code execution already.
> 
>> I know someone would jump in and say but roll back is great feature
>> and
>> its useful and....i know that but like i said might be not suiting
>> all
>> users (specially with limited space).
> Because it is.  There are things larger than package generations.  My
> current profile weighs 8.5GB according to du, much of which can be
> shared between generations.  A typical anime episode encoded with x264
> at 1080p weighs 1GB or more.  So one season of your favourite show is
> literally more data than all of your software.
> 
>> Current manual solution is to delete this extra mess using 2
>> commands:
>>
>> guix gc -d 1s && sudo guix system delete-generation
>>
>> This should be run whenever there is no space left, Or to get rid of
>> the
>> old stuff
> Just FYI deleting all that so often only puts unnecessary stress on
> your disk, because native inputs will have to be redownloaded and
> you're not even freeing up that much space.
> 
>> My suggestion is to have the ability to make Guix automatically just
>> having the latest up to date packages without extra consumed storage
>> (no
>> cache no generation no nothing more than having the latest packages
>> available in the distro).
> That's not very functional.  Again, you're putting more stress on your
> hardware by actively asking it to remove stuff.
> 
> Regards,
> Leo
> 

bug#47846: Feature Request: Add ability to disable having cache or generations

[Leo Prikler]

Hi,

Am Sonntag, den 18.04.2021, 14:40 +0000 schrieb bo0od:
>  > There is no active caching going on.
> 
> Not sure what do you mean by this.
Exactly what I said.  There is a philosophical difference between a
store, that keeps items as long as there's a referrer and a cache,
which keeps some items on a heuristic basis.

>  > but on a desktop with 500MB storage, you can keep several
> months of that around if you want to.
> 
> Im using 20GB+9GB swap, its nightmare you cant just upgrade without
> each 
> and everytime delete cache. So no, Sorry The statement isnt accurate 
> about 500MB. (my personal experience, not someone telling me nor 
> guessing things)
My bad, I meant to type 500GB (a fairly common disk size), but it turns
out my other laptop survives quite fine on 250.  Fair enough, it's not
32GB (common in phones), but then again, you'd run normally very
different packages on embedded systems.

And yeah, this is also personal experience, not someone telling me or
guessing, I merely made a typo.

> 
>  > Which is bad how?
> 
> Imagine i upgraded to FF version 79, but as well i have
> 78.9.2,78.9.0... 
> These are wasted software we are not hunting deer and keeping
> trophies, 
>   Dont get me wrong roll back is great/usable but not for 
> everyone/everytime case.
You do know, that Guix also has environments, that can be garbage
collected, as soon as the process exits, right?  If you use Icecat so
rarely, that upgrading it along with the rest of your profile makes no
sense, you could use those.  Not to mention w.r.t. security, using a
containerized icecat is probably a better idea.

>  >Just FYI deleting all that so often only puts unnecessary stress on
> your disk, because native inputs will have to be redownloaded and
> you're not even freeing up that much space.
> 
> There is no way i can upgrade without using them.
There are several ways of optimizing for profile size, one of which is
to not run huge browsers like icecat.  I have no idea what kind of
system you're trying to fit into 20GB , but a hard idea thinking it's
the right kind.

By the way, continuing from before, my /run/current-system, which
consists of the desktop template plus some extras, seems to weigh just
about 2GB, which would fit 5 times into 20GB while still letting me use
half of the disk.

>  > That's not very functional.  Again, you're putting more stress on
> your
> hardware by actively asking it to remove stuff.
> 
> If you mean by the method of removing, Thats not my job to know what
> is 
> the best method to be used, There are main distros like 
> debian,fedora..etc devs can look at them and see how they can 
> adopt/merge some methods.
What kind of advanced removal strategies are you talking about?
Traditional distros do not face this issue, because they're more or
less just dumping files into already existing locations, and don't
really worry whether something already exists there.  (Well, there are
varying degrees of worrying, but they are all incomplete.)  Binary
distros have it even easier, because they don't even attempt to build
from source (another issue if you're running a resource-constrained
device).

These so-called "removal methods" of traditional distros are
antithetical to Guix' design.  Asking us to behave just like a "main
distro", when we have made a clear decision not to, is not going to
please either side of the discussion.

Regards,
Leo

bug#47846: Feature Request: Add ability to disable having cache or generations

[bo0od]

 > My bad, I meant to type 500GB (a fairly common disk size), but it turns
 > out my other laptop survives quite fine on 250.  Fair enough, it's not
 > 32GB (common in phones), but then again, you'd run normally very
 > different packages on embedded systems.

yeah 100+ GB thats too big, not always having this space is easy or 
available.

 > There are several ways of optimizing for profile size, one of which is
 > to not run huge browsers like icecat.  I have no idea what kind of
 > system you're trying to fit into 20GB , but a hard idea thinking it's
 > the right kind.

I have debian,fedora,kali,ubuntu,trisquel/triskel,arch... all with only 
20GB space and working for testing purposes as im mostly working as 
software tester.

 > What kind of advanced removal strategies are you talking about?

I didnt suggested how its done in my ticket, I gave the issue and 
feature request as a solution to it but how to do it the best way i 
leave this to the devs to decide not me.

If out of ideas and nothing is available look at other distributions and 
see how its done and what can be taken from them and merge into guix to 
adopt this feature.

Leo Prikler:
> Hi,
> 
> Am Sonntag, den 18.04.2021, 14:40 +0000 schrieb bo0od:
>>   > There is no active caching going on.
>>
>> Not sure what do you mean by this.
> Exactly what I said.  There is a philosophical difference between a
> store, that keeps items as long as there's a referrer and a cache,
> which keeps some items on a heuristic basis.
> 
>>   > but on a desktop with 500MB storage, you can keep several
>> months of that around if you want to.
>>
>> Im using 20GB+9GB swap, its nightmare you cant just upgrade without
>> each
>> and everytime delete cache. So no, Sorry The statement isnt accurate
>> about 500MB. (my personal experience, not someone telling me nor
>> guessing things)
> My bad, I meant to type 500GB (a fairly common disk size), but it turns
> out my other laptop survives quite fine on 250.  Fair enough, it's not
> 32GB (common in phones), but then again, you'd run normally very
> different packages on embedded systems.
> 
> And yeah, this is also personal experience, not someone telling me or
> guessing, I merely made a typo.
> 
>>
>>   > Which is bad how?
>>
>> Imagine i upgraded to FF version 79, but as well i have
>> 78.9.2,78.9.0...
>> These are wasted software we are not hunting deer and keeping
>> trophies,
>>    Dont get me wrong roll back is great/usable but not for
>> everyone/everytime case.
> You do know, that Guix also has environments, that can be garbage
> collected, as soon as the process exits, right?  If you use Icecat so
> rarely, that upgrading it along with the rest of your profile makes no
> sense, you could use those.  Not to mention w.r.t. security, using a
> containerized icecat is probably a better idea.
> 
>>   >Just FYI deleting all that so often only puts unnecessary stress on
>> your disk, because native inputs will have to be redownloaded and
>> you're not even freeing up that much space.
>>
>> There is no way i can upgrade without using them.
> There are several ways of optimizing for profile size, one of which is
> to not run huge browsers like icecat.  I have no idea what kind of
> system you're trying to fit into 20GB , but a hard idea thinking it's
> the right kind.
> 
> By the way, continuing from before, my /run/current-system, which
> consists of the desktop template plus some extras, seems to weigh just
> about 2GB, which would fit 5 times into 20GB while still letting me use
> half of the disk.
> 
>>   > That's not very functional.  Again, you're putting more stress on
>> your
>> hardware by actively asking it to remove stuff.
>>
>> If you mean by the method of removing, Thats not my job to know what
>> is
>> the best method to be used, There are main distros like
>> debian,fedora..etc devs can look at them and see how they can
>> adopt/merge some methods.
> What kind of advanced removal strategies are you talking about?
> Traditional distros do not face this issue, because they're more or
> less just dumping files into already existing locations, and don't
> really worry whether something already exists there.  (Well, there are
> varying degrees of worrying, but they are all incomplete.)  Binary
> distros have it even easier, because they don't even attempt to build
> from source (another issue if you're running a resource-constrained
> device).
> 
> These so-called "removal methods" of traditional distros are
> antithetical to Guix' design.  Asking us to behave just like a "main
> distro", when we have made a clear decision not to, is not going to
> please either side of the discussion.
> 
> Regards,
> Leo
> 

bug#47846: Feature Request: Add ability to disable having cache or generations

[Leo Prikler]

Hi,

Am Sonntag, den 18.04.2021, 18:45 +0000 schrieb bo0od:
>  > My bad, I meant to type 500GB (a fairly common disk size), but it
> turns
>  > out my other laptop survives quite fine on 250.  Fair enough, it's
> not
>  > 32GB (common in phones), but then again, you'd run normally very
>  > different packages on embedded systems.
> 
> yeah 100+ GB thats too big, not always having this space is easy or 
> available.
It is common enough for desktop PCs and laptops, which is the use case
I'm talking about here.  If you're hosting a server, chances also are,
that you have that much, if not more space available.  For cases, in
which you have significantly less memory available, there is a bare-
bones template.

>  > There are several ways of optimizing for profile size, one of
> which is
>  > to not run huge browsers like icecat.  I have no idea what kind of
>  > system you're trying to fit into 20GB , but a hard idea thinking
> it's
>  > the right kind.
> 
> I have debian,fedora,kali,ubuntu,trisquel/triskel,arch... all with
> only 
> 20GB space and working for testing purposes as im mostly working as 
> software tester.
I'm fairly certain you should be able to get test environments, that
fit this size via `guix environment', but if that's your plan, you
shouldn't do much else with the space you have.  Might I ask if you're
the kind to keep a separate /home?

>  > What kind of advanced removal strategies are you talking about?
> 
> I didnt suggested how its done in my ticket, I gave the issue and 
> feature request as a solution to it but how to do it the best way i 
> leave this to the devs to decide not me.
"Please remove all my previous stuff whenever I upgrade a package" is a
rather specific feature request in my opinion.  I don't think there is
much room for bikeshedding different implementations of it, though of
course, Debian, Arch, Gentoo etc. are all different distributions, that
produce it as a side-effect of what they're actually trying to do.

> If out of ideas and nothing is available look at other distributions
> and 
> see how its done and what can be taken from them and merge into guix
> to 
> adopt this feature.
How did other distributions "adopt this feature" in your opinion?

Regards,
Leo

bug#47846: Feature Request: Add ability to disable having cache or generations

[bo0od]

 > It is common enough for desktop PCs and laptops, which is the use case
 > I'm talking about here.

The entire feature im requesting is not for common things but for 
specific use cases.

 > I'm fairly certain you should be able to get test environments, that
 > fit this size via `guix environment', but if that's your plan, you
 > shouldn't do much else with the space you have.

Im not sure im following how this solve the issue im describing.

 > Might I ask if you're
 > the kind to keep a separate /home?

No, I use everything default and try finding enhancements,bugs after 
that. (bugs easier to be reproduced by others when you have strange 
behaviors).

 > though of
 > course, Debian, Arch, Gentoo etc. are all different distributions, that
 > produce it as a side-effect of what they're actually trying to do.

"as a side-effect" not sure what do you mean.

 > How did other distributions "adopt this feature" in your opinion?

Not sure how is this question correct as other distributions doesnt have 
rollback/generations..etc options in order to adopt anything from others 
except NixOS which i didnt yet asked them about this (I hope i wont be 
forced to use Nix instead of Guix).

Good projects to look at which are actually rolling distros:

Voidlinux(xbps)<- Good starting point/features, Arch(pacman).
Worth mentioning: Kali(apt), Gentoo(portage).

And if Nix going to provide such a similar feature then that would be 
interested to check on as well (maybe easier way to see it as well 
implemented within guix).

Leo Prikler:
> Hi,
> 
> Am Sonntag, den 18.04.2021, 18:45 +0000 schrieb bo0od:
>>   > My bad, I meant to type 500GB (a fairly common disk size), but it
>> turns
>>   > out my other laptop survives quite fine on 250.  Fair enough, it's
>> not
>>   > 32GB (common in phones), but then again, you'd run normally very
>>   > different packages on embedded systems.
>>
>> yeah 100+ GB thats too big, not always having this space is easy or
>> available.
> It is common enough for desktop PCs and laptops, which is the use case
> I'm talking about here.  If you're hosting a server, chances also are,
> that you have that much, if not more space available.  For cases, in
> which you have significantly less memory available, there is a bare-
> bones template.
> 
>>   > There are several ways of optimizing for profile size, one of
>> which is
>>   > to not run huge browsers like icecat.  I have no idea what kind of
>>   > system you're trying to fit into 20GB , but a hard idea thinking
>> it's
>>   > the right kind.
>>
>> I have debian,fedora,kali,ubuntu,trisquel/triskel,arch... all with
>> only
>> 20GB space and working for testing purposes as im mostly working as
>> software tester.
> I'm fairly certain you should be able to get test environments, that
> fit this size via `guix environment', but if that's your plan, you
> shouldn't do much else with the space you have.  Might I ask if you're
> the kind to keep a separate /home?
> 
>>   > What kind of advanced removal strategies are you talking about?
>>
>> I didnt suggested how its done in my ticket, I gave the issue and
>> feature request as a solution to it but how to do it the best way i
>> leave this to the devs to decide not me.
> "Please remove all my previous stuff whenever I upgrade a package" is a
> rather specific feature request in my opinion.  I don't think there is
> much room for bikeshedding different implementations of it, though of
> course, Debian, Arch, Gentoo etc. are all different distributions, that
> produce it as a side-effect of what they're actually trying to do.
> 
>> If out of ideas and nothing is available look at other distributions
>> and
>> see how its done and what can be taken from them and merge into guix
>> to
>> adopt this feature.
> How did other distributions "adopt this feature" in your opinion?
> 
> Regards,
> Leo
> 

bug#47846: Feature Request: Add ability to disable having cache or generations

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 5945 bytes --]

bo0od schreef op za 17-04-2021 om 18:29 [+0000]:
> Hi There,
> 
> Current situation with the guix distro upgrade is:(as i understand)
> 
> A) User Packages: whenever there is an upgrade to package A version 1 to 
> new Version lets call it A version 2 , So the process is ADD A2 → SWITCH 
> to A2 → Cache A1 and so on.

There isn't really any caching, it's more like garbage collection like in
Guile, other Schemes & lisps, Java, ..., where old objects (package versions
& profiles that aren't referred anywhere anymore) are deleted.

Is that informative for you?

When the user upgrades A from version A1 to version A2, creating a new version
of the profile P2:

  * P1: original profile, referring to the binaries of A1
    (and other packages, which I'll ignore here)
  * The user asks guix to upgrade from A1 to A2.  So guix first builds A2.
  * Guix creates a profile P2, referring to the binaries of A2
  * P2 becomes the current profile.
  * P1 is kept around in case the user isn't satisfied, or is feeling nastolgic
    or something

> B) System Packages: Same process but it will be saved through generations

In case of user packages, old generations are also saved.
Try "guix package --list-generations".  I have 128 generations of bloat.
Lemme update ("guix package -u"), soon I'll have 129 generations of bloat.

So the bloat is even worse, and your point is even more compelling!

> This causes unpleasant actions to some users:
> 
> - Bloating the disk size

About 200 GiB or so in my case, though admittedly that's partially because
I never run "guix gc" or that command for deleting old generations

> - Having old unnecessary files/packages

Well, having ... They are just sitting there under /gnu/store. A user won't
accidentally see them or something.  Aside from
taking in disk space (see ‘Bloating the disk size’), this seems harmless.

> - Questionable security of the saved old versions.

As long as you don't run the old versions, you should be fine.
If you actually run the old versions, then you'll get the old packages
with old security bugs.  If they can connect to the network, best disconnect
first.

Only run old (and therefore possibly with publicly-known and unfixed security
issues) issues on trusted data!  Use case I have in mind:

  /.../old-profile-version/bin/tome4  (when you've disabled Internet access,
    for trying an old version of tome4 (a game) to see what has changes since).

>  As it depend if they 
> have access to suid or not (i didnt investigate this, but if they have 
> then thats big problem but this is not the ticket to discuss it)

"guix package -i" never creates setuid/setgid binaries.  The only setuid/setgid
binaries that guix creates are in /run/setuid-programs.  These are setuid/setgid
_copies_ of what's requested in the _current_ (or the one at boot, I forgot)
operating-system declaration.

> I know someone would jump in and say but roll back is great feature and 
> its useful and....i know that but like i said might be not suiting all 
> users (specially with limited space).

Yes, but let's at least keep the last few generations around.

E.g., I actually _almost_ never use the rollback mechanism.  The only
time was when I messed up the operating-system declaration, so I had to boot
into the previous system generation (is that the term?).

Of course, then there's a choice to make for _how many_ generations to keep
around ...

> Current manual solution is to delete this extra mess using 2 commands:
> 
> guix gc -d 1s && sudo guix system delete-generation

You should run them in the opposite order.
Also, "guix package --delete-generations" should be run for each user.

> This should be run whenever there is no space left,

Tricky ... reportedly, many software does not handle out-of-disk-space errors
well.  Also, letting "guix gc" and "guix package --delete-generations" run
takes some time.  So this would have to be run when there's only 10% disk
space left or something like that.

> Or to get rid of the old stuff

When you've the space, I recommend keeping the ‘old stuff’ around.
You'd never know whether you'll need it later!  In case you'll need it later,
keeping the ‘old stuff’ around saves on Internet traffic (saving some time)
(lessening the load on the substitute servers -> less network and disk I/O and
CPU usage --> less monetary costs, less environmental cost).

> My suggestion is to have the ability to make Guix automatically just 
> having the latest up to date packages

There's an unattended-upgrade-service for the system profile (if that's the
result of operating-system).  Maybe we can have something similar for user
profiles.

> without extra consumed storage (no cache no generation no nothing more than
> having the latest packages available in the distro).
> 
> So the process is ADD A2 → SWITCH to A2 → Delete A1 , Or Download A2 → 
> Replace over A1 and so on.

This in-place replacement you seem to be suggesting, is rather counter one of
the primary goals of Guix (and Nix, for that matter) --- functional package
management.

Automatically deleting the previous profile and trying to delete the previous
version whenever possible could be possible, but I'm not sure it's worth it.

You could try to implement it yourself, and try your modified guix out for
a while, and report whether it seems to work well, of course!

However, I believe the following would be easier in the short term
(and *very* likely to be accepted upstream):

Implement a graphical application (maybe using the guile-gnome bindings,
or as a web app run on localhost) that has a few buttons for collecting
garbage and deleting old profiles.  Guix could use some graphical stuff.

Of course, what you decide to hack on, what approaches you'll take, etc.
is up to you!

Greetings,
Maxime

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

bug#47846: Feature Request: Add ability to disable having cache or generations

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 460 bytes --]

Maxime Devos schreef op za 17-04-2021 om 22:07 [+0200]:
> bo0od schreef op za 17-04-2021 om 18:29 [+0000]:
> > 
> > [...]
> > - Bloating the disk size
> 
> About 200 GiB or so in my case, though admittedly that's partially because
> I never run "guix gc" or that command for deleting old generations

Correction -- it seems I have 129GiB in my home directory. (du --total $HOME)
I thought it was at most 40 GiB or so.  Replace 200GiB with 100GiB.


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

bug#47846: Feature Request: Add ability to disable having cache or generations

[bo0od]

 > There isn't really any caching

Im calling the saved old versions of software cache, So if there is 
better term to use good.

 >    * P1 is kept around in case the user isn't satisfied, or is 
feeling nastolgic
 >      or something

Yeah this is what im suggesting to have ability to disable this behavior 
as an optional choice for the user (not by default).

 > In case of user packages, old generations are also saved.

Yeah no more "old" anything, Just latest fresh only.

 > About 200 GiB or so in my case, though admittedly that's partially 
because
 > I never run "guix gc" or that command for deleting old generations

I have VM total is 29GB it goes as 20GB for guix system and 9GB swap = 
nightmare of space (other distros this is more than enough specially 
main/known one debian,fedora,trisquel/ubuntu...etc).

 > Well, having ... They are just sitting there under /gnu/store. A user 
won't
 > accidentally see them or something.  Aside from
 > taking in disk space (see ‘Bloating the disk size’), this seems harmless.

Yeah not suitable for low space usages specially like mobile phones or 
limited storage like VMs/VPS. So this is more of annoying thing to have 
thus suggesting to have ability to disable it.

 > As long as you don't run the old versions, you should be fine.
 > If you actually run the old versions, then you'll get the old packages
 > with old security bugs.  If they can connect to the network, best 
disconnect first.

Hmm not really always effective as guix system is as well targeting 
servers so as (might be in the future) can be targeting mobile phones 
both of them hard to keep internet off. But i got my answer to the 
question from leo famulari which is these packages doesnt have access to 
suid and thats the important part.

 > Yes, but let's at least keep the last few generations around.

Yeah this is optional for each user case, The default is that guix 
keeping it anyway but if someone want to disable it he should be able to 
do so and use guix just for getting latest software available.

 > So this would have to be run when there's only 10% disk
space left or something like that.

actually i get only 0.something left when upgrading guix on 20GB, So 
without running the commands just no upgrades.

 > When you've the space, I recommend keeping the ‘old stuff’ around.
 > You'd never know whether you'll need it later!  In case you'll need 
it later[..]

Thats true but not always you are able to have more space easily, 
Sometimes you are limited with payment if you are working on remote 
VPS/VM or using devices which are hard/expensive changing their hard 
like mobiles/tablets.

This issue doesnt talk about my own case scenario but as any other user 
which might have similar circumstances and guix without this suggested 
feature wont help these users.

 > There's an unattended-upgrade-service for the system profile (if 
that's the
 > result of operating-system).  Maybe we can have something similar for 
user
 > profiles.

Anything which can overcome the problem of having limited disk space.

 > This in-place replacement you seem to be suggesting, is rather 
counter one of
 > the primary goals of Guix (and Nix, for that matter) --- functional 
package
 > management.
 >
 > Automatically deleting the previous profile and trying to delete the 
previous
 > version whenever possible could be possible, but I'm not sure it's 
worth it.

Any method not a problem since this feature going to be optional side 
feature anyway.

Create the suitable way of doing it and i will test and report back if 
there is any problems.

 > Of course, what you decide to hack on, what approaches you'll take, etc.
 > is up to you!

So when i open feature request ticket i should answer myself and do it?

No, I want this from upstream to be implemented as a feature its not 
about me only but any scenario which has the same problem because guix 
doesnt give solutions for limited space (and 20GB is not small storage 
for GNU/Linux distro)



Maxime Devos:
> bo0od schreef op za 17-04-2021 om 18:29 [+0000]:
>> Hi There,
>>
>> Current situation with the guix distro upgrade is:(as i understand)
>>
>> A) User Packages: whenever there is an upgrade to package A version 1 to
>> new Version lets call it A version 2 , So the process is ADD A2 → SWITCH
>> to A2 → Cache A1 and so on.
> 
> There isn't really any caching, it's more like garbage collection like in
> Guile, other Schemes & lisps, Java, ..., where old objects (package versions
> & profiles that aren't referred anywhere anymore) are deleted.
> 
> Is that informative for you?
> 
> When the user upgrades A from version A1 to version A2, creating a new version
> of the profile P2:
> 
>    * P1: original profile, referring to the binaries of A1
>      (and other packages, which I'll ignore here)
>    * The user asks guix to upgrade from A1 to A2.  So guix first builds A2.
>    * Guix creates a profile P2, referring to the binaries of A2
>    * P2 becomes the current profile.
>    * P1 is kept around in case the user isn't satisfied, or is feeling nastolgic
>      or something
> 
>> B) System Packages: Same process but it will be saved through generations
> 
> In case of user packages, old generations are also saved.
> Try "guix package --list-generations".  I have 128 generations of bloat.
> Lemme update ("guix package -u"), soon I'll have 129 generations of bloat.
> 
> So the bloat is even worse, and your point is even more compelling!
> 
>> This causes unpleasant actions to some users:
>>
>> - Bloating the disk size
> 
> About 200 GiB or so in my case, though admittedly that's partially because
> I never run "guix gc" or that command for deleting old generations
> 
>> - Having old unnecessary files/packages
> 
> Well, having ... They are just sitting there under /gnu/store. A user won't
> accidentally see them or something.  Aside from
> taking in disk space (see ‘Bloating the disk size’), this seems harmless.
> 
>> - Questionable security of the saved old versions.
> 
> As long as you don't run the old versions, you should be fine.
> If you actually run the old versions, then you'll get the old packages
> with old security bugs.  If they can connect to the network, best disconnect
> first.
> 
> Only run old (and therefore possibly with publicly-known and unfixed security
> issues) issues on trusted data!  Use case I have in mind:
> 
>    /.../old-profile-version/bin/tome4  (when you've disabled Internet access,
>      for trying an old version of tome4 (a game) to see what has changes since).
> 
>>   As it depend if they
>> have access to suid or not (i didnt investigate this, but if they have
>> then thats big problem but this is not the ticket to discuss it)
> 
> "guix package -i" never creates setuid/setgid binaries.  The only setuid/setgid
> binaries that guix creates are in /run/setuid-programs.  These are setuid/setgid
> _copies_ of what's requested in the _current_ (or the one at boot, I forgot)
> operating-system declaration.
> 
>> I know someone would jump in and say but roll back is great feature and
>> its useful and....i know that but like i said might be not suiting all
>> users (specially with limited space).
> 
> Yes, but let's at least keep the last few generations around.
> 
> E.g., I actually _almost_ never use the rollback mechanism.  The only
> time was when I messed up the operating-system declaration, so I had to boot
> into the previous system generation (is that the term?).
> 
> Of course, then there's a choice to make for _how many_ generations to keep
> around ...
> 
>> Current manual solution is to delete this extra mess using 2 commands:
>>
>> guix gc -d 1s && sudo guix system delete-generation
> 
> You should run them in the opposite order.
> Also, "guix package --delete-generations" should be run for each user.
> 
>> This should be run whenever there is no space left,
> 
> Tricky ... reportedly, many software does not handle out-of-disk-space errors
> well.  Also, letting "guix gc" and "guix package --delete-generations" run
> takes some time.  So this would have to be run when there's only 10% disk
> space left or something like that.
> 
>> Or to get rid of the old stuff
> 
> When you've the space, I recommend keeping the ‘old stuff’ around.
> You'd never know whether you'll need it later!  In case you'll need it later,
> keeping the ‘old stuff’ around saves on Internet traffic (saving some time)
> (lessening the load on the substitute servers -> less network and disk I/O and
> CPU usage --> less monetary costs, less environmental cost).
> 
>> My suggestion is to have the ability to make Guix automatically just
>> having the latest up to date packages
> 
> There's an unattended-upgrade-service for the system profile (if that's the
> result of operating-system).  Maybe we can have something similar for user
> profiles.
> 
>> without extra consumed storage (no cache no generation no nothing more than
>> having the latest packages available in the distro).
>>
>> So the process is ADD A2 → SWITCH to A2 → Delete A1 , Or Download A2 →
>> Replace over A1 and so on.
> 
> This in-place replacement you seem to be suggesting, is rather counter one of
> the primary goals of Guix (and Nix, for that matter) --- functional package
> management.
> 
> Automatically deleting the previous profile and trying to delete the previous
> version whenever possible could be possible, but I'm not sure it's worth it.
> 
> You could try to implement it yourself, and try your modified guix out for
> a while, and report whether it seems to work well, of course!
> 
> However, I believe the following would be easier in the short term
> (and *very* likely to be accepted upstream):
> 
> Implement a graphical application (maybe using the guile-gnome bindings,
> or as a web app run on localhost) that has a few buttons for collecting
> garbage and deleting old profiles.  Guix could use some graphical stuff.
> 
> Of course, what you decide to hack on, what approaches you'll take, etc.
> is up to you!
> 
> Greetings,
> Maxime
>