Garbage Collector

Garbage Collector

[Gottfried]

[-- Attachment #1.1.1: Type: text/plain, Size: 422 bytes --]


Hi Guixers,

I would like to use guix gc.

The manual says that it is dangerous to use: "guix gc"
because it can delete too much.

So would it be a good idea to use: "guix gc -d 2m"
It would delete older than 2 month generations and also collect garbage.
Than the danger to delete something that the system needs is minimized.

Or is there still a better idea around?

Thanks for your answers

Gottfried

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3191 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: Garbage Collector

[(]

On Mon Jul 25, 2022 at 10:51 AM BST, Gottfried wrote:
> The manual says that it is dangerous to use: "guix gc"
> because it can delete too much.
I think you're misunderstanding what the manual means, and what garbage
collection entails.

(Disclaimer: I'm not a Guix expert, so it's possible this explanation is
partially incorrect.)

The store is a massive database, containing things like outputs,
derivations, and profiles. These are then symlinked into various
positions in the system, for example your user's profile:

ʃ readlink -f .guix-profile
/gnu/store/vpjlpmxnfsmbzzmv15das5gzcg308cdk-profile

which itself contains symlinks to subpaths of outputs:

ʃ readlink /gnu/store/vpjlpmxnfsmbzzmv15das5gzcg308cdk-profile/bin/vim
/gnu/store/kkc4ff1bxmb8qixiiy4rbc256m8l4rmz-vim-9.0.0000/bin/vim

In my profile, I have two packages installed:

ʃ guix package --list-generations
Generation 1 ...
  aerc ...
  vim ...

What happens if I remove `vim`? Well, we need to create an entirely new
profile (which does not contain Vim things). Our .guix-profile is now
linked to a completely different store path, this one without any
Vim-related symlinks:

ʃ readlink -f .guix-profile
/gnu/store/22ph0mlsd83k4p6wsc6276rhsk3qanxv-profile

ʃ ls /gnu/store/22ph0mlsd83k4p6wsc6276rhsk3qanxv-profile/bin
aerc

What happened to our Vim output path? Was it deleted?

ʃ test -e /gnu/store/kkc4ff1bxmb8qixiiy4rbc256m8l4rmz-vim-9.0.0000 && echo still exists
still exists

So, is it just hanging around in the store, taking up space? Not quite.
As you'll probably know, Guix supports rolling back to previous
generations:

ʃ guix package --list-generations
Generation 1 ...
  aerc ...
  vim ...

Generation 2 ...
  - vim ...

So Vim is actually still in use! But we've decided that we are
absolutely certain that we don't want Vim. Let's delete all records of
Vim's existence in our profile.

ʃ guix package --delete-generations
deleting /var/guix/profiles/per-user/paren/guix-profile-1-link

And now...

ʃ guix package --list-generations
Generation 929 ...
  aerc ...

No more Vim.

ʃ test -e /gnu/store/kkc4ff1bxmb8qixiiy4rbc256m8l4rmz-vim-9.0.0000 && echo still exists!
still exists!

What?! Vim really wants to stay alive. But it's not being used by
anything anymore, not even an old profile. So, how do we vanquish it?
With `guix gc` :)

ʃ guix gc
finding garbage collector roots...
deleting garbage...
[...]
[0 MiB] deleting '/gnu/store/kkc4ff1bxmb8qixiiy4rbc256m8l4rmz-vim-9.0.0000'
[...]
guix gc: freed 45.94845 MiBs

We did it! The Vim store path no longer exists (and neither does the Tcsh
store path, which is a dependency of Vim):

ʃ test -e /gnu/store/kkc4ff1bxmb8qixiiy4rbc256m8l4rmz-vim-9.0.0000 || echo gone!
gone!

So, `guix gc` makes sure never to touch anything that's still in use.
Quoting the manual,

> The garbage collector has a set of known roots: any file under
> /gnu/store reachable from a root is considered live and cannot be
> deleted; any other file is considered dead and may be deleted. The set
> of garbage collector roots (“GC roots” for short) includes default user
> profiles; by default, the symlinks under /var/guix/gcroots represent
> these GC roots. New GC roots can be added with guix build --root, for
> example (see Invoking guix build). The guix gc --list-roots command
> lists them.

This would make sense for a 'garbage collector'; after all, the term
usually refers to a system for automatically freeing unused memory, so
that it can be reused. If it were to accidentally free memory that's
still in use, someone's variable would suddenly point to invalid memory!
So the GC needs to be absolutely certain that nobody's using the memory.

The manual does say, though, that:

> Running guix gc with no arguments will collect as much garbage as it
> can, but that is often inconvenient: you may find yourself having to
> rebuild or re-download software that is “dead” from the GC viewpoint but
> that is necessary to build other pieces of software—e.g., the compiler
> tool chain.

And indeed, if we change our mind and decide we actually do want Vim, we
have to re-download it:

ʃ guix install vim
The following package will be installed:
   vim 9.0.0000

[...]

7.4 MB will be downloaded
[...]
 tcsh-6.22.03  323KiB ...
 vim-9.0.0000  7.1MiB ...
[...]

However, running `guix gc` is *never destructive or dangerous*, because
that would be counter to the entire purpose of a GC.

    -- (

Re: Garbage Collector

[Felix Lechner via]

Hi,

On Mon, Jul 25, 2022 at 2:52 AM Gottfried <gottfried@posteo.de> wrote:
>
> The manual says that it is dangerous to use: "guix gc"
> because it can delete too much.

I agree that 'guix gc' deletes too much, but it's probably not
dangerous. You will just see some downloads and builds repeated when
reconfiguring later.

I believe 'guix gc' deletes items that are not garbage to me (but I am
new to Guix). Based on friendly advice I received, I currently use
these settings:

(guix-service-type
  config => (guix-configuration
    (inherit config)
    (extra-options (list "--gc-keep-derivations=yes" "--gc-keep-outputs=yes"))
...

Kind regards
Felix Lechner

Re: Garbage Collector

[Gottfried]

[-- Attachment #1.1.1: Type: text/plain, Size: 1247 bytes --]

Hi,
when I look at your:

(guix-service-type
   config => (guix-configuration
     (inherit config)
     (extra-options (list "--gc-keep-derivations=yes" 
"--gc-keep-outputs=yes"))

Isn't there at the end one bracket too little, to close up the whole thing?

Because when I add that to my config.scm file the first bracket before 
guix-service-type appears violett in Emacs,
but there is no violet bracket at the end which closes it.

Gottfried


Am 25.07.22 um 17:20 schrieb Felix Lechner:
> Hi,
> 
> On Mon, Jul 25, 2022 at 2:52 AM Gottfried <gottfried@posteo.de> wrote:
>>
>> The manual says that it is dangerous to use: "guix gc"
>> because it can delete too much.
> 
> I agree that 'guix gc' deletes too much, but it's probably not
> dangerous. You will just see some downloads and builds repeated when
> reconfiguring later.
> 
> I believe 'guix gc' deletes items that are not garbage to me (but I am
> new to Guix). Based on friendly advice I received, I currently use
> these settings:
> 
>  > (guix-service-type
>   config => (guix-configuration
>     (inherit config)
>     (extra-options (list "--gc-keep-derivations=yes" "--gc-keep-outputs=yes"))


> ...
> 
> Kind regards
> Felix Lechner


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3191 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: Garbage Collector

[Gottfried]

[-- Attachment #1.1.1: Type: text/plain, Size: 3328 bytes --]

Hi again,

in my config.scm if I add this part

(guix-service-type
 >    config => (guix-configuration
 >      (inherit config)
 >      (extra-options (list "--gc-keep-derivations=yes" 
"--gc-keep-outputs=yes"))

at the end of my guix-service-types,
it seems to me there I have to make 4 brackets:
Or am I wrong?



My config.scm:

;; This is an operating system configuration generated
;; by the graphical installer.

(use-modules (gnu))
(use-package-modules cups scanner)
(use-service-modules cups desktop networking ssh xorg)

(operating-system
   (locale "de_DE.utf8")
   (timezone "Europe/Berlin")
   (keyboard-layout (keyboard-layout "de"))
   (host-name "Tuxedo")
   (users (cons* (user-account
                   (name "gfp")
                   (comment "Gfp")
                   (group "users")
                   (home-directory "/home/gfp")
                   (supplementary-groups
                     '("wheel" "netdev" "audio" "video")))
                 %base-user-accounts))
   (packages
     (append
       (list (specification->package "nss-certs"))			
       %base-packages))
   (services
     (append
       (list (service mate-desktop-service-type)
             (service enlightenment-desktop-service-type)
			(service cups-service-type
				(cups-configuration
					(web-interface? #t)
					(extensions (list cups-filters hplip))))			
			(service openssh-service-type)
             (service tor-service-type)
             (set-xorg-configuration
               (xorg-configuration
                 (keyboard-layout keyboard-layout))))
       (modify-services %desktop-services
		(sane-service-type _ => sane-backends))))
       (guix-service-type
         config => (guix-configuration
           (inherit config)
           (extra-options (list "--gc-keep-derivations=yes" 
"--gc-keep-outputs=yes"))

   (bootloader
     (bootloader-configuration
       (bootloader grub-efi-bootloader)
       (target "/boot/efi")
       (keyboard-layout keyboard-layout)))
   (swap-devices
     (list (uuid "51d5cd20-4513-4a02-9e35-df4338eccaa0")))
   (file-systems
     (cons* (file-system
              (mount-point "/boot/efi")
              (device (uuid "BB77-FE3B" 'fat32))
              (type "vfat"))
            (file-system
              (mount-point "/")
              (device
                (uuid "4fb0ed7c-61ab-45eb-be0b-ff527b320e6d"
                      'ext4))
              (type "ext4"))
            %base-file-systems)))


Gottfried


Am 25.07.22 um 17:20 schrieb Felix Lechner:
> Hi,
> 
> On Mon, Jul 25, 2022 at 2:52 AM Gottfried <gottfried@posteo.de> wrote:
>>
>> The manual says that it is dangerous to use: "guix gc"
>> because it can delete too much.
> 
> I agree that 'guix gc' deletes too much, but it's probably not
> dangerous. You will just see some downloads and builds repeated when
> reconfiguring later.
> 
> I believe 'guix gc' deletes items that are not garbage to me (but I am
> new to Guix). Based on friendly advice I received, I currently use
> these settings:
> 
> (guix-service-type
>    config => (guix-configuration
>      (inherit config)
>      (extra-options (list "--gc-keep-derivations=yes" "--gc-keep-outputs=yes"))
> ...
> 
> Kind regards
> Felix Lechner


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3191 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: Garbage Collector

[Gottfried]

[-- Attachment #1.1.1: Type: text/plain, Size: 3648 bytes --]

Hi, Guixers,

I think I managed to add the proposal of Felix Lechner and to set up my 
brackets.

Behind:
(sane-service-type _ => sane-backends))))

I remove one bracket (because the 4. brackets closes "services")

and

behind:

(guix-service-type
          config => (guix-configuration
            (inherit config)
            (extra-options (list "--gc-keep-derivations=yes"
"--gc-keep-outputs=yes"))

I add 3 brackets, to have 5 brackets (the 5. bracket closes "services").

This is how I understand it.
Am I right?

kind regards
Gottfried


My confi.scm before changing the brackets.


;; This is an operating system configuration generated
;; by the graphical installer.

(use-modules (gnu))
(use-package-modules cups scanner)
(use-service-modules cups desktop networking ssh xorg)

(operating-system
    (locale "de_DE.utf8")
    (timezone "Europe/Berlin")
    (keyboard-layout (keyboard-layout "de"))
    (host-name "Tuxedo")
    (users (cons* (user-account
                    (name "gfp")
                    (comment "Gfp")
                    (group "users")
                    (home-directory "/home/gfp")
                    (supplementary-groups
                      '("wheel" "netdev" "audio" "video")))
                  %base-user-accounts))
    (packages
      (append
        (list (specification->package "nss-certs"))			
        %base-packages))
    (services
      (append
        (list (service mate-desktop-service-type)
              (service enlightenment-desktop-service-type)
			(service cups-service-type
				(cups-configuration
					(web-interface? #t)
					(extensions (list cups-filters hplip))))			
			(service openssh-service-type)
              (service tor-service-type)
              (set-xorg-configuration
                (xorg-configuration
                  (keyboard-layout keyboard-layout))))
        (modify-services %desktop-services
		(sane-service-type _ => sane-backends))))
        (guix-service-type
          config => (guix-configuration
            (inherit config)
            (extra-options (list "--gc-keep-derivations=yes"
"--gc-keep-outputs=yes"))

    (bootloader
      (bootloader-configuration
        (bootloader grub-efi-bootloader)
        (target "/boot/efi")
        (keyboard-layout keyboard-layout)))
    (swap-devices
      (list (uuid "51d5cd20-4513-4a02-9e35-df4338eccaa0")))
    (file-systems
      (cons* (file-system
               (mount-point "/boot/efi")
               (device (uuid "BB77-FE3B" 'fat32))
               (type "vfat"))
             (file-system
               (mount-point "/")
               (device
                 (uuid "4fb0ed7c-61ab-45eb-be0b-ff527b320e6d"
                       'ext4))
               (type "ext4"))
             %base-file-systems)))


Gottfried


Am 25.07.22 um 17:20 schrieb Felix Lechner:
> Hi,
> 
> On Mon, Jul 25, 2022 at 2:52 AM Gottfried <gottfried@posteo.de> wrote:
>>
>> The manual says that it is dangerous to use: "guix gc"
>> because it can delete too much.
> 
> I agree that 'guix gc' deletes too much, but it's probably not
> dangerous. You will just see some downloads and builds repeated when
> reconfiguring later.
> 
> I believe 'guix gc' deletes items that are not garbage to me (but I am
> new to Guix). Based on friendly advice I received, I currently use
> these settings:
> 
> (guix-service-type
>    config => (guix-configuration
>      (inherit config)
>      (extra-options (list "--gc-keep-derivations=yes" "--gc-keep-outputs=yes"))
> ...
> 
> Kind regards
> Felix Lechner


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3191 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: Garbage Collector

[Ludovic Courtès]

Hi,

Felix Lechner via <help-guix@gnu.org> skribis:

> On Mon, Jul 25, 2022 at 2:52 AM Gottfried <gottfried@posteo.de> wrote:
>>
>> The manual says that it is dangerous to use: "guix gc"
>> because it can delete too much.
>
> I agree that 'guix gc' deletes too much, but it's probably not
> dangerous. You will just see some downloads and builds repeated when
> reconfiguring later.

Indeed, it’s not dangerous, and I don’t think the manual suggests that.

To avoid collecting things just to redownload/rebuild them later, I
usually ask ‘guix gc’ to free up some amount of space, as in:

  guix gc -F20G

That ensures 20G (more or less) are available on my disk and doesn’t try
to collect more than that.

Thanks,
Ludo’.

Re: Garbage Collector - for your Information

[Gottfried]

[-- Attachment #1.1.1: Type: text/plain, Size: 1563 bytes --]

Hi Guixers,

thanks for Your advice Ludo and all other Guixers,

for Your information (first time of using gc):

After using Guix System for 6 month I did a:
"guix gc"

It collected about 50 GB and deleted that.
(I wanted to do that to get to know how it collects without deleting 
generations)
Than I deleted all generations
I did a "guix gc" again
and it deleted approx 40 GB again.

After that I did a
guix pull,
guix package -u,
sudo guix system reconfigure /etc/config.scm

and it downloaded approx 2,2 GB of packages.

The result of that is, that in future I will do a
"guix gc -F20G"
in order not to delete too much and later to redownload and rebuild them.

Gottfried



Am 09.08.22 um 22:18 schrieb Ludovic Courtès:
> Hi,
> 
> Felix Lechner via <help-guix@gnu.org> skribis:
> 
>> On Mon, Jul 25, 2022 at 2:52 AM Gottfried <gottfried@posteo.de> wrote:
>>>
>>> The manual says that it is dangerous to use: "guix gc"
>>> because it can delete too much.
>>
>> I agree that 'guix gc' deletes too much, but it's probably not
>> dangerous. You will just see some downloads and builds repeated when
>> reconfiguring later.
> 
> Indeed, it’s not dangerous, and I don’t think the manual suggests that.
> 
> To avoid collecting things just to redownload/rebuild them later, I
> usually ask ‘guix gc’ to free up some amount of space, as in:
> 
>    guix gc -F20G
> 
> That ensures 20G (more or less) are available on my disk and doesn’t try
> to collect more than that.
> 
> Thanks,
> Ludo’.



[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3191 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]