guix-gc

guix-gc

[Ludovic Courtès]

Commit fe8ff02 adds a ‘guix-gc’ command to invoke the GC, similar to
part of the ‘nix-store’ command.  Below is a copy of the manual.

Comments welcome!

Ludo’.


Packages that are installed but not used may be "garbage-collected".
The `guix-gc' command allows users to explicitly run the garbage
collector to reclaim space from the `/nix/store' directory.

   The garbage collector has a set of known "roots": any file under
`/nix/store' reachable from a root is considered "live" and cannot be
deleted; any other file is considered "dead" and may be deleted.  The
set of garbage collector roots includes default user profiles, and may
be augmented with `guix-build --root', for example (*note Invoking
guix-build::).

   The `guix-gc' command has three mode of operations: it can be used
to garbage-collect any dead files (the default), to delete specific
files (the `--delete' option), or to print garbage-collector
information.  The available options are listed below:

`--collect-garbage[=MIN]'
`-C [MIN]'
     Collect garbage--i.e., unreachable `/nix/store' files and
     sub-directories.  This is the default operation when no option is
     specified.

     When MIN is given, stop once MIN bytes have been collected.  MIN
     may be a number of bytes, or it may include a unit as a suffix,
     such as `MiB' for mebibytes and `GB' for gigabytes.

     When MIN is omitted, collect as much garbage as possible.

`--delete'
`-d'
     Attempt to delete all the store files and directories specified as
     arguments.  This fails if some of the files are not in the store,
     or if they are still live.

`--list-dead'
     Show the list of dead files and directories still present in the
     store--i.e., files and directories no longer reachable from any
     root.

`--list-live'
     Show the list of live store files and directories.

Re: guix-gc

[Andreas Enge]

[-- Attachment #1: Type: text/plain, Size: 1167 bytes --]

Am Sonntag, 6. Januar 2013 schrieb Ludovic Courtès:
> Commit fe8ff02 adds a ‘guix-gc’ command to invoke the GC, similar to
> part of the ‘nix-store’ command.  Below is a copy of the manual.
> 
> Comments welcome!

When running "guix-gc" without specifying any root, the complete content of 
/nix/store was classified as dead. This looks a bit excessive to me; should 
not at least the gnu build system that would be used to build a package 
with the currently installed guix be preserved? Even if the user has not 
installed gcc in his environement, for instance, the "current" gcc should 
be kept, whereas any older gcc (I had quite a few of them corresponding to 
several versions of guix) should be deleted.

If I see it correctly, this means that the output of
   guix-build hello --dry-run
should be kept in the store.


I also got the following error on the first run:

deleting `/nix/store/j29s9129di8lzx7xbc9rj0ywx05pivij-readline-6.2'
error: build failed: getting status of 
`/nix/store/j29s9129di8lzx7xbc9rj0ywx05pivij-readline-6.2': No such file or 
directory

But the next run worked and deleted everything.

Andreas

[-- Attachment #2: Type: text/html, Size: 4730 bytes --]

Re: guix-gc

[Ludovic Courtès]

Hi!

Andreas Enge <andreas@enge.fr> skribis:

> When running "guix-gc" without specifying any root, the complete content of 
> /nix/store was classified as dead. This looks a bit excessive to me; should 
> not at least the gnu build system that would be used to build a package 
> with the currently installed guix be preserved? Even if the user has not 
> installed gcc in his environement, for instance, the "current" gcc should 
> be kept, whereas any older gcc (I had quite a few of them corresponding to 
> several versions of guix) should be deleted.

Well, anything that’s not reachable from a GC root is deleted.

Nix provides other options that extend this model to something closer to
what you say, namely ‘gc-keep-outputs’ (see
<http://hydra.nixos.org/build/3676991/download/1/nix/manual.html>.)

I guess we should add a guix-daemon option for that.

> If I see it correctly, this means that the output of
>    guix-build hello --dry-run
> should be kept in the store.

Currently it’s not the case, unless you use:

  guix-build --root=my-hello hello

and the keep the ‘my-hello’ symlink around.

> I also got the following error on the first run:
>
> deleting `/nix/store/j29s9129di8lzx7xbc9rj0ywx05pivij-readline-6.2'
> error: build failed: getting status of 
> `/nix/store/j29s9129di8lzx7xbc9rj0ywx05pivij-readline-6.2': No such file or 
> directory
>
> But the next run worked and deleted everything.

Hmm, weird.  If that happens again, perhaps you should try to see where
that directory is, by looking at /nix/store and at --list-dead and
--list-live, for instance.

Thanks,
Ludo’.

Re: guix-gc

[Andreas Enge]

[-- Attachment #1: Type: text/plain, Size: 918 bytes --]

Am Donnerstag, 10. Januar 2013 schrieb Ludovic Courtès:
> Nix provides other options that extend this model to something closer to
> what you say, namely ‘gc-keep-outputs’ (see
> <http://hydra.nixos.org/build/3676991/download/1/nix/manual.html>.)
> 
> I guess we should add a guix-daemon option for that.

If I understand the manual correctly, this is not exactly what I meant. I 
thought of simply keeping all packages that are needed to build a simple 
project, such as "hello". Or maybe just keeping gcc, as it takes really 
very long to build. But indeed, this is less critical once binary downloads 
from a build farm are possible.

Another option would be something close to debian's "apt-get autoclean": 
Keep all the packages that are still available in the sense that they would 
be built by calls to "guix-build", looping over the output of "guix-package 
--list-available".

Andreas

[-- Attachment #2: Type: text/html, Size: 3187 bytes --]

Re: guix-gc

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> Am Donnerstag, 10. Januar 2013 schrieb Ludovic Courtès:
>> Nix provides other options that extend this model to something closer to
>> what you say, namely ‘gc-keep-outputs’ (see
>> <http://hydra.nixos.org/build/3676991/download/1/nix/manual.html>.)
>> 
>> I guess we should add a guix-daemon option for that.
>
> If I understand the manual correctly, this is not exactly what I meant. I 
> thought of simply keeping all packages that are needed to build a simple 
> project, such as "hello". Or maybe just keeping gcc, as it takes really 
> very long to build. But indeed, this is less critical once binary downloads 
> from a build farm are possible.

I’ve actually installed or registered roots for such “precious”
packages, which prevents them from being GC’d.

> Another option would be something close to debian's "apt-get autoclean": 
> Keep all the packages that are still available in the sense that they would 
> be built by calls to "guix-build", looping over the output of "guix-package 
> --list-available".

For packages that use ‘gnu-build-system’, the prerequisites are a
superset of (@ (gnu packages base) final-inputs), which includes GCC,
Coreutils, libc, etc.  So you would want to keep those, I guess, right?

OTOH, imagine a hypothetical ‘python-build-system’, whose only implicit
inputs are Python and perhaps Coreutils.  Someone who only installs
Python packages only cares about keeping Python and Coreutils around.

And then, the GC.  The GC only sees derivations, and derivation outputs.
It doesn’t have the slightest idea of which outputs are valuable to the
user, and which aren’t.

So the normal way to address this problem is to tell the GC what’s
important: by installing packages, or by running ‘guix-build --root’.

The Nix daemon options I mentioned above can be used to do something
more: keeping the source tarballs of derivations whose outputs are still
present.

But as you note, all this will be much less of a concern when we can
download pre-built binaries.  Something to get in for 0.2, I guess.  :-)

WDYT?

Ludo’.

Re: guix-gc

[Andreas Enge]

[-- Attachment #1: Type: text/plain, Size: 557 bytes --]

Am Dienstag, 22. Januar 2013 schrieb Ludovic Courtès:
> For packages that use ‘gnu-build-system’, the prerequisites are a
> superset of (@ (gnu packages base) final-inputs), which includes GCC,
> Coreutils, libc, etc.  So you would want to keep those, I guess, right?

Yes, that was the suggestion.

> But as you note, all this will be much less of a concern when we can
> download pre-built binaries.  Something to get in for 0.2, I guess.  :-)

I agree; it is maybe too presumptuos to decide what the user would like to 
keep.

Andreas

[-- Attachment #2: Type: text/html, Size: 2947 bytes --]