Pierre Neidhardt writes: > Thanks for mentioning this. I'm very confused by the documentation however: > > ‘--gc-keep-outputs[=yes|no]’ > Tell whether the garbage collector (GC) must keep outputs of live > derivations. > > When set to “yes”, the GC will keep the outputs of any live > derivation available in the store—the ‘.drv’ files. The default is > “no”, meaning that derivation outputs are kept only if they are GC > roots. *Note Invoking guix gc::, for more on GC roots. > > Specifically this: "derivation outputs are kept only if they are GC roots'". > Would that mean that other live outputs are deleted?!? I must get this wrong! :p Live outputs are never eligible for garbage collection. The behavior of these options might be a little non-obvious, so I'll explain what I understand about it, and hopefully it will help. First, let's assume that you've set both --gc-keep-derivations=no and --gc-keep-outputs=no. Let's also suppose that you've installed hello@2.10 into your profile. Since your profile is a GC root, this means that hello@2.10's output (e.g., /gnu/store/...krfy-hello-2.10) is live. However, its derivation isn't live, since the derivation is neither in your profile nor in the output's closure. To query for the deriver of hello@2.10, you can do this: --8<---------------cut here---------------start------------->8--- $ guix build -d hello@2.10 /gnu/store/p2hmc4wv59kxvhvxa1dwjjps0g38ikc1-hello-2.10.drv --8<---------------cut here---------------end--------------->8--- That derivation describes precisely how to build hello@2.10, all the way down to how to fetch the sources. That's because it refers to other dependencies. Here they are: --8<---------------cut here---------------start------------->8--- $ guix gc --references $(guix build -d hello@2.10) /gnu/store/1bjlh3cyij46a3mwi0ikyrn1pb70bd8m-gawk-4.2.1.drv /gnu/store/1dv1gyyqfn50i2zrq4fvr7fbgwm3hlkx-hello-2.10-guile-builder /gnu/store/2n4fdhjq47cgifbp2wsmyapqxih73bm4-ld-wrapper-0.drv [... in total, there are 25 store items like this ...] --8<---------------cut here---------------end--------------->8--- You can consider these store items to be the "direct dependencies" of the hello@2.10 derivation. Many of them are derivations themselves, which makes sense because you cannot start the hello@2.10 derivation until after all the things it requires have been built. Furthermore, these derivations may in turn refer to other derivations. Here are the "direct and transitive dependencies" of the hello@2.10 derivation (which includes the hello@2.10 derivation itself): --8<---------------cut here---------------start------------->8--- $ guix gc --requisites $(guix build -d hello@2.10) /gnu/store/xc0wliwjngcsx26wh65dx07xnas4v146-gcc-4.9.4.tar.xz-builder /gnu/store/lmfvf4iwkzn4wibvb7ik3cadxq07pjvi-gcc-libvtv-runpath.patch /gnu/store/d57l8pc992bdqd20l5piqk9k47dqirzm-gcc-4.9-libsanitizer-fix.patch /gnu/store/5752c05gravrwk2kvy1n2zrgl9j0kxhg-gcc-4.9.4.tar.bz2.drv [... in total, there are 308 store items like this ...] --8<---------------cut here---------------end--------------->8--- These 308 store items make up the closure of the hello@2.10 derivation. Compare this to the closure of hello@2.10's output: --8<---------------cut here---------------start------------->8--- $ guix gc --requisites $(guix build hello@2.10) /gnu/store/1r3dlhi2vasb8yw630728jlrk40mygj1-bash-static-4.4.19 /gnu/store/vla5j7pbkpcp39lsdfsmz7m9azn48lr4-gcc-5.5.0-lib /gnu/store/l4lr0f5cjd0nbsaaf8b5dmcw1a1yypr3-glibc-2.27 /gnu/store/bihfrh609gkxb9dp7n96wlpigiv3krfy-hello-2.10 --8<---------------cut here---------------end--------------->8--- The output's closure contains only 4 items! And if you examine these 4 items, you'll find that none of them appear in the closure of the hello@2.10 derivation. Therefore, even though the output of hello@2.10 is live because you've installed it into your profile, none of those 309 store items that were required for building hello@2.10 are live. They are all eligible for garbage collection. However, if you set --gc-keep-derivations=yes, then the hello@2.10 derivation will be treated as live. As a result, every store item in the hello@2.10 derivation's closure becomes live, also. In this way, --gc-keep-derivations=yes causes liveness to flow from outputs to derivations. But we're not done yet. Consider one of the derivations that shows up in the hello@2.10' derivation's closure: /gnu/store/5752c05gravrwk2kvy1n2zrgl9j0kxhg-gcc-4.9.4.tar.bz2.drv This derivation builds a tarball of gcc's sources. Let's build it: --8<---------------cut here---------------start------------->8--- $ guix build /gnu/store/5752c05gravrwk2kvy1n2zrgl9j0kxhg-gcc-4.9.4.tar.bz2.drv /gnu/store/1j3mqrcp3y4xlb9jl5d0ri5aszn8mfii-gcc-4.9.4.tar.bz2 --8<---------------cut here---------------end--------------->8--- The output path of this derivation is /gnu/store/...mfii-gcc-4.9.4.tar.bz2, which appears in neither the closure of hello@2.10's derivation nor the closure of hello@2.10's output. So, if you had built this derivation while building hello@2.10, the derivation's output (/gnu/store/...mfii-gcc-4.9.4.tar.bz2) would be eligible for garbage collection even if you had specified --gc-keep-derivations=yes. This is where --gc-keep-outputs=yes comes in. If you specify it, then the /gnu/store/...mfii-gcc-4.9.4.tar.bz2 store item will be treated as live, since it is the output of a live derivation (/gnu/store/...kxhg-gcc-4.9.4.tar.bz2.drv). In this way, --gc-keep-outputs=yes causes liveness to flow from derivations to outputs. In Guix, the defaults are --gc-keep-outputs=no and --gc-keep-derivations=yes. In this configuration, derivations tend to be kept, and outputs tend to be collected. Derivations usually don't take up much space (invoke "guix size $(guix build -d hello)" to get an idea of how small they are), but their outputs can be large. If you set --gc-keep-outputs=yes in addition to --gc-keep-derivations=yes, you will retain package outputs that you install, as well as all the derivations necessary for building the package, as well as all the outputs that you built while running those derivations. If you build from source instead of downloading substitutes, this means you'll have a lot of intermediary outputs stored in your local file system, and you won't be able to collect them, since they'll all be live. You should try setting both to yes explicitly and see if this makes your experience any better. Hopefully it does! However, there is a chance it might actually cause even more disk usage, and you might not be able to collect enough garbage to free up space. Anyway, I hope that helps clarify what these options do. I had forgotten about it, myself, so it was fun to remind myself how it works. Pierre Neidhardt writes: > The store does not have timestamps, so removing old items would not work > I guess. But what about removing older versions first? Say I have > glibc-2.24 and glibc-2.25, only remove glibc-2.24 if that's enough to > free the required space. > > Maybe we could introduce a "whitelist" of packages to delete last. These might help, but I think the options I suggested are the best thing to try first. Do they help in your case? You might need to aggressively delete old profile generations, also, to make sure you have some garbage to collect. > In the end I figured that the most convenient way to clean up the >store > might be to do it manually with "guix gc --delete PATH". Well, not so > from the commandline, but with guix.el I thought we could do something > nice. So I gave it a (quick&dirty™) shot: > > https://gitlab.com/emacs-guix/emacs-guix/issues/2 > > With guix.el loaded, `eval' my example and run "M-x guix-store". > This will return a list of all the dead links, which you can sort by > name or by size. You can then mark items ("m") and delete them ("k"). > > What do you people think? I don't use emacs-guix, so I can't comment on it, I'm afraid. Maybe someday I'll get around to using it, and then I can say something useful! > While I'm at it, I'd like to note that something might be wrong with the > `-F` option: I never get the promised amount of free space back, only > about 2/3 of it. I think that's normal. My understanding is that if you say "guix gc -F 3GiB", you aren't asking Guix to free 3 GiB; you're asking Guix to try to free enough space so that when it's done, the store will have 3 GiB of free space. For example, if the store begins with 2 GiB of free space and there is 5 GiB of garbage, Guix will only collect 1 GiB in order to bring the free space up to 3 GiB. -- Chris