bug#41484: guix gc fails catastrophically if the disk has no space left

[Stefan <stefan-guix@vodafonemail.de>]

Hi!

While trying to build guix, my disk ran out of space.

building /gnu/store/w5l7acwbfvn0n8zj2v9by4j3jwbiimj9-guix-1.1.0-4.bdc801e.drv...
100% [#########################################################################]note: build failure may have been caused by lack of free disk space
builder for `/gnu/store/w5l7acwbfvn0n8zj2v9by4j3jwbiimj9-guix-1.1.0-4.bdc801e.drv' failed with exit code 1
build of /gnu/store/w5l7acwbfvn0n8zj2v9by4j3jwbiimj9-guix-1.1.0-4.bdc801e.drv failed
View build log at '/var/log/guix/drvs/w5/l7acwbfvn0n8zj2v9by4j3jwbiimj9-guix-1.1.0-4.bdc801e.drv.bz2'.
guix system: error: build of `/gnu/store/w5l7acwbfvn0n8zj2v9by4j3jwbiimj9-guix-1.1.0-4.bdc801e.drv' failed

stefan@guix ~$ df -h /
Dateisystem                                       Größe Benutzt Verf. Verw% Eingehängt auf
:/volume5/RaspberryPi/Betriebssysteme/guix-system  1,4T    1,4T     0  100% /

OK, well, there’s certainly a lot of garbage to collect. Due to bad experiences in the past with the same situation, I first try to collect only a little garbage.

stefan@guix ~$ guix gc -F 100M
guix gc: 100 MiB werden freigegeben
finding garbage collector roots...
note: can't create trash directory: creating directory `/gnu/store/trash': No space left on device
deleting garbage...
[0%] deleting '/gnu/store/g30aknl5yxk9jbpx2w7r5d0ak6xa5dl2-grub.cfg.lock'
[0%] deleting '/gnu/store/gidc7jrgbnvfm38qx4sc012kn9llgsvz-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04-collection.lock'
[0%] deleting '/gnu/store/9cpj3ir8fnsg5fiqs3x8alai713xzfz0-grub-keymap.de.lock'
[0%] deleting '/gnu/store/z08s3kwlnpqq6aa1v6ydni5qp2k7gkkb-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04'
[0%] deleting '/gnu/store/wv7iwd66q6n5775qa4hgaq9j9f2vyvl4-guix-1.1.0-4.bdc801e.lock'
[0%] deleting '/gnu/store/pab2ylsvas12824qw6j8q2h1qm3gh4x6-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04-union.lock'
[0%] deleting '/gnu/store/3ldyga22s6ym9g2pc9ycjwdwqb4i41v2-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04.lock'
[0%] deleting '/gnu/store/nfxgqhjyzwngiadn0l180hsbfnw3h4zy-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04-collection.lock'
[0%] deleting '/gnu/store/pab2ylsvas12824qw6j8q2h1qm3gh4x6-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04-union'
[0%] deleting '/gnu/store/dahbiixsbi1xhjiim9phl1vginm83bbm-grub.cfg.lock'
[0%] deleting '/gnu/store/61mwka1h77kkhylcs9jj4ij167qgfj39-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04.lock'
[0%] deleting '/gnu/store/q829h36vnwhbdlnzkgdrx2b51kp2mvdv-grub.cfg.lock'
[0%] deleting '/gnu/store/ragpi42x39sf5qwcmcmdn751fp4jfhmv-grub.cfg.lock'
[0%] deleting '/gnu/store/ncdsjfl245k6r7zym6w4sr9fyxrwhb38-grub.cfg.lock'
[0%] deleting '/gnu/store/z08s3kwlnpqq6aa1v6ydni5qp2k7gkkb-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04.lock'
[0%] deleting '/gnu/store/gidc7jrgbnvfm38qx4sc012kn9llgsvz-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04-collection'
[0%] deleting '/gnu/store/61mwka1h77kkhylcs9jj4ij167qgfj39-package-collection-grub-efi-raspberrypi-firmware-u-boot-rpi-3-2.04'
[0%] deleting '/gnu/store/hmy4ni005digrilzjf93wr9gaj5cq536-computed-files.lock'
[0%] deleting '/gnu/store/hmy4ni005digrilzjf93wr9gaj5cq536-computed-files'
error (ignored): aborting transaction: cannot rollback - no transaction is active
guix gc: error: committing transaction: disk I/O error
stefan@guix ~$ 

And that’s it. Repeating the guix gc command now results in this immediate error:

stefan@guix ~$ guix gc -F 100M
warning: SQLite database is busy (SQLITE_PROTOCOL)
guix gc: error: setting synchronous mode: locking protocol
stefan@guix ~$ guix gc -F 100M
warning: SQLite database is busy (SQLITE_PROTOCOL)
guix gc: error: setting synchronous mode: locking protocol

A reboot is needed to solve this SQLite error. But retrying all this leads to the same situation again and again. It is kind of a deadlock.

Well, as my guix system boots from an NFS share, in the past I solved this by removing other stuff – not related to guix – from the same disk. But right now there isn't unrelated stuff left to be deleted. 

I intend to remove the listed /gnu/store/… files by hand from the NFS server side (which is much easier than removing the read-only bind-mount.) Is this save at least?

This is all on a Raspberry Pi 3b aarch64 machine, if it matters.


Bye

Stefan