From: "Christian Thäter" <ct.guix@pipapo.org>
To: 51787@debbugs.gnu.org
Subject: bug#51787: GC takes more than 9 hours on berlin
Date: Mon, 13 Dec 2021 17:13:33 +0100 [thread overview]
Message-ID: <20211213171333.04105a3b@wolke.pipapo.org> (raw)
In-Reply-To: <87o85lhhrd.fsf@gnu.org>
On 2021-12-12 18:09, Mathieu Othacehe wrote:
>Hey,
>
>> OK, thanks. Looks it just finished removing the trash directory
>> content. I started a GC process from my session to monitor it
>> closely.
>
>Daily GC recap:
>
>* The GC process I started yesterday, did collect 5.5TiB in
> approximately 24 hours, that are now in the /gnu/store/trash
> directory.
>
>* The /gnu/store/trash directory contains 288910 entries. If those
>items
> are removed at the same rate than on the previous days, it will take
> days/months to delete them all.
>
>* I noticed that the upstream Nix GC process can now operate without
> locking. I think it shouldn't be too hard to port it to our fork or
> maybe rewrite the process in Guile while we are at it.
>
> That will not fix the slow hard-drives issues though.
While discussing this issue on IRC I came up with some idea:
'rmrfd' a system daemon that deletes huge trees in the background where
'-rf' stands for --really --fast :)
Actually this is an use case that happens for on my backup system too.
With that idea I just started coding and ran some experiments. For me
this looks quite feasible now and I will continue next days on this
small project. Any feedback or help would be welcomed!
The initial ideas and experiments are at https://github.com/cehteh/rmrfd
Note that the important part is that it will put some efforts into
freeing as much space as possible at begin of the freeing process,
Unlike just 'rm -rf' where space may only freed really late when the
last link count of the data goes to zero.
Cheers
Christian
next prev parent reply other threads:[~2021-12-14 3:13 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-12 11:49 bug#51787: GC takes more than 9 hours on berlin Mathieu Othacehe
2021-11-12 19:17 ` Mathieu Othacehe
2021-11-22 9:16 ` zimoun
2021-11-23 17:48 ` Ludovic Courtès
2021-11-25 13:24 ` Christopher Baines
2021-11-27 11:23 ` Ludovic Courtès
2021-12-03 9:45 ` Mathieu Othacehe
2021-12-10 6:24 ` Mathieu Othacehe
2021-12-10 10:21 ` Ludovic Courtès
2021-12-10 17:11 ` Mathieu Othacehe
2021-12-10 21:54 ` Ludovic Courtès
2021-12-11 9:42 ` Mathieu Othacehe
2021-12-12 17:09 ` Mathieu Othacehe
2021-12-13 16:13 ` Christian Thäter [this message]
2021-12-14 3:31 ` Christian Thäter
2021-12-17 10:55 ` Mathieu Othacehe
2021-12-17 13:06 ` Ricardo Wurmus
2021-12-17 14:08 ` Ricardo Wurmus
2021-12-10 17:11 ` Mathieu Othacehe
2021-11-27 11:11 ` Ludovic Courtès
2021-12-10 19:38 ` Ricardo Wurmus
2021-12-20 21:12 ` Ricardo Wurmus
2023-08-16 10:53 ` Maxim Cournoyer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211213171333.04105a3b@wolke.pipapo.org \
--to=ct.guix@pipapo.org \
--cc=51787@debbugs.gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.