From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id EOpGI8yqfmFkqwAAgWs5BA (envelope-from ) for ; Sun, 31 Oct 2021 15:40:12 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id tbq0HsyqfmHUKQAAbx9fmQ (envelope-from ) for ; Sun, 31 Oct 2021 14:40:12 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id EFECABF97 for ; Sun, 31 Oct 2021 15:40:11 +0100 (CET) Received: from localhost ([::1]:44100 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mhC0I-0007Au-Od for larch@yhetil.org; Sun, 31 Oct 2021 10:40:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36160) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mhC0A-0007AR-7Z for guix-patches@gnu.org; Sun, 31 Oct 2021 10:40:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:49472) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mhC09-0001Co-Vd for guix-patches@gnu.org; Sun, 31 Oct 2021 10:40:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mhC09-0004cV-Sk for guix-patches@gnu.org; Sun, 31 Oct 2021 10:40:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#51427] [PATCH] nix: libstore: Do not remove unused links when deleting specific items. Resent-From: Liliana Marie Prikler Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 31 Oct 2021 14:40:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 51427 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 51427@debbugs.gnu.org, Maxim Cournoyer Received: via spool by 51427-submit@debbugs.gnu.org id=B51427.163569115417680 (code B ref 51427); Sun, 31 Oct 2021 14:40:01 +0000 Received: (at 51427) by debbugs.gnu.org; 31 Oct 2021 14:39:14 +0000 Received: from localhost ([127.0.0.1]:32785 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mhBzN-0004b5-Ol for submit@debbugs.gnu.org; Sun, 31 Oct 2021 10:39:14 -0400 Received: from mail-wr1-f67.google.com ([209.85.221.67]:44808) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mhBzL-0004at-Tb for 51427@debbugs.gnu.org; Sun, 31 Oct 2021 10:39:12 -0400 Received: by mail-wr1-f67.google.com with SMTP id d13so24408018wrf.11 for <51427@debbugs.gnu.org>; Sun, 31 Oct 2021 07:39:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:subject:from:to:cc:date:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=F2Dxl6lA7AbP8VyW0XTsyRAmV50iMNc9ISlG/Fh8gEA=; b=LDsR3QEcuOflCoO4xuVmzD/+Ahtwrcamt9Zs0PGegEtGUmmWI0BEgyEvR05ave3uJe /ZvEnHoYQC6yhJQYrUapp1ALt1g/6nrpxt6qpjGpi/fP6FTB7tVmChOOlohL5QZtma3o 2fhnRgmUzkzgSYyC/pl368QS6U6FiW8Z9L9v1VjLaU+WvLaBaykGFxh3ghuFkjLrPxz3 gakTr8w0FRBUltezkfU+AxdJq8ZuLgj6Onz4YPxn6hqat4ci4WW8m1FKC+O9WbFUkcNH u3hFLy9BCCDePSiZrOUqUrEl640Pns60By7EQ6ntFX/tIY/300iVSrJmQ/4eULTvuw3p p5ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=F2Dxl6lA7AbP8VyW0XTsyRAmV50iMNc9ISlG/Fh8gEA=; b=s0luqigOGYRwU8NDVfRIp1xx9F3OVpqv+0UG4O8Jml7EDnCuJEChxUb8LBhLI4gUD6 phw3+LolXCqQdxWWvrgmbRwLDkdgGGNXRizxI+m/W1UhATITi5wUe4Nj7ThBQZCpT04d H4D4VHYjgYe7CaOBqBallR0dDxZcZnDPhQjV9GkN49D+PQX+TjJ+W/FWmjiirVBVUMu2 s8rWjwTZN0I6Lg/8Kn7ideVRdvzZscwcsXj6rJZndaXHvYzHyclFUDdg+FrMctfuSg5x CUemvG8RUn7oU+DT3oMOZTh3NTmWbgDpHi+p592ao6I66WZ0ytkzluwuSZ/N7WcYCibN fY4Q== X-Gm-Message-State: AOAM530ut9XM/SSSfUx1zWw1pY4IJIsWr+xasTZewYExvOP+8eyyIHRV KPXtUW0hemxOAi0II/T0Zhg= X-Google-Smtp-Source: ABdhPJyGMjbXEhmUBYyGWO76D4vLcyxVOiaJ2xVF9qKvzg6bOoS12ja2bPxlolH+uUep2zakh+Unaw== X-Received: by 2002:adf:9bdb:: with SMTP id e27mr28279411wrc.417.1635691146044; Sun, 31 Oct 2021 07:39:06 -0700 (PDT) Received: from nijino.fritz.box (85-127-52-93.dsl.dynamic.surfer.at. [85.127.52.93]) by smtp.gmail.com with ESMTPSA id y6sm11372365wrh.18.2021.10.31.07.39.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 31 Oct 2021 07:39:05 -0700 (PDT) Message-ID: <9f9a708f8ecbb49464e3d939be89638a53dc0d30.camel@gmail.com> From: Liliana Marie Prikler Date: Sun, 31 Oct 2021 15:39:04 +0100 In-Reply-To: <87h7cxp9tl.fsf@gnu.org> References: <20211027034918.4591-1-maxim.cournoyer@gmail.com> <87o8795j61.fsf@gnu.org> <5c2dd60acfaa7d74b7554babb3e223bc855bac8a.camel@gmail.com> <87h7cxp9tl.fsf@gnu.org> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.34.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1635691212; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=F2Dxl6lA7AbP8VyW0XTsyRAmV50iMNc9ISlG/Fh8gEA=; b=UKmB5cKU3KneeJ1gftJgFV7sf9QyVLMmaOWVgaNQ0O6zr/jdgSKc8xiJYM6io9uJBqF/WT 4m3vr8o0b9xlsYRRGcOjy7Izy36BlkgHI8/64ixiE88TPa148Hu/yybJn38Qhc9M0fbPCx PfNPEJ3Ivwpd56hthp0mI7/KSF8uXoO/HVwQaOZmr23X+BuQUnDenUQAZfKjiO4mghqVCN fOKugX74G8v0WUA00c+e0kVPGwHBxNmCIBd67a28G+upshB8MHPLZEKvEUtLrEZqbmsTw/ likvbx0+Ni6H3F1n/glF4N4zuLGVMx8SDZ4mRZ0JVyt8ln3wRIWj+efmrLIegA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1635691212; a=rsa-sha256; cv=none; b=hisp71CnisZiDTBbCahnqxKqZPfvwswJjMZDtx7p/69yqivVXi3JO7KPOyEpTYxNhiVtwq Mei3qFs4yI05PIyoC2QcieIGh8jSA7YIpsiP1qYyTrMtH+Gikb/HDKUsZtVp8n49OeFoJp MGl96bJKCUJWLMLKmXtLrtqOVZxy98Bxl0XFVoq0jDvsq5YFWnlN/HKUr64nqbZjhjYvAj uhyD+bHnVrp6CIQ4zBgczvkZNXM91fwVEN+g5uPFtQOZPCBBKSJK+S/9hAajWB7gGXJgqy mMONygWbmYt/LtFpEu0GOHXJA/32ePIrgnsEfpyBh9EQN+C5Ujqt03GwgxLi0A== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=LDsR3QEc; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: 0.18 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=LDsR3QEc; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: EFECABF97 X-Spam-Score: 0.18 X-Migadu-Scanner: scn0.migadu.com X-TUID: H/lgyq6tv8+1 Hi, Am Sonntag, den 31.10.2021, 15:07 +0100 schrieb Ludovic Courtès: > What do you mean? When doing VM testing, I regularly do ‘guix gc -D > /gnu/store/…-disk-image’ precisely to save space. Fortunately it > does have the intended effect of freeing a bunch of GiBs. Fair enough, different strokes and all. > > Plus, you could invoke it like > > > > guix gc -D dead-item dead-item live-item dead-item > > > > It would fail at live-item and then not continue to free the links > > of the two dead items prior. > > Yes, and that’s annoying, but it’s unrelated. :-) > > > So there's a few things we could do here: > > > > 1. simply fail and have the user deal with it (including the option > > of doing a normal `guix gc' or `guix gc -C 1') > > 2. remember which paths were live and dead and always clean up the > > links, only reporting errors afterwards > > 3. add an option to explicitly check the .links directory (which > > defaults to true for the current things, but could also be used to > > clean links after a liveness check or after a do-nothing `guix gc > > -F'). > > 4. ... > > You seem to be proposing to remove ‘-D’ altogether. I wrote no such thing. Obviously there needs to be a way of removing single items from the store, but what else to do is not so clear. It is only obvious that traversing all of .links is too expensive. > I agree it has the shortcomings you write, but I think it’s > occasionally useful nonetheless. > > My proposal would be either the status quo, or removing just the one > link that matters from /gnu/store/.links upon ‘-D’. > > Thoughts? There isn't "just the one link that matters" when it comes to removing multiple items -- heck, even if tasked to free up just 5MB rather than all of the garbage, traversing all .links is probably too expensive. Accepting that we might want to always delete links at the end, I think `guix gc' needs a way to record which links had their count go to 1 during garbage deletion, so that when it comes to deleting links only those are tried (and of course checked again to make sure their count is indeed still 1). This should be the preferred mode if less than some arbitrary large number of store items are affected (let's say 1024 or some multiple of it) or a total cleaning of .links has been forced. Though perhaps there's a way to do this without manual recording. Let's say we notice a link count going to one as we clear the trash. We could just add the link behind it to the trash right away to ensure that it is not reused by something else and then clean the trash a second time (we would have to check for potential race conditions in this case). WDYT? Liliana