From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id WMiWOVYR2l4/UwAA0tVLHw (envelope-from ) for ; Fri, 05 Jun 2020 09:33:10 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id kKmBNVYR2l4mMQAA1q6Kng (envelope-from ) for ; Fri, 05 Jun 2020 09:33:10 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 59F809401CD for ; Fri, 5 Jun 2020 09:33:10 +0000 (UTC) Received: from localhost ([::1]:39540 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jh8iq-0008Rm-RT for larch@yhetil.org; Fri, 05 Jun 2020 05:33:08 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39650) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jh8ik-0008Re-A2 for bug-guix@gnu.org; Fri, 05 Jun 2020 05:33:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:36790) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jh8ik-0004VP-0Y for bug-guix@gnu.org; Fri, 05 Jun 2020 05:33:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jh8ij-0004Bu-Sa for bug-guix@gnu.org; Fri, 05 Jun 2020 05:33:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#41607: Deleted store items are not actually deleted Resent-From: Christopher Marusich Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Fri, 05 Jun 2020 09:33:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 41607 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Chris Marusich Received: via spool by 41607-submit@debbugs.gnu.org id=B41607.159134955616075 (code B ref 41607); Fri, 05 Jun 2020 09:33:01 +0000 Received: (at 41607) by debbugs.gnu.org; 5 Jun 2020 09:32:36 +0000 Received: from localhost ([127.0.0.1]:48336 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jh8iK-0004BD-Bu for submit@debbugs.gnu.org; Fri, 05 Jun 2020 05:32:36 -0400 Received: from mail-pg1-f173.google.com ([209.85.215.173]:34524) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jh8iJ-0004Ax-Bj for 41607@debbugs.gnu.org; Fri, 05 Jun 2020 05:32:35 -0400 Received: by mail-pg1-f173.google.com with SMTP id m1so4915504pgk.1 for <41607@debbugs.gnu.org>; Fri, 05 Jun 2020 02:32:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=g6Cz7RI1FbWUBQKEjMCGt4Pkf/fTsuxOy/qCZ2UE62U=; b=KVXKndWZeCb4ktqI0PMtO85gWSSfS2UVaTTwRC1IQL4aHtgWSV15eG+ytjYtIKg9ML UPVn6vjzg0o+AN6dHMP6gt1njAH2yUuwJVfrWEeQHljlpQLr9Gns5HYkdqPJeZ+kxQGW PEBJAMsE4ONDWEwKZzrH0ihGKQ817NS+Z4a7ECchttmm2JAt9rIgb20zA432C1hRkMHC 5gLo7l9PyzLYiFlKvRtu9Lewr8h6BCCOU8r/e62w2LoxCduXdRIttqvGb3qw0t7ODnw4 6lsT2nwYEHcVz++Jvpb2RW7tuOhCuQTcRAfVB7LgQt80FetubDU2eJ8/CXO4mcPc3nmE mBAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=g6Cz7RI1FbWUBQKEjMCGt4Pkf/fTsuxOy/qCZ2UE62U=; b=B0FZ1Nuj8AsCNbpNzU4ScQ7dexWs992auIPmcO+5ujUNEYyQjsG5wgv5ageg47ZgxE 8EcyZMjc/EACVha5mJkye/GoPLE1rUtNCJNCqB1h2rQFHfNJEAXtyUejwaOyjvGV5ady SzdA7cch959WHmVgU8XKiT26fYxdMHKDHOtD0P7sMwQctSrzeyWBGlrApDLXYitBveJm d5ArgQJNMVfIsvMqlzDk6qSCFtH3OeBBFIjnSnjgucF7HpdwuGyFvc3b7pXpFUSOlMi2 tgxcNaOs2Tq+5Mmt4D7RPInLmrU0935RzRSzEDQQHEv1xfNmLcr5xHAOO7y0cTEiRLum g+ow== X-Gm-Message-State: AOAM533ZZkXteHSnWpdHRM4GKGaFmhXJ014RnGOyw/k/fupVtRb/f5v9 3a8xFVFjCRuRfsnUiF96xpA= X-Google-Smtp-Source: ABdhPJw3uv0V+JTEH4tyB1k8cHHSfpMVDImMhT4wIjMOYMSNCoCELUXk62+JHTk5gl2Wh4ZGrcBjaQ== X-Received: by 2002:a65:6795:: with SMTP id e21mr8786912pgr.282.1591349549123; Fri, 05 Jun 2020 02:32:29 -0700 (PDT) Received: from garuda-lan (c-73-97-103-127.hsd1.wa.comcast.net. [73.97.103.127]) by smtp.gmail.com with ESMTPSA id b140sm6731895pfb.119.2020.06.05.02.32.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 05 Jun 2020 02:32:27 -0700 (PDT) From: Christopher Marusich X-Google-Original-From: Christopher Marusich References: <20200528181043.GC23745@jasmine.lan> <20200529170820.GA30828@jasmine.lan> <20200529180245.GA3754@jasmine.lan> <20200529190942.GA8440@jasmine.lan> <87r1v0k8hl.fsf@gmail.com> <87eeqvcaau.fsf@gnu.org> <87r1uu1x9e.fsf@gmail.com> Date: Fri, 05 Jun 2020 02:32:20 -0700 In-Reply-To: <87r1uu1x9e.fsf@gmail.com> (Chris Marusich's message of "Thu, 04 Jun 2020 11:50:05 -0700") Message-ID: <87pnad6eor.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.0 (-) X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 41607@debbugs.gnu.org, Stephen Scheck Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=fail (rsa verify failed) header.d=gmail.com header.s=20161025 header.b=KVXKndWZ; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Spam-Score: -2.01 X-TUID: Xdu9aYlvMsi7 --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Chris Marusich writes: > Ludovic Court=C3=A8s writes: > >>> Should Guix do anything about this? We could change guix-daemon to take >>> correct action in the face of an XDEV error. We could also improve the >>> logging, since currently it silently swallows the XDEV error. >> >> I guess we could delete recursively right away upon EXDEV. It should be >> just two lines of code, right? > > I'll try making the change and report back. Yes, there are other cases > where we immediately delete without moving into the trash directory > (e.g., when the trash directory fails to be created), so it seems OK. Here is a patch. Turns out it's was just a one line change! If nobody has any further feedback on it, I'll go ahead and merge it to the master branch in the next couple days. I tested it in one of the Docker containers provided by Stephen which exhibited the problem. I built the new Guix inside the Docker container and verified that a path which was previously unable to be GC'd due to the EXDEV error, was now able to be successfully GC'd. My understanding is that the only reason why the guix-daemon attempts to move dead directories to the trash directory is to save time on deleting, since large directories could take a while to fully delete. If there is any reason why it might be unsafe to delete the directories directly in case of EXDEV (I cannot think of any), please let me know. =2D-=20 Chris --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-daemon-Handle-EXDEV-when-moving-to-trash-directory.patch Content-Transfer-Encoding: quoted-printable From=20505481a6a22819a42320f693988c3f8e13ded080 Mon Sep 17 00:00:00 2001 From: Chris Marusich Date: Thu, 4 Jun 2020 23:26:19 -0700 Subject: [PATCH] daemon: Handle EXDEV when moving to trash directory. Fixes . Reported by Stephen Scheck . * nix/libstore/gc.cc (LocalStore::deletePathRecursive): When we try to move a dead directory into the trashDir using rename(2) but it returns an EXDEV error, just delete the directory instead. This can happen in a Docker container when the directory is not on the "top layer". =2D-- nix/libstore/gc.cc | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/nix/libstore/gc.cc b/nix/libstore/gc.cc index 8bc4e01eb0..845fe278c7 100644 =2D-- a/nix/libstore/gc.cc +++ b/nix/libstore/gc.cc @@ -455,7 +455,10 @@ void LocalStore::deletePathRecursive(GCState & state, = const Path & path) throw SysError(format("unable to rename `%1%' to `%2%'") %= path % tmp); state.bytesInvalidated +=3D size; } catch (SysError & e) { =2D if (e.errNo =3D=3D ENOSPC) { + // In a Docker container, rename(2) returns EXDEV when the sou= rce + // and destination are not both on the "top layer". See: + // https://bugs.gnu.org/41607 + if (e.errNo =3D=3D ENOSPC || e.errNo =3D=3D EXDEV) { printMsg(lvlInfo, format("note: can't create move `%1%': %= 2%") % path % e.msg()); deleteGarbage(state, path); } =2D-=20 2.26.2 --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAl7aESQACgkQ3UCaFdgi Rp30yw//Vw6SV8tFac2hCw6CHi8vg8IqjNZdGEq+9sXztSSopCib7KQnoOT9LfqS uGfwQ7yZEeTcQV6H6dOqHB7PmsyljHwIlMeEXDk4v5hCprxdfEftPO0y71BXJdYW B4VgDYuqp2GbLzwV9UCSLLpddOamUZK5Rt/fFsFmx55oheBt5s5I+hExGbb/HxT4 bC9XZhWQpMrJn9DtWvWZ2Fujgy9cEBi7gqXbsJ7XrENU4WXwfgrRgLVxKi1iyJWc LxVFXBY1D1Ob4mvK+mpuWdmvuYMoMHVbxP/so+fbEGcXvvopmYUlarfF+kq8IqLJ +Il7wuuqYKb3u2oDkLZCJ6yrCHeox6/aaK9xB89Lz1DRIHJ29U7T+JlmNFckJ/0+ Vw5BYvLcX1JbJGhyPSEt16gkSRgjT2KA3vqs9Nd9iFZaec5sN+4DqBUGxuJXr718 UCCCe26dKo6sIzzOzkEoD/BAt5k79LusNzNmh7DSsq7ANY1WkdPNHdhM0iMmqdfL 3WOiOtsDq2Cre6BnFJx3wm1d0LZAB/nvvJRDfJ0oiUYo6EzBVydJd+txdmEkq5VG +QDY8ktJ+PHdUMRs2DjqlK5oGoHFfXxIVJ6Qi960aToMWJa+SGxEoe7FJWqI4YpJ mVZGWl46A7eLjjLvTwn8XTK9iFUKjliJJn8ru1C1d8Xftcvq05U= =Jycu -----END PGP SIGNATURE----- --==-=-=--