* bug#41607: Deleted store items are not actually deleted [not found] ` <CAKjnHz0Ce=w0DZPi6xE6GNQ4-ezmvjUr+5FqxsvCBdx4SU+VsQ@mail.gmail.com> @ 2020-05-29 19:09 ` Leo Famulari 2020-05-31 4:56 ` Chris Marusich 0 siblings, 1 reply; 13+ messages in thread From: Leo Famulari @ 2020-05-29 19:09 UTC (permalink / raw) To: Stephen Scheck; +Cc: 41607 From the discussion "Guix Docker image inflation" [0] on help-guix: On Fri, May 29, 2020 at 02:29:53PM -0400, Stephen Scheck wrote: > # Now try to delete it... > root@localhost /gnu/store# guix gc --delete > /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules > finding garbage collector roots... > [0 MiB] deleting > '/gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules' > deleting `/gnu/store/trash' > deleting unused links... > note: currently hard linking saves 1181.36 MiB > > # Still there... > root@localhost /gnu/store# du -hs > /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules > 210M /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules Okay, something is definitely not right. [0] https://lists.gnu.org/archive/html/help-guix/2020-05/msg00235.html ^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#41607: Deleted store items are not actually deleted 2020-05-29 19:09 ` bug#41607: Deleted store items are not actually deleted Leo Famulari @ 2020-05-31 4:56 ` Chris Marusich 2020-05-31 9:27 ` zimoun 2020-06-04 11:58 ` Ludovic Courtès 0 siblings, 2 replies; 13+ messages in thread From: Chris Marusich @ 2020-05-31 4:56 UTC (permalink / raw) To: Leo Famulari; +Cc: 41607, Stephen Scheck [-- Attachment #1: Type: text/plain, Size: 4298 bytes --] Hi Leo, Stephen, and others, I originally wrote a very detailed email describing my investigation of this issue and the results. However, I accidentally deleted it, so please bear with me as I write a rough summary instead. Leo Famulari <leo@famulari.name> writes: >>From the discussion "Guix Docker image inflation" [0] on help-guix: > > On Fri, May 29, 2020 at 02:29:53PM -0400, Stephen Scheck wrote: >> # Now try to delete it... >> root@localhost /gnu/store# guix gc --delete >> /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules >> finding garbage collector roots... >> [0 MiB] deleting >> '/gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules' >> deleting `/gnu/store/trash' >> deleting unused links... >> note: currently hard linking saves 1181.36 MiB >> >> # Still there... >> root@localhost /gnu/store# du -hs >> /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules >> 210M /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules > > Okay, something is definitely not right. > > [0] https://lists.gnu.org/archive/html/help-guix/2020-05/msg00235.html There are two problems. One is that Stephen's images are getting larger every day. The other is that Guix is failing to GC dead store items in the Docker container. The latter does not cause the former. The reason Guix is failing to GC dead items in the Docker container is because those dead items are not on the "top layer", so Docker returns an EXDEV error: https://docs.docker.com/storage/storagedriver/overlayfs-driver/ "Renaming directories: Calling rename(2) for a directory is allowed only when both the source and the destination path are on the top layer. Otherwise, it returns EXDEV error ('cross-device link not permitted'). Your application needs to be designed to handle EXDEV and fall back to a 'copy and unlink' strategy." You can observe this by running guix-daemon with strace in the container, and watching what happens when you try to delete one of the offending store items (make sure it is a directory). For example: --8<---------------cut here---------------start------------->8--- 685 rename("/gnu/store/xib50iqk3w1gw9l770mad59m9bi3bcpc-manual-database", "/gnu/store/trash/xib50iqk3w1gw9l770mad59m9bi3bcpc-manual-database") = -1 EXDEV (Invalid cross-device link) --8<---------------cut here---------------end--------------->8--- In most cases, when guix-daemon GC's a dead directory, it does this (see: nix/libstore/gc.cc): - Create a trash directory (usually /gnu/store/trash) - Move dead directories into the trash directory. - Delete the trash directory. The trash directory is on the "top layer" because it gets created in the running container. However, in practice many store items from lower layers are made dead when Stephen's script runs "guix pull" and deletes the old profiles. If any of those store items were directories, guix-daemon will fail to GC them because of an XDEV error. If this is confusing to you, I suggest you experiment with Docker a little bit, and look closely at the steps that Stephen's script is running. I outlined this in the email I accidentally deleted, but I'm a little too tired to reproduce it all a second time. I hope you'll understand. Should Guix do anything about this? We could change guix-daemon to take correct action in the face of an XDEV error. We could also improve the logging, since currently it silently swallows the XDEV error. However, even if we make those changes and Guix is able to GC the dead store items, it would not prevent Stephen's images from growing in size without bound. There would still be many store items that came from a prior image (i.e., a lower layer), became dead after running "guix pull" and deleting old profile generations, and still exist in the prior layer, even though they are not visible in the running container. This is due to Docker's design, in which the visible file system is the result of stitching together all the layers with overlayfs. To work around the issue, Stephen can build the images from the same base image, rather than daisy-chaining new images from old ones. That way, they would not accumulate layers without bound. -- Chris [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#41607: Deleted store items are not actually deleted 2020-05-31 4:56 ` Chris Marusich @ 2020-05-31 9:27 ` zimoun 2020-06-04 11:58 ` Ludovic Courtès 1 sibling, 0 replies; 13+ messages in thread From: zimoun @ 2020-05-31 9:27 UTC (permalink / raw) To: Chris Marusich; +Cc: 41607, Stephen Scheck Hi Chris, Thank you for the detailed explanations. On Sun, 31 May 2020 at 06:57, Chris Marusich <cmmarusich@gmail.com> wrote: > The trash directory is on the "top layer" because it gets created in the > running container. However, in practice many store items from lower > layers are made dead when Stephen's script runs "guix pull" and deletes > the old profiles. If any of those store items were directories, > guix-daemon will fail to GC them because of an XDEV error. If this is > confusing to you, I suggest you experiment with Docker a little bit, and > look closely at the steps that Stephen's script is running. I outlined > this in the email I accidentally deleted, but I'm a little too tired to > reproduce it all a second time. I hope you'll understand. > > Should Guix do anything about this? We could change guix-daemon to take > correct action in the face of an XDEV error. We could also improve the > logging, since currently it silently swallows the XDEV error. What is the correct action? Because somehow these items cannot be removed of the Docker chained images. However, the logging could report the ``error", e.g., "guix gc" lists all the items and their size and currently it says "[0MiB] deleting" so instead it could say "[XDEV error] not valid" with some words about that in the manual, section "Invking guix gc". WDYT? Cheers, simon ^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#41607: Deleted store items are not actually deleted 2020-05-31 4:56 ` Chris Marusich 2020-05-31 9:27 ` zimoun @ 2020-06-04 11:58 ` Ludovic Courtès 2020-06-04 18:50 ` Chris Marusich 1 sibling, 1 reply; 13+ messages in thread From: Ludovic Courtès @ 2020-06-04 11:58 UTC (permalink / raw) To: Chris Marusich; +Cc: 41607, Stephen Scheck Hi, Chris Marusich <cmmarusich@gmail.com> skribis: > The reason Guix is failing to GC dead items in the Docker container is > because those dead items are not on the "top layer", so Docker returns > an EXDEV error: > > https://docs.docker.com/storage/storagedriver/overlayfs-driver/ > > "Renaming directories: Calling rename(2) for a directory is allowed only > when both the source and the destination path are on the top > layer. Otherwise, it returns EXDEV error ('cross-device link not > permitted'). Your application needs to be designed to handle EXDEV and > fall back to a 'copy and unlink' strategy." > > You can observe this by running guix-daemon with strace in the > container, and watching what happens when you try to delete one of the > offending store items (make sure it is a directory). For example: > > 685 rename("/gnu/store/xib50iqk3w1gw9l770mad59m9bi3bcpc-manual-database", "/gnu/store/trash/xib50iqk3w1gw9l770mad59m9bi3bcpc-manual-database") = -1 EXDEV (Invalid cross-device link) > > In most cases, when guix-daemon GC's a dead directory, it does this > (see: nix/libstore/gc.cc): > > - Create a trash directory (usually /gnu/store/trash) > - Move dead directories into the trash directory. > - Delete the trash directory. > > The trash directory is on the "top layer" because it gets created in the > running container. However, in practice many store items from lower > layers are made dead when Stephen's script runs "guix pull" and deletes > the old profiles. If any of those store items were directories, > guix-daemon will fail to GC them because of an XDEV error. If this is > confusing to you, I suggest you experiment with Docker a little bit, and > look closely at the steps that Stephen's script is running. I outlined > this in the email I accidentally deleted, but I'm a little too tired to > reproduce it all a second time. I hope you'll understand. Interesting, thanks for the analysis! > Should Guix do anything about this? We could change guix-daemon to take > correct action in the face of an XDEV error. We could also improve the > logging, since currently it silently swallows the XDEV error. I guess we could delete recursively right away upon EXDEV. It should be just two lines of code, right? > To work around the issue, Stephen can build the images from the same > base image, rather than daisy-chaining new images from old ones. That > way, they would not accumulate layers without bound. Maybe that too. Thanks, Ludo’. ^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#41607: Deleted store items are not actually deleted 2020-06-04 11:58 ` Ludovic Courtès @ 2020-06-04 18:50 ` Chris Marusich 2020-06-05 9:32 ` Christopher Marusich 0 siblings, 1 reply; 13+ messages in thread From: Chris Marusich @ 2020-06-04 18:50 UTC (permalink / raw) To: Ludovic Courtès; +Cc: 41607, Stephen Scheck [-- Attachment #1: Type: text/plain, Size: 604 bytes --] Ludovic Courtès <ludo@gnu.org> writes: >> Should Guix do anything about this? We could change guix-daemon to take >> correct action in the face of an XDEV error. We could also improve the >> logging, since currently it silently swallows the XDEV error. > > I guess we could delete recursively right away upon EXDEV. It should be > just two lines of code, right? I'll try making the change and report back. Yes, there are other cases where we immediately delete without moving into the trash directory (e.g., when the trash directory fails to be created), so it seems OK. -- Chris [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#41607: Deleted store items are not actually deleted 2020-06-04 18:50 ` Chris Marusich @ 2020-06-05 9:32 ` Christopher Marusich 2020-06-05 9:36 ` zimoun ` (2 more replies) 0 siblings, 3 replies; 13+ messages in thread From: Christopher Marusich @ 2020-06-05 9:32 UTC (permalink / raw) To: Chris Marusich; +Cc: 41607, Stephen Scheck [-- Attachment #1.1: Type: text/plain, Size: 1483 bytes --] Chris Marusich <cmmarusich@gmail.com> writes: > Ludovic Courtès <ludo@gnu.org> writes: > >>> Should Guix do anything about this? We could change guix-daemon to take >>> correct action in the face of an XDEV error. We could also improve the >>> logging, since currently it silently swallows the XDEV error. >> >> I guess we could delete recursively right away upon EXDEV. It should be >> just two lines of code, right? > > I'll try making the change and report back. Yes, there are other cases > where we immediately delete without moving into the trash directory > (e.g., when the trash directory fails to be created), so it seems OK. Here is a patch. Turns out it's was just a one line change! If nobody has any further feedback on it, I'll go ahead and merge it to the master branch in the next couple days. I tested it in one of the Docker containers provided by Stephen which exhibited the problem. I built the new Guix inside the Docker container and verified that a path which was previously unable to be GC'd due to the EXDEV error, was now able to be successfully GC'd. My understanding is that the only reason why the guix-daemon attempts to move dead directories to the trash directory is to save time on deleting, since large directories could take a while to fully delete. If there is any reason why it might be unsafe to delete the directories directly in case of EXDEV (I cannot think of any), please let me know. -- Chris [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1.2: 0001-daemon-Handle-EXDEV-when-moving-to-trash-directory.patch --] [-- Type: text/x-patch, Size: 1572 bytes --] From 505481a6a22819a42320f693988c3f8e13ded080 Mon Sep 17 00:00:00 2001 From: Chris Marusich <cmmarusich@gmail.com> Date: Thu, 4 Jun 2020 23:26:19 -0700 Subject: [PATCH] daemon: Handle EXDEV when moving to trash directory. Fixes <https://bugs.gnu.org/41607>. Reported by Stephen Scheck <singularsyntax@gmail.com>. * nix/libstore/gc.cc (LocalStore::deletePathRecursive): When we try to move a dead directory into the trashDir using rename(2) but it returns an EXDEV error, just delete the directory instead. This can happen in a Docker container when the directory is not on the "top layer". --- nix/libstore/gc.cc | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/nix/libstore/gc.cc b/nix/libstore/gc.cc index 8bc4e01eb0..845fe278c7 100644 --- a/nix/libstore/gc.cc +++ b/nix/libstore/gc.cc @@ -455,7 +455,10 @@ void LocalStore::deletePathRecursive(GCState & state, const Path & path) throw SysError(format("unable to rename `%1%' to `%2%'") % path % tmp); state.bytesInvalidated += size; } catch (SysError & e) { - if (e.errNo == ENOSPC) { + // In a Docker container, rename(2) returns EXDEV when the source + // and destination are not both on the "top layer". See: + // https://bugs.gnu.org/41607 + if (e.errNo == ENOSPC || e.errNo == EXDEV) { printMsg(lvlInfo, format("note: can't create move `%1%': %2%") % path % e.msg()); deleteGarbage(state, path); } -- 2.26.2 [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply related [flat|nested] 13+ messages in thread
* bug#41607: Deleted store items are not actually deleted 2020-06-05 9:32 ` Christopher Marusich @ 2020-06-05 9:36 ` zimoun 2020-06-05 16:05 ` Stephen Scheck 2020-06-05 16:21 ` Ludovic Courtès 2 siblings, 0 replies; 13+ messages in thread From: zimoun @ 2020-06-05 9:36 UTC (permalink / raw) To: Christopher Marusich; +Cc: 41607, Stephen Scheck Hi Chris, On Fri, 5 Jun 2020 at 11:33, Christopher Marusich <cmmarusich@gmail.com> wrote: > Chris Marusich <cmmarusich@gmail.com> writes: > > Ludovic Courtès <ludo@gnu.org> writes: > Here is a patch. Turns out it's was just a one line change! If nobody > has any further feedback on it, I'll go ahead and merge it to the master > branch in the next couple days. Really cool! How can I test it? All the best, simon ^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#41607: Deleted store items are not actually deleted 2020-06-05 9:32 ` Christopher Marusich 2020-06-05 9:36 ` zimoun @ 2020-06-05 16:05 ` Stephen Scheck 2020-06-05 16:21 ` Ludovic Courtès 2 siblings, 0 replies; 13+ messages in thread From: Stephen Scheck @ 2020-06-05 16:05 UTC (permalink / raw) To: Christopher Marusich; +Cc: 41607 [-- Attachment #1: Type: text/plain, Size: 2764 bytes --] Very cool - thanks Chris! In the meantime, I've updated my build script to externalize the Guix environment from the Docker container. So far the daily builds are staying nice and small at ~197MB and one layer. The images and updated script are here if anybody is curious: https://hub.docker.com/r/singularsyntax/guix/tags https://gitlab.com/singularsyntax-docker-hub/guix/-/blob/master/.gitlab-ci.yml GitLab allows you to cache files between job stages and even full pipeline runs. I first tried to cache /var/guix and /gnu/store and mount them inside a container in which to perform `guix pull` etc. However, it seems that handling hard links on externally mounted file systems from within a container is problematic. I think passing `--disable-deduplication` to guix-daemon might resolve it, but I couldn't figure out where/how to change the Shepherd configuration to do that. So instead, I just copy the directories into and out of the container at start and end of the process. It seems to work. Mounting would probably speed up the process quite a bit if I could make it work. Cheers, -Steve On Fri, Jun 5, 2020 at 5:32 AM Christopher Marusich <cmmarusich@gmail.com> wrote: > Chris Marusich <cmmarusich@gmail.com> writes: > > > Ludovic Courtès <ludo@gnu.org> writes: > > > >>> Should Guix do anything about this? We could change guix-daemon to > take > >>> correct action in the face of an XDEV error. We could also improve the > >>> logging, since currently it silently swallows the XDEV error. > >> > >> I guess we could delete recursively right away upon EXDEV. It should be > >> just two lines of code, right? > > > > I'll try making the change and report back. Yes, there are other cases > > where we immediately delete without moving into the trash directory > > (e.g., when the trash directory fails to be created), so it seems OK. > > Here is a patch. Turns out it's was just a one line change! If nobody > has any further feedback on it, I'll go ahead and merge it to the master > branch in the next couple days. > > I tested it in one of the Docker containers provided by Stephen which > exhibited the problem. I built the new Guix inside the Docker container > and verified that a path which was previously unable to be GC'd due to > the EXDEV error, was now able to be successfully GC'd. > > My understanding is that the only reason why the guix-daemon attempts to > move dead directories to the trash directory is to save time on > deleting, since large directories could take a while to fully delete. > If there is any reason why it might be unsafe to delete the directories > directly in case of EXDEV (I cannot think of any), please let me know. > > -- > Chris > [-- Attachment #2: Type: text/html, Size: 3642 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#41607: Deleted store items are not actually deleted 2020-06-05 9:32 ` Christopher Marusich 2020-06-05 9:36 ` zimoun 2020-06-05 16:05 ` Stephen Scheck @ 2020-06-05 16:21 ` Ludovic Courtès 2020-06-07 1:31 ` Chris Marusich 2 siblings, 1 reply; 13+ messages in thread From: Ludovic Courtès @ 2020-06-05 16:21 UTC (permalink / raw) To: Christopher Marusich; +Cc: 41607, Stephen Scheck Howdy! Christopher Marusich <cmmarusich@gmail.com> skribis: > Here is a patch. Turns out it's was just a one line change! If nobody > has any further feedback on it, I'll go ahead and merge it to the master > branch in the next couple days. Yay! > From 505481a6a22819a42320f693988c3f8e13ded080 Mon Sep 17 00:00:00 2001 > From: Chris Marusich <cmmarusich@gmail.com> > Date: Thu, 4 Jun 2020 23:26:19 -0700 > Subject: [PATCH] daemon: Handle EXDEV when moving to trash directory. > > Fixes <https://bugs.gnu.org/41607>. > Reported by Stephen Scheck <singularsyntax@gmail.com>. > > * nix/libstore/gc.cc (LocalStore::deletePathRecursive): When we try to > move a dead directory into the trashDir using rename(2) but it returns > an EXDEV error, just delete the directory instead. This can happen in a > Docker container when the directory is not on the "top layer". > --- > nix/libstore/gc.cc | 5 ++++- > 1 file changed, 4 insertions(+), 1 deletion(-) > > diff --git a/nix/libstore/gc.cc b/nix/libstore/gc.cc > index 8bc4e01eb0..845fe278c7 100644 > --- a/nix/libstore/gc.cc > +++ b/nix/libstore/gc.cc > @@ -455,7 +455,10 @@ void LocalStore::deletePathRecursive(GCState & state, const Path & path) > throw SysError(format("unable to rename `%1%' to `%2%'") % path % tmp); > state.bytesInvalidated += size; > } catch (SysError & e) { > - if (e.errNo == ENOSPC) { > + // In a Docker container, rename(2) returns EXDEV when the source > + // and destination are not both on the "top layer". See: > + // https://bugs.gnu.org/41607 > + if (e.errNo == ENOSPC || e.errNo == EXDEV) { > printMsg(lvlInfo, format("note: can't create move `%1%': %2%") % path % e.msg()); > deleteGarbage(state, path); > } For consistency with (most) of the code, I’d suggest a /* */ comment. Otherwise LGTM, thank you! Ludo’. ^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#41607: Deleted store items are not actually deleted 2020-06-05 16:21 ` Ludovic Courtès @ 2020-06-07 1:31 ` Chris Marusich 2020-06-07 10:07 ` zimoun 0 siblings, 1 reply; 13+ messages in thread From: Chris Marusich @ 2020-06-07 1:31 UTC (permalink / raw) To: Ludovic Courtès, Stephen Scheck, zimoun; +Cc: 41607-close [-- Attachment #1: Type: text/plain, Size: 1016 bytes --] Hi everyone, I have committed the fix in d445c30ea6 and updated the guix package definition in ecbde6505c to ensure that the next time "guix pull" is run, the new guix-daemon version will be used. Ludovic Courtès <ludo@gnu.org> writes: > For consistency with (most) of the code, I’d suggest a /* */ comment. OK - I did this in d445c30ea6. Stephen Scheck <singularsyntax@gmail.com> writes: > I've updated my build script to externalize the Guix environment from > the Docker container. So far the daily builds are staying nice and > small at ~197MB and one layer. The images and updated script are here > if anybody is curious: > > https://hub.docker.com/r/singularsyntax/guix/tags > https://gitlab.com/singularsyntax-docker-hub/guix/-/blob/master/.gitlab-ci.yml Neat! I'm glad you were figure out a way to to keep the image from growing in size. It's good to know of a way to do that. Thank you for your help and have a nice day! I'm closing this bug report. -- Chris [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#41607: Deleted store items are not actually deleted 2020-06-07 1:31 ` Chris Marusich @ 2020-06-07 10:07 ` zimoun 2020-06-08 7:43 ` Chris Marusich 0 siblings, 1 reply; 13+ messages in thread From: zimoun @ 2020-06-07 10:07 UTC (permalink / raw) To: Chris Marusich; +Cc: 41607-close, Stephen Scheck Hi Chris, Thank you. On Sun, 7 Jun 2020 at 03:31, Chris Marusich <cmmarusich@gmail.com> wrote: > I have committed the fix in d445c30ea6 and updated the guix package > definition in ecbde6505c to ensure that the next time "guix pull" is > run, the new guix-daemon version will be used. Sorry to be late, I have things like that: -8<---------------cut here---------------start------------->8--- [548 MiB] deleting '/gnu/store/dkzivzn17qilmqdfpyps62b395wxhshh-openssl-1.1.1f' note: can't create move `/gnu/store/dkzivzn17qilmqdfpyps62b395wxhshh-openssl-1.1 unable to rename `/gnu/store/dkzivzn17qilmqdfpyps62b395wxhshh-openssl-1.1.1f' to `/gnu/store/trash/dkzivzn17qilmqdfpyps62b395wxhshh-openssl-1.1.1f': Invalid cross-device link [553 MiB] deleting '/gnu/store/yaajjri3cks8rhkb29z08d3q5waww0dx-packages.scm' --8<---------------cut here---------------end--------------->8--- I should do something wrong... All the best, simon ^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#41607: Deleted store items are not actually deleted 2020-06-07 10:07 ` zimoun @ 2020-06-08 7:43 ` Chris Marusich 0 siblings, 0 replies; 13+ messages in thread From: Chris Marusich @ 2020-06-08 7:43 UTC (permalink / raw) To: zimoun; +Cc: 41607-close, 41607, Stephen Scheck [-- Attachment #1: Type: text/plain, Size: 1296 bytes --] Hi zimoun, zimoun <zimon.toutoune@gmail.com> writes: > On Sun, 7 Jun 2020 at 03:31, Chris Marusich <cmmarusich@gmail.com> wrote: > >> I have committed the fix in d445c30ea6 and updated the guix package >> definition in ecbde6505c to ensure that the next time "guix pull" is >> run, the new guix-daemon version will be used. > > Sorry to be late, I have things like that: > > [548 MiB] deleting '/gnu/store/dkzivzn17qilmqdfpyps62b395wxhshh-openssl-1.1.1f' > note: can't create move > `/gnu/store/dkzivzn17qilmqdfpyps62b395wxhshh-openssl-1.1 unable to > rename `/gnu/store/dkzivzn17qilmqdfpyps62b395wxhshh-openssl-1.1.1f' to > `/gnu/store/trash/dkzivzn17qilmqdfpyps62b395wxhshh-openssl-1.1.1f': > Invalid cross-device link > [553 MiB] deleting '/gnu/store/yaajjri3cks8rhkb29z08d3q5waww0dx-packages.scm' That looks normal to me. Previously, the cross-device link error would be silently ignored and the directory would not be deleted - no message would be printed about the cross-device link error. Now, however, the cross-device link error is reported, and the directory is deleted. If you find that the directory is not actually deleted, please let me know so I can look into it more. When I tested it on my end, the directory was in fact deleted. -- Chris [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
[parent not found: <CAJ3okZ0vPpPnDQavrpufPXmSZWk2h+oN2SR7CDVwaEhg-h4h6g@mail.gmail.com>]
[parent not found: <CAKjnHz1kgwCOia5_hVYkuwaoOiJ55cATaF-Ed6_RsS-r5AELHw@mail.gmail.com>]
[parent not found: <CAJ3okZ2MhALsxXvWGJccE8feERDf4SihtpF=68s-APQbqaUMRA@mail.gmail.com>]
[parent not found: <CAKjnHz33fQrKa3KoQsDWh0pv-SgGZwPnn1Wgu6Kvf=SDGb2dWQ@mail.gmail.com>]
[parent not found: <CAJ3okZ0C2_wECSG4jWq1+ZaO=vL5Zbvyo-2kStaK0cTyF3fXgA@mail.gmail.com>]
[parent not found: <CAKjnHz35PVGM4BcJUML8=hLxXbJG-hcOg8PeZFjqCoTXe7EueQ@mail.gmail.com>]
* bug#41607: Deleted store items are not actually deleted [not found] ` <CAKjnHz35PVGM4BcJUML8=hLxXbJG-hcOg8PeZFjqCoTXe7EueQ@mail.gmail.com> @ 2020-05-29 23:36 ` zimoun 0 siblings, 0 replies; 13+ messages in thread From: zimoun @ 2020-05-29 23:36 UTC (permalink / raw) To: Stephen Scheck, 41607 -help-guix On Sat, 30 May 2020 at 00:11, Stephen Scheck <singularsyntax@gmail.com> wrote: >> Do you have '/var/' in your Docker image? Because it looks like the same than: > If you'd like, you can fetch the exact same image and look around yourself: > > docker pull singularsyntax/guix:master-a5374cd # same as singularsyntax/guix:latest Thanks! I have already did with singularsyntax/guix-bootstrap-alpine but then I hit some issues... Well another story. :-) Why there is (in both images) 13 "layers"? > CONTAINER=`docker run --detach --tty --privileged singularsyntax/guix:master-a5374cd` > docker exec --interactive --tty $CONTAINER /run/current-system/profile/bin/bash --login --8<---------------cut here---------------start------------->8--- for f in $(guix gc --list-dead); do du -sh $f; done | sort -h [...] 103M /gnu/store/flcfw8pcs2s0wnm78lp1jwid5lglky3j-guix-packages-base 103M /gnu/store/hl8h9krwiqwn51gydc13jz4846gq55pr-guix-packages-base 106M /gnu/store/mnvkhj7crqcd0d1xa0zqa3hd1f5hmv83-guix-packages-base 107M /gnu/store/6sggbpgg0zkbgxwf3wa2j15dis8z7cr1-guix-packages-base 107M /gnu/store/m0fv2xmfif5pxnfb1bscfvgyfx0x6xdc-guix-packages-base 107M /gnu/store/n339sr8c63f0nzja6yl8zfwy1jklj19j-guix-packages-base 107M /gnu/store/plaay02w581vx9ilyiv93sl1lw54n7h5-guix-packages-base 205M /gnu/store/5mhn1ynxvy7jihsknsnv3yspkkvc0r5s-guix-2e59ae238-modules 205M /gnu/store/8z9qc2bvq8azc08p4miq77yf2agk07aq-guix-843e77205-modules 205M /gnu/store/brbwlbnx56ms50kklyqk9fsf0xkwjjf9-guix-498e2e669-modules 205M /gnu/store/d3h4b7nvnms8d03ddi9b481dlxpykl7l-guix-5e3d16994-modules 205M /gnu/store/ibgjq1ampj8bldrabbsnwik2sr0gg3as-guix-a43fe7acd-modules 205M /gnu/store/l3amdz5xyhflg5wdzlxr2685dq5glic2-guix-527ab3125-modules 210M /gnu/store/0vwg9aqzs5xrk10vcs4dl105s3f42ilf-guix-b1affd477-modules 210M /gnu/store/47aack48aczpzm635axsy4jf2pvmwrv0-guix-ef1d475b0-modules 210M /gnu/store/hz2rn2l0jixg91q4rsdcwc489y71ll29-guix-05e1edf22-modules 210M /gnu/store/mych9fchln22pbhpc5syxyymx4hz496y-guix-8bd0b533b-modules 210M /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules --8<---------------cut here---------------end--------------->8--- Well, there is 11 generations which corresponds to --8<---------------cut here---------------start------------->8--- guix describe Generation 12 May 29 2020 22:51:28 (current) guix a5374cd repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: a5374cde918cfeae5c16b43b9f2dd2b24bc3564d --8<---------------cut here---------------end--------------->8--- But "docker pull" downloaded 13 layers, well because these numbers are closed, is it meaningful? Really naive question. Other said, is it not something from the Docker side? Well, it is like Guix deletes something but this something is empty (from the Guix side): --8<---------------cut here---------------start------------->8--- guix gc -D /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules finding garbage collector roots... [0 MiB] deleting '/gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules' deleting `/gnu/store/trash' deleting unused links... note: currently hard linking saves 1181.36 MiB du -sh /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules 210M /gnu/store/x7ns2xcp8lfg24zq7gr3y8ffczn1nsxp-guix-d79c917f2-modules --8<---------------cut here---------------end--------------->8--- abd the "real" target is still something on the filesytem side (du -sh). Bah, I am not enough familiar with Docker... Good night, simon ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2020-06-08 7:44 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <CAKjnHz1X26QhPEWZfOmAbEmZHQA1qdU_Dq7U6kVXUqG7vCvuhw@mail.gmail.com> [not found] ` <20200528181043.GC23745@jasmine.lan> [not found] ` <CAKjnHz0txxc+Bdc5=4TLLcxSmRpf2sCJswu-E36UJKvQqBA2kQ@mail.gmail.com> [not found] ` <20200529170820.GA30828@jasmine.lan> [not found] ` <CAKjnHz1b1EUXszQtUVz9p8zWu+h0HtPfCzrOmrXO+eyhH+AvVw@mail.gmail.com> [not found] ` <20200529180245.GA3754@jasmine.lan> [not found] ` <CAKjnHz0Ce=w0DZPi6xE6GNQ4-ezmvjUr+5FqxsvCBdx4SU+VsQ@mail.gmail.com> 2020-05-29 19:09 ` bug#41607: Deleted store items are not actually deleted Leo Famulari 2020-05-31 4:56 ` Chris Marusich 2020-05-31 9:27 ` zimoun 2020-06-04 11:58 ` Ludovic Courtès 2020-06-04 18:50 ` Chris Marusich 2020-06-05 9:32 ` Christopher Marusich 2020-06-05 9:36 ` zimoun 2020-06-05 16:05 ` Stephen Scheck 2020-06-05 16:21 ` Ludovic Courtès 2020-06-07 1:31 ` Chris Marusich 2020-06-07 10:07 ` zimoun 2020-06-08 7:43 ` Chris Marusich [not found] ` <CAJ3okZ0vPpPnDQavrpufPXmSZWk2h+oN2SR7CDVwaEhg-h4h6g@mail.gmail.com> [not found] ` <CAKjnHz1kgwCOia5_hVYkuwaoOiJ55cATaF-Ed6_RsS-r5AELHw@mail.gmail.com> [not found] ` <CAJ3okZ2MhALsxXvWGJccE8feERDf4SihtpF=68s-APQbqaUMRA@mail.gmail.com> [not found] ` <CAKjnHz33fQrKa3KoQsDWh0pv-SgGZwPnn1Wgu6Kvf=SDGb2dWQ@mail.gmail.com> [not found] ` <CAJ3okZ0C2_wECSG4jWq1+ZaO=vL5Zbvyo-2kStaK0cTyF3fXgA@mail.gmail.com> [not found] ` <CAKjnHz35PVGM4BcJUML8=hLxXbJG-hcOg8PeZFjqCoTXe7EueQ@mail.gmail.com> 2020-05-29 23:36 ` zimoun
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).