Diagnosing "In procedure symlink: No space left on device"

Diagnosing "In procedure symlink: No space left on device"

[David Wilson]

Hello Guix!

Recently on one of my machines I've not been able to update one of my profiles due to a "No space left on device" error (see snippet below).  My device is most definitely not out of space (/dev/dm-0 has 42gb free):

---
λ df -h
Filesystem      Size  Used Avail Use% Mounted on
none            7.8G     0  7.8G   0% /dev
/dev/dm-0       234G  180G   42G  82% /
/dev/nvme0n1p2  495M   43M  453M   9% /boot/efi
tmpfs           7.8G     0  7.8G   0% /dev/shm
none            7.8G   28K  7.8G   1% /run/systemd
none            7.8G     0  7.8G   0% /run/user
cgroup          7.8G     0  7.8G   0% /sys/fs/cgroup
tmpfs           1.6G  4.0K  1.6G   1% /run/user/1000
---

Here's the backtrace from `guix package -m <my manifest file>`

---
Backtrace:
 8 (primitive-load "/gnu/store/mablf146gb6vb0m1lj9a9yr4cfj…")
In guix/build/profiles.scm:
 157:2 7 (build-profile "/gnu/store/mdzxrmx68qkpakhy1n5vk6lzvnq…" …)
In unknown file:
 6 (hash-for-each #<procedure 7ffff2e7cb80 at guix/build/…> …)
 5 (hash-for-each #<procedure 7ffff2852e40 at guix/build/…> …)
 4 (hash-for-each #<procedure 7ffff2852a40 at guix/build/…> …)
 3 (hash-for-each #<procedure 7ffff2889e00 at guix/build/…> …)
 2 (hash-for-each #<procedure 7ffff783e200 at guix/build/…> …)
 1 (hash-for-each #<procedure 7ffff2e068c0 at guix/build/…> …)
 0 (symlink "/gnu/store/z74railpdshvi28i1rfa97fp2d9rg79d-…" …)

ERROR: In procedure symlink:
In procedure symlink: No space left on device
---

Is there some other reason that Guix might think that I'm out of space?  Is there any other command I can run that might help illuminate why I'm getting this error?

Thanks!

David

Re: Diagnosing "In procedure symlink: No space left on device"

[Vagrant Cascadian]

On 2020-02-10, David Wilson wrote:
> Recently on one of my machines I've not been able to update one of my
> profiles due to a "No space left on device" error (see snippet below).
> My device is most definitely not out of space (/dev/dm-0 has 42gb
> free):
>
> ---
> λ df -h
> Filesystem      Size  Used Avail Use% Mounted on
> none            7.8G     0  7.8G   0% /dev
> /dev/dm-0       234G  180G   42G  82% /
...
> Is there some other reason that Guix might think that I'm out of
> space?  Is there any other command I can run that might help
> illuminate why I'm getting this error?

Maybe you've run out of inodes?

  df -i


live well,
  vagrant

Re: Diagnosing "In procedure symlink: No space left on device"

[Julien Lepiller]

Le 10 février 2020 12:30:21 GMT-05:00, David Wilson <david@daviwil.com> a écrit :
>Hello Guix!
>
>Recently on one of my machines I've not been able to update one of my
>profiles due to a "No space left on device" error (see snippet below). 
>My device is most definitely not out of space (/dev/dm-0 has 42gb
>free):
>
>---
>λ df -h
>Filesystem      Size  Used Avail Use% Mounted on
>none            7.8G     0  7.8G   0% /dev
>/dev/dm-0       234G  180G   42G  82% /
>/dev/nvme0n1p2  495M   43M  453M   9% /boot/efi
>tmpfs           7.8G     0  7.8G   0% /dev/shm
>none            7.8G   28K  7.8G   1% /run/systemd
>none            7.8G     0  7.8G   0% /run/user
>cgroup          7.8G     0  7.8G   0% /sys/fs/cgroup
>tmpfs           1.6G  4.0K  1.6G   1% /run/user/1000
>---
>
>Here's the backtrace from `guix package -m <my manifest file>`
>
>---
>Backtrace:
> 8 (primitive-load "/gnu/store/mablf146gb6vb0m1lj9a9yr4cfj…")
>In guix/build/profiles.scm:
> 157:2 7 (build-profile "/gnu/store/mdzxrmx68qkpakhy1n5vk6lzvnq…" …)
>In unknown file:
> 6 (hash-for-each #<procedure 7ffff2e7cb80 at guix/build/…> …)
> 5 (hash-for-each #<procedure 7ffff2852e40 at guix/build/…> …)
> 4 (hash-for-each #<procedure 7ffff2852a40 at guix/build/…> …)
> 3 (hash-for-each #<procedure 7ffff2889e00 at guix/build/…> …)
> 2 (hash-for-each #<procedure 7ffff783e200 at guix/build/…> …)
> 1 (hash-for-each #<procedure 7ffff2e068c0 at guix/build/…> …)
> 0 (symlink "/gnu/store/z74railpdshvi28i1rfa97fp2d9rg79d-…" …)
>
>ERROR: In procedure symlink:
>In procedure symlink: No space left on device
>---
>
>Is there some other reason that Guix might think that I'm out of space?
>Is there any other command I can run that might help illuminate why I'm
>getting this error?
>
>Thanks!
>
>David

It coull be you ran out of inode (but I doubt it) or that the tmpfs ran out of space, not your root fs. What were you trying to build? Something big I suppose?

Re: Diagnosing "In procedure symlink: No space left on device"

[David Wilson]

Argh, yes, that seems to be the case:

---
λ df -i
/dev/dm-0      15597568 15597568       0  100% /
---

I could have sworn I checked that before!  Clearing out some large Git repos seems to be doing the trick.  Thanks a lot!

David

On Mon, Feb 10, 2020, at 9:53 AM, Vagrant Cascadian wrote:
> On 2020-02-10, David Wilson wrote:
> > Recently on one of my machines I've not been able to update one of my
> > profiles due to a "No space left on device" error (see snippet below).
> > My device is most definitely not out of space (/dev/dm-0 has 42gb
> > free):
> >
> > ---
> > λ df -h
> > Filesystem      Size  Used Avail Use% Mounted on
> > none            7.8G     0  7.8G   0% /dev
> > /dev/dm-0       234G  180G   42G  82% /
> ...
> > Is there some other reason that Guix might think that I'm out of
> > space?  Is there any other command I can run that might help
> > illuminate why I'm getting this error?
> 
> Maybe you've run out of inodes?
> 
>   df -i
> 
> 
> live well,
>   vagrant
>

Re: Diagnosing "In procedure symlink: No space left on device"

[David Wilson]

Turns out it was indeed a lack of inodes :/ 

Thanks!

On Mon, Feb 10, 2020, at 10:06 AM, Julien Lepiller wrote:
> Le 10 février 2020 12:30:21 GMT-05:00, David Wilson <david@daviwil.com> 
> a écrit :
> >Hello Guix!
> >
> >Recently on one of my machines I've not been able to update one of my
> >profiles due to a "No space left on device" error (see snippet below). 
> >My device is most definitely not out of space (/dev/dm-0 has 42gb
> >free):
> >
> >---
> >λ df -h
> >Filesystem      Size  Used Avail Use% Mounted on
> >none            7.8G     0  7.8G   0% /dev
> >/dev/dm-0       234G  180G   42G  82% /
> >/dev/nvme0n1p2  495M   43M  453M   9% /boot/efi
> >tmpfs           7.8G     0  7.8G   0% /dev/shm
> >none            7.8G   28K  7.8G   1% /run/systemd
> >none            7.8G     0  7.8G   0% /run/user
> >cgroup          7.8G     0  7.8G   0% /sys/fs/cgroup
> >tmpfs           1.6G  4.0K  1.6G   1% /run/user/1000
> >---
> >
> >Here's the backtrace from `guix package -m <my manifest file>`
> >
> >---
> >Backtrace:
> > 8 (primitive-load "/gnu/store/mablf146gb6vb0m1lj9a9yr4cfj…")
> >In guix/build/profiles.scm:
> > 157:2 7 (build-profile "/gnu/store/mdzxrmx68qkpakhy1n5vk6lzvnq…" …)
> >In unknown file:
> > 6 (hash-for-each #<procedure 7ffff2e7cb80 at guix/build/…> …)
> > 5 (hash-for-each #<procedure 7ffff2852e40 at guix/build/…> …)
> > 4 (hash-for-each #<procedure 7ffff2852a40 at guix/build/…> …)
> > 3 (hash-for-each #<procedure 7ffff2889e00 at guix/build/…> …)
> > 2 (hash-for-each #<procedure 7ffff783e200 at guix/build/…> …)
> > 1 (hash-for-each #<procedure 7ffff2e068c0 at guix/build/…> …)
> > 0 (symlink "/gnu/store/z74railpdshvi28i1rfa97fp2d9rg79d-…" …)
> >
> >ERROR: In procedure symlink:
> >In procedure symlink: No space left on device
> >---
> >
> >Is there some other reason that Guix might think that I'm out of space?
> >Is there any other command I can run that might help illuminate why I'm
> >getting this error?
> >
> >Thanks!
> >
> >David
> 
> It coull be you ran out of inode (but I doubt it) or that the tmpfs ran 
> out of space, not your root fs. What were you trying to build? 
> Something big I suppose?
>