Delete all previous derivations

Delete all previous derivations

[Joey Dumont]

Hi!

I am using guix on Arch Linux, installed through the AUR package, but with
a custom store directory: --with-store-dir=/cvmfs/soft.valandil.ca/2022.03.
I was running the daemon with options:

# /etc/systemd/system/guix-daemon.service.d/override.conf
[Service]
ExecStart=
ExecStart=/usr/bin/guix-daemon --build-users-group=guixbuild
--gc-keep-outputs=yes --gc-keep-derivations=yes

I have since updated my guix install to point to a new store: /cvmfs/
soft.valandil.ca/2022.04, but I keep getting errors when running guix pull:

$ guix pull
;;; WARNING: loading compiled file /usr/lib/guile/2.2/site-ccache/gnutls.go
failed:
;;; In procedure load-thunk-from-file: Invalid argument
;;; WARNING: loading compiled file /usr/lib/guile/2.2/site-ccache/gnutls.go
failed:
;;; In procedure load-thunk-from-file: Invalid argument
Updating channel 'guix-hpc' from Git repository at '
https://gitlab.inria.fr/guix-hpc/guix-hpc'...
Updating channel 'guix' from Git repository at '
https://git.savannah.gnu.org/git/guix.git'...
Authenticating channel 'guix', commits 9edb3f6 to 1d4f2cd (23 new
commits)...
Building from these channels:
  guix      https://git.savannah.gnu.org/git/guix.git 1d4f2cd
  guix-hpc  https://gitlab.inria.fr/guix-hpc/guix-hpc de22a12
guix pull: error: derivation `/cvmfs/
soft.valandil.ca/2022.03/compat/x86_64/linux/pf4z6hc1f9bwddsravnlacnb2i8p9xyl-guile-3.0.2.tar.xz.drv'
has incorrect output `/cvmfs/
soft.valandil.ca/2022.04/compat/x86_64/linux/92ip5misld973bsm73hfazwmd4n9d58z-guile-3.0.2.tar.xz',
should be `/cvmfs/
soft.valandil.ca/2022.03/compat/x86_64/linux/9gc2alnhkrilqdch2xzjg0vq9b44is43-guile-3.0.2.tar.xz
'

I tried stracing the guix pull, and deleted all cache entries I could,
~/.cache/{guile,guix}, and the previous store, but the error persists.
Where is that derivation stored so that it can be compared against the new
derivation? How can I delete it?

Thanks for any help.

Joey Dumont (Profile <http://blog.joey-dumont.ca/>)
The supreme elegance of Nature lies in its apparent simplicity.

Re: Delete all previous derivations

[zimoun]

Hi,

On Fri, 08 Apr 2022 at 23:14, Joey Dumont <joey.dumont@gmail.com> wrote:

> I am using guix on Arch Linux, installed through the AUR package, but with
> a custom store directory: --with-store-dir=/cvmfs/soft.valandil.ca/2022.03.
> I was running the daemon with options:
>
> # /etc/systemd/system/guix-daemon.service.d/override.conf
> [Service]
> ExecStart=
> ExecStart=/usr/bin/guix-daemon --build-users-group=guixbuild
> --gc-keep-outputs=yes --gc-keep-derivations=yes
>
> I have since updated my guix install to point to a new store: /cvmfs/
> soft.valandil.ca/2022.04, but I keep getting errors when running guix pull:

Hum, interesting setup… but then Guix will be confused, I guess.

> $ guix pull
> ;;; WARNING: loading compiled file /usr/lib/guile/2.2/site-ccache/gnutls.go
> failed:
> ;;; In procedure load-thunk-from-file: Invalid argument
> ;;; WARNING: loading compiled file /usr/lib/guile/2.2/site-ccache/gnutls.go
> failed:
> ;;; In procedure load-thunk-from-file: Invalid argument
> Updating channel 'guix-hpc' from Git repository at '
> https://gitlab.inria.fr/guix-hpc/guix-hpc'...
> Updating channel 'guix' from Git repository at '
> https://git.savannah.gnu.org/git/guix.git'...
> Authenticating channel 'guix', commits 9edb3f6 to 1d4f2cd (23 new
> commits)...
> Building from these channels:
>   guix      https://git.savannah.gnu.org/git/guix.git 1d4f2cd
>   guix-hpc  https://gitlab.inria.fr/guix-hpc/guix-hpc de22a12
> guix pull: error: derivation `/cvmfs/
> soft.valandil.ca/2022.03/compat/x86_64/linux/pf4z6hc1f9bwddsravnlacnb2i8p9xyl-guile-3.0.2.tar.xz.drv'
> has incorrect output `/cvmfs/
> soft.valandil.ca/2022.04/compat/x86_64/linux/92ip5misld973bsm73hfazwmd4n9d58z-guile-3.0.2.tar.xz',
> should be `/cvmfs/
> soft.valandil.ca/2022.03/compat/x86_64/linux/9gc2alnhkrilqdch2xzjg0vq9b44is43-guile-3.0.2.tar.xz
> '

Ah, Guix is really confused. :-)

> I tried stracing the guix pull, and deleted all cache entries I could,
> ~/.cache/{guile,guix}, and the previous store, but the error persists.
> Where is that derivation stored so that it can be compared against the new
> derivation? How can I delete it?

IIUC, you have 2 stores.  It means now you have 2 SQLite databases and
so on.  Well, I do not know how it can work.

You pulled with the old store.  So the current guix command points to an
item to this old store.  To change the store location
(--with-store-dir), you have recompiled Guix, right?

Then, did you run

    ./pre-inst-env guix pull -p ~/.config/guix/current
    ~/.config/guix/current/bin/guix pull

?


Cheers,
simon

Re: Delete all previous derivations

[Joey Dumont]

I was toying around the idea of having guix as a compatibility layer for an
HPC software stack, similar to what Compute Canada had with nix before they
switched to Gentoo (
http://users.ugent.be/~kehoste/eum18/easybuild_nix_cvmfs_compute_canada_bart_oldeman.pdf),
hence the weird setup. I am aware that the Guix HPC guys recently published
a layer between modules and guix shell, which I find interesting.

It took a little while to try your command, as I built guix using Arch's
package manager, which doesn't install pre-inst-env. Also, I built in a
chroot, and pre-inst-env contains hard-coded references to the build
directory. I guess I am a very weird user :)

But even with that I still get the same errors. I don't want to keep the
two stores simultaneously, I'm fine with fully destroying the old one
(although I would be interested in knowing how to properly rollover from
one store to another).

Thanks!

Joey Dumont (Profile <http://blog.joey-dumont.ca/>)
The supreme elegance of Nature lies in its apparent simplicity.


On Fri, 29 Apr 2022 at 12:57, zimoun <zimon.toutoune@gmail.com> wrote:

> Hi,
>
> On Fri, 08 Apr 2022 at 23:14, Joey Dumont <joey.dumont@gmail.com> wrote:
>
> > I am using guix on Arch Linux, installed through the AUR package, but
> with
> > a custom store directory: --with-store-dir=/cvmfs/
> soft.valandil.ca/2022.03.
> > I was running the daemon with options:
> >
> > # /etc/systemd/system/guix-daemon.service.d/override.conf
> > [Service]
> > ExecStart=
> > ExecStart=/usr/bin/guix-daemon --build-users-group=guixbuild
> > --gc-keep-outputs=yes --gc-keep-derivations=yes
> >
> > I have since updated my guix install to point to a new store: /cvmfs/
> > soft.valandil.ca/2022.04, but I keep getting errors when running guix
> pull:
>
> Hum, interesting setup… but then Guix will be confused, I guess.
>
> > $ guix pull
> > ;;; WARNING: loading compiled file
> /usr/lib/guile/2.2/site-ccache/gnutls.go
> > failed:
> > ;;; In procedure load-thunk-from-file: Invalid argument
> > ;;; WARNING: loading compiled file
> /usr/lib/guile/2.2/site-ccache/gnutls.go
> > failed:
> > ;;; In procedure load-thunk-from-file: Invalid argument
> > Updating channel 'guix-hpc' from Git repository at '
> > https://gitlab.inria.fr/guix-hpc/guix-hpc'...
> > Updating channel 'guix' from Git repository at '
> > https://git.savannah.gnu.org/git/guix.git'...
> > Authenticating channel 'guix', commits 9edb3f6 to 1d4f2cd (23 new
> > commits)...
> > Building from these channels:
> >   guix      https://git.savannah.gnu.org/git/guix.git 1d4f2cd
> >   guix-hpc  https://gitlab.inria.fr/guix-hpc/guix-hpc de22a12
> > guix pull: error: derivation `/cvmfs/
> >
> soft.valandil.ca/2022.03/compat/x86_64/linux/pf4z6hc1f9bwddsravnlacnb2i8p9xyl-guile-3.0.2.tar.xz.drv
> '
> > has incorrect output `/cvmfs/
> >
> soft.valandil.ca/2022.04/compat/x86_64/linux/92ip5misld973bsm73hfazwmd4n9d58z-guile-3.0.2.tar.xz
> ',
> > should be `/cvmfs/
> >
> soft.valandil.ca/2022.03/compat/x86_64/linux/9gc2alnhkrilqdch2xzjg0vq9b44is43-guile-3.0.2.tar.xz
> > '
>
> Ah, Guix is really confused. :-)
>
> > I tried stracing the guix pull, and deleted all cache entries I could,
> > ~/.cache/{guile,guix}, and the previous store, but the error persists.
> > Where is that derivation stored so that it can be compared against the
> new
> > derivation? How can I delete it?
>
> IIUC, you have 2 stores.  It means now you have 2 SQLite databases and
> so on.  Well, I do not know how it can work.
>
> You pulled with the old store.  So the current guix command points to an
> item to this old store.  To change the store location
> (--with-store-dir), you have recompiled Guix, right?
>
> Then, did you run
>
>     ./pre-inst-env guix pull -p ~/.config/guix/current
>     ~/.config/guix/current/bin/guix pull
>
> ?
>
>
> Cheers,
> simon
>

Re: Delete all previous derivations

[zimoun]

Hi,

On Thu, 19 May 2022 at 21:14, Joey Dumont <joey.dumont@gmail.com> wrote:
> I was toying around the idea of having guix as a compatibility layer for an
> HPC software stack, similar to what Compute Canada had with nix before they
> switched to Gentoo (
> http://users.ugent.be/~kehoste/eum18/easybuild_nix_cvmfs_compute_canada_bart_oldeman.pdf),
> hence the weird setup. I am aware that the Guix HPC guys recently published
> a layer between modules and guix shell, which I find interesting.

I am missing why 2 stores (or more) are required for this compatibility
layer.


> It took a little while to try your command, as I built guix using Arch's
> package manager, which doesn't install pre-inst-env. Also, I built in a
> chroot, and pre-inst-env contains hard-coded references to the build
> directory. I guess I am a very weird user :)

It was a question. :-)  Thanks for the try and the report.

How do you do otherwise to have this multi-store setup?


> But even with that I still get the same errors. I don't want to keep the
> two stores simultaneously, I'm fine with fully destroying the old one
> (although I would be interested in knowing how to properly rollover from
> one store to another).

Sorry, but I am still missing what you want to achieve.

The store is a global cache mounted somewhere.  The daemon uses this
cache to create a symlinks forest and build some profiles.

Now, you create another cache and let the daemon points to it.
Therefore, the elements from the previous profiles refers to the first
cache location and thus they cannot be reachable.

You get this:

--8<---------------cut here---------------start------------->8---
guix pull: error: derivation `/cvmfs/
soft.valandil.ca/2022.03/compat/x86_64/linux/pf4z6hc1f9bwddsravnlacnb2i8p9xyl-guile-3.0.2.tar.xz.drv'
has incorrect output `/cvmfs/
soft.valandil.ca/2022.04/compat/x86_64/linux/92ip5misld973bsm73hfazwmd4n9d58z-guile-3.0.2.tar.xz',
should be `/cvmfs/
soft.valandil.ca/2022.03/compat/x86_64/linux/9gc2alnhkrilqdch2xzjg0vq9b44is43-guile-3.0.2.tar.xz
--8<---------------cut here---------------end--------------->8---

It means the profile, say ~/.config/current/guix, had been built using
the store located at ’soft.valandil.ca/2022.03’. Then, you reconfigure
the daemon to point to another location for the store
’soft.valandil.ca/2022.04’.

When you pull, the profile ~/.config/current/guix is expecting elements
in soft.valandil.ca/2022.03 but they cannot be reachable since you
replaced the location of the cache without rewriting the information in
the profile.  Bang!

The question is: what do you want to achieve?  Why do you want this
various stores based on dates?


Maybe I am incorrect, but from my understanding,

 1. the store should always be mounted in the same location,
 2. ’guix gc’ is the only command for managing the store,
 3. if you want backup or snapshot of the store, then you should use
 another tool working with the volumes.


A poor man solution for what I understand to your issue could be:

 a) ’guix gc’ for cleaning the current store mounted at /soft.ca/store
 b) copy the volume mounted at /soft.ca/store to another volume
 c) mount this other volume to /soft.ca/store

This way, the necessary elements are kept from one store/volume to the
other.


Cheers,
simon