Store on separate partition

Store on separate partition

[Carlos Sánchez de La Lama]

Hi all,

my (not very big) root partition is almost full, even after
garbage-collecting. I am trying to move the store to a separate
partition, but this is problematic as you need the store at boot time
for the system utilities (and to mount partitions, actually).

Is there a known way to achieve this?

Thanks!

Carlos

Re: Store on separate partition

[Vincent Legoll]

Hello,

> my (not very big) root partition is almost full, even after
> garbage-collecting. I am trying to move the store to a separate
> partition, but this is problematic as you need the store at boot time
> for the system utilities (and to mount partitions, actually).

I did it once and it worked (if I remember right) and I think it's because
all the required tools for mounting it are in the initrd...

But I did it at installation time, and not afterwards.

Maybe I misremember...

-- 
Vincent Legoll

Re: Store on separate partition

[Alex Kost]

Carlos Sánchez de La Lama (2016-08-24 10:06 +0300) wrote:

> Hi all,
>
> my (not very big) root partition is almost full, even after
> garbage-collecting. I am trying to move the store to a separate
> partition, but this is problematic as you need the store at boot time
> for the system utilities (and to mount partitions, actually).
>
> Is there a known way to achieve this?

I tried to do it recently but failed.  My understanding is that it's
impossible, but I may be wrong.

-- 
Alex

Re: Store on separate partition

[Carlos Sánchez de La Lama]

>> my (not very big) root partition is almost full, even after
>> garbage-collecting. I am trying to move the store to a separate
>> partition, but this is problematic as you need the store at boot time
>> for the system utilities (and to mount partitions, actually).
>
> I tried to do it recently but failed.  My understanding is that it's
> impossible, but I may be wrong.

After giving it some more thinking, I also think this is not
possible. The kernel image itself is in the store (and must be, because
you can have several system generations, each using a different
kernel). Same with initrd images.

I have finally moved the whole root to a bigger disk, not much different
to moving the store as root size (aside from the store) is rather small.

Thanks!

Carlos

Re: Store on separate partition

[Tomáš Čech]

[-- Attachment #1: Type: text/plain, Size: 1154 bytes --]

On Wed, Aug 24, 2016 at 12:37:49PM +0200, Carlos Sánchez de La Lama wrote:
>>> my (not very big) root partition is almost full, even after
>>> garbage-collecting. I am trying to move the store to a separate
>>> partition, but this is problematic as you need the store at boot time
>>> for the system utilities (and to mount partitions, actually).
>>
>> I tried to do it recently but failed.  My understanding is that it's
>> impossible, but I may be wrong.
>
>After giving it some more thinking, I also think this is not
>possible. The kernel image itself is in the store (and must be, because
>you can have several system generations, each using a different
>kernel). Same with initrd images.
>
>I have finally moved the whole root to a bigger disk, not much different
>to moving the store as root size (aside from the store) is rather small.

I think it is impossible now but to achieve that you'd need only:

1] to store /gnu/store on separate partition (so grub will find it with its --search)
2] and have some bind mounts ready in initrd stage.

It would be nice to have more flexibility in this area.

Best regards,

S_W

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Store on separate partition

[Tomáš Čech]

[-- Attachment #1: Type: text/plain, Size: 1561 bytes --]

On Thu, Aug 25, 2016 at 04:30:37PM +0200, Tomáš Čech wrote:
>On Wed, Aug 24, 2016 at 12:37:49PM +0200, Carlos Sánchez de La Lama wrote:
>>>>my (not very big) root partition is almost full, even after
>>>>garbage-collecting. I am trying to move the store to a separate
>>>>partition, but this is problematic as you need the store at boot time
>>>>for the system utilities (and to mount partitions, actually).
>>>
>>>I tried to do it recently but failed.  My understanding is that it's
>>>impossible, but I may be wrong.
>>
>>After giving it some more thinking, I also think this is not
>>possible. The kernel image itself is in the store (and must be, because
>>you can have several system generations, each using a different
>>kernel). Same with initrd images.
>>
>>I have finally moved the whole root to a bigger disk, not much different
>>to moving the store as root size (aside from the store) is rather small.
>
>I think it is impossible now but to achieve that you'd need only:
>
>1] to store /gnu/store on separate partition (so grub will find it with its --search)
>2] and have some bind mounts ready in initrd stage.
>
>It would be nice to have more flexibility in this area.

According to
https://www.gnu.org/software/guix/manual/guix.html#File-Systems it
seems that 'bind-mount' is supported flag so together with
'needed-for-boot?' it should be doable.

normal mount
/dev/sdX --> /mnt/store-partition

bind mount
/mnt/store-partition/gnu --> /gnu

grub should find your kernel and initrd with --search

S_W

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: Store on separate partition

[Ludovic Courtès]

Hello,

Alex Kost <alezost@gmail.com> skribis:

> Carlos Sánchez de La Lama (2016-08-24 10:06 +0300) wrote:
>
>> Hi all,
>>
>> my (not very big) root partition is almost full, even after
>> garbage-collecting. I am trying to move the store to a separate
>> partition, but this is problematic as you need the store at boot time
>> for the system utilities (and to mount partitions, actually).
>>
>> Is there a known way to achieve this?
>
> I tried to do it recently but failed.  My understanding is that it's
> impossible, but I may be wrong.

It would be interesting to fix it.  In theory there’s no reason why it
shouldn’t work (you need to mark the /gnu/store file system as
‘needed-for-boot?’.)

If you or Carlos could try it and report details to bug-guix@gnu.org, we
could start from there.

Thanks!

Ludo’.

Re: Store on separate partition

[Carlos Sánchez de La Lama]

>>> I am trying to move the store to a separate partition, but this is
>>> problematic as you need the store at boot time for the system
>>> utilities (and to mount partitions, actually).
>>>
>> I tried to do it recently but failed.  My understanding is that it's
>> impossible, but I may be wrong.
>
> It would be interesting to fix it.  In theory there’s no reason why it
> shouldn’t work (you need to mark the /gnu/store file system as
> ‘needed-for-boot?’.)
>
> If you or Carlos could try it and report details to bug-guix@gnu.org, we
> could start from there.

I got it more-or-less working. What I did is a plain 0.11 install, but
before the "herd start cow-store /mnt" step I had prepared the target
filesystems so I had:

/dev/sda1 mount as /mnt
/dev/sdb1 mounted as /mnt/gnu

To make the store go into /dev/sdb1

My system definition included:

--8<---------------cut here---------------start------------->8---
  (file-systems (cons* (file-system
                         (device "my-root")
                         (title 'label)
                         (mount-point "/")
                         (type "ext4"))
                       (file-system
                         (device "/dev/sdb1")
                         (mount-point "/gnu")
                         (type "ext4")
                         (needed-for-boot? #t))
                       %base-file-systems))
--8<---------------cut here---------------end--------------->8---

When rebooting the installed system, I have to edit GRUB command line
(pressing 'e'), which was:

--8<---------------cut here---------------start------------->8---
  search --file --set /gnu/store/2qh58nr4sxc5jwv2ng9sqq06niab8sjp-linux-libre-4.7/bzImage
  linux /gnu/store/2qh58nr4sxc5jwv2ng9sqq06niab8sjp-linux-libre-4.7/bzImage --root=my-root --system=/gnu/store/4ji3d5v5ci91l69fgi4hdi73kc558whp-system --load=/gnu/store/4ji3d5v5ci91l69fgi4hdi73kc558whp-system/boot
  initrd /gnu/store/4ji3d5v5ci91l69fgi4hdi73kc558whp-system/initrd
--8<---------------cut here---------------end--------------->8---

By:

1) removing "/gnu" from search command, so GRUB root (not kernel root)
   is correctly set as (hd1,msdos1), i.e. /dev/sdb1
2) removing "/gnu" from linux command, so the kernel is correctly loaded
   by GRUB (/gnu is not removed from --system or --load, as those
   commands are interpreted by gnu/build/linux-boot.scm after all
   needed-for-boot filesystems have been mounted in their correct mount
   points).
3) changing initrd path. Removing "/gnu" is not enough, as the path in
   grub.cfg is a symlink to the real initrd, pointing into /gnu/store
   which is not available at initrd-loading time. So I placed the
   synlink destination there, removing "/gnu", so:

initrd /store/<hash>-base-initrd/initrd

And voilá, I can boot into my newly installed GuixSD with /gnu/store in
a separate partition.

Some other files (modules, background image) fail to be accesses by GRUB
as well with generated grub.cfg, but those are not essential to make the
system boot, in my system at least.

BR

Carlos

Re: Store on separate partition

[Ludovic Courtès]

Hi,

csanchezdll@gmail.com (Carlos Sánchez de La Lama) skribis:

> When rebooting the installed system, I have to edit GRUB command line
> (pressing 'e'), which was:
>
>   search --file --set /gnu/store/2qh58nr4sxc5jwv2ng9sqq06niab8sjp-linux-libre-4.7/bzImage
>   linux /gnu/store/2qh58nr4sxc5jwv2ng9sqq06niab8sjp-linux-libre-4.7/bzImage --root=my-root --system=/gnu/store/4ji3d5v5ci91l69fgi4hdi73kc558whp-system --load=/gnu/store/4ji3d5v5ci91l69fgi4hdi73kc558whp-system/boot
>   initrd /gnu/store/4ji3d5v5ci91l69fgi4hdi73kc558whp-system/initrd
>
> By:
>
> 1) removing "/gnu" from search command, so GRUB root (not kernel root)
>    is correctly set as (hd1,msdos1), i.e. /dev/sdb1
> 2) removing "/gnu" from linux command, so the kernel is correctly loaded
>    by GRUB (/gnu is not removed from --system or --load, as those
>    commands are interpreted by gnu/build/linux-boot.scm after all
>    needed-for-boot filesystems have been mounted in their correct mount
>    points).
> 3) changing initrd path. Removing "/gnu" is not enough, as the path in
>    grub.cfg is a symlink to the real initrd, pointing into /gnu/store
>    which is not available at initrd-loading time. So I placed the
>    synlink destination there, removing "/gnu", so:
>
> initrd /store/<hash>-base-initrd/initrd
>
> And voilá, I can boot into my newly installed GuixSD with /gnu/store in
> a separate partition.

Ooh, I see.

We should fix our grub.cfg generation to take this into account
somehow.  I’ve reported filed it to bug-guix@gnu.org:

  http://debbugs.gnu.org/cgi/bugreport.cgi?bug=24346

Please email 24346@debbugs.gnu.org if you have suggestions on how to
fix it.

Thanks!

Ludo’.

Re: Store on separate partition

[John Darrington]

[-- Attachment #1: Type: text/plain, Size: 705 bytes --]

On Wed, Aug 31, 2016 at 11:21:32PM +0200, Ludovic Court??s wrote:

     > And voil??, I can boot into my newly installed GuixSD with /gnu/store in
     > a separate partition.

That's great.

It would be wonderful if we could go one step further, and have source derivations on
a separate partition to binary derivations.  Those two kinds of object have significantly,
different access patterns, and would benefit from being stored on different media.

J'
     

-- 
Avoid eavesdropping.  Send strong encryted email.
PGP Public key ID: 1024D/2DE827B3 
fingerprint = 8797 A26D 0854 2EAB 0285  A290 8A67 719C 2DE8 27B3
See http://sks-keyservers.net or any PGP keyserver for public key.


[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: Store on separate partition

[Vincent Legoll]

Hello,

> It would be wonderful if we could go one step further, and have source derivations on
> a separate partition to binary derivations.  Those two kinds of object have significantly,
> different access patterns, and would benefit from being stored on different media.

+1

and then it would be cool to split guix-daemon's --gc-keep-derivations into
--gc-keep-source-derivations & --gc-keep-binary-derivations...

-- 
Vincent Legoll

bug#24344: [PATCH] Generate grub.cfg with correct paths when store is not in root partition

[Carlos Sánchez de La Lama]

Took a while to produce a working patch :S

A little explanation of what it does:

guix/scripts/system.scm: when constructing old entries, we know
"systems" are the system symlinks in /var/guix/profile, and not a store
derivation. No need for gexps then, which is good as resolving the
symlink using "readlink" cannot be done by the builders /var is not in
the chroot).

gnu/system.scm: for the current system, initrd is inside the system
derivattion in the store. Resolve the symlink at build time.

gnu/system/grub.scm: strip the mount point on the paths GRUB will use,
so GRUB modules, background images. kernels and initrds are correcty
loaded.

This is the first time I do something non-trivial in scheme, so there
might be an easier way to achieve the same result. Nonetheless, I have
tested it on my system and seems to work ok.

* guix/scripts/system.scm (previous-grub-entries): resolve initrd
  symlink for old entries (on the host).
* gnu/system.scm (operating-system-grub.cfg): resolve initrd symlink for
  current system (on the container).
* gnu/system/grub.scm: strip mount-point from GRUB filenames.

---
 gnu/system.scm          |  2 +-
 gnu/system/grub.scm     | 29 ++++++++++++++++++++---------
 guix/scripts/system.scm |  6 +++---
 3 files changed, 24 insertions(+), 13 deletions(-)

diff --git a/gnu/system.scm b/gnu/system.scm
index 0802010..1346f49 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -723,7 +723,7 @@ listed in OS.  The C library expects to find it under
                                    #~(string-append "--load=" #$system
                                                     "/boot")
                                    (operating-system-kernel-arguments os)))
-                           (initrd #~(string-append #$system "/initrd"))))))
+                           (initrd #~(readlink (string-append #$system "/initrd")))))))
     (grub-configuration-file (operating-system-bootloader os)
                              store-fs entries
                              #:old-entries old-entries)))
diff --git a/gnu/system/grub.scm b/gnu/system/grub.scm
index 45b46ca..56cb081 100644
--- a/gnu/system/grub.scm
+++ b/gnu/system/grub.scm
@@ -62,6 +62,14 @@
 ;;;
 ;;; Code:
 
+(define (strip-mount-point fs file)
+  (let ((mount-point (file-system-mount-point fs)))
+    (if (string=? mount-point "/")
+	file
+	#~(if (string-prefix? #$mount-point #$file)
+	      (substring #$file (string-length #$mount-point))
+	      #$file))))
+
 (define-record-type* <grub-image>
   grub-image make-grub-image
   grub-image?
@@ -183,7 +191,8 @@ the store is.  SYSTEM must be the target system string---e.g.,
                      (symbol->string (assoc-ref colors 'bg)))))
 
   (define font-file
-    #~(string-append #$grub "/share/grub/unicode.pf2"))
+    (strip-mount-point root-fs
+		       #~(string-append #$grub "/share/grub/unicode.pf2")))
 
   (mlet* %store-monad ((image (grub-background-image config)))
     (return (and image
@@ -209,7 +218,7 @@ fi~%"
                            #$(grub-root-search root-fs font-file)
                            #$font-file
 
-                           #$image
+                           #$(strip-mount-point root-fs image)
                            #$(theme-colors grub-theme-color-normal)
                            #$(theme-colors grub-theme-color-highlight))))))
 
@@ -254,17 +263,19 @@ corresponding to old generations of the system."
   (define entry->gexp
     (match-lambda
      (($ <menu-entry> label linux arguments initrd)
-      #~(format port "menuentry ~s {
+      (let ((linux (strip-mount-point store-fs linux))
+            (initrd (strip-mount-point store-fs initrd)))
+        #~(format port "menuentry ~s {
   ~a
   linux ~a/~a ~a
   initrd ~a
 }~%"
-                #$label
-                #$(grub-root-search store-fs
-                                    #~(string-append #$linux "/"
-                                                     #$linux-image-name))
-                #$linux #$linux-image-name (string-join (list #$@arguments))
-                #$initrd))))
+                  #$label
+                  #$(grub-root-search store-fs
+                                      #~(string-append #$linux "/"
+                                                       #$linux-image-name))
+                  #$linux #$linux-image-name (string-join (list #$@arguments))
+                  #$initrd)))))
 
   (mlet %store-monad ((sugar (eye-candy config store-fs system #~port)))
     (define builder
diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
index 953c624..5ff98a0 100644
--- a/guix/scripts/system.scm
+++ b/guix/scripts/system.scm
@@ -391,10 +391,10 @@ it atomically, and then run OS's activation script."
         (linux kernel)
         (linux-arguments
          (cons* (string-append "--root=" root-device)
-                #~(string-append "--system=" #$system)
-                #~(string-append "--load=" #$system "/boot")
+                (string-append "--system=" system)
+                (string-append "--load=" system "/boot")
                 kernel-arguments))
-        (initrd #~(string-append #$system "/initrd"))))))
+        (initrd (readlink (string-append system "/initrd")))))))
 
   (let* ((numbers (generation-numbers profile))
          (systems (map (cut generation-file-name profile <>)
--