* [bug#27401] [PATCH] doc: Suggest a QEMU image size large enough for the system examples. @ 2017-06-17 5:14 Leo Famulari 2017-06-17 12:20 ` Marius Bakke 2017-06-19 17:55 ` Jan Nieuwenhuizen 0 siblings, 2 replies; 8+ messages in thread From: Leo Famulari @ 2017-06-17 5:14 UTC (permalink / raw) To: 27401 * doc/guix.texi (Installing GuixSD in a Virtual Machine): Increase suggested image size from 5 GB to 50 GB. --- doc/guix.texi | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index db0e2fbd7..4933a98dd 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -7796,11 +7796,11 @@ Create a disk image that will hold the installed system. To make a qcow2-formatted disk image, use the @command{qemu-img} command: @example -qemu-img create -f qcow2 guixsd.img 5G +qemu-img create -f qcow2 guixsd.img 50G @end example -The resulting file will be much smaller than 5GB, but it will grow as -the virtualized storage device is filled up. +The resulting file will be much smaller than 50 GB (typically less than +1 MB), but it will grow as the virtualized storage device is filled up. @item Boot the USB installation image in an VM: -- 2.13.1 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [bug#27401] [PATCH] doc: Suggest a QEMU image size large enough for the system examples. 2017-06-17 5:14 [bug#27401] [PATCH] doc: Suggest a QEMU image size large enough for the system examples Leo Famulari @ 2017-06-17 12:20 ` Marius Bakke 2017-06-19 17:55 ` Jan Nieuwenhuizen 1 sibling, 0 replies; 8+ messages in thread From: Marius Bakke @ 2017-06-17 12:20 UTC (permalink / raw) To: Leo Famulari, 27401 [-- Attachment #1: Type: text/plain, Size: 161 bytes --] Leo Famulari <leo@famulari.name> writes: > * doc/guix.texi (Installing GuixSD in a Virtual Machine): Increase suggested > image size from 5 GB to 50 GB. LGTM. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 487 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* [bug#27401] [PATCH] doc: Suggest a QEMU image size large enough for the system examples. 2017-06-17 5:14 [bug#27401] [PATCH] doc: Suggest a QEMU image size large enough for the system examples Leo Famulari 2017-06-17 12:20 ` Marius Bakke @ 2017-06-19 17:55 ` Jan Nieuwenhuizen 2017-06-19 21:48 ` Leo Famulari 1 sibling, 1 reply; 8+ messages in thread From: Jan Nieuwenhuizen @ 2017-06-19 17:55 UTC (permalink / raw) To: Leo Famulari; +Cc: 27401 Leo Famulari writes: > * doc/guix.texi (Installing GuixSD in a Virtual Machine): Increase suggested > image size from 5 GB to 50 GB. LGTM, however -- What about something like this patch too? I seems that the VMs we create (although they are of the ballooning cow2 kind), are 2GiB -- i.e., almost unusable for Real Stuff (TM)? $ git diff diff --git a/Makefile.am b/Makefile.am index a3b3e557a..ff2ae120b 100644 --- a/Makefile.am +++ b/Makefile.am @@ -592,7 +592,7 @@ GUIXSD_VM_IMAGE_BASE = guixsd-vm-image-$(PACKAGE_VERSION) GUIXSD_INSTALLATION_IMAGE_SIZE ?= 950MiB # Size of the VM image (for x86_64 typically). -GUIXSD_VM_IMAGE_SIZE ?= 2GiB +GUIXSD_VM_IMAGE_SIZE ?= 50GiB # The release process works in several phases: # -- Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [bug#27401] [PATCH] doc: Suggest a QEMU image size large enough for the system examples. 2017-06-19 17:55 ` Jan Nieuwenhuizen @ 2017-06-19 21:48 ` Leo Famulari 2017-06-20 18:03 ` Jan Nieuwenhuizen 0 siblings, 1 reply; 8+ messages in thread From: Leo Famulari @ 2017-06-19 21:48 UTC (permalink / raw) To: Jan Nieuwenhuizen; +Cc: 27401 [-- Attachment #1: Type: text/plain, Size: 942 bytes --] On Mon, Jun 19, 2017 at 07:55:03PM +0200, Jan Nieuwenhuizen wrote: > Leo Famulari writes: > > > * doc/guix.texi (Installing GuixSD in a Virtual Machine): Increase suggested > > image size from 5 GB to 50 GB. > > LGTM, however -- What about something like this patch too? I seems that > the VMs we create (although they are of the ballooning cow2 kind), are > 2GiB -- i.e., almost unusable for Real Stuff (TM)? I chose 2GiB for those images because it's pretty easy to grow a partition and ext4 filesystem from a running system. On the other hand, I couldn't find an easy way to shrink them. We can't predict the size of the virtual block device storage provided by a VPS hoster, so I chose this small size with these expectations: 1) 2GiB will never be too big 2) users will resize after booting I didn't come up with a better solution for the problem of not knowing the storage size in advance. What do you think? [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* [bug#27401] [PATCH] doc: Suggest a QEMU image size large enough for the system examples. 2017-06-19 21:48 ` Leo Famulari @ 2017-06-20 18:03 ` Jan Nieuwenhuizen 2017-06-23 21:13 ` Leo Famulari 0 siblings, 1 reply; 8+ messages in thread From: Jan Nieuwenhuizen @ 2017-06-20 18:03 UTC (permalink / raw) To: Leo Famulari; +Cc: 27401 Leo Famulari writes: >> LGTM, however -- What about something like this patch too? I seems that >> the VMs we create (although they are of the ballooning cow2 kind), are >> 2GiB -- i.e., almost unusable for Real Stuff (TM)? > > I chose 2GiB for those images because it's pretty easy to grow a > partition and ext4 filesystem from a running system. On the other hand, > I couldn't find an easy way to shrink them. > > We can't predict the size of the virtual block device storage provided > by a VPS hoster, Ah, sure. I missed that aspect. > so I chose this small size with these expectations: > 1) 2GiB will never be too big Yes. > 2) users will resize after booting When I was asked to help with this, I was confronted with a 2GiB sda1 and right behind that an EFI sda2. That was running in vmware (don't ask ;-). We hesitated, removed the sda2 and did the resize thing. Sadly I have no further information, instead of testing the reboot we decided to go for qemu. > I didn't come up with a better solution for the problem of not knowing > the storage size in advance. What do you think? Yes, I think you made the right choice. Greetings, janneke -- Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.org Freelance IT http://JoyofSource.com | Avatar® http://AvatarAcademy.com ^ permalink raw reply [flat|nested] 8+ messages in thread
* [bug#27401] [PATCH] doc: Suggest a QEMU image size large enough for the system examples. 2017-06-20 18:03 ` Jan Nieuwenhuizen @ 2017-06-23 21:13 ` Leo Famulari 2017-06-26 11:51 ` [bug#27401] Guessing the disk image size in 'guix system' Ludovic Courtès 0 siblings, 1 reply; 8+ messages in thread From: Leo Famulari @ 2017-06-23 21:13 UTC (permalink / raw) To: Jan Nieuwenhuizen; +Cc: 27401 [-- Attachment #1: Type: text/plain, Size: 815 bytes --] On Tue, Jun 20, 2017 at 08:03:17PM +0200, Jan Nieuwenhuizen wrote: > Leo Famulari writes: > > 2) users will resize after booting > > When I was asked to help with this, I was confronted with a 2GiB sda1 > and right behind that an EFI sda2. That was running in vmware (don't > ask ;-). We hesitated, removed the sda2 and did the resize thing. Ah, I didn't plan for that sort of disk layout. > Sadly I have no further information, instead of testing the reboot we > decided to go for qemu. > > > I didn't come up with a better solution for the problem of not knowing > > the storage size in advance. What do you think? > > Yes, I think you made the right choice. Okay. I'd like to clarify and say that I think we could do something more sophisticated, but I'm not sure what we would do :) [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* [bug#27401] Guessing the disk image size in 'guix system' 2017-06-23 21:13 ` Leo Famulari @ 2017-06-26 11:51 ` Ludovic Courtès 2017-06-29 22:30 ` Ludovic Courtès 0 siblings, 1 reply; 8+ messages in thread From: Ludovic Courtès @ 2017-06-26 11:51 UTC (permalink / raw) To: Leo Famulari; +Cc: 27401 [-- Attachment #1: Type: text/plain, Size: 614 bytes --] Hello! Leo Famulari <leo@famulari.name> skribis: > On Tue, Jun 20, 2017 at 08:03:17PM +0200, Jan Nieuwenhuizen wrote: [...] >> > I didn't come up with a better solution for the problem of not knowing >> > the storage size in advance. What do you think? >> >> Yes, I think you made the right choice. > > Okay. I'd like to clarify and say that I think we could do something > more sophisticated, but I'm not sure what we would do :) We can do “guix size $(guix system build foo.scm)” and add 10% or so to that. Maybe something like this mostly-untested patch would work? Ludo’. [-- Attachment #2: Type: text/x-patch, Size: 7861 bytes --] diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm index 392737d07..15cc0d993 100644 --- a/gnu/system/vm.scm +++ b/gnu/system/vm.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2013, 2014, 2015, 2016 Ludovic Courtès <ludo@gnu.org> +;;; Copyright © 2013, 2014, 2015, 2016, 2017 Ludovic Courtès <ludo@gnu.org> ;;; Copyright © 2016 Christopher Allan Webber <cwebber@dustycloud.org> ;;; Copyright © 2016 Leo Famulari <leo@famulari.name> ;;; Copyright © 2017 Mathieu Othacehe <m.othacehe@gmail.com> @@ -108,8 +108,7 @@ (references-graphs #f) (memory-size 256) (disk-image-format "qcow2") - (disk-image-size - (* 100 (expt 2 20)))) + (disk-image-size 'guess)) "Evaluate EXP in a QEMU virtual machine running LINUX with INITRD (a derivation). In the virtual machine, EXP has access to all its inputs from the store; it should put its output files in the `/xchg' directory, which is @@ -138,19 +137,24 @@ made available under the /xchg CIFS share." (define builder ;; Code that launches the VM that evaluates EXP. (with-imported-modules (source-module-closure '((guix build utils) + (guix build store-copy) (gnu build vm))) #~(begin (use-modules (guix build utils) + (guix build store-copy) (gnu build vm)) - (let ((inputs '#$(list qemu coreutils)) - (linux (string-append #$linux "/" - #$(system-linux-image-file-name))) - (initrd (string-append #$initrd "/initrd")) - (loader #$loader) - (graphs '#$(match references-graphs - (((graph-files . _) ...) graph-files) - (_ #f)))) + (let* ((inputs '#$(list qemu coreutils)) + (linux (string-append #$linux "/" + #$(system-linux-image-file-name))) + (initrd (string-append #$initrd "/initrd")) + (loader #$loader) + (graphs '#$(match references-graphs + (((graph-files . _) ...) graph-files) + (_ #f))) + (size #$(if (eq? 'guess disk-image-size) + #~(round (* 1.1 (closure-size graphs))) + disk-image-size))) (set-path-environment-variable "PATH" '("bin") inputs) @@ -160,7 +164,7 @@ made available under the /xchg CIFS share." #:memory-size #$memory-size #:make-disk-image? #$make-disk-image? #:disk-image-format #$disk-image-format - #:disk-image-size #$disk-image-size + #:disk-image-size size #:references-graphs graphs))))) (gexp->derivation name builder @@ -198,10 +202,13 @@ the image." (expression->derivation-in-linux-vm name (with-imported-modules (source-module-closure '((gnu build vm) + (guix build store-copy) (guix build utils))) #~(begin (use-modules (gnu build vm) - (guix build utils)) + (guix build store-copy) + (guix build utils) + (srfi srfi-26)) (let ((inputs '#$(append (list qemu parted e2fsprogs dosfstools) @@ -227,9 +234,14 @@ the image." #:copy-closures? #$copy-inputs? #:register-closures? #$register-closures? #:system-directory #$os-drv)) + (root-size #$(if (eq? 'guess disk-image-size) + #~(closure-size + (map (cut string-append "/xchg/" <>) + graphs)) + (- disk-image-size + (* 50 (expt 2 20))))) (partitions (list (partition - (size #$(- disk-image-size - (* 50 (expt 2 20)))) + (size root-size) (label #$file-system-label) (file-system #$file-system-type) (flags '(boot)) diff --git a/guix/build/store-copy.scm b/guix/build/store-copy.scm index a296bdf78..fe2eb6f69 100644 --- a/guix/build/store-copy.scm +++ b/guix/build/store-copy.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2013, 2014 Ludovic Courtès <ludo@gnu.org> +;;; Copyright © 2013, 2014, 2017 Ludovic Courtès <ludo@gnu.org> ;;; ;;; This file is part of GNU Guix. ;;; @@ -20,7 +20,9 @@ #:use-module (guix build utils) #:use-module (srfi srfi-1) #:use-module (ice-9 rdelim) + #:use-module (ice-9 ftw) #:export (read-reference-graph + closure-size populate-store)) ;;; Commentary: @@ -46,6 +48,37 @@ The data at PORT is the format produced by #:references-graphs." (loop (read-line port) result))))) +(define (file-size file) + "Return the size of bytes of FILE, entering it if FILE is a directory." + (file-system-fold (const #t) + (lambda (file stat result) ;leaf + (+ (stat:size stat) result)) + (lambda (directory stat result) ;down + (+ (stat:size stat) result)) + (lambda (directory stat result) ;up + result) + (lambda (file stat result) ;skip + result) + (lambda (file stat errno result) + (format (current-error-port) + "file-size: ~a: ~a~%" file + (strerror errno)) + result) + 0 + file + lstat)) + +(define (closure-size reference-graphs) + "Return an estimate of the size of the closure described by +REFERENCE-GRAPHS, a list of reference-graph files." + (define (graph-from-file file) + (call-with-input-file file read-reference-graph)) + + (define items + (delete-duplicates (append-map graph-from-file reference-graphs))) + + (reduce + 0 (map file-size items))) + (define* (populate-store reference-graphs target) "Populate the store under directory TARGET with the items specified in REFERENCE-GRAPHS, a list of reference-graph files." diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm index 35675cc01..7e20b10da 100644 --- a/guix/scripts/system.scm +++ b/guix/scripts/system.scm @@ -854,7 +854,7 @@ Some ACTIONS support additional ARGS.\n")) (build-hook? . #t) (max-silent-time . 3600) (verbosity . 0) - (image-size . ,(* 900 (expt 2 20))) + (image-size . guess) (install-bootloader? . #t))) \f ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [bug#27401] Guessing the disk image size in 'guix system' 2017-06-26 11:51 ` [bug#27401] Guessing the disk image size in 'guix system' Ludovic Courtès @ 2017-06-29 22:30 ` Ludovic Courtès 0 siblings, 0 replies; 8+ messages in thread From: Ludovic Courtès @ 2017-06-29 22:30 UTC (permalink / raw) To: Leo Famulari; +Cc: 27401 ludo@gnu.org (Ludovic Courtès) skribis: > We can do “guix size $(guix system build foo.scm)” and add 10% or so to > that. > > Maybe something like this mostly-untested patch would work? I’ve committed a variant of this patch as a8ac4f081a9a679498ea42ccfe001f218bba3043. I’ve tested it with several OS configs and it worked, except for “lightweight-desktop.tmpl” where it failed with a confusing GRUB error that might have been caused by ENOSPC. Feedback and suggestion to improve the estimate welcome! Ludo’. ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2017-06-29 22:31 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2017-06-17 5:14 [bug#27401] [PATCH] doc: Suggest a QEMU image size large enough for the system examples Leo Famulari 2017-06-17 12:20 ` Marius Bakke 2017-06-19 17:55 ` Jan Nieuwenhuizen 2017-06-19 21:48 ` Leo Famulari 2017-06-20 18:03 ` Jan Nieuwenhuizen 2017-06-23 21:13 ` Leo Famulari 2017-06-26 11:51 ` [bug#27401] Guessing the disk image size in 'guix system' Ludovic Courtès 2017-06-29 22:30 ` Ludovic Courtès
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/guix.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.