VM image: can we simplify its use?

VM image: can we simplify its use?

[Ludovic Courtès]

Hello Guix!

I had criticism about the VM image:

  https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/system/examples/vm-image.tmpl

Namely that you have to manually set up networking, and then to manually
resize the image just to be able to run ‘guix pull’, is considered
inconvenient (rightly so).

Would it be an option to install connman or something in the image?

What about making the image bigger?  I would expect the QCOW2 file to be
small, if the disk is full of zero.

Thoughts?

Ludo’.

Re: VM image: can we simplify its use?

[Gábor Boskovits]

Hello,

Ludovic Courtès <ludo@gnu.org> ezt írta (időpont: 2019. ápr. 4., Cs, 16:57):
>
> Hello Guix!
>
> I had criticism about the VM image:
>
>   https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/system/examples/vm-image.tmpl
>
> Namely that you have to manually set up networking, and then to manually
> resize the image just to be able to run ‘guix pull’, is considered
> inconvenient (rightly so).
>

I believe that expanding the documentation a bit on how to increase
the -img size,
addin why this is needed, and adding a fist time boot script, to
adjust the partition and
filesystem size to the size of the image would be helpful. Wdyt?

> Would it be an option to install connman or something in the image?

I believe this might be ok, but maybe we could just get by adding a
dhcp service?
That would solve most of the installs, and if that does not work, then
you need at
least some manual steps anyways. Or am I missing sommething?

>
> What about making the image bigger?  I would expect the QCOW2 file to be
> small, if the disk is full of zero.
>
> Thoughts?
>
> Ludo’.
>

Best regards,
g_bor

Re: VM image: can we simplify its use?

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 1033 bytes --]

On Thu, Apr 04, 2019 at 04:56:51PM +0200, Ludovic Courtès wrote:
> I had criticism about the VM image:
[...]
> Thoughts?

Originally I added the VM image to address a specific use case at one VM
hoster. I had hoped it would catch on more than it did and that the VM
image would be improved to avoid these manual steps.

In the meantime, people are using the VM image just to try out Guix —
for example, in a local QEMU instance.

It doesn't give a great impression, so let's change it.

We should offer something like a "Live Guix" image for people to try the
full Guix system without installing anything, and this should be more
fully-featured than the current Guix VM image.

Let's replace the current VM image with this "Live Guix" thing for the
next release :) Should we adopt 'desktop.tmpl' for this?

Eventually we should also offer a Guix system service that works like
cloud-init [0] and can replace the manual work around networking,
partitioning, etc. Help wanted!

[0]
https://cloud-init.io/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: VM image: can we simplify its use?

[Daniel Jiang]

[-- Attachment #1: Type: text/plain, Size: 1462 bytes --]

Hello,

I think I can help! Missed the deadline for GSoC application because of
school stuff but that shouldn't stop me from contributing to Guix. This
sounds like a task on a level I can accomplish (though I might still need
some guidance/pointers of where to start).

Sincerely,
Daniel Jiang

On Tue, Apr 9, 2019 at 3:48 PM Leo Famulari <leo@famulari.name> wrote:

> On Thu, Apr 04, 2019 at 04:56:51PM +0200, Ludovic Courtès wrote:
> > I had criticism about the VM image:
> [...]
> > Thoughts?
>
> Originally I added the VM image to address a specific use case at one VM
> hoster. I had hoped it would catch on more than it did and that the VM
> image would be improved to avoid these manual steps.
>
> In the meantime, people are using the VM image just to try out Guix —
> for example, in a local QEMU instance.
>
> It doesn't give a great impression, so let's change it.
>
> We should offer something like a "Live Guix" image for people to try the
> full Guix system without installing anything, and this should be more
> fully-featured than the current Guix VM image.
>
> Let's replace the current VM image with this "Live Guix" thing for the
> next release :) Should we adopt 'desktop.tmpl' for this?
>
> Eventually we should also offer a Guix system service that works like
> cloud-init [0] and can replace the manual work around networking,
> partitioning, etc. Help wanted!
>
> [0]
> https://cloud-init.io/
>

[-- Attachment #2: Type: text/html, Size: 1975 bytes --]

Re: VM image: can we simplify its use?

[Ludovic Courtès]

Hello Daniel,

Daniel Jiang <jngdnl@gmail.com> skribis:

> I think I can help! Missed the deadline for GSoC application because of
> school stuff but that shouldn't stop me from contributing to Guix. This
> sounds like a task on a level I can accomplish (though I might still need
> some guidance/pointers of where to start).

You’re welcome to give it a try!  Like Leo wrote, you could essentially
start from the gnu/system/examples/desktop.tmpl OS.  You can pass it to
‘guix system vm’ image to try it out, until you’re satisfied with the
result.

At that point you can try ‘guix system vm-image’ and see how big the
result is.

Let us know how it goes!

Thanks,
Ludo’.

Re: VM image: can we simplify its use?

[Ludovic Courtès]

Hi Leo,

Leo Famulari <leo@famulari.name> skribis:

> Originally I added the VM image to address a specific use case at one VM
> hoster. I had hoped it would catch on more than it did and that the VM
> image would be improved to avoid these manual steps.
>
> In the meantime, people are using the VM image just to try out Guix —
> for example, in a local QEMU instance.
>
> It doesn't give a great impression, so let's change it.

Agreed.

> We should offer something like a "Live Guix" image for people to try the
> full Guix system without installing anything, and this should be more
> fully-featured than the current Guix VM image.
>
> Let's replace the current VM image with this "Live Guix" thing for the
> next release :) Should we adopt 'desktop.tmpl' for this?

I think we could start from there.  With GNOME, that’d be 2+ GiB
probably, which is pretty big; otherwise we could come up with a lighter
setup, say SLiM + Xfce (+ IceCat?).

Thoughts?

> Eventually we should also offer a Guix system service that works like
> cloud-init [0] and can replace the manual work around networking,
> partitioning, etc. Help wanted!

It seems cloud-init is largely about running some sort of a YAMLish
shell script that runs commands to obtain the desired state in terms of
installed packages, system setup, and all that, right?  To some extent
that seems redundant with a typical OS declaration, no?

Thanks for your feedback!

Ludo’.

Re: VM image: can we simplify its use?

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 434 bytes --]

Leo Famulari <leo@famulari.name> writes:

> We should offer something like a "Live Guix" image for people to try the
> full Guix system without installing anything, and this should be more
> fully-featured than the current Guix VM image.

Interested parties would probably do well to examine prior art at places
such as:

https://www.debian.org/devel/debian-live/

Surely there are some things we can adapt.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

New VM image

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 751 bytes --]

Hello Guix!

The attached config file can be used to produce a 2.9G QCOW image
(uncompressed) that internally appears to have a 20G file system:

  guix system vm-image --image-size=20G vm-live.tmpl

The image logs into Xfce and from there people can spawn a terminal, run
‘guix package -i icecat’, etc.

I think it would fill the need of those who want to see what Guix is
like, and also the need of developers who want to test their software on
Guix without installing it (I’d argue that installing Guix on top of
their distro is simpler, but some people will use the VM image for that
purpose anyway.)

What do people think?

If that’s fine with you I can commit it and update the doc & makefiles.

Thanks,
Ludo’.


[-- Attachment #2: the config --]
[-- Type: text/plain, Size: 2896 bytes --]

;; This is an operating system configuration for a VM image.
;; Modify it as you see fit and instantiate the changes by running:
;;
;;   guix system reconfigure /etc/config.scm
;;

(use-modules (gnu) (srfi srfi-1))
(use-service-modules xorg desktop)
(use-package-modules bootloaders nvi wget xorg)

(define vm-image-motd (plain-file "motd" "
\x1b[1;37mThis is the GNU system.  Welcome!\x1b[0m

This instance of Guix is a template for virtualized environments.
You can reconfigure the whole system by adjusting /etc/config.scm
and running:

  guix system reconfigure /etc/config.scm

\x1b[1;33mConsider setting a password for the 'root' and 'guest' \
accounts.\x1b[0m
"))

(define this-file
  (local-file "./vm-live.tmpl" "config.scm"))


(operating-system
  (host-name "gnu")
  (timezone "Etc/UTC")
  (locale "en_US.utf8")
  (keyboard-layout (keyboard-layout "us" "altgr-intl"))

  (bootloader (bootloader-configuration
               (bootloader grub-bootloader)
               (target "/dev/sda")
               (terminal-outputs '(console))))
  (file-systems (cons (file-system
                        (mount-point "/")
                        (device "/dev/vda")
                        (type "ext4"))
                      %base-file-systems))

  (users (cons (user-account
                (name "guest")
                (comment "GNU Guix Live")
                (password "")                     ;no password
                (group "users")
                (supplementary-groups '("wheel" "netdev"
                                        "audio" "video")))
               %base-user-accounts))

  (packages (cons* nvi wget %base-packages))

  (services (cons* (service xfce-desktop-service-type)

                   ;; Copy this file to /etc/config.scm in the OS.
                   (simple-service 'config-file etc-service-type
                                   `(("config.scm" ,this-file)))

                   ;; Choose SLiM, which is lighter than the default GDM.
                   (service slim-service-type
                            (slim-configuration
                             (auto-login? #t)
                             (default-user "guest")
                             (xorg-configuration
                              (xorg-configuration
                               (keyboard-layout keyboard-layout)))))

                   (remove (lambda (service)
                             (eq? gdm-service-type (service-kind service)))
                           (modify-services %desktop-services
                             (login-service-type config =>
                                                 (login-configuration
                                                  (inherit config)
                                                  (motd vm-image-motd)))))))

  ;; Allow resolution of '.local' host names with mDNS.
  (name-service-switch %mdns-host-lookup-nss))

Re: New VM image

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 1018 bytes --]

On Thu, Apr 25, 2019 at 06:44:07PM +0200, Ludovic Courtès wrote:
> The attached config file can be used to produce a 2.9G QCOW image
> (uncompressed) that internally appears to have a 20G file system:

A little big but I don't think we can shrink it any more unfortunately.

> What do people think?

It looks good to me, thank you!

>   (file-systems (cons (file-system
>                         (mount-point "/")
>                         (device "/dev/vda")
>                         (type "ext4"))
>                       %base-file-systems))
> 
>   (users (cons (user-account
>                 (name "guest")
>                 (comment "GNU Guix Live")
>                 (password "")                     ;no password
>                 (group "users")
>                 (supplementary-groups '("wheel" "netdev"
>                                         "audio" "video")))
>                %base-user-accounts))

We will probably get some cons / cons* confusion in #guix but that's
okay.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: New VM image

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 3193 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> The attached config file can be used to produce a 2.9G QCOW image
> (uncompressed) that internally appears to have a 20G file system:

Nice!

> I think it would fill the need of those who want to see what Guix is
> like, and also the need of developers who want to test their software on
> Guix without installing it (I’d argue that installing Guix on top of
> their distro is simpler, but some people will use the VM image for that
> purpose anyway.)
>
> What do people think?

I like it!  Is it intended to replace gnu/system/examples/vm-image.tmpl?

The phrase "live image" suggests a live USB stick that can be booted as
a live system.  To avoid confusion, would it be better to just call it a
"VM image"?  I don't feel that strongly about it.

> (define this-file
>   (local-file "./vm-live.tmpl" "config.scm"))

Cute!  But FYI, I tried building this and it failed at first, since I
renamed the file and forgot about this reference.  I then re-ran the
command from the same directory as the file, which of course worked.

I built it, booted with 1 CPU and 1024 MB of RAM, logged in, and
successfully built GNU Hello.  I confirmed that changes persist across
reboots, and that the file system appears to have about 16 GB of space
left.

When I booted into the system, I noticed some things:

- There are many folders on the desktop.  For example, there is one for
  "/sys/fs/cgroup/memory".  A minor blemish, but rather odd.

- The "guest" user cannot run "sudo".  However, after you invoke
  "passwd" and change the password to something, you can invoke sudo.  I
  guess this is a side effect of using an empty password.  Would it be
  hard to arrange for the "guest" user to be allowed passwordless sudo?
  I guess I should have listened to your comment in the config file and
  given root/guest a password from the start!

- "sudo herd status" reports that term-auto and user-homes is stopped,
  which I think might be intended, but I'm not sure.  I also see
  possibly unnecessary services: wpa-supplicant, upower-daemon.  The VM
  probably doesn't need wpa-supplicant, but maybe it needs
  upower-daemon?  Not sure.

- There is a "Web Browser" link at the bottom of the screen.  It begs to
  be clicked.  But when you click it, the system asks you to choose a
  default web browser, and there are no options.  Hmm.

- "sudo guix system reconfigure /etc/config.scm" failed because "error:
  missing field initializers (home-directory)".  I then ran "guix pull"
  and tried again with "sudo -E guix system reconfigure
  /etc/config.scm", since I remembered that the proper way to do this
  using my freshly pulled Guix would be to use -E.  I got a new error:
  "error: device '/dev/vda' not found: No such file or directory", which
  occurred because the VM has /dev/sda but not /dev/vda.  Maybe we can
  modify the config file to use sda instead?

I hope that helps.  I think it's better to provide a VM image than none
at all.  Even though it's a little bumpy, it's still a decent way to try
out Guix if you don't want to install it on your existing distro.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: New VM image

[Ludovic Courtès]

Hello Chris,

Chris Marusich <cmmarusich@gmail.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:

[...]

>> I think it would fill the need of those who want to see what Guix is
>> like, and also the need of developers who want to test their software on
>> Guix without installing it (I’d argue that installing Guix on top of
>> their distro is simpler, but some people will use the VM image for that
>> purpose anyway.)
>>
>> What do people think?
>
> I like it!  Is it intended to replace gnu/system/examples/vm-image.tmpl?

Yes.  I’ve just committed it (538b99f31fb3eb41f90d6eb0e658e26e72123553)
before seeing your message, but I’m of course happy to take your
feedback into account.

> The phrase "live image" suggests a live USB stick that can be booted as
> a live system.  To avoid confusion, would it be better to just call it a
> "VM image"?  I don't feel that strongly about it.

Yes, definitely, sorry for the confusion.  I do not plan to change the
“VM” terminology in the manual and on the download page.

>> (define this-file
>>   (local-file "./vm-live.tmpl" "config.scm"))
>
> Cute!  But FYI, I tried building this and it failed at first, since I
> renamed the file and forgot about this reference.  I then re-ran the
> command from the same directory as the file, which of course worked.

Yeah, we could maybe extract the actual file name from
(current-source-location).

> When I booted into the system, I noticed some things:
>
> - There are many folders on the desktop.  For example, there is one for
>   "/sys/fs/cgroup/memory".  A minor blemish, but rather odd.

Yeah, that’s a longstanding issue with GLib/GIO: that library has code
to find out which mount points are “system” mount points (things that
should not be shown in the UI), but for some reason, it doesn’t work
correctly for us:

  https://lists.gnu.org/archive/html/help-guix/2017-11/msg00087.html

Any help to fix or work around the issue would be welcome!

> - The "guest" user cannot run "sudo".  However, after you invoke
>   "passwd" and change the password to something, you can invoke sudo.  I
>   guess this is a side effect of using an empty password.  Would it be
>   hard to arrange for the "guest" user to be allowed passwordless sudo?
>   I guess I should have listened to your comment in the config file and
>   given root/guest a password from the start!

What would it take to allow for password-less sudo?  We could do that.

> - "sudo herd status" reports that term-auto and user-homes is stopped,
>   which I think might be intended, but I'm not sure.  I also see
>   possibly unnecessary services: wpa-supplicant, upower-daemon.  The VM
>   probably doesn't need wpa-supplicant, but maybe it needs
>   upower-daemon?  Not sure.

True, the VM currently uses NM + wpa_supplicant, which we could
certainly replace with the DHCP client.

> - There is a "Web Browser" link at the bottom of the screen.  It begs to
>   be clicked.  But when you click it, the system asks you to choose a
>   default web browser, and there are no options.  Hmm.

Yeah, you first need to install one.

> - "sudo guix system reconfigure /etc/config.scm" failed because "error:
>   missing field initializers (home-directory)".

That error will vanish when we update ‘guix’ i guess.

>   I then ran "guix pull"
>   and tried again with "sudo -E guix system reconfigure
>   /etc/config.scm", since I remembered that the proper way to do this
>   using my freshly pulled Guix would be to use -E.  I got a new error:
>   "error: device '/dev/vda' not found: No such file or directory", which
>   occurred because the VM has /dev/sda but not /dev/vda.  Maybe we can
>   modify the config file to use sda instead?

It all depends on how you start QEMU: the device would be /dev/vda if
you use virtio, and /dev/hda otherwise.  In the manual, we should
probably give the incantation that leads it to use virtio:

  -device virtio-blk,drive=myhd -drive if=none,file=/tmp/t.img,id=myhd

The obvious downside is that it’s more verbose than just “-hda”.  The
upside is that it’ll perform better.

Thoughts?

Thanks,
Ludo’.

Re: New VM image

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 847 bytes --]

Hi Ludo,
Hi Chris,

> What would it take to allow for password-less sudo?  We could do that.

Example from /etc/sudoers:

guest ALL=NOPASSWD:/run/current-system/profile/bin/mount /x

> > - "sudo herd status" reports that term-auto and user-homes is stopped,
> >   which I think might be intended, but I'm not sure.

user-homes is a one-shot service, so technically it's good if it's stopped.
Maybe it makes sense to say that it's a one-shot service, or not list it
at all by default or something.

term-auto is cheating in order to be able to present a login on the serial
console if the Linux kernel is started with a serial console.  If the
Linux kernel is started without a serial console (normal for x86_64 laptops
etc), then term-auto will just stop.  It's mostly intended for ARM machines
(and x86_64 servers, I guess).

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: New VM image

[Ludovic Courtès]

Ludovic Courtès <ludo@gnu.org> skribis:

> Chris Marusich <cmmarusich@gmail.com> skribis:

[...]

>>> (define this-file
>>>   (local-file "./vm-live.tmpl" "config.scm"))
>>
>> Cute!  But FYI, I tried building this and it failed at first, since I
>> renamed the file and forgot about this reference.  I then re-ran the
>> command from the same directory as the file, which of course worked.
>
> Yeah, we could maybe extract the actual file name from
> (current-source-location).

Done in 1ac6c33caec2c2a4aaadda49cb0febb5bfbf264c.

>> When I booted into the system, I noticed some things:
>>
>> - There are many folders on the desktop.  For example, there is one for
>>   "/sys/fs/cgroup/memory".  A minor blemish, but rather odd.
>
> Yeah, that’s a longstanding issue with GLib/GIO: that library has code
> to find out which mount points are “system” mount points (things that
> should not be shown in the UI), but for some reason, it doesn’t work
> correctly for us:
>
>   https://lists.gnu.org/archive/html/help-guix/2017-11/msg00087.html

Fixed in 74685a4369033e79a776c5dec1c8480e8b446f6b.  The solution turned
out to be disappointingly trivial and is another example of this Unix
baggage we’re still carrying…

>> - "sudo herd status" reports that term-auto and user-homes is stopped,
>>   which I think might be intended, but I'm not sure.  I also see
>>   possibly unnecessary services: wpa-supplicant, upower-daemon.  The VM
>>   probably doesn't need wpa-supplicant, but maybe it needs
>>   upower-daemon?  Not sure.
>
> True, the VM currently uses NM + wpa_supplicant, which we could
> certainly replace with the DHCP client.

Done 

>>   I then ran "guix pull"
>>   and tried again with "sudo -E guix system reconfigure
>>   /etc/config.scm", since I remembered that the proper way to do this
>>   using my freshly pulled Guix would be to use -E.  I got a new error:
>>   "error: device '/dev/vda' not found: No such file or directory", which
>>   occurred because the VM has /dev/sda but not /dev/vda.  Maybe we can
>>   modify the config file to use sda instead?
>
> It all depends on how you start QEMU: the device would be /dev/vda if
> you use virtio, and /dev/hda otherwise.  In the manual, we should
> probably give the incantation that leads it to use virtio:
>
>   -device virtio-blk,drive=myhd -drive if=none,file=/tmp/t.img,id=myhd

I’ve done that in 05b60752a67af294c35d8f4218487c9a44b5fdee.

I’ve also fixed sudo along the lines of what Danny suggested in commit
6e0521eeede6bd06bc083073145413a04771aad3.

I think it’s pretty decent now, or at least there are fewer rough edges!
:-)

Ludo’.

Re: New VM image

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 198 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> I think it’s pretty decent now, or at least there are fewer rough edges!
> :-)

That's fantastic!  I'm glad my feedback was helpful.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]