Testing the installer

Testing the installer

[Ludovic Courtès]

Hello Guix!

One of the lessons from the 1.0.0 screw-up was that we should test the
graphical installer itself:

  https://guix.gnu.org/blog/2019/gnu-guix-1.0.1-released/

I think we should try to do that before the next release; not doing it
means testing by hand, which also takes a lot of time.

One idea that I had was that we could change the installer so that it
listens for connections on some pre-defined Unix-domain socket.  When it
gets a connection, it would perform a dialog with its client: sent it a
summary of the current form, wait for its choice, and so on.  The forms
would thus be unavailable from the keyboard: essentially ‘run-form’
calls would be replaced by ‘draw-form’ calls in that mode.

Our test infrastructure would thus connect (from a marionette) to the
installer and participate in that dialog.  It could even take a QEMU
screenshot at each step.

How does that sound?  Am I overlooking things?

I guess the difficulty will be to represent questions and answers.

Ludo’.

Re: Testing the installer

[sirgazil]

 ---- On Sun, 12 Jan 2020 17:13:54 -0500 Ludovic Courtès <ludo@gnu.org> wrote ----
 > Hello Guix!
 > 
 > One of the lessons from the 1.0.0 screw-up was that we should test the
 > graphical installer itself:

And, if possible, not test only using virtual machines, but real ones, because I remember my installation in a virtual machine went smoothly, but installing in my real machine was rough (network devices not detected, installer generated config.scm with truncated UUID, unknown error when trying to restart installation).

Re: Testing the installer

[sirgazil]

 ---- On Sun, 12 Jan 2020 18:34:54 -0500 sirgazil <sirgazil@zoho.com> wrote ----
 >  ---- On Sun, 12 Jan 2020 17:13:54 -0500 Ludovic Courtès <ludo@gnu.org> wrote ----
 >  > Hello Guix!
 >  > 
 >  > One of the lessons from the 1.0.0 screw-up was that we should test the
 >  > graphical installer itself:
 > 
 > And, if possible, not test only using virtual machines, but real ones, because I remember my installation in a virtual machine went smoothly, but installing in my real machine was rough (network devices not detected, installer generated config.scm with truncated UUID, unknown error when trying to restart installation).
 > 

By the way, in my old notes about my installation of the Guix System 1.0.0, I have something about a problem related to the order in which the installer checks for network availability. 

My computer had another OS installed at that time. If I recall correctly, I started the Guix System installation process, told the installer to use the whole disk for installation, and only after formatting the disk, the installer told me there was a connection problem, and I couldn't install the system. So I couldn't boot to the old OS at that point. Fortunately, people on #guix helped me solve the network problem (related to bug) and I could install the Guix System.

So, if this is still the case, I think the installer should check for network connection before attempting to do any destructive operation.

---
https://sirgazil.bitbucket.io/

Re: Testing the installer

[Mathieu Othacehe]

Hello Ludo,

That sounds like a nice idea! I guess we can discuss it more deeply
during Guix Days, but here are a few thoughts:

* Testing from a VM won't cover HW issues with undetected wifi networks
and kmscon rendering issues, but could cover most partitioning issues.

* This kind of client/server protocol would make even easier to
switch to a gtk graphical installer.

* Before the next release, it would be nice to fix the "installer always
fail when restarted" issue than has been bitten lots of people.

Thanks,

Mathieu

Ludovic Courtès writes:

> Hello Guix!
>
> One of the lessons from the 1.0.0 screw-up was that we should test the
> graphical installer itself:
>
>   https://guix.gnu.org/blog/2019/gnu-guix-1.0.1-released/
>
> I think we should try to do that before the next release; not doing it
> means testing by hand, which also takes a lot of time.
>
> One idea that I had was that we could change the installer so that it
> listens for connections on some pre-defined Unix-domain socket.  When it
> gets a connection, it would perform a dialog with its client: sent it a
> summary of the current form, wait for its choice, and so on.  The forms
> would thus be unavailable from the keyboard: essentially ‘run-form’
> calls would be replaced by ‘draw-form’ calls in that mode.
>
> Our test infrastructure would thus connect (from a marionette) to the
> installer and participate in that dialog.  It could even take a QEMU
> screenshot at each step.
>
> How does that sound?  Am I overlooking things?
>
> I guess the difficulty will be to represent questions and answers.
>
> Ludo’.

Re: Testing the installer

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 1177 bytes --]

Hello!

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

> That sounds like a nice idea! I guess we can discuss it more deeply
> during Guix Days, but here are a few thoughts:
>
> * Testing from a VM won't cover HW issues with undetected wifi networks
> and kmscon rendering issues, but could cover most partitioning issues.

Yeah, but for that we need manual testing, there’s no other option.

> * This kind of client/server protocol would make even easier to
> switch to a gtk graphical installer.

Well, if we replace the explicit Newt calls with a higher-level
interface, yes.

Besides, the protocol may prove useful for accessibility because it
provides semantics.

> * Before the next release, it would be nice to fix the "installer always
> fail when restarted" issue than has been bitten lots of people.

Yes!

I’ve pushed ‘wip-installer-test’, which implements said protocol.  See
screenshot below!  :-)

What’s left to do now is to write an actual test that uses it.

Speaking of which, does anyone know if there’s a way to share a
Unix-domain socket between a QEMU guest and its host?  (Over 9p?)

Thanks,
Ludo’.


[-- Attachment #2: the dialog --]
[-- Type: image/png, Size: 18497 bytes --]

Re: Testing the installer

[Mathieu Othacehe]

Hey!

> I’ve pushed ‘wip-installer-test’, which implements said protocol.  See
> screenshot below!  :-)

Woow, that's impressive :)

I just gave you access to Guile-newt repository so that you can
integrate your changes. I will create a new release once its done.

In the future, I would have liked to keep open the possibility of having
one installer with multiple graphical backends (newt, gtk, cli). That
would mean that we would need to factorize the protocol part in the
backend agnostic part of the installer.

Anyway, for now it would be completely over-engineered, as we don't know
yet if a new backend will come one day.

Please let me know if you want help on some point but, I think that I'll
try to tackle the "installer cannot restart issue".

Thanks,

Mathieu

Re: Testing the installer

[Gábor Boskovits]

Hello Mathieu,

Mathieu Othacehe <m.othacehe@gmail.com> ezt írta (időpont: 2020. jan.
23., Cs, 10:42):
>
>
> Hey!
>
> > I’ve pushed ‘wip-installer-test’, which implements said protocol.  See
> > screenshot below!  :-)
>
> Woow, that's impressive :)
>
> I just gave you access to Guile-newt repository so that you can
> integrate your changes. I will create a new release once its done.
>
> In the future, I would have liked to keep open the possibility of having
> one installer with multiple graphical backends (newt, gtk, cli). That
> would mean that we would need to factorize the protocol part in the
> backend agnostic part of the installer.
>
> Anyway, for now it would be completely over-engineered, as we don't know
> yet if a new backend will come one day.

This is a bit off topic, but I am about to create an automatic
installer, that is
a candidate for a new backend. It currently works by providing a configuration
record, and then executes a few things from the installer.

(It does a lookup for a candidate installation target disk, autopartitions it,
then injects the bootloader and file-system fields into an operating-system
template, and finally does a system installation.)

I will have a look at the new protocol soon, to see if this project
can benefit from that.

>
> Please let me know if you want help on some point but, I think that I'll
> try to tackle the "installer cannot restart issue".
>
> Thanks,
>
> Mathieu
>

Best regards,
g_bor
-- 
OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21

Re: Testing the installer

[Bengt Richter]

Hello Gábor,

On +2020-01-23 12:21:27 +0100, Gábor Boskovits wrote:
[...]

> This is a bit off topic, but I am about to create an automatic
> installer, that is
> a candidate for a new backend. It currently works by providing a configuration
> record, and then executes a few things from the installer.
> 
> (It does a lookup for a candidate installation target disk, autopartitions it,
> then injects the bootloader and file-system fields into an operating-system
> template, and finally does a system installation.)
>

Will your installer be able to create /boot and / images that will
work with Coreboot/SeaBios/grub legacy-only (not UEFI) bootable systems 
(like PureOS)?

IOW, what kind of BIOS systems is it dependent on?

> I will have a look at the new protocol soon, to see if this project
> can benefit from that.
[...]

> 
> Best regards,
> g_bor
> -- 
> OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21
> 

-- 
Regards,
Bengt Richter

Re: Testing the installer

[Gábor Boskovits]

Hello,

Bengt Richter <bokr@bokr.com> ezt írta (időpont: 2020. jan. 24., P, 0:08):
>
> Hello Gábor,
>
> On +2020-01-23 12:21:27 +0100, Gábor Boskovits wrote:
> [...]
>
> > This is a bit off topic, but I am about to create an automatic
> > installer, that is
> > a candidate for a new backend. It currently works by providing a configuration
> > record, and then executes a few things from the installer.
> >
> > (It does a lookup for a candidate installation target disk, autopartitions it,
> > then injects the bootloader and file-system fields into an operating-system
> > template, and finally does a system installation.)
> >
>
> Will your installer be able to create /boot and / images that will
> work with Coreboot/SeaBios/grub legacy-only (not UEFI) bootable systems
> (like PureOS)?

It is based on the current installation image, and it accepts a Guix
System configuration
template, so it works on systems that Guix System can currently operate on.
Right now the bootloader configuration is limited to that what the
autopartitoning in the
installer does for the option 'All in one partiton', which means that
UEFI or legacy is
autodetected, and installed accordingly. Does that answer you question?

>
> IOW, what kind of BIOS systems is it dependent on?
>
> > I will have a look at the new protocol soon, to see if this project
> > can benefit from that.
> [...]
>
> >
> > Best regards,
> > g_bor
> > --
> > OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21
> >
>
> --
> Regards,
> Bengt Richter


Best regards,
g_bor
-- 
OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21

Re: Testing the installer

[Ludovic Courtès]

Hi Gábor,

Gábor Boskovits <boskovits@gmail.com> skribis:

> This is a bit off topic, but I am about to create an automatic
> installer, that is
> a candidate for a new backend. It currently works by providing a configuration
> record, and then executes a few things from the installer.
>
> (It does a lookup for a candidate installation target disk, autopartitions it,
> then injects the bootloader and file-system fields into an operating-system
> template, and finally does a system installation.)
>
> I will have a look at the new protocol soon, to see if this project
> can benefit from that.

Yes, there may be things to share here.  Alternately, the automated
installer could directly poke at the installer’s APIs instead of being
an external program.

Something to discuss at the Guix Days!  :-)

Ludo’.

Re: Testing the installer

[Ludovic Courtès]

Hi!

Mathieu Othacehe <m.othacehe@gmail.com> skribis:

>> I’ve pushed ‘wip-installer-test’, which implements said protocol.  See
>> screenshot below!  :-)
>
> Woow, that's impressive :)

Glad you like it.  :-)

> I just gave you access to Guile-newt repository so that you can
> integrate your changes. I will create a new release once its done.

Thanks, will do!

> In the future, I would have liked to keep open the possibility of having
> one installer with multiple graphical backends (newt, gtk, cli). That
> would mean that we would need to factorize the protocol part in the
> backend agnostic part of the installer.
>
> Anyway, for now it would be completely over-engineered, as we don't know
> yet if a new backend will come one day.

Yeah, I think that can come later.

> Please let me know if you want help on some point but, I think that I'll
> try to tackle the "installer cannot restart issue".

It’d be great if you could address that issue!

For my part, I’ll try to get with at least one test to make sure the
approach is viable.  Then I’ll need help to write new tests and to
improve things here and there.

Also, for 1.0, we had also messed up with things like MATE.  I’d like us
to have system tests for the desktops we propose: GNOME, MATE, etc.
That would mean at least ensuring that a config like that of
‘desktop.tmpl’ allows us to log in via GDM, and then to start a terminal
in the desktop (I suppose we could do that using the keyboard, with
‘marionette-type’ etc.  NixOS tests are a good source of inspiration:

  https://github.com/NixOS/nixpkgs/blob/master/nixos/tests/gnome3.nix

Help welcome!

Ludo’.

Re: Testing the installer

[Jonathan Brielmaier]

On 12.01.20 23:13, Ludovic Courtès wrote:
> Hello Guix!
>
> One of the lessons from the 1.0.0 screw-up was that we should test the
> graphical installer itself:
>
>    https://guix.gnu.org/blog/2019/gnu-guix-1.0.1-released/
>
> I think we should try to do that before the next release; not doing it
> means testing by hand, which also takes a lot of time.

I did a quick test by hand and opened a bug:
https://issues.guix.gnu.org/issue/39199

Testing the installer

[swedebugia]

[-- Attachment #1: Type: text/plain, Size: 340 bytes --]

Hi
I see two branches with wip-installer.

What is the difference between those? Wip-installer2 having a newer master merged in? 

I would like to try installing via the installer to test it. Is it working? 

I have a system that already has Ubuntu.

Can the installer handle dual-boot detection/setup yet?
-- 
Cheers Swedebugia 

[-- Attachment #2: Type: text/html, Size: 384 bytes --]

Re: Testing the installer

[Ricardo Wurmus]

Hi swedebugia,

> I see two branches with wip-installer.
>
> What is the difference between those? Wip-installer2 having a newer master merged in?

The “wip-installer2” is newer and has seen more work than the original
“wip-installer”, which has been abandoned.  If you want to work on it I
recommend looking at “wip-installer2”.

> I would like to try installing via the installer to test it. Is it working?

I don’t know.  From what I remember it still had some rather rough edges
and was less guided than you would expect from other installers.

> Can the installer handle dual-boot detection/setup yet?

I don’t think so.

--
Ricardo