System tests

System tests

[Ludovic Courtès]

Hello Guix!

I just pushed a couple of new modules to run system tests.  As done in
NixOS¹ (in Perl + Bash), the idea is to run the system under test in a
VM and to instrument the guest system with a backdoor so we can fiddle
with it from the outside.

Here, the backdoor is obviously a Guile REPL, which is pretty cool.  :-)

Currently there’s only one test series, in (gnu tests base).  You’re
welcome to look at it and comment!  The ‘marionette’ procedures used in
there are what allows us to execute code in the system under test.

In the system under test, we can of course call any system call that
Guile provides, we can use (gnu system herd) to query the Shepherd, etc.
The ‘marionette-type’ procedure allows us to simulate typing at the
keyboard, although that’s a bit harder to use due to lack of
synchronization.

Anyway, to run the test:

  make check-system

Feedback welcome!

Ludo’.

¹ https://github.com/NixOS/nixpkgs/blob/master/nixos/lib/test-driver/Machine.pm

Commit 958dd3ce68 will break emacs interface after "guix pull"

[Alex Kost]

Note: if you use emacs interface from a git checkout [as described in
the bottom of (info "(guix) Emacs Initial Setup")], everything should
work as expected.

Hello Ludo, this is not related to system tests, but in this patchset
you also moved 'memoize' to (guix combinators).  In commit 86670ca4b7 I
adjusted emacs interface to use it.

However, it is not enough (see [1]): currently, if a user does "guix
pull", (s)he gets the recent guile code with the new (guix combinators)
module, but the emacs stuff is still old as it comes with the
development snapshot.

So in this case emacs interface will not work (complaining about unbound
'memoize' variable).  So I think it's the time to update guix-devel
package.  Is it OK to do it now?

In general, it is not perfect that "guix pull" updates only guile
modules, and not emacs interface.  I recall there were similar (but much
smaller) breakages when emacs stuff and guile code (from
"~/.config/guix/latest") were out of sync.

I'm just whining, I don't have a wish to improve "guix pull" :-)

[1] https://gnunet.org/bot/log/guix/2016-05-05#T1020612

-- 
Alex

Re: Commit 958dd3ce68 will break emacs interface after "guix pull"

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Hello Ludo, this is not related to system tests, but in this patchset
> you also moved 'memoize' to (guix combinators).  In commit 86670ca4b7 I
> adjusted emacs interface to use it.

Ooops, sorry for the breakage.  I guess I don’t master ‘grep’ yet.  ;-)

> However, it is not enough (see [1]): currently, if a user does "guix
> pull", (s)he gets the recent guile code with the new (guix combinators)
> module, but the emacs stuff is still old as it comes with the
> development snapshot.
>
> So in this case emacs interface will not work (complaining about unbound
> 'memoize' variable).

Indeed.

> So I think it's the time to update guix-devel package.  Is it OK to do
> it now?

Yes, we could do that, but the problem remains on foreign distros
though.

What about simply #:re-export (memoize) in (guix utils) for now?

> In general, it is not perfect that "guix pull" updates only guile
> modules, and not emacs interface.  I recall there were similar (but much
> smaller) breakages when emacs stuff and guile code (from
> "~/.config/guix/latest") were out of sync.

Yes, that’s definitely one of the shortcomings we should address:

  http://debbugs.gnu.org/cgi/bugreport.cgi?bug=22629

Thanks for the report!

Ludo’.

Re: Commit 958dd3ce68 will break emacs interface after "guix pull"

[Alex Kost]

[-- Attachment #1: Type: text/plain, Size: 1871 bytes --]

Ludovic Courtès (2016-05-06 13:36 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> Hello Ludo, this is not related to system tests, but in this patchset
>> you also moved 'memoize' to (guix combinators).  In commit 86670ca4b7 I
>> adjusted emacs interface to use it.
>
> Ooops, sorry for the breakage.  I guess I don’t master ‘grep’ yet.  ;-)

No problem, I keep an eye emacs UI :-)

>> However, it is not enough (see [1]): currently, if a user does "guix
>> pull", (s)he gets the recent guile code with the new (guix combinators)
>> module, but the emacs stuff is still old as it comes with the
>> development snapshot.
>>
>> So in this case emacs interface will not work (complaining about unbound
>> 'memoize' variable).
>
> Indeed.
>
>> So I think it's the time to update guix-devel package.  Is it OK to do
>> it now?
>
> Yes, we could do that, but the problem remains on foreign distros
> though.

Not really: on foreign distros the recommended way (it's probably the
only way except using git checkout) to use emacs interface is to install
guix into a user profile, so once we update guix-devel snapshot, it can
be updated by a user, and this breakage will be fixed.

> What about simply #:re-export (memoize) in (guix utils) for now?

Oh, indeed, this is a good workaround, thanks!  OK for the attached
patch or should I add some comment there?

>> In general, it is not perfect that "guix pull" updates only guile
>> modules, and not emacs interface.  I recall there were similar (but much
>> smaller) breakages when emacs stuff and guile code (from
>> "~/.config/guix/latest") were out of sync.
>
> Yes, that’s definitely one of the shortcomings we should address:
>
>   http://debbugs.gnu.org/cgi/bugreport.cgi?bug=22629

Yes, I'm aware of this, thanks (and sorry that I'm not the taker).


[-- Attachment #2: 0001-guix-utils-Re-export-memoize.patch --]
[-- Type: text/x-patch, Size: 983 bytes --]

From e07a1bc4b33f232f3812b687caca2c34ddfcf339 Mon Sep 17 00:00:00 2001
From: Alex Kost <alezost@gmail.com>
Date: Fri, 6 May 2016 21:19:06 +0300
Subject: [PATCH] guix: utils: Re-export 'memoize'.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Suggested by Ludovic Courtès <ludo@gnu.org>.

* guix/utils.scm: Re-export 'memoize' to avoid a potential breakage of
  emacs interface.  See
  <http://lists.gnu.org/archive/html/guix-devel/2016-05/msg00146.html>
  for details.
---
 guix/utils.scm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/guix/utils.scm b/guix/utils.scm
index f18bbd1..64ecdd7 100644
--- a/guix/utils.scm
+++ b/guix/utils.scm
@@ -44,6 +44,7 @@
   #:use-module (ice-9 format)
   #:use-module ((ice-9 iconv) #:select (bytevector->string))
   #:use-module (system foreign)
+  #:re-export (memoize)
   #:export (bytevector->base16-string
             base16-string->bytevector
 
-- 
2.7.3

Re: Commit 958dd3ce68 will break emacs interface after "guix pull"

[Ludovic Courtès]

Alex Kost <alezost@gmail.com> skribis:

> Ludovic Courtès (2016-05-06 13:36 +0300) wrote:

[...]

>>> So I think it's the time to update guix-devel package.  Is it OK to do
>>> it now?
>>
>> Yes, we could do that, but the problem remains on foreign distros
>> though.
>
> Not really: on foreign distros the recommended way (it's probably the
> only way except using git checkout) to use emacs interface is to install
> guix into a user profile, so once we update guix-devel snapshot, it can
> be updated by a user, and this breakage will be fixed.

At any rate, I’ll probably update it very soon.

>> What about simply #:re-export (memoize) in (guix utils) for now?
>
> Oh, indeed, this is a good workaround, thanks!  OK for the attached
> patch or should I add some comment there?


[...]

> --- a/guix/utils.scm
> +++ b/guix/utils.scm
> @@ -44,6 +44,7 @@
>    #:use-module (ice-9 format)
>    #:use-module ((ice-9 iconv) #:select (bytevector->string))
>    #:use-module (system foreign)
> +  #:re-export (memoize)

Just add a comment there, like “for backwards compatibility”.

Thank you!

Ludo’.

Re: Commit 958dd3ce68 will break emacs interface after "guix pull"

[Alex Kost]

Ludovic Courtès (2016-05-08 19:04 +0300) wrote:

> Alex Kost <alezost@gmail.com> skribis:
>
>> Ludovic Courtès (2016-05-06 13:36 +0300) wrote:
 [...]
>>> What about simply #:re-export (memoize) in (guix utils) for now?
>>
>> Oh, indeed, this is a good workaround, thanks!  OK for the attached
>> patch or should I add some comment there?
>
> [...]
>
>> --- a/guix/utils.scm
>> +++ b/guix/utils.scm
>> @@ -44,6 +44,7 @@
>>    #:use-module (ice-9 format)
>>    #:use-module ((ice-9 iconv) #:select (bytevector->string))
>>    #:use-module (system foreign)
>> +  #:re-export (memoize)
>
> Just add a comment there, like “for backwards compatibility”.

Done and commited, thanks.

-- 
Alex

Re: System tests

[Ludovic Courtès]

Hello!

ludo@gnu.org (Ludovic Courtès) skribis:

> I just pushed a couple of new modules to run system tests.  As done in
> NixOS¹ (in Perl + Bash), the idea is to run the system under test in a
> VM and to instrument the guest system with a backdoor so we can fiddle
> with it from the outside.
>
> Here, the backdoor is obviously a Guile REPL, which is pretty cool.  :-)
>
> Currently there’s only one test series, in (gnu tests base).  You’re
> welcome to look at it and comment!  The ‘marionette’ procedures used in
> there are what allows us to execute code in the system under test.
>
> In the system under test, we can of course call any system call that
> Guile provides, we can use (gnu system herd) to query the Shepherd, etc.
> The ‘marionette-type’ procedure allows us to simulate typing at the
> keyboard, although that’s a bit harder to use due to lack of
> synchronization.
>
> Anyway, to run the test:
>
>   make check-system

Commit 94b4274d0dc5768bac255980c7e785bd3dff261f adds a (pretty
expensive) test for the installation procedure, in (gnu tests install).

It works by:

  1. Rebuilding the current Guix, i.e., the Guix in the current load
     path; see ‘current-guix’ in (gnu packages package-management) for
     the tricks that are used.

  2. Building the installation image defined in (gnu system install)
     with:

        a. the current Guix (instead of the ‘guix-devel’ snapshot
           necessary so we know what packages it will use);

        b. the marionette service for test instrumentation;

        c. additional GC roots added via the new ‘gc-root-service-type’,
           to ensure that the store items needed by the system we will
           install are already available in the installation image (this
           is necessary since the installation image has no network
           access and wouldn’t be able to download/build stuff as
           happens “in real life”.)

  3. Booting this installation image and running the installation
     procedure as documented in “System Installation”.  The result is a
     qcow2 image containing the installed system.

  4. Booting the installed system and running the “basic” test series
     described above and defined in (gnu tests base).

This proved to be more involved than I initially hoped for, but I like
the end result.  :-)

Next step is to use it to test the fix for encrypted roots at
<http://bugs.gnu.org/21843>.

If everything goes well, Hydra will eventually run these tests
(currently evaluation is failing badly…).

Comments welcome!

Ludo’.

Re: System tests

[Ludovic Courtès]

ludo@gnu.org (Ludovic Courtès) skribis:

> If everything goes well, Hydra will eventually run these tests

It does!

  https://hydra.gnu.org/job/gnu/master/test.basic.x86_64-linux
  https://hydra.gnu.org/job/gnu/master/test.installed-os.x86_64-linux
  …

Ludo’.