Getting Unit Test Results Out Of Guix Build

Getting Unit Test Results Out Of Guix Build

[Phil Beadling]

Hi all,

Given Guix builds packages in a container, is there any way of getting a
file out of the build container for further processing?

In my use case - I generate some junit test XML output during the build
process and want to render that to a webpage rather than send it to stdout
as part of the build process?

It is possible to export the nar file using guix archive --export, and
possible to keep failed build artifacts using -K, but there doesn't seem to
be a way to dump out a file from a successful build?

Is this against the philosphy of purely-functional builds?  Obviously it is
a side-effect to dump out an XML file, but no more so than sending build
logs to stdout?  Is it possible to dump outputs without breaking
referential transparency on the inputs - i.e. it should be possible to
configure a directory that is write-only for the container, but can be read
by users outside the container?

Or am I missing something obvious?

Thanks,
Phil.

Re: Getting Unit Test Results Out Of Guix Build

[raingloom]

On Mon, 25 Oct 2021 13:30:40 +0100
Phil Beadling <phil@beadling.co.uk> wrote:

> Hi all,
> 
> Given Guix builds packages in a container, is there any way of
> getting a file out of the build container for further processing?
> 
> In my use case - I generate some junit test XML output during the
> build process and want to render that to a webpage rather than send
> it to stdout as part of the build process?
> 
> It is possible to export the nar file using guix archive --export, and
> possible to keep failed build artifacts using -K, but there doesn't
> seem to be a way to dump out a file from a successful build?
> 
> Is this against the philosphy of purely-functional builds?  Obviously
> it is a side-effect to dump out an XML file, but no more so than
> sending build logs to stdout?  Is it possible to dump outputs without
> breaking referential transparency on the inputs - i.e. it should be
> possible to configure a directory that is write-only for the
> container, but can be read by users outside the container?
> 
> Or am I missing something obvious?
> 
> Thanks,
> Phil.

Couldn't this just be another package output? Maybe not the most elegant
solution, since those are not usually used this way, but it can sort of
work. Just add a phase that copies the logs to the "tests" output if
it exists. Kinda like how the "debug" is currently used.

Re: Getting Unit Test Results Out Of Guix Build

[Phil]

Hi,

raingloom writes:


> Couldn't this just be another package output? Maybe not the most elegant
> solution, since those are not usually used this way, but it can sort of
> work. Just add a phase that copies the logs to the "tests" output if
> it exists. Kinda like how the "debug" is currently used.

Yes this could work - so for my-package I could just define
my-package:tests which would add an extra test directory to the package
in the store which I would copy the test results into.

The files will be small so I could just leave them in the store, but
also I could remove the my-package:tests once I'm done using the file,
to keep the store tidy?

It might even be possible to write something that augments this onto a
package automatically.

I'll give this a whirl and if I produce anything useful I'll reply with
the details.

Thanks!

Re: Getting Unit Test Results Out Of Guix Build

[Phil Beadling]

Hi,

On Tue, 26 Oct 2021 at 19:31, Phil <phil@beadling.co.uk> wrote:

> Hi,
>
> raingloom writes:
>
>
> > Couldn't this just be another package output? Maybe not the most elegant
> > solution, since those are not usually used this way, but it can sort of
> > work. Just add a phase that copies the logs to the "tests" output if
> > it exists. Kinda like how the "debug" is currently used.
>
> Yes this could work - so for my-package I could just define
> my-package:tests which would add an extra test directory to the package
> in the store which I would copy the test results into.
>
>
I gave this a got it almost working with one significant snag I was hoping
someone might be able to help with?

Once the build is complete is possible to retrieve the location of the test
files using a simple REPL script - In my example this is nested in Groovy
script in my case which will interpolate the ${packageExpression} for me to
give a proper Scheme expression (this isn't important just mentioning it so
the script makes sense):













*(use-modules (guix packages)             (gnu packages)             (guix
store)             (guix derivations))(define my-drv  (with-store store
(run-with-store store      (package->derivation
${packageExpression}))))(format #t "~a~%" (assoc-ref
(derivation->output-paths my-drv) "test"))*


The below details are no overly important (IMHO), but script is fed into a
Groovy variable called testLocationSCript, and called from shell script
like so (localCannelSwitch is just a "-L /path/to/my/local/channel"
variable):
TESTDIR=`guix repl ${localChannelSwitch} -- <(cat <<<
'${testLocationScript}')`


This all works great *until* you actually have a unit test failure - then
because that unit test failure will fail the "guix build", the
"package->derivation" command seems to try to rebuild the package, which of
course fails again for exactly the same reason.as the original failure and
provides no location for derivation->output-paths.

*So what I need is a way of asking what will the output path be, without
actually building, or inspite of a build failure?*

This must be techincally possible because mid-build the output paths are
available - so they are known ahead of build completion.

So, does anyone have a way of getting the output directories from Guix
without first requiring a successful build?


Thanks!

Re: Getting Unit Test Results Out Of Guix Build

[raingloom]

On Thu, 4 Nov 2021 13:52:02 +0000
Phil Beadling <phil@beadling.co.uk> wrote:

> Hi,
> 
> On Tue, 26 Oct 2021 at 19:31, Phil <phil@beadling.co.uk> wrote:
> 
> > Hi,
> >
> > raingloom writes:
> >
> >  
> > > Couldn't this just be another package output? Maybe not the most
> > > elegant solution, since those are not usually used this way, but
> > > it can sort of work. Just add a phase that copies the logs to the
> > > "tests" output if it exists. Kinda like how the "debug" is
> > > currently used.  
> >
> > Yes this could work - so for my-package I could just define
> > my-package:tests which would add an extra test directory to the
> > package in the store which I would copy the test results into.
> >
> >  
> I gave this a got it almost working with one significant snag I was
> hoping someone might be able to help with?
> 
> Once the build is complete is possible to retrieve the location of
> the test files using a simple REPL script - In my example this is
> nested in Groovy script in my case which will interpolate the
> ${packageExpression} for me to give a proper Scheme expression (this
> isn't important just mentioning it so the script makes sense):
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> *(use-modules (guix packages)             (gnu packages)
> (guix store)             (guix derivations))(define my-drv
> (with-store store (run-with-store store      (package->derivation
> ${packageExpression}))))(format #t "~a~%" (assoc-ref
> (derivation->output-paths my-drv) "test"))*
> 
> 
> The below details are no overly important (IMHO), but script is fed
> into a Groovy variable called testLocationSCript, and called from
> shell script like so (localCannelSwitch is just a "-L
> /path/to/my/local/channel" variable):
> TESTDIR=`guix repl ${localChannelSwitch} -- <(cat <<<
> '${testLocationScript}')`
> 
> 
> This all works great *until* you actually have a unit test failure -
> then because that unit test failure will fail the "guix build", the
> "package->derivation" command seems to try to rebuild the package,
> which of course fails again for exactly the same reason.as the
> original failure and provides no location for
> derivation->output-paths.
> 
> *So what I need is a way of asking what will the output path be,
> without actually building, or inspite of a build failure?*
> 
> This must be techincally possible because mid-build the output paths
> are available - so they are known ahead of build completion.
> 
> So, does anyone have a way of getting the output directories from Guix
> without first requiring a successful build?
> 
> 
> Thanks!

The --dry-run option is the closest thing that I know of. Not sure what
its Scheme equivalent is.

Re: Getting Unit Test Results Out Of Guix Build

[Phil]

Hi,

raingloom writes:

> The --dry-run option is the closest thing that I know of. Not sure what
> its Scheme equivalent is.

Thanks yes - I think this suffers from a similar issue to my own
approach in that when a build fails, the store item is removed, along
with the test results contained within.  So getting the directory was
only half the battle.  In this case you could use "-K" and have
separate logic to deal with success (using a new output) and failure
(trawl /tmp for the results) but this strikes me as too
complicated. "-K" won't work with build offloading either.

And alternative approach I have been looking at is (carefully) using
--chroot-directory option on the daemon.  It's not ideal as it risks a
leaky container, but an extra directory can added to the build container
for the sole purpose of test results.  It seems to me if I make the
directory owned-by but read-only to my build pipeline account (which
calls guix and collects the results), and give the directory a group id
of guixbuild, but make the group write-only, then I can have a situation
where the extra directory doesn't pollute the container, allowing build
artifacts to flow outwards only, with no files being able to flow into
the build.

The problem I see with this is if (like me) you have a machines.scm
offloading builds you need a way of retrieving the artifacts off the
remote workers (I am assuming they won't sync automatically, but haven't
yet tested).  A crude solution to this is simply to NFS mount or similar
the chroot'ed directory so each worker shares the same directory.

Any more comments/ideas welcome - I'll report back if I get something
palatable working.