Re: Parallel guix builds can trample?

[Phil <phil@beadling.co.uk>]

Hi - more details below.

Ricardo Wurmus writes:

>
> How are you using Guix with this?  Do you generate Guix package
> expressions?  Do you use “guix build --with-commit”?
>

The situation is like this - if we had a directory of clones of my
channel:

- pr-1
- pr-2
- pr-3
- pr-4
... and so on

Initially all the clones are taken from the master branch of my
channel and are all identical - but we change the version and commit to
match the head of each PR branch as per below.

Each clone looks like this:
- pr-1
      - my-package.scm
- pr-2
      - my-package.scm
and so on....

Each my-package.scm has a package like below - the inital packages are all
identical, but my system effectively seds the version and commit values
like the below.  These values are never committed back to master they
are used only as local channels to build each PR to test each build
still passes.

(define-public my-package
  (package
    (name "my-package")
    (version "this-is-different-for-each-pr")  ;; replace master version
    (source
      (git-checkout
        (url "ssh://same@repo:7999/same/repo.git")
        (commit "this-is-different-for-each-pr") ;; replace master version 
everything else remains the same in the package....


At this point we have lots of local channels referencing different commits, in
the same package, ready to build - so I spawn them all simultaneously -
the equivalent pseudo-shell that I will mock up today would be:

# define some sort of return code array:
RC=[]

for dir in pr-dirs
  RC[${dir}]=`guix build -K -L ${dir} my-package & 2>&1 > /tmp/${dir}.log`  # note the ampersand
wait

for rc in $RC
  if $rc.value != 0:
    report the failure of build $rc.key

What I'm seeing occasionally is that the logs and return code for say directory pr-1
and appearing in the guix build for pr-3 or pr-6 instead.

We know this becuse the code is different enough in pr-1 that it's logs
are unique across all the PRs.  We can also check the source code if the
build fails using --keep-failed to show it doesn't match the commit id
in the package used to build it.

Hopefully that makes sense?  I can post the actual shell script once
I've written the mock.