Automatically building packages affected by submitted patches

Automatically building packages affected by submitted patches

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 2785 bytes --]

Hey,

In short, following on from some previous emails about Patchwork [1] and
tracking and inspecting how Guix changes over time [2], I've got to the
point where I have a very rough setup for building packages changed by
patches sent to the guix-patches mailing list.

1: http://lists.gnu.org/archive/html/guix-devel/2018-10/msg00638.html
2: https://lists.gnu.org/archive/html/guix-devel/2019-02/msg00089.html

If you want to see this in action, assuming for a given patch series
that all goes to plan, patches sent to the guix-patches mailing list as
normal should appear in the Patchwork web interface [3], and after some
processing, a specification should be inserted in to this Cuirass
instances [4] which builds the packages affected by the patch series.

3: https://patchwork.cbaines.net/project/guix-patches/list/
4: https://cuirass.cbaines.net/

For a specific example, here [5] is series 700 (a Patchwork id). There
are a number of intermediate steps, but this is the specification in
Cuirass [6].

5: https://patchwork.cbaines.net/project/guix-patches/list/?series=700
6: https://cuirass.cbaines.net/jobset/series-700

If you want to see how this is held together at the moment, the main
script can be found here [7]. That script repeatedly in a loop:

 - Looks at patch series in the Patchwork database that have been
   processed through the patchwork-test-series job in Laminar.
 - Gets the recorded commits for the base commit and head commit.
 - Fetches the list of affected packages from the Guix Data Service.
 - Assuming the list is available, ensures that a Cuirass specification
   exists to build the affected packages.

7: https://laminar.cbaines.net/cfg/scripts/guix-patchwork-helper

This is very much an initial prototype, and from this I've got a load of
ideas about how better to set things up.

In terms of features I'd like to work towards next, the main thing on my
mind is doing something useful with the data from building these
packages. With the goal of displaying a check in Patchwork about the
build status of the affected packages, I need to compare what's been
build by my Cuirass instance, with what https://ci.guix.info/ has
built. To do this, my current plan is to have the Guix Data Service
monitor a number of Cuirass instances somehow, extract information from
them and store it. You'd then be able to get a comparison for the build
status using the results gathered from the two Cuirass instances from
the Guix Data Service.

Going back to some of the aims I have with this, I think something like
this could help people review and test there own patch submissions, as
well as assisting those taking the time to review patches that others
have submitted.

There's a few more related things on my mind, but I'll end this email
here.

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Automatically building packages affected by submitted patches

[Ricardo Wurmus]

Hey Chris,

> In short, following on from some previous emails about Patchwork [1] and
> tracking and inspecting how Guix changes over time [2], I've got to the
> point where I have a very rough setup for building packages changed by
> patches sent to the guix-patches mailing list.

This is great!

> With the goal of displaying a check in Patchwork about the
> build status of the affected packages, I need to compare what's been
> build by my Cuirass instance, with what https://ci.guix.info/ has
> built. To do this, my current plan is to have the Guix Data Service
> monitor a number of Cuirass instances somehow, extract information from
> them and store it.

Would it not make sense to have the build farm perform the builds
instead of having a separate Cuirass instance?

>  - Looks at patch series in the Patchwork database that have been
>    processed through the patchwork-test-series job in Laminar.

Is the Laminar job something that could become part of Cuirass itself?

--
Ricardo

Re: Automatically building packages affected by submitted patches

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 3474 bytes --]


Ricardo Wurmus <rekado@elephly.net> writes:

>> In short, following on from some previous emails about Patchwork [1] and
>> tracking and inspecting how Guix changes over time [2], I've got to the
>> point where I have a very rough setup for building packages changed by
>> patches sent to the guix-patches mailing list.
>
> This is great!

I'm glad you think so :)

>> With the goal of displaying a check in Patchwork about the
>> build status of the affected packages, I need to compare what's been
>> build by my Cuirass instance, with what https://ci.guix.info/ has
>> built. To do this, my current plan is to have the Guix Data Service
>> monitor a number of Cuirass instances somehow, extract information from
>> them and store it.
>
> Would it not make sense to have the build farm perform the builds
> instead of having a separate Cuirass instance?

I'm using my Cuirass instance because it's easy to test with.

But yes, I can see advantages in using the main build farm to perform
the builds, especially in potentially providing substitutes more
promptly. Were you thinking of anything in particular?

>>  - Looks at patch series in the Patchwork database that have been
>>    processed through the patchwork-test-series job in Laminar.
>
> Is the Laminar job something that could become part of Cuirass itself?

The patchwork-test-series job [1] running within Laminar is doing a few
things:

 - It takes the patches from patchwork.cbaines.net and applies them to
   the master branch.

 - (part 1) It pushes the resulting Git commits/branch up to [2].

 - (part 2) It instructs the Guix Data Service to fetch the base and tip
   of the branch, and load the data from those commits.

 - (part 3) It also stores the hashes of the commits in the same
   database used by Patchwork.

The above 3 parts of the process are a bit complicated, so I'll attempt
to explain how they fit together.

If you want to instruct Cuirass to build all the packages that are new
or that have been changed, you need to know what the state was before
the patches, and what the state with the patches looks like. This is
where part 2 comes in. Once the Guix Data Service has processed the two
commits, it can provide this information.

This is where part 1 comes in providing the patches in a more machine
readable form to the Guix Data Service and Cuirass, as they both fetch
from the Git branch created in part 1.

The Guix Data Service takes time to work out what's changed between the
two revisions. This is where storing the commit hashes against the
Patchwork series in the same database as Patchwork (part 3) comes
in. That allows regularly checking if there are any Patchwork series
that don't have a specification in Cuirass, and attempting to create one
using the information from the Guix Data Service.

1: https://laminar.cbaines.net/cfg/jobs/patchwork-test-series.run
2: https://git.cbaines.net/guix/patches

So back to your question, I don't think any of the above would fit
neatly within Cuirass. I'm still thinking of Cuirass as a build service,
you describe what you want it to build, and it goes away and does that.

One thing I've become aware of recently is how similar Cuirass and the
Guix Data Service actually are. The evaluations part of Cuirass and
storing the derivations is very similar to the Guix Data Service.

Do let me know if you have more thoughts or questions...

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Automatically building packages affected by submitted patches

[Ludovic Courtès]

Hello!

Christopher Baines <mail@cbaines.net> skribis:

> For a specific example, here [5] is series 700 (a Patchwork id). There
> are a number of intermediate steps, but this is the specification in
> Cuirass [6].
>
> 5: https://patchwork.cbaines.net/project/guix-patches/list/?series=700
> 6: https://cuirass.cbaines.net/jobset/series-700

Amazing!  Looks like you’re really close to reaching your initial goal!

> In terms of features I'd like to work towards next, the main thing on my
> mind is doing something useful with the data from building these
> packages. With the goal of displaying a check in Patchwork about the
> build status of the affected packages, I need to compare what's been
> build by my Cuirass instance, with what https://ci.guix.info/ has
> built. To do this, my current plan is to have the Guix Data Service
> monitor a number of Cuirass instances somehow, extract information from
> them and store it. You'd then be able to get a comparison for the build
> status using the results gathered from the two Cuirass instances from
> the Guix Data Service.

I’m also not sure why you need to compare things.  Looking at
<https://cuirass.cbaines.net/jobset/series-700> and
<https://cuirass.cbaines.net/specifications>, it looks like Cuirass is
already building the subset of packages that depend on the modified
package(s):

  "subset":["r-dnacopy@1.56.0","r-dnacopy@1.56.0","coq-flocq@3.1.0","coq-interval@3.4.0"]

(Though what is “r-dnacopy” doing here?)

So I would think that you only need to check the status of these 4 jobs,
no?  What would you need to ask ci.guix.info?

Going forward, I agree with Ricardo that we could start running all this
on ci.guix.info, whenever you think is the right time.  I guess we could
start with the Guix Data Service, which is already useful in itself.

Thanks!

Ludo’.

Re: Automatically building packages affected by submitted patches

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 3071 bytes --]


Ludovic Courtès <ludo@gnu.org> writes:

> Christopher Baines <mail@cbaines.net> skribis:
>
>> For a specific example, here [5] is series 700 (a Patchwork id). There
>> are a number of intermediate steps, but this is the specification in
>> Cuirass [6].
>>
>> 5: https://patchwork.cbaines.net/project/guix-patches/list/?series=700
>> 6: https://cuirass.cbaines.net/jobset/series-700
>
> Amazing!  Looks like you’re really close to reaching your initial goal!

Thanks :)

>> In terms of features I'd like to work towards next, the main thing on my
>> mind is doing something useful with the data from building these
>> packages. With the goal of displaying a check in Patchwork about the
>> build status of the affected packages, I need to compare what's been
>> build by my Cuirass instance, with what https://ci.guix.info/ has
>> built. To do this, my current plan is to have the Guix Data Service
>> monitor a number of Cuirass instances somehow, extract information from
>> them and store it. You'd then be able to get a comparison for the build
>> status using the results gathered from the two Cuirass instances from
>> the Guix Data Service.
>
> I’m also not sure why you need to compare things.  Looking at
> <https://cuirass.cbaines.net/jobset/series-700> and
> <https://cuirass.cbaines.net/specifications>, it looks like Cuirass is
> already building the subset of packages that depend on the modified
> package(s):
>
>   "subset":["r-dnacopy@1.56.0","r-dnacopy@1.56.0","coq-flocq@3.1.0","coq-interval@3.4.0"]
>
> (Though what is “r-dnacopy” doing here?)

The query in the Guix data service to determine what's changed between
two revisions can't handle with two packages having the same name. It's
been fixed now, but there were two definitions of the r-dnacopy package.

> So I would think that you only need to check the status of these 4 jobs,
> no?  What would you need to ask ci.guix.info?

Yeah, knowing what the after effect of the patches are is pretty useful,
but I think that being able to compare the before and after state would
give an even more complete picture.

Consider if a patch affects 10 packages, and with that patch, 6 of the
ten affected packages fail to build. If previously all 10 of those
packages build successfully, then maybe this patch is a bit of a step
backwards. However, if previously all 10 of these packages failed to
build, then this patch is a definite improvement.

So, I think it would be useful to gather the information from
ci.guix.info about what packages have built and failed to build so that
this can be used to reveal more about the effect of patches.

> Going forward, I agree with Ricardo that we could start running all this
> on ci.guix.info, whenever you think is the right time.  I guess we could
> start with the Guix Data Service, which is already useful in itself.

Great, I'm not sure it's quite ready yet, but I think setting up the
Guix Data Service as more of a proper thing would be a good step
forward.

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]