Patchwork + automated checking and testing of patches

Patchwork + automated checking and testing of patches

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 2636 bytes --]

Hey,

For a while, I've been thinking about ways in which contributing code
changes to Guix can be better, in particular, how things like
automatically checking and testing patches might be beneficial.

Some tooling for doing checks already exists, like guix lint, but you
can't run that on a patch, you need some way of getting from the text
sent to the mailing list to some state in a git repository.

I haven't used Patchwork [1], but it has been tried previously with Guix
[2][3]. It also seems to be a step forward in terms of taking what's
sent to the mailing list and organising it.

1: http://jk.ozlabs.org/projects/patchwork/
2: https://lists.gnu.org/archive/html/guix-devel/2016-05/msg00056.html
3: https://patchwork.sourceware.org/project/guix/list/

I've now written a very rough package and service for Patchwork [4], and
managed to setup a instance here [5]. With the help of an email account
subscribed to both guix-patches and guix-commits, getmail and a couple
of scripts, it should also collect new patches sent to guix-patches, and
mark those that have been merged to the master branch as "Accepted" [6].

4: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=33185
5: https://patchwork.cbaines.net/
6: https://patchwork.cbaines.net/project/guix-patches/list/?state=3

What I'm hoping now to do, is to try and use Patchwork to coordinate
automated checking and testing of patch series. I haven't thought much
about this yet, but Patchwork does have something for recording
"checks", demonstrated here [7]. There has also been a talk at FOSDEM
about this kind of thing [8] as well as a blog post with more detail
[9].

7: https://patches.dpdk.org/patch/43055/
8: https://archive.fosdem.org/2017/schedule/event/patchwork_jenkins/
9: https://that.guru/blog/patchwork-and-ci-in-a-tree/

I don't intend to do anything with Jenkins, as I think that wouldn't be
maintainable, but I think setting up some system to check some of the
following would be beneficial:

 - If a patch series applies to master
 - If ./configure and make run successfully
 - If building a guix package with the patch applied works
 - If running guix lint for all the new/changed packages works
 - If building all the new/changed packages works
 - If running the tests and system tests works

I think there are also some review activities that are better done at
scale. One example I can think of is comparing the contents of source
tarballs and store outputs for new packages to those of existing
packages. If there is some overlap, it may suggest that there is some
bundled code/data that could be removed.

Let me know if you have any thoughts,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Patchwork + automated checking and testing of patches

[Tobias Geerinckx-Rice]

Christopher,

Christopher Baines wrote:
> I've now written a very rough package and service for Patchwork 
> [4], and
> managed to setup a instance here [5]. With the help of an email 
> account
> subscribed to both guix-patches and guix-commits, getmail and a 
> couple
> of scripts, it should also collect new patches sent to 
> guix-patches, and
> mark those that have been merged to the master branch as 
> "Accepted" [6].
>
> 4: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=33185
> 5: https://patchwork.cbaines.net/
> 6: 
> https://patchwork.cbaines.net/project/guix-patches/list/?state=3

Niiice. If that's ‘rough‘, I look forward to what's to come.

A lost battle, perhaps, but does Patchwork provide an option to 
mangle e-mail addresses?

Kind regards,

T G-R

Re: Patchwork + automated checking and testing of patches

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1235 bytes --]


Tobias Geerinckx-Rice <me@tobias.gr> writes:

> Christopher Baines wrote:
>> I've now written a very rough package and service for Patchwork [4],
>> and
>> managed to setup a instance here [5]. With the help of an email
>> account
>> subscribed to both guix-patches and guix-commits, getmail and a
>> couple
>> of scripts, it should also collect new patches sent to guix-patches,
>> and
>> mark those that have been merged to the master branch as "Accepted"
>> [6].
>>
>> 4: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=33185
>> 5: https://patchwork.cbaines.net/
>> 6: https://patchwork.cbaines.net/project/guix-patches/list/?state=3
>
> Niiice. If that's ‘rough‘, I look forward to what's to come.

Thanks :) It's the Patchwork package, service and me running getmail in
a screen session that's a bit rough.

> A lost battle, perhaps, but does Patchwork provide an option to mangle
> e-mail addresses?

Unfortunately, not that I can see. It should be possible to add though,
I can see that some preprocessing goes on for the text in comments for
example, and Patchwork maintains a people table with email addresses, so
you should be able to remove or alter any occurances of any email
addresses.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Patchwork + automated checking and testing of patches

[Ludovic Courtès]

Hello!

Christopher Baines <mail@cbaines.net> skribis:

> I haven't used Patchwork [1], but it has been tried previously with Guix
> [2][3]. It also seems to be a step forward in terms of taking what's
> sent to the mailing list and organising it.
>
> 1: http://jk.ozlabs.org/projects/patchwork/
> 2: https://lists.gnu.org/archive/html/guix-devel/2016-05/msg00056.html
> 3: https://patchwork.sourceware.org/project/guix/list/
>
> I've now written a very rough package and service for Patchwork [4], and
> managed to setup a instance here [5]. With the help of an email account
> subscribed to both guix-patches and guix-commits, getmail and a couple
> of scripts, it should also collect new patches sent to guix-patches, and
> mark those that have been merged to the master branch as "Accepted" [6].
>
> 4: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=33185
> 5: https://patchwork.cbaines.net/
> 6: https://patchwork.cbaines.net/project/guix-patches/list/?state=3

Back when we tried, it had a couple of shortcomings:

  1. It would not automatically detect which patches have been merged;

  2. It would not present patch series correctly.

From what you write it looks like #1 has been fixed, but the web
interface suggests that #2 isn’t quite fixed yet, is that correct?

> I don't intend to do anything with Jenkins, as I think that wouldn't be
> maintainable, but I think setting up some system to check some of the
> following would be beneficial:
>
>  - If a patch series applies to master
>  - If ./configure and make run successfully
>  - If building a guix package with the patch applied works
>  - If running guix lint for all the new/changed packages works
>  - If building all the new/changed packages works
>  - If running the tests and system tests works

I think these are all things we’d love to see.

Apart from item #1, the rest is pretty much Guix-specific.  My
suggestion would be to write tooling piecemeal: for example, things that
allow us to determine the set of packages added or upgraded by a patch
(we actually have a bit of that with ‘guix pull’ and (guix inferior)),
things to apply patches, etc.

We don’t want to reimplement Patchwork, GitLab, etc. so anything that
can be borrowed from there is welcome.

What might be nice is integration with Cuirass: if it had an HTTP API to
easily request a branch build, we could easily hook into it; and then,
we can extended its existing HTTP API as we see fit to make it easy to
retrieve info about build statuses and so on.

All this is easier said than done, but my point is that we should try
and identify easily doable bits so we can incrementally build such a
tool.

Thanks!

Ludo’.

Re: Patchwork + automated checking and testing of patches

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 3372 bytes --]


Ludovic Courtès <ludo@gnu.org> writes:

> Hello!
>
> Christopher Baines <mail@cbaines.net> skribis:
>
>> ...
>>
>> I've now written a very rough package and service for Patchwork [4], and
>> managed to setup a instance here [5]. With the help of an email account
>> subscribed to both guix-patches and guix-commits, getmail and a couple
>> of scripts, it should also collect new patches sent to guix-patches, and
>> mark those that have been merged to the master branch as "Accepted" [6].
>>
>> ...
>
> Back when we tried, it had a couple of shortcomings:
>
>   1. It would not automatically detect which patches have been merged;
>
>   2. It would not present patch series correctly.
>
> From what you write it looks like #1 has been fixed, but the web
> interface suggests that #2 isn’t quite fixed yet, is that correct?

On the detecting merged patches, that's definately working for some
patches though. I don't fully understand what criteria it's using
though, as it's comparing the commits that come through to the master
branch, and I bet it's possible to confuse it a bit by tweaking patches
before pushing them.

Regarding patch series, I don't know much about the specifics of this,
and I don't know much about Patchwork, but just comparing a few patches
on the older version [1], and the newer version [2], it looks like it's
better. Take this patch [3], it's part of a series, but you can't
tell. However, with this patch [4], you can see the series and related
patches towards the top of the page, and also a link to download the
whole series as an mbox. How does this look to you?

1: https://patchwork.sourceware.org/project/guix/list/
2: https://patchwork.cbaines.net/project/guix-patches/list/
3: https://patchwork.sourceware.org/patch/18558/
4: https://patchwork.cbaines.net/patch/66/

>> I don't intend to do anything with Jenkins, as I think that wouldn't be
>> maintainable, but I think setting up some system to check some of the
>> following would be beneficial:
>>
>>  - If a patch series applies to master
>>  - If ./configure and make run successfully
>>  - If building a guix package with the patch applied works
>>  - If running guix lint for all the new/changed packages works
>>  - If building all the new/changed packages works
>>  - If running the tests and system tests works
>
> I think these are all things we’d love to see.
>
> Apart from item #1, the rest is pretty much Guix-specific.  My
> suggestion would be to write tooling piecemeal: for example, things that
> allow us to determine the set of packages added or upgraded by a patch
> (we actually have a bit of that with ‘guix pull’ and (guix inferior)),
> things to apply patches, etc.
>
> We don’t want to reimplement Patchwork, GitLab, etc. so anything that
> can be borrowed from there is welcome.
>
> What might be nice is integration with Cuirass: if it had an HTTP API to
> easily request a branch build, we could easily hook into it; and then,
> we can extended its existing HTTP API as we see fit to make it easy to
> retrieve info about build statuses and so on.
>
> All this is easier said than done, but my point is that we should try
> and identify easily doable bits so we can incrementally build such a
> tool.

That all matches up well with what I've been thinking about :)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Patchwork + automated checking and testing of patches

[Ludovic Courtès]

Hello!

Christopher Baines <mail@cbaines.net> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:

[...]

>> Back when we tried, it had a couple of shortcomings:
>>
>>   1. It would not automatically detect which patches have been merged;
>>
>>   2. It would not present patch series correctly.
>>
>> From what you write it looks like #1 has been fixed, but the web
>> interface suggests that #2 isn’t quite fixed yet, is that correct?
>
> On the detecting merged patches, that's definately working for some
> patches though. I don't fully understand what criteria it's using
> though, as it's comparing the commits that come through to the master
> branch, and I bet it's possible to confuse it a bit by tweaking patches
> before pushing them.
>
> Regarding patch series, I don't know much about the specifics of this,
> and I don't know much about Patchwork, but just comparing a few patches
> on the older version [1], and the newer version [2], it looks like it's
> better. Take this patch [3], it's part of a series, but you can't
> tell. However, with this patch [4], you can see the series and related
> patches towards the top of the page, and also a link to download the
> whole series as an mbox. How does this look to you?

It looks better than the old Patchwork version, but I think
<https://patchwork.cbaines.net/project/guix-patches/list/> still leaves
a bit to be desired.  For instance, it shows every message, even for
series.

At this point I find <https://issues.guix.info/search?query=is%3Aopen>
clearer and less cluttered, though it needs some improvements (e.g.,
most-recent-first sorting, listing all the open issues, etc.), which I
think should be relatively easy to do.  Also, it is directly linked to
Debbugs and its notion of issues and associated properties (bug number,
owner, status, tags, etc.), which is good IMO.

WDYT?

Thanks,
Ludo’.

Re: Patchwork + automated checking and testing of patches

[Gábor Boskovits]

Hello,

Ludovic Courtès <ludo@gnu.org> ezt írta (időpont: 2018. nov. 6., K, 14:33):
>
> Hello!
>
> Christopher Baines <mail@cbaines.net> skribis:
>
> > Ludovic Courtès <ludo@gnu.org> writes:
>
> [...]
>
> >> Back when we tried, it had a couple of shortcomings:
> >>
> >>   1. It would not automatically detect which patches have been merged;
> >>
> >>   2. It would not present patch series correctly.
> >>
> >> From what you write it looks like #1 has been fixed, but the web
> >> interface suggests that #2 isn’t quite fixed yet, is that correct?
> >
> > On the detecting merged patches, that's definately working for some
> > patches though. I don't fully understand what criteria it's using
> > though, as it's comparing the commits that come through to the master
> > branch, and I bet it's possible to confuse it a bit by tweaking patches
> > before pushing them.
> >
> > Regarding patch series, I don't know much about the specifics of this,
> > and I don't know much about Patchwork, but just comparing a few patches
> > on the older version [1], and the newer version [2], it looks like it's
> > better. Take this patch [3], it's part of a series, but you can't
> > tell. However, with this patch [4], you can see the series and related
> > patches towards the top of the page, and also a link to download the
> > whole series as an mbox. How does this look to you?
>
> It looks better than the old Patchwork version, but I think
> <https://patchwork.cbaines.net/project/guix-patches/list/> still leaves
> a bit to be desired.  For instance, it shows every message, even for
> series.
>
> At this point I find <https://issues.guix.info/search?query=is%3Aopen>
> clearer and less cluttered, though it needs some improvements (e.g.,
> most-recent-first sorting, listing all the open issues, etc.), which I
> think should be relatively easy to do.  Also, it is directly linked to
> Debbugs and its notion of issues and associated properties (bug number,
> owner, status, tags, etc.), which is good IMO.

I have noticed that issues.guix.info goes to 504 quite often, Ricardo said it
was because we only have one backend. So it might be a good idea to get
some infrastructure up there. WDYT?

>
> WDYT?
>
> Thanks,
> Ludo’.
>

g_bor

Re: Patchwork + automated checking and testing of patches

[Ricardo Wurmus]

Gábor Boskovits <boskovits@gmail.com> writes:

> I have noticed that issues.guix.info goes to 504 quite often, Ricardo said it
> was because we only have one backend. So it might be a good idea to get
> some infrastructure up there. WDYT?

The software needs to be modified a little, e.g. to add fibers.  I’ll
try to implement this soon.

Currently, there’s just a single server process, and it blocks while it
computes a response to a request.

--
Ricardo

Re: Patchwork + automated checking and testing of patches

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 3101 bytes --]


Ludovic Courtès <ludo@gnu.org> writes:

> Hello!
>
> Christopher Baines <mail@cbaines.net> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>
> [...]
>
>>> Back when we tried, it had a couple of shortcomings:
>>>
>>>   1. It would not automatically detect which patches have been merged;
>>>
>>>   2. It would not present patch series correctly.
>>>
>>> From what you write it looks like #1 has been fixed, but the web
>>> interface suggests that #2 isn’t quite fixed yet, is that correct?
>>
>> On the detecting merged patches, that's definately working for some
>> patches though. I don't fully understand what criteria it's using
>> though, as it's comparing the commits that come through to the master
>> branch, and I bet it's possible to confuse it a bit by tweaking patches
>> before pushing them.
>>
>> Regarding patch series, I don't know much about the specifics of this,
>> and I don't know much about Patchwork, but just comparing a few patches
>> on the older version [1], and the newer version [2], it looks like it's
>> better. Take this patch [3], it's part of a series, but you can't
>> tell. However, with this patch [4], you can see the series and related
>> patches towards the top of the page, and also a link to download the
>> whole series as an mbox. How does this look to you?
>
> It looks better than the old Patchwork version, but I think
> <https://patchwork.cbaines.net/project/guix-patches/list/> still leaves
> a bit to be desired.  For instance, it shows every message, even for
> series.
>
> At this point I find <https://issues.guix.info/search?query=is%3Aopen>
> clearer and less cluttered, though it needs some improvements (e.g.,
> most-recent-first sorting, listing all the open issues, etc.), which I
> think should be relatively easy to do.  Also, it is directly linked to
> Debbugs and its notion of issues and associated properties (bug number,
> owner, status, tags, etc.), which is good IMO.
>
> WDYT?

So, I was trying to work out how to update a series of patches, and
asked on the Patchwork mailing list [1], and discovered there's actually
another codebase with different features [2]. This Patchwork-FDO fork
does have a page that just displays series [3].

1: https://lists.ozlabs.org/pipermail/patchwork/2018-November/005674.html
2: https://gitlab.freedesktop.org/patchwork-fdo/patchwork-fdo/
3: https://patchwork.freedesktop.org/project/intel-gfx/series/?ordering=-last_updated

In terms of patch tracking, I'm thinking on Patchwork and Debbugs
independently, while I think it would be possible to integrate the two,
Patchwork has the tracking capabilities of Debbugs, so if Patchwork ever
becomes useful for automated testing of patches, it would probably
remove the need to use Debbugs to track patches (not actual bugs and
issues).

Anyway, I think the next thing to try and do is get something
(Patchwork, Patchwork-FDO, Debbugs/issues.guix.info) triggering tests
for patches. I still need to compare the two Patchwork codebases to see
if they differ in this respect.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Patchwork + automated checking and testing of patches

[Ludovic Courtès]

Hi Christopher,

Christopher Baines <mail@cbaines.net> skribis:

> So, I was trying to work out how to update a series of patches, and
> asked on the Patchwork mailing list [1], and discovered there's actually
> another codebase with different features [2]. This Patchwork-FDO fork
> does have a page that just displays series [3].
>
> 1: https://lists.ozlabs.org/pipermail/patchwork/2018-November/005674.html
> 2: https://gitlab.freedesktop.org/patchwork-fdo/patchwork-fdo/
> 3: https://patchwork.freedesktop.org/project/intel-gfx/series/?ordering=-last_updated

Interesting.

> In terms of patch tracking, I'm thinking on Patchwork and Debbugs
> independently, while I think it would be possible to integrate the two,
> Patchwork has the tracking capabilities of Debbugs, so if Patchwork ever
> becomes useful for automated testing of patches, it would probably
> remove the need to use Debbugs to track patches (not actual bugs and
> issues).

Yes maybe, though Debbugs has features that make it attractive to me
(SOAP interface and Emacs interface, properties attached to “bugs”, etc.)

> Anyway, I think the next thing to try and do is get something
> (Patchwork, Patchwork-FDO, Debbugs/issues.guix.info) triggering tests
> for patches. I still need to compare the two Patchwork codebases to see
> if they differ in this respect.

Sounds like a plan!

Thanks,
Ludo’.

Re: Patchwork + automated checking and testing of patches

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 4373 bytes --]


Christopher Baines <mail@cbaines.net> writes:

> I've now written a very rough package and service for Patchwork [4], and
> managed to setup a instance here [5]. With the help of an email account
> subscribed to both guix-patches and guix-commits, getmail and a couple
> of scripts, it should also collect new patches sent to guix-patches, and
> mark those that have been merged to the master branch as "Accepted" [6].
>
> 4: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=33185
> 5: https://patchwork.cbaines.net/
> 6: https://patchwork.cbaines.net/project/guix-patches/list/?state=3

So, this has now moved forward a bit since this update. There is a
Summary at the bottom which you might want to skip ahead to.

I'm now testing out 3 services working together for testing patches:
 - https://patchwork.cbaines.net/
 - https://patchwork-fdo.cbaines.net/
 - https://laminar.cbaines.net/

Previously, I was just running Patchwork, and it was just picking up
patches from the mailing list, and marking committed patches as
accepted.

I mailed the Patchwork mailing list [1], as I had a question about
re-sending patches, and one thing to come out of that was the
realisation that I'd managed to stumble upon the documentation for a
fork of Patchwork.

1: https://lists.ozlabs.org/pipermail/patchwork/2018-November/005674.html

It turns out that there is a FreeDesktop fork [2][3][4]. It's
significant, as I think it does some things better than the upstream
version of Patchwork. One example is the Series page. For easy
comparison, I'm now running both, so [5] is upstream patchwork, and [6]
is the FreeDesktop fork.

2: https://patchwork.freedesktop.org/
3: https://gitlab.freedesktop.org/patchwork-fdo/patchwork-fdo/
4: https://patchwork-freedesktop.readthedocs.io/en/latest/

5: https://patchwork.cbaines.net/
6: https://patchwork-fdo.cbaines.net/

In addition to that, I wanted to continue working towards testing the
patches. Now Patchwork doesn't provide a framework for that, it just
provides a way to record the results of the testing. Maybe Cuirass can
do this in the future, but for now, I went with writing some bash
scripts, using some software called Laminar [1] as a useful
framework. It's tagline is "Continuous Integration the Light Way", and
it's something I've been wanted to try out for a little while.

7: https://laminar.ohwg.net/

I've configured 3 jobs [8] per patchwork service (fdo is short for
FreeDesktop):

 - patchwork(-fdo)-test-series:
   To test an individual series of patches.

 - patchwork(-fdo)-test-new:
   To trigger the testing of a series for all new (untested or updated)
   series.

 - patchwork(-fdo)-test-all:
   To trigger the testing of all open series, useful when the master
   branch of guix is updated.

8: https://laminar.cbaines.net/jobs

Laminar is configured by some files in /var/lib/laminar/cfg, so I've
added some NGinx configuration to make that public here [9]. Currently,
the patchwork(-fdo)-test-series jobs just:
 - Updates it's copy of the guix repository
 - Downloads the mbox from the relevant patchwork service
 - Tries applying it
 - Reports what happened to the relevant patchwork service

9: https://laminar.cbaines.net/cfg/

=== Summary ===

This is still very much a prototype, but I think it's nearing the point
where it actually might be doing something useful.

Currently, taking the service running the FreeDesktop fork of Patchwork,
you can go to the series page [10], and it will show you a list of patch
series, along with a indication of the "tests" status, which currently
is just whether the patch applied successfully to the master branch.

10: https://patchwork-fdo.cbaines.net/project/guix-patches/series/

The Patchwork service running the upstream code is very similar, you
just have to click in to the individual patches to get the information
on the "Checks", for an example, have a look at [11]

11: https://patchwork.cbaines.net/patch/233/

Next, I'm planning to work on extending the test series job to actually
start running some code, maybe perhaps just applying the patch, and then
running make at first. Then maybe trying to use the (guix inferior)
module code to intelligently determine changed packages that need
building. If you have something that does this, or want to write
something to do this, that would be awesome.

Any questions or comments, just let me know,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Patchwork + automated checking and testing of patches

[Ludovic Courtès]

Hi!

Christopher Baines <mail@cbaines.net> skribis:

> This is still very much a prototype, but I think it's nearing the point
> where it actually might be doing something useful.
>
> Currently, taking the service running the FreeDesktop fork of Patchwork,
> you can go to the series page [10], and it will show you a list of patch
> series, along with a indication of the "tests" status, which currently
> is just whether the patch applied successfully to the master branch.
>
> 10: https://patchwork-fdo.cbaines.net/project/guix-patches/series/

The test part is nice!

> The Patchwork service running the upstream code is very similar, you
> just have to click in to the individual patches to get the information
> on the "Checks", for an example, have a look at [11]
>
> 11: https://patchwork.cbaines.net/patch/233/

Neat.

> Next, I'm planning to work on extending the test series job to actually
> start running some code, maybe perhaps just applying the patch, and then
> running make at first. Then maybe trying to use the (guix inferior)
> module code to intelligently determine changed packages that need
> building. If you have something that does this, or want to write
> something to do this, that would be awesome.

For this I would suggest building the code using (guix self).  You can
do that either in Scheme using ‘inferior-for-channels’ or with something
like:

  guix pull --url=file://… -p ./test-guix

As for determining the list of new and upgraded packages, see
‘display-new/upgraded-packages’ in (guix scripts pull).  We can separate
the computational part from the UI part in there.

Perhaps we could integrate the tests with Mumi as well.

Thanks for the great work!

Ludo’.

Re: Patchwork + automated checking and testing of patches

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 691 bytes --]

Hi Chris,

This is really cool stuff!  Thanks for looking into it.

Christopher Baines <mail@cbaines.net> writes:

> I don't intend to do anything with Jenkins, as I think that wouldn't be
> maintainable, but I think setting up some system to check some of the
> following would be beneficial:

I'm actually a little hesitant to ask, since I don't have time to
contribute to this right now, and I don't want to bike-shed.  However,
I'm just curious: why do you think using Jenkins wouldn't be
maintainable?  I've read about Jenkins but haven't used it personally,
and I've heard good things about it, so I'm curious to know why you
prefer to avoid it entirely.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Patchwork + automated checking and testing of patches

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1939 bytes --]


Chris Marusich <cmmarusich@gmail.com> writes:

> Hi Chris,
>
> This is really cool stuff!  Thanks for looking into it.
>
> Christopher Baines <mail@cbaines.net> writes:
>
>> I don't intend to do anything with Jenkins, as I think that wouldn't be
>> maintainable, but I think setting up some system to check some of the
>> following would be beneficial:
>
> I'm actually a little hesitant to ask, since I don't have time to
> contribute to this right now, and I don't want to bike-shed.  However,
> I'm just curious: why do you think using Jenkins wouldn't be
> maintainable?  I've read about Jenkins but haven't used it personally,
> and I've heard good things about it, so I'm curious to know why you
> prefer to avoid it entirely.

Sure, I've used Jenkins in different contexts, and indeed currently use
Jenkins, so I've got some experience here.

I'm obviously ignoring all the good parts here, but one source of data
is Debian. It used to have a package for Jenkins, and you can see some
of the work/issues here [1] and the thread about it's removal here [2].

One issue in particular to call out is some potential bootstrapping
issues [3] that seem to have been encountered.

It seems sensible to me to use Guix when doing things related to the
Guix project, like automated testing of patches (dogfooding, if you know
the term). Some of the things I've mentioned here lead me to doubt that
Jenkins will at some point be available through Guix.

So that's something about the maintainability of a Guix package, but
there's an operational component to this as well. The attack surface it
offers is maybe larger than sometimes necessary, if you only need a
read-only web interface for example.

1:
https://bugs.debian.org/cgi-bin/pkgreport.cgi?archive=both;dist=unstable;ordering=normal;repeatmerged=0;src=jenkins
2: https://lists.debian.org/debian-java/2016/01/msg00019.html
3: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=714596

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Patchwork + automated checking and testing of patches

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 1869 bytes --]

Hi Chris,

Christopher Baines <mail@cbaines.net> writes:

> Chris Marusich <cmmarusich@gmail.com> writes:
>
>> [...] Why do you think using Jenkins wouldn't be maintainable? [...]
>
> Sure, I've used Jenkins in different contexts, and indeed currently use
> Jenkins, so I've got some experience here.
>
> I'm obviously ignoring all the good parts here, but one source of data
> is Debian. It used to have a package for Jenkins, and you can see some
> of the work/issues here [1] and the thread about it's removal here [2].
>
> One issue in particular to call out is some potential bootstrapping
> issues [3] that seem to have been encountered.
>
> It seems sensible to me to use Guix when doing things related to the
> Guix project, like automated testing of patches (dogfooding, if you know
> the term). Some of the things I've mentioned here lead me to doubt that
> Jenkins will at some point be available through Guix.
>
> So that's something about the maintainability of a Guix package, but
> there's an operational component to this as well. The attack surface it
> offers is maybe larger than sometimes necessary, if you only need a
> read-only web interface for example.
>
> 1:
> https://bugs.debian.org/cgi-bin/pkgreport.cgi?archive=both;dist=unstable;ordering=normal;repeatmerged=0;src=jenkins
> 2: https://lists.debian.org/debian-java/2016/01/msg00019.html
> 3: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=714596

Wow!  I had no idea Debian had encountered such difficulties with
Jenkins.  And your point about attack surface is also valid - if we can
get by with a read-only web interface, it's much less to worry about.
After reading the links you've provided and hearing your argument, I'm
inclined to agree that avoiding Jenkins is the right call at this time.

Thank you for explaining things!

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Patchwork + automated checking and testing of patches

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1024 bytes --]

So, today at the Guix Days event in Brussels I've made a small change to
the Laminar configuration related to patchwork.cbaines.net.

Now, when processing a series of patches, it should create a Git branch
and push it to a repository.

For example, for this patch series [1], you can see it being processed
here [2]. The result of applying the patches is pushed up to this Git
repository [3], and you can see the commit range here [4].

1: https://patchwork.cbaines.net/project/guix-patches/list/?series=482
2: https://laminar.cbaines.net/jobs/patchwork-test-series/826
3: https://git.cbaines.net/guix/patches/log/?h=series-482-version-1
4: https://git.cbaines.net/guix/patches/log/?h=series-482-version-1&qt=range&q=base-for-series-482-version-1..series-482-version-1

I'm hoping that having the patches in a Git repository can enable using
things like `guix pull` to build a Guix with the patches to then extract
information about what's changed, as well as potentially being useful
for getting Cuirass to build things...

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Patchwork + automated checking and testing of patches

[Ludovic Courtès]

Heya!

Christopher Baines <mail@cbaines.net> skribis:

> So, today at the Guix Days event in Brussels I've made a small change to
> the Laminar configuration related to patchwork.cbaines.net.

Damn it, I didn’t know you had done that while we were there, kudos!

> Now, when processing a series of patches, it should create a Git branch
> and push it to a repository.
>
> For example, for this patch series [1], you can see it being processed
> here [2]. The result of applying the patches is pushed up to this Git
> repository [3], and you can see the commit range here [4].
>
> 1: https://patchwork.cbaines.net/project/guix-patches/list/?series=482
> 2: https://laminar.cbaines.net/jobs/patchwork-test-series/826
> 3: https://git.cbaines.net/guix/patches/log/?h=series-482-version-1
> 4: https://git.cbaines.net/guix/patches/log/?h=series-482-version-1&qt=range&q=base-for-series-482-version-1..series-482-version-1

Neat!

> I'm hoping that having the patches in a Git repository can enable using
> things like `guix pull` to build a Guix with the patches to then extract
> information about what's changed, as well as potentially being useful
> for getting Cuirass to build things...

Yup, this is really promising, looks like you’re getting there!  As
discussed at the Greek restaurant, it’d be nice to see how much of the
job Laminar runs can be turned into (guix …) modules, though that can
come later.

Anyway, as this is shaping up, let us know if you need input on the
remaining bits, like using the channel API to extract information about
the new branch, or fiddling with Cuirass.

Thanks,
Ludo’.

Re: Patchwork + automated checking and testing of patches

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 2171 bytes --]


Ludovic Courtès <ludo@gnu.org> writes:

> Heya!
>
> Christopher Baines <mail@cbaines.net> skribis:
>
>> So, today at the Guix Days event in Brussels I've made a small change to
>> the Laminar configuration related to patchwork.cbaines.net.
>
> Damn it, I didn’t know you had done that while we were there, kudos!
>
>> Now, when processing a series of patches, it should create a Git branch
>> and push it to a repository.
>>
>> For example, for this patch series [1], you can see it being processed
>> here [2]. The result of applying the patches is pushed up to this Git
>> repository [3], and you can see the commit range here [4].
>>
>> 1: https://patchwork.cbaines.net/project/guix-patches/list/?series=482
>> 2: https://laminar.cbaines.net/jobs/patchwork-test-series/826
>> 3: https://git.cbaines.net/guix/patches/log/?h=series-482-version-1
>> 4: https://git.cbaines.net/guix/patches/log/?h=series-482-version-1&qt=range&q=base-for-series-482-version-1..series-482-version-1
>
> Neat!

Thanks, however, one thing I've noticed is that git apply seems to be
much more reliable at applying patches than git am. It often fails with
"Patch is empty".

I'm not sure why, it seems to work for the patches I've sent with git
send-email, but maybe it's very particular about the formatting...

>> I'm hoping that having the patches in a Git repository can enable using
>> things like `guix pull` to build a Guix with the patches to then extract
>> information about what's changed, as well as potentially being useful
>> for getting Cuirass to build things...
>
> Yup, this is really promising, looks like you’re getting there!  As
> discussed at the Greek restaurant, it’d be nice to see how much of the
> job Laminar runs can be turned into (guix …) modules, though that can
> come later.
>
> Anyway, as this is shaping up, let us know if you need input on the
> remaining bits, like using the channel API to extract information about
> the new branch, or fiddling with Cuirass.

I've made a bit more progress with this over the last couple of days,
I'll start another thread for that though.

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: Patchwork + automated checking and testing of patches

[Björn Höfling]

[-- Attachment #1: Type: text/plain, Size: 793 bytes --]

On Fri, 08 Feb 2019 12:04:20 +0000
Christopher Baines <mail@cbaines.net> wrote:

> Thanks, however, one thing I've noticed is that git apply seems to be
> much more reliable at applying patches than git am. It often fails
> with "Patch is empty".
> 
> I'm not sure why, it seems to work for the patches I've sent with git
> send-email, but maybe it's very particular about the formatting...

Maybe you already thought about it, but:

When I use it manually, I save the whole patch-series as an mbox-file,
including the "--cover-letter", i.e. the first email saying [0/n]. When
I then say "git am patch.mbox" it complains there is nothing in. I then
have to say "git am --skip" and it goes on automatically.

Could that also be the case with your patchwork-setup?

Björn


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]