Update on automating testing of patches and qa.guix.gnu.org

Update on automating testing of patches and qa.guix.gnu.org

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1957 bytes --]

Hey!

I last emailed guix-devel on this topic back in September:

  https://lists.gnu.org/archive/html/guix-devel/2022-09/msg00054.html
  https://lists.gnu.org/archive/html/guix-devel/2022-09/msg00159.html

Along with some general improvements and fixes, the following things
might be interesting:

 - There are now some bigger badges for the overall status of patch
   issues, thanks Arun!

 - The sort order for the patches page should now be more useful

 - The creation of the patch branches is now handled here (rather than
   through scripts run by Laminar)

 - There is a mailto link on the issue page to toggle the moreinfo
   status, thanks Efraim for the idea!

 - The issue page has a "review checklist". I based this off of
   something Arun suggested for Mumi. The form doesn't do anything, I'm
   just using it as a reminder of things to check when reviewing
   patches.

In terms of next steps:

 - I'm still looking to move this repository on to Savannah, my initial
   attempt failed, so I'm not sure how to do this.

 - Currently it's hard to distinguish "Unknown" builds in to builds
   where some dependency has failed, and builds which just haven't
   happened yet. I'm going to try and change the Guix Data Service to
   provide information on blocking builds in the comparison.

 - For more things, see the README
   https://git.cbaines.net/guix/qa-frontpage/about/


I'm finding this really useful to highlight simple patches which have
had some testing done automatically. I'd encourage others who are
interested in merging patches to try looking at the patches showing up
as green, as hopefully they're in a good state. If there's any which
aren't (e.g. needs some changes or more discussion), you can mark it as
moreinfo to push it down the list (there's a link on the right to do
this).

Do let me know if you have any comments or questions! This should be
easy to hack on as well, and I'm happy to try and help.

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: Update on automating testing of patches and qa.guix.gnu.org

[zimoun]

Hi Chris,

Thanks!  Really cool.

On sam., 05 nov. 2022 at 12:24, Christopher Baines <mail@cbaines.net> wrote:

>  - The creation of the patch branches is now handled here (rather than
>    through scripts run by Laminar)

Is it possible to have access to the branch?  As discussed with Josselin
at 10 Years Days, we could have in mind the workflow:

    guix time-machine --url=https://qa.guix.gnu.org \
                      --branch=58812                \
         -- shell --symlink

for quickly testing a specific patch without going to manually clone,
apply the patch, then run ./bootstrap, ./configure and last use
./pre-inst-env.


> I'm finding this really useful to highlight simple patches which have
> had some testing done automatically. I'd encourage others who are
> interested in merging patches to try looking at the patches showing up
> as green, as hopefully they're in a good state.

Do you expose the list of these green patches?  It could be cool to have
a command line – an option of “guix weather” or a specific CLI – to
collect information of the status.  WDYT?

>                                                 If there's any which
> aren't (e.g. needs some changes or more discussion), you can mark it as
> moreinfo to push it down the list (there's a link on the right to do
> this).

Mark it as ’moreinfo’ via Debbugs?  Or something else?


Cheers,
simon

Re: Update on automating testing of patches and qa.guix.gnu.org

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 2538 bytes --]


zimoun <zimon.toutoune@gmail.com> writes:

> Hi Chris,
>
> Thanks!  Really cool.
>
> On sam., 05 nov. 2022 at 12:24, Christopher Baines <mail@cbaines.net> wrote:
>
>>  - The creation of the patch branches is now handled here (rather than
>>    through scripts run by Laminar)
>
> Is it possible to have access to the branch?  As discussed with Josselin
> at 10 Years Days, we could have in mind the workflow:
>
>     guix time-machine --url=https://qa.guix.gnu.org \
>                       --branch=58812                \
>          -- shell --symlink
>
> for quickly testing a specific patch without going to manually clone,
> apply the patch, then run ./bootstrap, ./configure and last use
> ./pre-inst-env.

For the Git branches, there's Gitolite+cgit+some other stuff setup here
https://git.guix-patches.cbaines.net/git/guix-patches

So yes, this command should work (obviously, disabling authentication
isn't good):

  guix time-machine \
    --url=https://git.guix-patches.cbaines.net/git/guix-patches \
    --branch=issue-58812 --disable-authentication -- shell --symlink

Eventually I'd like to move this off of a machine I'm paying for, plus
move it on to a .guix domain. Also, even though the channel instance for
some branches might have been built by the data.qa.guix.gnu.org Guix
Data Service, this isn't done in a way that substitutes are available,
so that's not ideal.

>> I'm finding this really useful to highlight simple patches which have
>> had some testing done automatically. I'd encourage others who are
>> interested in merging patches to try looking at the patches showing up
>> as green, as hopefully they're in a good state.
>
> Do you expose the list of these green patches?  It could be cool to have
> a command line – an option of “guix weather” or a specific CLI – to
> collect information of the status.  WDYT?

I'm using this list here https://qa.guix.gnu.org/patches and just
looking at the ones at the top with the green circle by them.

An API endpoint could easily be added though if that's useful.

>>                                                 If there's any which
>> aren't (e.g. needs some changes or more discussion), you can mark it as
>> moreinfo to push it down the list (there's a link on the right to do
>> this).
>
> Mark it as ’moreinfo’ via Debbugs?  Or something else?

Yep. I'm currently trying to keep the qa-frontpage relatively stateless,
so it's just reading the tags in debbugs.

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: Update on automating testing of patches and qa.guix.gnu.org

[zimoun]

Hi Chris,

On Mon, 07 Nov 2022 at 10:36, Christopher Baines <mail@cbaines.net> wrote:

> For the Git branches, there's Gitolite+cgit+some other stuff setup here
> https://git.guix-patches.cbaines.net/git/guix-patches
>
> So yes, this command should work (obviously, disabling authentication
> isn't good):
>
>   guix time-machine \
>     --url=https://git.guix-patches.cbaines.net/git/guix-patches \
>     --branch=issue-58812 --disable-authentication -- shell --symlink

That’s awesome!  It just works.  It could be a booster for reviewing
patches because:

 1. substitutes are sometimes available; which is really helpful for
 people using low-power machines,

 2. it simplifies the workflow for testing.

However, something about “guix time-machine” is not cached as expected.
Running twice in a row the same command and then

    Computing Guix derivation for 'x86_64-linux'...

is computed each time; when it should not, IIUC.   Another story. :-)


> Eventually I'd like to move this off of a machine I'm paying for, plus
> move it on to a .guix domain. Also, even though the channel instance for
> some branches might have been built by the data.qa.guix.gnu.org Guix
> Data Service, this isn't done in a way that substitutes are available,
> so that's not ideal.

What do you mean by «this isn't done in a way that substitutes are
available»?


> I'm using this list here https://qa.guix.gnu.org/patches and just
> looking at the ones at the top with the green circle by them.
>
> An API endpoint could easily be added though if that's useful.

Personally, I find easier to query and get back a list that I can
process instead of having open my webbrowser. :-)


>>>                                                 If there's any which
>>> aren't (e.g. needs some changes or more discussion), you can mark it as
>>> moreinfo to push it down the list (there's a link on the right to do
>>> this).
>>
>> Mark it as ’moreinfo’ via Debbugs?  Or something else?
>
> Yep. I'm currently trying to keep the qa-frontpage relatively stateless,
> so it's just reading the tags in debbugs.

Ok, cool!

If I remember correctly, the Data Service provides some lint
information.  We could also imagine a commit-message linter.  However,
the number of tags is rather limited.  Some time ago, we discussed
’usertag’ or else.  Well, what could be done in this area of “automatic”
tagging?


Cheers,
simon

Re: Update on automating testing of patches and qa.guix.gnu.org

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

>>   guix time-machine \
>>     --url=https://git.guix-patches.cbaines.net/git/guix-patches \
>>     --branch=issue-58812 --disable-authentication -- shell --symlink

[...]

> However, something about “guix time-machine” is not cached as expected.
> Running twice in a row the same command and then
>
>     Computing Guix derivation for 'x86_64-linux'...
>
> is computed each time; when it should not, IIUC.   Another story. :-)

It’s expected: caching it disabled when passing
‘--disable-authentication’ (commit
9f371f23ebfa20f70b3bfd55dc459b683f21ba91).

But yeah, that’s annoying.

Ludo’.