Re: August/November update on qa.guix.gnu.org and related things

Re: August/November update on qa.guix.gnu.org and related things

[Suhail]

Christopher Baines <mail@cbaines.net> writes:

>  - The README is published at https://qa.guix.gnu.org/README

The README seems more focused on task planning and TODO items than
explaining how to use the qa.guix.gnu.org website. Could you please
provide a reference for the latter?

Specifically, I submitted a patch some while ago:
<https://qa.guix.gnu.org/issue/66644>. Its QA status is marked as
unknown with a few items highlighted in red. While the UI helps draw
attention to those items, it's not clear (to me) how to remedy them and
who is responsible for doing what. I.e., what are the next steps? I
would like to get that patch reviewed and merged in some way, but I
don't know what, if anything, I can do to help with the matter.

> I'd also really like to see some testing of the patch review feature
> in QA, since I think trying to get people without commit access
> reviewing patches will really help speed up getting things reviewed
> and merged.

Is there a document that outlines how to get started and/or
pre-requisites that one must have before reviewing certain aspects?

-- 
Suhail

This email is not an offer capable of acceptance, does not evidence an
intention to enter into an agreement, has no operative effect until a
definitive agreement is signed in writing by both parties, and that no
party should act in reliance on the email or any representations of the
sender until a definitive agreement is signed in writing by both
parties.

This email may contain information that is privileged, confidential
and/or exempt from disclosure.  No waiver whatsoever is intended by
sending this e-mail which is intended only for the named recipient(s).
Unauthorized use, dissemination or copying is prohibited.  If you
receive this email in error, please notify the sender and destroy all
copies of this email.

Re: August/November update on qa.guix.gnu.org and related things

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 2251 bytes --]


Suhail <suhail@bayesians.ca> writes:

> Christopher Baines <mail@cbaines.net> writes:
>
>>  - The README is published at https://qa.guix.gnu.org/README
>
> The README seems more focused on task planning and TODO items than
> explaining how to use the qa.guix.gnu.org website. Could you please
> provide a reference for the latter?

There isn't much documentation for QA, mostly because I want to see it
improve so that you don't need documentation to use it.

> Specifically, I submitted a patch some while ago:
> <https://qa.guix.gnu.org/issue/66644>. Its QA status is marked as
> unknown with a few items highlighted in red. While the UI helps draw
> attention to those items, it's not clear (to me) how to remedy them and
> who is responsible for doing what. I.e., what are the next steps? I
> would like to get that patch reviewed and merged in some way, but I
> don't know what, if anything, I can do to help with the matter.

I had a look at the QA page for #66644 and yeah, the red highlighted
bits where builds which hadn't happened yet.

I think it's fair to say that these shouldn't be styled the same as
failed builds, so I've changed the styling now. I've also added a new
issue status for when QA is waiting on builds to happen to provide more
information.

So yeah, QA isn't currently pointing out anything for you to do on this
issue.

>> I'd also really like to see some testing of the patch review feature
>> in QA, since I think trying to get people without commit access
>> reviewing patches will really help speed up getting things reviewed
>> and merged.
>
> Is there a document that outlines how to get started and/or
> pre-requisites that one must have before reviewing certain aspects?

On the page for each issue on qa.guix.gnu.org, there's a list of common
things to check. That form gives a way to record a review when you think
the patches look good to merge.

There's also some content in the manual that might be useful when
reviewing patches:

  https://guix.gnu.org/en/manual/devel/en/html_node/Packaging-Guidelines.html
  https://guix.gnu.org/en/manual/devel/en/html_node/Submitting-Patches.html

But there's no pre-requisites to reviewing Guix patches, so the best way
to learn is to start looking to review things.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: August/November update on qa.guix.gnu.org and related things

[Suhail]

"Christopher Baines" <mail@cbaines.net> writes:

> There isn't much documentation for QA

Understood.

Is the preferred place to ask questions regd the QA service this mailing
list?

> I think it's fair to say that these shouldn't be styled the same as
> failed builds, so I've changed the styling now.

The neutral blue works better; thank you. On a related note, the
specific build status on data.qa.guix.gnu.org for the "now blue" entries
is "Scheduled". Why does that get presented as "Unknown" in QA? IMO,
either "Scheduled" or "Pending" (in case it's important to maintain a
distinction from the build status of individual jobs as on
data.qa.guix.gnu.org) would be clearer than "Unknown".

> I've also added a new issue status for when QA is waiting on builds to
> happen to provide more information.

This being "Investigate"? Out of curiosity, and in a similar vein as
above, why not simply "Scheduled" or "Pending"? Or is it that it has had
"Scheduled" build jobs for far too long and thus requires someone else
with more privileged access (than myself) to investigate the cause?

I.e., Investigate is a verb and thus makes me wonder what the object is
(what needs to be investigated) and who the subject is (by whom)?
Shouldn't the QA issue status be an adjective instead?

> So yeah, QA isn't currently pointing out anything for you to do on
> this issue.

Okay, thank you for the clarification.

> There's also some content in the manual that might be useful when
> reviewing patches:
>
>   https://guix.gnu.org/en/manual/devel/en/html_node/Packaging-Guidelines.html
>   https://guix.gnu.org/en/manual/devel/en/html_node/Submitting-Patches.html

Perhaps linking to these from the "Mark patches as reviewed" section on
QA would be helpful?

> But there's no pre-requisites to reviewing Guix patches, so the best
> way to learn is to start looking to review things.

I imagine some of the "common things to check" will get automated in the
near future (e.g. whether or not the changes are adding to the lint
warnings), whereas some others will stay manual (e.g. are things "well
written"). Personally, for such subjective measures (i.e., the latter) I
find having some examples of "what good looks like" readily available
quite helpful. In case the intent is to make it easier for newcomers to
the project (i.e., those who've not yet internalized this knowledge) to
contribute, providing such prototypical examples by linking to commits,
descriptions etc in the existing source tree would help.

-- 
Suhail

This email is not an offer capable of acceptance, does not evidence an
intention to enter into an agreement, has no operative effect until a
definitive agreement is signed in writing by both parties, and that no
party should act in reliance on the email or any representations of the
sender until a definitive agreement is signed in writing by both
parties.

This email may contain information that is privileged, confidential
and/or exempt from disclosure.  No waiver whatsoever is intended by
sending this e-mail which is intended only for the named recipient(s).
Unauthorized use, dissemination or copying is prohibited.  If you
receive this email in error, please notify the sender and destroy all
copies of this email.

Re: August/November update on qa.guix.gnu.org and related things

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 2994 bytes --]


Suhail <suhail@bayesians.ca> writes:

> "Christopher Baines" <mail@cbaines.net> writes:
>
>> There isn't much documentation for QA
>
> Understood.
>
> Is the preferred place to ask questions regd the QA service this mailing
> list?

Yep.

>> I think it's fair to say that these shouldn't be styled the same as
>> failed builds, so I've changed the styling now.
>
> The neutral blue works better; thank you. On a related note, the
> specific build status on data.qa.guix.gnu.org for the "now blue" entries
> is "Scheduled". Why does that get presented as "Unknown" in QA? IMO,
> either "Scheduled" or "Pending" (in case it's important to maintain a
> distinction from the build status of individual jobs as on
> data.qa.guix.gnu.org) would be clearer than "Unknown".

I guess the result of the builds is unknown when they're scheduled.

>> I've also added a new issue status for when QA is waiting on builds to
>> happen to provide more information.
>
> This being "Investigate"? Out of curiosity, and in a similar vein as
> above, why not simply "Scheduled" or "Pending"? Or is it that it has had
> "Scheduled" build jobs for far too long and thus requires someone else
> with more privileged access (than myself) to investigate the cause?
>
> I.e., Investigate is a verb and thus makes me wonder what the object is
> (what needs to be investigated) and who the subject is (by whom)?
> Shouldn't the QA issue status be an adjective instead?

Yeah, this is just me copying and pasting the code for the badge. I've
changed the text to pending now. That's not as specific as "QA is
waiting on the results of builds" but at least it hints at that.

>> There's also some content in the manual that might be useful when
>> reviewing patches:
>>
>>   https://guix.gnu.org/en/manual/devel/en/html_node/Packaging-Guidelines.html
>>   https://guix.gnu.org/en/manual/devel/en/html_node/Submitting-Patches.html
>
> Perhaps linking to these from the "Mark patches as reviewed" section on
> QA would be helpful?

Yes, although I think the documentation maybe needs looking at a bit to
make it a bit more useful for patch review.

>> But there's no pre-requisites to reviewing Guix patches, so the best
>> way to learn is to start looking to review things.
>
> I imagine some of the "common things to check" will get automated in the
> near future (e.g. whether or not the changes are adding to the lint
> warnings), whereas some others will stay manual (e.g. are things "well
> written"). Personally, for such subjective measures (i.e., the latter) I
> find having some examples of "what good looks like" readily available
> quite helpful. In case the intent is to make it easier for newcomers to
> the project (i.e., those who've not yet internalized this knowledge) to
> contribute, providing such prototypical examples by linking to commits,
> descriptions etc in the existing source tree would help.

Yes, and while I think working on the docs is important, maybe QA can
link and show various examples.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: August/November update on qa.guix.gnu.org and related things

[Tomas Volf]

[-- Attachment #1: Type: text/plain, Size: 1496 bytes --]

On 2023-11-02 16:38:29 +0000, Suhail wrote:
> [..]
> -- 
> Suhail
> 
> This email is not an offer capable of acceptance, does not evidence an
> intention to enter into an agreement, has no operative effect until a
> definitive agreement is signed in writing by both parties, and that no
> party should act in reliance on the email or any representations of the
> sender until a definitive agreement is signed in writing by both
> parties.

I am unsure what is the takeaway from this paragraph.  If you, for example,
offer do implement something, should people wait for written agreement before
being able to assume you will do so?  So anything you say here should not be
relied upon?

> 
> This email may contain information that is privileged, confidential
> and/or exempt from disclosure.  No waiver whatsoever is intended by
> sending this e-mail which is intended only for the named recipient(s).
> Unauthorized use, dissemination or copying is prohibited.  If you
> receive this email in error, please notify the sender and destroy all
> copies of this email.
> 
>

I would suggest not sending a privileged, confidential and/or exempt from
disclosure information to a public mailing list.  Also, I am not sure you cannot
prohibit me from distributing the information once I receive it, since I did not
sign any NDA or similar document.

T.

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: August/November update on qa.guix.gnu.org and related things

[Suhail]

"Tomas Volf" <~@wolfsden.cz> writes:

> On 2023-11-02 16:38:29 +0000, Suhail wrote:
>> [..]
>> -- 
>> Suhail
>> 
>> This email is not an offer capable of acceptance, does not evidence an
>> intention to enter into an agreement, has no operative effect until a
>> definitive agreement is signed in writing by both parties, and that no
>> party should act in reliance on the email or any representations of the
>> sender until a definitive agreement is signed in writing by both
>> parties.
>
> I am unsure what is the takeaway from this paragraph.

I am not a lawyer, so take my interpretation with sufficient grains of
salt. To me, the takeaway is that a message with that signature does not
constitute a legally binding contract and thus limits liability.

> I would suggest not sending a privileged, confidential and/or exempt
> from disclosure information to a public mailing list.

You're correct. Thank you for catching that. I've gotten rid of the
offending clause.

-- 
Suhail

This email is not an offer capable of acceptance, does not evidence an
intention to enter into an agreement, has no operative effect until a
definitive agreement is signed in writing by both parties, and that no
party should act in reliance on the email or any representations of the
sender until a definitive agreement is signed in writing by both
parties.

August/November update on qa.guix.gnu.org and related things

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 2647 bytes --]

Hey,

The last update I sent out was back in July [1].

1: https://lists.gnu.org/archive/html/guix-devel/2023-07/msg00144.html

There have been quite a few changes over the last few months, here's a
summary of the changes:

 - QA now stores when it's submitted builds for a issue/branch and
   cancels them when they're no longer needed

 - The README is published at https://qa.guix.gnu.org/README

 - The output when applying patches is stored and displayed if there was
   a failure

 - There's a review feature for marking patches as looking good

 - There are more issue statuses and support filtering by status

 - Merged issues are now handled

 - Patches can be applied to non-master branch

 - The latest patch series are processed, rather than the latest issues

 - There's a page specifically about reproducible builds
   (qa.guix.gnu.org/reproducible-builds)


I've also been doing little bit of work towards speeding up the data
service processing revisions. I've disabled computing the system test
derivations on data.qa.guix.gnu.org as that takes a significant amount
of time, and they aren't being used yet.

The formatting linter takes quite a bit of time, and I've got an open
patch to speed it up:

  https://issues.guix.gnu.org/66796

The performance of computing cross derivations also seems like it could
be improved, I've sent some details here:

  https://lists.gnu.org/archive/html/guix-devel/2023-10/msg00257.html

# Next steps

I'm still planning to stop hosting both data.guix.gnu.org and
data.qa.guix.gnu.org at the end of the year, which is only a few weeks
away now. The qa-frontpage which runs qa.guix.gnu.org is dependent on
data.qa.guix.gnu.org. There's been some discussion of potentially moving
some or all of this to hosting organised through the project/Guix
Europe, but I'm not sure anything has happened on this yet.

I've added a few TODO's to the list in the README off the back of
discussion at the Reproducible Builds summit.

In my mind, while there's still lots to do in the qa-frontpage,
addressing problems in the data service is probably still the most
important thing to do. There's still some reliability issues to
investigate further as well as improving the performance of processing
revisions. If the data service can be sped up, QA will be able to give
feedback faster, and it'll scale to handle more patches.

I'd also really like to see some testing of the patch review feature in
QA, since I think trying to get people without commit access reviewing
patches will really help speed up getting things reviewed and merged.

Let me know if you have any comments or questions!

Thanks,

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]