bug#53533: [DISCUSSION] Quality of services in reproducible build environment Guix

[Leo Famulari <leo@famulari.name>]

On Tue, Jan 25, 2022 at 08:45:22PM +0000, Sharlatan Hellseher wrote:
> The current QA for the accepted changes in Gujx is far away from
> trustible. For example, some
> changes in package update may cause a faileur of the whole chain of
> packages depending on it. It
> would be nice to have some soft policy of changes, check list or some
> procedure to have a "stable"
> branch which may guarantee all packages build successfully and pass of
> all enabled tests.

People are definitely supposed to check that their changes don't break
things before they push them, but it doesn't always happen.

It's not easy to make sure that all packages build successfully. I've
never seen 100% of packages build successfully since I joined Guix in
2015. It's a nice goal but it requires some work... a lot more work.
Everyone is invited to help. Probably, the first step is to remove
several hundred packages that don't build; it might even be a couple
thousand.

> I would like to conclude from the CI is which commit broke how many
> packages.

Agreed, this is a very important missing feature. Also, we need the
capability to compare commits in terms of CI results.

Our CI software is called Cuirass:

https://git.savannah.gnu.org/cgit/guix/guix-cuirass.git

We need more people to help develop Cuirass!

> Some missing practice of packaging:
> - Some essential message of the reason why tests were disabled and any
> sort of suggestions on how to
>   make them enabled. Contact upstream if required.

Everyone is supposed to do this when writing packages. It's a failure of
the code review process if that is not happening.

> - Before sending patch make sure (at least for the localhost
> architecture) it's built, linted and in
>   case of bumping version - all dependent chain still can be built.

This is supposed to be done for patches that go to the master branch.
That is, patches that affect <300 packages. Other patches that affect
more than 300 packages are batched on development branches such as
'core-updates' [0], and then we need a lot of Guix contributors to help
get all the packages building again. Maybe we should remove broken
packages more casually, like I suggested above.

>   Excess changes which could be prevented with just local attempt to build
>   #+begin_src sh
> git log --grep="Fix build" --pretty="%h %s %cd - %cn" | wc -l
>   #+end_src
>   : 1025

We have 92225 commits total:

------
$ git log --oneline | wc -l
92225
------

So, about 1.1% of them are "Fix build" commits.

However, not all of these commits were made on the master branch. Many
of them are on development branches such as 'core-updates' and
'staging', and so users never experienced the problems that were fixed.

That doesn't mean that Guix QA is perfect, but rather that your
measurement is inaccurate and misleading.

[0] To learn more about development branches like core-updates, see
item 8:
https://guix.gnu.org/manual/en/html_node/Submitting-Patches.html