Frequently testing fixed output derivations

Frequently testing fixed output derivations

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1647 bytes --]

Hey,

Something I noticed when looking at things guix.cbaines.net failed to
build, is that some failures were because a fixed output derivation had
failed (because of a 404 or hash mismatch for example).

I've got around to adding a page to the Guix Data Service to list all
the fixed output derivations that are related to packages, which also
shows the latest build for that particular derivation. It's very slow at
the moment, I need to work more on the PostgreSQL query it uses, but I
have done most of the builds for all the ~15000 (x86_64-linux)
derivations now. If this link loads for you, it should show ~28
derivations that failed to build [1].

1: https://data.guix-patches.cbaines.net/revision/78fd422cd0e1c894280ef48de8b3a5d42db2eb80/fixed-output-package-derivations?system=x86_64-linux&target=none&latest_build_status=failed&all_results=on

There's some nuance here, not all these derivations could cause
problems, as a user might have a substitute available for them, but they
might cause problems for those building without substitutes (although
substitutes should always be fine to use for fixed output
derivations). Because the fixed output derivations themselves attempt to
fallback to mirrors or fetching via the hash, this isn't yet a good test
on whether a file download necessarily works for example. Maybe there
could be a way of disabling that functionality for testing.

My plan is to improve the performace issues with this page, and schedule
builds each week for every derivation (spread out over the week). That
way there should be up to date information, making it possible to fix
things soon after they break.

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

Re: Frequently testing fixed output derivations

[Ludovic Courtès]

Hi,

Christopher Baines <mail@cbaines.net> skribis:

> Something I noticed when looking at things guix.cbaines.net failed to
> build, is that some failures were because a fixed output derivation had
> failed (because of a 404 or hash mismatch for example).
>
> I've got around to adding a page to the Guix Data Service to list all
> the fixed output derivations that are related to packages, which also
> shows the latest build for that particular derivation. It's very slow at
> the moment, I need to work more on the PostgreSQL query it uses, but I
> have done most of the builds for all the ~15000 (x86_64-linux)
> derivations now. If this link loads for you, it should show ~28
> derivations that failed to build [1].
>
> 1: https://data.guix-patches.cbaines.net/revision/78fd422cd0e1c894280ef48de8b3a5d42db2eb80/fixed-output-package-derivations?system=x86_64-linux&target=none&latest_build_status=failed&all_results=on

Nice.

One could also run ‘guix lint -c source’, but it takes ages and misses
hash mismatches (it just checks whether there’s “something” at the given
URL).

> My plan is to improve the performace issues with this page, and schedule
> builds each week for every derivation (spread out over the week). That
> way there should be up to date information, making it possible to fix
> things soon after they break.

That’s a great idea.  Would it be also an option to have plots of the
fraction of origin derivations that fail over time?  We’d start an
emergency rescue plan on IRC every time we cross a certain threshold.

Thanks,
Ludo’.