bug#33737: do not attempt to build a package known to be broken

bug#33737: do not attempt to build a package known to be broken

[Ricardo Wurmus]

The default use case is for users to have substitutes enabled and to
fetch binaries from a build farm.  The build farm tries to build all
packages, but not all builds succeed.  The daemon caches build failures.

This information, however, is not available for users.  The default
behaviour of Guix is to attempt building the package locally.  This is
fine for packages that the build farm hasn’t attempted yet, but it is a
waste of time when the build has already been attempted without success.

I propose that “guix publish” should also convey information about
failed builds.  The Guix client can then abort right away without
wasting time and power to attempt a doomed build.  Users may override
this with “--fallback” or a new option.

--
Ricardo

bug#33737: do not attempt to build a package known to be broken

[Björn Höfling]

[-- Attachment #1: Type: text/plain, Size: 645 bytes --]

On Fri, 14 Dec 2018 05:41:37 +0100
Ricardo Wurmus <rekado@elephly.net> wrote:

> The default use case is for users to have substitutes enabled and to
> fetch binaries from a build farm.  The build farm tries to build all
> packages, but not all builds succeed.  The daemon caches build
> failures.
> 
> This information, however, is not available for users.  The default
> behaviour of Guix is to attempt building the package locally.  This is
> fine for packages that the build farm hasn’t attempted yet, but it is
> a waste of time when the build has already been attempted without
> success.

+1

Very good idea.

Björn

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

bug#33737: do not attempt to build a package known to be broken

[znavko]

[-- Attachment #1: Type: text/plain, Size: 1049 bytes --]

Package can have additional fields in guix repository, like
architecture, version, buid_status, source_type
build_status may contain such info. 
source_type could show if the package source url is one of those public git services (thus, user could enable proxy first).


Dec 15, 2018, 6:54 PM by bjoern.hoefling@bjoernhoefling.de:

> On Fri, 14 Dec 2018 05:41:37 +0100
> Ricardo Wurmus <> rekado@elephly.net <mailto:rekado@elephly.net>> > wrote:
>
>> The default use case is for users to have substitutes enabled and to
>> fetch binaries from a build farm.  The build farm tries to build all
>> packages, but not all builds succeed.  The daemon caches build
>> failures.
>>
>> This information, however, is not available for users.  The default
>> behaviour of Guix is to attempt building the package locally.  This is
>> fine for packages that the build farm hasn’t attempted yet, but it is
>> a waste of time when the build has already been attempted without
>> success.
>>
>
> +1
>
> Very good idea.
>
> Björn
>


[-- Attachment #2: Type: text/html, Size: 2534 bytes --]

bug#33737: do not attempt to build a package known to be broken

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 1061 bytes --]

On Fri, 14 Dec 2018 05:41:37 +0100
Ricardo Wurmus <rekado@elephly.net> wrote:

> The default use case is for users to have substitutes enabled and to
> fetch binaries from a build farm.  The build farm tries to build all
> packages, but not all builds succeed.  The daemon caches build failures.
> 
> This information, however, is not available for users.  The default
> behaviour of Guix is to attempt building the package locally.  This is
> fine for packages that the build farm hasn’t attempted yet, but it is a
> waste of time when the build has already been attempted without success.
> 
> I propose that “guix publish” should also convey information about
> failed builds.  The Guix client can then abort right away without
> wasting time and power to attempt a doomed build.  Users may override
> this with “--fallback” or a new option.

As long as we distinguish transient build machine errors (disk full, can't
resolve hostname, too many names in directory, no inodes left etc) from actual
errors in the source code, +1.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

bug#33737: do not attempt to build a package known to be broken

[Ricardo Wurmus]

Danny Milosavljevic <dannym@scratchpost.org> writes:

>> I propose that “guix publish” should also convey information about
>> failed builds.  The Guix client can then abort right away without
>> wasting time and power to attempt a doomed build.  Users may override
>> this with “--fallback” or a new option.
>
> As long as we distinguish transient build machine errors (disk full, can't
> resolve hostname, too many names in directory, no inodes left etc) from actual
> errors in the source code, +1.

I was thinking of something very simple.  The daemon is already capable
of caching build failures; I think it may be good to simply pass that
information on to clients.

I don’t know how we could distinguish different kinds of build failures
with the current daemon.

--
Ricardo

bug#33737: do not attempt to build a package known to be broken

[Ludovic Courtès]

Hello!

Ricardo Wurmus <rekado@elephly.net> skribis:

> Danny Milosavljevic <dannym@scratchpost.org> writes:
>
>>> I propose that “guix publish” should also convey information about
>>> failed builds.  The Guix client can then abort right away without
>>> wasting time and power to attempt a doomed build.  Users may override
>>> this with “--fallback” or a new option.

Good idea!

The difficulty is that the protocol between ‘guix substitute --query’
and the daemon is currently very much Boolean: either ‘guix substitute’
provides the binary or it doesn’t.  We’d have to adjust it.

>> As long as we distinguish transient build machine errors (disk full, can't
>> resolve hostname, too many names in directory, no inodes left etc) from actual
>> errors in the source code, +1.
>
> I was thinking of something very simple.  The daemon is already capable
> of caching build failures; I think it may be good to simply pass that
> information on to clients.

The daemon tries to cache only “persistent” failures.  In particular,
local failures likely due to lack of space are not cached.

However, the daemon currently doesn’t distinguish between ENOSPC and
actual failures for offloaded builds.  We may be able to fix this by
having ‘guix offload’ check the available disk space upon failure.

Then there’s the problem of non-deterministic failures.  For these it
would be nicer to let the user build stuff anyway.  There’s no good
solution here.

Ludo’.