bug#39387: Builds are *not* offloaded when using the "--keep-failed" or -K option.

bug#39387: Builds are *not* offloaded when using the "--keep-failed" or -K option.

[Maxim Cournoyer]

Consider the following, where 10.42.0.199 is a machine described in
/etc/guix/machines.scm for offloading:

$ ./pre-inst-env guix build php
The following derivation will be built:
   /gnu/store/dj9qi9lj4xz0dk6agq9mndh8mhi8ifrg-php-7.4.1.drv
process 8241 acquired build slot '/var/guix/offload/10.42.0.199:22/0'
load on machine '10.42.0.199' is 0.04 (normalized: 0.04)
building /gnu/store/dj9qi9lj4xz0dk6agq9mndh8mhi8ifrg-php-7.4.1.drv...
^C

$ ./pre-inst-env guix build php --keep-failed
The following derivation will be built:
   /gnu/store/dj9qi9lj4xz0dk6agq9mndh8mhi8ifrg-php-7.4.1.drv
building /gnu/store/dj9qi9lj4xz0dk6agq9mndh8mhi8ifrg-php-7.4.1.drv...
starting phase `set-SOURCE-DATE-EPOCH'
phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds
starting phase `set-paths'
environment variable `PATH' set to
`/gnu/store/niqa6dlvw4r2bfbabn65kq1kgdfwcb4b-pkg-config-0.29.2/bin:/gnu/store/qz06d1dz19wkrmibcr1485wqqia9zzhi-bison-3.5.1/bin:/gnu/store/3fzw8qb3cmrdkr46y55njjrfkvxd53lr-intltool-0.51.0/bin:
[...]

In the second case, the build is started on the local machine instead of
being dispatched to the offload machine.  Presumably, this is done so
that the files are available locally; but I'd prefer if it'd offload and
keep the files on the remote.  An alternative would be to copy the remote build directory
locally after failure.

bug#39387: Builds are *not* offloaded when using the "--keep-failed" or -K option.

[Ludovic Courtès]

Hi Maxim,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> In the second case, the build is started on the local machine instead of
> being dispatched to the offload machine.  Presumably, this is done so
> that the files are available locally; but I'd prefer if it'd offload and
> keep the files on the remote.  An alternative would be to copy the remote build directory
> locally after failure.

Indeed, it’s a feature!  :-)  It was discussed here:

  https://issues.guix.gnu.org/issue/38312

There we discussed the possibility of printing a warning, which wasn’t
implemented.

I’m closing in the meantime, but let me know what you think!

Thank you,
Ludo’.

bug#39387: Builds are *not* offloaded when using the "--keep-failed" or -K option.

[Maxim Cournoyer]

Hello!

Ludovic Courtès <ludo@gnu.org> writes:

> Hi Maxim,
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> In the second case, the build is started on the local machine instead of
>> being dispatched to the offload machine.  Presumably, this is done so
>> that the files are available locally; but I'd prefer if it'd offload and
>> keep the files on the remote.  An alternative would be to copy the remote build directory
>> locally after failure.
>
> Indeed, it’s a feature!  :-)  It was discussed here:
>
>   https://issues.guix.gnu.org/issue/38312
>
> There we discussed the possibility of printing a warning, which wasn’t
> implemented.

I see!

> I’m closing in the meantime, but let me know what you think!

OK!  I think I now see the technical difficulties in implementing a
useful, offloaded interrupted build: the build artifacts, even if copied
locally, would require syncing all the build time dependencies in order
to allow sourcing the "environment-variables" script and be able to
debug.  That's more the job of 'guix publish', not of 'guix offload'.

Maxim

bug#39387: Builds are *not* offloaded when using the "--keep-failed" or -K option.

[Ludovic Courtès]

Hi Maxim,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:

[...]

>> Indeed, it’s a feature!  :-)  It was discussed here:
>>
>>   https://issues.guix.gnu.org/issue/38312
>>
>> There we discussed the possibility of printing a warning, which wasn’t
>> implemented.
>
> I see!
>
>> I’m closing in the meantime, but let me know what you think!
>
> OK!  I think I now see the technical difficulties in implementing a
> useful, offloaded interrupted build: the build artifacts, even if copied
> locally, would require syncing all the build time dependencies in order
> to allow sourcing the "environment-variables" script and be able to
> debug.  That's more the job of 'guix publish', not of 'guix offload'.

Right.  Ideally the build directory of an offloaded could be copied back
to the host of the caller, but that’s not implemented and a bit tricky
maybe.

Before this change, my main use of ‘--no-build-hook’ would be together
with ‘-K’ for that reason.

Ludo’.

bug#39387: Builds are *not* offloaded when using the "--keep-failed" or -K option.

[Maxim Cournoyer]

Ludovic Courtès <ludo@gnu.org> writes:

> Hi Maxim,
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> In the second case, the build is started on the local machine instead of
>> being dispatched to the offload machine.  Presumably, this is done so
>> that the files are available locally; but I'd prefer if it'd offload and
>> keep the files on the remote.  An alternative would be to copy the remote build directory
>> locally after failure.
>
> Indeed, it’s a feature!  :-)  It was discussed here:
>
>   https://issues.guix.gnu.org/issue/38312
>
> There we discussed the possibility of printing a warning, which wasn’t
> implemented.
>
> I’m closing in the meantime, but let me know what you think!
>
> Thank you,
> Ludo’.

Still, it'd be nice to have an option to keep failed build on the
*remote* machines.  I don't think that's possible currently?

Maxim

bug#39387: Builds are *not* offloaded when using the "--keep-failed" or -K option.

[Ludovic Courtès]

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> Still, it'd be nice to have an option to keep failed build on the
> *remote* machines.  I don't think that's possible currently?

In some cases it’s not even desirable because you could fill up /tmp
without noticing.

But note that another option is to do:

  GUIX_DAEMON_SOCKET=ssh://remote.example.org guix build foo -K

WDYT?

Ludo’.

bug#39387: Builds are *not* offloaded when using the "--keep-failed" or -K option.

[Maxim Cournoyer]

Hello Ludovic!

Ludovic Courtès <ludo@gnu.org> writes:

> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> Still, it'd be nice to have an option to keep failed build on the
>> *remote* machines.  I don't think that's possible currently?
>
> In some cases it’s not even desirable because you could fill up /tmp
> without noticing.
>
> But note that another option is to do:
>
>   GUIX_DAEMON_SOCKET=ssh://remote.example.org guix build foo -K
>
> WDYT?
>
> Ludo’.

This fulfill my use case well and appears to be sufficiently documented
already.  We're gold!

Thank you,

Maxim