guix weather exit status?

guix weather exit status?

[Leo Famulari]

When a substitute is not available for all specified substitute servers,
`guix weather` exits with a return code of '1', signaling failure.

For example:

------
$ ./pre-inst-env guix weather linux-libre; echo $?
computing 1 package derivations for x86_64-linux...
looking for 1 store items on http://ci.guix.gnu.org...
http://ci.guix.gnu.org
  100.0% substitutes available (1 out of 1)
  at least 234.8 MiB of nars (compressed)
  86.3 MiB on disk (uncompressed)

  38 queued builds
      aarch64-linux: 31 (81.6%)
      powerpc64le-linux: 1 (2.6%)
      armhf-linux: 1 (2.6%)
      i686-linux: 5 (13.2%)
  build rate: .00 builds per hour
      x86_64-linux: 0.00 builds per hour
      aarch64-linux: 0.00 builds per hour
      i686-linux: 0.00 builds per hour
looking for 1 store items on http://bordeaux.guix.gnu.org...
http://bordeaux.guix.gnu.org
  0.0% substitutes available (0 out of 1)
  unknown substitute sizes
  0.0 MiB on disk (uncompressed)
  (continuous integration information unavailable)
1
------

I expected it to return successfully because a substitute is available,
but I see that it could also make sense if "success" means that a
substitute is available from all servers.

However, the current behaviour breaks the use case of "wait until a
substitute is available and then download it", like this for example:

$ while true; do guix weather linux-libre && break; sleep 600; done; guix build --no-grafts linux-libre

I'm not sure if this topic was discussed previously... your thoughts?

Re: guix weather exit status?

[Ludovic Courtès]

Hi!

Leo Famulari <leo@famulari.name> skribis:

> When a substitute is not available for all specified substitute servers,
> `guix weather` exits with a return code of '1', signaling failure.
>
> For example:
>
> ------
> $ ./pre-inst-env guix weather linux-libre; echo $?
> computing 1 package derivations for x86_64-linux...
> looking for 1 store items on http://ci.guix.gnu.org...
> http://ci.guix.gnu.org
>   100.0% substitutes available (1 out of 1)
>   at least 234.8 MiB of nars (compressed)
>   86.3 MiB on disk (uncompressed)
>
>   38 queued builds
>       aarch64-linux: 31 (81.6%)
>       powerpc64le-linux: 1 (2.6%)
>       armhf-linux: 1 (2.6%)
>       i686-linux: 5 (13.2%)
>   build rate: .00 builds per hour
>       x86_64-linux: 0.00 builds per hour
>       aarch64-linux: 0.00 builds per hour
>       i686-linux: 0.00 builds per hour
> looking for 1 store items on http://bordeaux.guix.gnu.org...
> http://bordeaux.guix.gnu.org
>   0.0% substitutes available (0 out of 1)
>   unknown substitute sizes
>   0.0 MiB on disk (uncompressed)
>   (continuous integration information unavailable)
> 1
> ------
>
> I expected it to return successfully because a substitute is available,
> but I see that it could also make sense if "success" means that a
> substitute is available from all servers.

Quoth the manual (info "(guix) Invoking guix weather"):

  ‘guix weather’ exits with a non-zero code when the fraction of
  available substitutes is below 100%.

> However, the current behaviour breaks the use case of "wait until a
> substitute is available and then download it", like this for example:
>
> $ while true; do guix weather linux-libre && break; sleep 600; done; guix build --no-grafts linux-libre
>
> I'm not sure if this topic was discussed previously... your thoughts?

I agree we could change (or rather refine) semantics to exit with
non-zero when overall coverage is below 100%.  In the example above, it
should return 0.

Thanks,
Ludo’.

Re: guix weather exit status?

[Leo Famulari]

On Sat, Jul 10, 2021 at 04:41:44PM +0200, Ludovic Courtès wrote:
> I agree we could change (or rather refine) semantics to exit with
> non-zero when overall coverage is below 100%.  In the example above, it
> should return 0.

Maybe we could distinguish between various cases like this:

0 means 100% coverage
1 means a substitute is available, but not from all servers
2 means a substitute is not available

This would preserve the current meaning of 0, while still allowing `guix
weather` to be used in scripts to "wait for a substitute".

Re: guix weather exit status?

[zimoun]

Hi,

On Sat, 10 Jul 2021 at 18:06, Leo Famulari <leo@famulari.name> wrote:
> On Sat, Jul 10, 2021 at 04:41:44PM +0200, Ludovic Courtès wrote:
>> I agree we could change (or rather refine) semantics to exit with
>> non-zero when overall coverage is below 100%.  In the example above, it
>> should return 0.
>
> Maybe we could distinguish between various cases like this:
>
> 0 means 100% coverage
> 1 means a substitute is available, but not from all servers
> 2 means a substitute is not available
>
> This would preserve the current meaning of 0, while still allowing `guix
> weather` to be used in scripts to "wait for a substitute".

What about two or more packages?  “guix weather foo bar” when the
package ’foo’ is available on one server and ’bar’ is not at all.  Or if
’foo’ is available on one server and ’bar’ on the other.  Etc.

We could have:

 0 100% all servers
 1 100% from one server
 2 100% from a server mix
 3 one or more substitute is missing

WDYT?

Cheers,
simon

Re: guix weather exit status?

[Kaelyn]

Hi,

I'm fairly new to Guix and haven't made much use of guix weather, but had a thought about the exit status.

On Sunday, July 11th, 2021 at 4:40 AM, zimoun <zimon.toutoune@gmail.com> wrote:

> Hi,
>
> On Sat, 10 Jul 2021 at 18:06, Leo Famulari leo@famulari.name wrote:
>
> > On Sat, Jul 10, 2021 at 04:41:44PM +0200, Ludovic Courtès wrote:
> >
> > > I agree we could change (or rather refine) semantics to exit with
> > >
> > > non-zero when overall coverage is below 100%. In the example above, it
> > >
> > > should return 0.
> >
> > Maybe we could distinguish between various cases like this:
> >
> > 0 means 100% coverage
> >
> > 1 means a substitute is available, but not from all servers
> >
> > 2 means a substitute is not available
> >
> > This would preserve the current meaning of 0, while still allowing `guix weather` to be used in scripts to "wait for a substitute".
>
> What about two or more packages? “guix weather foo bar” when the
>
> package ’foo’ is available on one server and ’bar’ is not at all. Or if
>
> ’foo’ is available on one server and ’bar’ on the other. Etc.
>
> We could have:
>
> 0 100% all servers
>
> 1 100% from one server
>
> 2 100% from a server mix
>
> 3 one or more substitute is missing

It looks to me like there are two overlapping use cases for guix weather and it's exit status (and sorry if they aren't the clearest): checking substitute availability for download, and substitute progress/coverage among servers.

There is already a "--coverage" flag to guix weather that the help describes as "show substitute coverage for packages with at least COUNT dependents". What if a new flag was added, say "-a" / "--available", which specifies the number of servers a single substitute must be available from to be considered available, possibly with a default of "1". That way guix weather can exit 0 when needed for both of the use cases.

For example, with the original output that showed 100% substitute availability from ci.guix.gnu.org and 0% availability from bordeaux.guix.gnu.org for a single package and exited 1 as a result:
* "guix weather -a 1" would exit 0
* "guix weather -a 2" would exit 1 because it was available from less than 2 servers.

That way, even when multiple packages are specified at once, guix weather would exit 0 as long as all of the packages met the requested minimum availability.

Cheers,
Kaelyn


P.S. If folks like the idea, I'm volunteering to implement it. ;)