collection of “guix pull“ bug reports

collection of “guix pull“ bug reports

[Simon Tournier]

Hi,

The bug tracker – or guix-devel and/or help-guix – reports several
similar issues: “guix pull” fails and then run it again (or again and
again and …) makes it pass.  Well, part of the collection:

https://issues.guix.gnu.org/issue/62830
https://issues.guix.gnu.org/issue/63451
https://issues.guix.gnu.org/issue/63830
https://issues.guix.gnu.org/issue/64489
https://issues.guix.gnu.org/issue/64659
https://issues.guix.gnu.org/issue/64753
https://issues.guix.gnu.org/issue/64963

It is hard to know if all have the same root and/or they are the trace
of the same bug but there is an issue with “guix pull”.  For example
#64302 and #64489 had been merged but based on #64489 message, I am not
convinced the issue is the same as the reported by #64302.  Anyway.

Recently, I had to run several times “guix pull” in a row.  And on
another machine, I did it with --no-substitutes because I was tired.

Well, I do not know how to tackle this: most of the time the issue is
not reproducible because it depends on substitute servers.

Any idea about what could be unexpected or what needs some love?

Cheers,
simon

Re: collection of “guix pull“ bug reports

[Maxim Cournoyer]

Hi Simon,

Simon Tournier <zimon.toutoune@gmail.com> writes:

> Hi,
>
> The bug tracker – or guix-devel and/or help-guix – reports several
> similar issues: “guix pull” fails and then run it again (or again and
> again and …) makes it pass.  Well, part of the collection:
>
> https://issues.guix.gnu.org/issue/62830
> https://issues.guix.gnu.org/issue/63451
> https://issues.guix.gnu.org/issue/63830
> https://issues.guix.gnu.org/issue/64489
> https://issues.guix.gnu.org/issue/64659
> https://issues.guix.gnu.org/issue/64753
> https://issues.guix.gnu.org/issue/64963
>
> It is hard to know if all have the same root and/or they are the trace
> of the same bug but there is an issue with “guix pull”.  For example
> #64302 and #64489 had been merged but based on #64489 message, I am not
> convinced the issue is the same as the reported by #64302.  Anyway.
>
> Recently, I had to run several times “guix pull” in a row.  And on
> another machine, I did it with --no-substitutes because I was tired.
>
> Well, I do not know how to tackle this: most of the time the issue is
> not reproducible because it depends on substitute servers.
>
> Any idea about what could be unexpected or what needs some love?

I haven't checked the above links, but I think something that would help
in this regard is better handling of network issues.  E.g, don't print a
backtrace on the first connection failure; retry maybe 3 times then
print a helpful error mentioning the network appears unreliable.

-- 
Thanks,
Maxim

Re: collection of “guix pull“ bug reports

[Simon Tournier]

Hi Maxim,

On Sat, 26 Aug 2023 at 22:34, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

>> https://issues.guix.gnu.org/issue/62830
>> https://issues.guix.gnu.org/issue/63451
>> https://issues.guix.gnu.org/issue/63830
>> https://issues.guix.gnu.org/issue/64489
>> https://issues.guix.gnu.org/issue/64659
>> https://issues.guix.gnu.org/issue/64753
>> https://issues.guix.gnu.org/issue/64963

[...]

>> Any idea about what could be unexpected or what needs some love?
>
> I haven't checked the above links, but I think something that would help
> in this regard is better handling of network issues.  E.g, don't print a
> backtrace on the first connection failure; retry maybe 3 times then
> print a helpful error mentioning the network appears unreliable.

IMHO, this collection raises two questions:

  1. Why does it happen?  To say it explicitly, I am almost sure that
  something is not smoothly working as expected on server side.  For
  instance, I had ‘guix pull’ troubles with a machine and I am doubtful
  that this machine has network issue (this machine is using the fast
  network link of my Univ. employer :-))

  2. What could be done on client side for reducing the annoyance?  I
  agree that substitute failures should be better handled.  For example,
  retry 3 times then display an error message.  Etc.


Cheers,
simon

PS: About #2, please give a look at these annoyances:

  + Issue 1 and 2: Options --fallabck and --no-substitutes
  + Issue 3: Non consistent message for substitutes and/or fallback

from <https://simon.tournier.info/posts/2023-06-23-hackathon-repro.html>.

Re: collection of “guix pull“ bug reports

[Kierin Bell]

Simon Tournier <zimon.toutoune@gmail.com> writes:

Hi everyone,

> Hi Maxim,
>
> On Sat, 26 Aug 2023 at 22:34, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>
>>> https://issues.guix.gnu.org/issue/62830
>>> https://issues.guix.gnu.org/issue/63451
>>> https://issues.guix.gnu.org/issue/63830
>>> https://issues.guix.gnu.org/issue/64489
>>> https://issues.guix.gnu.org/issue/64659
>>> https://issues.guix.gnu.org/issue/64753
>>> https://issues.guix.gnu.org/issue/64963
>
> [...]
>
>>> Any idea about what could be unexpected or what needs some love?
>>
>> I haven't checked the above links, but I think something that would help
>> in this regard is better handling of network issues.  E.g, don't print a
>> backtrace on the first connection failure; retry maybe 3 times then
>> print a helpful error mentioning the network appears unreliable.
>
> IMHO, this collection raises two questions:
>
>   1. Why does it happen?  To say it explicitly, I am almost sure that
>   something is not smoothly working as expected on server side.  For
>   instance, I had ‘guix pull’ troubles with a machine and I am doubtful
>   that this machine has network issue (this machine is using the fast
>   network link of my Univ. employer :-))
>
>   2. What could be done on client side for reducing the annoyance?  I
>   agree that substitute failures should be better handled.  For example,
>   retry 3 times then display an error message.  Etc.
>
>
> Cheers,
> simon
>
> PS: About #2, please give a look at these annoyances:
>
>   + Issue 1 and 2: Options --fallabck and --no-substitutes
>   + Issue 3: Non consistent message for substitutes and/or fallback
>
> from <https://simon.tournier.info/posts/2023-06-23-hackathon-repro.html>.
>
>
>

I've had most of these issues.  FWIW, some of them (like
https://issues.guix.gnu.org/issue/63451) only manifested when I had been
using old ethernet dongles or WiFi cards which severely limited
bandwidth.  Even with better hardware, the 'substitute died
unexpectedly' issues still happen sometimes.

I definitely vote for adding some redundancy here, too.

Thanks
-- 
Kierin Bell
GPG Key: FCF2 5F08 EA4F 2E3D C7C3  0D41 D14A 8CD3 2D97 0B36

Re: collection of “guix pull“ bug reports

[Csepp]

Simon Tournier <zimon.toutoune@gmail.com> writes:

> Hi Maxim,
>
> On Sat, 26 Aug 2023 at 22:34, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>
>>> https://issues.guix.gnu.org/issue/62830
>>> https://issues.guix.gnu.org/issue/63451
>>> https://issues.guix.gnu.org/issue/63830
>>> https://issues.guix.gnu.org/issue/64489
>>> https://issues.guix.gnu.org/issue/64659
>>> https://issues.guix.gnu.org/issue/64753
>>> https://issues.guix.gnu.org/issue/64963
>
> [...]
>
>>> Any idea about what could be unexpected or what needs some love?
>>
>> I haven't checked the above links, but I think something that would help
>> in this regard is better handling of network issues.  E.g, don't print a
>> backtrace on the first connection failure; retry maybe 3 times then
>> print a helpful error mentioning the network appears unreliable.
>
> IMHO, this collection raises two questions:
>
>   1. Why does it happen?  To say it explicitly, I am almost sure that
>   something is not smoothly working as expected on server side.  For
>   instance, I had ‘guix pull’ troubles with a machine and I am doubtful
>   that this machine has network issue (this machine is using the fast
>   network link of my Univ. employer :-))
>
>   2. What could be done on client side for reducing the annoyance?  I
>   agree that substitute failures should be better handled.  For example,
>   retry 3 times then display an error message.  Etc.
>
>
> Cheers,
> simon
>
> PS: About #2, please give a look at these annoyances:
>
>   + Issue 1 and 2: Options --fallabck and --no-substitutes
>   + Issue 3: Non consistent message for substitutes and/or fallback
>
> from <https://simon.tournier.info/posts/2023-06-23-hackathon-repro.html>.

There also needs to be an option to just retry forever IMHO.  On my
netbook it is very costly to re-run guix pull's non-substitutable parts
so I'd rather Guix just kept trying in case of network errors.

collection of “guix pull“ bug reports

[Simon Tournier]

Hi,

>>> https://issues.guix.gnu.org/issue/62830
>>> https://issues.guix.gnu.org/issue/63451
>>> https://issues.guix.gnu.org/issue/63830
>>> https://issues.guix.gnu.org/issue/64489
>>> https://issues.guix.gnu.org/issue/64659
>>> https://issues.guix.gnu.org/issue/64753
>>> https://issues.guix.gnu.org/issue/64963

More are reported…

>>> Any idea about what could be unexpected or what needs some love?
>>
>> I haven't checked the above links, but I think something that would help
>> in this regard is better handling of network issues.  E.g, don't print a
>> backtrace on the first connection failure; retry maybe 3 times then
>> print a helpful error mentioning the network appears unreliable.
>
> IMHO, this collection raises two questions:
>
>   1. Why does it happen?  To say it explicitly, I am almost sure that
>   something is not smoothly working as expected on server side.  For
>   instance, I had ‘guix pull’ troubles with a machine and I am doubtful
>   that this machine has network issue (this machine is using the fast
>   network link of my Univ. employer :-))
>
>   2. What could be done on client side for reducing the annoyance?  I
>   agree that substitute failures should be better handled.  For example,
>   retry 3 times then display an error message.  Etc.

Well, most of the time, the error is transient so it is hard to debug.
On client side (#2), well I have ideas how to fake a faulty network on
my end.  What could be done on server side?

Cheers,
simon