bug#41878: Handshake Error

bug#41878: Handshake Error

[Quinten Gruenthal]

[-- Attachment #1: Type: text/plain, Size: 1820 bytes --]

I received the following error when performing a guix pull:

|substitute: guix substitute: error: TLS error in procedure 'handshake':
Error in the pull function.
killing process 8828
Backtrace:
          11 (primitive-load
"/gnu/store/pl48b057h6yg8w6f7hafiilcc44d0fn6-compute-guix-derivation")
In ice-9/eval.scm:
    155:9 10 (_ _)
    159:9  9 (_ #(#(#(#(#(#(#(#(#(#(#(#(#(#<directory (guile-user)
7fc5ff085f?> ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?))
In ./guix/store.scm:
  2025:24  8 (run-with-store #<store-connection 256.99 7fc5fd407640> _
#:guile-for-build _ #:system _ #:target _)
   1859:8  7 (_ _)
In ./guix/gexp.scm:
   243:18  6 (_ _)
   1061:2  5 (_ _)
    921:2  4 (_ _)
    782:4  3 (_ _)
In ./guix/store.scm:
  1907:12  2 (_ #<store-connection 256.99 7fc5fd407640>)
   1356:5  1 (map/accumulate-builds #<store-connection 256.99 7fc5fd407640>
_ _)
  1367:15  0 (_ #<store-connection 256.99 7fc5fd407640> _ _)

./guix/store.scm:1367:15: ERROR:
  1. &store-protocol-error:
      message: "`/gnu/store/5r3sb6bj6pppn4h35a35956mv5qrd011-guix-command
substitute' died unexpectedly"
      status: 1
guix pull: error: You found a bug: the program
'/gnu/store/pl48b057h6yg8w6f7hafiilcc44d0fn6-compute-guix-derivation'
failed to compute the derivation for Guix (version:
"cf48f0fc4c40a2ec0b38a445e1e13f37722a0ade"; system: "x86_64-linux";
host version: "ecf92194a55188a9c217d76617378749db063453"; pull-version: 1).
Please report it by email to <bug-guix@gnu.org>.

I've seen this kind of error before in other software and I believe it
occurs when a handshake can't be performed in time and the connection is
dropped; I was able to successfully pull a few minutes later under better
conditions. That being said I bring this up because a problem in my network
(I think) is being treated like an error in your program.

[-- Attachment #2: Type: text/html, Size: 2194 bytes --]

bug#41878: Handshake Error

[Ludovic Courtès]

Hi,

Quinten Gruenthal <quintengruenthal@gmail.com> skribis:

> |substitute: guix substitute: error: TLS error in procedure 'handshake':
> Error in the pull function.

[...]

> I've seen this kind of error before in other software and I believe it
> occurs when a handshake can't be performed in time and the connection is
> dropped; I was able to successfully pull a few minutes later under better
> conditions. That being said I bring this up because a problem in my network
> (I think) is being treated like an error in your program.

Yes, it looks like a connectivity issue, either on your side or on the
side of https://ci.guix.gnu.org (though I’m unaware of downtime today).

Could you try again?

Thanks,
Ludo’.

bug#41878: Handshake Error

[Quinten Gruenthal]

[-- Attachment #1: Type: text/plain, Size: 1571 bytes --]

Sure; though, as I mentioned before I did try again and it did work. Doing
a pull once more I failed with:

guix substitute: error: TLS error in procedure 'handshake': Error in the
pull function.
substitution of
/gnu/store/k1da7bv579d8gvwdrakd9l4hxswknff2-guix-module-union failed
guix pull: error: some substitutes for the outputs of derivation
`/gnu/store/k2gdq7ahjaqmdgjr9xwywqpxvzazn467-guix-e07573432.drv' failed
(usually happens due to networking issues); try `--fallback' to build
derivation from source

As with before, I was also able to succeed on a subsequent pull; but this
error seemed to be handled better as it suggested both the cause of the
problem and a possible solution.

On Mon, Jun 15, 2020 at 12:42 PM Ludovic Courtès <ludo@gnu.org> wrote:

> Hi,
>
> Quinten Gruenthal <quintengruenthal@gmail.com> skribis:
>
> > |substitute: guix substitute: error: TLS error in procedure 'handshake':
> > Error in the pull function.
>
> [...]
>
> > I've seen this kind of error before in other software and I believe it
> > occurs when a handshake can't be performed in time and the connection is
> > dropped; I was able to successfully pull a few minutes later under better
> > conditions. That being said I bring this up because a problem in my
> network
> > (I think) is being treated like an error in your program.
>
> Yes, it looks like a connectivity issue, either on your side or on the
> side of https://ci.guix.gnu.org (though I’m unaware of downtime today).
>
> Could you try again?
>
> Thanks,
> Ludo’.
>

[-- Attachment #2: Type: text/html, Size: 2133 bytes --]

bug#41878: Handshake Error

[Ludovic Courtès]

Hi,

Quinten Gruenthal <quintengruenthal@gmail.com> skribis:

> Sure; though, as I mentioned before I did try again and it did work. Doing
> a pull once more I failed with:
>
> guix substitute: error: TLS error in procedure 'handshake': Error in the
> pull function.
> substitution of
> /gnu/store/k1da7bv579d8gvwdrakd9l4hxswknff2-guix-module-union failed
> guix pull: error: some substitutes for the outputs of derivation
> `/gnu/store/k2gdq7ahjaqmdgjr9xwywqpxvzazn467-guix-e07573432.drv' failed
> (usually happens due to networking issues); try `--fallback' to build
> derivation from source

Could it be that you’re on a flaky network connection?

> As with before, I was also able to succeed on a subsequent pull; but this
> error seemed to be handled better as it suggested both the cause of the
> problem and a possible solution.

If the connection is dropped, there’s little we can do, but maybe you’re
suggesting better error reporting?

Thanks,
Ludo’.

PS: Please keep the bug Cc’d.

bug#41878: Handshake Error

[Bengt Richter]

Hi,

On +2020-06-16 10:07:51 +0200, Ludovic Courtès wrote:
> Hi,
> 
> Quinten Gruenthal <quintengruenthal@gmail.com> skribis:
> 
> > Sure; though, as I mentioned before I did try again and it did work. Doing
> > a pull once more I failed with:
> >
> > guix substitute: error: TLS error in procedure 'handshake': Error in the
> > pull function.
> > substitution of
> > /gnu/store/k1da7bv579d8gvwdrakd9l4hxswknff2-guix-module-union failed
> > guix pull: error: some substitutes for the outputs of derivation
> > `/gnu/store/k2gdq7ahjaqmdgjr9xwywqpxvzazn467-guix-e07573432.drv' failed
> > (usually happens due to networking issues); try `--fallback' to build
> > derivation from source
> 
> Could it be that you’re on a flaky network connection?
> 
> > As with before, I was also able to succeed on a subsequent pull; but this
> > error seemed to be handled better as it suggested both the cause of the
> > problem and a possible solution.
> 
> If the connection is dropped, there’s little we can do, but maybe you’re
> suggesting better error reporting?

Hm, are gexp's checkpointable?

Some ftp can reconnect and continue, IUUC. Idk how that works with https.
Can one adjust timeouts? Continue on an alternate mirror?
E.g. for downloading substitutes?

In general, I hate it when I get 1.0GB through a 1.1GB ISO and lose it all,
esp when I pay by GBs. Can guix help? :)

BTW, I think I have heard of security risks in restarting, but perhaps
with end-to-end integrity checks it is not an issue despite poss MM trying?

> 
> Thanks,
> Ludo’.
> 
> PS: Please keep the bug Cc’d.
> 
> 
> 

-- 
Regards,
Bengt Richter

bug#41878: Handshake Error

[Quinten Gruenthal]

[-- Attachment #1: Type: text/plain, Size: 3637 bytes --]

Yes and yes. I didn't state explicitly that it was a problem with my
network only because I didn't want to appear overconfident in my amateur
diagnosis and I definitely prefer the second error message to the one that
dumps a trace and prompts the filing of a bug. This:

substitute: guix substitute: error: TLS error in procedure 'handshake':
Error in the pull function.
killing process 8828
Backtrace:
          11 (primitive-load
"/gnu/store/pl48b057h6yg8w6f7hafiilcc44d0fn6-compute-guix-derivation")
In ice-9/eval.scm:
    155:9 10 (_ _)
    159:9  9 (_ #(#(#(#(#(#(#(#(#(#(#(#(#(#<directory (guile-user)
7fc5ff085f?> ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?) ?))
In ./guix/store.scm:
  2025:24  8 (run-with-store #<store-connection 256.99 7fc5fd407640> _
#:guile-for-build _ #:system _ #:target _)
   1859:8  7 (_ _)
In ./guix/gexp.scm:
   243:18  6 (_ _)
   1061:2  5 (_ _)
    921:2  4 (_ _)
    782:4  3 (_ _)
In ./guix/store.scm:
  1907:12  2 (_ #<store-connection 256.99 7fc5fd407640>)
   1356:5  1 (map/accumulate-builds #<store-connection 256.99 7fc5fd407640>
_ _)
  1367:15  0 (_ #<store-connection 256.99 7fc5fd407640> _ _)

./guix/store.scm:1367:15: ERROR:
  1. &store-protocol-error:
      message: "`/gnu/store/5r3sb6bj6pppn4h35a35956mv5qrd011-guix-command
substitute' died unexpectedly"
      status: 1
guix pull: error: You found a bug: the program
'/gnu/store/pl48b057h6yg8w6f7hafiilcc44d0fn6-compute-guix-derivation'
failed to compute the derivation for Guix (version:
"cf48f0fc4c40a2ec0b38a445e1e13f37722a0ade"; system: "x86_64-linux";
host version: "ecf92194a55188a9c217d76617378749db063453"; pull-version: 1).
Please report it by email to <bug-guix@gnu.org>.

is much different than this:

guix substitute: error: TLS error in procedure 'handshake': Error in the
pull function.
substitution of
/gnu/store/k1da7bv579d8gvwdrakd9l4hxswknff2-guix-module-union failed
guix pull: error: some substitutes for the outputs of derivation
`/gnu/store/k2gdq7ahjaqmdgjr9xwywqpxvzazn467-guix-e07573432.drv' failed
(usually happens due to networking issues); try `--fallback' to build
derivation from source

especially to one as new to scheme as myself. I can't say why the two
errors were different as I'm not familiar mechanically with what phases
pull goes through but I can say the first error occurred during the
beginning of the pull and the second occurred in the wrap up thing it does
towards the end.


On Tue, Jun 16, 2020 at 1:07 AM Ludovic Courtès <ludo@gnu.org> wrote:

> Hi,
>
> Quinten Gruenthal <quintengruenthal@gmail.com> skribis:
>
> > Sure; though, as I mentioned before I did try again and it did work.
> Doing
> > a pull once more I failed with:
> >
> > guix substitute: error: TLS error in procedure 'handshake': Error in the
> > pull function.
> > substitution of
> > /gnu/store/k1da7bv579d8gvwdrakd9l4hxswknff2-guix-module-union failed
> > guix pull: error: some substitutes for the outputs of derivation
> > `/gnu/store/k2gdq7ahjaqmdgjr9xwywqpxvzazn467-guix-e07573432.drv' failed
> > (usually happens due to networking issues); try `--fallback' to build
> > derivation from source
>
> Could it be that you’re on a flaky network connection?
>
> > As with before, I was also able to succeed on a subsequent pull; but this
> > error seemed to be handled better as it suggested both the cause of the
> > problem and a possible solution.
>
> If the connection is dropped, there’s little we can do, but maybe you’re
> suggesting better error reporting?
>
> Thanks,
> Ludo’.
>
> PS: Please keep the bug Cc’d.
>

[-- Attachment #2: Type: text/html, Size: 4493 bytes --]

bug#41878: Handshake Error

[Ludovic Courtès]

Hi,

Quinten Gruenthal <quintengruenthal@gmail.com> skribis:

> Yes and yes. I didn't state explicitly that it was a problem with my
> network only because I didn't want to appear overconfident in my amateur
> diagnosis and I definitely prefer the second error message to the one that
> dumps a trace and prompts the filing of a bug. This:
>
> substitute: guix substitute: error: TLS error in procedure 'handshake':
> Error in the pull function.

I see.  I’ll take a look and see how we can improve on this.

Thanks,
Ludo’.

bug#41878: 'guix substitute' and 'guix pull' fail gracelessly on flaky networks

[Brendan Tildesley]

I have not looked closely but from observation I think currently guix 
first decides if it is going to commit to using a substitute, or falling 
back to building locally, by checking if substitutes are available then 
committing to a method. This differs from the concept of a fallback in 
my head, which would involve trying option B only after option A has 
been tried and failed. guix's way means there are a class of failures 
where guix simply gives up and stops instead of falling back.

In my experience, probably 10% of the time I try a guix pull; guix 
package -u ., there is some weird network error that doesn't happen the 
second time I run it. Perhaps it would be sufficient to simply try twice 
for every substitute; accumulate a list of failed substitutes and retry 
them after iterating through the list of substitutes to download, then 
if that fails try building from source. only then are we allowed to give up.

bug#41878: 'guix substitute' and 'guix pull' fail gracelessly on flaky networks

[zimoun]

Dear,

On Sun, 30 Aug 2020 at 20:06, Brendan Tildesley <mail@brendan.scot> wrote:
> I have not looked closely but from observation I think currently guix first
> decides if it is going to commit to using a substitute, or falling back to
> building locally, by checking if substitutes are available then committing to
> a method. This differs from the concept of a fallback in my head, which would
> involve trying option B only after option A has been tried and failed. guix's
> way means there are a class of failures where guix simply gives up and stops
> instead of falling back.

What do you mean?

> In my experience, probably 10% of the time I try a guix pull; guix package -u
> ., there is some weird network error that doesn't happen the second time I run
> it. Perhaps it would be sufficient to simply try twice for every substitute;
> accumulate a list of failed substitutes and retry them after iterating through
> the list of substitutes to download, then if that fails try building from
> source. only then are we allowed to give up.

It rings a bell to me.  Something about the configuration of Cuirass and
the build farm serving the substitutes; related to caching.  But I am
not able to find the relevant pointer.


All the best,
simon