unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* 1.2.1 pre-release testing
@ 2021-04-17  3:40 jcguu95
  2021-04-17  5:43 ` Leo Famulari
                   ` (3 more replies)
  0 siblings, 4 replies; 35+ messages in thread
From: jcguu95 @ 2021-04-17  3:40 UTC (permalink / raw)
  To: guix-devel

Hi all!

First time posting here. Please let me know if I missed anything.

This is about my testing of 1.2.1 pre-release
(<https://ci.guix.gnu.org/build/175632/details>). I wiped out my machine
and installed it two times. Here are some summary about the experience.

1. Graphical Installation

   I chose the graphical method for the first installation. Everything
   went well, except my wired internet connection failed as it
   indicated. Weirdly enough, my ethernet cable has been always on my
   machine for a while, and never failed. Anyway, I managed to get wifi
   working, and that helped complete my installation process.

2. CLI-based Installation

   2.1. Better support for using git during installation process?

   Before `guix system init /mnt/etc/config.scm /mnt', I'd like to getch
   config.scm from my github repo. So I ran `guix package -i git'. But
   when I `git clone ...' as usual, there's an error "git:
   'remote-https' is not a git command .See 'git --help'".

   What's going on? Why does `git clone' fails when I'm installing guix
   system?

   2.2. Unexpected failure

   After running `guix system init /mnt/etc/config.scm /mnt' for 5 to 10
   minutes, I got the error

   ,----
   |    substitute: updating substitutes from 'https://ci.guix.gnu.org'... 0.0%guix substitute: error: TSL error in procedure 'write_to_session_port': Resource temporarily unavailable, try again.
   | 
   |    guix system: error: `/gnu/store/1nmwil4cs...vc2p-guix-1.2.0-21.4dff6ec/bin/guix substitute` died unexpectedly
   |    root@gnu ~# _
   `----

   Retrying it did resolve the problem. And I got a working guix system
   out of it :)



In summary, though there was some minor issue, I was able/lucky to
resolve it. Thanks for all the hard work. And please feel free to let me
know if I can help further.

Cheers,

Jin


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: 1.2.1 pre-release testing
  2021-04-17  3:40 1.2.1 pre-release testing jcguu95
@ 2021-04-17  5:43 ` Leo Famulari
  2021-04-17 11:35   ` GIT_EXEC_PATH " François
  2021-04-17  9:32 ` pelzflorian (Florian Pelz)
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 35+ messages in thread
From: Leo Famulari @ 2021-04-17  5:43 UTC (permalink / raw)
  To: jcguu95; +Cc: guix-devel

On Fri, Apr 16, 2021 at 10:40:55PM -0500, jcguu95 wrote:
> First time posting here. Please let me know if I missed anything.

Thanks for your report!

>    I chose the graphical method for the first installation. Everything
>    went well, except my wired internet connection failed as it
>    indicated. Weirdly enough, my ethernet cable has been always on my
>    machine for a while, and never failed. Anyway, I managed to get wifi
>    working, and that helped complete my installation process.

I wonder, can anyone else reproduce the wired connection failure? It
worked for me with the graphical installer. It would be a shame to
release with a bug that breaks wired connections in the installer.

>    2.1. Better support for using git during installation process?
> 
>    Before `guix system init /mnt/etc/config.scm /mnt', I'd like to getch
>    config.scm from my github repo. So I ran `guix package -i git'. But
>    when I `git clone ...' as usual, there's an error "git:
>    'remote-https' is not a git command .See 'git --help'".
> 
>    What's going on? Why does `git clone' fails when I'm installing guix
>    system?

Weird, we need to look into that. I wonder if it's related to
'$GIT_SSL_CAINFO'? Or something else about find the X.509 certificates?
Check here for more info on that:

https://guix.gnu.org/manual/en/html_node/X_002e509-Certificates.html

I also found some suggestions regarding $GIT_EXEC_PATH needing to be
set.

It would be nice to somehow offer "fetch a remote config.scm" within the
graphical installer. Something for 1.3.0, I suppose!

>    After running `guix system init /mnt/etc/config.scm /mnt' for 5 to 10
>    minutes, I got the error
> 
>    ,----
>    |    substitute: updating substitutes from 'https://ci.guix.gnu.org'... 0.0%guix substitute: error: TSL error in procedure 'write_to_session_port': Resource temporarily unavailable, try again.
>    | 
>    |    guix system: error: `/gnu/store/1nmwil4cs...vc2p-guix-1.2.0-21.4dff6ec/bin/guix substitute` died unexpectedly
>    |    root@gnu ~# _
>    `----
> 
>    Retrying it did resolve the problem. And I got a working guix system
>    out of it :)

Hm :/ I hope it doesn't happen too often. I wonder how we can debug
this?

> 
> In summary, though there was some minor issue, I was able/lucky to
> resolve it. Thanks for all the hard work. And please feel free to let me
> know if I can help further.
> 
> Cheers,
> 
> Jin
> 


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: 1.2.1 pre-release testing
  2021-04-17  3:40 1.2.1 pre-release testing jcguu95
  2021-04-17  5:43 ` Leo Famulari
@ 2021-04-17  9:32 ` pelzflorian (Florian Pelz)
  2021-04-17 12:58   ` Guu, Jin-Cheng
  2021-04-17 17:27 ` Leo Famulari
  2021-04-18  9:38 ` [1.2.1 pre-release testing] substitute downloading and TLS errors pelzflorian (Florian Pelz)
  3 siblings, 1 reply; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-17  9:32 UTC (permalink / raw)
  To: jcguu95; +Cc: guix-devel

Hello Jin.  Thank you for testing!

On Fri, Apr 16, 2021 at 10:40:55PM -0500, jcguu95 wrote:
> 1. Graphical Installation
> 
>    I chose the graphical method for the first installation. Everything
>    went well, except my wired internet connection failed as it
>    indicated. Weirdly enough, my ethernet cable has been always on my
>    machine for a while, and never failed. […]

Does the ethernet cable work on the installed system?  Or on other
free distros?

In the distant past, I had trouble using the built-in Ethernet on my
11-year old Macbook until (I think) a newer Linux-libre kernel release
added support.  Maybe this is similar.

If Ethernet is only broken in the installer though, then the installer
is at fault.

Regards,
Florian


^ permalink raw reply	[flat|nested] 35+ messages in thread

* GIT_EXEC_PATH Re: 1.2.1 pre-release testing
  2021-04-17  5:43 ` Leo Famulari
@ 2021-04-17 11:35   ` François
  2021-04-17 17:24     ` Leo Famulari
  0 siblings, 1 reply; 35+ messages in thread
From: François @ 2021-04-17 11:35 UTC (permalink / raw)
  To: Leo Famulari; +Cc: guix-devel, jcguu95

Hello,

On Sat, Apr 17, 2021 at 01:43:33AM -0400, Leo Famulari wrote:
> On Fri, Apr 16, 2021 at 10:40:55PM -0500, jcguu95 wrote:
> >    2.1. Better support for using git during installation process?
> > 
> >    Before `guix system init /mnt/etc/config.scm /mnt', I'd like to getch
> >    config.scm from my github repo. So I ran `guix package -i git'. But
> >    when I `git clone ...' as usual, there's an error "git:
> >    'remote-https' is not a git command .See 'git --help'".

> I also found some suggestions regarding $GIT_EXEC_PATH needing to be
> set.

GIT_EXEC_PATH is very probably at fault.

It is set via profile to allow to use git subcommands installed by other
packages but it has no sane default when we are outside a profile.

Perhaps git should have a wrapper to manage this use-case.

François


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: 1.2.1 pre-release testing
  2021-04-17  9:32 ` pelzflorian (Florian Pelz)
@ 2021-04-17 12:58   ` Guu, Jin-Cheng
  2021-04-17 13:17     ` pelzflorian (Florian Pelz)
  0 siblings, 1 reply; 35+ messages in thread
From: Guu, Jin-Cheng @ 2021-04-17 12:58 UTC (permalink / raw)
  To: pelzflorian (Florian Pelz); +Cc: guix-devel

> Does the ethernet cable work on the installed system?  Or on other
> free distros?

It has never failed on me before, even when I was installing arch
linux. So I was a bit surprise yesterday. It had also worked before in
the Guix System I got from the official - so I suppose that's a
completely free distro. Maybe it was just a tiny hiccup..

> If Ethernet is only broken in the installer though, then the installer
> is at fault.

Urgh.. I should have tested it more extensively before installing
using cli. At that point that wasn't a huge problem for me, since my
wifi happened to work..

Jin


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: 1.2.1 pre-release testing
  2021-04-17 12:58   ` Guu, Jin-Cheng
@ 2021-04-17 13:17     ` pelzflorian (Florian Pelz)
  0 siblings, 0 replies; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-17 13:17 UTC (permalink / raw)
  To: Guu, Jin-Cheng; +Cc: guix-devel

Hello Jin,

On Sat, Apr 17, 2021 at 07:58:30AM -0500, Guu, Jin-Cheng wrote:
> > Does the ethernet cable work on the installed system?  Or on other
> > free distros?
> 
> It has never failed on me before, even when I was installing arch
> linux. So I was a bit surprise yesterday.

Arch ships with non-free drivers and firmware.

> It had also worked before in
> the Guix System I got from the official - so I suppose that's a
> completely free distro.

If you had used Guix System on the same Ethernet connection on the
same machine, no nonfree channels and it worked, then the installer
image is at fault.  I will test later today if there is anything is
wrong on any of my machines.

Regards,
Florian


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: GIT_EXEC_PATH Re: 1.2.1 pre-release testing
  2021-04-17 11:35   ` GIT_EXEC_PATH " François
@ 2021-04-17 17:24     ` Leo Famulari
  2021-04-17 18:07       ` François
  0 siblings, 1 reply; 35+ messages in thread
From: Leo Famulari @ 2021-04-17 17:24 UTC (permalink / raw)
  To: François; +Cc: guix-devel, jcguu95

On Sat, Apr 17, 2021 at 01:35:45PM +0200, François wrote:
> GIT_EXEC_PATH is very probably at fault.
> 
> It is set via profile to allow to use git subcommands installed by other
> packages but it has no sane default when we are outside a profile.
> 
> Perhaps git should have a wrapper to manage this use-case.

I think it does work properly in Guix, but requires the user to log in
again after installing Git. That's because a lot of Guix stuff is set up
in the login shell initialization scripts. It might even be enough to do
`bash --login` after installing the package.

I guess that many of us install Git right away, so we don't notice a
problem.


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: 1.2.1 pre-release testing
  2021-04-17  3:40 1.2.1 pre-release testing jcguu95
  2021-04-17  5:43 ` Leo Famulari
  2021-04-17  9:32 ` pelzflorian (Florian Pelz)
@ 2021-04-17 17:27 ` Leo Famulari
  2021-04-17 21:57   ` pelzflorian (Florian Pelz)
  2021-04-18  9:38 ` [1.2.1 pre-release testing] substitute downloading and TLS errors pelzflorian (Florian Pelz)
  3 siblings, 1 reply; 35+ messages in thread
From: Leo Famulari @ 2021-04-17 17:27 UTC (permalink / raw)
  To: guix-devel

I re-installed my Guix System based on the ISO found here:

https://ci.guix.gnu.org/build/175632/details

That's '/gnu/store/lw1j4gn8h51cbfp9dg0g025ngkg83sw1-image.iso.drv'.

It's a simple system, but the process worked for me.

It successfully recognized and re-used my existing /home partition.

No trouble with the laptop's ethernet connection, and no trouble with
missing substitutes or failing substitution.


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: GIT_EXEC_PATH Re: 1.2.1 pre-release testing
  2021-04-17 17:24     ` Leo Famulari
@ 2021-04-17 18:07       ` François
  0 siblings, 0 replies; 35+ messages in thread
From: François @ 2021-04-17 18:07 UTC (permalink / raw)
  To: Leo Famulari; +Cc: guix-devel, jcguu95


On Sat, Apr 17, 2021 at 01:24:51PM -0400, Leo Famulari wrote:
> On Sat, Apr 17, 2021 at 01:35:45PM +0200, François wrote:
> > GIT_EXEC_PATH is very probably at fault.
> > 
> > It is set via profile to allow to use git subcommands installed by other
> > packages but it has no sane default when we are outside a profile.
> > 
> > Perhaps git should have a wrapper to manage this use-case.
> 
> I think it does work properly in Guix, but requires the user to log in
> again after installing Git. That's because a lot of Guix stuff is set up
> in the login shell initialization scripts. It might even be enough to do
> `bash --login` after installing the package.

I had exactly the same problem on foreing distro after installing git
and trying to use it without reloading ~/.guix-profile/etc/profile in
my current shell. So yes, it is just a matter of refreshing the current
environment.

> I guess that many of us install Git right away, so we don't notice a
> problem.

Probably.


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: 1.2.1 pre-release testing
  2021-04-17 17:27 ` Leo Famulari
@ 2021-04-17 21:57   ` pelzflorian (Florian Pelz)
  2021-04-18  0:09     ` Guu, Jin-Cheng
  0 siblings, 1 reply; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-17 21:57 UTC (permalink / raw)
  To: Leo Famulari, Guu, Jin-Cheng; +Cc: guix-devel

Hello Jin,

On Sat, Apr 17, 2021 at 01:27:09PM -0400, Leo Famulari wrote:
> I re-installed my Guix System based on the ISO found here:
> 
> https://ci.guix.gnu.org/build/175632/details
> 
> That's '/gnu/store/lw1j4gn8h51cbfp9dg0g025ngkg83sw1-image.iso.drv'.

Ethernet works on the four machines I tested.  Probably it is an issue
with missing hardware support in the Linux-libre kernel, but you could
try again or look at h-node.org if they have or have not listed your
laptop having working Ethernet.

Regards,
Florian


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: 1.2.1 pre-release testing
  2021-04-17 21:57   ` pelzflorian (Florian Pelz)
@ 2021-04-18  0:09     ` Guu, Jin-Cheng
  2021-04-18  0:19       ` Leo Famulari
  2021-04-18  8:57       ` pelzflorian (Florian Pelz)
  0 siblings, 2 replies; 35+ messages in thread
From: Guu, Jin-Cheng @ 2021-04-18  0:09 UTC (permalink / raw)
  To: pelzflorian (Florian Pelz); +Cc: guix-devel

Not sure if it's appropriate to say here.. but when I was installing,
there was non-free kernel in my machine. And that's why my wifi was
working.

My machine is Lenovo's b50-80, which isn't on the site.
> https://h-node.org/notebooks/catalogue/en/1/1/undef/undef/undef/undef/compatibility/undef/undef/undef?search_string=Lenovo+B50-80&submit=Search

I've just tried again the same ISO, and chose "wired". It works right
away this time. I think it was just a tiny hiccup last time. Should
have tried more times then.. sorry about that!

Jin

On 4/17/21, pelzflorian (Florian Pelz) <pelzflorian@pelzflorian.de> wrote:
> Hello Jin,
>
> On Sat, Apr 17, 2021 at 01:27:09PM -0400, Leo Famulari wrote:
>> I re-installed my Guix System based on the ISO found here:
>>
>> https://ci.guix.gnu.org/build/175632/details
>>
>> That's '/gnu/store/lw1j4gn8h51cbfp9dg0g025ngkg83sw1-image.iso.drv'.
>
> Ethernet works on the four machines I tested.  Probably it is an issue
> with missing hardware support in the Linux-libre kernel, but you could
> try again or look at h-node.org if they have or have not listed your
> laptop having working Ethernet.
>
> Regards,
> Florian
>


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: 1.2.1 pre-release testing
  2021-04-18  0:09     ` Guu, Jin-Cheng
@ 2021-04-18  0:19       ` Leo Famulari
  2021-04-18  8:57       ` pelzflorian (Florian Pelz)
  1 sibling, 0 replies; 35+ messages in thread
From: Leo Famulari @ 2021-04-18  0:19 UTC (permalink / raw)
  To: Guu, Jin-Cheng; +Cc: guix-devel

On Sat, Apr 17, 2021 at 07:09:59PM -0500, Guu, Jin-Cheng wrote:
> I've just tried again the same ISO, and chose "wired". It works right
> away this time. I think it was just a tiny hiccup last time. Should
> have tried more times then.. sorry about that!

Thanks a lot for testing again! We really appreciate it :)


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: 1.2.1 pre-release testing
  2021-04-18  0:09     ` Guu, Jin-Cheng
  2021-04-18  0:19       ` Leo Famulari
@ 2021-04-18  8:57       ` pelzflorian (Florian Pelz)
  1 sibling, 0 replies; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-18  8:57 UTC (permalink / raw)
  To: Guu, Jin-Cheng; +Cc: guix-devel

On Sat, Apr 17, 2021 at 07:09:59PM -0500, Guu, Jin-Cheng wrote:
> Not sure if it's appropriate to say here.. but when I was installing,
> there was non-free kernel in my machine. And that's why my wifi was
> working.

If I remember correctly, on some machines the wifi only works after a
reboot from a non-free system.

> My machine is Lenovo's b50-80, which isn't on the site.

I find Z50 with working wifi and ethernet
<https://h-node.org/notebooks/view/en/1848/Z50-75-80EC00MYFR/2/1/Lenovo/undef/undef/notebook/compatibility/undef/undef/50>
and G50 with*out* working wifi
<https://h-node.org/notebooks/view/en/2196/G50-45-80E30181US/2/1/Lenovo/undef/undef/notebook/compatibility/undef/undef/50>.
This won’t be fixed at the Guix level though.

Regards,
Florian


^ permalink raw reply	[flat|nested] 35+ messages in thread

* [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-17  3:40 1.2.1 pre-release testing jcguu95
                   ` (2 preceding siblings ...)
  2021-04-17 17:27 ` Leo Famulari
@ 2021-04-18  9:38 ` pelzflorian (Florian Pelz)
  2021-04-18  9:44   ` bug#47867: " pelzflorian (Florian Pelz)
  2021-04-18 10:15   ` Ludovic Courtès
  3 siblings, 2 replies; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-18  9:38 UTC (permalink / raw)
  To: jcguu95; +Cc: guix-devel, bug-guix

On Fri, Apr 16, 2021 at 10:40:55PM -0500, jcguu95 wrote:
>    2.2. Unexpected failure
> 
>    After running `guix system init /mnt/etc/config.scm /mnt' for 5 to 10
>    minutes, I got the error
> 
>    ,----
>    |    substitute: updating substitutes from 'https://ci.guix.gnu.org'... 0.0%guix substitute: error: TSL error in procedure 'write_to_session_port': Resource temporarily unavailable, try again.
>    | 
>    |    guix system: error: `/gnu/store/1nmwil4cs...vc2p-guix-1.2.0-21.4dff6ec/bin/guix substitute` died unexpectedly
>    |    root@gnu ~# _
>    `----
> 
>    Retrying it did resolve the problem. And I got a working guix system
>    out of it :)

I’ve got the same TLS error (though the procedure for me is
‘write_to_session_record_port’.

I needed to restart the installer at the last step, this stopped while
downloading enlightenment with no error but it does not continue.

Regards,
Florian


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-18  9:38 ` [1.2.1 pre-release testing] substitute downloading and TLS errors pelzflorian (Florian Pelz)
@ 2021-04-18  9:44   ` pelzflorian (Florian Pelz)
  2021-04-18  9:48     ` pelzflorian (Florian Pelz)
  2021-04-18 10:15   ` Ludovic Courtès
  1 sibling, 1 reply; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-18  9:44 UTC (permalink / raw)
  To: jcguu95; +Cc: guix-devel, 47867

On Sun, Apr 18, 2021 at 11:38:56AM +0200, pelzflorian (Florian Pelz) wrote:
> I needed to restart the installer at the last step, this stopped while
> downloading enlightenment with no error but it does not continue.

Note: The last message was

enlightenment-0.24.2  27.0MiB                                  3.7MiB/s 00:07 [################] 100.0%

But now after maybe 10 minutes it finally continued and died with the
same TLS error about write_to_session_record_port and Resource not
available.


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-18  9:44   ` bug#47867: " pelzflorian (Florian Pelz)
@ 2021-04-18  9:48     ` pelzflorian (Florian Pelz)
  2021-04-18  9:57       ` pelzflorian (Florian Pelz)
  0 siblings, 1 reply; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-18  9:48 UTC (permalink / raw)
  Cc: guix-devel, 47867

On Sun, Apr 18, 2021 at 11:44:25AM +0200, pelzflorian (Florian Pelz) wrote:
> But now after maybe 10 minutes it finally continued and died with the
> same TLS error about write_to_session_record_port and Resource not
> available.

The error happened while

substitute: updating substitutes from […]


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-18  9:48     ` pelzflorian (Florian Pelz)
@ 2021-04-18  9:57       ` pelzflorian (Florian Pelz)
  0 siblings, 0 replies; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-18  9:57 UTC (permalink / raw)
  To: guix-devel, 47867

On Sun, Apr 18, 2021 at 11:48:47AM +0200, pelzflorian (Florian Pelz) wrote:
> On Sun, Apr 18, 2021 at 11:44:25AM +0200, pelzflorian (Florian Pelz) wrote:
> > But now after maybe 10 minutes it finally continued and died with the
> > same TLS error about write_to_session_record_port and Resource not
> > available.
> 
> The error happened while
> 
> substitute: updating substitutes from […]

After starting one more time from the last installer step it
downloaded the same enlightenment substitute again and got stuck the
same way.


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-18  9:38 ` [1.2.1 pre-release testing] substitute downloading and TLS errors pelzflorian (Florian Pelz)
  2021-04-18  9:44   ` bug#47867: " pelzflorian (Florian Pelz)
@ 2021-04-18 10:15   ` Ludovic Courtès
  2021-04-18 11:33     ` pelzflorian (Florian Pelz)
  1 sibling, 1 reply; 35+ messages in thread
From: Ludovic Courtès @ 2021-04-18 10:15 UTC (permalink / raw)
  To: pelzflorian (Florian Pelz); +Cc: guix-devel, jcguu95, 47867

Hi Florian,

"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:

> On Fri, Apr 16, 2021 at 10:40:55PM -0500, jcguu95 wrote:
>>    2.2. Unexpected failure
>> 
>>    After running `guix system init /mnt/etc/config.scm /mnt' for 5 to 10
>>    minutes, I got the error
>> 
>>    ,----
>>    |    substitute: updating substitutes from 'https://ci.guix.gnu.org'... 0.0%guix substitute: error: TSL error in procedure 'write_to_session_port': Resource temporarily unavailable, try again.
>>    | 
>>    |    guix system: error: `/gnu/store/1nmwil4cs...vc2p-guix-1.2.0-21.4dff6ec/bin/guix substitute` died unexpectedly
>>    |    root@gnu ~# _
>>    `----
>> 
>>    Retrying it did resolve the problem. And I got a working guix system
>>    out of it :)
>
> I’ve got the same TLS error (though the procedure for me is
> ‘write_to_session_record_port’.
>
> I needed to restart the installer at the last step, this stopped while
> downloading enlightenment with no error but it does not continue.

Was this in a VM?  Could it be that networking was unstable (e.g., you
were running this over a flaky WiFi connection)?  How reproducible is
this?  Any tips on how to reproduce it?

I did a full bare-bones OS install in a VM, which went fine, but maybe
that’s not representative of your setup.

Thanks,
Ludo’.


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-18 10:15   ` Ludovic Courtès
@ 2021-04-18 11:33     ` pelzflorian (Florian Pelz)
  2021-04-18 12:10       ` pelzflorian (Florian Pelz)
  0 siblings, 1 reply; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-18 11:33 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel, jcguu95, 47867

On Sun, Apr 18, 2021 at 12:15:47PM +0200, Ludovic Courtès wrote:
> Was this in a VM?

No, an Asrock Beebox (real x86_64 hardware).


> Could it be that networking was unstable (e.g., you
> were running this over a flaky WiFi connection)?

It is a stable Ethernet connection.


> How reproducible is
> this?  Any tips on how to reproduce it?
> 
> I did a full bare-bones OS install in a VM, which went fine, but maybe
> that’s not representative of your setup.
> 
> Thanks,
> Ludo’.

The installer completed just now after continuing from the last
install step another time (well with an unrelated error by
grub-install that it could not determine the canonical path of
/boot/efi).  I will try reproducing, but Jin and me both had this
error.

Regards,
Florian


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-18 11:33     ` pelzflorian (Florian Pelz)
@ 2021-04-18 12:10       ` pelzflorian (Florian Pelz)
  2021-04-19  8:05         ` pelzflorian (Florian Pelz)
  0 siblings, 1 reply; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-18 12:10 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel, jcguu95, 47867

On Sun, Apr 18, 2021 at 01:33:37PM +0200, pelzflorian (Florian Pelz) wrote:
> (well with an unrelated error by
> grub-install that it could not determine the canonical path of
> /boot/efi).

Probably

> I will try reproducing, but Jin and me both had this
> error.

I got the error again with enlightenment.

On IRC Ricardo/rekado suspected
c7c7f068c15e419aaf5ef616516aa5ad4e55c2fa, I will try reverting it.

Regards,
Florian


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-18 12:10       ` pelzflorian (Florian Pelz)
@ 2021-04-19  8:05         ` pelzflorian (Florian Pelz)
  2021-04-19  8:22           ` pelzflorian (Florian Pelz)
                             ` (2 more replies)
  0 siblings, 3 replies; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-19  8:05 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel, jcguu95, 47867

On Sun, Apr 18, 2021 at 02:10:06PM +0200, pelzflorian (Florian Pelz) wrote:
> On IRC Ricardo/rekado suspected
> c7c7f068c15e419aaf5ef616516aa5ad4e55c2fa, I will try reverting it.

It was necessary to revert follow-up commits before reverting
c7c7f068c15e419aaf5ef616516aa5ad4e55c2fa.

I tried on Guix master commit fef2f08bc640f78cc0a86fc7be3eccbc07b5e98c
the following commands:

sed -i '/vi/d' po/packages/LINGUAS #otherwise errors; Julien fixed this by now I think
git add po/packages/LINGUAS
git commit -m fixLINGUAS
git revert 2d73086262e1fb33cd0f0f16f74a495fe06b38aa
git revert 673e5276f6b4dda4bfa9dd5bb70220fc8b17abd2
git revert 45fce38fb0b6c6796906149ade145b8d3594c1c6
git revert 9da5ec7099b992a8969a17627548cd341c01bd90
git revert c5ab78f90b111839508d0ab10c0e5ac2038002ca
git revert b48204259aa9cad80c5b23a4060e2d796007ec7a
git revert c37e3b92ad0334ba2fe7ee4e98631f0a4edeee21
git revert 728c90862eb967a1679efdb69e26d7ba37cdc3a3
git revert fd5b77503e852b78a43e1bee4d6bdfbbb1f27e8f
git revert 112692c0d546d35cd67c5dc70dbd1dc609b18f64
git revert ee3226e9d54891c7e696912245e4904435be191c
git revert e2572aa95007558b008f04decff095f46d20e087
git revert 20c08a8a45d0f137ead7c05e720456b2aea44402
git revert 187e97096825d2bcceb144cead6eccc27385acd7
git revert 8116cc66733134a8fb6f9117d4648288b83c8356
git revert b9d058e3f7982eed04cc748cc0b24173a3969c52
git revert 7c85877fdf964694061e3192eac35723ebc047bf
git revert f50f5751fff4cfc6d5abba9681054569694b7a5c
git revert 05f38ca8dccc49d9ec70d1f63461976b97d74d52
git revert 7b812f7c84c43455cdd68a0e51b6ded018afcc8e
git revert f50b501a7472f4f237023831aa415a948115d1d1
git revert 205833b72c5517915a47a50dbe28e7024dc74e57
git revert e2e853ddb0d964e7b0300334ab51ab89950c2376
git revert 87734503a4a1761a4432b601352e5b02340a97f0
git revert 08acee2f98321d57c4faa3ba641a5543b74f10a4
git revert fbd61b5d3de353bfa468641d087bc53aaa1e63a5
git revert be5a75ebb5988b87b2392e2113f6590f353dd6cd
git revert c7c7f068c15e419aaf5ef616516aa5ad4e55c2fa
./bootstrap
./configure --localstatedir=/var
make
GUIX_ALLOW_ME_TO_USE_PRIVATE_COMMIT=y make update-guix-package
# disabling #:tests? on the guix package was *not* necessary once all reverts were in place
make
./pre-inst-env guix system image -t iso9660 gnu/system/install.scm

I dd’d the resulting image to a USB drive and the TLS errors no longer
appear (tried installing everything two times now; without reverts
downloading enlightenment needed retrys everytime).

I suppose if we leave this bug in the new Guix release, then some
substitutes like enlightenment get stuck and fail to download for some
people and therefore the installation fails.  I suspect restarting the
install causes grub-install to fail because it cannot find /boot/efi
anymore, but maybe my grub-install error had other reasons.

I will bisect which reverts are needed, but it takes time …

Regards,
Florian


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-19  8:05         ` pelzflorian (Florian Pelz)
@ 2021-04-19  8:22           ` pelzflorian (Florian Pelz)
  2021-04-19 11:02           ` pelzflorian (Florian Pelz)
  2021-04-20  1:21           ` pelzflorian (Florian Pelz)
  2 siblings, 0 replies; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-19  8:22 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel, jcguu95, 47867

On Mon, Apr 19, 2021 at 10:05:25AM +0200, pelzflorian (Florian Pelz) wrote:
> I dd’d the resulting image to a USB drive and the TLS errors no longer
> appear (tried installing everything two times now; without reverts
> downloading enlightenment needed retrys everytime).
> […] I suspect restarting the
> install causes grub-install to fail because it cannot find /boot/efi
> anymore, but maybe my grub-install error had other reasons.

No sorry, grub-install still fails, even with no TLS errors no prior
installation restarts.  grub-install failing is an unrelated bug.  I
had not let the install run past the “copying everthing to /mnt” step
last time.


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-19  8:05         ` pelzflorian (Florian Pelz)
  2021-04-19  8:22           ` pelzflorian (Florian Pelz)
@ 2021-04-19 11:02           ` pelzflorian (Florian Pelz)
  2021-04-20  1:21           ` pelzflorian (Florian Pelz)
  2 siblings, 0 replies; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-19 11:02 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel, jcguu95, 47867

On Mon, Apr 19, 2021 at 10:05:25AM +0200, pelzflorian (Florian Pelz) wrote:
> I will bisect which reverts are needed, but it takes time …

My UEFI broke; the PC cannot display anything anymore.  Don’t wait for
my debugging.


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-19  8:05         ` pelzflorian (Florian Pelz)
  2021-04-19  8:22           ` pelzflorian (Florian Pelz)
  2021-04-19 11:02           ` pelzflorian (Florian Pelz)
@ 2021-04-20  1:21           ` pelzflorian (Florian Pelz)
  2021-04-20 15:27             ` pelzflorian (Florian Pelz)
  2 siblings, 1 reply; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-20  1:21 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel, jcguu95, 47867

After flashing the CMOS I can test again.

Without reverts TLS errors happen while or after downloading the
enlightenment substitute from the installer.

With all of those reverts *no* TLS errors happen:

On Mon, Apr 19, 2021 at 10:05:25AM +0200, pelzflorian (Florian Pelz) wrote:
> git revert 2d73086262e1fb33cd0f0f16f74a495fe06b38aa
> git revert 673e5276f6b4dda4bfa9dd5bb70220fc8b17abd2
> git revert 45fce38fb0b6c6796906149ade145b8d3594c1c6
> git revert 9da5ec7099b992a8969a17627548cd341c01bd90
> git revert c5ab78f90b111839508d0ab10c0e5ac2038002ca
> git revert b48204259aa9cad80c5b23a4060e2d796007ec7a
> git revert c37e3b92ad0334ba2fe7ee4e98631f0a4edeee21
> git revert 728c90862eb967a1679efdb69e26d7ba37cdc3a3
> git revert fd5b77503e852b78a43e1bee4d6bdfbbb1f27e8f
> git revert 112692c0d546d35cd67c5dc70dbd1dc609b18f64
> git revert ee3226e9d54891c7e696912245e4904435be191c
> git revert e2572aa95007558b008f04decff095f46d20e087
> git revert 20c08a8a45d0f137ead7c05e720456b2aea44402
> git revert 187e97096825d2bcceb144cead6eccc27385acd7
> git revert 8116cc66733134a8fb6f9117d4648288b83c8356
> git revert b9d058e3f7982eed04cc748cc0b24173a3969c52
> git revert 7c85877fdf964694061e3192eac35723ebc047bf
> git revert f50f5751fff4cfc6d5abba9681054569694b7a5c
> git revert 05f38ca8dccc49d9ec70d1f63461976b97d74d52
> git revert 7b812f7c84c43455cdd68a0e51b6ded018afcc8e
> git revert f50b501a7472f4f237023831aa415a948115d1d1
> git revert 205833b72c5517915a47a50dbe28e7024dc74e57
> git revert e2e853ddb0d964e7b0300334ab51ab89950c2376
> git revert 87734503a4a1761a4432b601352e5b02340a97f0
> git revert 08acee2f98321d57c4faa3ba641a5543b74f10a4
> git revert fbd61b5d3de353bfa468641d087bc53aaa1e63a5
> git revert be5a75ebb5988b87b2392e2113f6590f353dd6cd

The bad commit must be one of those.
c7c7f068c15e419aaf5ef616516aa5ad4e55c2fa is not the problem.

I will bisect tomorrow.

Regards,
Florian


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-20  1:21           ` pelzflorian (Florian Pelz)
@ 2021-04-20 15:27             ` pelzflorian (Florian Pelz)
  2021-04-20 17:03               ` pelzflorian (Florian Pelz)
  2021-04-20 22:38               ` Ludovic Courtès
  0 siblings, 2 replies; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-20 15:27 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel, jcguu95, 47867

On Tue, Apr 20, 2021 at 03:21:13AM +0200, pelzflorian (Florian Pelz) wrote:
> > git revert be5a75ebb5988b87b2392e2113f6590f353dd6cd

It seems this is the bad commit.  Downloading the enlightenment
substitute got stuck and after a few minutes displayed the usual TLS
error.

Regards,
Florian


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-20 15:27             ` pelzflorian (Florian Pelz)
@ 2021-04-20 17:03               ` pelzflorian (Florian Pelz)
  2021-04-20 18:00                 ` Leo Famulari
  2021-04-20 22:38               ` Ludovic Courtès
  1 sibling, 1 reply; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-20 17:03 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel, jcguu95, 47867

On Tue, Apr 20, 2021 at 05:27:54PM +0200, pelzflorian (Florian Pelz) wrote:
> On Tue, Apr 20, 2021 at 03:21:13AM +0200, pelzflorian (Florian Pelz) wrote:
> > > git revert be5a75ebb5988b87b2392e2113f6590f353dd6cd
> 
> It seems this is the bad commit.  Downloading the enlightenment
> substitute got stuck and after a few minutes displayed the usual TLS
> error.

P.S. There are no issues on my Macbook on the same internet
connection, the issue is only with the Beebox PC.

Regards,
Florian


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-20 17:03               ` pelzflorian (Florian Pelz)
@ 2021-04-20 18:00                 ` Leo Famulari
  2021-04-20 19:41                   ` pelzflorian (Florian Pelz)
  0 siblings, 1 reply; 35+ messages in thread
From: Leo Famulari @ 2021-04-20 18:00 UTC (permalink / raw)
  To: pelzflorian (Florian Pelz); +Cc: guix-devel, jcguu95, 47867

On Tue, Apr 20, 2021 at 07:03:26PM +0200, pelzflorian (Florian Pelz) wrote:
> On Tue, Apr 20, 2021 at 05:27:54PM +0200, pelzflorian (Florian Pelz) wrote:
> > It seems this is the bad commit.  Downloading the enlightenment
> > substitute got stuck and after a few minutes displayed the usual TLS
> > error.
> 
> P.S. There are no issues on my Macbook on the same internet
> connection, the issue is only with the Beebox PC.

To clarify, you're running the same Guix on both machines, but it only
fails on one of them?


^ permalink raw reply	[flat|nested] 35+ messages in thread

* bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-20 18:00                 ` Leo Famulari
@ 2021-04-20 19:41                   ` pelzflorian (Florian Pelz)
  0 siblings, 0 replies; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-20 19:41 UTC (permalink / raw)
  To: Leo Famulari; +Cc: guix-devel, jcguu95, 47867

On Tue, Apr 20, 2021 at 02:00:54PM -0400, Leo Famulari wrote:
> On Tue, Apr 20, 2021 at 07:03:26PM +0200, pelzflorian (Florian Pelz) wrote:
> > On Tue, Apr 20, 2021 at 05:27:54PM +0200, pelzflorian (Florian Pelz) wrote:
> > > It seems this is the bad commit.  Downloading the enlightenment
> > > substitute got stuck and after a few minutes displayed the usual TLS
> > > error.
> > 
> > P.S. There are no issues on my Macbook on the same internet
> > connection, the issue is only with the Beebox PC.
> 
> To clarify, you're running the same Guix on both machines, but it only
> fails on one of them?

No sorry I was mistaken, sorry for misguiding you.

I had thought so, but the actions were different on both machines.
Only full installs fail.

Doing a full install with the Enlightenment desktop environment fails
with TLS errors on the Beebox PC.

guix build enlightenment succeeds on the Macbook (with same Guix
commit), but

Running
mount /dev/sda2 /mnt
guile -c '((@ (gnu build install) mount-cow-store) "/mnt" "/tmp/guix-inst")'
guix build enlightenment
succeeds though.  (Without mount-cow-store disk space is insufficient.)

So I will try to do a full reinstall with Enlightenment on the Macbook
now after a backup.

Regards,
Florian




^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-20 15:27             ` pelzflorian (Florian Pelz)
  2021-04-20 17:03               ` pelzflorian (Florian Pelz)
@ 2021-04-20 22:38               ` Ludovic Courtès
  2021-04-21 20:17                 ` pelzflorian (Florian Pelz)
  1 sibling, 1 reply; 35+ messages in thread
From: Ludovic Courtès @ 2021-04-20 22:38 UTC (permalink / raw)
  To: pelzflorian (Florian Pelz); +Cc: guix-devel, jcguu95, 47867

[-- Attachment #1: Type: text/plain, Size: 1656 bytes --]

Hi Florian,

"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:

> On Tue, Apr 20, 2021 at 03:21:13AM +0200, pelzflorian (Florian Pelz) wrote:
>> > git revert be5a75ebb5988b87b2392e2113f6590f353dd6cd
>
> It seems this is the bad commit.  Downloading the enlightenment
> substitute got stuck and after a few minutes displayed the usual TLS
> error.

Note that on master there have been changes in this area since this
commit, in particular 20c08a8a45d0f137ead7c05e720456b2aea44402.

I assume the error we’re after is still this one:

>>    |    substitute: updating substitutes from 'https://ci.guix.gnu.org'... 0.0%guix substitute: error: TSL error in procedure 'write_to_session_port': Resource temporarily unavailable, try again.

I believe the attached patch “addresses” this problem.

Now, I’m a bit skeptical: the error above is GNUTLS_E_AGAIN, which
happens if sendmsg(2) returns EAGAIN, which supposedly only happens on
datagram (UDP) sockets or on non-blocking sockets, neither of which
applies here.

This change should be safe, though since I’m not sure why we’re getting
EAGAIN in the first place, I wonder what could happen (infinite loop?).

Could you give it a try?

Can you reproduce the substitute issue in a simpler environment?
For instance, by running:

  rm -rf ~/.cache/guix/substitute/
  ./pre-inst-env guix weather $(guix package -A |head -2000 |cut -f1) 

This makes 2000 pipelined GETs to https://ci.guix.gnu.org, which is
probably similar to what you observe during system installation.

Thanks for the thorough debugging and bisecting, as always!

Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 2170 bytes --]

diff --git a/guix/http-client.scm b/guix/http-client.scm
index a2e11a1b73..51bba250d5 100644
--- a/guix/http-client.scm
+++ b/guix/http-client.scm
@@ -38,7 +38,7 @@
   #:use-module (guix utils)
   #:use-module (guix base64)
   #:autoload   (gcrypt hash) (sha256)
-  #:autoload   (gnutls) (error/invalid-session)
+  #:autoload   (gnutls) (error/invalid-session error/again)
   #:use-module ((guix build utils)
                 #:select (mkdir-p dump-port))
   #:use-module ((guix build download)
@@ -163,7 +163,8 @@ reusing stale cached connections."
       (if (or (and (eq? key 'system-error)
                    (= EPIPE (system-error-errno `(,key ,@args))))
               (and (eq? key 'gnutls-error)
-                   (eq? (first args) error/invalid-session))
+                   (memq (first args)
+                         (list error/again error/invalid-session)))
               (memq key
                     '(bad-response bad-header bad-header-component)))
           #f
diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm
index 48309f9b3a..65940591a9 100755
--- a/guix/scripts/substitute.scm
+++ b/guix/scripts/substitute.scm
@@ -45,7 +45,7 @@
                 #:select (uri-abbreviation nar-uri-abbreviation
                           (open-connection-for-uri
                            . guix:open-connection-for-uri)))
-  #:autoload   (gnutls) (error/invalid-session)
+  #:autoload   (gnutls) (error/invalid-session error/again)
   #:use-module (guix progress)
   #:use-module ((guix build syscalls)
                 #:select (set-thread-name))
@@ -417,7 +417,8 @@ server certificates."
         (if (or (and (eq? key 'system-error)
                      (= EPIPE (system-error-errno `(,key ,@args))))
                 (and (eq? key 'gnutls-error)
-                     (eq? (first args) error/invalid-session))
+                     (memq (first args)
+                           (list error/again error/invalid-session)))
                 (memq key '(bad-response bad-header bad-header-component)))
             (proc (open-connection-for-uri/cached uri
                                                   #:verify-certificate? #f

^ permalink raw reply related	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-20 22:38               ` Ludovic Courtès
@ 2021-04-21 20:17                 ` pelzflorian (Florian Pelz)
  2021-04-23  9:19                   ` Ludovic Courtès
  0 siblings, 1 reply; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-21 20:17 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel, jcguu95, 47867

Sorry for the slow response.

On Wed, Apr 21, 2021 at 12:38:58AM +0200, Ludovic Courtès wrote:
> Hi Florian,
> 
> "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:
> > On Tue, Apr 20, 2021 at 03:21:13AM +0200, pelzflorian (Florian Pelz) wrote:
> >> > git revert be5a75ebb5988b87b2392e2113f6590f353dd6cd
> > It seems this is the bad commit.  Downloading the enlightenment
> > substitute got stuck and after a few minutes displayed the usual TLS
> > error.
> Note that on master there have been changes in this area since this
> commit, in particular 20c08a8a45d0f137ead7c05e720456b2aea44402.

I have tested 20c08a in my reverts, it is neither the bad commit nor
does it fix it.
> I assume the error we’re after is still this one:
> 
> >>    |    substitute: updating substitutes from 'https://ci.guix.gnu.org'... 0.0%guix substitute: error: TSL error in procedure 'write_to_session_port': Resource temporarily unavailable, try again.

Yes.

> I believe the attached patch “addresses” this problem.

It still gets stuck (sometimes with enlightenment, one time with
udisks, restarting the install fixed it once).  After getting stuck,
this different error message is shown now; no TLS error (copied by
manual typing, there may be typos):

gtk-doc-1.28  653KiB                2.4MiB/s 00:00 [####################] 100.0%
udisks-2.8.4  842KiB                1.6MiB/s 00:00 [####################] 100.0%

substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%Backtrace:
substitute: In ice-9/boot-9.scm:
substitute:   1736:10 17 (with-exception-handler _ _ #:unwind? _ # _)
substitute: In unknown file:
substitute:           16 (apply-smob/0 #<thunk 7fc2dbf60520>)
substitute: In ice-9/boot-9.scm:
substitute:     718:2 15 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
substitute: In ice-9/eval.scm:
substitute:     619:8 14 (_ #(#(#<directory (guile-user) 7fc2dbf63c80>)))
substitute: In guix/ui.scm:
substitute:   2164:12 13 (run-guix-command _ . _)
substitute: In ice-9/boot-9.scm:
substitute:   1736:10 12 (with-exception-handler _ _ #:unwind? _ # _)
substitute:   1736:10 11 (with-exception-handler _ _ #:unwind? _ # _)
substitute:   1731:15 10 (with-exception-handler _ _ #:unwind? _ # _)
substitute: In guix/scripts/substitute.scm:
substitute:    745:18  9 (_)
substitute:    346:26  8 (process-query #<output: file 4> _ #:cache-urls _ #:acl _)
substitute: In guix/substitutes.scm:
substitute:    358:27  7 (lookup-narinfos/diverse _ _ #<procedure 7fc2d3b15000 …>
substitute:    315:31  6 (lookup-narinfos _ _ #:open-connection _ # _)
substitute:    238:26  5 (fetch-narinfos _ _ #:open-connection _ # _)
substitute: In ice-9/boot-9.scm:
substitute:   1669:16  4 (raise-exception _ #:continuable? _)
substitute:   1669:16  3 (raise-exception _ #:continuable? _)
substitute:   1764:13  2 (_ #<&compound-exception _ components: assertion-fail…>)
substitute:   1669:16  1 (raise-exception _ #:continuable? _)
substitute:   1669:16  0 (raise-exception _ #:continuable? _)
substitute:
substitute: In ice-9/boot-9.scm:1669:16 In procedure raise-exception:
substitute: In procedure %read-line: Wrong type argument in position 1 (expecting open input port): #<closed: string 7fc2d8796070>
guix system: error: `/gnu/store/k3n98i1fk9awd5ydv4ry4k4rlpp7i13m7-guix-1.2.0-22.c467718/bin/guix substitute' died unexpectedly

Command failed with exit code 1.
Press Enter to continue.

> Can you reproduce the substitute issue in a simpler environment?
> For instance, by running:
> 
>   rm -rf ~/.cache/guix/substitute/
>   ./pre-inst-env guix weather $(guix package -A |head -2000 |cut -f1) 

Nope, this always works without issue.  Ricardo had written about TLS
errors with 0ad some two or three days ago, but I do not get errors
with 0ad.  The issue is present when installing Enlightenment DE from
the installer on both my Macbook and my Beebox every time, but not
when installing to a VM (but maybe it was just luck)

Regards,
Florian


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-21 20:17                 ` pelzflorian (Florian Pelz)
@ 2021-04-23  9:19                   ` Ludovic Courtès
  2021-04-23 14:07                     ` pelzflorian (Florian Pelz)
  0 siblings, 1 reply; 35+ messages in thread
From: Ludovic Courtès @ 2021-04-23  9:19 UTC (permalink / raw)
  To: pelzflorian (Florian Pelz); +Cc: guix-devel, jcguu95, 47867

[-- Attachment #1: Type: text/plain, Size: 3874 bytes --]

Hi Florian,

(Cc: Chris who’s also familiar with (guix http-client).)

"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:

> It still gets stuck (sometimes with enlightenment, one time with
> udisks, restarting the install fixed it once).  After getting stuck,
> this different error message is shown now; no TLS error (copied by
> manual typing, there may be typos):
>
> gtk-doc-1.28  653KiB                2.4MiB/s 00:00 [####################] 100.0%
> udisks-2.8.4  842KiB                1.6MiB/s 00:00 [####################] 100.0%
>
> substitute: updating substitutes from 'https://ci.guix.gnu.org'... 100.0%Backtrace:
> substitute: In ice-9/boot-9.scm:
> substitute:   1736:10 17 (with-exception-handler _ _ #:unwind? _ # _)
> substitute: In unknown file:
> substitute:           16 (apply-smob/0 #<thunk 7fc2dbf60520>)
> substitute: In ice-9/boot-9.scm:
> substitute:     718:2 15 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
> substitute: In ice-9/eval.scm:
> substitute:     619:8 14 (_ #(#(#<directory (guile-user) 7fc2dbf63c80>)))
> substitute: In guix/ui.scm:
> substitute:   2164:12 13 (run-guix-command _ . _)
> substitute: In ice-9/boot-9.scm:
> substitute:   1736:10 12 (with-exception-handler _ _ #:unwind? _ # _)
> substitute:   1736:10 11 (with-exception-handler _ _ #:unwind? _ # _)
> substitute:   1731:15 10 (with-exception-handler _ _ #:unwind? _ # _)
> substitute: In guix/scripts/substitute.scm:
> substitute:    745:18  9 (_)
> substitute:    346:26  8 (process-query #<output: file 4> _ #:cache-urls _ #:acl _)
> substitute: In guix/substitutes.scm:
> substitute:    358:27  7 (lookup-narinfos/diverse _ _ #<procedure 7fc2d3b15000 …>
> substitute:    315:31  6 (lookup-narinfos _ _ #:open-connection _ # _)
> substitute:    238:26  5 (fetch-narinfos _ _ #:open-connection _ # _)
> substitute: In ice-9/boot-9.scm:
> substitute:   1669:16  4 (raise-exception _ #:continuable? _)
> substitute:   1669:16  3 (raise-exception _ #:continuable? _)
> substitute:   1764:13  2 (_ #<&compound-exception _ components: assertion-fail…>)
> substitute:   1669:16  1 (raise-exception _ #:continuable? _)
> substitute:   1669:16  0 (raise-exception _ #:continuable? _)
> substitute:
> substitute: In ice-9/boot-9.scm:1669:16 In procedure raise-exception:
> substitute: In procedure %read-line: Wrong type argument in position 1 (expecting open input port): #<closed: string 7fc2d8796070>
> guix system: error: `/gnu/store/k3n98i1fk9awd5ydv4ry4k4rlpp7i13m7-guix-1.2.0-22.c467718/bin/guix substitute' died unexpectedly

I think I got it: commit 205833b72c5517915a47a50dbe28e7024dc74e57 (then
carried over in 45fce38fb0b6c6796906149ade145b8d3594c1c6) introduced a
call to ‘connect’ in non-tail position.  Once that recursive call to
‘connect’ had completed, ‘http-multiple-get’ would go on in ‘loop’
trying to re-process responses, but at that point there aren’t any
responses left to process.

This problem could only happen if a networking exception would occur
while sending HTTP requests for narinfos.  Thus, it was most likely to
occur when interleaving substitutions and queries, as in the snippet you
provided above, because then ‘http-multiple-get’ was more likely to be
passed a stale reused connection.

Florian, could you try again with the attached patch?

If you have the courage, it would be awesome if you could also try the
patch without the ‘error/again’ bits.  It’s possible that they aren’t
needed now.  I double-checked and the GnuTLS Guile bindings already
handle GNUTLS_E_AGAIN and GNUTLS_E_INTERRUPTED, so my guess is that this
was just a side effect of dealing with stale TLS sessions:

  https://gitlab.com/gnutls/gnutls/-/blob/master/guile/src/core.c#L1042

Thanks a lot for your help!

Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 3201 bytes --]

diff --git a/guix/http-client.scm b/guix/http-client.scm
index a2e11a1b73..b9cf0b1a4b 100644
--- a/guix/http-client.scm
+++ b/guix/http-client.scm
@@ -38,7 +38,7 @@
   #:use-module (guix utils)
   #:use-module (guix base64)
   #:autoload   (gcrypt hash) (sha256)
-  #:autoload   (gnutls) (error/invalid-session)
+  #:autoload   (gnutls) (error/invalid-session error/again)
   #:use-module ((guix build utils)
                 #:select (mkdir-p dump-port))
   #:use-module ((guix build download)
@@ -163,7 +163,8 @@ reusing stale cached connections."
       (if (or (and (eq? key 'system-error)
                    (= EPIPE (system-error-errno `(,key ,@args))))
               (and (eq? key 'gnutls-error)
-                   (eq? (first args) error/invalid-session))
+                   (memq (first args)
+                         (list error/again error/invalid-session)))
               (memq key
                     '(bad-response bad-header bad-header-component)))
           #f
@@ -207,15 +208,14 @@ returning."
         ;; Inherit the HTTP proxying property from P.
         (set-http-proxy-port?! buffer (http-proxy-port? p))
 
-        (unless (false-if-networking-error
-                 (begin
-                   (for-each (cut write-request <> buffer) batch)
-                   (put-bytevector p (get))
-                   (force-output p)
-                   #t))
-          ;; If PORT becomes unusable, open a fresh connection and retry.
-          (close-port p)                          ; close the broken port
-          (connect #f requests result)))
+        ;; Swallow networking errors that could occur due to connection reuse
+        ;; and the like; they will be handled down the road when trying to
+        ;; read responses.
+        (false-if-networking-error
+         (begin
+           (for-each (cut write-request <> buffer) batch)
+           (put-bytevector p (get))
+           (force-output p))))
 
       ;; Now start processing responses.
       (let loop ((sent      batch)
diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm
index 48309f9b3a..65940591a9 100755
--- a/guix/scripts/substitute.scm
+++ b/guix/scripts/substitute.scm
@@ -45,7 +45,7 @@
                 #:select (uri-abbreviation nar-uri-abbreviation
                           (open-connection-for-uri
                            . guix:open-connection-for-uri)))
-  #:autoload   (gnutls) (error/invalid-session)
+  #:autoload   (gnutls) (error/invalid-session error/again)
   #:use-module (guix progress)
   #:use-module ((guix build syscalls)
                 #:select (set-thread-name))
@@ -417,7 +417,8 @@ server certificates."
         (if (or (and (eq? key 'system-error)
                      (= EPIPE (system-error-errno `(,key ,@args))))
                 (and (eq? key 'gnutls-error)
-                     (eq? (first args) error/invalid-session))
+                     (memq (first args)
+                           (list error/again error/invalid-session)))
                 (memq key '(bad-response bad-header bad-header-component)))
             (proc (open-connection-for-uri/cached uri
                                                   #:verify-certificate? #f

^ permalink raw reply related	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-23  9:19                   ` Ludovic Courtès
@ 2021-04-23 14:07                     ` pelzflorian (Florian Pelz)
  2021-04-24 17:13                       ` Ludovic Courtès
  0 siblings, 1 reply; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-23 14:07 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel, jcguu95, 47867

Success!  Thank you.
65;6003;1c
On Fri, Apr 23, 2021 at 11:19:28AM +0200, Ludovic Courtès wrote:
> Florian, could you try again with the attached patch?

It succeeds on two full installs of Enlightenment, no errors, no
prolonged getting stuck.


> If you have the courage, it would be awesome if you could also try the
> patch without the ‘error/again’ bits.

This fails.  I tried with

-        (unless (false-if-networking-error
-                 (begin
-                   (for-each (cut write-request <> buffer) batch)
-                   (put-bytevector p (get))
-                   (force-output p)
-                   #t))
-          ;; If PORT becomes unusable, open a fresh connection and retry.
-          (close-port p)                          ; close the broken port
-          (connect #f requests result)))
+        ;; Swallow networking errors that could occur due to connection reuse
+        ;; and the like; they will be handled down the road when trying to
+        ;; read responses.
+        (false-if-networking-error
+         (begin
+           (for-each (cut write-request <> buffer) batch)
+           (put-bytevector p (get))
+           (force-output p))))


only and not the rest of your patch, on Guix git master where the full
patch had worked, it fails again with TLS errors (the same error of
Resource temporarily unavailable in procedure
'write_to_session_record_port') after downloading the enlightenment
substitute.


> I double-checked and the GnuTLS Guile bindings already
> handle GNUTLS_E_AGAIN and GNUTLS_E_INTERRUPTED, so my guess is that this
> was just a side effect of dealing with stale TLS sessions:
>   https://gitlab.com/gnutls/gnutls/-/blob/master/guile/src/core.c#L1042

Strange,.

Regards,
Florian


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-23 14:07                     ` pelzflorian (Florian Pelz)
@ 2021-04-24 17:13                       ` Ludovic Courtès
  2021-04-25  0:20                         ` pelzflorian (Florian Pelz)
  2021-06-21 19:18                         ` Ludovic Courtès
  0 siblings, 2 replies; 35+ messages in thread
From: Ludovic Courtès @ 2021-04-24 17:13 UTC (permalink / raw)
  To: pelzflorian (Florian Pelz); +Cc: guix-devel, jcguu95, 47867

Hi Florian,

"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:

> Success!  Thank you.

\o/

> On Fri, Apr 23, 2021 at 11:19:28AM +0200, Ludovic Courtès wrote:
>> Florian, could you try again with the attached patch?
>
> It succeeds on two full installs of Enlightenment, no errors, no
> prolonged getting stuck.

Yay.

>> I double-checked and the GnuTLS Guile bindings already
>> handle GNUTLS_E_AGAIN and GNUTLS_E_INTERRUPTED, so my guess is that this
>> was just a side effect of dealing with stale TLS sessions:
>>   https://gitlab.com/gnutls/gnutls/-/blob/master/guile/src/core.c#L1042
>
> Strange,.

My bad: I was looking at ‘read_from_session_record_port’, but the
problem is in ‘write_to_session_record_port’.  All is clear now.

I’ve pushed these as two separate patches:

  c50db7156d http-client: Remove exception mishandling in 'http-multiple-get'.
  02d62978f4 http-client, substitute: Gracefully handle GnuTLS EAGAIN/EINTR.

This bug should be gone now.

I’ll go ahead and fix ‘write_to_session_record_port’ in GnuTLS.

Thanks a lot for the quick and thorough testing!

Ludo’.


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-24 17:13                       ` Ludovic Courtès
@ 2021-04-25  0:20                         ` pelzflorian (Florian Pelz)
  2021-06-21 19:18                         ` Ludovic Courtès
  1 sibling, 0 replies; 35+ messages in thread
From: pelzflorian (Florian Pelz) @ 2021-04-25  0:20 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel, 47867-done, jcguu95

On Sat, Apr 24, 2021 at 07:13:01PM +0200, Ludovic Courtès wrote:
> I’ve pushed these as two separate patches:
> 
>   c50db7156d http-client: Remove exception mishandling in 'http-multiple-get'.
>   02d62978f4 http-client, substitute: Gracefully handle GnuTLS EAGAIN/EINTR.

I assume they will be cherry-picked to the version-1.3.0 branch and
the guix package will be updated.

> This bug should be gone now.

Yes, after and only after locally updating the guix package I can
create good installer images.

Closing.

Note though that "Guu, Jin-Cheng" <jcguu95@gmail.com> reported the TLS
errors before me, but to guix-devel and not as a bug report.

> I’ll go ahead and fix ‘write_to_session_record_port’ in GnuTLS.

Thank you for all the work!

Regards,
Florian


^ permalink raw reply	[flat|nested] 35+ messages in thread

* Re: bug#47867: [1.2.1 pre-release testing] substitute downloading and TLS errors
  2021-04-24 17:13                       ` Ludovic Courtès
  2021-04-25  0:20                         ` pelzflorian (Florian Pelz)
@ 2021-06-21 19:18                         ` Ludovic Courtès
  1 sibling, 0 replies; 35+ messages in thread
From: Ludovic Courtès @ 2021-06-21 19:18 UTC (permalink / raw)
  To: pelzflorian (Florian Pelz); +Cc: guix-devel, jcguu95, 47867

Hi,

Ludovic Courtès <ludo@gnu.org> skribis:

>>> I double-checked and the GnuTLS Guile bindings already
>>> handle GNUTLS_E_AGAIN and GNUTLS_E_INTERRUPTED, so my guess is that this
>>> was just a side effect of dealing with stale TLS sessions:
>>>   https://gitlab.com/gnutls/gnutls/-/blob/master/guile/src/core.c#L1042
>>
>> Strange,.
>
> My bad: I was looking at ‘read_from_session_record_port’, but the
> problem is in ‘write_to_session_record_port’.  All is clear now.
>
> I’ve pushed these as two separate patches:
>
>   c50db7156d http-client: Remove exception mishandling in 'http-multiple-get'.
>   02d62978f4 http-client, substitute: Gracefully handle GnuTLS EAGAIN/EINTR.
>
> This bug should be gone now.
>
> I’ll go ahead and fix ‘write_to_session_record_port’ in GnuTLS.

It took a while but this is now fixed in GnuTLS:

  https://gitlab.com/gnutls/gnutls/-/merge_requests/1417

I *think* the fix will be included in 3.7.3 (it’s not in 3.7.2, released
a couple of weeks ago).

Ludo’.


^ permalink raw reply	[flat|nested] 35+ messages in thread

end of thread, other threads:[~2021-06-21 19:18 UTC | newest]

Thread overview: 35+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-04-17  3:40 1.2.1 pre-release testing jcguu95
2021-04-17  5:43 ` Leo Famulari
2021-04-17 11:35   ` GIT_EXEC_PATH " François
2021-04-17 17:24     ` Leo Famulari
2021-04-17 18:07       ` François
2021-04-17  9:32 ` pelzflorian (Florian Pelz)
2021-04-17 12:58   ` Guu, Jin-Cheng
2021-04-17 13:17     ` pelzflorian (Florian Pelz)
2021-04-17 17:27 ` Leo Famulari
2021-04-17 21:57   ` pelzflorian (Florian Pelz)
2021-04-18  0:09     ` Guu, Jin-Cheng
2021-04-18  0:19       ` Leo Famulari
2021-04-18  8:57       ` pelzflorian (Florian Pelz)
2021-04-18  9:38 ` [1.2.1 pre-release testing] substitute downloading and TLS errors pelzflorian (Florian Pelz)
2021-04-18  9:44   ` bug#47867: " pelzflorian (Florian Pelz)
2021-04-18  9:48     ` pelzflorian (Florian Pelz)
2021-04-18  9:57       ` pelzflorian (Florian Pelz)
2021-04-18 10:15   ` Ludovic Courtès
2021-04-18 11:33     ` pelzflorian (Florian Pelz)
2021-04-18 12:10       ` pelzflorian (Florian Pelz)
2021-04-19  8:05         ` pelzflorian (Florian Pelz)
2021-04-19  8:22           ` pelzflorian (Florian Pelz)
2021-04-19 11:02           ` pelzflorian (Florian Pelz)
2021-04-20  1:21           ` pelzflorian (Florian Pelz)
2021-04-20 15:27             ` pelzflorian (Florian Pelz)
2021-04-20 17:03               ` pelzflorian (Florian Pelz)
2021-04-20 18:00                 ` Leo Famulari
2021-04-20 19:41                   ` pelzflorian (Florian Pelz)
2021-04-20 22:38               ` Ludovic Courtès
2021-04-21 20:17                 ` pelzflorian (Florian Pelz)
2021-04-23  9:19                   ` Ludovic Courtès
2021-04-23 14:07                     ` pelzflorian (Florian Pelz)
2021-04-24 17:13                       ` Ludovic Courtès
2021-04-25  0:20                         ` pelzflorian (Florian Pelz)
2021-06-21 19:18                         ` Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).