From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 6N9SCLdP22DyBQAAgWs5BA (envelope-from ) for ; Tue, 29 Jun 2021 18:52:07 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id cKX6A7dP22BwSwAAB5/wlQ (envelope-from ) for ; Tue, 29 Jun 2021 16:52:07 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 488AB24E0E for ; Tue, 29 Jun 2021 18:52:06 +0200 (CEST) Received: from localhost ([::1]:39470 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lyGxv-0001P4-PV for larch@yhetil.org; Tue, 29 Jun 2021 12:52:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:32924) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lyGvy-0007FE-Bk for bug-guix@gnu.org; Tue, 29 Jun 2021 12:50:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:43650) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lyGvy-0002Bw-3q for bug-guix@gnu.org; Tue, 29 Jun 2021 12:50:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lyGvx-00025L-Sp for bug-guix@gnu.org; Tue, 29 Jun 2021 12:50:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#48468: substitute server connection timeout Resent-From: Mathieu Othacehe Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 29 Jun 2021 16:50:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 48468 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 48468-submit@debbugs.gnu.org id=B48468.16249853777979 (code B ref 48468); Tue, 29 Jun 2021 16:50:01 +0000 Received: (at 48468) by debbugs.gnu.org; 29 Jun 2021 16:49:37 +0000 Received: from localhost ([127.0.0.1]:55196 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lyGvY-00024d-RT for submit@debbugs.gnu.org; Tue, 29 Jun 2021 12:49:37 -0400 Received: from eggs.gnu.org ([209.51.188.92]:34882) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lyGvY-00024S-3x for 48468@debbugs.gnu.org; Tue, 29 Jun 2021 12:49:36 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:34964) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lyGvQ-0001sO-LC; Tue, 29 Jun 2021 12:49:29 -0400 Received: from [2a01:e0a:19b:d9a0:f2f7:a404:c3d3:f8b4] (port=51100 helo=meije) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lyGvQ-0002tg-DC; Tue, 29 Jun 2021 12:49:28 -0400 From: Mathieu Othacehe References: <87lf8e4l42.fsf@gnu.org> <87im2bfj3i.fsf@gnu.org> Date: Tue, 29 Jun 2021 18:49:26 +0200 In-Reply-To: <87im2bfj3i.fsf@gnu.org> ("Ludovic =?UTF-8?Q?Court=C3=A8s?="'s message of "Fri, 18 Jun 2021 14:33:21 +0200") Message-ID: <87sg108vl5.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 48468@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1624985526; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=2KHYUmbG9I1UxlTtl2r95FGnNfMf1O2EoRWL9dxTzOo=; b=hvizWRY8UWnKWvDl1HJbOLfx243Q8lhGS7DkHwzyOP2uyWa656FIdL7uoQ68JnQXGkYBcT 0kxoJHcwew11i4sskZ98AB8sTQG1grKiWATFiku+/QRJR7txgOiI7nn8bAFmJVB0Q42U/X 99AP6dLHyVRAJ7OnZkOCdQdixqSeF6/Wl3QgkuE3kHFYMvJeCejsL01iV7TjnlMka+nqhz OKixEJIrAEsH7FIHwodDFp+Kd8Anw0M4StvnvQPaOrotQlQxhVwnQk66zkjC6PRO4wEega THwjkS/RUnBoaezGzukNDFU+7N3b5+mYEkHynwF42TOlD5sJLbwN4XmMOxgELQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1624985526; a=rsa-sha256; cv=none; b=p/Z8CJ+Ihf6Q+hv3QiaWFyq6u994LV+6y9z030KXnHE3mjm7VeOI5UnLdBkbcbdgZmslLt L5Yfmpe1ZG0w2x95zGPRvJ7Wmm2cTzySmiH6SKh6jLlnhc9ANDNRsyuK/U4gDBHzWLJbgw rsLg65QjoWFiaonLjaKbGxSu5PTcBc4MIMW14XVQE6ptGKMEh8COAIwpyd+Ixb0BtZ4bgk z/Eb7QH3/YHbBH+3Onv2gdYpSGscMkkLNDOYgHrhh4lygBBWTxMlL/EnkW76IHuM6a2GZd 6zJzdJXMqOoU+RfA0kPbstOWFdYR63j9LJ1cV+lk2sgKMXg4zctgpiKgpUXlBg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Spam-Score: -2.92 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Queue-Id: 488AB24E0E X-Spam-Score: -2.92 X-Migadu-Scanner: scn1.migadu.com X-TUID: rzXC45FWHE3u Hey, > But maybe I=E2=80=99m not looking at the right thing. > > Do you have evidence or a reproducer? Yes, adding the following debug message here: --8<---------------cut here---------------start------------->8--- --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -416,6 +416,7 @@ server certificates." ;; and retry. We might also get 'bad-response or a similar ;; exception from (web response) later on, once we've sent the ;; request, or a ERROR/INVALID-SESSION from GnuTLS. + (pk key args) (if (or (and (eq? key 'system-error) (=3D EPIPE (system-error-errno `(,key ,@args)))) (and (eq? key 'gnutls-error) --8<---------------cut here---------------end--------------->8--- and using substitutes from a publish server without keep alive support (before 0b8fa24), prints something like: --8<---------------cut here---------------start------------->8--- downloading from http://192.168.1.51:8080/nar/gzip/lkmiyfzi1bq571yn4gypbcs5= vn4fpma7-texlive-cm-51265 ... texlive-cm-51265 = = = 7.8MiB/s 00:00 | 1.7MiB transferred substituting /gnu/store/l788x07ska5vffayz0gayv4hsx5flxal-module-import-comp= iled... ;;; (bad-header (read-header-line #)) downloading from http://192.168.1.51:8080/nar/gzip/l788x07ska5vffayz0gayv4h= sx5flxal-module-import-compiled ... module-import-compiled = = = 7.5MiB/s 00:00 | 85KiB transferred substituting /gnu/store/1s1lrnxlkjwxshk5q2w97ig3clc6n06f-ruby-2.6.5... ;;; (bad-header (read-header-line #)) --8<---------------cut here---------------end--------------->8--- That's because the connection is closed by the publish server each time a NAR is sent. This particular behaviour is fixed by: https://issues.guix.gnu.org/48556. I hoped that it would decrease the load of the Cuirass publish server that was flooded by worker connections. While the situation is better, there are still a lot of substitute timeout errors on Cuirass. I have also observed those timeouts using the publish server behind ci.guix.gnu.org. This makes me think that there's something else that is preventing the server from honoring every connection request in less than 5 seconds (the timeout duration). Thanks, Mathieu