From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:8:6d80::]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 0AQfL0huXGCa9wAAgWs5BA (envelope-from ) for ; Thu, 25 Mar 2021 12:04:40 +0100 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id UIWvKkhuXGDKQgAAbx9fmQ (envelope-from ) for ; Thu, 25 Mar 2021 11:04:40 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 7DF6C1AAB7 for ; Thu, 25 Mar 2021 12:04:40 +0100 (CET) Received: from localhost ([::1]:55382 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lPNn5-0005D5-Ka for larch@yhetil.org; Thu, 25 Mar 2021 07:04:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56470) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lPNmU-0004kQ-Uj for guix-patches@gnu.org; Thu, 25 Mar 2021 07:04:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:54015) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lPNmU-0007vs-Lc for guix-patches@gnu.org; Thu, 25 Mar 2021 07:04:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lPNmU-0001Vf-Hj for guix-patches@gnu.org; Thu, 25 Mar 2021 07:04:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#47288] [PATCH v3 1/2] guix: http-client: Tweak http-multiple-get error handling. References: <20210321004338.31867-1-mail@cbaines.net> In-Reply-To: <20210321004338.31867-1-mail@cbaines.net> Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 25 Mar 2021 11:04:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 47288 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 47288@debbugs.gnu.org Received: via spool by 47288-submit@debbugs.gnu.org id=B47288.16166701995738 (code B ref 47288); Thu, 25 Mar 2021 11:04:02 +0000 Received: (at 47288) by debbugs.gnu.org; 25 Mar 2021 11:03:19 +0000 Received: from localhost ([127.0.0.1]:37326 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lPNln-0001UT-7o for submit@debbugs.gnu.org; Thu, 25 Mar 2021 07:03:19 -0400 Received: from mira.cbaines.net ([212.71.252.8]:50966) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lPNll-0001UF-Ox for 47288@debbugs.gnu.org; Thu, 25 Mar 2021 07:03:18 -0400 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id CA5CD27BC5C for <47288@debbugs.gnu.org>; Thu, 25 Mar 2021 11:03:16 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 8a272564 for <47288@debbugs.gnu.org>; Thu, 25 Mar 2021 11:03:16 +0000 (UTC) From: Christopher Baines Date: Thu, 25 Mar 2021 11:03:15 +0000 Message-Id: <20210325110316.862-1-mail@cbaines.net> X-Mailer: git-send-email 2.30.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1616670280; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=8z2T3BHf+kpRY6Io02vkUAWl+pH0R+52W2ERSK2ujv8=; b=BknekjR98h3uwNjAzuVXyRseGDaF5wDYvPgQdSBpds6EmoBFX1nV7PoUjEeqhI6W/Exws5 eKwOTW0RqZ7ReihffFUmuzMYIocTX7E+puhWPWzOLqnCWpjVl7wdST0iGYO0QSRy/USGnv UZVfQa3wkCucooIK21ugQr2SWrr1aQiuc3uJhg1osLw1kSsdpctEglralJcJOqoIl449y+ ub6sfu4G49b1DEdVVmZQoW+qtjmZzDCraKY2qONbBXrtachLRCxKyt8qJ9bk3UGskhTsO5 8LtHdORCkURYysMFenDXYv/5Z8AFchV/pmPyQjSYvJgeKqn7qIzrkZcO1RFUZA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1616670280; a=rsa-sha256; cv=none; b=jok2Mt8UOrRJqw0VeW6+bcgkiIyTTYTmXiI+AMvACyLctuhnSVLOrW0vluuEFxyLyF88K9 xyJbiou+emYFFL1ugxyNNhijf8ahdnPc20bzHL38R0o0ir/vyeN4U0Y1doDIFqsU9cqa1v yE5ZrqgESU844HD3ULcpWqjdY2bEYXQoX+VGkeT3VaM3s8j8Mhen9fff/7RuAEy+bTKbAi jcpzJ1NSnLspRx3oPOYERfWHORt6/h8DS2CAAW737IjSbUvaVc4aCRX88MhZ1qO2ltKpUa edCO5Zp6YHfBf58kkX2ogkdc5kMVXPrxLgHSvxK5VROYtb7IrLmKBvuNkPsBJQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: 2.58 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: 7DF6C1AAB7 X-Spam-Score: 2.58 X-Migadu-Scanner: scn0.migadu.com X-TUID: ylAMna0tEQlN This isn't meant to change the way errors are handled, and arguably makes the code harder to read, but it's a uninformed attempt to improve the performance (following on from a performance regression in 205833b72c5517915a47a50dbe28e7024dc74e57). I'm guessing something about Guile internals makes calling (loop ...) within the catch bit less performant than avoiding this and calling (loop ...) after the catch bit has finished. Since this happens lots, this seems to be sufficient to make guix weather a lot slower than it was before. Anecdotal testing of guix weather suggests this change might work. * guix/http-client.scm (http-multiple-get): Tweak how the second catch statement works. --- guix/http-client.scm | 77 +++++++++++++++++++++++++------------------- 1 file changed, 43 insertions(+), 34 deletions(-) diff --git a/guix/http-client.scm b/guix/http-client.scm index 4b4c14ed0b..adbfbc0d6e 100644 --- a/guix/http-client.scm +++ b/guix/http-client.scm @@ -219,42 +219,51 @@ returning." (remainder (connect p remainder result)))) ((head tail ...) - (catch #t - (lambda () - (let* ((resp (read-response p)) - (body (response-body-port resp)) - (result (proc head resp body result))) - ;; The server can choose to stop responding at any time, - ;; in which case we have to try again. Check whether - ;; that is the case. Note that even upon "Connection: - ;; close", we can read from BODY. - (match (assq 'connection (response-headers resp)) - (('connection 'close) - (close-port p) - (connect #f ;try again - (drop requests (+ 1 processed)) - result)) - (_ - (loop tail (+ 1 processed) result))))) ;keep going - (lambda (key . args) - ;; If PORT was cached and the server closed the connection - ;; in the meantime, we get EPIPE. In that case, open a - ;; fresh connection and retry. We might also get - ;; 'bad-response or a similar exception from (web response) - ;; later on, once we've sent the request, or a - ;; ERROR/INVALID-SESSION from GnuTLS. - (if (or (and (eq? key 'system-error) - (= EPIPE (system-error-errno `(,key ,@args)))) - (and (eq? key 'gnutls-error) - (eq? (first args) error/invalid-session)) - (memq key - '(bad-response bad-header bad-header-component))) - (begin - (close-port p) - (connect #f ; try again + (match + (catch #t + (lambda () + (let* ((resp (read-response p)) + (body (response-body-port resp)) + (result (proc head resp body result))) + ;; The server can choose to stop responding at any time, + ;; in which case we have to try again. Check whether + ;; that is the case. Note that even upon "Connection: + ;; close", we can read from BODY. + (match (assq 'connection (response-headers resp)) + (('connection 'close) + (close-port p) + (list 'connect + #f (drop requests (+ 1 processed)) result)) - (apply throw key args)))))))))) + (_ + (list 'loop tail (+ 1 processed) result))))) + (lambda (key . args) + ;; If PORT was cached and the server closed the connection + ;; in the meantime, we get EPIPE. In that case, open a + ;; fresh connection and retry. We might also get + ;; 'bad-response or a similar exception from (web response) + ;; later on, once we've sent the request, or a + ;; ERROR/INVALID-SESSION from GnuTLS. + (if (or (and (eq? key 'system-error) + (= EPIPE (system-error-errno `(,key ,@args)))) + (and (eq? key 'gnutls-error) + (eq? (first args) error/invalid-session)) + (memq key + '(bad-response + bad-header + bad-header-component))) + (begin + (close-port p) + (list 'connect + #f + (drop requests processed) + result)) + (apply throw key args)))) + (('connect . args) + (apply connect args)) + (('loop . args) + (apply loop args))))))))) ;;; -- 2.30.1