From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 2AhXE87sh2HPQgEAgWs5BA (envelope-from ) for ; Sun, 07 Nov 2021 16:12:14 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id 0JnhDs7sh2GDBwAAbx9fmQ (envelope-from ) for ; Sun, 07 Nov 2021 15:12:14 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 0BFB115195 for ; Sun, 7 Nov 2021 16:12:14 +0100 (CET) Received: from localhost ([::1]:41198 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mjjq9-0000hZ-5t for larch@yhetil.org; Sun, 07 Nov 2021 10:12:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:40792) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mjjpy-0000hQ-Mv for bug-guix@gnu.org; Sun, 07 Nov 2021 10:12:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:42783) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mjjpy-0007sB-ET for bug-guix@gnu.org; Sun, 07 Nov 2021 10:12:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mjjpy-0000Oy-4S for bug-guix@gnu.org; Sun, 07 Nov 2021 10:12:02 -0500 X-Loop: help-debbugs@gnu.org Subject: bug#51472: substitute servers should be preferred according to their coverage rate Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sun, 07 Nov 2021 15:12:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 51472 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Maxim Cournoyer Received: via spool by 51472-submit@debbugs.gnu.org id=B51472.16362978781480 (code B ref 51472); Sun, 07 Nov 2021 15:12:02 +0000 Received: (at 51472) by debbugs.gnu.org; 7 Nov 2021 15:11:18 +0000 Received: from localhost ([127.0.0.1]:54328 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mjjpF-0000Nn-Hb for submit@debbugs.gnu.org; Sun, 07 Nov 2021 10:11:17 -0500 Received: from eggs.gnu.org ([209.51.188.92]:42824) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mjjpC-0000NV-Og for 51472@debbugs.gnu.org; Sun, 07 Nov 2021 10:11:16 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:47078) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mjjp7-0007fC-I1; Sun, 07 Nov 2021 10:11:09 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:In-Reply-To:Date:References:Subject:To: From; bh=nJSxpCipyFyDxVuK2ZXAJmhQPidnqevm8rVWfwgF+40=; b=I3XIZjXC3ifYv6+AOefZ +zhysSKzVPLzTHGC+cdDmhk1iWaMcYJuBWAzJJ45NWuwSYgvOIoooCqcZhZGX+eFfH937+yRC/3aG zZfV3UtymCIt5jqFdB5GW0lWlFp4U8Svsl4rqO8KnZOhf+998dQWZOOBuENxYauceQZleJEeTgwjH pPZruTqSKnVUBL3UnoZWV1ZcXWq0bVTOvX7qEY3ngSwGtMxcwrSRF8sx7PrlEMi38Bh6Dd4oL0yuh 5LnwRGbRW5CCB6/G0eUk/cdqcZfXviPsvK1ik8QTB1cGkGPzwhIHdhMcZ40yS2Z56JIMCr5Ht8oSv mhQ0Y/L9QFSfDg==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:55538 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mjjp5-0006mg-0v; Sun, 07 Nov 2021 10:11:09 -0500 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <87k0hwa8ic.fsf@gmail.com> Date: Sun, 07 Nov 2021 16:11:04 +0100 In-Reply-To: <87k0hwa8ic.fsf@gmail.com> (Maxim Cournoyer's message of "Thu, 28 Oct 2021 22:07:39 -0400") Message-ID: <874k8oknlj.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 51472@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1636297934; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=nJSxpCipyFyDxVuK2ZXAJmhQPidnqevm8rVWfwgF+40=; b=S7bBsCjwVST6XGAKcEEZWavL3Znps43Xxqmu6OMXbfn5XcCjtxPDnzciDMM7epbvM7V4CY f6kpVQ3MgeypTrxgRlfUnlbTRlCFvsmHAADknw96BiRnIhatV5atRgEWvAG9flbq4NKo4d YdMujD4FIqYXEIBE/lcbLO/tIy4lbCPKDwys0hcYET7uv8yHRihE2ECuhaGrQ7f7Rw+mBR SmD0bPiz/d5sg8Qf84LeSnHmrYk19su9jCB78hqZuxX0HRybmJvLmWopv/BUnFPPvBHgcn IzXEzwLHMd+g2D5bMZJ/qKqzah5DKtgXsnWpWRvizvKu/J007nJU4t65ep0UJA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1636297934; a=rsa-sha256; cv=none; b=N58P9bYbblvHSQzHKI256aPVFcdNXV2LFaO0ZC84o0NyMcA+6KO8ELE02Uggugl90JCLfh NKwiH6Xi17iz9kamiUtdCoBlxN+7GFYJFNKQg/PIsM2th33uFZU9/YY6UbB2kB/eyHyKjo BlgDqHnjGnD9sCjq87iEFKEeuFfFbR0XOwQlDnRAldv4J91dsMjIpelA/19zx4wXjsfoUk I08at1nX2Ij9bTXrQXFinnV9VWho6vOAEQzDHIAmoT/3SZzVebe+axJuf1ox50rFnfsAkw ym63lmHQaeYzgolRoUHfilrFbr1Wy/hzL29n6UvBcQ3V0XmR4aYeqw+Sr5PXLQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gnu.org header.s=fencepost-gnu-org header.b=I3XIZjXC; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Spam-Score: -2.92 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gnu.org header.s=fencepost-gnu-org header.b=I3XIZjXC; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Queue-Id: 0BFB115195 X-Spam-Score: -2.92 X-Migadu-Scanner: scn0.migadu.com X-TUID: bk5gS06BiL4U Hi, Maxim Cournoyer skribis: > When using substitute servers discovery, I've noticed that if one of the > substitute servers doesn't have any substitutes available, it'll keep > getting tried instead of others, leading to a slide-show of substitutes > updates such as: > > normalized load on machine '127.0.0.1' is 0.04 > building /gnu/store/ajd0hx104702jpz2ycdwgrnyrv8jsp6d-xorg-server-21.1.0.t= ar.xz.drv... > process 9195 acquired build slot '/var/guix/offload/127.0.0.1:6666/1' > normalized load on machine '127.0.0.1' is 0.04 > building /gnu/store/49rqi3wpvdm5pv6in9pamzdvg0wscrl8-xorgproto-2021.5.drv= ... > substitute: updating substitutes from 'http://192.168.10.102:80'... 0.0% > substitute: updating substitutes from 'http://192.168.10.102:80'... 0.0% > substitute: updating substitutes from 'http://192.168.10.102:80'... 0.0% > substitute: updating substitutes from 'http://192.168.10.102:80'... 0.0% > substitute: updating substitutes from 'http://192.168.10.102:80'... 0.0% We=E2=80=99d need to check why this particular server is checked repeatedly. The fact that it displays =E2=80=9C0.0%=E2=80=9D doesn=E2=80=99t mean that = the server lacks substitutes, but that it does not reply to =E2=80=98GET /xyz.narinfo=E2=80= =99 requests, for example because it=E2=80=99s off-line (see .) > We should implement some scheme to prefer querying high-substitute > servers first, instead of wasting time querying servers always failed > queries; this would greatly improve performance when using substitute > discovery for example combined with low coverage. There are several problems with that. First one is that you can=E2=80=99t = tell what substitute coverage is until you=E2=80=99ve actually made those GET requests. Second one is that substitute coverage varies and it=E2=80=99s n= ot an absolute measure; for example, if a server provides substitutes for only 0.1% of all the packages, but that=E2=80=99s precisely the 0.1% you care ab= out, it=E2=80=99s more valuable than the one that has 99% of the packages but la= cks those you want. There are other issues such as the fact that current semantics is to respect the order of substitute URLs, which is presumably chosen by the user according to their own criteria: download speed, bandwidth usage, etc. I hope this makes sense! Ludo=E2=80=99.