From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id kJduD1K3E2FiMQEAgWs5BA (envelope-from ) for ; Wed, 11 Aug 2021 13:41:06 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id 2MwTC1K3E2GMbQAAB5/wlQ (envelope-from ) for ; Wed, 11 Aug 2021 11:41:06 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 705C81F36C for ; Wed, 11 Aug 2021 13:41:05 +0200 (CEST) Received: from localhost ([::1]:59170 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mDmbY-0003n5-EY for larch@yhetil.org; Wed, 11 Aug 2021 07:41:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47938) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mDmYq-0001q8-Sa for guix-devel@gnu.org; Wed, 11 Aug 2021 07:38:18 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:55524) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mDmYp-0007D8-N1; Wed, 11 Aug 2021 07:38:16 -0400 Received: from [2a01:e0a:19b:d9a0:f2f7:a404:c3d3:f8b4] (port=44524 helo=meije) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mDmYp-0000X5-9l; Wed, 11 Aug 2021 07:38:15 -0400 From: Mathieu Othacehe To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: Substitute timeouts References: <875ywec3oo.fsf@gnu.org> <87pmukmeau.fsf@gnu.org> Date: Wed, 11 Aug 2021 13:38:13 +0200 In-Reply-To: <87pmukmeau.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Wed, 11 Aug 2021 13:06:01 +0200") Message-ID: <87sfzg43fe.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1628682065; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=irUztpkh0Mr/mZirFd/1clyqrmq3UaAGDK6Yj0wSLbA=; b=O5iQlcKvw45nSlo86/6lL+FQSwM5Tg2ADspodj01vkxhZzRsVBnYetqtT66wS6ULEUSJsK FjWTBACvmMVq42gmifDZGbZvE7N8aYgrz4fSPrHizXO79JE6YkmttYKBqLH7cG8wNrXc8O 3x6kHAObSMe1G4x+VEitTurlR7myYPOmyAppJy8FgocMfFme8hboh1MnrAUK34TXUvQqwY zTll8nBK84j2H9bzzgX93NyBUBYpE5zJYBbra7MEZEkBldolOB1I1gW5rySPLbkCodtzCu PhR7olBi0/S8BPG1Ezw/k4PL4MQ0Yvi6CuC36A3JXqYNDZtIK+zGiQXGTYuXCg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1628682065; a=rsa-sha256; cv=none; b=OYaMQmJxVCqr6KYFb+Ec2Waa8hALexzsgDi433ote3ThpH/otJX5caHlsnJHHFnMEXMf7y tvKXoVbGg0Z+nYpoTP6C0/WpoBLRbd5MYajjP/CQ9OOFaVUJQxQAlFwntS+GSer3tSVUpx 34H8PBwLhesHoyzfvtrD9fI3Cxp/TGbhLp87lqHkNJgpgsK7UbIkKWFWJQ5+RUrFpY8F73 zIBroCN9VT6yJfcYGVrpAOziNQd9sm31cOP5LUorsOvDALt4asQEkgZwKICA72CUzKa3EH oxRgWYNnr0L8brnMD2MS2Hn3QZpPsmbFwV7U7berWDcZSZ9uAW5ejrUSYg1/hQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -3.58 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 705C81F36C X-Spam-Score: -3.58 X-Migadu-Scanner: scn1.migadu.com X-TUID: ObfGPegYAxKn Hey Ludo, Thanks for taking the time to read my wall of text :D. > Yeah, it=E2=80=99s a double-edged sword. If this is a problem on the mai= n =E2=80=98guix > publish=E2=80=99 server, we can lower the bypass threshold, which is curr= ently > 50=C2=A0MiB: > > https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/modul= es/sysadmin/services.scm#n450 > > WDYT? That would maybe help, but on the other hand, I would prefer to find a more definitive solution :). > First, in terms of UI, you=E2=80=99d have a command sitting there and doi= ng > nothing, which can be off-putting. Second, clients have no idea how > long they=E2=80=99re going to wait; it could be that the nar is going to = be > baked within seconds, or it could take 20mn if the baking queue is > already crowded or if the user is asking for a big store item like > libreoffice. Third, in many cases, building locally is likely to be > faster than waiting for substitutes to be available (the majority of > packages build very quickly, though the few most popular leaf packages > take a long time to build). It would be interesting to monitor the status of the baking workers. Could it really take 20 minutes to bake a substitute from your experience? Personally, I have always found this baking 404 and bypass cache a bit misleading. When substituting libreoffice, I would much rather wait a few minutes than trying to build it while there's an almost ready substitute. I get that this is a personal choice and maybe it should be an optional behaviour. >> It will also allow the Cuirass build farm to use directly the main guix >> publish server, simplifying the current CI setup. > > The only reason why Cuirass runs its own publish server is to avoid > overloading the main one? No, the main reason is that with the use of a publish cache, the Cuirass workers would probably hit 404 errors while the substitutes are being baked. Using a publish server without cache was a way to work around it. The motivation of the 202 waiting patch was to solve both problems at once. Maybe I should explore the narinfo dedicated thread solution as a short term solution, while starting to think about a more long term solution based on Fiber/Nginx. A Cuirass dedicated solution could also be to declare a build successful only when a nar is available and stop using a non-caching publish server. Thanks, Mathieu