From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id EXuqKlU/lF9JEwAA0tVLHw (envelope-from ) for ; Sat, 24 Oct 2020 14:51:01 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id UIVKJlU/lF/pXgAAbx9fmQ (envelope-from ) for ; Sat, 24 Oct 2020 14:51:01 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 41813940111 for ; Sat, 24 Oct 2020 14:51:01 +0000 (UTC) Received: from localhost ([::1]:56412 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWKsl-0007Bj-S7 for larch@yhetil.org; Sat, 24 Oct 2020 10:50:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39664) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWKrr-0006NQ-M1 for guix-patches@gnu.org; Sat, 24 Oct 2020 10:50:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:50940) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kWKrq-0006S7-9P for guix-patches@gnu.org; Sat, 24 Oct 2020 10:50:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kWKrq-0001En-7q for guix-patches@gnu.org; Sat, 24 Oct 2020 10:50:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#44193] [PATCH 0/1] 'guix publish --cache' can publish items not yet cached Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 24 Oct 2020 14:50:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 44193 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 44193@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.16035509844723 (code B ref -1); Sat, 24 Oct 2020 14:50:02 +0000 Received: (at submit) by debbugs.gnu.org; 24 Oct 2020 14:49:44 +0000 Received: from localhost ([127.0.0.1]:34253 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWKrY-0001E7-6W for submit@debbugs.gnu.org; Sat, 24 Oct 2020 10:49:44 -0400 Received: from lists.gnu.org ([209.51.188.17]:43150) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kWKrV-0001Dx-Ba for submit@debbugs.gnu.org; Sat, 24 Oct 2020 10:49:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39644) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kWKrV-0006Gn-12 for guix-patches@gnu.org; Sat, 24 Oct 2020 10:49:41 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:47624) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kWKrS-0006RB-73; Sat, 24 Oct 2020 10:49:39 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=51756 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1kWKrQ-0004dT-Hx; Sat, 24 Oct 2020 10:49:37 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Sat, 24 Oct 2020 16:49:29 +0200 Message-Id: <20201024144929.4529-1-ludo@gnu.org> X-Mailer: git-send-email 2.28.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -3.3 (---) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: scn0 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: -0.51 X-TUID: aUjYpDIfQAB6 Hello! The ‘--cache’ mode of ‘guix publish’ is nice for many reasons. Until now though, it would only return 200 to narinfo and nar requests corresponding to items already in cache—already “baked”. Thus, the first narinfo request for an item would always return 404; one would have to wait until the item is baked to get 200 and download the substitute. If you’re fetching substitutes for popular packages on a popular instance, the extra delay is most likely invisible. However, if you’re using an instance with few users, or if you’re interesting in substitutes for an package that’s not popular, the behavior described above is a showstopper. (Many people here have criticized it in the past. :-)) This patch changes the behavior of ‘--cache’: if a store item is not yet in cache, and if it’s “small enough”, then narinfo/nar requests for it immediately return 200, as in the no-cache mode. You’re trading possibly increased server resource usage for reduced publication delay. To put an upper bound on the extra resource usage, the ‘--cache-bypass-threshold’ allows users to control the maximum size of a store item that can receive this treatment. An interesting use case that can benefit from this change is .drv substitutes: .drv themselves can be substitutes, so you can run: guix build /gnu/store/….drv and the daemon will fetch the .drv and its closure and start building it (since commit 9c9982dc0c8c38ce3821b154b7e92509c1564317). It’s not something we use much so far, but maybe we could put it to good use in the future (I know Chris has been playing with it in the context of the Data Service). Thoughts? Ludo’. Ludovic Courtès (1): publish: Add '--cache-bypass-threshold'. doc/guix.texi | 24 +++++++++++- guix/scripts/publish.scm | 85 ++++++++++++++++++++++++++++++++-------- tests/publish.scm | 43 ++++++++++++++++++-- 3 files changed, 130 insertions(+), 22 deletions(-) -- 2.28.0