From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id AJmJGW1ESGccbgEAe85BDQ:P1 (envelope-from ) for ; Thu, 28 Nov 2024 10:22:37 +0000 Received: from aspmx1.migadu.com ([2001:41d0:403:4876::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2.migadu.com with LMTPS id AJmJGW1ESGccbgEAe85BDQ (envelope-from ) for ; Thu, 28 Nov 2024 11:22:37 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=elephly.net header.s=zoho header.b=B+CwqGyU; dmarc=none; arc=pass ("zohomail.com:s=zohoarc:i=1"); spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=2; s=key1; d=yhetil.org; t=1732789357; a=rsa-sha256; cv=pass; b=YGmnSikDnTbT/okFduxreVQnJhnxKi8pitfEVRCaaPnyuWZH71CKHJ6vHf6Ei8pzkAKkFk 53HGkzMjcHAOgv/EIezj9W/rcqNUbVmAKK2aOyCi64FM0szB+tJ2nbtJv2NAtANPJkeTwt KLMCEhWNXEAWVyY+EPfkkWE/4C4dqosbwKS2MuFU8q7mfIICabMZHzRIYvGx/jpUyv9uZg x6JJiMljRyleZ303uRGBwQKiAZ4TeYrlDfNovFoIwjw5Kw8AO9QMwQNjeDGcZ29AjvB9Qz WvwG/ZQqkSExByY+aVsuXLeIZFqSKcQkfPvBZMRu/3KnB9mIftHvaxudjpIxqw== ARC-Authentication-Results: i=2; aspmx1.migadu.com; dkim=pass header.d=elephly.net header.s=zoho header.b=B+CwqGyU; dmarc=none; arc=pass ("zohomail.com:s=zohoarc:i=1"); spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1732789357; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=lebdaFlnYmF3G3yqZL1O002TB4rTXP0pv56e4v+rpKQ=; b=Esr8e8Itj8HqAfKD9w8GaUKTG0MywmLz1suTzHr+DGD8Ax6Elj6bvHulIJcwxKchdoxdwG jSpTgQ8vUKJbNfQmte6MyfhNzo9FcESMIPQy4MdrujOPkXiuktCM9mF+dAcDas+MsC+L21 aHZpDOkrF8kEsETyVrTJCnX1izJanBnBvb7qti1OyQEgph8RANsU6L1LQdvwHzfcUs3+vN qpJPbdRUC+dLp7xlAaWyTl+ztynW5W11SmRQVEBj/0kOitG2jexZcnj1a5UW0CpqT0RIr7 gK+ndvKphixvITAfxnEhM76o/AvPKybVuf+j/hTiymOSmk7VIgjxz+W3yaxEZA== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id B64741AC5D for ; Thu, 28 Nov 2024 11:22:36 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tGbeh-0005PS-NX; Thu, 28 Nov 2024 05:21:51 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tGbea-0005P4-AP for guix-devel@gnu.org; Thu, 28 Nov 2024 05:21:46 -0500 Received: from sender4-of-o51.zoho.com ([136.143.188.51]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tGbeY-0006UU-8n; Thu, 28 Nov 2024 05:21:44 -0500 ARC-Seal: i=1; a=rsa-sha256; t=1732789297; cv=none; d=zohomail.com; s=zohoarc; b=XnKpQbnx0gT6Tgrjuj3fSOLGk9n0/lUPIu6HZcTLLXLVZHASvgfWWlqAi+EyVbZ2+GNeZuGcbePXJTD15usGtWwy++HPNFUj0fHuu0gUe3VcPM5fCEL6Eu7tYoMTDgg07ACfILjjNd1702y56qp3YlORFsuV2Y6IQRR1r/4O38o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1732789297; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=lebdaFlnYmF3G3yqZL1O002TB4rTXP0pv56e4v+rpKQ=; b=h+8Q1tO0Aqnp3DqXMRPOkV/gUQ92p3q6kthuTbnDePcN9p/iSNpXiHqXERmrt9DltH27AQfEabCb9eu/QrRfAEVG/ff4lWIxZ9hP/P+EwM2L4RkrLC8jso8sTkDVuZ72gSBjRG6O4XtcyObN6ualKBeUuQVowow8Zhwo08KZgS0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1732789297; s=zoho; d=elephly.net; i=rekado@elephly.net; h=From:From:To:To:Cc:Cc:Subject:Subject:In-Reply-To:References:Date:Date:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To; bh=lebdaFlnYmF3G3yqZL1O002TB4rTXP0pv56e4v+rpKQ=; b=B+CwqGyUuuIWcmpP1h4I+/bBWKqAGYSDENmxWNfy8Bj2d0DPslMAIQEU6FyLC14j LtH4EwcLN+q/yFebRzcUXd8Pd0FCQBiCSINSpIVlBKNQKJ5/WscQvYcHwhNb3+sbqxe N4LPKKw73SXUMc/y1Y+s+wwmTUs21QFsNqyJlV3U= Received: by mx.zohomail.com with SMTPS id 1732789297195307.6407946982614; Thu, 28 Nov 2024 02:21:37 -0800 (PST) From: Ricardo Wurmus To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: guix-devel@gnu.org Subject: Re: Caching calls to download-to-store? In-Reply-To: <87zflj69ln.fsf@gnu.org> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?= =?utf-8?Q?s?= message of "Thu, 28 Nov 2024 09:33:40 +0100") References: <87zflkctrz.fsf@elephly.net> <87zflj69ln.fsf@gnu.org> User-Agent: mu4e 1.12.7; emacs 29.4 Date: Thu, 28 Nov 2024 11:21:34 +0100 Message-ID: <87iks7d5g1.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External Received-SPF: pass client-ip=136.143.188.51; envelope-from=rekado@elephly.net; helo=sender4-of-o51.zoho.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -8.67 X-Spam-Score: -8.67 X-Migadu-Queue-Id: B64741AC5D X-Migadu-Scanner: mx10.migadu.com X-TUID: uH9HnWU5Y33G Ludovic Court=C3=A8s writes: > Hi, > > Ricardo Wurmus skribis: > >> the CRAN updater downloads every upstream archive three times. The >> importer memoizes calls to the download procedure, but one download that >> is out of reach is the very first. >> >> It is triggered by package-update/url-fetch in (guix upstream). It uses >> download-to-store, which will unconditionally fetch whatever URL it is >> given. > > Why three times (and not two)? Not entirely sure, but it's tracked here https://issues.guix.gnu.org/48792 https://issues.guix.gnu.org/44256 I'm already caching some downloads in the updater, but it's not always working as intended. (I have an unfinished fix for this locally.) >> I would very much like to parameterize download-to-store, so that we can >> optionally cache the results of its invocation. > > I=E2=80=99m not sure exactly how to do that, but why not. > > Another option would be to register GC roots (or mere symlinks) in > ~/.cache for things that are downloaded, and then to use > =E2=80=98If-Modified-Since=E2=80=99 plus some default TTL to determine wh= ether to > re-download things. That's a good idea. Currently, we download to arbitrary temporary directories. The downloads performed by importers vanish immediately. --=20 Ricardo