From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id kB5bGZt44mHTHgEAgWs5BA (envelope-from ) for ; Sat, 15 Jan 2022 08:32:43 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id gHGsFpt44mGxHAEA9RJhRA (envelope-from ) for ; Sat, 15 Jan 2022 08:32:43 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id DFE0837F9A for ; Sat, 15 Jan 2022 08:32:42 +0100 (CET) Received: from localhost ([::1]:41394 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1n8dYI-0004hr-1B for larch@yhetil.org; Sat, 15 Jan 2022 02:32:42 -0500 Received: from eggs.gnu.org ([209.51.188.92]:58806) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n8dWa-0004hK-Dt for guix-devel@gnu.org; Sat, 15 Jan 2022 02:30:56 -0500 Received: from mx1.dismail.de ([78.46.223.134]:22753) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1n8dWY-000149-1I for guix-devel@gnu.org; Sat, 15 Jan 2022 02:30:56 -0500 Received: from mx1.dismail.de (localhost [127.0.0.1]) by mx1.dismail.de (OpenSMTPD) with ESMTP id a1079501 for ; Sat, 15 Jan 2022 08:30:50 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=dismail.de; h=date :message-id:from:to:subject:references:mime-version:content-type :content-transfer-encoding; s=20190914; bh=eJ5JwYEXscHUr1Ahi3mxD e0m81fDs3wIl4DAg4wthB8=; b=KkYN0X89Mp7XKwasEgo8ArMAPKHIFoGn9zSUd DYVLB71oiLlbPH1btO6UifpfARpDHi17q+Gmkhmq//ES6wG0l9pM1laq4I5aWP7A J3ajoZeI2HNYjNPwrBVPu+NNxfgWYI71L63CIJ1f0rbpqw0zT62sS/mWxc9F8wgl 0VcWKGJAoh6owAHyLI3CoMYmLzk/T/rjNhF0pWzSVyyb0Q14L6gYaq7xwvcpnX6I JBJ8zXAYGw23surTfoMGp89M24eV3U7gUqETrP0D00pBeQIPyzgAWQoVZa0y9+XZ fQXQFAN+mQdKIzx1nyzP1aZzf294YT+yR3dX6qh29UB27gffA== Received: from smtp1.dismail.de ( [10.240.26.11]) by mx1.dismail.de (OpenSMTPD) with ESMTP id fc20a793 for ; Sat, 15 Jan 2022 08:30:50 +0100 (CET) Received: from smtp1.dismail.de (localhost [127.0.0.1]) by smtp1.dismail.de (OpenSMTPD) with ESMTP id bf9d5b63 for ; Sat, 15 Jan 2022 08:30:50 +0100 (CET) Received: by dismail.de (OpenSMTPD) with ESMTPSA id 47fee19e (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO) for ; Sat, 15 Jan 2022 08:30:49 +0100 (CET) Date: Sat, 15 Jan 2022 02:30:47 -0500 Message-ID: <20220115023047.GB14101@gac.attlocal.net> From: jgart To: Guix Devel Subject: Caching The Python World: guix shell --container References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=78.46.223.134; envelope-from=jgart@dismail.de; helo=mx1.dismail.de X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1642231963; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=wsiYgYmg166lcO9evjU90YKroj7EisrfHZ98evGi07E=; b=IU6k/Nl05yWsUo2s7xV/2Qi91VrwEaWWpCdftlCN6Pkwfu28JkKiCLMQQyXqPjtriceHUd fnmc9c4N4YAWfCQEg9WCM/X4Lov53KywJYxw/EpP3/J6KtHNHWg/Y9n6/jmWIdAI+3Nqp9 DfeXUWAmc7D8Lul8rwOG/NT5gkVqdJHmF83y0Dp3rLwbxfa5irAcNJY1aXVhcJalJL6H6e XQ5isIs4CvdnxqbPiXw04RtXefwa9/tjiyu45jWouYmroInGsh3/NWZOyNhrInYeOh7RTZ 90JR2NoaDpY3UqVVqzbWP0k4LRDKn9egDgD8nmL9P8CWgbM0eu9mx6mXTuanHQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1642231963; a=rsa-sha256; cv=none; b=bhz/VdF5j8ViO4TKZJT6N7jIk9bvAJopfvYvX1l+k6RgfTc9QYPH0FZ/ITIUKR7OORR4MW u6KOr2vV1MkBbgkEjjb4e5dGUzVx7D3xKg67cYb+TA2hW3mv2M9T/BD/LzHkhzoUVtIQ6k Bi5/IbQjxLpDww4xavZAcKcszffyeRvlUXDYR1I5siAJ5wD/5JDwo4xQ4S1uUOjK8+hQzG cBGF+IqGVUiDBNhgMWsiIzM7hdXOKWK5RQb5KTjiWxZ86Pqo324+g2omIS2Yxw/oXjIQpD LMzP+hqGWgTc+wYLuM1IOlLPTZihK3MKXwZ0eEu31EENlb7dfMm3WKQhut0Qiw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=dismail.de header.s=20190914 header.b=KkYN0X89; dmarc=pass (policy=reject) header.from=dismail.de; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -7.62 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=dismail.de header.s=20190914 header.b=KkYN0X89; dmarc=pass (policy=reject) header.from=dismail.de; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: DFE0837F9A X-Spam-Score: -7.62 X-Migadu-Scanner: scn0.migadu.com X-TUID: PPST+MXeviSg Hi Guixers, I've been developing an app[0] that launches jupyter notebooks in "the cloud" in guix containers[1]. In other words, the user submits a url for a git repo containing the jupyter notebook and the app runs the notebook in a guix container serving it to the user's browser tab. The app auto-detects a manifest.scm in the root of the jupyter notebook's project repository. The app then builds the container and all the dependencies for the notebook specified by the manifest.scm. The issue is the following: Sometimes a manifest with 10 notebook dependencies or even less will take a long time (> 3 minutes) for Guix to build the container with everything specified by the manifest. For example, this repository containing a guix manifest caused the browser to timeout out today after minutes of waiting for guix to build the container. https://github.com/BonfaceKilz/tsaf-analysis-of-bxd-mouse-colonies/blob/main/manifest.scm Is there a way to pre-cache all the guix packaged binaries or a subset thereof *ahead of time* so that if the user needs a particular package/library they can get it near instantaneously[2]/on demand when the app calls guix shell? In other words, I am interested in pre-caching "the python world" packaged by GNU Guix upstream. Is there currently a convenient way to tell Guix to build "all python-* packages" and cache them? One way that comes to mind is dumping all the python packages to a manifest and regularly updating that manifest as new packages are added by maintainers. Does this sound like a sane way to solve this container build waiting issue? Any thoughts/advice is much appreciated. all best, jgart [0]: https://git.genenetwork.org/jgart/binderlite [1]: https://guix.gnu.org/en/blog/2015/container-provisioning-with-guix/ [2]: https://guix.gnu.org/en/blog/2021/from-guix-environment-to-guix-shell/ "hot cache, in 0.1 seconds"