From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id KBwILMkzWGFGpgAAgWs5BA (envelope-from ) for ; Sat, 02 Oct 2021 12:26:17 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id 4ITSJ8kzWGHCTAAA1q6Kng (envelope-from ) for ; Sat, 02 Oct 2021 10:26:17 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 67A712D8F5 for ; Sat, 2 Oct 2021 12:26:17 +0200 (CEST) Received: from localhost ([::1]:53648 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcDg-0003ML-Ix for larch@yhetil.org; Sat, 02 Oct 2021 06:26:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:38432) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mWcBY-0002HV-KD for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46433) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcBY-0000Cy-CX for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:04 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mWcBY-0007ZM-9J for guix-patches@gnu.org; Sat, 02 Oct 2021 06:24:04 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50960] [PATCH 10/10] shell: Maintain a profile cache. Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 02 Oct 2021 10:24:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50960 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50960@debbugs.gnu.org Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 50960-submit@debbugs.gnu.org id=B50960.163317019328986 (code B ref 50960); Sat, 02 Oct 2021 10:24:04 +0000 Received: (at 50960) by debbugs.gnu.org; 2 Oct 2021 10:23:13 +0000 Received: from localhost ([127.0.0.1]:57971 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAj-0007XM-3g for submit@debbugs.gnu.org; Sat, 02 Oct 2021 06:23:13 -0400 Received: from eggs.gnu.org ([209.51.188.92]:33150) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWcAY-0007UY-Li for 50960@debbugs.gnu.org; Sat, 02 Oct 2021 06:23:03 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:49206) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mWcAT-0007hX-G3; Sat, 02 Oct 2021 06:22:57 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=36460 helo=gnu.org) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mWcAT-0007gb-6e; Sat, 02 Oct 2021 06:22:57 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Date: Sat, 2 Oct 2021 12:22:40 +0200 Message-Id: <20211002102240.27815-10-ludo@gnu.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20211002102240.27815-1-ludo@gnu.org> References: <20211002102240.27815-1-ludo@gnu.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1633170377; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=5Ag+bUs+DluTMGNAdAD+Ho+RU5iuNn4wIbCbv4AqL+4=; b=j06BUogDdi1b2DztZIfAjCXCroDrOxsVID7tK2/14h7q8Ahqd+VxdaBCIUpyIbEy1S7rzL qJE2XNqIdZc27sAPlMabxjdqDs8ZyMpr4I5vcvT34JwUFyqm0oEHdvKL99ImxoMYWMKfDS vvjxhpNuAdJObliDN9PQz070yDXleooF6S0Q2Hma9b8q9NNUH3s+rtdEEi+l9ljcp+kmqq lkrZW7lmotndz7RX985L4J9GyztBgDnnXj847OlXPYZUDctOnkMCFkhq2J1MJDkyDGxZab X/SrPtWel2Pk+QucjDMVydPiap9h3N8Rl7LM1u3qNXRInU12A4gDWSfbahJqSQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1633170377; a=rsa-sha256; cv=none; b=mwhqEhL+ViL1IeywiRFRNHqvUWaXd/i63nTfB+oX4B1SV3MlKRY0nfaA+HgJ60RMfqqit+ KhOaVrc+r/9hiJW2fcK8ntCHljgS/mlxkT45k/G/ZzhMM86ewLYbhB65In+l3o/pxDi3UI KwF2O4MDYTEcPJR12HH9A6QHg9NPS+6YZzHVbDzMWjmdIa+S/yyIi1LnqJPvRrDg29tFaP McN0Dyp37vw5xKIghBqVHuF/JTJllgjPzv+bIFQHmpY4dMi6rdBlm0qX9iuCZFubqLRLll 1oR/ba0nn/3DAXELLZRt8TkLlhD5RHwMQ6WoVMjRPppKYg0J+iApticgJKT/Gw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: 2.79 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: 67A712D8F5 X-Spam-Score: 2.79 X-Migadu-Scanner: scn1.migadu.com X-TUID: 2X9Q3rYUKO9Q With this change, running "guix shell" (no arguments) is equivalent to: guix environment -r ~/.cache/guix/profiles/some-root -l guix.scm This is the cache miss. On cache hit, it's equivalent to: guix environment -p ~/.cache/guix/profiles/some-root ... which can run in 0.1s. * guix/scripts/shell.scm (auto-detect-manifest): Looked for a cached GC root to the profile and use it. (%profile-cache-directory): New variable. (profile-cache-key, profile-cached-gc-root): New procedures. (guix-shell)[cache-entries, entry-expiration]: New procedures. Add call to 'maybe-remove-expired-cache-entries'. --- guix/scripts/shell.scm | 90 +++++++++++++++++++++++++++++++++++++++--- 1 file changed, 84 insertions(+), 6 deletions(-) diff --git a/guix/scripts/shell.scm b/guix/scripts/shell.scm index 2f15befbd3..7c116cc770 100644 --- a/guix/scripts/shell.scm +++ b/guix/scripts/shell.scm @@ -29,6 +29,15 @@ #:use-module (srfi srfi-37) #:use-module (srfi srfi-71) #:use-module (ice-9 match) + #:autoload (guix base32) (bytevector->base32-string) + #:autoload (rnrs bytevectors) (string->utf8) + #:autoload (guix utils) (cache-directory) + #:autoload (guix describe) (current-channels) + #:autoload (guix channels) (channel-commit) + #:autoload (gcrypt hash) (sha256) + #:use-module ((guix build utils) #:select (mkdir-p)) + #:use-module (guix cache) + #:use-module ((ice-9 ftw) #:select (scandir)) #:export (guix-shell)) (define (show-help) @@ -161,16 +170,85 @@ Return the modified OPTS." (warning (G_ "no packages specified; creating an empty environment~%")) opts) (file + ;; Load environment from FILE; if possible, use/maintain a GC root to + ;; the corresponding profile in cache. (info (G_ "loading environment from '~a'...~%") file) - (match (basename file) - ("guix.scm" - (alist-cons 'load `(package ,file) opts)) - ("manifest.scm" - (alist-cons 'manifest file opts))))))) + (let* ((root (profile-cached-gc-root file)) + (stat (and root (false-if-exception (lstat root))))) + (if (and stat + (<= (stat:mtime ((@ (guile) stat) file)) + (stat:mtime stat))) + (let ((now (current-time))) + ;; Update the atime on ROOT to reflect usage. + (utime root + now (stat:mtime stat) + 0 (stat:mtimensec stat) + AT_SYMLINK_NOFOLLOW) + (alist-cons 'profile root opts)) ;load right away + (let ((opts (match (basename file) + ("guix.scm" + (alist-cons 'load `(package ,file) opts)) + ("manifest.scm" + (alist-cons 'manifest file opts))))) + (if (and root (not (assq-ref opts 'gc-root))) + (begin + (if stat + (delete-file root) + (mkdir-p (dirname root))) + (alist-cons 'gc-root root opts)) + opts)))))))) + + +;;; +;;; Profile cache. +;;; + +(define %profile-cache-directory + ;; Directory where profiles created by 'guix shell' alone (without extra + ;; options) are cached. + (make-parameter (string-append (cache-directory #:ensure? #f) + "/profiles"))) + +(define (profile-cache-key file) + "Return the cache key for the profile corresponding to FILE, a 'guix.scm' or +'manifest.scm' file, or #f if we lack channel information." + (match (current-channels) + (() #f) + (((= channel-commit commits) ...) + (let ((stat (stat file))) + (bytevector->base32-string + (sha256 (string->utf8 + (string-append (string-join commits) ":" + (basename file) ":" + (number->string (stat:dev stat)) ":" + (number->string (stat:ino stat)))))))))) + +(define (profile-cached-gc-root file) + "Return the cached GC root for FILE, a 'guix.scm' or 'manifest.scm' file, or +#f if we lack information to cache it." + (match (profile-cache-key file) + (#f #f) + (key (string-append (%profile-cache-directory) "/" key)))) (define-command (guix-shell . args) (category development) (synopsis "spawn one-off software environments") - (guix-environment* (parse-args args))) + (define (cache-entries directory) + (filter-map (match-lambda + ((or "." "..") #f) + (file (string-append directory "/" file))) + (or (scandir directory) '()))) + + (define* (entry-expiration file) + ;; Return the time at which FILE, a cached profile, is considered expired. + (match (false-if-exception (lstat file)) + (#f 0) ;FILE may have been deleted in the meantime + (st (+ (stat:atime st) (* 60 60 24 7))))) + + (let ((result (guix-environment* (parse-args args)))) + (maybe-remove-expired-cache-entries (%profile-cache-directory) + cache-entries + #:entry-expiration entry-expiration) + result)) -- 2.33.0