From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms1.migadu.com with LMTPS id SOW9AOjfJGZ6qgAA62LTzQ:P1 (envelope-from ) for ; Sun, 21 Apr 2024 11:44:08 +0200 Received: from aspmx1.migadu.com ([2001:41d0:303:e224::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1.migadu.com with LMTPS id SOW9AOjfJGZ6qgAA62LTzQ (envelope-from ) for ; Sun, 21 Apr 2024 11:44:08 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1713692647; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=2gj0QOlLNzYINLb77R97kpYiV8+WmMOR4j2latO1DtU=; b=ei5fe6R6RpHNCS/TBZktjDxZtEEkUeHmeTwqceNdddb64vHjD9uNszRmGgdqSS9TsGfv4d jDCirzT4vPh9dfE2iJ+m1CYrntLdmEL9QtotezLngKw4g4DngIoYr9ASbY54hdDRMSJWKT 6Mz0r78VTw6k/jvHAK7traDpd/+MoDoEXYu7eTHMqkqc3GLi2uNwIqD1fBmO+rpW/httWP 6RtVvT63ctW8IQ8J84FhTp8sEVlZn2AV/R6DIef2N5ag3MyGoXs+VUJWmYdVA7HxHW7kFD nXpK3wm6j9mGQ0pq7VkoHc9mYRAVL5oqvkWXLvb+eOEa+tbcQojkyEvGioDDpQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1713692647; a=rsa-sha256; cv=none; b=kBf0FpElS22wBnZr/IaU3HBJBvFn/yxpa/uHfKJw35RBoNcbpJtxibjmzL662x0Fr54w2p 6/TK9Ba7/jClsPIwykohnT+jucyQeLd24fxjSx78i78fo5/rvEnGnEE+ujWeh4/CGNtflR RkXoV6VBrXLp+B5KxT8FlPPG8meqmnp/53yEFmIcP4V8rhFXE225EashkxbpP9isF8FRny faCbkg/2t9FTHpEIDmOMnXM8imMr64nPUkGsVq2NOHyDjq1gngfNnINdKzVNwqqcX2WxBX 33zAg8Q2dc7spjjR5bIlFu3KL5VMN+0m0OMtTWPjab1b8OSy3VeJ2Gfp862dQg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=none Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 8C4691F800 for ; Sun, 21 Apr 2024 11:44:07 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ryTjq-0005Ss-Ig; Sun, 21 Apr 2024 05:43:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ryTjj-0005PE-EP for guix-patches@gnu.org; Sun, 21 Apr 2024 05:43:51 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ryTji-0002lV-G6; Sun, 21 Apr 2024 05:43:50 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ryTjv-0006TR-DT; Sun, 21 Apr 2024 05:44:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#70494] [PATCH 01/23] store: database: Register derivation outputs. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, rekado@elephly.net, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Sun, 21 Apr 2024 09:44:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 70494 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 70494@debbugs.gnu.org Cc: Christopher Baines , Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Ricardo Wurmus , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 70494-submit@debbugs.gnu.org id=B70494.171369260124404 (code B ref 70494); Sun, 21 Apr 2024 09:44:03 +0000 Received: (at 70494) by debbugs.gnu.org; 21 Apr 2024 09:43:21 +0000 Received: from localhost ([127.0.0.1]:41756 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ryTjD-0006LA-0T for submit@debbugs.gnu.org; Sun, 21 Apr 2024 05:43:20 -0400 Received: from mira.cbaines.net ([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:51273) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ryTiw-0006HM-CR for 70494@debbugs.gnu.org; Sun, 21 Apr 2024 05:43:04 -0400 Received: from localhost (unknown [212.132.255.10]) by mira.cbaines.net (Postfix) with ESMTPSA id 1EA3427BBE2; Sun, 21 Apr 2024 10:42:46 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id e25e3982; Sun, 21 Apr 2024 09:42:45 +0000 (UTC) From: Christopher Baines Date: Sun, 21 Apr 2024 10:42:19 +0100 Message-ID: X-Mailer: git-send-email 2.41.0 In-Reply-To: <87bk632h36.fsf@cbaines.net> References: <87bk632h36.fsf@cbaines.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -3.38 X-Spam-Score: -3.38 X-Migadu-Queue-Id: 8C4691F800 X-Migadu-Scanner: mx12.migadu.com X-TUID: pWMaCJ1WmU8D From: Caleb Ristvedt * guix/store/database.scm (register-derivation-outputs, registered-derivation-outputs): New procedures (register-valid-path): Call register-derivation-outputs for derivations. Co-authored-by: Christopher Baines Change-Id: Id958709f36f24ee1c9c375807e8146a9d1cc4259 --- guix/store/database.scm | 49 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/guix/store/database.scm b/guix/store/database.scm index a847f9d2f0..6a9acc2aef 100644 --- a/guix/store/database.scm +++ b/guix/store/database.scm @@ -22,6 +22,9 @@ (define-module (guix store database) #:use-module (sqlite3) #:use-module (guix config) + #:use-module (guix serialization) + #:use-module (guix store) + #:use-module (guix derivations) #:use-module (guix store deduplication) #:use-module (guix base16) #:use-module (guix progress) @@ -44,7 +47,9 @@ (define-module (guix store database) valid-path-id register-valid-path + register-derivation-outputs register-items + registered-derivation-outputs %epoch reset-timestamps vacuum-database)) @@ -206,6 +211,26 @@ (define-inlinable (assert-integer proc in-range? key number) "Integer ~A out of range: ~S" (list key number) (list number)))) +(define (register-derivation-outputs db drv) + "Register all output paths of DRV as being produced by it (note that +this doesn't mean 'already produced by it', but rather just 'associated with +it')." + (let ((stmt (sqlite-prepare + db + " +INSERT OR REPLACE INTO DerivationOutputs (drv, id, path) +SELECT id, :outid, :outpath FROM ValidPaths WHERE path = :drvpath;" + #:cache? #t))) + (for-each (match-lambda + ((outid . ($ path)) + (sqlite-bind-arguments stmt + #:drvpath (derivation-file-name + drv) + #:outid outid + #:outpath path) + (sqlite-step-and-reset stmt))) + (derivation-outputs drv)))) + (define (add-references db referrer references) "REFERRER is the id of the referring store item, REFERENCES is a list ids of items referred to." @@ -284,6 +309,11 @@ (define* (register-valid-path db #:key path (references '()) (sqlite-step-and-reset stmt) (last-insert-row-id db))))) + (when (derivation-path? path) + (register-derivation-outputs db + (read-derivation-from-file + path))) + ;; Call 'path-id' on each of REFERENCES. This ensures we get a ;; "non-NULL constraint" failure if one of REFERENCES is unregistered. (add-references db id @@ -331,6 +361,25 @@ (define %epoch ;; When it all began. (make-time time-utc 0 1)) +(define (registered-derivation-outputs db drv) + "Get the list of (id, output-path) pairs registered for DRV." + (let ((stmt (sqlite-prepare + db + " +SELECT id, path +FROM DerivationOutputs +WHERE drv in (SELECT id from ValidPaths where path = :drv)" + #:cache? #t))) + (sqlite-bind-arguments stmt #:drv drv) + (let ((result (sqlite-fold (lambda (current prev) + (match current + (#(id path) + (cons (cons id path) + prev)))) + '() stmt))) + (sqlite-reset stmt) + result))) + (define* (register-items db items #:key prefix (registration-time (timestamp)) base-commit: 92af4ea17f70207fbbf2513f677f3171d4eafd41 -- 2.41.0