From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:306:2d92::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id GA91J8VzJWVmhQEAauVa8A:P1 (envelope-from ) for ; Tue, 10 Oct 2023 17:54:45 +0200 Received: from aspmx1.migadu.com ([2001:41d0:306:2d92::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id GA91J8VzJWVmhQEAauVa8A (envelope-from ) for ; Tue, 10 Oct 2023 17:54:45 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 5B43954A1E for ; Tue, 10 Oct 2023 17:54:45 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=gnu.org header.s=fencepost-gnu-org header.b=NFc6HSpX; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1696953285; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=4as3NM9UlW7/47mUcDiVUbPDw1/t+XHBzOyTUzrcECU=; b=t9T3dpuQj94qkQafS6trtFqD8XbWRtaLENvsVyaKPKWMNdZo4FVctfbKUa6vszsSrOH0pb 90oxo8QIFuYzgtHOJ8h1sryHKKsl/h6kLJRCxh068IckzH61TNC3nFQLwtEYQCZ/vwVD7t 515ZZU1SiR9fzRgf01gi4VkqmBCIhJ6FvLDiHIB6bGwJK4vh6T8eAmwsbDypWzVvezU1C+ 1LBx4BqS8aaC5hQoKMVrHUBdF2zQaDx/LFXiLKXh4iBNAxvYCIsHxpg8cxCRasEBd3Otaf e8VbDuwi2tY7hNuI7PDEoNx3p0EKIL3BCzmAUaVnrmFciG+KklVKNTKAXX8Ceg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("body hash did not verify") header.d=gnu.org header.s=fencepost-gnu-org header.b=NFc6HSpX; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1696953285; a=rsa-sha256; cv=none; b=leKwshauTJpbO1jS/6tnReNMoxIMN+1OcgkFTGOabnfrhUbCBLZEXCNhO9FlInAZ5T9epH pVdm3JjCcgQQE0M3mrleiJRvaGgZFXwzsi9qnuT8VIIOx+dXXPmUnCqwrJepPg+QXhMbQ1 PAQVI7nwS6oLoU3hMn5m/cFv3QJ/ejPu/D0120mp5sCQdNM1SDW62WSU/5XfaxWKD4a5Fj dTO/D2wCXCKb8yzJee6H3vkLDwSKJueAhTomEMvWeXieS/RTzcv2t/Y2JYyskDSvZ+JfDD sp1TIlMKfhBzuDBzULpm1thdAhmcO0ZocXW/ksv/GNoHk6H2MKNiE0KOopk63A== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qqF3j-0000eK-5S; Tue, 10 Oct 2023 11:54:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qqF3J-0000bf-8U for bug-guix@gnu.org; Tue, 10 Oct 2023 11:53:45 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qqF3E-0000F7-Ml for bug-guix@gnu.org; Tue, 10 Oct 2023 11:53:44 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1qqF3Z-0006bR-UM for bug-guix@gnu.org; Tue, 10 Oct 2023 11:54:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#54447: cuirass: missing derivation error Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Tue, 10 Oct 2023 15:54:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 54447 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Mathieu Othacehe Cc: 54447@debbugs.gnu.org, guix-sysadmin Received: via spool by 54447-submit@debbugs.gnu.org id=B54447.169695320825330 (code B ref 54447); Tue, 10 Oct 2023 15:54:01 +0000 Received: (at 54447) by debbugs.gnu.org; 10 Oct 2023 15:53:28 +0000 Received: from localhost ([127.0.0.1]:36744 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qqF32-0006aU-0S for submit@debbugs.gnu.org; Tue, 10 Oct 2023 11:53:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:59882) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qqF2z-0006aG-8x for 54447@debbugs.gnu.org; Tue, 10 Oct 2023 11:53:26 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qqF2X-00006l-C8; Tue, 10 Oct 2023 11:52:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-Version:Date:References:In-Reply-To:Subject:To: From; bh=UGjkdV5U8aqc0r2Z2Nt5UyDLOqFbO4V1MEP7zwdyLH8=; b=NFc6HSpX9DLrCFZYeeSu 5yQK0gNsTI/KEHx6yTZnoQ7DRXesYT5iL1A2tEc1rP/rLdS2doEm82L1GzMiyEOTZFDUmrJFIcZMk l5l+0uVJ/v+IB/TJ357q5eqA/OoiLsKQCZRTi5tYajG0eDfaK1DJWCy0Wc4fKi0eTdaf2AQRQEUxQ 6pxX8Bbf9ISJuXbv/9KUPNF2cqDUkdUPRXXm1rfHKe3YUSu0CMAQh9CyPHgKLkgTGcl+Jf/SS27no pylNn0qxC9Z6OQCafz0xp9hLzvYfnV2gBBr+YczPYDq62J8mUfcn9LbsyQFyDk1lcnjAbdheVEWgf F7dFNjl7r7MtXg==; From: Ludovic =?UTF-8?Q?Court=C3=A8s?= In-Reply-To: <877d8r4etz.fsf@gnu.org> (Mathieu Othacehe's message of "Fri, 18 Mar 2022 13:36:56 +0100") References: <877d8r4etz.fsf@gnu.org> Date: Tue, 10 Oct 2023 17:52:54 +0200 Message-ID: <87r0m2v5ih.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: bug-guix-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Spam-Score: -5.78 X-Spam-Score: -5.78 X-Migadu-Queue-Id: 5B43954A1E X-Migadu-Scanner: mx2.migadu.com X-TUID: OVyt77uLwdWF --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello! Mathieu Othacehe skribis: > A lot of builds, among them ~20 system tests[1], are failing with: > "cannot build missing derivation > ?/gnu/store/hs6kp1lqgymhyp3jndc0dsp0pn4psgv0-gui-installed-desktop-os-enc= rypted.drv?" > errors. I have a disappointingly simple hypothesis for this. Remember that =E2=80=9Cmissing derivation=E2=80=9D errors happen primarily for system tes= ts. Turns out that =E2=80=98cleanup-cuirass-roots=E2=80=99 in maintenance.git, = used as an mcron job, explicitly removes GC roots for things like *-os-encrypted once they=E2=80=99re more than two days old, as well as GC roots for the corresponding .drv. I think this was increasing the likelihood that a .drv would be GC=E2=80=99= d by the time we run the test: under high load=C2=B9, it=E2=80=99s plausible tha= t a system test wouldn=E2=80=99t be built within two days after it=E2=80=99s been queu= ed. I=E2=80=99m proposing the change below to address this; I don=E2=80=99t thi= nk we need =E2=80=98--gc-keep-outputs --gc-keep-derivations=E2=80=99 anymore now that = we keep things in =E2=80=98guix publish=E2=80=99 cache first and foremost. Thoughts? In addition to the mcron job, Cuirass=E2=80=99s own =E2=80=98register-gc-ro= ots=E2=80=99 procedure periodically deletes GC roots older than =E2=80=98%gc-roots-ttl= =E2=80=99 (30 days in practice). That=E2=80=99s okay, except that it would be safer to d= elete GC roots for a .drv if and only if it=E2=80=99s been built already. Thanks, Ludo=E2=80=99. =C2=B9 The queue was often processed slowly, with many workers remaining id= le due to the bug fixed by . --=-=-= Content-Type: text/x-patch Content-Disposition: inline diff --git a/hydra/modules/sysadmin/services.scm b/hydra/modules/sysadmin/services.scm index fecfdde..e6f2b44 100644 --- a/hydra/modules/sysadmin/services.scm +++ b/hydra/modules/sysadmin/services.scm @@ -110,9 +110,7 @@ ((guix config) => ,(make-config.scm))) #~(begin (use-modules (ice-9 ftw) - (srfi srfi-1) - (guix store) - (guix derivations)) + (srfi srfi-1)) (define %roots-directory "/var/guix/profiles/per-user/cuirass/cuirass") @@ -157,28 +155,6 @@ deleted)) deleted)) - (define (root-target root) - ;; Return the store item ROOT refers to. - (string-append (%store-prefix) "/" (basename root))) - - (define (derivation-referrers store item) - ;; Return the referrers of the derivers of ITEM. - (let* ((derivers (valid-derivers store item)) - (referrers (append-map (lambda (drv) - (referrers store drv)) - derivers))) - (delete-duplicates referrers))) - - (define (delete-gc-root-for-derivation drv) - ;; Delete the GC root for DRV, if any. - (catch 'system-error - (lambda () - (let ((item (derivation-path->output-path drv))) - (delete-file - (string-append %roots-directory - "/" (basename drv))))) - (const #f))) - ;; Note: 'scandir' would introduce too much overhead due ;; to the large number of entries that it would sort. (define deleted @@ -197,17 +173,7 @@ (for-each (lambda (file) (display file port) (newline port)) - deleted))) - - ;; Since we run 'guix-daemon --gc-keep-outputs - ;; --gc-keep-derivations', also remove GC roots for the outputs of - ;; derivations that refer to the derivers of DELETED. - (for-each delete-gc-root-for-derivation - (with-store store - (append-map (lambda (root) - (derivation-referrers - store (root-target root))) - deleted)))))))) + deleted)))))))) (define (gc-jobs threshold) "Return the garbage collection mcron jobs. The garbage collection @@ -251,8 +217,7 @@ collection instead." (build-accounts (* build-accounts-to-max-jobs-ratio max-jobs)) (extra-options (list "--max-jobs" (number->string max-jobs) - "--cores" (number->string cores) - "--gc-keep-outputs" "--gc-keep-derivations")))) + "--cores" (number->string cores))))) ;;; --=-=-=--