From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0.migadu.com ([2001:41d0:303:e16b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id eBH7JV/XQGeNVgEAqHPOHw:P1 (envelope-from ) for ; Fri, 22 Nov 2024 19:11:27 +0000 Received: from aspmx1.migadu.com ([2001:41d0:303:e16b::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0.migadu.com with LMTPS id eBH7JV/XQGeNVgEAqHPOHw (envelope-from ) for ; Fri, 22 Nov 2024 20:11:27 +0100 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=debbugs.gnu.org header.s=debbugs-gnu-org header.b=tQhA53er; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20230601 header.b=H8bWeief; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1732302687; a=rsa-sha256; cv=none; b=ONH+aYeV2tRPM8BOZgeuAmhHpgEUR9DI8DzNXEfkvF4KesG/kumjfVu1oNj8EEO0l7Szoz qfB001uM+wWmCwLdDvkzGzXU7k+duN/sSr4mdF8v4zTuXSk3fi0lh7cFbKZ8i4pT4gmH7z glcY+RhpMhLPCpeazUgJhK5QVzSqDS76w5rVP1Pm4SZRbdALCTb/vP2GZob69UZdkejlvS oyYobj/1k5OpbY8eGD1BEciOuirvFiobP6L4t54bYbdbaoUTjx6NlmSxAZjFjgm86dbWk9 P6hDyQLLRIRllbk9SV1JiuPfmBOpLOpPI85wNr7xWDNU9cpUSMjE8N3QDfaXyw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=debbugs.gnu.org header.s=debbugs-gnu-org header.b=tQhA53er; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20230601 header.b=H8bWeief; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1732302687; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=8o9R/iC8JNZ8XZ7k62//tJ1gfK9u5OqPGPqttkCnP/g=; b=P5lWsONQdL/tZHxZYYKdLFduPtS64iFdaMnc6NzibL+xq7BqIFTU9rmJoWOt+GJTDk0fxf lwoNWIuOZDmydAJ3eug1zKU3M+6hDmtZu9svIT9TgU6xeLGj6jv6DHOtZuPJFR+jx3Nh/q qPrHZjVhCfwuXqn57uw0aSDaSP9NZOrRfWn3o7rsDHJS8WyLw5rHeJVYvn0s6RKeqbmzTo u3KO8R7rPMGjh/2hOqBW2mMgVZLoyC3hYiV6xiqnNwd8KbP6g/MxQmWE/S74mbsoFS+b1k ts/vfyCy27ZJzkDTqcpLUFXDPDP2L/BrLSRnSv0vOnUKCEFYK9fN0iWlUutsZg== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 2FB9F45C9 for ; Fri, 22 Nov 2024 20:11:27 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tEZ3c-0007Dr-QG; Fri, 22 Nov 2024 14:11:08 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tEZ3a-0007DH-97 for guix-patches@gnu.org; Fri, 22 Nov 2024 14:11:06 -0500 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tEZ3Y-0006zN-U2; Fri, 22 Nov 2024 14:11:04 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:References:In-Reply-To:Date:From:To:Subject; bh=8o9R/iC8JNZ8XZ7k62//tJ1gfK9u5OqPGPqttkCnP/g=; b=tQhA53erLJnZZLM7CnxliHqoePDcAZ/C8P6UZvkiH6CvJH4L5dzrSfEo0gqlxulVHDTX8kUEGAof9HaN+CIj8Cej9sq7LzGK5Xyv9eE7cm630Mki9bbDJuMAZGfuQLgf9+UIgRi3/EBgx9QS86E7lK1r6FWP+36WiOyhKPC4W1mtqBDw+dxuVMKel3rwSQYo6vrq6q8Mb/le1izErtNiyfBOcz/TLyPjIAIw5JmKYBU+G62DPda542zPHz/xuHeMfmQIw+Etozg3Z3ygyQb7ZBgT+etMXhe3FOzcXzLpd5y+7+SqTPlkbv4UAmMvg3m9asl44svDe97lRtN0wKhAew==; Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1tEZ3W-0003Ad-GB; Fri, 22 Nov 2024 14:11:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#74481] [PATCH 1/2] git: Catch Git errors when updating cached checkout. Resent-From: Simon Tournier Original-Sender: "Debbugs-submit" Resent-CC: guix@cbaines.net, dev@jpoiret.xyz, ludo@gnu.org, othacehe@gnu.org, zimon.toutoune@gmail.com, me@tobias.gr, guix-patches@gnu.org Resent-Date: Fri, 22 Nov 2024 19:11:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 74481 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 74481@debbugs.gnu.org Cc: Simon Tournier , Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice X-Debbugs-Original-Xcc: Christopher Baines , Josselin Poiret , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Mathieu Othacehe , Simon Tournier , Tobias Geerinckx-Rice Received: via spool by 74481-submit@debbugs.gnu.org id=B74481.173230265612163 (code B ref 74481); Fri, 22 Nov 2024 19:11:02 +0000 Received: (at 74481) by debbugs.gnu.org; 22 Nov 2024 19:10:56 +0000 Received: from localhost ([127.0.0.1]:55543 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tEZ3P-0003A6-W7 for submit@debbugs.gnu.org; Fri, 22 Nov 2024 14:10:56 -0500 Received: from mail-wr1-f45.google.com ([209.85.221.45]:52544) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1tEZ3L-00039W-UE for 74481@debbugs.gnu.org; Fri, 22 Nov 2024 14:10:53 -0500 Received: by mail-wr1-f45.google.com with SMTP id ffacd0b85a97d-38246333e12so2257554f8f.1 for <74481@debbugs.gnu.org>; Fri, 22 Nov 2024 11:10:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732302586; x=1732907386; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8o9R/iC8JNZ8XZ7k62//tJ1gfK9u5OqPGPqttkCnP/g=; b=H8bWeiefdZvIk3fbTJI/TYBEihdVngwPKQc9n3NgUDLPTF+Q4fhPfd2l+ausEFD9EA CpZmPWJalgoMdb8rVjNIT/WPGevQNIwdRTSMc6jb90RMyBy7LQSX8Wov3+zAIZbbSi1Y FQe7LzPwPXPqgfWGYCtfR30epFY6tz5xTS1x49tBM9QJ9d6RPZaq4Crtgp1pznL4j/9W cdl4JB3IllcZz76Mn3optj/YpivE/NO7rhG4wjx8yZGYuk5JtW6GZaLyv8LLNVQvOPBa LxWsh+gtYyMMbW1NprBD0JdUAqaIBC44puwPq+j9QlRG47FV6K/IXSxQJcsV6Kqxte1N Y3Iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732302586; x=1732907386; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8o9R/iC8JNZ8XZ7k62//tJ1gfK9u5OqPGPqttkCnP/g=; b=QdpBMSrvuKzi5QMZ/QkoC/OvVQlzvvZGrfkkRTlRME4/G8M4BmnOzDK4jendviSoW2 USbTjsqMC13uvTQ/l5EEH4oxbbdSa1Eb7mR97OApds74m87cs7hY5MuZFBsXTMfFMVk6 HSdC62TPHuOd8gCO8vueu3LAxw5pnwqLgz5TR+0hUTSxp8dkOEd0+ZLlJBCZQo1lLULi 1pylT+CcaTEy5AvnuF7Qb/yCmb5v/ynCpRlOcJws506CUgzaozvU6VVhKbgUDAvhUO4J g9zCD6D6xGxPlJ6wnzIXmNX1bkhtALyJ+0vmAOnm1IH3AzISXt3gGjwJ5YBDx5ys0bf8 OFfA== X-Gm-Message-State: AOJu0YwhLzFh+s/JxlMAD67jo8Gtl+oq39dxW/xaIQIKb1H37OWelgG0 teDOu4FD2f4PWCKwzQ8i6EaCAprzTTPCl0tPIX58twYM7+0ZrsrhNgfv2g== X-Gm-Gg: ASbGncsFmzUlDWjrPUg7JlrQLtugIiE9rgcPzr35mkrWUZt0Zqx6yLxdv1xyT0+LoVo Paj3dNskc1XQ97fdsKhCocY/TP6E18K1wN8ajB3yhu9IM7WgTK302oOkrXCcMmIkfnzmhvu/553 FjDrWtlw94+czgkySc+Ns7FlDjcCtt7xw8lJPFEZskdVy6ac9ESYU8PEtj5ALh1xtXERts5ZOsr DcP8GThzfmBHBhRRkxIu6OqiriDyAf2JHmz6a6oxlYiXDtxl/hSNaOc13sI5On9d3l6 X-Google-Smtp-Source: AGHT+IFlkbO7gcc3wgmqcFsLXFczQVLFyxSY1QNTB6n+MXLUgcyy1CCOfx9msbyqtpCm4CSwdSyI4A== X-Received: by 2002:a05:6000:178a:b0:37d:47e0:45fb with SMTP id ffacd0b85a97d-38260b53851mr3743804f8f.21.1732302586040; Fri, 22 Nov 2024 11:09:46 -0800 (PST) Received: from localhost.localdomain ([193.48.40.241]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3825fb260besm3186355f8f.43.2024.11.22.11.09.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Nov 2024 11:09:45 -0800 (PST) From: Simon Tournier Date: Fri, 22 Nov 2024 20:09:26 +0100 Message-ID: X-Mailer: git-send-email 2.46.0 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-Migadu-Queue-Id: 2FB9F45C9 X-Migadu-Scanner: mx13.migadu.com X-Migadu-Spam-Score: -5.47 X-Spam-Score: -5.47 X-TUID: RzL7Am3PkjZP * guix/git.scm (resolve-reference): Catch Git error when reference does not exist and return #false. (switch-to-ref): Adjust. (update-cached-checkout)[close-and-clean!]: New helper. Catch Git error when reference does not exist and warn. Return #false values when reference does not exist. Change-Id: If6e244fe40ebee978ec8de51a6a68bcbd4a2c79e --- guix/git.scm | 128 ++++++++++++++++++++++++++++++--------------------- 1 file changed, 75 insertions(+), 53 deletions(-) diff --git a/guix/git.scm b/guix/git.scm index 410cd4c153..328e2d5c8c 100644 --- a/guix/git.scm +++ b/guix/git.scm @@ -5,7 +5,7 @@ ;;; Copyright © 2021 Marius Bakke ;;; Copyright © 2022 Maxime Devos ;;; Copyright © 2023 Tobias Geerinckx-Rice -;;; Copyright © 2023 Simon Tournier +;;; Copyright © 2023, 2024 Simon Tournier ;;; ;;; This file is part of GNU Guix. ;;; @@ -265,7 +265,7 @@ (define (tag->commit repository tag) (define (resolve-reference repository ref) "Resolve the branch, commit or tag specified by REF, and return the -corresponding Git object." +corresponding Git object. Return #false if REF is not found." (let resolve ((ref ref)) (match ref (('branch . branch) @@ -281,9 +281,12 @@ (define (resolve-reference repository ref) ;; can't be sure it's available. Furthermore, 'string->oid' used to ;; read out-of-bounds when passed a string shorter than 40 chars, ;; which is why we delay calls to it below. - (if (< len 40) - (object-lookup-prefix repository (string->oid commit) len) - (object-lookup repository (string->oid commit))))) + (catch 'git-error + (lambda () + (if (< len 40) + (object-lookup-prefix repository (string->oid commit) len) + (object-lookup repository (string->oid commit)))) + (lambda _ #f)))) (('tag-or-commit . str) (cond ((and (string-contains str "-g") (match (string-split str #\-) @@ -300,7 +303,10 @@ (define (resolve-reference repository ref) => (lambda (commit) (resolve `(commit . ,commit)))) ((or (> (string-length str) 40) (not (string-every char-set:hex-digit str))) - (resolve `(tag . ,str))) ;definitely a tag + (catch 'git-error ;definitely a tag + (lambda () + (resolve `(tag . ,str))) + (lambda _ #f))) (else (catch 'git-error (lambda () @@ -336,13 +342,15 @@ (define (switch-to-ref repository ref) (define obj (resolve-reference repository ref)) - (reset repository obj RESET_HARD) + (and obj + (begin + (reset repository obj RESET_HARD) - ;; There might still be untracked files in REPOSITORY due to an interrupted - ;; checkout for example; delete them. - (delete-untracked-files repository) + ;; There might still be untracked files in REPOSITORY due to an interrupted + ;; checkout for example; delete them. + (delete-untracked-files repository) - (object-id obj)) + (object-id obj)))) (define (call-with-repository directory proc) (let ((repository #f)) @@ -562,6 +570,31 @@ (define* (update-cached-checkout url (string-append directory "/" file))) (or (scandir directory) '()))) + (define (close-and-clean! repository) + (repository-close! repository) + + ;; Update CACHE-DIRECTORY's mtime to so the cache logic sees it. + (match (gettimeofday) + ((seconds . microseconds) + (let ((nanoseconds (* 1000 microseconds))) + (utime cache-directory + seconds seconds + nanoseconds nanoseconds)))) + + ;; Run 'git gc' if needed. + (maybe-run-git-gc cache-directory) + + ;; When CACHE-DIRECTORY is a sub-directory of the default cache + ;; directory, remove expired checkouts that are next to it. + (let ((parent (dirname cache-directory))) + (when (string=? parent (%repository-cache-directory)) + (maybe-remove-expired-cache-entries parent cache-entries + #:entry-expiration + cached-checkout-expiration + #:delete-entry delete-checkout + #:cleanup-period + %checkout-cache-cleanup-period)))) + (define canonical-ref ;; We used to require callers to specify "origin/" for each branch, which ;; made little sense since the cache should be transparent to them. So @@ -583,56 +616,45 @@ (define* (update-cached-checkout url ;; Only fetch remote if it has not been cloned just before. (when (and cache-exists? (not (reference-available? repository ref))) - (remote-fetch (remote-lookup repository "origin") - #:fetch-options (make-default-fetch-options))) + (catch 'git-error + (lambda () + (remote-fetch (remote-lookup repository "origin") + #:fetch-options (make-default-fetch-options))) + (lambda _ + (warning (G_ "Git cannot update the cache of ~a~%") url)))) (when recursive? (update-submodules repository #:log-port log-port #:fetch-options (make-default-fetch-options))) ;; Note: call 'commit-relation' from here because it's more efficient ;; than letting users re-open the checkout later on. - (let* ((oid (if check-out? + (let ((oid (if check-out? (switch-to-ref repository canonical-ref) (object-id - (resolve-reference repository canonical-ref)))) - (new (and starting-commit - (commit-lookup repository oid))) - (old (and starting-commit - (false-if-git-not-found - (commit-lookup repository - (string->oid starting-commit))))) - (relation (and starting-commit - (if old - (commit-relation old new) - 'unrelated)))) - - ;; Reclaim file descriptors and memory mappings associated with - ;; REPOSITORY as soon as possible. - (repository-close! repository) - - ;; Update CACHE-DIRECTORY's mtime to so the cache logic sees it. - (match (gettimeofday) - ((seconds . microseconds) - (let ((nanoseconds (* 1000 microseconds))) - (utime cache-directory - seconds seconds - nanoseconds nanoseconds)))) - - ;; Run 'git gc' if needed. - (maybe-run-git-gc cache-directory) - - ;; When CACHE-DIRECTORY is a sub-directory of the default cache - ;; directory, remove expired checkouts that are next to it. - (let ((parent (dirname cache-directory))) - (when (string=? parent (%repository-cache-directory)) - (maybe-remove-expired-cache-entries parent cache-entries - #:entry-expiration - cached-checkout-expiration - #:delete-entry delete-checkout - #:cleanup-period - %checkout-cache-cleanup-period))) - - (values cache-directory (oid->string oid) relation))))) + (resolve-reference repository canonical-ref))))) + (if (not oid) + (begin + (warning (G_ "Git cannot find the reference ~a~%") (match canonical-ref + ((_ . ref) ref))) + (close-and-clean! repository) + (values #f #f #f)) + (let* ((new (and starting-commit + (commit-lookup repository oid))) + (old (and starting-commit + (false-if-git-not-found + (commit-lookup repository + (string->oid starting-commit))))) + (relation (and starting-commit + (if old + (commit-relation old new) + 'unrelated)))) + + ;; Reclaim file descriptors and memory mappings associated with + ;; REPOSITORY as soon as possible. + + (close-and-clean! repository) + + (values cache-directory (oid->string oid) relation))))))) (define* (latest-repository-commit store url #:key -- 2.46.0