From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id GP7UFE6x62KtiQAAbAwnHQ (envelope-from ) for ; Thu, 04 Aug 2022 13:45:18 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id YHa/FE6x62JrXAEAauVa8A (envelope-from ) for ; Thu, 04 Aug 2022 13:45:18 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 1603A427B2 for ; Thu, 4 Aug 2022 13:45:18 +0200 (CEST) Received: from localhost ([::1]:49274 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oJZHw-0006mM-7G for larch@yhetil.org; Thu, 04 Aug 2022 07:45:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:58282) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oJZHi-0006jO-AI for bug-guix@gnu.org; Thu, 04 Aug 2022 07:45:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:33477) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oJZHh-0002f9-Vk for bug-guix@gnu.org; Thu, 04 Aug 2022 07:45:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1oJZHh-0006te-RQ for bug-guix@gnu.org; Thu, 04 Aug 2022 07:45:01 -0400 X-Loop: help-debbugs@gnu.org Subject: bug#56398: (guix git) fails to check out repos with nested submodules Resent-From: =?UTF-8?Q?Andr=C3=A9?= Batista Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 04 Aug 2022 11:45:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 56398 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 56398@debbugs.gnu.org Received: via spool by 56398-submit@debbugs.gnu.org id=B56398.165961346526436 (code B ref 56398); Thu, 04 Aug 2022 11:45:01 +0000 Received: (at 56398) by debbugs.gnu.org; 4 Aug 2022 11:44:25 +0000 Received: from localhost ([127.0.0.1]:51459 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oJZH6-0006sK-NA for submit@debbugs.gnu.org; Thu, 04 Aug 2022 07:44:25 -0400 Received: from mx1.riseup.net ([198.252.153.129]:56346) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oJZH1-0006s1-8B for 56398@debbugs.gnu.org; Thu, 04 Aug 2022 07:44:23 -0400 Received: from fews2.riseup.net (fews2-pn.riseup.net [10.0.1.84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.riseup.net", Issuer "R3" (not verified)) by mx1.riseup.net (Postfix) with ESMTPS id 4Lz6MK47B6zDrfd; Thu, 4 Aug 2022 11:44:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=riseup.net; s=squak; t=1659613453; bh=in3Y79DmsWshf+9Xk9adtSKmgitRdEsUMnXhffZNNtI=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=dM8z9l0aGP0rqRjQAcvyCWyqUgV3/ghWk74oHov5xsJgK+h2xzIM7bsEA27xTtXac vLfLOuvCoWK8jZaBCaoId5I9FHWQyJlyDEVpXmh4FXFaaxRh4Dut6YQfKMw4GQS1Yp XfpttlS4Jcb1/k3g/yTDD3zTFcri2vRO0fFOHI6s= X-Riseup-User-ID: 6D476C40040550CD79FAE87EE0C6A1EB71A55AC55DA168EF2F0B908BA410E5AE Received: from [127.0.0.1] (localhost [127.0.0.1]) by fews2.riseup.net (Postfix) with ESMTPSA id 4Lz6MJ25Zfz214k; Thu, 4 Aug 2022 11:44:12 +0000 (UTC) Date: Thu, 4 Aug 2022 08:43:49 -0300 From: =?UTF-8?Q?Andr=C3=A9?= Batista Message-ID: References: <87sfnf4n7c.fsf@inria.fr> <87pmigxb5r.fsf@inria.fr> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="u4zcwAmZBclUJiwX" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <87pmigxb5r.fsf@inria.fr> X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1659613518; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=QdCClStkR13Me2H1u8GMUsrrYzd8VOTOts+zPTpNzQk=; b=M8KWTkQR2lKB6cYqHKCkqNl+aUG2BGXMoyymxx94RWYL7wah1AmrJB5lh4/Ynip9K02xoE HHxc4POi2ujg6xGHfjmZ0fhYIyFgVgEvMgVvd77YOWMZ6pYjbsIQo5SuTzAgHTMK8QfErN HWkcmJEPuBjqUtEbc4v/6T/5sbpq8NXLVQDAD4HuWGCfJtbxRTGyMrAgv7QNPqJfdMCqtl Nvii1xuQTxZFIWFp4EAk1zgRnTN8U9PC43QvBoTredlY+CmKnKiHsKCxLt5FpZNyhLBjrb QPM0Il/NQiGnh6SA0XXIJQaK+aNh+36uqbO9wSLdjsoDtQoqAbT9F5yjQctRjQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1659613518; a=rsa-sha256; cv=none; b=IPGOmnR5UrvIqF5B6/nlAk3hM9/gQLau4B8eJXCYEEUqpIJTCiyzUmxzQqZ6Xq8NhDC00/ yK2QLIph+hTOBf7rfy8R6Ty74E3NXurmbLpwkwKlKUSOZJpDkGYDhJSgNgli8RWrHv9M2H zcMZjgNZzMIbMwTmx5DhRAZsSREv3/eyl13Wm6MGjsCNYfRfikfZ54Bw6EbTnHVpQBmjkv SeHEzpXqyDxqUTjC83o7Yz2oBpy8fDIb5ORxguqTF0CQt8Qyq11BEo0fLUON3M2EEQ6epA fLkCBA4W7MNgkMsoHw1h/Gqfw7HtLeVnbzF8Mzs3H7lJhs/UdR6+KPh+E52CXw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=riseup.net header.s=squak header.b=dM8z9l0a; dmarc=fail reason="SPF not aligned (relaxed)" header.from=riseup.net (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: 7.09 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=riseup.net header.s=squak header.b=dM8z9l0a; dmarc=fail reason="SPF not aligned (relaxed)" header.from=riseup.net (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 1603A427B2 X-Spam-Score: 7.09 X-Migadu-Scanner: scn0.migadu.com X-TUID: c9SxAa1HVfJf --u4zcwAmZBclUJiwX Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit Hi and I'm sorry for the delay.. sex 08 jul 2022 às 10:26:40 (1657286800), ludovic.courtes@inria.fr enviou: > If we do this: > > --8<---------------cut here---------------start------------->8--- > scheme@(guix git)> (define r (repository-open "/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/")) > scheme@(guix git)> (define mod (submodule-lookup r "third-party/googletest")) > scheme@(guix git)> mod > $13 = # > scheme@(guix git)> (submodule-update $13) > ice-9/boot-9.scm:1685:16: In procedure raise-exception: > Git error: failed to resolve path '/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/third-party/googletest/.git': No such file or directory Nice trick! > (...) > access("/home/ludo/.cache/guix/checkouts/4pe5bqkmsmcros5oviwzvmnjlie6jyhx2dciznz7shwawckb32sq/third-party/googletest/.git", F_OK) = -1 ENOENT (No such file or directory) > (...) > > Thus, looking at ‘git_submodule_update’ in libgit2, it looks as if this > condition was true: > > (submodule_status & GIT_SUBMODULE_STATUS_WD_UNINITIALIZED) > > Hmm, thoughts? Well, I guess ENOENT != GIT_ENOTFOUND and, in that case, I think we are better off gracefully failing to update it than trying to "branch" our local repo. The patch below tests for the directory's existence before proceding with the update. With this patch applied I've managed to refresh pytorch, its submodules and got the following code: --8<---------------cut here---------------start------------->8--- diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index b19af8a1d5..174ba3d39b 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -2871,7 +2871,7 @@ (define-public xnnpack (define-public python-pytorch (package (name "python-pytorch") - (version "1.12.0") + (version "82782") (source (origin (method git-fetch) (uri (git-reference @@ -2881,7 +2881,7 @@ (define-public python-pytorch (file-name (git-file-name name version)) (sha256 (base32 - "0pdqi91qzgyx947zv4pw2fdj9vpqvdhfzw1ydjd4mpqm8g5njgnz")) + "0zshqfqv3lcwyym3l8zx675chnhpxn14c4nr1c2b7ci1zis785va")) (patches (search-patches "python-pytorch-system-libraries.patch" "python-pytorch-runpath.patch")) (modules '((guix build utils))) --8<---------------cut here---------------end--------------->8--- Where "82782" is a ciflow/trunk reference to commit 700dba518be03ee0c0d6389162b5907a13838f49. I've also thought of filtering the submodules list and passing the resulting list instead, but came to conclude that it would clutter the code instead of simplifying it. I hope that helps! --- --u4zcwAmZBclUJiwX Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline; filename="git-scm.patch" Content-Transfer-Encoding: 8bit >From 10bbb79f87f3728c347e33a101add8cb740e9469 Mon Sep 17 00:00:00 2001 In-Reply-To: <56398@debbugs.gnu.org> References: <56398@debbugs.gnu.org> From: =?UTF-8?q?Andr=C3=A9=20Batista?= Date: Thu, 4 Aug 2022 08:07:35 -0300 Subject: [PATCH] guix: git: Gracefully handle missing submodules when updating. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit To: debbugs@gnu.org Fixes . Reported by Ludovic Courtès . * guix/git.scm (update-submodules): Check if submodule directory exists before trying to update it. --- guix/git.scm | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/guix/git.scm b/guix/git.scm index 631bf577d3..d6a82fb86c 100644 --- a/guix/git.scm +++ b/guix/git.scm @@ -320,20 +320,22 @@ (define* (update-submodules repository (fetch-options #f)) "Update the submodules of REPOSITORY, a Git repository object." (for-each (lambda (name) - (let ((submodule (submodule-lookup repository name))) + (let* ((submodule (submodule-lookup repository name)) + (directory (string-append + (repository-working-directory repository) + "/" (submodule-path submodule)))) (format log-port (G_ "updating submodule '~a'...~%") name) - (submodule-update submodule - #:fetch-options fetch-options) - - ;; Recurse in SUBMODULE. - (let ((directory (string-append - (repository-working-directory repository) - "/" (submodule-path submodule)))) - (with-repository directory repository - (update-submodules repository - #:fetch-options fetch-options - #:log-port log-port))))) + (if (file-exists? directory) + ((lambda () + (submodule-update submodule + #:fetch-options fetch-options) + + ;; Recurse in SUBMODULE. + (with-repository directory repository + (update-submodules repository + #:fetch-options fetch-options + #:log-port log-port))))))) (repository-submodules repository))) (define-syntax-rule (false-if-git-not-found exp) -- 2.36.0 --u4zcwAmZBclUJiwX--