From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id uGJxCsU0amG2pwAAgWs5BA (envelope-from ) for ; Sat, 16 Oct 2021 04:11:17 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id wIMpBsU0amHxAgAAB5/wlQ (envelope-from ) for ; Sat, 16 Oct 2021 02:11:17 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 3B350D204 for ; Sat, 16 Oct 2021 04:11:14 +0200 (CEST) Received: from localhost ([::1]:43590 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mbZAG-0002sY-7g for larch@yhetil.org; Fri, 15 Oct 2021 22:11:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:48810) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mbZA6-0002sJ-RF for guix-patches@gnu.org; Fri, 15 Oct 2021 22:11:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:57338) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mbZA6-0002dN-IB for guix-patches@gnu.org; Fri, 15 Oct 2021 22:11:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mbZA6-0004UW-DW for guix-patches@gnu.org; Fri, 15 Oct 2021 22:11:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#51091] [PATCH v2] guix: opam: Do not fail when refreshing. Resent-From: Julien Lepiller Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 16 Oct 2021 02:11:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 51091 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 51091@debbugs.gnu.org, Xinglu Chen Received: via spool by 51091-submit@debbugs.gnu.org id=B51091.163435022617217 (code B ref 51091); Sat, 16 Oct 2021 02:11:02 +0000 Received: (at 51091) by debbugs.gnu.org; 16 Oct 2021 02:10:26 +0000 Received: from localhost ([127.0.0.1]:40651 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mbZ9V-0004Tc-EO for submit@debbugs.gnu.org; Fri, 15 Oct 2021 22:10:25 -0400 Received: from lepiller.eu ([89.234.186.109]:35668) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mbZ9R-0004TP-7M for 51091@debbugs.gnu.org; Fri, 15 Oct 2021 22:10:24 -0400 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id df1c9c26; Sat, 16 Oct 2021 02:10:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date:from :to:cc:subject:message-id:in-reply-to:references:mime-version :content-type; s=dkim; bh=IPNDWEJCEwENk7ejKtUVydY5I/Z7J9NF7fDY38 K5Uws=; b=H/DXYNmFMH5Ny/O6z8pGEb7AHhMO2Qq0Mnl7lOfsSZSXnYH0dOV4Kr vRSksk9VxcKnGdgtzc1WwPD9Zm03x+u4uMJZ4M+fxtCbQte29qHqaQr9SApKWVWQ DMR+Em9VcNR3xnQfxbLT9apVxxuUxaGSjPdIaKX1fBmn2uL4YIk8lN2agyvBbjZC hPJGnuvE9sDnDapd+SEtvGnHODRmlYSoVjnQlnvp0da3707xu5d94qfpKc+/jGo6 DQ5pR4h4XoPq1kjAX5Gs/OFTwc/e9AOlvOct/iduK68vpWgFR2gKWboqFbrMqHYV MhfXjX26FnXvVkiAiEC/Q0Ls77wmmHLA== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id ad558339 (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO); Sat, 16 Oct 2021 02:10:17 +0000 (UTC) Date: Sat, 16 Oct 2021 04:10:09 +0200 From: Julien Lepiller Message-ID: <20211016041009.5b4ddff0@tachikoma.lepiller.eu> In-Reply-To: <871r4nrc9t.fsf_-_@gnu.org> References: <20211008050310.407d6b23@tachikoma.lepiller.eu> <87y272lepg.fsf@yoctocell.xyz> <871r4nrc9t.fsf_-_@gnu.org> X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="MP_/kJPvXDquHX6Rc/CDmw1XNkI" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1634350276; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=ReJV9NwxiNOLNzfUDfCHC/Gc5s4M+46Zw9W+gwh2Z+Y=; b=sBkexIFFyulG10JQCDNZlgosqQyAnpTIauUyUyB+DsFkvXqAtmcNeiwv5SV2CSZ24a0uY2 N1BoKWuwDtuhUBG+z+sdrTVVx4sCUnnd6YNrxZT1rh4EbK6JYRzOOm/CWbRo9jAK/G+W9j Q3qwFfuNdm72LNJbu7TIPukIgqjTFAh3Q7HKTBsyLCqoBpj3WyTl9pAQOL69/t3Bb9yCPO md5YGno3OQ5v8EeiOb+Ee+wW2cwrW9iYuySEbxfOXccyUNMxcTOTTL1GWobS9yBcdsJ1n4 8ibC5b1XK7gxCRz5uDW8YclbZ9pW+3dqBDn+dlYalY4QfTTBqog0SpYWzSwLgQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1634350276; a=rsa-sha256; cv=none; b=rI7/avUTn3PjW5Xn+Vu04D5LQG5Y1po4Db5NekIHWJbgivzGmzYhmSnIg7FyecZzPGBc/E 4fKVbWbPWNK25HjP0tjzYNuPzETThGM1WIgh96r+d4HkXDUEN+neyTKbWUkGKqBXlwGiJK 66yslJT2HhZvWLq15OLMk2+B7xzFrX6m/1HMlEUlPY6alfotU2Lid9yXV0yufnwhlt2PmP BOkpLW43hZSvbGExmLiq1mWRPGnmEwd61sM0k8VZiZY8EED/hrNFkHmKqJNjWwiEyxul+S wV6dYRRVV3DDWE/UCPrJbJsOk+0eoviq+w8uRmn/6fyfRmQGCCdpKEzU/cUbfQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=lepiller.eu header.s=dkim header.b="H/DXYNmF"; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Spam-Score: -1.32 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=lepiller.eu header.s=dkim header.b="H/DXYNmF"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=lepiller.eu (policy=none); spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Migadu-Queue-Id: 3B350D204 X-Spam-Score: -1.32 X-Migadu-Scanner: scn0.migadu.com X-TUID: 9Mc9wlKMNah6 --MP_/kJPvXDquHX6Rc/CDmw1XNkI Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline Thanks for the reviews! I've managed to use conditions, which was not easy because I didn't notice there was already a "condition" defined, and that lead to weird error messages. Attached is the updated patch. --MP_/kJPvXDquHX6Rc/CDmw1XNkI Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0001-import-opam-Do-not-fail-when-refreshing.patch >From 0cadac6c3dabea8b986cd59d97c84beaf7a33325 Mon Sep 17 00:00:00 2001 Message-Id: <0cadac6c3dabea8b986cd59d97c84beaf7a33325.1634350078.git.julien@lepiller.eu> From: Julien Lepiller Date: Fri, 8 Oct 2021 04:58:27 +0200 Subject: [PATCH] import: opam: Do not fail when refreshing. Because we throw an error when a package is not in the opam repository, the updater would crash when encountering a package that is not in opam but uses the ocaml-build-system, such as opam itself. This catches the error and continues without updating said package, and lets us update the rest of the packages. * guix/scripts/import/opam.scm (guix-import-opam): Catch not-found condition and leave. * guix/import/opam.scm (&opam-not-found-error): New condition type. (opam-fetch): Raise condition instead of leaving. (latest-release): Catch not-found condition and warn. (conditional): Rename from `condition'. * tests/opam.scm (parse-conditions): Change accordingly. --- guix/import/opam.scm | 45 +++++++++++++++++++++++++----------- guix/scripts/import/opam.scm | 31 ++++++++++++++----------- tests/opam.scm | 2 +- 3 files changed, 49 insertions(+), 29 deletions(-) diff --git a/guix/import/opam.scm b/guix/import/opam.scm index fe13d29f03..cd746f6dc6 100644 --- a/guix/import/opam.scm +++ b/guix/import/opam.scm @@ -30,6 +30,8 @@ #:use-module (srfi srfi-1) #:use-module (srfi srfi-2) #:use-module ((srfi srfi-26) #:select (cut)) + #:use-module (srfi srfi-34) + #:use-module (srfi srfi-35) #:use-module ((web uri) #:select (string->uri uri->string)) #:use-module ((guix build utils) #:select (dump-port find-files mkdir-p)) #:use-module (guix build-system) @@ -47,12 +49,16 @@ opam-recursive-import %opam-updater + &opam-not-found-error + opam-not-found-error? + opam-not-found-error-package + ;; The following patterns are exported for testing purposes. string-pat multiline-string list-pat dict - condition)) + conditional)) ;; Define a PEG parser for the opam format (define-peg-pattern comment none (and "#" (* COMMCHR) "\n")) @@ -85,7 +91,7 @@ (and (or conditional-value ground-value) (* SP) (ignore "&") (* SP) (or group-pat conditional-value ground-value))) (define-peg-pattern ground-value body (and (or multiline-string string-pat choice-pat list-pat var) (* SP))) -(define-peg-pattern conditional-value all (and ground-value (* SP) condition)) +(define-peg-pattern conditional-value all (and ground-value (* SP) conditional)) (define-peg-pattern string-pat all (and QUOTE (* STRCHR) QUOTE)) (define-peg-pattern list-pat all (and (ignore "[") (* SP) (* (and value (* SP))) (ignore "]"))) (define-peg-pattern var all (+ (or (range #\a #\z) "-"))) @@ -96,7 +102,7 @@ QUOTE QUOTE QUOTE)) (define-peg-pattern dict all (and (ignore "{") (* SP) records (* SP) (ignore "}"))) -(define-peg-pattern condition body (and (ignore "{") condition-form (ignore "}"))) +(define-peg-pattern conditional body (and (ignore "{") condition-form (ignore "}"))) (define-peg-pattern condition-form body (and @@ -310,6 +316,10 @@ path to the repository." (list dependency (list 'unquote (string->symbol dependency)))) (ocaml-names->guix-names lst))) +(define-condition-type &opam-not-found-error &error + opam-not-found-error? + (package opam-not-found-error-package)) + (define* (opam-fetch name #:optional (repositories-specs '("opam"))) (or (fold (lambda (repository others) (match (find-latest-version name repository) @@ -318,7 +328,7 @@ path to the repository." (_ others))) #f (filter-map get-opam-repository repositories-specs)) - (leave (G_ "package '~a' not found~%") name))) + (raise (condition (&opam-not-found-error (package name)))))) (define* (opam->guix-package name #:key (repo '()) version) "Import OPAM package NAME from REPOSITORIES (a list of names, URLs or local @@ -409,16 +419,23 @@ package in OPAM." (define (latest-release package) "Return an for the latest release of PACKAGE." - (and-let* ((opam-name (guix-package->opam-name package)) - (opam-file (opam-fetch opam-name)) - (version (assoc-ref opam-file "version")) - (opam-content (assoc-ref opam-file "metadata")) - (url-dict (metadata-ref opam-content "url")) - (source-url (metadata-ref url-dict "src"))) - (upstream-source - (package (package-name package)) - (version version) - (urls (list source-url))))) + (catch #t + (lambda _ + (and-let* ((opam-name (guix-package->opam-name package)) + (opam-file (guard* (c ((opam-not-found-error? c) + (warning (G_ "package '~a' not found~%") + (opam-not-found-error-package c)) + #f)) + (opam-fetch opam-name))) + (version (assoc-ref opam-file "version")) + (opam-content (assoc-ref opam-file "metadata")) + (url-dict (metadata-ref opam-content "url")) + (source-url (metadata-ref url-dict "src"))) + (upstream-source + (package (package-name package)) + (version version) + (urls (list source-url))))) + (const #f))) (define %opam-updater (upstream-updater diff --git a/guix/scripts/import/opam.scm b/guix/scripts/import/opam.scm index 834ac34cb0..043d05d717 100644 --- a/guix/scripts/import/opam.scm +++ b/guix/scripts/import/opam.scm @@ -93,20 +93,23 @@ Import and convert the opam package for PACKAGE-NAME.\n")) (reverse opts)))) (match args ((package-name) - (if (assoc-ref opts 'recursive) - ;; Recursive import - (map (match-lambda - ((and ('package ('name name) . rest) pkg) - `(define-public ,(string->symbol name) - ,pkg)) - (_ #f)) - (opam-recursive-import package-name #:repo repo)) - ;; Single import - (let ((sexp (opam->guix-package package-name #:repo repo))) - (unless sexp - (leave (G_ "failed to download meta-data for package '~a'~%") - package-name)) - sexp))) + (guard* (c ((opam-not-found-error? c) + (leave (G_ "package '~a' not found~%") + (opam-not-found-error-package c)))) + (if (assoc-ref opts 'recursive) + ;; Recursive import + (map (match-lambda + ((and ('package ('name name) . rest) pkg) + `(define-public ,(string->symbol name) + ,pkg)) + (_ #f)) + (opam-recursive-import package-name #:repo repo)) + ;; Single import + (let ((sexp (opam->guix-package package-name #:repo repo))) + (unless sexp + (leave (G_ "failed to download meta-data for package '~a'~%") + package-name)) + sexp)))) (() (leave (G_ "too few arguments~%"))) ((many ...) diff --git a/tests/opam.scm b/tests/opam.scm index 31b4ea41ff..96b748bcd9 100644 --- a/tests/opam.scm +++ b/tests/opam.scm @@ -171,7 +171,7 @@ url { ("{a: \"b\"\nc: \"d\"}" . (dict (record "a" (string-pat "b")) (record "c" (string-pat "d")))))) (test-opam-syntax - "parse-conditions" condition + "parse-conditions" conditional '(("" . #f) ("{}" . #f) ("{build}" . (condition-var "build")) -- 2.33.0 --MP_/kJPvXDquHX6Rc/CDmw1XNkI--