From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id AEvkDIDrAGKXegEAgWs5BA (envelope-from ) for ; Mon, 07 Feb 2022 10:50:56 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id sChzCYDrAGK4IgEAauVa8A (envelope-from ) for ; Mon, 07 Feb 2022 10:50:56 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 9A650B38E for ; Mon, 7 Feb 2022 10:50:55 +0100 (CET) Received: from localhost ([::1]:57400 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nH0fe-0003s3-HX for larch@yhetil.org; Mon, 07 Feb 2022 04:50:54 -0500 Received: from eggs.gnu.org ([209.51.188.92]:35768) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nGzgs-0003ea-9C for guix-patches@gnu.org; Mon, 07 Feb 2022 03:48:22 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:46983) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nGzgo-0001bP-EJ for guix-patches@gnu.org; Mon, 07 Feb 2022 03:48:05 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nGzgo-00006W-8Y for guix-patches@gnu.org; Mon, 07 Feb 2022 03:48:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#53828] [PATCH] guix: opam: Allow importing local files. Resent-From: Julien Lepiller Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 07 Feb 2022 08:48:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 53828 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Xinglu Chen , 53828@debbugs.gnu.org Received: via spool by 53828-submit@debbugs.gnu.org id=B53828.164422362232653 (code B ref 53828); Mon, 07 Feb 2022 08:48:02 +0000 Received: (at 53828) by debbugs.gnu.org; 7 Feb 2022 08:47:02 +0000 Received: from localhost ([127.0.0.1]:40880 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nGzfp-0008UU-8o for submit@debbugs.gnu.org; Mon, 07 Feb 2022 03:47:01 -0500 Received: from lepiller.eu ([89.234.186.109]:46778) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nGzfm-0008UK-Uo for 53828@debbugs.gnu.org; Mon, 07 Feb 2022 03:47:00 -0500 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id 6c8b50a5; Mon, 7 Feb 2022 08:46:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date:from :to:subject:in-reply-to:references:message-id:mime-version :content-type:content-transfer-encoding; s=dkim; bh=EoFFm+flW95U vQBoMMrG5zhLGbXongkby4wNpu7Id2E=; b=PGOkcOEkNQjYQxG6NOskrJJ+GuNm y2Up0/0eGAOPiRhVFNJdOVaSygd2BuA6deZdTKrgmKV8VPkN12D4SJPtzaXrbvER T+ImL4N7lAq+hQ9611lB27/PALnEEYdFrPwKFHai0pmnhKe+onCH9RM0hrKldIuZ thMX+/0izUIelQfQeC1NqCDVDtoAN0rXf7As7TIfb4Xi77R3hx19zmURngVTj890 AwXdeRzR0yoSIjgXIiU8dUqudzjXplhmc9HWOPFi6aYMkMenhB6/LDoghU0MejZE VXZciEWyt4BC34vRW7KOQDxhGtOM4V0+1OWrHxf2wtLAC5+s3LK41hlQ2w== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id cbcb3ff2 (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO); Mon, 7 Feb 2022 08:46:55 +0000 (UTC) Date: Mon, 07 Feb 2022 09:46:51 +0100 From: Julien Lepiller User-Agent: K-9 Mail for Android In-Reply-To: <87k0e713pg.fsf@yoctocell.xyz> References: <20220206224130.2220aae9@tachikoma.lepiller.eu> <87k0e713pg.fsf@yoctocell.xyz> Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1644227455; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=0k1infuuAZVYai6e09K3NcEuOJ1SG2lFAwiI4w4qSs0=; b=XTuMufAVgd3fRVNLevrN/sLsrnnErIyQTJmfamnESAynVrGhAUnZhop0AaNyRgBMic+w1p 6z49j3ATYi1AZWFSr3DL62GDo6GtMNp+hHghQDPAyjcAJktHrVYnnqHi5pgLEbGiBJXnU8 s0icVWfO4NgR4C3h/qDWJHhJmReK1KsGFDr1aff+dIhlhNwc+oGM3V1TsmIxskomZxV0+G KgncmW1ITFKS7ZwG8GHJIRIlDT3t4TQWoKazDhIBJDHco7WgFFHIjp/Lh+6Q4GMCsfMyUy us4r5SOPlWfOhGvzd9AHnOpFBGvdzIXzUJ3/ZgoxKfCrvd6hCqmW8abv7rWLJQ== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1644227455; a=rsa-sha256; cv=none; b=otscMbF92BHu/sj83U+seoCXBUUyaHma/mS7qCTCcQlKnmAbfrij9pf7LNcMwfiwZiLKow lQEK+dGN/wq0wU6hQcPr/vh+dqMTi+/65TCKUw/ivMMtPyCZGbt7zMWwH/Z4V3NPuTZJLj 9HNS77yymwATsFk17mP6EFlBRj5VG3DpH1X7yXtyhtvY/P3NWw3wAyGyDVJEG4wnRSC9C7 CtS/pWxAbpe++ZnRlDRtOdNc0DkJPzHt1c3jyXdfDZ7GfgjDI96DGiHr1R4CHjTg6JtvjR pBvvquYvIi+jBHuf0cP+ihWGBjjYUY/ZcwIKjjcEBHpl0xO0i9TkgHyVn1YyTw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=lepiller.eu header.s=dkim header.b=PGOkcOEk; dmarc=fail reason="SPF not aligned (relaxed)" header.from=lepiller.eu (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -2.83 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=lepiller.eu header.s=dkim header.b=PGOkcOEk; dmarc=fail reason="SPF not aligned (relaxed)" header.from=lepiller.eu (policy=none); spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 9A650B38E X-Spam-Score: -2.83 X-Migadu-Scanner: scn0.migadu.com X-TUID: Td5tcEOWzx1K On February 7, 2022 9:27:39 AM GMT+01:00, Xinglu Chen wrote: >Julien schrieb am Sonntag der 06=2E Februar 2022 um 22:41 +01: > >> Hi Guix! >> >> This patch lets you import opam packages from a local file=2E This is a >> feature that was requested a few times, and there it is :) >> >> Now, if you have an OCaml package sources that provides an opam file >> (usually called opam, or with the =2Eopam extension), you can import th= e >> package, recursively or not with something like: >> >> guix import opam foo -f foo=2Eopam -r >> >> instead of fetching foo from the repositories, it will fetch the data >> directly from the given file=2E WDYT? > >Why just limit it to local files? What about accepting a tarball on the >internet or any VCS repository? This could generally be useful for any >importer=2E That's a good idea, but I'm not sure how to recognize these cases=2E > >Also, since we already know the name of the package, maybe specifying >the directory that contains the OCaml project would be enough? Sure, that could be useful for your suggestion below to use the local dire= ctory as the source=2E > > guix import opam foo --type=3Dlocal /path/to/directory > guix import opam foo --type=3Dtarball https://example=2Eorg/foo=2Etar= =2Egz > guix import opam foo --type=3Dgit https://git=2Eexample=2Eorg/foo > =E2=80=A6 > >Or am I asking for too much? :-) > >> From b027391357eef4d6fa1bb8741528cf9650aef568 Mon Sep 17 00:00:00 2001 >> Message-Id: >> From: Julien Lepiller >> Date: Sun, 6 Feb 2022 22:35:14 +0100 >> Subject: [PATCH] guix: opam: Allow importing local files=2E > >It should be =E2=80=9Cimport: opam: =E2=80=A6=E2=80=9D=2E > >> * guix/scripts/import/opam=2Escm (guix-import-opam): Support `--file` f= lag=2E >> * guix/import/utils=2Escm (recursive-import): Add optional file argumen= t=2E >> Pass it to the first iteration when available=2E >> * guix/import/opam=2Escm (opam-fetch): Fetch from file when available= =2E >> (opam->guix-source): Factorize source out of opam->guix-package=2E >> (opam->guix-package): Add file argument=2E Ensure it does not fail whe= n >> name, version and url are not available=2E >> --- >> guix/import/opam=2Escm | 101 +++++++++++++++++++--------------= -- >> guix/import/utils=2Escm | 17 +++--- >> guix/scripts/import/opam=2Escm | 11 +++- >> 3 files changed, 75 insertions(+), 54 deletions(-) >> >> diff --git a/guix/import/opam=2Escm b/guix/import/opam=2Escm >> index a6f6fe8c9f=2E=2E05f79110f8 100644 >> --- a/guix/import/opam=2Escm >> +++ b/guix/import/opam=2Escm >> @@ -310,8 +310,11 @@ (define (dependency-list->inputs lst) >> (map string->symbol >> (ocaml-names->guix-names lst))) >> =20 >> -(define* (opam-fetch name #:optional (repositories-specs '("opam"))) >> - (or (fold (lambda (repository others) >> +(define* (opam-fetch name #:optional (repositories-specs '("opam")) (f= ile #f)) >> + (or (and file (let ((metadata (get-metadata file))) >> + `(("metadata" =2E ,metadata) >> + ("version" =2E #f)))) >> + (fold (lambda (repository others) >> (match (find-latest-version name repository) >> ((_ version file) `(("metadata" ,@(get-metadata file)) >> ("version" =2E ,version))) >> @@ -320,17 +323,29 @@ (define* (opam-fetch name #:optional (repositorie= s-specs '("opam"))) >> (filter-map get-opam-repository repositories-specs)) >> (warning (G_ "opam: package '~a' not found~%") name))) >> =20 >> -(define* (opam->guix-package name #:key (repo 'opam) version) >> +(define (opam->guix-source url-dict) >> + (let ((source-url (and url-dict >> + (or (metadata-ref url-dict "src") >> + (metadata-ref url-dict "archive"))))) >> + (if source-url >> + (call-with-temporary-output-file >> + (lambda (temp port) >> + (and (url-fetch source-url temp) >> + `(origin >> + (method url-fetch) >> + (uri ,source-url) >> + (sha256 (base32 ,(guix-hash-url temp))))))) >> + 'no-source-information))) > >I would use =E2=80=98and-let*=E2=80=99 instead of =E2=80=98let=E2=80=99, = and drop the =E2=80=98if=E2=80=99 form=2E That would return #f and make the ocher and-let* in opam->guix-package fai= l=2E > >The ability to import from a local file seems to mostly be useful when >the package isn=E2=80=99t meant to be submitted to Guix, right? Wouldn= =E2=80=99t it >make more sense to make the source a =E2=80=98local-file=E2=80=99 object = instead of just >setting it to #f? Excellent idea=2E > >> +(define* (opam->guix-package name #:key (repo '("opam")) (file #f) ver= sion) >> "Import OPAM package NAME from REPOSITORY (a directory name) or, if >> -REPOSITORY is #f, from the official OPAM repository=2E Return a 'pack= age' sexp >> +REPOSITORY is #f, from the official OPAM repository=2E When FILE is d= efined, >> +import the packaged defined in the opam FILE instead=2E Return a 'pac= kage' sexp > >s/packaged/package/ > >> (and-let* ((with-opam (if (member "opam" repo) repo (cons "opam" rep= o))) >> - (opam-file (opam-fetch name with-opam)) >> - (version (assoc-ref opam-file "version")) >> + (opam-file (opam-fetch name with-opam file)) >> (opam-content (assoc-ref opam-file "metadata")) >> - (url-dict (metadata-ref opam-content "url")) >> - (source-url (or (metadata-ref url-dict "src") >> - (metadata-ref url-dict "archive"))) >> + (source (opam->guix-source (metadata-ref opam-content "ur= l"))) >> (requirements (metadata-ref opam-content "depends")) >> (names (dependency-list->names requirements)) >> (dependencies (filter-dependencies names)) >> @@ -344,47 +359,41 @@ (define* (opam->guix-package name #:key (repo 'op= am) version) >> (not (member name '("dune" "jbuilder= ")))) >> native-dependencies)))) >> (let ((use-dune? (member "dune" names))) >> - (call-with-temporary-output-file >> - (lambda (temp port) >> - (and (url-fetch source-url temp) >> - (values >> - `(package >> - (name ,(ocaml-name->guix-name name)) >> - (version ,version) >> - (source >> - (origin >> - (method url-fetch) >> - (uri ,source-url) >> - (sha256 (base32 ,(guix-hash-url temp))))) >> - (build-system ,(if use-dune? >> - 'dune-build-system >> - 'ocaml-build-system)) >> - ,@(if (null? inputs) >> - '() >> - `((propagated-inputs (list ,@inputs)))) >> - ,@(if (null? native-inputs) >> - '() >> - `((native-inputs (list ,@native-inputs)))) >> - ,@(if (equal? name (guix-name->opam-name (ocaml= -name->guix-name name))) >> - '() >> - `((properties >> - ,(list 'quasiquote `((upstream-name =2E= ,name)))))) >> - (home-page ,(metadata-ref opam-content "homepag= e")) >> - (synopsis ,(metadata-ref opam-content "synopsis= ")) >> - (description ,(beautify-description >> - (metadata-ref opam-content "desc= ription"))) >> - (license ,(spdx-string->license >> - (metadata-ref opam-content "license"= )))) >> - (filter >> - (lambda (name) >> - (not (member name '("dune" "jbuilder")))) >> - dependencies)))))))) >> + (values >> + `(package >> + (name ,(and name (ocaml-name->guix-name name))) >> + (version ,(assoc-ref opam-file "version")) >> + (source ,source) >> + (build-system ,(if use-dune? >> + 'dune-build-system >> + 'ocaml-build-system)) >> + ,@(if (null? inputs) >> + '() >> + `((propagated-inputs (list ,@inputs)))) >> + ,@(if (null? native-inputs) >> + '() >> + `((native-inputs (list ,@native-inputs)))) >> + ,@(if (and name (equal? name (guix-name->opam-name (ocam= l-name->guix-name name)))) >> + '() >> + `((properties >> + ,(list 'quasiquote `((upstream-name =2E ,name)))= ))) >> + (home-page ,(metadata-ref opam-content "homepage")) >> + (synopsis ,(metadata-ref opam-content "synopsis")) >> + (description ,(beautify-description >> + (metadata-ref opam-content "description")= )) >> + (license ,(spdx-string->license >> + (metadata-ref opam-content "license")))) >> + (filter >> + (lambda (name) >> + (not (member name '("dune" "jbuilder")))) >> + dependencies))))) >> =20 >> -(define* (opam-recursive-import package-name #:key repo) >> +(define* (opam-recursive-import package-name #:key repo file) >> (recursive-import package-name >> #:repo->guix-package opam->guix-package >> #:guix-name ocaml-name->guix-name >> - #:repo repo)) >> + #:repo repo >> + #:file file)) >> =20 >> (define (guix-name->opam-name name) >> (if (string-prefix? "ocaml-" name) >> diff --git a/guix/import/utils=2Escm b/guix/import/utils=2Escm >> index 1c3cfa3e0b=2E=2Eab35b8a4fc 100644 >> --- a/guix/import/utils=2Escm >> +++ b/guix/import/utils=2Escm >> @@ -470,7 +470,7 @@ (define (topological-sort nodes >> (set-insert (node-name head) visited)))))))) >> =20 >> (define* (recursive-import package-name >> - #:key repo->guix-package guix-name version = repo >> + #:key repo->guix-package guix-name version = repo file >> #:allow-other-keys) >> "Return a list of package expressions for PACKAGE-NAME and all its >> dependencies, sorted in topological order=2E For each package, > >The docstrings should also be updated=2E