From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id oCnuJEfTsl8jaQAA0tVLHw (envelope-from ) for ; Mon, 16 Nov 2020 19:30:15 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id oLXFIEfTsl9uMQAAB5/wlQ (envelope-from ) for ; Mon, 16 Nov 2020 19:30:15 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A666E940367 for ; Mon, 16 Nov 2020 19:30:14 +0000 (UTC) Received: from localhost ([::1]:48780 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kekCb-00053G-JB for larch@yhetil.org; Mon, 16 Nov 2020 14:30:13 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:46296) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kejrD-0001xm-Hd for guix-patches@gnu.org; Mon, 16 Nov 2020 14:08:07 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:45729) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kejrC-0001GE-Fj for guix-patches@gnu.org; Mon, 16 Nov 2020 14:08:07 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kejrC-0002tb-AI for guix-patches@gnu.org; Mon, 16 Nov 2020 14:08:06 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#38408] [PATCH v17 8/8] import: crate: Use existing package satisfying semver requirement. Resent-From: Hartmut Goebel Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 16 Nov 2020 19:08:06 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 38408 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 38408@debbugs.gnu.org X-Debbugs-Original-To: 38408@debbugs.gnu.org, guix-patches@gnu.org Received: via spool by 38408-submit@debbugs.gnu.org id=B38408.160555368511120 (code B ref 38408); Mon, 16 Nov 2020 19:08:06 +0000 Received: (at 38408) by debbugs.gnu.org; 16 Nov 2020 19:08:05 +0000 Received: from localhost ([127.0.0.1]:57272 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kejrA-0002t8-Se for submit@debbugs.gnu.org; Mon, 16 Nov 2020 14:08:05 -0500 Received: from mail-out.m-online.net ([212.18.0.9]:40235) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kejr9-0002sQ-1r for 38408@debbugs.gnu.org; Mon, 16 Nov 2020 14:08:03 -0500 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4CZdrL3d8wz1qs3l; Mon, 16 Nov 2020 20:08:02 +0100 (CET) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4CZdrL3M4Xz1qqkX; Mon, 16 Nov 2020 20:08:02 +0100 (CET) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id NPa1t-ZMq-AK; Mon, 16 Nov 2020 20:08:00 +0100 (CET) Received: from hermia.goebel-consult.de (ppp-188-174-59-253.dynamic.mnet-online.de [188.174.59.253]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS; Mon, 16 Nov 2020 20:08:00 +0100 (CET) Received: from thisbe.fritz.box (thisbe.fritz.box [192.168.110.24]) by hermia.goebel-consult.de (Postfix) with ESMTP id F189060505; Mon, 16 Nov 2020 20:08:28 +0100 (CET) From: Hartmut Goebel Date: Mon, 16 Nov 2020 20:07:28 +0100 Message-Id: <7a551f3ffbd32779fdc46f0e39198ba55c2cefbe.1605552418.git.h.goebel@crazy-compilers.com> X-Mailer: git-send-email 2.21.3 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-Spam-Score: -1.7 (-) X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: "Guix-patches" X-Scanner: ns3122888.ip-94-23-21.eu Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-patches-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-patches-bounces@gnu.org X-Spam-Score: -0.01 X-TUID: yvBiNGFSC1NS If a package satisfying the dependency's semver requirement already exists, use it. Prior to this change the highest version matching the semver requirement was used (and imported in case it was not defined as package already). When resolving a dependency (now done in `sort-map-dependencies`), first search for a package matching the semver requirement and only if this fails reach out for a crate. * guix/import/crate.scm (crate->guix-package)[find-package-version]: New function. [dependency-name+version]: New function. [sort-map-dependencies]: Use it instead of lambda function. * tests/crate.scm (test-doctool-crate, test-doctool-dependencies): New variables. ("self-test …", "cargo-recursive-import-hoors-existing-packages"): New tests. --- guix/import/crate.scm | 37 ++++++++++++++----- tests/crate.scm | 83 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 112 insertions(+), 8 deletions(-) diff --git a/guix/import/crate.scm b/guix/import/crate.scm index b133529ba7..3bc261b04e 100644 --- a/guix/import/crate.scm +++ b/guix/import/crate.scm @@ -32,6 +32,7 @@ #:use-module (guix packages) #:use-module (guix upstream) #:use-module (guix utils) + #:use-module (gnu packages) #:use-module (ice-9 match) #:use-module (ice-9 regex) #:use-module (json) @@ -92,7 +93,7 @@ (requirement crate-dependency-requirement "req")) ;string (module-autoload! (current-module) - '(semver) '(string->semver semversemver semver->string semversemver-range semver-range-contains?)) @@ -235,6 +236,21 @@ look up the development dependencs for the given crate." (or version (crate-latest-version crate)))) + ;; find the highest existing package that fulfills the semver + (define (find-package-version name range) + (let* ((semver-range (string->semver-range range)) + (versions + (sort + (filter (lambda (version) + (semver-range-contains? semver-range version)) + (map (lambda (pkg) + (string->semver (package-version pkg))) + (find-packages-by-name + (crate-name->package-name name)))) + semverstring (last versions))))) + ;; find the highest version of a crate that fulfills the semver (define (find-crate-version crate range) (let* ((semver-range (string->semver-range range)) @@ -251,6 +267,17 @@ look up the development dependencs for the given crate." (and (not (null-list? versions)) (second (last versions))))) + (define (dependency-name+version dep) + (let* ((name (crate-dependency-id dep)) + (req (crate-dependency-requirement dep)) + (existing-version (find-package-version name req))) + (if existing-version + (list name existing-version) + (let* ((crate (lookup-crate* name)) + (ver (find-crate-version crate req))) + (list name + (crate-version-number ver)))))) + (define version* (and crate (find-crate-version crate version-number))) @@ -258,13 +285,7 @@ look up the development dependencs for the given crate." ;; sort and map the dependencies to a list containing ;; pairs of (name version) (define (sort-map-dependencies deps) - (sort (map (lambda (dep) - (let* ((name (crate-dependency-id dep)) - (crate (lookup-crate* name)) - (req (crate-dependency-requirement dep)) - (ver (find-crate-version crate req))) - (list name - (crate-version-number ver)))) + (sort (map dependency-name+version deps) (match-lambda* (((name _) ...) (apply string-cilicense "MIT/Apache-2.0")) + + +(define test-doctool-crate + "{ + \"crate\": { + \"max_version\": \"2.2.2\", + \"name\": \"leaf-bob\", + \"description\": \"summary\", + \"homepage\": \"http://example.com\", + \"repository\": \"http://example.com\", + \"keywords\": [\"dummy\", \"test\"], + \"categories\": [\"test\"] + \"actual_versions\": [ + { \"id\": 234280, + \"num\": \"2.2.2\", + \"license\": \"MIT OR Apache-2.0\", + \"links\": { + \"dependencies\": \"/api/v1/crates/doctool/2.2.2/dependencies\" + } + } + ] + } +}") + +;; FIXME: This test depends on some existing packages +(define test-doctool-dependencies + "{ + \"dependencies\": [ + { + \"crate_id\": \"docopt\", + \"kind\": \"normal\", + \"req\": \"^0.8.1\" + } + ] +}") + + +(test-assert "self-test: rust-docopt 0.8.x is gone, please adjust the test case" + (not (null? (find-packages-by-name "rust-docopt" "0.8")))) + +(test-assert "cargo-recursive-import-hoors-existing-packages" + (mock ((guix http-client) http-fetch + (lambda (url . rest) + (match url + ("https://crates.io/api/v1/crates/doctool" + (open-input-string test-doctool-crate)) + ("https://crates.io/api/v1/crates/doctool/2.2.2/download" + (set! test-source-hash + (bytevector->nix-base32-string + (sha256 (string->bytevector "empty file\n" "utf-8")))) + (open-input-string "empty file\n")) + ("https://crates.io/api/v1/crates/doctool/2.2.2/dependencies" + (open-input-string test-doctool-dependencies)) + (_ (error "Unexpected URL: " url))))) + (match (crate-recursive-import "doctool") + (((define-public 'rust-doctool-2 + (package + (name "rust-doctool") + (version "2.2.2") + (source + (origin + (method url-fetch) + (uri (crate-uri "doctool" version)) + (file-name + (string-append name "-" version ".tar.gz")) + (sha256 + (base32 + (? string? hash))))) + (build-system cargo-build-system) + (arguments + ('quasiquote (#:cargo-inputs + (("rust-docopt" + ('unquote 'rust-docopt-0.8)))))) + (home-page "http://example.com") + (synopsis "summary") + (description "summary") + (license (list license:expat license:asl2.0))))) + #t) + (x + (pk 'fail x #f))))) + (test-end "crate") -- 2.21.3