From mboxrd@z Thu Jan 1 00:00:00 1970 From: Cyril Roelandt Subject: Re: [PATCH] import: pypi: read requirements from wheels. Date: Sun, 24 Jan 2016 21:26:13 +0100 Message-ID: <56A53365.5090404@gmail.com> References: <1453414080-23296-1-git-send-email-tipecaml@gmail.com> <87twm2n3je.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:48472) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aNREp-0005Jh-Es for guix-devel@gnu.org; Sun, 24 Jan 2016 15:26:20 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aNREl-00084f-VC for guix-devel@gnu.org; Sun, 24 Jan 2016 15:26:19 -0500 In-Reply-To: <87twm2n3je.fsf@gnu.org> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: =?UTF-8?Q?Ludovic_Court=c3=a8s?= Cc: guix-devel@gnu.org On 01/24/2016 09:08 PM, Ludovic Courtès wrote: > Cyril Roelandt skribis: > >> * guix/import/pypi.scm (latest-wheel-release): New function. > > s/function/procedure/ :-) > > Please also mention the changes in ‘guess-requirements’, > ‘compute-inputs’, etc. > > So do I get it right that pypi now provides packages both in Wheels and > in “traditional” format, but that Wheels provides more info about > dependencies? > > IOW: What does this buy us? :-) When uploading a package to PyPI, one may provide a wheel in addition to a tarball. The wheel is built by setuptools, and contains metadata, which includes *all* the requirements specified by the authors. It is much better than reading "requirements.txt", because this file is not mandatory. Since wheels may not be provided, we need to keep the old way of retrieving dependencies, even though it is not extremely reliable. > >> +(define (latest-wheel-release pypi-package) >> + "Return the url of the wheel for the latest release of pypi-package, of #f if > > Line a bit long. s/of/or/ > >> + (define (read-wheel-metadata wheel-archive) >> + ;; Given WHEEL-ARCHIVE, a ZIP Python wheel archive, return the package's >> + ;; requirements. >> + (let* ((dirname (string-append >> + (string-join >> + (list-head >> + (string-split (last (string-split wheel-url #\/)) #\-) 2) >> + "-") > > "-" should be aligned with (list-head. > > I would be best to turn this transformation into a top-level procedure, > say ‘wheel-url->extracted-directory’. > >> + ".dist-info")) >> + (json-file (string-append dirname "/metadata.json"))) >> + (and (system* "unzip" "-q" wheel-archive json-file) >> + (dynamic-wind >> + (const #t) >> + (lambda () >> + (call-with-input-file json-file >> + (lambda (port) >> + (let* ((metadata (json->scm port)) >> + (run_requires (hash-ref metadata "run_requires")) >> + (requirements (hash-ref (list-ref run_requires 0) >> + "requires"))) >> + (map (lambda (r) >> + (python->package-name (clean-requirement r))) >> + requirements))))) >> + (lambda () >> + (delete-file json-file) >> + (rmdir dirname)))))) > > Eventually I wonder if we should do this in a derivation instead of > hoping for ‘unzip’ & co. to be available there (“eventually”, because > the problem is already present with the ‘requirements.txt’ thingie.) > I think it is fine as is, seeing how it makes Python packaging much easier :) > Could you add a test in tests/pypi.scm? > Will do! Cyril.