From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ben Woodcroft Subject: Re: pypi URL change Date: Mon, 25 Apr 2016 09:21:16 +1000 Message-ID: <571D54EC.2030000@gmail.com> References: <571C754A.1050506@gmail.com> <571D4811.4050300@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:60821) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1auTLD-0005KX-9q for guix-devel@gnu.org; Sun, 24 Apr 2016 19:21:28 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1auTLA-0006CH-3o for guix-devel@gnu.org; Sun, 24 Apr 2016 19:21:27 -0400 Received: from mail-pf0-x243.google.com ([2607:f8b0:400e:c00::243]:36825) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1auTL9-0006C7-SY for guix-devel@gnu.org; Sun, 24 Apr 2016 19:21:24 -0400 Received: by mail-pf0-x243.google.com with SMTP id p185so15657986pfb.3 for ; Sun, 24 Apr 2016 16:21:23 -0700 (PDT) In-Reply-To: <571D4811.4050300@gmail.com> List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Cyril Roelandt , guix-devel@gnu.org On 25/04/16 08:26, Cyril Roelandt wrote: > On 04/24/2016 09:27 AM, Ben Woodcroft wrote: >> Hi, >> >> Today I tried using pypi-url to autogenerate a package that was only >> released today. Unfortunately, it seems that the URL scheme has changed >> and I was unable to download, though I believe this only affects newly >> released package versions. I found this on the nix mailing list: >> http://comments.gmane.org/gmane.linux.distributions.nixos/20158 >> >>> So, previously PyPI used URLs like : >>> //packages//{python version}/{name[0]}/{name}/{filename} >>> >>> Now it uses: >>> //packages//{hash[:2]}/{hash[2:4]}/{hash[4:]}/{filename} >>> Where hash is blake2b(file_content, digest_size=32).hexdigest().lower() >> Is there a blake2b implemented in Guile? In the meantime, the URL will have to be specified manually: 'guix import pypi ...' provides the correct and updated URL. >> > The URL should always be correct, because the pypi importer can detect > whether the URL build by pypi-uri is wrong (see make-pypi-sexp). > > Now, in order to handle this new pattern, maybe pypi-uri should: > > 1) try to compute the "new URI" > 2) issue a "curl --head $uri" using some Guile magic > 3) return the URI or compute the "old URI" depending on 2) > > WDYT? Are suggesting that pypi-uri queries pypi each time it is called, even during a build? That seems a little wasteful. I think there is actually 2 problems here. 1) 'guix refresh -u' updates the package with 'pypi-uri', not the URL it couldn't guess. 2) The new URLs are even uglier than the old one and a working pypi-uri would hide them from our eyes. Fixing (1) should be easier and guards against future URL changes too, but fixing (2) would be good too. I'm not familiar with Guile bindings, but there is a C library for blake2b so perhaps could simply bind to that? https://github.com/BLAKE2/libb2 Thoughts? Thanks, ben