bug#37071: guix import pypi httpie fails

bug#37071: guix import pypi httpie fails

[Robert Vollmert]

$ guix import pypi httpie
 …0.2.tar.gz  83KiB                   291KiB/s 00:00 [##################] 100.0%
 ….py3-none-any.whl  58KiB            201KiB/s 00:00 [##################] 100.0%
guix import: warning: Failed to extract file: httpie-1.0.2.dist-info/METADATA from wheel.
Backtrace:
          15 (primitive-load "/home/rob/.config/guix/current/bin/guix")
In guix/ui.scm:
  1692:12 14 (run-guix-command _ . _)
In guix/scripts/import.scm:
   115:11 13 (guix-import . _)
In guix/scripts/import/pypi.scm:
   102:23 12 (guix-import-pypi . _)
In guix/memoization.scm:
     98:0 11 (_ #<hash-table 11e5b20 0/31> ("httpie") _)
In unknown file:
          10 (_ #<procedure 11f9580 at guix/memoization.scm:179:32 …> …)
In ice-9/boot-9.scm:
    829:9  9 (catch _ _ #<procedure 7f2711b7fd48 at guix/import/pyp…> …)
In guix/utils.scm:
    635:8  8 (call-with-temporary-output-file _)
In guix/import/pypi.scm:
   384:25  7 (_ "/tmp/guix-file.A9OwRK" _)
In guix/utils.scm:
    635:8  6 (call-with-temporary-output-file #<procedure 1091c20 at…>)
In guix/import/utils.scm:
   133:10  5 (_ "/tmp/guix-file.jFzPxe" _)
    123:4  4 (url-fetch _ _)
In guix/build/download.scm:
    763:4  3 (url-fetch "/tmp/guix-file.A9OwRK" "/tmp/guix-file.jFz…" …)
In srfi/srfi-1.scm:
   679:15  2 (append-map _ _ . _)
   592:17  1 (map1 (#f))
In guix/build/download.scm:
    728:8  0 (maybe-expand-mirrors _ _)

guix/build/download.scm:728:8: In procedure maybe-expand-mirrors:
In procedure struct_vtable: Wrong type argument in position 1 (expecting struct): #f

bug#37071: guix import pypi httpie fails

[Nicolas Goaziou]

Hello,

Robert Vollmert <rob@vllmrt.net> writes:

> $ guix import pypi httpie
>  …0.2.tar.gz  83KiB                   291KiB/s 00:00 [##################] 100.0%
>  ….py3-none-any.whl  58KiB            201KiB/s 00:00 [##################] 100.0%
> guix import: warning: Failed to extract file: httpie-1.0.2.dist-info/METADATA from wheel.
> Backtrace:

[...]

>
> guix/build/download.scm:728:8: In procedure maybe-expand-mirrors:
> In procedure struct_vtable: Wrong type argument in position
> 1 (expecting struct): #f

FWIW, I cannot reproduce it. I get

  (package
    (name "python-httpie")
    (version "1.0.2")
    (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "httpie" version))
        (sha256
          (base32
            "1ax22jh5lpjywpj7lsl072wdhr1pxiqzmxhyph5diwxxzs2nqrzw"))))
    (build-system python-build-system)
    (propagated-inputs
      `(("python-colorama" ,python-colorama)
        ("python-pygments" ,python-pygments)
        ("python-requests" ,python-requests)))
    (home-page "http://httpie.org/")
    (synopsis
      "HTTPie - a CLI, cURL-like tool for humans.")
    (description
      "HTTPie - a CLI, cURL-like tool for humans.")
    (license license:bsd-3))


Regards,

-- 
Nicolas Goaziou

bug#37071: guix import pypi httpie fails

[Robert Vollmert]

> On 18. Aug 2019, at 13:28, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
> Robert Vollmert <rob@vllmrt.net> writes:
> 
>> $ guix import pypi httpie
>> …0.2.tar.gz  83KiB                   291KiB/s 00:00 [##################] 100.0%
>> ….py3-none-any.whl  58KiB            201KiB/s 00:00 [##################] 100.0%
>> guix import: warning: Failed to extract file: httpie-1.0.2.dist-info/METADATA from wheel.
>> Backtrace:
> 
> [...]
> 
>> 
>> guix/build/download.scm:728:8: In procedure maybe-expand-mirrors:
>> In procedure struct_vtable: Wrong type argument in position
>> 1 (expecting struct): #f
> 
> FWIW, I cannot reproduce it. I get

Thanks for looking into this!

I’ll try making sure everything is up to date, and try again.

bug#37071: guix import pypi httpie fails

[Robert Vollmert]

> On 18. Aug 2019, at 13:49, Robert Vollmert <rob@vllmrt.net> wrote:
> 
> 
> 
>> On 18. Aug 2019, at 13:28, Nicolas Goaziou <mail@nicolasgoaziou.fr> wrote:
>> Robert Vollmert <rob@vllmrt.net> writes:
>> 
>>> $ guix import pypi httpie
>>> …0.2.tar.gz  83KiB                   291KiB/s 00:00 [##################] 100.0%
>>> ….py3-none-any.whl  58KiB            201KiB/s 00:00 [##################] 100.0%
>>> guix import: warning: Failed to extract file: httpie-1.0.2.dist-info/METADATA from wheel.
>>> Backtrace:
>> 
>> [...]
>> 
>>> 
>>> guix/build/download.scm:728:8: In procedure maybe-expand-mirrors:
>>> In procedure struct_vtable: Wrong type argument in position
>>> 1 (expecting struct): #f
>> 
>> FWIW, I cannot reproduce it. I get
> 
> Thanks for looking into this!
> 
> I’ll try making sure everything is up to date, and try again.

Just to note that the error persists after guix pull. Weird.

@Nicolas: Did your test use a recently pulled guix?

bug#37071: guix import pypi httpie fails

[Nicolas Goaziou]

Hello,

Robert Vollmert <rob@vllmrt.net> writes:

> Just to note that the error persists after guix pull. Weird.
>
> @Nicolas: Did your test use a recently pulled guix?

I confirm the command succeeds with

Génération 34	18 août 2019 22:18:31	(actuelle)
  guix 4167181
    URL du dépôt : https://git.savannah.gnu.org/git/guix.git
    branche: master
    commit : 41671813f13cc4965f3674fdf4eb17506b49caf2

Not sure it helps, though.

Did you try to recompile guix after make clean-go? IIRC I encountered
a similar problem a couple of weeks ago.

Regards,

-- 
Nicolas Goaziou

bug#37071: guix import pypi httpie fails

[Ludovic Courtès]

Hi,

Robert Vollmert <rob@vllmrt.net> skribis:

> $ guix import pypi httpie
>  …0.2.tar.gz  83KiB                   291KiB/s 00:00 [##################] 100.0%
>  ….py3-none-any.whl  58KiB            201KiB/s 00:00 [##################] 100.0%
> guix import: warning: Failed to extract file: httpie-1.0.2.dist-info/METADATA from wheel.
> Backtrace:
>           15 (primitive-load "/home/rob/.config/guix/current/bin/guix")
> In guix/ui.scm:
>   1692:12 14 (run-guix-command _ . _)
> In guix/scripts/import.scm:
>    115:11 13 (guix-import . _)
> In guix/scripts/import/pypi.scm:
>    102:23 12 (guix-import-pypi . _)
> In guix/memoization.scm:
>      98:0 11 (_ #<hash-table 11e5b20 0/31> ("httpie") _)
> In unknown file:
>           10 (_ #<procedure 11f9580 at guix/memoization.scm:179:32 …> …)
> In ice-9/boot-9.scm:
>     829:9  9 (catch _ _ #<procedure 7f2711b7fd48 at guix/import/pyp…> …)
> In guix/utils.scm:
>     635:8  8 (call-with-temporary-output-file _)
> In guix/import/pypi.scm:
>    384:25  7 (_ "/tmp/guix-file.A9OwRK" _)
> In guix/utils.scm:
>     635:8  6 (call-with-temporary-output-file #<procedure 1091c20 at…>)
> In guix/import/utils.scm:
>    133:10  5 (_ "/tmp/guix-file.jFzPxe" _)
>     123:4  4 (url-fetch _ _)
> In guix/build/download.scm:
>     763:4  3 (url-fetch "/tmp/guix-file.A9OwRK" "/tmp/guix-file.jFz…" …)
> In srfi/srfi-1.scm:
>    679:15  2 (append-map _ _ . _)
>    592:17  1 (map1 (#f))
> In guix/build/download.scm:
>     728:8  0 (maybe-expand-mirrors _ _)
>
> guix/build/download.scm:728:8: In procedure maybe-expand-mirrors:
> In procedure struct_vtable: Wrong type argument in position 1 (expecting struct): #f

The key here is the message “Failed to extract […] METADATA from wheel”.

When that happens, I think we miss too much information to do anything
sensible.  Maxim, do you confirm?

If you add ‘unzip’ to $PATH, the problem should vanish.

We should probably process wheels & co. in derivations to avoid relying
on tools that may or may not be in $PATH.

Thoughts?

Ludo’.

bug#37071: guix import pypi httpie fails

[Maxim Cournoyer]

Hello,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
> Robert Vollmert <rob@vllmrt.net> skribis:
>
>> $ guix import pypi httpie
>>  …0.2.tar.gz  83KiB                   291KiB/s 00:00 [##################] 100.0%
>>  ….py3-none-any.whl  58KiB            201KiB/s 00:00 [##################] 100.0%
>> guix import: warning: Failed to extract file: httpie-1.0.2.dist-info/METADATA from wheel.
>> Backtrace:
>>           15 (primitive-load "/home/rob/.config/guix/current/bin/guix")
>> In guix/ui.scm:
>>   1692:12 14 (run-guix-command _ . _)
>> In guix/scripts/import.scm:
>>    115:11 13 (guix-import . _)
>> In guix/scripts/import/pypi.scm:
>>    102:23 12 (guix-import-pypi . _)
>> In guix/memoization.scm:
>>      98:0 11 (_ #<hash-table 11e5b20 0/31> ("httpie") _)
>> In unknown file:
>>           10 (_ #<procedure 11f9580 at guix/memoization.scm:179:32 …> …)
>> In ice-9/boot-9.scm:
>>     829:9  9 (catch _ _ #<procedure 7f2711b7fd48 at guix/import/pyp…> …)
>> In guix/utils.scm:
>>     635:8  8 (call-with-temporary-output-file _)
>> In guix/import/pypi.scm:
>>    384:25  7 (_ "/tmp/guix-file.A9OwRK" _)
>> In guix/utils.scm:
>>     635:8  6 (call-with-temporary-output-file #<procedure 1091c20 at…>)
>> In guix/import/utils.scm:
>>    133:10  5 (_ "/tmp/guix-file.jFzPxe" _)
>>     123:4  4 (url-fetch _ _)
>> In guix/build/download.scm:
>>     763:4  3 (url-fetch "/tmp/guix-file.A9OwRK" "/tmp/guix-file.jFz…" …)
>> In srfi/srfi-1.scm:
>>    679:15  2 (append-map _ _ . _)
>>    592:17  1 (map1 (#f))
>> In guix/build/download.scm:
>>     728:8  0 (maybe-expand-mirrors _ _)
>>
>> guix/build/download.scm:728:8: In procedure maybe-expand-mirrors:
>> In procedure struct_vtable: Wrong type argument in position 1 (expecting struct): #f
>
> The key here is the message “Failed to extract […] METADATA from wheel”.
>
> When that happens, I think we miss too much information to do anything
> sensible.  Maxim, do you confirm?
>
> If you add ‘unzip’ to $PATH, the problem should vanish.
>
> We should probably process wheels & co. in derivations to avoid relying
> on tools that may or may not be in $PATH.
>
> Thoughts?
>
> Ludo’.

I can't seem to reproduce, even with 'unzip' removed from my profile:

--8<---------------cut here---------------start------------->8---
Starting download of /tmp/guix-file.Bdb5Iq
From https://files.pythonhosted.org/packages/d5/a4/ab61c1dbfdef33c7b7f5f7df0d79eb5cd55a106601a4acc17f983f320b4a/httpie-1.0.3.tar.gz...
 …0.3.tar.gz  85KiB                   1.4MiB/s 00:00 [##################] 100.0%

Starting download of /tmp/guix-file.F4KKQY
From https://files.pythonhosted.org/packages/da/08/22487f04aa2cb831c5e02c9286f1d145f81b06bf9bbf1e73b70d2dc15d7f/httpie-1.0.3-py2.py3-none-any.whl...
 ….py3-none-any.whl  58KiB            1.1MiB/s 00:00 [##################] 100.0%
guix import: warning: Failed to extract file: httpie-1.0.3.dist-info/METADATA from wheel.
(package
  (name "python-httpie")
  (version "1.0.3")
  (source
    (origin
      (method url-fetch)
      (uri (pypi-uri "httpie" version))
      (sha256
        (base32
          "103fcigpxf4nqmrdqjnyz7d9n4n16906slwmmqqc0gkxv8hnw6vd"))))
  (build-system python-build-system)
  (propagated-inputs
    `(("python-pygments" ,python-pygments)
      ("python-requests" ,python-requests)))
  (home-page "http://httpie.org/")
  (synopsis
    "HTTPie - a CLI, cURL-like tool for humans.")
  (description
    "HTTPie - a CLI, cURL-like tool for humans.")
  (license license:bsd-3))
--8<---------------cut here---------------end--------------->8---

It's hard to investigate the problem without being able to reproduce,
but, what the code is supposed to do is

a. Parse the METADATA file of the wheel archive.
b. If that fails, parse the first requires.txt file found in a source
archive of the package.

When both fail, empty lists are returned for the required dependencies
and optional test dependencies (and this should just give you a package
without inputs rather than a backtrace).

The later case is covered by the functional test named
"pypi->guix-package, no usable requirement file." in
guix/tests/pypi.scm.

HTH!

Maxim