guix import pypi uwsgi

guix import pypi uwsgi

[Danny Milosavljevic]

Hi,

I tried to create a package for uwsgi by

$ guix import pypi uwsgi

...From https://pypi.python.org/packages/source/u/uWSGI/uwsgi-2.0.12.tar.gz...
...
(package
  (name "python-uwsgi")
  (version "2.0.12")
  (source
    (origin
      (method url-fetch)
      (uri (pypi-uri "uWSGI" version))
      (sha256
        (base32
          "02g46dnw5j1iw8fsq392bxbk8d21b9pdgb3ypcinv3b4jzdm2srh"))))
  (build-system python-build-system)
  (inputs
    `(("python-setuptools" ,python-setuptools)))
  (home-page "UNKNOWN")
  (synopsis "The uWSGI server")
  (description "The uWSGI server")
  (license #f))

Then I put that (and define-public python-uwsgi ...) into python.scm and did

  $ guix package -i python-uwsgi

I get the following error message:

Starting download of /gnu/store/d16s3cw2arg89pm7djh6xrdkf0dnckyy-uWSGI-2.0.12.tar.gz
From https://pypi.python.org/packages/source/u/uWSGI/uWSGI-2.0.12.tar.gz...
ERROR: download failed "https://pypi.python.org/packages/source/u/uWSGI/uWSGI-2.0.12.tar.gz" 404 "Not Found"
failed to download "/gnu/store/d16s3cw2arg89pm7djh6xrdkf0dnckyy-uWSGI-2.0.12.tar.gz" from "https://pypi.python.org/packages/source/u/uWSGI/uWSGI-2.0.12.tar.gz"
builder for `/gnu/store/d5jfk4l2yvxfd3pw0fndwx2bxjwg1gpl-uWSGI-2.0.12.tar.gz.drv' failed to produce output path `/gnu/store/d16s3cw2arg89pm7djh6xrdkf0dnckyy-uWSGI-2.0.12.tar.gz'
cannot build derivation `/gnu/store/xqwnqx8yw63i864z0c3vd3vr9c8xcflr-python-uwsgi-2.0.12.drv': 1 dependencies couldn't be built
guix package: error: build failed: build of `/gnu/store/xqwnqx8yw63i864z0c3vd3vr9c8xcflr-python-uwsgi-2.0.12.drv' failed

(Note the wrong basename)

Re: guix import pypi uwsgi

[Nils Gillmann]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> Hi,
>
> I tried to create a package for uwsgi by
>
> $ guix import pypi uwsgi
>
> ...From https://pypi.python.org/packages/source/u/uWSGI/uwsgi-2.0.12.tar.gz...
> ...
> (package
>   (name "python-uwsgi")
>   (version "2.0.12")
>   (source
>     (origin
>       (method url-fetch)
>       (uri (pypi-uri "uWSGI" version))

I did not package with python so far, but is it not possible that
you set the url to fetch from with pypi-uri?

>       (sha256
>         (base32
>           "02g46dnw5j1iw8fsq392bxbk8d21b9pdgb3ypcinv3b4jzdm2srh"))))
>   (build-system python-build-system)
>   (inputs
>     `(("python-setuptools" ,python-setuptools)))
>   (home-page "UNKNOWN")
>   (synopsis "The uWSGI server")
>   (description "The uWSGI server")
>   (license #f))
>
> Then I put that (and define-public python-uwsgi ...) into python.scm and did
>
>   $ guix package -i python-uwsgi
>
> I get the following error message:
>
> Starting download of /gnu/store/d16s3cw2arg89pm7djh6xrdkf0dnckyy-uWSGI-2.0.12.tar.gz
> From https://pypi.python.org/packages/source/u/uWSGI/uWSGI-2.0.12.tar.gz...
> ERROR: download failed "https://pypi.python.org/packages/source/u/uWSGI/uWSGI-2.0.12.tar.gz" 404 "Not Found"
> failed to download "/gnu/store/d16s3cw2arg89pm7djh6xrdkf0dnckyy-uWSGI-2.0.12.tar.gz" from "https://pypi.python.org/packages/source/u/uWSGI/uWSGI-2.0.12.tar.gz"
> builder for `/gnu/store/d5jfk4l2yvxfd3pw0fndwx2bxjwg1gpl-uWSGI-2.0.12.tar.gz.drv' failed to produce output path `/gnu/store/d16s3cw2arg89pm7djh6xrdkf0dnckyy-uWSGI-2.0.12.tar.gz'
> cannot build derivation `/gnu/store/xqwnqx8yw63i864z0c3vd3vr9c8xcflr-python-uwsgi-2.0.12.drv': 1 dependencies couldn't be built
> guix package: error: build failed: build of `/gnu/store/xqwnqx8yw63i864z0c3vd3vr9c8xcflr-python-uwsgi-2.0.12.drv' failed
>
> (Note the wrong basename)
>
>

-- 
ng
personal contact: http://krosos.sdf.org
EDN: https://wiki.c3d2.de/EDN

Re: guix import pypi uwsgi

[Danny Milosavljevic]

Hi,

On Fri, 18 Mar 2016 21:32:44 +0100
Nils Gillmann <niasterisk@grrlz.net> wrote:

> I did not package with python so far, but is it not possible that
> you set the url to fetch from with pypi-uri?

According to guix/build-system/python.scm the answer is no:

(define* (pypi-uri name version #:optional (extension ".tar.gz"))
  "Return a URI string for the Python package hosted on the Python Package
Index (PyPI) corresponding to NAME and VERSION.  EXTENSION is the file name
extension, such as '.tar.gz'."
  (string-append "https://pypi.python.org/packages/source/"
                 (string-take name 1) "/" name "/"
                 name "-" version extension))


That's why I bring it up. 

I can just build the URL myself - no problem. But there's a problem in guix when one guix part generates what another guix part can't process (note: guix import even downloaded the file just fine!).

Re: guix import pypi uwsgi

[Nils Gillmann]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> Hi,
>
> On Fri, 18 Mar 2016 21:32:44 +0100
> Nils Gillmann <niasterisk@grrlz.net> wrote:
>
>> I did not package with python so far, but is it not possible that
>> you set the url to fetch from with pypi-uri?
>
> According to guix/build-system/python.scm the answer is no:
>
> (define* (pypi-uri name version #:optional (extension ".tar.gz"))
>   "Return a URI string for the Python package hosted on the Python Package
> Index (PyPI) corresponding to NAME and VERSION.  EXTENSION is the file name
> extension, such as '.tar.gz'."
>   (string-append "https://pypi.python.org/packages/source/"
>                  (string-take name 1) "/" name "/"
>                  name "-" version extension))
>
>
> That's why I bring it up. 
>
> I can just build the URL myself - no problem. But there's a problem in guix when one guix part generates what another guix part can't process (note: guix import even downloaded the file just fine!).

Looks like a bug for me, maybe parallel to this thread you could
open a bug on the bugs list so that it can be tracked.

pypi import should be changed to be able to deliver the correct
url and adjust if a part (like file.tar.gz) differs from the name
it is known at in pypi.

-- 
ng
personal contact: http://krosos.sdf.org
EDN: https://wiki.c3d2.de/EDN

Re: guix import pypi uwsgi

[Ludovic Courtès]

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> On Fri, 18 Mar 2016 21:32:44 +0100
> Nils Gillmann <niasterisk@grrlz.net> wrote:
>
>> I did not package with python so far, but is it not possible that
>> you set the url to fetch from with pypi-uri?
>
> According to guix/build-system/python.scm the answer is no:
>
> (define* (pypi-uri name version #:optional (extension ".tar.gz"))
>   "Return a URI string for the Python package hosted on the Python Package
> Index (PyPI) corresponding to NAME and VERSION.  EXTENSION is the file name
> extension, such as '.tar.gz'."
>   (string-append "https://pypi.python.org/packages/source/"
>                  (string-take name 1) "/" name "/"
>                  name "-" version extension))
>
>
> That's why I bring it up. 
>
> I can just build the URL myself - no problem. But there's a problem in guix when one guix part generates what another guix part can't process (note: guix import even downloaded the file just fine!).

So the bug is that ‘guix import’ should emit ‘pypi-uri’ calls if and
only if the URL matches the usual pypi.python.org scheme, right?

Ludo’.

Re: guix import pypi uwsgi

[Danny Milosavljevic]

Hi,

On Sat, 19 Mar 2016 21:56:45 +0100
ludo@gnu.org (Ludovic Courtès) wrote:

> So the bug is that ‘guix import’ should emit ‘pypi-uri’ calls if and
> only if the URL matches the usual pypi.python.org scheme, right?

Yes.

guix/import/pypi.scm's make-pypi-sexp even knows the source-url. But then in the source origin it prints '(pypi-uri ,name....

It should only do that if pypi-uri can actually reproduce it, and otherwise use string-append.

That would be easiest if it actually tried to call pypi-uri on its own and checked whether its result is equal to the source-url.

guix/import/pypi.scm even already imports guix/build-system/python.scm and so make-pypi-sexp could call pypi-uri. But it doesn't right now.

Something like this:

diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
index d54bb9f..6889f0e 100644
--- a/guix/import/pypi.scm
+++ b/guix/import/pypi.scm
@@ -194,7 +194,8 @@ VERSION, SOURCE-URL, HOME-PAGE, SYNOPSIS, DESCRIPTION, and LICENSE."
              (version ,version)
              (source (origin
                        (method url-fetch)
-                       (uri (pypi-uri ,name version))
+                       (uri ,(if (equal? (pypi-uri name version) source-url) `(pypi-uri ,name version)
+                                                                             source-url))
                        (sha256
                         (base32
                          ,(guix-hash-url temp)))))


Note that then, it hardcodes the version number. Note sure whether heuristics to extract it from a random URL and repplace it by 'version would be reliable, though.

With the patch, I get:

$ guix import pypi uwsgi
Starting download of /tmp/guix-file.NdDsTJ
From https://pypi.python.org/packages/source/u/uWSGI/uwsgi-2.0.12.tar.gz...
 uwsgi-2.0.12.tar.gz  766KiB        594KiB/s 00:01 [####################] 100.0%
tar: uwsgi-2.0.12/requirements.txt: Not found in archive
tar: Exiting with failure status due to previous errors
guix import: warning: 'tar xf' failed with exit code 512
(package
  (name "python-uwsgi")
  (version "2.0.12")
  (source
    (origin
      (method url-fetch)
      (uri "https://pypi.python.org/packages/source/u/uWSGI/uwsgi-2.0.12.tar.gz")
      (sha256
        (base32
          "02g46dnw5j1iw8fsq392bxbk8d21b9pdgb3ypcinv3b4jzdm2srh"))))
  (build-system python-build-system)
  (inputs
    `(("python-setuptools" ,python-setuptools)))
  (home-page "UNKNOWN")
  (synopsis "The uWSGI server")
  (description "The uWSGI server")
  (license #f))
$ 

For comparison, for a package where pypi-uri works, I get

$ guix import pypi waitress
;;; note: source file /home/dannym/src/guix/guix/import/pypi.scm
;;;       newer than compiled /home/dannym/src/guix/guix/import/pypi.go

Starting download of /tmp/guix-file.4QDL89
From https://pypi.python.org/packages/source/w/waitress/waitress-0.9.0b0.tar.gz...
 waitress-0.9.0b0.tar.gz  125KiB    236KiB/s 00:01 [####################] 100.0%
tar: waitress-0.9.0b0/requirements.txt: Not found in archive
tar: Exiting with failure status due to previous errors
guix import: warning: 'tar xf' failed with exit code 512
(package
  (name "python-waitress")
  (version "0.9.0b0")
  (source
    (origin
      (method url-fetch)
      (uri (pypi-uri "waitress" version))
      (sha256
        (base32
          "019584z449hc3j8q51jbvrp3wjxzzbshid2x2w8xk2xbg5inj151"))))
  (build-system python-build-system)
  (inputs
    `(("python-setuptools" ,python-setuptools)))
  (home-page "https://github.com/Pylons/waitress")
  (synopsis "Waitress WSGI server")
  (description "Waitress WSGI server")
  (license #f))
$ 

Note non-hardcoded version.

Re: guix import pypi uwsgi

[Ludovic Courtès]

Danny Milosavljevic <dannym@scratchpost.org> skribis:

> Something like this:
>
> diff --git a/guix/import/pypi.scm b/guix/import/pypi.scm
> index d54bb9f..6889f0e 100644
> --- a/guix/import/pypi.scm
> +++ b/guix/import/pypi.scm
> @@ -194,7 +194,8 @@ VERSION, SOURCE-URL, HOME-PAGE, SYNOPSIS, DESCRIPTION, and LICENSE."
>               (version ,version)
>               (source (origin
>                         (method url-fetch)
> -                       (uri (pypi-uri ,name version))
> +                       (uri ,(if (equal? (pypi-uri name version) source-url) `(pypi-uri ,name version)
> +                                                                             source-url))
>                         (sha256
>                          (base32
>                           ,(guix-hash-url temp)))))

Indeed.  Pushed something along these lines in commit
522773b70024272555aab0448fae8606add4c582.

Thanks!

Ludo’.