[PATCH] guix: python-build-system: Properly inform caller about test status.

[PATCH] guix: python-build-system: Properly inform caller about test status.

[Danny Milosavljevic]

* guix/build/python-build-system.scm (check): Properly inform caller about test status.
---
 guix/build/python-build-system.scm | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/guix/build/python-build-system.scm b/guix/build/python-build-system.scm
index 3f280b0ac..d46739827 100644
--- a/guix/build/python-build-system.scm
+++ b/guix/build/python-build-system.scm
@@ -136,12 +136,13 @@
       ;; build/lib in some cases, e.g. if the source is in a sub-directory
       ;; (given with `package_dir`). This will by copied to the output, too,
       ;; so we need to remove.
-      (let ((before (find-files "build" "\\.egg-info$" #:directories? #t)))
-        (call-setuppy test-target '() use-setuptools?)
-        (let* ((after (find-files "build" "\\.egg-info$" #:directories? #t))
-               (inter (lset-difference eqv? after before)))
-          (for-each delete-file-recursively inter)))
-    #t))
+      (let* ((before (find-files "build" "\\.egg-info$" #:directories? #t))
+             (status (call-setuppy test-target '() use-setuptools?))
+             (after (find-files "build" "\\.egg-info$" #:directories? #t))
+             (inter (lset-difference eqv? after before)))
+        (for-each delete-file-recursively inter)
+        status)
+      #t))
 
 (define (get-python-version python)
   (let* ((version     (last (string-split python #\-)))

Re: [PATCH] guix: python-build-system: Properly inform caller about test status.

[Hartmut Goebel]

Am 05.01.2017 um 00:30 schrieb Danny Milosavljevic:
>        ;; build/lib in some cases, e.g. if the source is in a sub-directory
>        ;; (given with `package_dir`). This will by copied to the output, too,
>        ;; so we need to remove.

Maybe you could take the change updating my bad English comment :-)

Otherwise LGTM.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: [PATCH] guix: python-build-system: Properly inform caller about test status.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1788 bytes --]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> * guix/build/python-build-system.scm (check): Properly inform caller about test status.
> ---
>  guix/build/python-build-system.scm | 13 +++++++------
>  1 file changed, 7 insertions(+), 6 deletions(-)
>
> diff --git a/guix/build/python-build-system.scm b/guix/build/python-build-system.scm
> index 3f280b0ac..d46739827 100644
> --- a/guix/build/python-build-system.scm
> +++ b/guix/build/python-build-system.scm
> @@ -136,12 +136,13 @@
>        ;; build/lib in some cases, e.g. if the source is in a sub-directory
>        ;; (given with `package_dir`). This will by copied to the output, too,
>        ;; so we need to remove.
> -      (let ((before (find-files "build" "\\.egg-info$" #:directories? #t)))
> -        (call-setuppy test-target '() use-setuptools?)
> -        (let* ((after (find-files "build" "\\.egg-info$" #:directories? #t))
> -               (inter (lset-difference eqv? after before)))
> -          (for-each delete-file-recursively inter)))
> -    #t))
> +      (let* ((before (find-files "build" "\\.egg-info$" #:directories? #t))
> +             (status (call-setuppy test-target '() use-setuptools?))
> +             (after (find-files "build" "\\.egg-info$" #:directories? #t))
> +             (inter (lset-difference eqv? after before)))
> +        (for-each delete-file-recursively inter)
> +        status)
> +      #t))

This is fixed with e46a043ecd9f9ef46d1d44393362bb7016454544 from the
'python-tests' branch. Speaking of which, there are still some failing
packages in that branch listed at:
https://hydra.gnu.org/eval/109407?compare=master#tabs-now-fail

I'll try to take care of the remaining failures in the next week so we
can merge this branch. Contributions are always welcome however ;-)


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [PATCH] guix: python-build-system: Properly inform caller about test status.

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 626 bytes --]

On Thu, Jan 05, 2017 at 02:31:41PM +0100, Marius Bakke wrote:
> Danny Milosavljevic <dannym@scratchpost.org> writes:
> 
> > * guix/build/python-build-system.scm (check): Properly inform caller about test status.

> This is fixed with e46a043ecd9f9ef46d1d44393362bb7016454544 from the
> 'python-tests' branch. Speaking of which, there are still some failing
> packages in that branch listed at:
> https://hydra.gnu.org/eval/109407?compare=master#tabs-now-fail
> 
> I'll try to take care of the remaining failures in the next week so we
> can merge this branch. Contributions are always welcome however ;-)

+1 :)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

python-tests: Sphinx

[Danny Milosavljevic]

Hi Marius,

> This is fixed with e46a043ecd9f9ef46d1d44393362bb7016454544 from the
> 'python-tests' branch. Speaking of which, there are still some failing
> packages in that branch listed at:
> https://hydra.gnu.org/eval/109407?compare=master#tabs-now-fail
> 
> I'll try to take care of the remaining failures in the next week so we
> can merge this branch. Contributions are always welcome however ;-)

Can we update Sphinx in the python-tests branch please? Then some more things will fail...

(The reason I sent the patchsets was because Ludo suggested to test Sphinx breakage in the python-tests branch - that's also why it has this strange split into multiple patchsets...)

Re: python-tests: Sphinx

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 742 bytes --]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> Hi Marius,
>
>> This is fixed with e46a043ecd9f9ef46d1d44393362bb7016454544 from the
>> 'python-tests' branch. Speaking of which, there are still some failing
>> packages in that branch listed at:
>> https://hydra.gnu.org/eval/109407?compare=master#tabs-now-fail
>> 
>> I'll try to take care of the remaining failures in the next week so we
>> can merge this branch. Contributions are always welcome however ;-)
>
> Can we update Sphinx in the python-tests branch please? Then some more things will fail...

Right, I will add them when I get time to work on it in a few days.
Could you make sure they apply, and send an updated patchset with commit
message fixes? TIA!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[PATCH v2 0/6] Make python-oslosphinx work in python-tests branch.

[Danny Milosavljevic]

Preparations to make python-oslosphinx work in python-tests branch.

Danny Milosavljevic (6):
  gnu: python-testtools: Update to 1.4.0.
  gnu: Add python-fastimport.
  gnu: Add python-dulwich.
  gnu: Add python-reno.
  gnu: python-mox3: Remove native-input dependency (since the tests
    don't run anyway).
  gnu: python-sphinx: Update to 1.5.1 and enable tests.

 gnu/packages/openstack.scm |  46 +++++++++++++++++-
 gnu/packages/python.scm    | 114 +++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 149 insertions(+), 11 deletions(-)

[PATCH v2 1/6] gnu: python-testtools: Update to 1.4.0.

[Danny Milosavljevic]

* gnu/packages/python.scm (python-testtools): Update to 1.4.0.
  [source]: Use pypi-uri.
  [arguments]: New field.
---
 gnu/packages/python.scm | 29 ++++++++++++++++++++++++-----
 1 file changed, 24 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 9a44b5881..d53a10116 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -1982,17 +1982,36 @@ subprocess and see the output as well as any file modifications.")
 (define-public python-testtools
   (package
     (name "python-testtools")
-    (version "1.0.0")
+    (version "1.4.0")
     (source
      (origin
        (method url-fetch)
-       (uri (string-append
-             "https://pypi.python.org/packages/source/t/testtools/testtools-"
-             version ".tar.gz"))
+       (uri (pypi-uri "testtools" version))
        (sha256
         (base32
-         "1dyml28ykpl5jb9khdmcdvhy1cxqingys6qvj2k04fzlaj6z3bbx"))))
+         "1vw8yljnd75d396hhw6s2hrf4cclzy845ifd5am0lxsl235z3i8c"))))
     (build-system python-build-system)
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'fix-module-imports
+           (lambda _
+             (substitute* "setup.py"
+               (("'unittest2>=0.8.0',") ""))
+             (substitute* '("testtools/testcase.py"
+                            "testtools/testsuite.py"
+                            "testtools/run.py"
+                            "testtools/tests/test_run.py"
+                            "testtools/tests/test_testsuite.py"
+                            "testtools/tests/test_deferredruntest.py")
+               ;; unittest2 is a backport of Python2.7 features to Python 2.4.
+               (("import unittest2 as unittest") "import unittest")
+               (("import unittest2") "import unittest as unittest2")
+               (("from unittest2 import") "from unittest import"))
+             (substitute* "testtools/tests/test_testresult.py"
+               ;; NUL in source code is not allowed (raises ValueError).
+               (("\\x00\\x04") "\\x04"))
+             #t)))))
     (propagated-inputs
      `(("python-mimeparse" ,python-mimeparse)
        ("python-extras" ,python-extras)))

Re: [PATCH v2 1/6] gnu: python-testtools: Update to 1.4.0.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 162 bytes --]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> * gnu/packages/python.scm (python-testtools): Update to 1.4.0.

Why not go with the latest version 2.2.0?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [PATCH v2 1/6] gnu: python-testtools: Update to 1.4.0.

[Danny Milosavljevic]

On Wed, 18 Jan 2017 18:39:24 +0100
Marius Bakke <mbakke@fastmail.com> wrote:

> Danny Milosavljevic <dannym@scratchpost.org> writes:
> 
> > * gnu/packages/python.scm (python-testtools): Update to 1.4.0.  
> 
> Why not go with the latest version 2.2.0?

It depends on traceback2 backports etc which means for our Python3 version I have to patch out the traceback2 imports for Python3 packages only - it's more complicated than 1.4.0. If it's OK let's update to 1.4.0 now and update to 2.2.0 later? Otherwise I have the feeling I'd have to update and patch all Python packages before anything works :)

But let's keep in mind that we should update testtools to 2.2.0 later.

Re: [PATCH v2 1/6] gnu: python-testtools: Update to 1.4.0.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1077 bytes --]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> On Wed, 18 Jan 2017 18:39:24 +0100
> Marius Bakke <mbakke@fastmail.com> wrote:
>
>> Danny Milosavljevic <dannym@scratchpost.org> writes:
>> 
>> > * gnu/packages/python.scm (python-testtools): Update to 1.4.0.  
>> 
>> Why not go with the latest version 2.2.0?
>
> It depends on traceback2 backports etc which means for our Python3
> version I have to patch out the traceback2 imports for Python3
> packages only - it's more complicated than 1.4.0.

The traceback2 module works on python3 as well. I don't think patching
imports will be necessary?

> If it's OK let's update to 1.4.0 now and update to 2.2.0 later?
> Otherwise I have the feeling I'd have to update and patch all Python
> packages before anything works :)

There is a moderate chance testtools-1.4.0, requests-2.12.4 and
sphinx-1.5.1 requires updating a bunch of python packages anyway :)

But I see we don't currently have traceback2, so I'm fine with 1.4.0
for now just to avoid packaging it. The rest LGTM, let's get this in!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

python2-traceback2, python2-linecache2

[Danny Milosavljevic]

Hi,

> The traceback2 module works on python3 as well. I don't think patching
> imports will be necessary?

Maybe it works. However, I think it's silly to package a "traceback" module for Python 3.5 which is a backport of the "traceback" module included in the CPython 3.5 interpreter when we use the CPython 3.5 interpreter in the first place.

Same for python-linecache2.

Therefore, I would have another patchset which adds them for Python 2 only and patches them out otherwise (s/traceback2/traceback/g).

Adding them for Python 3.5 would just be asking for them to become outdated for no reason (they are part of Python 3.5 and will update with it - their origin is Python 3.5).

That said, I'm happy that this is being discussed - I'm not sure what the right way to go forward is.

>But I see we don't currently have traceback2, so I'm fine with 1.4.0 for now just to avoid packaging it. The rest LGTM, let's get this [python-testtools] in!

OK!

Re: python2-traceback2, python2-linecache2

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1747 bytes --]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> Hi,
>
>> The traceback2 module works on python3 as well. I don't think patching
>> imports will be necessary?
>
> Maybe it works. However, I think it's silly to package a "traceback"
> module for Python 3.5 which is a backport of the "traceback" module
> included in the CPython 3.5 interpreter when we use the CPython 3.5
> interpreter in the first place.

There are other modules extracted from python3 and packaged for
python2+3. "enum34", "contextlib2", etc. AFAICT traceback was added
already in 3.0: https://docs.python.org/3.0/library/traceback.html

> Same for python-linecache2.
>
> Therefore, I would have another patchset which adds them for Python 2
> only and patches them out otherwise (s/traceback2/traceback/g).

If a module written for python3 imports "traceback2", that won't work
anyway with the standard library since it's called just "traceback". I
would much rather package a "superfluous" module, instead of
conditionally patching packages to use the built-in for python3 only.

> Adding them for Python 3.5 would just be asking for them to become
> outdated for no reason (they are part of Python 3.5 and will update
> with it - their origin is Python 3.5).
>
> That said, I'm happy that this is being discussed - I'm not sure what
> the right way to go forward is.

Me neither :-) but it's obviously something we need to tackle
eventually. Copied in Hartmut, maybe he's got some insight?

>>But I see we don't currently have traceback2, so I'm fine with 1.4.0
>>for now just to avoid packaging it. The rest LGTM, let's get this
>>[python-testtools] in!
>
> OK!

...and the rest of this series ;-)

I wonder if we should update scipy and numpy while at it. Thoughts?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: python2-traceback2, python2-linecache2

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 452 bytes --]

Marius Bakke <mbakke@fastmail.com> writes:

> I wonder if we should update scipy and numpy while at it. Thoughts?

Ugh. I tried this out (plus matplotlib), but the latest matplotlib
(2.0.0) has a circular dependency on ipython. Great.

The motivation is that the scipy failure stems from the Sphinx update.
I think I'll add an older Sphinx for now to get the ball rolling again;
we should start a new 'python-updates' branch as soon as this is merged.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

'python-tests' merge

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 705 bytes --]


>> I wonder if we should update scipy and numpy while at it. Thoughts?
>
> Ugh. I tried this out (plus matplotlib), but the latest matplotlib
> (2.0.0) has a circular dependency on ipython. Great.
>
> The motivation is that the scipy failure stems from the Sphinx update.
> I think I'll add an older Sphinx for now to get the ball rolling again;
> we should start a new 'python-updates' branch as soon as this is merged.

Turns out sphinx-1.5.1 broke matplotlib again. I've changed both
matplotlib and scipy to use python-sphinx@1.2.3 in a50e030141.

@Leo: Can you start a new evaluation? Hopefully we can get this merged
before the next 'core-updates' evaluation. Alternatively merge it to
that branch.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: python2-traceback2, python2-linecache2

[Hartmut Goebel]

Am 19.01.2017 um 16:01 schrieb Marius Bakke:
> Me neither :-) but it's obviously something we need to tackle
> eventually. Copied in Hartmut, maybe he's got some insight?

Sorry. I posted my answer in a follow up to another mail of this thread.

For the records: I'm afraid we need to package the modules for the next
time. I filed some bug reports there.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: python2-traceback2, python2-linecache2

[Hartmut Goebel]

Am 19.01.2017 um 15:34 schrieb Danny Milosavljevic:
> Maybe it works. However, I think it's silly to package a "traceback" module for Python 3.5 which is a backport of the "traceback" module included in the CPython 3.5 interpreter when we use the CPython 3.5 interpreter in the first place.

Well, the root cause are badly written packages. We ought address this,
but it's not the primary packages job.

I'm in favour of keeping traceback2 and linecache2, since I'm afraid
there might be more package requiring them.

I already files some bug-reports to these projects:

https://github.com/testing-cabal/testtools/issues/246
https://github.com/testing-cabal/testtools/issues/247
https://github.com/testing-cabal/traceback2/issues/10
https://github.com/testing-cabal/linecache2/issues/13

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: [PATCH v2 1/6] gnu: python-testtools: Update to 1.4.0.

[Danny Milosavljevic]

Committed to python-tests as 7adb90311f8ce5e44c5bd1ecba28dcb26bfabebe .

[PATCH v2 2/6] gnu: Add python-fastimport.

[Danny Milosavljevic]

* gnu/packages/python.scm (python-fastimport, python2-fastimport):
  New variables.
---
 gnu/packages/python.scm | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index d53a10116..6c7a6353d 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -10551,6 +10551,27 @@ concurrent HTTP client library for python using @code{gevent}.")
 (define-public python2-geventhttpclient
   (package-with-python2 python-geventhttpclient))
 
+(define-public python-fastimport
+  (package
+    (name "python-fastimport")
+    (version "0.9.6")
+    (source
+      (origin
+        (method url-fetch)
+        (uri (pypi-uri "fastimport" version))
+        (sha256
+          (base32 "1aqjsin4rmqm7ln4j0p73fzxifws6c6ikgyhav7r137m2ixsxl43"))))
+    (build-system python-build-system)
+    (home-page "https://github.com/jelmer/python-fastimport")
+    (synopsis "VCS fastimport parser and generator in Python")
+    (description "This package provides a parser for and generator of the Git
+@url{https://www.kernel.org/pub/software/scm/git/docs/git-fast-import.html,fastimport}
+format.")
+    (license license:gpl2+)))
+
+(define-public python2-fastimport
+  (package-with-python2 python-fastimport))
+
 (define-public python-twisted
   (package
     (name "python-twisted")

Re: [PATCH v2 2/6] gnu: Add python-fastimport.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 149 bytes --]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> * gnu/packages/python.scm (python-fastimport, python2-fastimport):
>   New variables.

LGTM!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [PATCH v2 2/6] gnu: Add python-fastimport.

[Danny Milosavljevic]

Committed as b585e361844e0ab527d10880698ee0489347b1c6 to python-tests.

[PATCH v2 3/6] gnu: Add python-dulwich.

[Danny Milosavljevic]

* gnu/packages/python.scm (python-dulwich, python2-dulwich): New variables.
---
 gnu/packages/python.scm | 44 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 6c7a6353d..b064c9f47 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -12740,3 +12740,47 @@ Features:
 @item Compiles templates into optimized, yet readable, Python code.
 @end enumerate")
     (license (license:x11-style "file://LICENSE"))))
+
+(define-public python-dulwich
+  (package
+    (name "python-dulwich")
+    (version "0.16.3")
+    (source
+      (origin
+        (method url-fetch)
+        (uri (pypi-uri "dulwich" version))
+        (sha256
+          (base32 "0fl47vzfgc3w3rmhn8naii905cjqcp0vc68iyvymxp7567hh6als"))))
+    (build-system python-build-system)
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (add-before 'check 'fix-tests
+           (lambda* (#:key inputs #:allow-other-keys)
+             (substitute* "dulwich/tests/compat/utils.py"
+               (("'git'") (string-append "'"
+                                         (assoc-ref inputs "git")
+                                         "/bin/git'")))
+
+
+             (substitute* '("dulwich/tests/test_repository.py"
+                            "dulwich/tests/test_hooks.py")
+               (("#!/bin/sh") (string-append "#!" (which "sh"))))
+             (setenv "TEST_RUNNER" "unittest")
+             (setenv "PYTHONHASHSEED" "random")
+             #t)))))
+    (propagated-inputs
+     `(("python-fastimport" ,python-fastimport)))
+    (native-inputs
+     `(("python-mock" ,python-mock)
+       ("python-geventhttpclient" ,python-geventhttpclient)
+       ("git" ,git)))
+    (home-page "https://www.dulwich.io/")
+    (synopsis "Git implementation in Python")
+    (description "Dulwich is an implementation of the Git file formats and
+protocols written in pure Python.")
+    ;; dual-licensed
+    (license (list license:asl2.0 license:gpl2+))))
+
+(define-public python2-dulwich
+  (package-with-python2 python-dulwich))

Re: [PATCH v2 3/6] gnu: Add python-dulwich.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 2570 bytes --]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> * gnu/packages/python.scm (python-dulwich, python2-dulwich): New variables.
> ---
>  gnu/packages/python.scm | 44 ++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 44 insertions(+)
>
> diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
> index 6c7a6353d..b064c9f47 100644
> --- a/gnu/packages/python.scm
> +++ b/gnu/packages/python.scm
> @@ -12740,3 +12740,47 @@ Features:
>  @item Compiles templates into optimized, yet readable, Python code.
>  @end enumerate")
>      (license (license:x11-style "file://LICENSE"))))
> +
> +(define-public python-dulwich
> +  (package
> +    (name "python-dulwich")
> +    (version "0.16.3")
> +    (source
> +      (origin
> +        (method url-fetch)
> +        (uri (pypi-uri "dulwich" version))

Upstream provides the same releaases on their home page. I would add

  (uri (list (string-append "https://www.dulwich.io/releases/dulwich-"
                            version ".tar.gz")
             (pypi-uri "cram" version)))

> +        (sha256
> +          (base32 "0fl47vzfgc3w3rmhn8naii905cjqcp0vc68iyvymxp7567hh6als"))))
> +    (build-system python-build-system)
> +    (arguments
> +     `(#:phases
> +       (modify-phases %standard-phases
> +         (add-before 'check 'fix-tests
> +           (lambda* (#:key inputs #:allow-other-keys)
> +             (substitute* "dulwich/tests/compat/utils.py"
> +               (("'git'") (string-append "'"
> +                                         (assoc-ref inputs "git")
> +                                         "/bin/git'")))

Why is this necessary? Git is available in PATH, no?

> +             (substitute* '("dulwich/tests/test_repository.py"
> +                            "dulwich/tests/test_hooks.py")
> +               (("#!/bin/sh") (string-append "#!" (which "sh"))))
> +             (setenv "TEST_RUNNER" "unittest")
> +             (setenv "PYTHONHASHSEED" "random")
> +             #t)))))
> +    (propagated-inputs
> +     `(("python-fastimport" ,python-fastimport)))
> +    (native-inputs
> +     `(("python-mock" ,python-mock)
> +       ("python-geventhttpclient" ,python-geventhttpclient)
> +       ("git" ,git)))
> +    (home-page "https://www.dulwich.io/")
> +    (synopsis "Git implementation in Python")
> +    (description "Dulwich is an implementation of the Git file formats and
> +protocols written in pure Python.")
> +    ;; dual-licensed
> +    (license (list license:asl2.0 license:gpl2+))))

Please clarify whether it's both, or either. Otherwise LGTM, thanks!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [PATCH v2 3/6] gnu: Add python-dulwich.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 3097 bytes --]

On Wed, Jan 18, 2017 at 06:48:46PM +0100, Marius Bakke wrote:
> Danny Milosavljevic <dannym@scratchpost.org> writes:
> 
> > * gnu/packages/python.scm (python-dulwich, python2-dulwich): New variables.
> > ---
> >  gnu/packages/python.scm | 44 ++++++++++++++++++++++++++++++++++++++++++++
> >  1 file changed, 44 insertions(+)
> >
> > diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
> > index 6c7a6353d..b064c9f47 100644
> > --- a/gnu/packages/python.scm
> > +++ b/gnu/packages/python.scm
> > @@ -12740,3 +12740,47 @@ Features:
> >  @item Compiles templates into optimized, yet readable, Python code.
> >  @end enumerate")
> >      (license (license:x11-style "file://LICENSE"))))
> > +
> > +(define-public python-dulwich
> > +  (package
> > +    (name "python-dulwich")
> > +    (version "0.16.3")
> > +    (source
> > +      (origin
> > +        (method url-fetch)
> > +        (uri (pypi-uri "dulwich" version))
> 
> Upstream provides the same releaases on their home page. I would add
> 
>   (uri (list (string-append "https://www.dulwich.io/releases/dulwich-"
>                             version ".tar.gz")
>              (pypi-uri "cram" version)))
> 
> > +        (sha256
> > +          (base32 "0fl47vzfgc3w3rmhn8naii905cjqcp0vc68iyvymxp7567hh6als"))))
> > +    (build-system python-build-system)
> > +    (arguments
> > +     `(#:phases
> > +       (modify-phases %standard-phases
> > +         (add-before 'check 'fix-tests
> > +           (lambda* (#:key inputs #:allow-other-keys)
> > +             (substitute* "dulwich/tests/compat/utils.py"
> > +               (("'git'") (string-append "'"
> > +                                         (assoc-ref inputs "git")
> > +                                         "/bin/git'")))
> 
> Why is this necessary? Git is available in PATH, no?

or alternatively, would (which "git") work?

> 
> > +             (substitute* '("dulwich/tests/test_repository.py"
> > +                            "dulwich/tests/test_hooks.py")
> > +               (("#!/bin/sh") (string-append "#!" (which "sh"))))
> > +             (setenv "TEST_RUNNER" "unittest")
> > +             (setenv "PYTHONHASHSEED" "random")
> > +             #t)))))
> > +    (propagated-inputs
> > +     `(("python-fastimport" ,python-fastimport)))
> > +    (native-inputs
> > +     `(("python-mock" ,python-mock)
> > +       ("python-geventhttpclient" ,python-geventhttpclient)
> > +       ("git" ,git)))
> > +    (home-page "https://www.dulwich.io/")
> > +    (synopsis "Git implementation in Python")
> > +    (description "Dulwich is an implementation of the Git file formats and
> > +protocols written in pure Python.")
> > +    ;; dual-licensed
> > +    (license (list license:asl2.0 license:gpl2+))))
> 
> Please clarify whether it's both, or either. Otherwise LGTM, thanks!



-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v2 3/6] gnu: Add python-dulwich.

[Danny Milosavljevic]

> or alternatively, would (which "git") work?

Yes. I'll use it.

Re: [PATCH v2 3/6] gnu: Add python-dulwich.

[Danny Milosavljevic]

> Why is this necessary? Git is available in PATH, no?

Yes, but their tests use Popen with a custom environment which doesn't include the PATH variable at all.

Also, if one messes it up the git invocation tests will be skipped and "succeed"!

Re: [PATCH v2 3/6] gnu: Add python-dulwich.

[Danny Milosavljevic]

Committed as d83d87225980cec4926aec03280ad3533b778281 to python-tests.

[PATCH v2 4/6] gnu: Add python-reno.

[Danny Milosavljevic]

* gnu/packages/openstack.scm (python-reno, python-reno2): New variables.
---
 gnu/packages/openstack.scm | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/gnu/packages/openstack.scm b/gnu/packages/openstack.scm
index b3e3f08c2..0e50236af 100644
--- a/gnu/packages/openstack.scm
+++ b/gnu/packages/openstack.scm
@@ -19,6 +19,7 @@
 ;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
 
 (define-module (gnu packages openstack)
+  #:use-module (gnu packages gnupg)
   #:use-module (gnu packages python)
   #:use-module (gnu packages ssh)
   #:use-module (gnu packages tls)
@@ -550,6 +551,47 @@ in transmittable and storable formats, such as JSON and MessagePack.")
 (define-public python2-oslo.serialization
   (package-with-python2 python-oslo.serialization))
 
+(define-public python-reno
+  (package
+    (name "python-reno")
+    (version "2.0.3")
+    (source
+      (origin
+        (method url-fetch)
+        (uri (pypi-uri "reno" version))
+        (sha256
+          (base32 "1i2wnn5fnm3jm5774pahg000q0lma5i913hml91bbbm2mybphndd"))))
+    (build-system python-build-system)
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (add-before 'check 'init-git
+           (lambda _
+             ;; reno expects a git repo
+             (zero? (system* "git" "init")))))))
+    (propagated-inputs
+      `(("python-babel" ,python-babel)
+        ("python-dulwich" ,python-dulwich)
+        ("python-pyyaml" ,python-pyyaml)
+        ("python-six" ,python-six)))
+    (native-inputs
+      `(("python-testtools" ,python-testtools)
+        ("python-pbr" ,python-pbr)
+        ("python-testscenarios" ,python-testscenarios)
+        ("python-testrepository" ,python-testrepository)
+        ("python-mock" ,python-mock)
+        ("python-oslotest" ,python-oslotest)
+        ("gnupg" ,gnupg)
+        ("git" ,git)))
+    (home-page "http://docs.openstack.org/developer/reno/")
+    (synopsis "Release notes manager")
+    (description "Reno is a tool for storing release notes in a git repository
+and building documentation from them.")
+    (license asl2.0)))
+
+(define-public python2-reno
+  (package-with-python2 python-reno))
+
 (define-public python-oslosphinx
   (package
     (name "python-oslosphinx")

Re: [PATCH v2 4/6] gnu: Add python-reno.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 2624 bytes --]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> * gnu/packages/openstack.scm (python-reno, python-reno2): New variables.
> ---
>  gnu/packages/openstack.scm | 42 ++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 42 insertions(+)
>
> diff --git a/gnu/packages/openstack.scm b/gnu/packages/openstack.scm
> index b3e3f08c2..0e50236af 100644
> --- a/gnu/packages/openstack.scm
> +++ b/gnu/packages/openstack.scm
> @@ -19,6 +19,7 @@
>  ;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
>  
>  (define-module (gnu packages openstack)
> +  #:use-module (gnu packages gnupg)
>    #:use-module (gnu packages python)
>    #:use-module (gnu packages ssh)
>    #:use-module (gnu packages tls)
> @@ -550,6 +551,47 @@ in transmittable and storable formats, such as JSON and MessagePack.")
>  (define-public python2-oslo.serialization
>    (package-with-python2 python-oslo.serialization))
>  
> +(define-public python-reno
> +  (package
> +    (name "python-reno")
> +    (version "2.0.3")
> +    (source
> +      (origin
> +        (method url-fetch)
> +        (uri (pypi-uri "reno" version))
> +        (sha256
> +          (base32 "1i2wnn5fnm3jm5774pahg000q0lma5i913hml91bbbm2mybphndd"))))
> +    (build-system python-build-system)
> +    (arguments
> +     `(#:phases
> +       (modify-phases %standard-phases
> +         (add-before 'check 'init-git
> +           (lambda _
> +             ;; reno expects a git repo
> +             (zero? (system* "git" "init")))))))

Heh. Why doesn't reno init on its own, then. Oh well.

LGTM!

> +    (propagated-inputs
> +      `(("python-babel" ,python-babel)
> +        ("python-dulwich" ,python-dulwich)
> +        ("python-pyyaml" ,python-pyyaml)
> +        ("python-six" ,python-six)))
> +    (native-inputs
> +      `(("python-testtools" ,python-testtools)
> +        ("python-pbr" ,python-pbr)
> +        ("python-testscenarios" ,python-testscenarios)
> +        ("python-testrepository" ,python-testrepository)
> +        ("python-mock" ,python-mock)
> +        ("python-oslotest" ,python-oslotest)
> +        ("gnupg" ,gnupg)
> +        ("git" ,git)))
> +    (home-page "http://docs.openstack.org/developer/reno/")
> +    (synopsis "Release notes manager")
> +    (description "Reno is a tool for storing release notes in a git repository
> +and building documentation from them.")
> +    (license asl2.0)))
> +
> +(define-public python2-reno
> +  (package-with-python2 python-reno))
> +
>  (define-public python-oslosphinx
>    (package
>      (name "python-oslosphinx")

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [PATCH v2 4/6] gnu: Add python-reno.

[Danny Milosavljevic]

Committed to python-tests as c8d7e063d0b576615fb481e0abc55c0aeab609b8

Re: [PATCH v2 4/6] gnu: Add python-reno.

[Danny Milosavljevic]

> * gnu/packages/openstack.scm (python-reno, python-reno2): New variables.
                                                   ^^^ typo here. 

I committed and pushed it already. Sorry.

[PATCH v2 5/6] gnu: python-mox3: Remove python-oslosphinx dependency (since the tests don't run anyway).

[Danny Milosavljevic]

* gnu/packages/python.scm (python-mox3)[native-inputs]: Remove python-oslosphinx.
---
 gnu/packages/openstack.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/openstack.scm b/gnu/packages/openstack.scm
index 0e50236af..277a443b0 100644
--- a/gnu/packages/openstack.scm
+++ b/gnu/packages/openstack.scm
@@ -153,11 +153,11 @@ guidelines}.")
            "0njmh40i1lg5mzn9hc2ax83adj6dli455j6xifilrw27c4wlkjzx"))))
     (build-system python-build-system)
     (arguments
-     ;; TODO: Requires newer version of testtools (>=1.4.0).
+     ;; TODO: Resolve dependency cycle and re-enable.
      '(#:tests? #f))
     (native-inputs
       `(("python-fixtures" ,python-fixtures)
-        ("python-oslosphinx" ,python-oslosphinx)
+        ;("python-oslosphinx" ,python-oslosphinx)
         ("python-pbr" ,python-pbr)
         ("python-sphinx" ,python-sphinx)
         ("python-testtools" ,python-testtools)))

Re: [PATCH v2 5/6] gnu: python-mox3: Remove python-oslosphinx dependency (since the tests don't run anyway).

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1288 bytes --]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> * gnu/packages/python.scm (python-mox3)[native-inputs]: Remove python-oslosphinx.
                 ^^^^^^^^^^
                 This path seems wrong :)

Otherwise LGTM. If you look around in openstack.scm, I've had to do
similar things in a bunch of packages. We should try to update all of
them, after this branch is merged :-)


> ---
>  gnu/packages/openstack.scm | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/gnu/packages/openstack.scm b/gnu/packages/openstack.scm
> index 0e50236af..277a443b0 100644
> --- a/gnu/packages/openstack.scm
> +++ b/gnu/packages/openstack.scm
> @@ -153,11 +153,11 @@ guidelines}.")
>             "0njmh40i1lg5mzn9hc2ax83adj6dli455j6xifilrw27c4wlkjzx"))))
>      (build-system python-build-system)
>      (arguments
> -     ;; TODO: Requires newer version of testtools (>=1.4.0).
> +     ;; TODO: Resolve dependency cycle and re-enable.
>       '(#:tests? #f))
>      (native-inputs
>        `(("python-fixtures" ,python-fixtures)
> -        ("python-oslosphinx" ,python-oslosphinx)
> +        ;("python-oslosphinx" ,python-oslosphinx)
>          ("python-pbr" ,python-pbr)
>          ("python-sphinx" ,python-sphinx)
>          ("python-testtools" ,python-testtools)))

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [PATCH v2 5/6] gnu: python-mox3: Remove python-oslosphinx dependency (since the tests don't run anyway).

[Danny Milosavljevic]

> > * gnu/packages/python.scm (python-mox3)[native-inputs]: Remove python-oslosphinx.  
>                  ^^^^^^^^^^
>                  This path seems wrong :)

Fixed and committed as 376c4619cf8ad683fba5d04513a5298c0a77cfdb to python-tests.

[PATCH v2 6/6] gnu: python-sphinx: Update to 1.5.1 and enable tests.

[Danny Milosavljevic]

* gnu/packages/python.scm (python-sphinx): Update to 1.5.1.
  [arguments]: New field.
  [native-inputs]: Add graphviz, python-html5lib, python-mock, python-nose,
  python-requests.
---
 gnu/packages/python.scm | 20 ++++++++++++++++----
 1 file changed, 16 insertions(+), 4 deletions(-)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index b064c9f47..3b18ed441 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -67,6 +67,7 @@
   #:use-module (gnu packages ghostscript)
   #:use-module (gnu packages gl)
   #:use-module (gnu packages glib)
+  #:use-module (gnu packages graphviz)
   #:use-module (gnu packages gstreamer)
   #:use-module (gnu packages gtk)
   #:use-module (gnu packages icu4c)
@@ -2964,18 +2965,23 @@ reStructuredText.")
 (define-public python-sphinx
   (package
     (name "python-sphinx")
-    (version "1.4.8")
+    (version "1.5.1")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "Sphinx" version))
        (sha256
         (base32
-         "0zvh8wwhm6gy21rr0cg42znsy4zzv2mnsxxk9gmn5y1ycn7rgbs1"))))
+         "1i8p9idnli4gr0y4x67yakbdk5w6a0xjzhrg6bg51y9d1fi7fslf"))))
     (build-system python-build-system)
     (arguments
-     ;; FIXME: Missing dependencies.
-     '(#:tests? #f))
+     `(#:phases
+       (modify-phases %standard-phases
+         (replace 'check
+           (lambda _
+             ;; Requires Internet access.
+             (delete-file "tests/test_build_linkcheck.py")
+             (zero? (system* "make" "test")))))))
     (propagated-inputs
      `(("python-imagesize" ,python-imagesize)
        ("python-sphinx-alabaster-theme"
@@ -2986,6 +2992,12 @@ reStructuredText.")
        ("python-jinja2" ,python-jinja2)
        ("python-pygments" ,python-pygments)
        ("python-six" ,python-six)))
+    (native-inputs
+     `(("graphviz" ,graphviz)
+       ("python-html5lib" ,python-html5lib)
+       ("python-mock" ,python-mock)
+       ("python-nose" ,python-nose)
+       ("python-requests" ,python-requests)))
     (home-page "http://sphinx-doc.org/")
     (synopsis "Python documentation generator")
     (description "Sphinx is a tool that makes it easy to create documentation

Re: [PATCH v2 6/6] gnu: python-sphinx: Update to 1.5.1 and enable tests.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 258 bytes --]

Danny Milosavljevic <dannym@scratchpost.org> writes:

> * gnu/packages/python.scm (python-sphinx): Update to 1.5.1.
>   [arguments]: New field.
>   [native-inputs]: Add graphviz, python-html5lib, python-mock, python-nose,
>   python-requests.

LGTM, thanks!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: [PATCH v2 6/6] gnu: python-sphinx: Update to 1.5.1 and enable tests.

[Danny Milosavljevic]

Commited to python-tests as c227458ac84273c12c254827e3311e9df2798767. Adapted python2-sphinx to include python2-enum34 to make their tests work in the same commit.