[bug#34204] [PATCH 0/3] Mkdocs and its dependencies

[bug#34204] [PATCH 0/3] Mkdocs and its dependencies

[Mathieu Lirzin]

Hello,

Here are a sequence of patches which adds the Mkdocs documentation generator
to the set of Guix packages.

Mathieu Lirzin (3):
  gnu: python-tornado: Update to 5.1.1.
  gnu: Add Python LiveReload.
  gnu: Add Mkdocs.

 gnu/packages/documentation.scm | 35 ++++++++++++++++++++++++++++++++++
 gnu/packages/python-web.scm    | 27 ++++++++++++++++++++++++--
 2 files changed, 60 insertions(+), 2 deletions(-)

-- 
2.20.1

[bug#34204] [PATCH 1/3] gnu: python-tornado: Update to 5.1.1.

[Mathieu Lirzin]

* gnu/packages/python-web.scm (python-tornado): Update to 5.1.1.
---
 gnu/packages/python-web.scm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index cee4973086..19ff7d2dc4 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -27,6 +27,7 @@
 ;;; Copyright © 2018 Nicolas Goaziou <mail@nicolasgoaziou.fr>
 ;;; Copyright © 2018 Mathieu Othacehe <m.othacehe@gmail.com>
 ;;; Copyright © 2018 Maxim Cournoyer <maxim.cournoyer@gmail.com>
+;;; Copyright © 2019 Mathieu Lirzin <mthl@gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -665,13 +666,13 @@ teams extension for python-openid.")
 (define-public python-tornado
   (package
     (name "python-tornado")
-    (version "4.5.1")
+    (version "5.1.1")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "tornado" version))
        (sha256
-        (base32 "1zbkgcdfq81k298awrm8p0xwbwwn2p3nbizdglzfbkskhai082fv"))))
+        (base32 "02clqk2116jbnq8lnaqmdw3p52nqrd9ib59r4xz2ll43fpcmhlaf"))))
     (build-system python-build-system)
     (arguments
      '(;; FIXME: Two tests error out with:
-- 
2.20.1

[bug#34204] [PATCH 2/3] gnu: Add Python LiveReload.

[Mathieu Lirzin]

* gnu/packages/python-web.scm (python-livereload): New variable.
---
 gnu/packages/python-web.scm | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/gnu/packages/python-web.scm b/gnu/packages/python-web.scm
index 19ff7d2dc4..24fa4ec60a 100644
--- a/gnu/packages/python-web.scm
+++ b/gnu/packages/python-web.scm
@@ -2923,3 +2923,25 @@ underlies Mozilla Persona.")
      "This is a Python library for interacting with the Firefox Accounts
 ecosystem.")
     (license license:mpl2.0)))
+
+(define-public python-livereload
+  (package
+    (name "python-livereload")
+    (version "2.6.0")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "livereload" version))
+              (sha256
+               (base32
+                "19s9lrzq33ds3d580q99y0zg8f1vhyz6afpisz0mb49l3p6sccp6"))))
+    (build-system python-build-system)
+    (propagated-inputs
+     `(("python-six" ,python-six)
+       ("python-tornado" ,python-tornado)))
+    (home-page "https://github.com/lepture/python-livereload")
+    (synopsis "Live reload server in Python")
+    (description
+     "This provides the @code{livereload} command line utility for starting a
+server in a directory.  By default, it listens to port 35729 which is the
+common port for LiveReload browser extensions.")
+    (license license:bsd-3)))
-- 
2.20.1

[bug#34204] [PATCH 3/3] gnu: Add Mkdocs.

[Mathieu Lirzin]

* gnu/packages/documentation.scm (mkdocs): New variable.
---
 gnu/packages/documentation.scm | 35 ++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)

diff --git a/gnu/packages/documentation.scm b/gnu/packages/documentation.scm
index 9425df707c..2eb1f29013 100644
--- a/gnu/packages/documentation.scm
+++ b/gnu/packages/documentation.scm
@@ -7,6 +7,7 @@
 ;;; Copyright © 2017 Kei Kebreau <kkebreau@posteo.net>
 ;;; Copyright © 2017 Efraim Flashner <efraim@flashner.co.il>
 ;;; Copyright © 2018 Tobias Geerinckx-Rice <me@tobias.gr>
+;;; Copyright © 2019 Mathieu Lirzin <mthl@gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -29,10 +30,13 @@
   #:use-module (guix download)
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system cmake)
+  #:use-module (guix build-system python)
   #:use-module (gnu packages)
   #:use-module (gnu packages autotools)
   #:use-module (gnu packages bash)
   #:use-module (gnu packages python)
+  #:use-module (gnu packages python-web)
+  #:use-module (gnu packages python-xyz)
   #:use-module (gnu packages bison)
   #:use-module (gnu packages docbook)
   #:use-module (gnu packages flex)
@@ -219,3 +223,34 @@ sort, and search the document catalog.  It will also be able to communicate
 with catalog servers on the Net to search for documents which are not on the
 local system.")
     (license lgpl2.1+)))
+
+(define-public mkdocs
+  (package
+    (name "mkdocs")
+    (version "1.0.4")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "mkdocs" version))
+       (sha256
+        (base32
+         "0fg9w6rdskwnn7knri7xzrd26k9svwqlxvdr0kk5spfpm8ll7lqp"))))
+    (build-system python-build-system)
+    (arguments
+     ;; XXX: Tests are failing.
+     ;; AttributeError: module 'mkdocs.plugins' has no attribute 'get_plugins'
+     '(#:tests? #f))
+    (propagated-inputs
+     `(("python-click" ,python-click)
+       ("python-jinja2" ,python-jinja2)
+       ("python-livereload" ,python-livereload)
+       ("python-markdown" ,python-markdown)
+       ("python-pyyaml" ,python-pyyaml)
+       ("python-tornado" ,python-tornado)))
+    (home-page "https://www.mkdocs.org")
+    (synopsis "Project documentation with Markdown")
+    (description
+     "This is a fast and simple static site generator that's geared towards
+building project documentation.  Documentation source files are written in
+Markdown, and configured with a single YAML configuration file.")
+    (license bsd-3)))
-- 
2.20.1

[bug#34204] [PATCH 1/3] gnu: python-tornado: Update to 5.1.1.

[Ricardo Wurmus]

Hi Mathieu,

> * gnu/packages/python-web.scm (python-tornado): Update to 5.1.1.

Thanks for the patch!

>  (define-public python-tornado
>    (package
>      (name "python-tornado")
> -    (version "4.5.1")
> +    (version "5.1.1")
>      (source
>       (origin
>         (method url-fetch)
>         (uri (pypi-uri "tornado" version))
>         (sha256
> -        (base32 "1zbkgcdfq81k298awrm8p0xwbwwn2p3nbizdglzfbkskhai082fv"))))
> +        (base32 "02clqk2116jbnq8lnaqmdw3p52nqrd9ib59r4xz2ll43fpcmhlaf"))))
>      (build-system python-build-system)
>      (arguments
>       '(;; FIXME: Two tests error out with:

The arguments field says:

    (arguments
     '(;; FIXME: Two tests error out with:
       ;; AssertionError: b'Error in atexit._run_exitfuncs:\nFileNotF[44 chars]ry\n' != b''
       ;; #:phases
       ;; (modify-phases %standard-phases
       ;;   (replace 'check
       ;;     (lambda _
       ;;       ;; 'setup.py test' hits an AssertionError on BSD-specific
       ;;       ;; "tornado/platform/kqueue.py". This is the supported method:
       ;;       (invoke- "python" "-m" "tornado.test")
       ;;       #t)))
       #:tests? #f))

Is this still the case or can we enable the tests?

A number of important packages depend on python-tornado, such as
python-matplotlib, jupyter, and python-scikit-image.  Since this is a
major version bump I wonder if you have tried building some of these
related packages to make sure they don’t fail.

(We can of course respond to breakage once we discover it when building
the new evaluations, but it would be nicer if we could have some
reassurance.)

-- 
Ricardo

[bug#34204] [PATCH 2/3] gnu: Add Python LiveReload.

[Ricardo Wurmus]

Mathieu Lirzin <mthl@gnu.org> writes:

> * gnu/packages/python-web.scm (python-livereload): New variable.

LGTM!

-- 
Ricardo

[bug#34204] [PATCH 3/3] gnu: Add Mkdocs.

[Ricardo Wurmus]

Hi Mathieu,

> * gnu/packages/documentation.scm (mkdocs): New variable.
[…]
> +
> +(define-public mkdocs
> +  (package
> +    (name "mkdocs")
> +    (version "1.0.4")
> +    (source
> +     (origin
> +       (method url-fetch)
> +       (uri (pypi-uri "mkdocs" version))
> +       (sha256
> +        (base32
> +         "0fg9w6rdskwnn7knri7xzrd26k9svwqlxvdr0kk5spfpm8ll7lqp"))))
> +    (build-system python-build-system)
> +    (arguments
> +     ;; XXX: Tests are failing.
> +     ;; AttributeError: module 'mkdocs.plugins' has no attribute 'get_plugins'
> +     '(#:tests? #f))

Hmm, any idea why this happens?  Are the plugins not included?  Is there
an upstream bug report about this?

Other than that it looks good to me.

-- 
Ricardo

[bug#34204] [PATCH 1/3] gnu: python-tornado: Update to 5.1.1.

[Mathieu Lirzin]

Hello Ricardo,

Ricardo Wurmus <rekado@elephly.net> writes:

> The arguments field says:
>
>     (arguments
>      '(;; FIXME: Two tests error out with:
>        ;; AssertionError: b'Error in atexit._run_exitfuncs:\nFileNotF[44 chars]ry\n' != b''
>        ;; #:phases
>        ;; (modify-phases %standard-phases
>        ;;   (replace 'check
>        ;;     (lambda _
>        ;;       ;; 'setup.py test' hits an AssertionError on BSD-specific
>        ;;       ;; "tornado/platform/kqueue.py". This is the supported method:
>        ;;       (invoke- "python" "-m" "tornado.test")
>        ;;       #t)))
>        #:tests? #f))
>
> Is this still the case or can we enable the tests?

It doesn't seem to work.  Here is the error I got:

--8<---------------cut here---------------start------------->8---
starting phase `check'
running "python setup.py" with command "test" and parameters ()
running test
running egg_info
writing tornado.egg-info/PKG-INFO
writing dependency_links to tornado.egg-info/dependency_links.txt
writing top-level names to tornado.egg-info/top_level.txt
reading manifest file 'tornado.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching 'docs/build'
warning: no files found matching 'tornado/test/README'
writing manifest file 'tornado.egg-info/SOURCES.txt'
running build_ext
building 'tornado.speedups' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/include/python3.7m -c tornado/speedups.c -o build/temp.linux-x86_64-3.7/tornado/speedups.o
gcc -pthread -shared -Wl,-rpath=/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib build/temp.linux-x86_64-3.7/tornado/speedups.o -L/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib -lpython3.7m -o /tmp/guix-build-python-tornado-5.1.1.drv-0/tornado-5.1.1/tornado/speedups.cpython-37m-x86_64-linux-gnu.so
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "setup.py", line 195, in <module>
    **kwargs
  File "/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib/python3.7/site-packages/setuptools/__init__.py", line 129, in setup
    return distutils.core.setup(**attrs)
  File "/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib/python3.7/site-packages/setuptools/command/test.py", line 226, in run
    self.run_tests()
  File "/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib/python3.7/site-packages/setuptools/command/test.py", line 248, in run_tests
    exit=False,
  File "/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib/python3.7/unittest/main.py", line 100, in __init__
    self.parseArgs(argv)
  File "/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib/python3.7/unittest/main.py", line 124, in parseArgs
    self._do_discovery(argv[2:])
  File "/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib/python3.7/unittest/main.py", line 244, in _do_discovery
    self.createTests(from_discovery=True, Loader=Loader)
  File "/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib/python3.7/unittest/main.py", line 154, in createTests
    self.test = loader.discover(self.start, self.pattern, self.top)
  File "/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib/python3.7/unittest/loader.py", line 347, in discover
    tests = list(self._find_tests(start_dir, pattern))
  File "/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib/python3.7/unittest/loader.py", line 404, in _find_tests
    full_path, pattern, namespace)
  File "/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib/python3.7/unittest/loader.py", line 481, in _find_test_path
    tests = self.loadTestsFromModule(package, pattern=pattern)
  File "/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib/python3.7/site-packages/setuptools/command/test.py", line 52, in loadTestsFromModule
    tests.append(self.loadTestsFromName(submodule))
  File "/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib/python3.7/unittest/loader.py", line 191, in loadTestsFromName
    return self.loadTestsFromModule(obj)
  File "/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib/python3.7/site-packages/setuptools/command/test.py", line 52, in loadTestsFromModule
    tests.append(self.loadTestsFromName(submodule))
  File "/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib/python3.7/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/tmp/guix-build-python-tornado-5.1.1.drv-0/tornado-5.1.1/tornado/platform/kqueue.py", line 22, in <module>
    assert hasattr(select, 'kqueue'), 'kqueue not supported'
AssertionError: kqueue not supported
Backtrace:
           5 (primitive-load "/gnu/store/g5qipxm84gwwndfybxim7n906mr…")
In ice-9/eval.scm:
   191:35  4 (_ _)
In srfi/srfi-1.scm:
   863:16  3 (every1 #<procedure 735020 at /gnu/store/4r04fsfcryy5h…> …)
In /gnu/store/4r04fsfcryy5h4v2h3g6lzlibafmmdjm-module-import/guix/build/gnu-build-system.scm:
   799:28  2 (_ _)
In /gnu/store/4r04fsfcryy5h4v2h3g6lzlibafmmdjm-module-import/guix/build/python-build-system.scm:
    142:8  1 (check #:tests? _ #:test-target _ #:use-setuptools? _)
In /gnu/store/4r04fsfcryy5h4v2h3g6lzlibafmmdjm-module-import/guix/build/utils.scm:
    616:6  0 (invoke _ . _)

/gnu/store/4r04fsfcryy5h4v2h3g6lzlibafmmdjm-module-import/guix/build/utils.scm:616:6: In procedure invoke:
Throw to key `srfi-34' with args `(#<condition &invoke-error [program: "python" arguments: ("-c" "import setuptools, tokenize;__file__='setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\\r\\n', '\\n');f.close();exec(compile(code, __file__, 'exec'))" "test") exit-status: 1 term-signal: #f stop-signal: #f] 491d40>)'.
builder for `/gnu/store/1naw0y63i0dwmmyrngadsqf1g66vra6h-python-tornado-5.1.1.drv' failed with exit code 1
build of /gnu/store/1naw0y63i0dwmmyrngadsqf1g66vra6h-python-tornado-5.1.1.drv failed
View build log at '/var/log/guix/drvs/1n/aw0y63i0dwmmyrngadsqf1g66vra6h-python-tornado-5.1.1.drv.bz2'.
guix build: error: build of `/gnu/store/1naw0y63i0dwmmyrngadsqf1g66vra6h-python-tornado-5.1.1.drv' failed
--8<---------------cut here---------------end--------------->8---

> A number of important packages depend on python-tornado, such as
> python-matplotlib, jupyter, and python-scikit-image.  Since this is a
> major version bump I wonder if you have tried building some of these
> related packages to make sure they don’t fail.
>
> (We can of course respond to breakage once we discover it when building
> the new evaluations, but it would be nicer if we could have some
> reassurance.)

I was a bit lazy by not checking such kind of things beforehand.

The upgrade of ‘python-tornado’ didn't triggered a rebuild of
‘python-matplotlib’.  ‘python-python-terminado’, ‘python-scikit-image’,
and ‘jupyter’ succeeds building on my machine.

Thanks for the review.

-- 
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

[bug#34204] [PATCH 3/3] gnu: Add Mkdocs.

[Mathieu Lirzin]

[-- Attachment #1: Type: text/plain, Size: 1755 bytes --]

Hello Ricardo,

Ricardo Wurmus <rekado@elephly.net> writes:

>> * gnu/packages/documentation.scm (mkdocs): New variable.
> […]
>> +
>> +(define-public mkdocs
>> +  (package
>> +    (name "mkdocs")
>> +    (version "1.0.4")
>> +    (source
>> +     (origin
>> +       (method url-fetch)
>> +       (uri (pypi-uri "mkdocs" version))
>> +       (sha256
>> +        (base32
>> +         "0fg9w6rdskwnn7knri7xzrd26k9svwqlxvdr0kk5spfpm8ll7lqp"))))
>> +    (build-system python-build-system)
>> +    (arguments
>> +     ;; XXX: Tests are failing.
>> +     ;; AttributeError: module 'mkdocs.plugins' has no attribute 'get_plugins'
>> +     '(#:tests? #f))
>
> Hmm, any idea why this happens?  Are the plugins not included?  Is there
> an upstream bug report about this?

I didn't dig much, but after some search I have found the Nix package
definition [1] and it seems that the test suite requires ‘nosetests’ and
excludes 3 tests needs to be excluded to make the test suite pass.

Without worrying about how to exclude tests for now, I tried to add the
following to the ‘mkdocs’ package definition:

    (arguments '(#:test-target "nosetests"))
    (native-inputs
     `(("python-mock" ,python-mock)
       ("python-nose" ,python-nose)))

I get errors like the following:

--8<---------------cut here---------------start------------->8---
AssertionError: ([('docs_dir', ValidationError("The path /tmp/guix-build-mkdocs-1.0.4.drv-0/mkdocs-1.0.4/mkdocs/tests/integration/minimal/docs isn't an existing directory."))], [])
--8<---------------cut here---------------end--------------->8---

Any idea how to fix this kind of issue?

You can take a look at the attached build log for more details.

Thanks


[-- Attachment #2: build log --]
[-- Type: application/octet-stream, Size: 8433 bytes --]

[-- Attachment #3: Type: text/plain, Size: 182 bytes --]


[1] https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/tools/documentation/mkdocs/default.nix

-- 
Mathieu Lirzin
GPG: F2A3 8D7E EB2B 6640 5761  070D 0ADE E100 9460 4D37

bug#34204: [PATCH 0/3] Mkdocs and its dependencies

[Maxim Cournoyer]

Hello,

Mathieu Lirzin <mthl@gnu.org> writes:

> Hello,
>
> Here are a sequence of patches which adds the Mkdocs documentation generator
> to the set of Guix packages.
>
> Mathieu Lirzin (3):
>   gnu: python-tornado: Update to 5.1.1.

We now have python-tornado 6.1 in Guix.

>   gnu: Add Python LiveReload.

That's been committed already.

>   gnu: Add Mkdocs.

There's now a python-mkdocs package at version 1.1.2 in Guix.

Thus, closing.

Thank you!

Maxim