[bug#27888] [PATCH 01/18] gnu: python@3: Update to 3.6.2.

[bug#27888] [PATCH 01/18] gnu: python@3: Update to 3.6.2.

[Marius Bakke]

* gnu/packages/patches/python-3.5-fix-tests.patch: Rename to ...
* gnu/packages/patches/python-3-fix-tests.patch: ... this. Adjust and disable
more tests.
* gnu/packages/patches/python-3.5-getentropy-on-old-kernels.patch: Delete file.
* gnu/local.mk (dist_patch_DATA): Adjust accordingly.
* gnu/packages/python.scm (python-2.7)[argument]: Patch '/bin/sh' in one more file.
(python-3.5): Rename to ...
(python-3.6): ... this. Update to 3.6.2.
[source](patches): Remove obsolete patch, adjust to patch rename.
(python-3): Use PYTHON-3.6.
---
 gnu/local.mk                                       |   3 +-
 gnu/packages/patches/python-3-fix-tests.patch      | 130 ++++
 gnu/packages/patches/python-3.5-fix-tests.patch    |  69 --
 .../python-3.5-getentropy-on-old-kernels.patch     | 720 ---------------------
 gnu/packages/python.scm                            |  12 +-
 5 files changed, 137 insertions(+), 797 deletions(-)
 create mode 100644 gnu/packages/patches/python-3-fix-tests.patch
 delete mode 100644 gnu/packages/patches/python-3.5-fix-tests.patch
 delete mode 100644 gnu/packages/patches/python-3.5-getentropy-on-old-kernels.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index baada1cf0..62ef61589 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -932,8 +932,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/python-2.7-source-date-epoch.patch	\
   %D%/packages/patches/python-3-deterministic-build-info.patch	\
   %D%/packages/patches/python-3-search-paths.patch		\
-  %D%/packages/patches/python-3.5-fix-tests.patch		\
-  %D%/packages/patches/python-3.5-getentropy-on-old-kernels.patch	\
+  %D%/packages/patches/python-3-fix-tests.patch			\
   %D%/packages/patches/python-dendropy-fix-tests.patch		\
   %D%/packages/patches/python-fix-tests.patch			\
   %D%/packages/patches/python-genshi-add-support-for-python-3.4-AST.patch	\
diff --git a/gnu/packages/patches/python-3-fix-tests.patch b/gnu/packages/patches/python-3-fix-tests.patch
new file mode 100644
index 000000000..dec58dd9c
--- /dev/null
+++ b/gnu/packages/patches/python-3-fix-tests.patch
@@ -0,0 +1,130 @@
+Additional test fixes which affect Python 3.5 (and presumably later) but not
+prior revisions of Python.
+
+--- Lib/test/test_pathlib.py     2014-03-01 03:02:36.088311000 +0100
++++ Lib/test/test_pathlib.py     2014-03-01 04:56:37.768311000 +0100
+@@ -2132,8 +2132,7 @@
+         self.assertEqual(given, expect)
+         self.assertEqual(set(p.rglob("FILEd*")), set())
+ 
+-    @unittest.skipUnless(hasattr(pwd, 'getpwall'),
+-                         'pwd module does not expose getpwall()')
++    @unittest.skipIf(True, "Guix builder home is '/' which causes trouble for these tests")
+     def test_expanduser(self):
+         P = self.cls
+         support.import_module('pwd')
+--- Lib/test/test_tarfile.py        2016-02-24 19:22:52.597208055 +0000
++++ Lib/test/test_tarfile.py     2016-02-24 20:50:48.941950135 +0000
+@@ -2305,11 +2305,14 @@
+     try:
+         import pwd, grp
+     except ImportError:
+         return False
+-    if pwd.getpwuid(0)[0] != 'root':
+-        return False
+-    if grp.getgrgid(0)[0] != 'root':
++    try:
++        if pwd.getpwuid(0)[0] != 'root':
++            return False
++        if grp.getgrgid(0)[0] != 'root':
++            return False
++    except KeyError:
+         return False
+     return True
+
+
+--- Lib/test/test_asyncio/test_base_events.py
++++ Lib/test/test_asyncio/test_base_events.py
+@@ -1216,6 +1216,8 @@
+         self._test_create_connection_ip_addr(m_socket, False)
+
+     @patch_socket
++    @unittest.skipUnless(support.is_resource_enabled('network'),
++                         'network is not enabled')
+     def test_create_connection_service_name(self, m_socket):
+         m_socket.getaddrinfo = socket.getaddrinfo
+         sock = m_socket.socket.return_value
+
+--- Lib/test/test_pdb.py.org    2017-03-12 03:09:01.991856701 +0100
++++ Lib/test/test_pdb.py        2017-03-12 03:26:17.742572869 +0100
+
+For some reason, KeyboardInterrupts do not work in the build
+environment (lack of controlling TTY?). Just change the expected
+outcome. Unfortunately, this will make it fail for users running
+`python -m test test_pdb test_pdb` interactively.
+
+@@ -928,11 +928,11 @@
+     > <doctest test.test_pdb.test_pdb_issue_20766[0]>(6)test_function()
+     -> print('pdb %d: %s' % (i, sess._previous_sigint_handler))
+     (Pdb) continue
+-    pdb 1: <built-in function default_int_handler>
++    pdb 1: Handlers.SIG_IGN
+     > <doctest test.test_pdb.test_pdb_issue_20766[0]>(5)test_function()
+     -> sess.set_trace(sys._getframe())
+     (Pdb) continue
+-    pdb 2: <built-in function default_int_handler>
++    pdb 2: Handlers.SIG_IGN
+     """
+
+ class PdbTestCase(unittest.TestCase):
+--- Lib/test/test_socket.py
++++ Lib/test/test_socket.py
+@@ -802,6 +802,8 @@
+         if not fqhn in all_host_names:
+             self.fail("Error testing host resolution mechanisms. (fqdn: %s, all: %s)" % (fqhn, repr(all_host_names)))
+ 
++    @unittest.skipUnless(support.is_resource_enabled('network'),
++                         'network is not enabled')
+     def test_host_resolution(self):
+         for addr in [support.HOST, '10.0.0.1', '255.255.255.255']:
+             self.assertEqual(socket.gethostbyname(addr), addr)
+--- Lib/test/test_spwd.py
++++ Lib/test/test_spwd.py
+@@ -5,8 +5,7 @@
+ spwd = support.import_module('spwd')
+ 
+ 
+-@unittest.skipUnless(hasattr(os, 'geteuid') and os.geteuid() == 0,
+-                     'root privileges required')
++@unittest.skipUnless(os.path.exists("/etc/shadow"), 'spwd tests require /etc/shadow')
+ class TestSpwdRoot(unittest.TestCase):
+ 
+     def test_getspall(self):
+@@ -56,8 +55,7 @@
+             self.assertRaises(TypeError, spwd.getspnam, bytes_name)
+ 
+ 
+-@unittest.skipUnless(hasattr(os, 'geteuid') and os.geteuid() != 0,
+-                     'non-root user required')
++@unittest.skipUnless(os.path.exists("/etc/shadow"), 'spwd tests require /etc/shadow')
+ class TestSpwdNonRoot(unittest.TestCase):
+ 
+     def test_getspnam_exception(self):
+--- Lib/test/test_regrtest.py
++++ Lib/test/test_regrtest.py
+@@ -700,6 +700,7 @@
+         output = self.run_tests('--fromfile', filename)
+         self.check_executed_tests(output, tests)
+ 
++    @unittest.skipIf(True, 'Keyboard interrupts do not work in the Guix build environment.')
+     def test_interrupted(self):
+         code = TEST_INTERRUPTED
+         test = self.create_test('sigint', code=code)
+@@ -717,6 +718,7 @@
+                  % (self.TESTNAME_REGEX, len(tests)))
+         self.check_line(output, regex)
+ 
++    @unittest.skipIf(True, 'Keyboard interrupts do not work in the Guix build environment.')
+     def test_slow_interrupted(self):
+         # Issue #25373: test --slowest with an interrupted test
+         code = TEST_INTERRUPTED
+--- Lib/test/test_generators.py
++++ Lib/test/test_generators.py
+@@ -29,6 +29,7 @@
+         else:
+             return "FAILED"
+ 
++    @unittest.skipIf(True, 'Keyboard interrupts do not work in the Guix build environment')
+     def test_raise_and_yield_from(self):
+         gen = self.generator1()
+         gen.send(None)
diff --git a/gnu/packages/patches/python-3.5-fix-tests.patch b/gnu/packages/patches/python-3.5-fix-tests.patch
deleted file mode 100644
index 9778b88db..000000000
--- a/gnu/packages/patches/python-3.5-fix-tests.patch
+++ /dev/null
@@ -1,69 +0,0 @@
-Additional test fixes which affect Python 3.5 (and presumably later) but not
-prior revisions of Python.
-
---- Lib/test/test_pathlib.py     2014-03-01 03:02:36.088311000 +0100
-+++ Lib/test/test_pathlib.py     2014-03-01 04:56:37.768311000 +0100
-@@ -1986,8 +1986,9 @@
-         expect = set() if not support.fs_is_case_insensitive(BASE) else given
-         self.assertEqual(given, expect)
-         self.assertEqual(set(p.rglob("FILEd*")), set())
- 
-+    @unittest.skipIf(True, "Guix builder home is '/' which causes trouble for these tests")
-     def test_expanduser(self):
-         P = self.cls
-         support.import_module('pwd')
-         import pwd
---- Lib/test/test_tarfile.py        2016-02-24 19:22:52.597208055 +0000
-+++ Lib/test/test_tarfile.py     2016-02-24 20:50:48.941950135 +0000
-@@ -2305,11 +2305,14 @@
-     try:
-         import pwd, grp
-     except ImportError:
-         return False
--    if pwd.getpwuid(0)[0] != 'root':
--        return False
--    if grp.getgrgid(0)[0] != 'root':
-+    try:
-+        if pwd.getpwuid(0)[0] != 'root':
-+            return False
-+        if grp.getgrgid(0)[0] != 'root':
-+            return False
-+    except KeyError:
-         return False
-     return True
-
-
---- Lib/test/test_asyncio/test_base_events.py
-+++ Lib/test/test_asyncio/test_base_events.py
-@@ -1216,6 +1216,8 @@
-         self._test_create_connection_ip_addr(m_socket, False)
-
-     @patch_socket
-+    @unittest.skipUnless(support.is_resource_enabled('network'),
-+                         'network is not enabled')
-     def test_create_connection_service_name(self, m_socket):
-         m_socket.getaddrinfo = socket.getaddrinfo
-         sock = m_socket.socket.return_value
-
---- Lib/test/test_pdb.py.org    2017-03-12 03:09:01.991856701 +0100
-+++ Lib/test/test_pdb.py        2017-03-12 03:26:17.742572869 +0100
-
-For some reason, KeyboardInterrupts do not work in the build
-environment (lack of controlling TTY?). Just change the expected
-outcome. Unfortunately, this will make it fail for users running
-`python -m test test_pdb test_pdb` interactively.
-
-@@ -928,11 +928,11 @@
-     > <doctest test.test_pdb.test_pdb_issue_20766[0]>(6)test_function()
-     -> print('pdb %d: %s' % (i, sess._previous_sigint_handler))
-     (Pdb) continue
--    pdb 1: <built-in function default_int_handler>
-+    pdb 1: Handlers.SIG_IGN
-     > <doctest test.test_pdb.test_pdb_issue_20766[0]>(5)test_function()
-     -> sess.set_trace(sys._getframe())
-     (Pdb) continue
--    pdb 2: <built-in function default_int_handler>
-+    pdb 2: Handlers.SIG_IGN
-     """
-
- class PdbTestCase(unittest.TestCase):
diff --git a/gnu/packages/patches/python-3.5-getentropy-on-old-kernels.patch b/gnu/packages/patches/python-3.5-getentropy-on-old-kernels.patch
deleted file mode 100644
index 8a12b5b44..000000000
--- a/gnu/packages/patches/python-3.5-getentropy-on-old-kernels.patch
+++ /dev/null
@@ -1,720 +0,0 @@
-This patch resolves a compatibility issue when compiled against glibc 2.25
-and run runder kernels < 3.17:
-
-https://bugzilla.redhat.com/show_bug.cgi?id=1410175
-
-Upstream bug URL: https://bugs.python.org/issue29157
-
-Patch copied from upstream source repository:
-
-https://hg.python.org/cpython/rev/8125d9a8152b
-
-# HG changeset patch
-# User Victor Stinner <victor.stinner@gmail.com>
-# Date 1483957133 -3600
-# Node ID 8125d9a8152b79e712cb09c7094b9129b9bcea86
-# Parent  337461574c90281630751b6095c4e1baf380cf7d
-Issue #29157: Prefer getrandom() over getentropy()
-
-Copy and then adapt Python/random.c from default branch. Difference between 3.5
-and default branches:
-
-* Python 3.5 only uses getrandom() in non-blocking mode: flags=GRND_NONBLOCK
-* If getrandom() fails with EAGAIN: py_getrandom() immediately fails and
-  remembers that getrandom() doesn't work.
-* Python 3.5 has no _PyOS_URandomNonblock() function: _PyOS_URandom()
-  works in non-blocking mode on Python 3.5
-
-diff --git a/Python/random.c b/Python/random.c
---- Python/random.c
-+++ Python/random.c
-@@ -1,6 +1,9 @@
- #include "Python.h"
- #ifdef MS_WINDOWS
- #  include <windows.h>
-+/* All sample MSDN wincrypt programs include the header below. It is at least
-+ * required with Min GW. */
-+#  include <wincrypt.h>
- #else
- #  include <fcntl.h>
- #  ifdef HAVE_SYS_STAT_H
-@@ -37,10 +40,9 @@ win32_urandom_init(int raise)
-     return 0;
- 
- error:
--    if (raise)
-+    if (raise) {
-         PyErr_SetFromWindowsErr(0);
--    else
--        Py_FatalError("Failed to initialize Windows random API (CryptoGen)");
-+    }
-     return -1;
- }
- 
-@@ -53,8 +55,9 @@ win32_urandom(unsigned char *buffer, Py_
- 
-     if (hCryptProv == 0)
-     {
--        if (win32_urandom_init(raise) == -1)
-+        if (win32_urandom_init(raise) == -1) {
-             return -1;
-+        }
-     }
- 
-     while (size > 0)
-@@ -63,11 +66,9 @@ win32_urandom(unsigned char *buffer, Py_
-         if (!CryptGenRandom(hCryptProv, (DWORD)chunk, buffer))
-         {
-             /* CryptGenRandom() failed */
--            if (raise)
-+            if (raise) {
-                 PyErr_SetFromWindowsErr(0);
--            else
--                Py_FatalError("Failed to initialized the randomized hash "
--                        "secret using CryptoGen)");
-+            }
-             return -1;
-         }
-         buffer += chunk;
-@@ -76,58 +77,23 @@ win32_urandom(unsigned char *buffer, Py_
-     return 0;
- }
- 
--/* Issue #25003: Don't use getentropy() on Solaris (available since
-- * Solaris 11.3), it is blocking whereas os.urandom() should not block. */
--#elif defined(HAVE_GETENTROPY) && !defined(sun)
--#define PY_GETENTROPY 1
--
--/* Fill buffer with size pseudo-random bytes generated by getentropy().
--   Return 0 on success, or raise an exception and return -1 on error.
--
--   If fatal is nonzero, call Py_FatalError() instead of raising an exception
--   on error. */
--static int
--py_getentropy(unsigned char *buffer, Py_ssize_t size, int fatal)
--{
--    while (size > 0) {
--        Py_ssize_t len = Py_MIN(size, 256);
--        int res;
--
--        if (!fatal) {
--            Py_BEGIN_ALLOW_THREADS
--            res = getentropy(buffer, len);
--            Py_END_ALLOW_THREADS
--
--            if (res < 0) {
--                PyErr_SetFromErrno(PyExc_OSError);
--                return -1;
--            }
--        }
--        else {
--            res = getentropy(buffer, len);
--            if (res < 0)
--                Py_FatalError("getentropy() failed");
--        }
--
--        buffer += len;
--        size -= len;
--    }
--    return 0;
--}
--
--#else
-+#else /* !MS_WINDOWS */
- 
- #if defined(HAVE_GETRANDOM) || defined(HAVE_GETRANDOM_SYSCALL)
- #define PY_GETRANDOM 1
- 
--/* Call getrandom()
-+/* Call getrandom() to get random bytes:
-+
-    - Return 1 on success
--   - Return 0 if getrandom() syscall is not available (failed with ENOSYS or
--     EPERM) or if getrandom(GRND_NONBLOCK) failed with EAGAIN (system urandom
--     not initialized yet) and raise=0.
-+   - Return 0 if getrandom() is not available (failed with ENOSYS or EPERM),
-+     or if getrandom(GRND_NONBLOCK) failed with EAGAIN (system urandom not
-+     initialized yet).
-    - Raise an exception (if raise is non-zero) and return -1 on error:
--     getrandom() failed with EINTR and the Python signal handler raised an
--     exception, or getrandom() failed with a different error. */
-+     if getrandom() failed with EINTR, raise is non-zero and the Python signal
-+     handler raised an exception, or if getrandom() failed with a different
-+     error.
-+
-+   getrandom() is retried if it failed with EINTR: interrupted by a signal. */
- static int
- py_getrandom(void *buffer, Py_ssize_t size, int raise)
- {
-@@ -142,16 +108,19 @@ py_getrandom(void *buffer, Py_ssize_t si
-      * see https://bugs.python.org/issue26839. To avoid this, use the
-      * GRND_NONBLOCK flag. */
-     const int flags = GRND_NONBLOCK;
-+    char *dest;
-     long n;
- 
-     if (!getrandom_works) {
-         return 0;
-     }
- 
-+    dest = buffer;
-     while (0 < size) {
- #ifdef sun
-         /* Issue #26735: On Solaris, getrandom() is limited to returning up
--           to 1024 bytes */
-+           to 1024 bytes. Call it multiple times if more bytes are
-+           requested. */
-         n = Py_MIN(size, 1024);
- #else
-         n = Py_MIN(size, LONG_MAX);
-@@ -161,34 +130,35 @@ py_getrandom(void *buffer, Py_ssize_t si
- #ifdef HAVE_GETRANDOM
-         if (raise) {
-             Py_BEGIN_ALLOW_THREADS
--            n = getrandom(buffer, n, flags);
-+            n = getrandom(dest, n, flags);
-             Py_END_ALLOW_THREADS
-         }
-         else {
--            n = getrandom(buffer, n, flags);
-+            n = getrandom(dest, n, flags);
-         }
- #else
-         /* On Linux, use the syscall() function because the GNU libc doesn't
--         * expose the Linux getrandom() syscall yet. See:
--         * https://sourceware.org/bugzilla/show_bug.cgi?id=17252 */
-+           expose the Linux getrandom() syscall yet. See:
-+           https://sourceware.org/bugzilla/show_bug.cgi?id=17252 */
-         if (raise) {
-             Py_BEGIN_ALLOW_THREADS
--            n = syscall(SYS_getrandom, buffer, n, flags);
-+            n = syscall(SYS_getrandom, dest, n, flags);
-             Py_END_ALLOW_THREADS
-         }
-         else {
--            n = syscall(SYS_getrandom, buffer, n, flags);
-+            n = syscall(SYS_getrandom, dest, n, flags);
-         }
- #endif
- 
-         if (n < 0) {
--            /* ENOSYS: getrandom() syscall not supported by the kernel (but
--             * maybe supported by the host which built Python). EPERM:
--             * getrandom() syscall blocked by SECCOMP or something else. */
-+            /* ENOSYS: the syscall is not supported by the kernel.
-+               EPERM: the syscall is blocked by a security policy (ex: SECCOMP)
-+               or something else. */
-             if (errno == ENOSYS || errno == EPERM) {
-                 getrandom_works = 0;
-                 return 0;
-             }
-+
-             if (errno == EAGAIN) {
-                 /* getrandom(GRND_NONBLOCK) fails with EAGAIN if the system
-                    urandom is not initialiazed yet. In this case, fall back on
-@@ -202,32 +172,101 @@ py_getrandom(void *buffer, Py_ssize_t si
-             }
- 
-             if (errno == EINTR) {
--                if (PyErr_CheckSignals()) {
--                    if (!raise) {
--                        Py_FatalError("getrandom() interrupted by a signal");
-+                if (raise) {
-+                    if (PyErr_CheckSignals()) {
-+                        return -1;
-                     }
--                    return -1;
-                 }
- 
--                /* retry getrandom() */
-+                /* retry getrandom() if it was interrupted by a signal */
-                 continue;
-             }
- 
-             if (raise) {
-                 PyErr_SetFromErrno(PyExc_OSError);
-             }
--            else {
--                Py_FatalError("getrandom() failed");
-+            return -1;
-+        }
-+
-+        dest += n;
-+        size -= n;
-+    }
-+    return 1;
-+}
-+
-+#elif defined(HAVE_GETENTROPY)
-+#define PY_GETENTROPY 1
-+
-+/* Fill buffer with size pseudo-random bytes generated by getentropy():
-+
-+   - Return 1 on success
-+   - Return 0 if getentropy() syscall is not available (failed with ENOSYS or
-+     EPERM).
-+   - Raise an exception (if raise is non-zero) and return -1 on error:
-+     if getentropy() failed with EINTR, raise is non-zero and the Python signal
-+     handler raised an exception, or if getentropy() failed with a different
-+     error.
-+
-+   getentropy() is retried if it failed with EINTR: interrupted by a signal. */
-+static int
-+py_getentropy(char *buffer, Py_ssize_t size, int raise)
-+{
-+    /* Is getentropy() supported by the running kernel? Set to 0 if
-+       getentropy() failed with ENOSYS or EPERM. */
-+    static int getentropy_works = 1;
-+
-+    if (!getentropy_works) {
-+        return 0;
-+    }
-+
-+    while (size > 0) {
-+        /* getentropy() is limited to returning up to 256 bytes. Call it
-+           multiple times if more bytes are requested. */
-+        Py_ssize_t len = Py_MIN(size, 256);
-+        int res;
-+
-+        if (raise) {
-+            Py_BEGIN_ALLOW_THREADS
-+            res = getentropy(buffer, len);
-+            Py_END_ALLOW_THREADS
-+        }
-+        else {
-+            res = getentropy(buffer, len);
-+        }
-+
-+        if (res < 0) {
-+            /* ENOSYS: the syscall is not supported by the running kernel.
-+               EPERM: the syscall is blocked by a security policy (ex: SECCOMP)
-+               or something else. */
-+            if (errno == ENOSYS || errno == EPERM) {
-+                getentropy_works = 0;
-+                return 0;
-+            }
-+
-+            if (errno == EINTR) {
-+                if (raise) {
-+                    if (PyErr_CheckSignals()) {
-+                        return -1;
-+                    }
-+                }
-+
-+                /* retry getentropy() if it was interrupted by a signal */
-+                continue;
-+            }
-+
-+            if (raise) {
-+                PyErr_SetFromErrno(PyExc_OSError);
-             }
-             return -1;
-         }
- 
--        buffer += n;
--        size -= n;
-+        buffer += len;
-+        size -= len;
-     }
-     return 1;
- }
--#endif
-+#endif /* defined(HAVE_GETENTROPY) && !defined(sun) */
-+
- 
- static struct {
-     int fd;
-@@ -235,136 +274,123 @@ static struct {
-     ino_t st_ino;
- } urandom_cache = { -1 };
- 
-+/* Read random bytes from the /dev/urandom device:
- 
--/* Read 'size' random bytes from py_getrandom(). Fall back on reading from
--   /dev/urandom if getrandom() is not available.
-+   - Return 0 on success
-+   - Raise an exception (if raise is non-zero) and return -1 on error
- 
--   Call Py_FatalError() on error. */
--static void
--dev_urandom_noraise(unsigned char *buffer, Py_ssize_t size)
-+   Possible causes of errors:
-+
-+   - open() failed with ENOENT, ENXIO, ENODEV, EACCES: the /dev/urandom device
-+     was not found. For example, it was removed manually or not exposed in a
-+     chroot or container.
-+   - open() failed with a different error
-+   - fstat() failed
-+   - read() failed or returned 0
-+
-+   read() is retried if it failed with EINTR: interrupted by a signal.
-+
-+   The file descriptor of the device is kept open between calls to avoid using
-+   many file descriptors when run in parallel from multiple threads:
-+   see the issue #18756.
-+
-+   st_dev and st_ino fields of the file descriptor (from fstat()) are cached to
-+   check if the file descriptor was replaced by a different file (which is
-+   likely a bug in the application): see the issue #21207.
-+
-+   If the file descriptor was closed or replaced, open a new file descriptor
-+   but don't close the old file descriptor: it probably points to something
-+   important for some third-party code. */
-+static int
-+dev_urandom(char *buffer, Py_ssize_t size, int raise)
- {
-     int fd;
-     Py_ssize_t n;
- 
--    assert (0 < size);
-+    if (raise) {
-+        struct _Py_stat_struct st;
- 
--#ifdef PY_GETRANDOM
--    if (py_getrandom(buffer, size, 0) == 1) {
--        return;
-+        if (urandom_cache.fd >= 0) {
-+            /* Does the fd point to the same thing as before? (issue #21207) */
-+            if (_Py_fstat_noraise(urandom_cache.fd, &st)
-+                || st.st_dev != urandom_cache.st_dev
-+                || st.st_ino != urandom_cache.st_ino) {
-+                /* Something changed: forget the cached fd (but don't close it,
-+                   since it probably points to something important for some
-+                   third-party code). */
-+                urandom_cache.fd = -1;
-+            }
-+        }
-+        if (urandom_cache.fd >= 0)
-+            fd = urandom_cache.fd;
-+        else {
-+            fd = _Py_open("/dev/urandom", O_RDONLY);
-+            if (fd < 0) {
-+                if (errno == ENOENT || errno == ENXIO ||
-+                    errno == ENODEV || errno == EACCES) {
-+                    PyErr_SetString(PyExc_NotImplementedError,
-+                                    "/dev/urandom (or equivalent) not found");
-+                }
-+                /* otherwise, keep the OSError exception raised by _Py_open() */
-+                return -1;
-+            }
-+            if (urandom_cache.fd >= 0) {
-+                /* urandom_fd was initialized by another thread while we were
-+                   not holding the GIL, keep it. */
-+                close(fd);
-+                fd = urandom_cache.fd;
-+            }
-+            else {
-+                if (_Py_fstat(fd, &st)) {
-+                    close(fd);
-+                    return -1;
-+                }
-+                else {
-+                    urandom_cache.fd = fd;
-+                    urandom_cache.st_dev = st.st_dev;
-+                    urandom_cache.st_ino = st.st_ino;
-+                }
-+            }
-+        }
-+
-+        do {
-+            n = _Py_read(fd, buffer, (size_t)size);
-+            if (n == -1)
-+                return -1;
-+            if (n == 0) {
-+                PyErr_Format(PyExc_RuntimeError,
-+                        "Failed to read %zi bytes from /dev/urandom",
-+                        size);
-+                return -1;
-+            }
-+
-+            buffer += n;
-+            size -= n;
-+        } while (0 < size);
-     }
--    /* getrandom() failed with ENOSYS or EPERM,
--       fall back on reading /dev/urandom */
--#endif
--
--    fd = _Py_open_noraise("/dev/urandom", O_RDONLY);
--    if (fd < 0) {
--        Py_FatalError("Failed to open /dev/urandom");
--    }
--
--    while (0 < size)
--    {
--        do {
--            n = read(fd, buffer, (size_t)size);
--        } while (n < 0 && errno == EINTR);
--
--        if (n <= 0) {
--            /* read() failed or returned 0 bytes */
--            Py_FatalError("Failed to read bytes from /dev/urandom");
--            break;
--        }
--        buffer += n;
--        size -= n;
--    }
--    close(fd);
--}
--
--/* Read 'size' random bytes from py_getrandom(). Fall back on reading from
--   /dev/urandom if getrandom() is not available.
--
--   Return 0 on success. Raise an exception and return -1 on error. */
--static int
--dev_urandom_python(char *buffer, Py_ssize_t size)
--{
--    int fd;
--    Py_ssize_t n;
--    struct _Py_stat_struct st;
--#ifdef PY_GETRANDOM
--    int res;
--#endif
--
--    if (size <= 0)
--        return 0;
--
--#ifdef PY_GETRANDOM
--    res = py_getrandom(buffer, size, 1);
--    if (res < 0) {
--        return -1;
--    }
--    if (res == 1) {
--        return 0;
--    }
--    /* getrandom() failed with ENOSYS or EPERM,
--       fall back on reading /dev/urandom */
--#endif
--
--    if (urandom_cache.fd >= 0) {
--        /* Does the fd point to the same thing as before? (issue #21207) */
--        if (_Py_fstat_noraise(urandom_cache.fd, &st)
--            || st.st_dev != urandom_cache.st_dev
--            || st.st_ino != urandom_cache.st_ino) {
--            /* Something changed: forget the cached fd (but don't close it,
--               since it probably points to something important for some
--               third-party code). */
--            urandom_cache.fd = -1;
--        }
--    }
--    if (urandom_cache.fd >= 0)
--        fd = urandom_cache.fd;
-     else {
--        fd = _Py_open("/dev/urandom", O_RDONLY);
-+        fd = _Py_open_noraise("/dev/urandom", O_RDONLY);
-         if (fd < 0) {
--            if (errno == ENOENT || errno == ENXIO ||
--                errno == ENODEV || errno == EACCES)
--                PyErr_SetString(PyExc_NotImplementedError,
--                                "/dev/urandom (or equivalent) not found");
--            /* otherwise, keep the OSError exception raised by _Py_open() */
-             return -1;
-         }
--        if (urandom_cache.fd >= 0) {
--            /* urandom_fd was initialized by another thread while we were
--               not holding the GIL, keep it. */
--            close(fd);
--            fd = urandom_cache.fd;
--        }
--        else {
--            if (_Py_fstat(fd, &st)) {
-+
-+        while (0 < size)
-+        {
-+            do {
-+                n = read(fd, buffer, (size_t)size);
-+            } while (n < 0 && errno == EINTR);
-+
-+            if (n <= 0) {
-+                /* stop on error or if read(size) returned 0 */
-                 close(fd);
-                 return -1;
-             }
--            else {
--                urandom_cache.fd = fd;
--                urandom_cache.st_dev = st.st_dev;
--                urandom_cache.st_ino = st.st_ino;
--            }
-+
-+            buffer += n;
-+            size -= n;
-         }
-+        close(fd);
-     }
--
--    do {
--        n = _Py_read(fd, buffer, (size_t)size);
--        if (n == -1) {
--            return -1;
--        }
--        if (n == 0) {
--            PyErr_Format(PyExc_RuntimeError,
--                    "Failed to read %zi bytes from /dev/urandom",
--                    size);
--            return -1;
--        }
--
--        buffer += n;
--        size -= n;
--    } while (0 < size);
--
-     return 0;
- }
- 
-@@ -376,8 +402,8 @@ dev_urandom_close(void)
-         urandom_cache.fd = -1;
-     }
- }
-+#endif /* !MS_WINDOWS */
- 
--#endif
- 
- /* Fill buffer with pseudo-random bytes generated by a linear congruent
-    generator (LCG):
-@@ -400,29 +426,98 @@ lcg_urandom(unsigned int x0, unsigned ch
-     }
- }
- 
-+/* Read random bytes:
-+
-+   - Return 0 on success
-+   - Raise an exception (if raise is non-zero) and return -1 on error
-+
-+   Used sources of entropy ordered by preference, preferred source first:
-+
-+   - CryptGenRandom() on Windows
-+   - getrandom() function (ex: Linux and Solaris): call py_getrandom()
-+   - getentropy() function (ex: OpenBSD): call py_getentropy()
-+   - /dev/urandom device
-+
-+   Read from the /dev/urandom device if getrandom() or getentropy() function
-+   is not available or does not work.
-+
-+   Prefer getrandom() over getentropy() because getrandom() supports blocking
-+   and non-blocking mode and Python requires non-blocking RNG at startup to
-+   initialize its hash secret: see the PEP 524.
-+
-+   Prefer getrandom() and getentropy() over reading directly /dev/urandom
-+   because these functions don't need file descriptors and so avoid ENFILE or
-+   EMFILE errors (too many open files): see the issue #18756.
-+
-+   Only use RNG running in the kernel. They are more secure because it is
-+   harder to get the internal state of a RNG running in the kernel land than a
-+   RNG running in the user land. The kernel has a direct access to the hardware
-+   and has access to hardware RNG, they are used as entropy sources.
-+
-+   Note: the OpenSSL RAND_pseudo_bytes() function does not automatically reseed
-+   its RNG on fork(), two child processes (with the same pid) generate the same
-+   random numbers: see issue #18747. Kernel RNGs don't have this issue,
-+   they have access to good quality entropy sources.
-+
-+   If raise is zero:
-+
-+   - Don't raise an exception on error
-+   - Don't call the Python signal handler (don't call PyErr_CheckSignals()) if
-+     a function fails with EINTR: retry directly the interrupted function
-+   - Don't release the GIL to call functions.
-+*/
-+static int
-+pyurandom(void *buffer, Py_ssize_t size, int raise)
-+{
-+#if defined(PY_GETRANDOM) || defined(PY_GETENTROPY)
-+    int res;
-+#endif
-+
-+    if (size < 0) {
-+        if (raise) {
-+            PyErr_Format(PyExc_ValueError,
-+                         "negative argument not allowed");
-+        }
-+        return -1;
-+    }
-+
-+    if (size == 0) {
-+        return 0;
-+    }
-+
-+#ifdef MS_WINDOWS
-+    return win32_urandom((unsigned char *)buffer, size, raise);
-+#else
-+
-+#if defined(PY_GETRANDOM) || defined(PY_GETENTROPY)
-+#ifdef PY_GETRANDOM
-+    res = py_getrandom(buffer, size, raise);
-+#else
-+    res = py_getentropy(buffer, size, raise);
-+#endif
-+    if (res < 0) {
-+        return -1;
-+    }
-+    if (res == 1) {
-+        return 0;
-+    }
-+    /* getrandom() or getentropy() function is not available: failed with
-+       ENOSYS, EPERM or EAGAIN. Fall back on reading from /dev/urandom. */
-+#endif
-+
-+    return dev_urandom(buffer, size, raise);
-+#endif
-+}
-+
- /* Fill buffer with size pseudo-random bytes from the operating system random
-    number generator (RNG). It is suitable for most cryptographic purposes
-    except long living private keys for asymmetric encryption.
- 
--   Return 0 on success, raise an exception and return -1 on error. */
-+   Return 0 on success. Raise an exception and return -1 on error. */
- int
- _PyOS_URandom(void *buffer, Py_ssize_t size)
- {
--    if (size < 0) {
--        PyErr_Format(PyExc_ValueError,
--                     "negative argument not allowed");
--        return -1;
--    }
--    if (size == 0)
--        return 0;
--
--#ifdef MS_WINDOWS
--    return win32_urandom((unsigned char *)buffer, size, 1);
--#elif defined(PY_GETENTROPY)
--    return py_getentropy(buffer, size, 0);
--#else
--    return dev_urandom_python((char*)buffer, size);
--#endif
-+    return pyurandom(buffer, size, 1);
- }
- 
- void
-@@ -463,13 +558,14 @@ void
-         }
-     }
-     else {
--#ifdef MS_WINDOWS
--        (void)win32_urandom(secret, secret_size, 0);
--#elif defined(PY_GETENTROPY)
--        (void)py_getentropy(secret, secret_size, 1);
--#else
--        dev_urandom_noraise(secret, secret_size);
--#endif
-+        int res;
-+
-+        /* _PyRandom_Init() is called very early in the Python initialization
-+           and so exceptions cannot be used (use raise=0). */
-+        res = pyurandom(secret, secret_size, 0);
-+        if (res < 0) {
-+            Py_FatalError("failed to get random numbers to initialize Python");
-+        }
-     }
- }
- 
-@@ -481,8 +577,6 @@ void
-         CryptReleaseContext(hCryptProv, 0);
-         hCryptProv = 0;
-     }
--#elif defined(PY_GETENTROPY)
--    /* nothing to clean */
- #else
-     dev_urandom_close();
- #endif
-
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index a088fefb8..a047b617c 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -201,6 +201,7 @@
                                   '("Lib/subprocess.py"
                                     "Lib/popen2.py"
                                     "Lib/distutils/tests/test_spawn.py"
+                                    "Lib/test/support/__init__.py"
                                     "Lib/test/test_subprocess.py"))
                (("/bin/sh") (which "sh")))
 
@@ -326,23 +327,22 @@ data types.")
 ;; Current 2.x version.
 (define-public python-2 python-2.7)
 
-(define-public python-3.5
+(define-public python-3.6
   (package (inherit python-2)
-    (version "3.5.3")
+    (version "3.6.2")
     (source (origin
               (method url-fetch)
               (uri (string-append "https://www.python.org/ftp/python/"
                                   version "/Python-" version ".tar.xz"))
               (patches (search-patches
                         "python-fix-tests.patch"
-                        "python-3.5-fix-tests.patch"
-                        "python-3.5-getentropy-on-old-kernels.patch"
+                        "python-3-fix-tests.patch"
                         "python-3-deterministic-build-info.patch"
                         "python-3-search-paths.patch"))
               (patch-flags '("-p0"))
               (sha256
                (base32
-                "1c6v1n9nz4mlx9mw1125fxpmbrgniqdbbx9hnqx44maqazb2mzpf"))
+                "1ab4vlpdax1ihpiyiwchlgsk36apl4kgdw271wvl9l8ywhxpfacj"))
               (snippet
                '(begin
                   (for-each delete-file
@@ -359,7 +359,7 @@ data types.")
                                         "/site-packages"))))))))
 
 ;; Current 3.x version.
-(define-public python-3 python-3.5)
+(define-public python-3 python-3.6)
 
 ;; Current major version.
 (define-public python python-3)
-- 
2.13.3

[bug#27888] [PATCH 02/18] gnu: python-py: Update to 1.4.34.

[Marius Bakke]

* gnu/packages/python.scm (python-py, python2-py): Update to 1.4.34.
---
 gnu/packages/python.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index a047b617c..42f5d0472 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -1855,14 +1855,14 @@ standard library.")
 (define-public python-py
   (package
     (name "python-py")
-    (version "1.4.32")
+    (version "1.4.34")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "py" version))
        (sha256
         (base32
-         "19s1pql9pq85h1qzsdwgyb8a3k1qgkvh33b02m8kfqhizz8rzf64"))))
+         "1qyd5z0hv8ymxy84v5vig3vps2fvhcf4bdlksb3r03h549fmhb8g"))))
     (build-system python-build-system)
     (arguments
      ;; FIXME: "ImportError: 'test' module incorrectly imported from
-- 
2.13.3

[bug#27888] [PATCH 03/18] gnu: python-six: Really enable tests.

[Marius Bakke]

* gnu/packages/python.scm (python-six)[arguments]: Override 'check' phase.
---
 gnu/packages/python.scm | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 42f5d0472..dd61389b8 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -1179,6 +1179,12 @@ password storage.")
        (base32
         "0snmb8xffb3vsma0z67i0h0w2g2dy0p3gsgh9gi4i0kgc5l8spqh"))))
     (build-system python-build-system)
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (replace 'check
+           (lambda _
+             (zero? (system* "py.test" "-v")))))))
     (native-inputs
      `(("python-py" ,python-py)
        ("python-pytest" ,python-pytest)))
-- 
2.13.3

[bug#27888] [PATCH 04/18] gnu: Remove python-pytest@2.

[Marius Bakke]

* gnu/packages/python.scm (python-pytest): Update to 3.0.7.
[source]: Move substitution to ...
[arguments]: ... here. Incorporate phase from pytest-3.0.
[native-inputs]: Add PYTHON-HYPOTHESIS.
(python-pytest-3.0): Remove variable.
(python2-pytest-3.0): Rename to ...
(python2-pytest): ... this.
[native-inputs]: Add PYTHON2-ENUM34.
* gnu/packages/backup.scm (borg)[native-inputs]: Replace PYTHON-PYTEST-3.0
with PYTHON-PYTEST.
* gnu/packages/calendar.scm (khal): Likewise.
* gnu/packages/python.scm (python-sphinx-1.6, python-sphinx-1.5.3,
python-pexpect, python-cryptography, python-pyopenssl, python-pytest-warnings,
python-pytest-capturelog, python-pytest-catchlog, python-marshmallow,
python-apispec, python-flasgger): Likewise.
* gnu/packages/web.scm (python-internetarchive): Likewise.
---
 gnu/packages/backup.scm   |  4 +--
 gnu/packages/calendar.scm |  3 +-
 gnu/packages/python.scm   | 86 ++++++++++++++++++-----------------------------
 gnu/packages/web.scm      |  2 +-
 4 files changed, 35 insertions(+), 60 deletions(-)

diff --git a/gnu/packages/backup.scm b/gnu/packages/backup.scm
index 20b375abe..974cc8fe6 100644
--- a/gnu/packages/backup.scm
+++ b/gnu/packages/backup.scm
@@ -534,9 +534,7 @@ detection, and lossless compression.")
     (native-inputs
      `(("python-cython" ,python-cython)
        ("python-setuptools-scm" ,python-setuptools-scm)
-       ;; Borg 1.0.8's test suite uses 'tmpdir_factory', which was introduced in
-       ;; pytest 2.8.
-       ("python-pytest" ,python-pytest-3.0)
+       ("python-pytest" ,python-pytest)
        ;; For generating the documentation.
        ("python-sphinx" ,python-sphinx)
        ("python-sphinx-rtd-theme" ,python-sphinx-rtd-theme)))
diff --git a/gnu/packages/calendar.scm b/gnu/packages/calendar.scm
index b022c0efa..bd2381aed 100644
--- a/gnu/packages/calendar.scm
+++ b/gnu/packages/calendar.scm
@@ -113,8 +113,7 @@ data units.")
                                    "/share/zoneinfo/Zulu"))
             (zero? (system* "py.test" "tests")))))))
     (native-inputs
-      ;; XXX Uses tmpdir_factory, introduced in pytest 2.8.
-     `(("python-pytest" ,python-pytest-3.0)
+     `(("python-pytest" ,python-pytest)
        ("python-pytest-cov" ,python-pytest-cov)
        ("python-setuptools-scm" ,python-setuptools-scm)
        ;; Required for tests
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index dd61389b8..18067d388 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -1889,28 +1889,36 @@ code introspection, and logging.")
 (define-public python-pytest
   (package
     (name "python-pytest")
-    (version "2.7.3")
+    (version "3.0.7")
     (source
      (origin
        (method url-fetch)
-       (uri (string-append
-             "https://pypi.python.org/packages/source/p/pytest/pytest-"
-             version ".tar.gz"))
+       (uri (pypi-uri "pytest" version))
        (sha256
         (base32
-         "1z4yi986f9n0p8qmzmn21m21m8j1x78hk3505f89baqm6pdw7afm"))
-       (modules '((guix build utils)))
-       (snippet
-        ;; One of the tests involves the /usr directory, so it fails.
-        '(substitute* "testing/test_argcomplete.py"
-           (("def test_remove_dir_prefix\\(self\\):")
-            "@pytest.mark.xfail\n    def test_remove_dir_prefix(self):")))))
+         "1asc4b2nd2a4f0g3r12y97rslq5wliji7b73wwkvdrm5s7mrc1mp"))))
     (build-system python-build-system)
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (add-before 'check 'disable-invalid-tests
+           (lambda _
+             ;; Some tests involves the /usr directory, and fails.
+             (substitute* "testing/test_argcomplete.py"
+               (("def test_remove_dir_prefix\\(self\\):")
+                "@pytest.mark.xfail\n    def test_remove_dir_prefix(self):"))
+             (substitute* "testing/test_argcomplete.py"
+               (("def test_remove_dir_prefix" line)
+                (string-append "@pytest.mark.skip"
+                               "(reason=\"Assumes that /usr exists.\")\n    "
+                               line)))
+             #t)))))
     (propagated-inputs
      `(("python-py" ,python-py)))
     (native-inputs
      `(;; Tests need the "regular" bash since 'bash-final' lacks `compgen`.
        ("bash" ,bash)
+       ("python-hypothesis" ,python-hypothesis)
        ("python-nose" ,python-nose)
        ("python-mock" ,python-mock)))
     (home-page "http://pytest.org")
@@ -1919,42 +1927,12 @@ code introspection, and logging.")
      "Pytest is a testing tool that provides auto-discovery of test modules
 and functions, detailed info on failing assert statements, modular fixtures,
 and many external plugins.")
+    (properties `((python2-variant . ,(delay python2-pytest))))
     (license license:expat)))
 
 (define-public python2-pytest
-  (package-with-python2 python-pytest))
-
-;; Some packages require a newer pytest.
-(define-public python-pytest-3.0
-  (package
-    (inherit python-pytest)
-    (name "python-pytest")
-    (version "3.0.7")
-    (source (origin
-              (method url-fetch)
-              (uri (pypi-uri "pytest" version))
-              (sha256
-               (base32
-                "1asc4b2nd2a4f0g3r12y97rslq5wliji7b73wwkvdrm5s7mrc1mp"))))
-    (arguments
-     `(#:phases
-       (modify-phases %standard-phases
-         (add-before 'check 'disable-invalid-test
-           (lambda _
-             (substitute* "testing/test_argcomplete.py"
-               (("def test_remove_dir_prefix" line)
-                (string-append "@pytest.mark.skip"
-                               "(reason=\"Assumes that /usr exists.\")\n    "
-                               line)))
-             #t)))))
-    (native-inputs
-     `(("python-hypothesis" ,python-hypothesis)
-       ,@(package-native-inputs python-pytest)))
-    (properties `((python2-variant . ,(delay python2-pytest-3.0))))))
-
-(define-public python2-pytest-3.0
   (let ((base (package-with-python2
-                (strip-python2-variant python-pytest-3.0))))
+                (strip-python2-variant python-pytest))))
     (package (inherit base)
       (native-inputs
         `(("python2-enum34" ,python2-enum34)
@@ -3296,7 +3274,7 @@ sources.")
      `(("python-sphinxcontrib-websupport" ,python-sphinxcontrib-websupport)
        ,@(package-propagated-inputs python-sphinx)))
     (native-inputs
-     `(("python-pytest" ,python-pytest-3.0)
+     `(("python-pytest" ,python-pytest)
        ("imagemagick" ,imagemagick) ; for "convert"
        ,@(package-native-inputs python-sphinx)))
     (properties '())))
@@ -3314,7 +3292,7 @@ sources.")
         (base32
          "0kw1axswbvaavr8ggyf4qr6hnisnrzlbkkcdada69vk1x9xjassg"))))
     (native-inputs
-     `(("python-pytest" ,python-pytest-3.0)
+     `(("python-pytest" ,python-pytest)
        ,@(package-native-inputs python-sphinx)))))
 
 (define-public python2-sphinx
@@ -5241,7 +5219,7 @@ cluster without needing to write any wrapper code yourself.")
          (replace 'check (lambda _ (zero? (system* "nosetests" "-v")))))))
     (native-inputs
      `(("python-nose" ,python-nose)
-       ("python-pytest" ,python-pytest-3.0)
+       ("python-pytest" ,python-pytest)
        ("man-db" ,man-db)
        ("which" ,which)
        ("bash-full" ,bash)))                 ;full Bash for 'test_replwrap.py'
@@ -7691,7 +7669,7 @@ responses, rather than doing any computation.")
        ("python-hypothesis" ,python-hypothesis)
        ("python-pretend" ,python-pretend)
        ("python-pytz" ,python-pytz)
-       ("python-pytest" ,python-pytest-3.0)))
+       ("python-pytest" ,python-pytest)))
     (home-page "https://github.com/pyca/cryptography")
     (synopsis "Cryptographic recipes and primitives for Python")
     (description
@@ -7751,7 +7729,7 @@ message digests and key derivation functions.")
     (native-inputs
      `(("python-flaky" ,python-flaky)
        ("python-pretend" ,python-pretend)
-       ("python-pytest" ,python-pytest-3.0)))
+       ("python-pytest" ,python-pytest)))
     (home-page "https://github.com/pyca/pyopenssl")
     (synopsis "Python wrapper module around the OpenSSL library")
     (description
@@ -14493,7 +14471,7 @@ for Flask.")
          "0gf2dpahpl5igb7jh1sr9acj3z3gp7zahqdqb69nk6wx01c8kc1g"))))
     (build-system python-build-system)
     (propagated-inputs
-     `(("pytest" ,python-pytest-3.0)))
+     `(("pytest" ,python-pytest)))
     (home-page "https://github.com/fschulze/pytest-warnings")
     (synopsis "Pytest plugin to list Python warnings in pytest report")
     (description
@@ -14517,7 +14495,7 @@ pytest report.")
          "038049nyjl7di59ycnxvc9nydivc5m8np3hqq84j2iirkccdbs5n"))))
     (build-system python-build-system)
     (propagated-inputs
-     `(("pytest" ,python-pytest-3.0)))
+     `(("pytest" ,python-pytest)))
     (home-page "http://bitbucket.org/memedough/pytest-capturelog/overview")
     (synopsis "Pytest plugin to catch log messages")
     (description
@@ -14542,7 +14520,7 @@ pytest report.")
     (native-inputs
      `(("unzip" ,unzip)))
     (propagated-inputs
-     `(("pytest" ,python-pytest-3.0)))
+     `(("pytest" ,python-pytest)))
     (home-page "https://github.com/eisensheng/pytest-catchlog")
     (synopsis "Pytest plugin to catch log messages")
     (description
@@ -15533,7 +15511,7 @@ address is valid and really exists.")
      `(("python-dateutil" ,python-dateutil)
        ("python-simplejson" ,python-simplejson)))
     (native-inputs
-     `(("python-pytest-3.0" ,python-pytest-3.0)
+     `(("python-pytest" ,python-pytest)
        ("python-pytz" ,python-pytz)))
     (home-page "https://github.com/marshmallow-code/marshmallow")
     (synopsis "Convert complex datatypes to and from native
@@ -15580,7 +15558,7 @@ complex datatypes to and from native Python datatypes.")
     (propagated-inputs
      `(("python-pyyaml" ,python-pyyaml)))
     (native-inputs
-     `(("python-pytest-3.0" ,python-pytest-3.0)
+     `(("python-pytest" ,python-pytest)
        ("python-flask" ,python-flask)
        ("python-marshmallow" ,python-marshmallow)
        ("python-tornado" ,python-tornado)
@@ -15633,7 +15611,7 @@ Swagger 2.0).")
        ("python-flake8" ,python-flake8)
        ("python-flask-restful" ,python-flask-restful)
        ("python-flex" ,python-flex)
-       ("python-pytest-3.0" ,python-pytest-3.0)
+       ("python-pytest" ,python-pytest)
        ("python-pytest-cov" ,python-pytest-cov)
        ("python-marshmallow" ,python-marshmallow)
        ("python-apispec" ,python-apispec)))
diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index 9c3373f8d..f691705d5 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -4763,7 +4763,7 @@ command-line arguments or read from stdin.")
        ("python-schema" ,python-schema-0.5)
        ("python-backports-csv" ,python-backports-csv)))
     (native-inputs
-     `(("python-pytest-3.0" ,python-pytest-3.0)
+     `(("python-pytest" ,python-pytest)
        ("python-pytest-capturelog" ,python-pytest-capturelog)
        ("python-responses" ,python-responses)))
     (home-page "https://github.com/jjjake/internetarchive")
-- 
2.13.3

[bug#27888] [PATCH 05/18] gnu: python-pytest: Add a minimal variant for bootstrapping.

[Marius Bakke]

* gnu/packages/python.scm (python-pytest-bootstrap, python2-pytest-bootstrap):
New variables.
(python-pytest, python2-pytest): Inherit from bootstrap variants.
(python-six, python-hypothesis, python-pytest-runner,
python-mccabe)[native-inputs]: Use PYTHON-PYTEST-BOOTSTRAP instead of
PYTHON-PYTEST.
---
 gnu/packages/python.scm | 42 ++++++++++++++++++++++++++----------------
 1 file changed, 26 insertions(+), 16 deletions(-)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 18067d388..2fe74a4e4 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -1187,7 +1187,7 @@ password storage.")
              (zero? (system* "py.test" "-v")))))))
     (native-inputs
      `(("python-py" ,python-py)
-       ("python-pytest" ,python-pytest)))
+       ("python-pytest" ,python-pytest-bootstrap)))
     (home-page "http://pypi.python.org/pypi/six/")
     (synopsis "Python 2 and 3 compatibility utilities")
     (description
@@ -1886,9 +1886,9 @@ code introspection, and logging.")
 (define-public python2-py
   (package-with-python2 python-py))
 
-(define-public python-pytest
+(define python-pytest-bootstrap
   (package
-    (name "python-pytest")
+    (name "python-pytest-bootstrap")
     (version "3.0.7")
     (source
      (origin
@@ -1899,6 +1899,25 @@ code introspection, and logging.")
          "1asc4b2nd2a4f0g3r12y97rslq5wliji7b73wwkvdrm5s7mrc1mp"))))
     (build-system python-build-system)
     (arguments
+     `(#:tests? #f))
+    (propagated-inputs
+     `(("python-py" ,python-py)))
+    (home-page "http://pytest.org")
+    (synopsis "Python testing library")
+    (description
+     "Pytest is a testing tool that provides auto-discovery of test modules
+and functions, detailed info on failing assert statements, modular fixtures,
+and many external plugins.")
+    (license license:expat)))
+
+(define-public python2-pytest-bootstrap
+  (package-with-python2 python-pytest-bootstrap))
+
+(define-public python-pytest
+  (package
+    (inherit python-pytest-bootstrap)
+    (name "python-pytest")
+    (arguments
      `(#:phases
        (modify-phases %standard-phases
          (add-before 'check 'disable-invalid-tests
@@ -1913,22 +1932,13 @@ code introspection, and logging.")
                                "(reason=\"Assumes that /usr exists.\")\n    "
                                line)))
              #t)))))
-    (propagated-inputs
-     `(("python-py" ,python-py)))
     (native-inputs
      `(;; Tests need the "regular" bash since 'bash-final' lacks `compgen`.
        ("bash" ,bash)
        ("python-hypothesis" ,python-hypothesis)
        ("python-nose" ,python-nose)
        ("python-mock" ,python-mock)))
-    (home-page "http://pytest.org")
-    (synopsis "Python testing library")
-    (description
-     "Pytest is a testing tool that provides auto-discovery of test modules
-and functions, detailed info on failing assert statements, modular fixtures,
-and many external plugins.")
-    (properties `((python2-variant . ,(delay python2-pytest))))
-    (license license:expat)))
+    (properties `((python2-variant . ,(delay python2-pytest-bootstrap))))))
 
 (define-public python2-pytest
   (let ((base (package-with-python2
@@ -1998,7 +2008,7 @@ supports coverage of subprocesses.")
                (string-append "version = \"" ,version "\"")))
             #t)))))
     (native-inputs
-     `(("python-pytest" ,python-pytest)
+     `(("python-pytest" ,python-pytest-bootstrap)
        ("python-setuptools-scm" ,python-setuptools-scm)))
     (home-page "https://github.com/pytest-dev/pytest-runner")
     (synopsis "Invoke py.test as a distutils command")
@@ -6722,7 +6732,7 @@ PEP 8.")
             "0yr08a36h8lqlif10l4xcikbbig7q8f41gqywir7rrvnv3mi4aws"))))
     (build-system python-build-system)
     (native-inputs
-      `(("python-pytest" ,python-pytest)
+      `(("python-pytest" ,python-pytest-bootstrap)
         ("python-pytest-runner" ,python-pytest-runner)))
     (home-page "https://github.com/flintwork/mccabe")
     (synopsis "McCabe checker, plugin for flake8")
@@ -9774,7 +9784,7 @@ Amazon Web Services (AWS) API.")
     (build-system python-build-system)
     (native-inputs
      `(("python-flake8" ,python-flake8)
-       ("python-pytest" ,python-pytest)))
+       ("python-pytest" ,python-pytest-bootstrap)))
     (synopsis "Library for property based testing")
     (description "Hypothesis is a library for testing your Python code against a
 much larger range of examples than you would ever want to write by hand.  It’s
-- 
2.13.3

[bug#27888] [PATCH 06/18] gnu: python-setuptools-scm: Update to 1.15.6.

[Marius Bakke]

* gnu/packages/python.scm (python-setuptools-scm): Update to 1.15.6.
---
 gnu/packages/python.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 2fe74a4e4..c40f56978 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -5250,13 +5250,13 @@ child application and control it as if a human were typing commands.")
 (define-public python-setuptools-scm
   (package
     (name "python-setuptools-scm")
-    (version "1.15.0")
+    (version "1.15.6")
     (source (origin
               (method url-fetch)
               (uri (pypi-uri "setuptools_scm" version))
               (sha256
                (base32
-                "0bwyc5markib0i7i2qlyhdzxhiywzxbkfiapldma8m91m82jvwfs"))))
+                "0pzvfmx8s20yrgkgwfbxaspz2x1g38qv61jpm0ns91lrb22ldas9"))))
     (build-system python-build-system)
     (home-page "https://github.com/pypa/setuptools_scm/")
     (synopsis "Manage Python package versions in SCM metadata")
-- 
2.13.3

[bug#27888] [PATCH 07/18] gnu: python-pytest: Update to 3.1.3.

[Marius Bakke]

* gnu/packages/python.scm (python-pytest-bootstrap): Update to 3.1.3.
[native-inputs]: Add PYTHON-SETUPTOOLS-SCM.
(python-pytest)[native-inputs]: Inherit.
---
 gnu/packages/python.scm | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index c40f56978..955bd670a 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -1889,17 +1889,19 @@ code introspection, and logging.")
 (define python-pytest-bootstrap
   (package
     (name "python-pytest-bootstrap")
-    (version "3.0.7")
+    (version "3.1.3")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "pytest" version))
        (sha256
         (base32
-         "1asc4b2nd2a4f0g3r12y97rslq5wliji7b73wwkvdrm5s7mrc1mp"))))
+         "01k2abl6x60ac7wx5k9rw602n1b5r39xix6sjly5c974ywr1hph9"))))
     (build-system python-build-system)
     (arguments
      `(#:tests? #f))
+    (native-inputs
+     `(("python-setuptools-scm" ,python-setuptools-scm)))
     (propagated-inputs
      `(("python-py" ,python-py)))
     (home-page "http://pytest.org")
@@ -1937,7 +1939,8 @@ and many external plugins.")
        ("bash" ,bash)
        ("python-hypothesis" ,python-hypothesis)
        ("python-nose" ,python-nose)
-       ("python-mock" ,python-mock)))
+       ("python-mock" ,python-mock)
+       ,@(package-native-inputs python-pytest-bootstrap)))
     (properties `((python2-variant . ,(delay python2-pytest-bootstrap))))))
 
 (define-public python2-pytest
-- 
2.13.3

[bug#27888] [PATCH 08/18] gnu: python-pyflakes: Update to 1.5.0.

[Marius Bakke]

* gnu/packages/python.scm (python-pyflakes): Update to 1.5.0.
---
 gnu/packages/python.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 955bd670a..d0a979a2a 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -6703,14 +6703,14 @@ PEP 8.")
 (define-public python-pyflakes
   (package
     (name "python-pyflakes")
-    (version "1.0.0")
+    (version "1.5.0")
     (source
       (origin
         (method url-fetch)
         (uri (pypi-uri "pyflakes" version))
         (sha256
           (base32
-            "0qs2sgqszq7wcplis8509wk2ygqcrwzbs1ghfj3svvivq2j377pk"))))
+            "1x1pcca4a24k4pw8x1c77sgi58cg1wl2k38mp8a25k608pzls3da"))))
     (build-system python-build-system)
     (home-page
       "https://github.com/pyflakes/pyflakes")
-- 
2.13.3

[bug#27888] [PATCH 09/18] gnu: python-mccabe: Update to 0.6.1.

[Marius Bakke]

* gnu/packages/python.scm (python-mccabe): Update to 0.6.1.
---
 gnu/packages/python.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index d0a979a2a..6fd0f014c 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -6725,14 +6725,14 @@ PEP 8.")
 (define-public python-mccabe
   (package
     (name "python-mccabe")
-    (version "0.4.0")
+    (version "0.6.1")
     (source
       (origin
         (method url-fetch)
         (uri (pypi-uri "mccabe" version))
         (sha256
           (base32
-            "0yr08a36h8lqlif10l4xcikbbig7q8f41gqywir7rrvnv3mi4aws"))))
+            "07w3p1qm44hgxf3vvwz84kswpsx6s7kvaibzrsx5dzm0hli1i3fx"))))
     (build-system python-build-system)
     (native-inputs
       `(("python-pytest" ,python-pytest-bootstrap)
-- 
2.13.3

[bug#27888] [PATCH 10/18] gnu: python-mock@2: Add dependency on python-six.

[Marius Bakke]

* gnu/packages/python.scm (python-mock-2)[propagated-inputs]: Add PYTHON-SIX.
---
 gnu/packages/python.scm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 6fd0f014c..37b8c5594 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -970,6 +970,7 @@ have been used.")
           "1flbpksir5sqrvq2z0dp8sl4bzbadg21sj4d42w3klpdfvgvcn5i"))))
     (propagated-inputs
      `(("python-pbr" ,python-pbr-minimal)
+       ("python-six" ,python-six)
        ,@(package-propagated-inputs python-mock)))))
 
 (define-public python-setuptools
-- 
2.13.3

[bug#27888] [PATCH 11/18] gnu: Add python2-enum.

[Marius Bakke]

* gnu/packages/python.scm (python2-enum): New variable.
---
 gnu/packages/python.scm | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 37b8c5594..0d90d2017 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -1547,6 +1547,28 @@ bug tracker.")
     (home-page "http://www.liquidx.net/pybugz/")
     (license license:gpl2)))
 
+(define-public python2-enum
+  (package
+    (name "python2-enum")
+    (version "0.4.6")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "enum" version))
+              (sha256
+               (base32
+                "13lk3yrwj42vl30kw3c194f739nrfrdg64s6i0v2p636n4k8brsl"))))
+    (build-system python-build-system)
+    (arguments
+     `(#:python ,python-2))
+    (home-page "http://pypi.python.org/pypi/enum/")
+    (synopsis "Robust enumerated type support in Python")
+    (description
+     "This provides a module for robust enumerations in Python.  It has
+been superseded by the Python standard library and is provided only for
+compatibility.")
+    ;; Choice of either license.
+    (license (list license:gpl3+ license:psfl))))
+
 (define-public python-enum34
   (package
     (name "python-enum34")
-- 
2.13.3

[bug#27888] [PATCH 12/18] gnu: python-flake8: Update to 3.4.1.

[Marius Bakke]

* gnu/packages/python.scm (python-flake8): Update to 3.4.1.
[source]: Remove obsolete snippet.
[arguments]: Override 'check' phase.
[propagated-inputs]: Remove PYTHON-PEP8. Add PYTHON-PYCODESTYLE and
PYTHON-SETUPTOOLS.
[native-inputs]: Change PYTHON-MOCK to PYTHON-MOCK-2. Remove PYTHON-NOSE. Add
PYTHON-PYTEST-BOOTSTRAP and PYTHON-PYTEST-RUNNER.
[properties]: Declare python2 variant.
* gnu/packages/python.scm (python-flake8)[propagated-inputs]: Add PYTHON2-ENUM
and PYTHON2-CONFIGPARSER.
---
 gnu/packages/python.scm | 41 +++++++++++++++++++++++++----------------
 1 file changed, 25 insertions(+), 16 deletions(-)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 0d90d2017..a23d6aa1c 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -6831,39 +6831,48 @@ complexity of Python source code.")
 (define-public python-flake8
   (package
     (name "python-flake8")
-    (version "2.5.4")
+    (version "3.4.1")
     (source
       (origin
         (method url-fetch)
         (uri (pypi-uri "flake8" version))
         (sha256
           (base32
-            "0bs9cz4fr99r2rwig1b8jwaadl1nan7kgpdzqwj0bwbckwbmh7nc"))
-        (modules '((guix build utils)))
-        (snippet
-         '(begin
-            ;; Remove pre-compiled .pyc files from source.
-            (for-each delete-file-recursively
-                      (find-files "." "__pycache__" #:directories? #t))
-            (for-each delete-file (find-files "." "\\.pyc$"))
-            #t))))
-    (build-system python-build-system)
-    (propagated-inputs
-      `(("python-pep8" ,python-pep8)
+            "1n0i38592vy3q0x2a9bf8z6rhhn04i30wsn5i5zzcj7qkxvl8062"))))
+    (build-system python-build-system)
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (delete 'check)
+         (add-after 'install 'check
+          (lambda* (#:key inputs outputs #:allow-other-keys)
+            (add-installed-pythonpath inputs outputs)
+            (zero? (system* "pytest" "-v")))))))
+    (propagated-inputs
+      `(("python-pycodestyle" ,python-pycodestyle)
         ("python-pyflakes" ,python-pyflakes)
+        ;; flake8 depends on a newer setuptools than provided by python.
+        ("python-setuptools" ,python-setuptools)
         ("python-mccabe" ,python-mccabe)))
     (native-inputs
-      `(("python-mock" ,python-mock) ; TODO: only required for < 3.3
-        ("python-nose" ,python-nose)))
+      `(("python-mock" ,python-mock-2) ; TODO: only required for < 3.3
+        ("python-pytest" ,python-pytest-bootstrap)
+        ("python-pytest-runner" ,python-pytest-runner)))
     (home-page "https://gitlab.com/pycqa/flake8")
     (synopsis
       "The modular source code checker: pep8, pyflakes and co")
     (description
       "Flake8 is a wrapper around PyFlakes, pep8 and python-mccabe.")
+    (properties `((python2-variant . ,(delay python2-flake8))))
     (license license:expat)))
 
 (define-public python2-flake8
-  (package-with-python2 python-flake8))
+  (let ((base (package-with-python2 (strip-python2-variant python-flake8))))
+    (package (inherit base)
+      (propagated-inputs
+       `(("python2-configparser" ,python2-configparser)
+         ("python2-enum" ,python2-enum)
+          ,@(package-propagated-inputs base))))))
 
 (define-public python-flake8-polyfill
   (package
-- 
2.13.3

[bug#27888] [PATCH 13/18] gnu: Add python-linecache2.

[Marius Bakke]

* gnu/packages/python.scm (python-linecache2, python2-linecache2): New variables.
---
 gnu/packages/python.scm | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index a23d6aa1c..be556959d 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -1819,6 +1819,30 @@ interfaces and processes.")
 (define-public python2-nose2
   (package-with-python2 python-nose2))
 
+(define-public python-linecache2
+  (package
+    (name "python-linecache2")
+    (version "1.0.0")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "linecache2" version))
+              (sha256
+               (base32
+                "0z79g3ds5wk2lvnqw0y2jpakjf32h95bd9zmnvp7dnqhf57gy9jb"))))
+    (build-system python-build-system)
+    (arguments `(#:tests? #f)) ; FIXME: dependency cycle on unittest2.
+    (native-inputs
+     `(("python-pbr" ,python-pbr-minimal)))
+    (home-page "https://github.com/testing-cabal/linecache2")
+    (synopsis "Backports of the linecache module")
+    (description
+     "Backports of the linecache module to older versions of Python.")
+    ;; No explicit license, but code is taken from Python standard library.
+    (license license:psfl)))
+
+(define-public python2-linecache2
+  (package-with-python2 python-linecache2))
+
 (define-public python-unittest2
   (package
     (name "python-unittest2")
-- 
2.13.3

[bug#27888] [PATCH 14/18] gnu: Add python-traceback2.

[Marius Bakke]

* gnu/packages/python.scm (python-traceback2, python2-traceback2): New variables.
---
 gnu/packages/python.scm | 25 +++++++++++++++++++++++++
 1 file changed, 25 insertions(+)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index be556959d..77efcf1ed 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -1843,6 +1843,31 @@ interfaces and processes.")
 (define-public python2-linecache2
   (package-with-python2 python-linecache2))
 
+(define-public python-traceback2
+  (package
+    (name "python-traceback2")
+    (version "1.4.0")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "traceback2" version))
+              (sha256
+               (base32
+                "0c1h3jas1jp1fdbn9z2mrgn3jj0hw1x3yhnkxp7jw34q15xcdb05"))))
+    (build-system python-build-system)
+    (arguments `(#:tests? #f)) ; FIXME: Dependency cycle on unittest2.
+    (native-inputs
+     `(("python-pbr" ,python-pbr-minimal)))
+    (propagated-inputs
+     `(("python-linecache2" ,python-linecache2)))
+    (home-page "https://github.com/testing-cabal/traceback2")
+    (synopsis "Backports of the traceback module")
+    (description
+     "Backports of the traceback module to older versions of Python.")
+    (license license:psfl)))
+
+(define-public python2-traceback2
+  (package-with-python2 python-traceback2))
+
 (define-public python-unittest2
   (package
     (name "python-unittest2")
-- 
2.13.3

[bug#27888] [PATCH 15/18] gnu: python-unittest2: Update to 1.1.0.

[Marius Bakke]

* gnu/packages/python.scm (python-unittest2): Update to 1.1.0.
[source]: Use PYPI-URI. Add python3 compatibility patch.
[arguments]: Add phases to loosen dependency requirements and run tests.
[propagated-inputs]: Add PYTHON-SIX and PYTHON-TRACEBACK2.
(python2-unittest2): Use 'package-with-python2'.
* gnu/packages/patches/python-unittest2-python3-compat.patch: New file.
* gnu/local.mk (dist_patch_DATA): Register it.
---
 gnu/local.mk                                       |  1 +
 .../patches/python-unittest2-python3-compat.patch  | 34 ++++++++++++++++++
 gnu/packages/python.scm                            | 42 +++++++++++-----------
 3 files changed, 57 insertions(+), 20 deletions(-)
 create mode 100644 gnu/packages/patches/python-unittest2-python3-compat.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 62ef61589..ce3d821d9 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -957,6 +957,7 @@ dist_patch_DATA =						\
   %D%/packages/patches/python2-pygobject-2-gi-info-type-error-domain.patch \
   %D%/packages/patches/python-pygpgme-fix-pinentry-tests.patch	\
   %D%/packages/patches/python2-subprocess32-disable-input-test.patch	\
+  %D%/packages/patches/python-unittest2-python3-compat.patch	\
   %D%/packages/patches/qemu-CVE-2017-7493.patch			\
   %D%/packages/patches/qemu-CVE-2017-8112.patch			\
   %D%/packages/patches/qemu-CVE-2017-8309.patch			\
diff --git a/gnu/packages/patches/python-unittest2-python3-compat.patch b/gnu/packages/patches/python-unittest2-python3-compat.patch
new file mode 100644
index 000000000..fe0afe559
--- /dev/null
+++ b/gnu/packages/patches/python-unittest2-python3-compat.patch
@@ -0,0 +1,34 @@
+Skip tests that fail with newer versions of Python.
+
+Patch copied from Gentoo:
+
+https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/unittest2/files/unittest2-1.1.0-python3.5-test.patch
+
+diff --git a/unittest2/test/test_loader.py b/unittest2/test/test_loader.py
+index 683f662..347eea5 100644
+--- a/unittest2/test/test_loader.py
++++ b/unittest2/test/test_loader.py
+@@ -509,6 +509,7 @@ class Test_TestLoader(unittest2.TestCase):
+     #
+     # What happens when an impossible name is given, relative to the provided
+     # `module`?
++    @unittest.skipIf(sys.version_info[:2] >= (3, 5), "python 3.5 has problems here")
+     def test_loadTestsFromName__relative_malformed_name(self):
+         loader = unittest.TestLoader()
+ 
+@@ -811,6 +812,7 @@ class Test_TestLoader(unittest2.TestCase):
+     # TestCase or TestSuite instance."
+     #
+     # What happens when presented with an impossible module name?
++    @unittest.skipIf(sys.version_info[:2] >= (3, 5), "python 3.5 has problems here")
+     def test_loadTestsFromNames__malformed_name(self):
+         loader = unittest2.TestLoader()
+ 
+@@ -918,6 +920,7 @@ class Test_TestLoader(unittest2.TestCase):
+     # "The method optionally resolves name relative to the given module"
+     #
+     # What happens when presented with an impossible attribute name?
++    @unittest.skipIf(sys.version_info[:2] >= (3, 5), "python 3.5 has problems here")
+     def test_loadTestsFromNames__relative_malformed_name(self):
+         loader = unittest.TestLoader()
+ 
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 77efcf1ed..3217de524 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -1871,17 +1871,33 @@ interfaces and processes.")
 (define-public python-unittest2
   (package
     (name "python-unittest2")
-    (version "0.5.1")
+    (version "1.1.0")
     (source
      (origin
        (method url-fetch)
-       (uri (string-append
-             "https://pypi.python.org/packages/source/u/unittest2py3k/unittest2py3k-"
-             version ".tar.gz"))
+       (uri (pypi-uri "unittest2" version))
+       (patches
+        (search-patches "python-unittest2-python3-compat.patch"))
        (sha256
         (base32
-         "00yl6lskygcrddx5zspkhr0ibgvpknl4678kkm6s626539grq93q"))))
+         "0y855kmx7a8rnf81d3lh5lyxai1908xjp0laf4glwa4c8472m212"))))
     (build-system python-build-system)
+    (arguments
+     '(#:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'adjust-dependencies
+           (lambda _
+             ;; Don't check for separate argparse module when it's shipped
+             ;; in all Python versions we support (2.7+).
+             (substitute* "setup.py"
+               (("'argparse',") ""))
+             #t))
+         (replace 'check
+           (lambda _
+             (zero? (system* "python" "-m" "unittest2" "discover" "--verbose")))))))
+    (propagated-inputs
+     `(("python-six" ,python-six)
+       ("python-traceback2" ,python-traceback2)))
     (home-page "http://pypi.python.org/pypi/unittest2")
     (synopsis "Python unit testing library")
     (description
@@ -1890,21 +1906,7 @@ standard library.")
     (license license:psfl)))
 
 (define-public python2-unittest2
-  (package (inherit python-unittest2)
-    (name "python2-unittest2")
-    (version "0.5.1")
-    (source
-     (origin
-       (method url-fetch)
-       (uri (string-append
-             "https://pypi.python.org/packages/source/u/unittest2/unittest2-"
-             version ".tar.gz"))
-       (sha256
-        (base32
-         "0wbs4i4x3x7klr3v35ss6p9mcqz883i1xgcpkhvl7n2lyv6yhpda"))))
-    (arguments
-     `(#:python ,python-2
-       #:tests? #f)))) ; no setup.py test command
+  (package-with-python2 python-unittest2))
 
 (define-public python-pafy
   (package
-- 
2.13.3

[bug#27888] [PATCH 16/18] gnu: Add python2-funcsigs.

[Marius Bakke]

* gnu/packages/python.scm (python2-funcsigs): New variable.
---
 gnu/packages/python.scm | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 3217de524..e1bc3905a 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -1819,6 +1819,27 @@ interfaces and processes.")
 (define-public python2-nose2
   (package-with-python2 python-nose2))
 
+(define-public python2-funcsigs
+  (package
+    (name "python2-funcsigs")
+    (version "1.0.2")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "funcsigs" version))
+              (sha256
+               (base32
+                "0l4g5818ffyfmfs1a924811azhjj8ax9xd1cffr1mzd3ycn0zfx7"))))
+    (build-system python-build-system)
+    (arguments
+     `(#:python ,python-2))
+    (native-inputs
+     `(("python2-unittest2" ,python2-unittest2)))
+    (home-page "http://funcsigs.readthedocs.org")
+    (synopsis "Python function signatures from PEP362")
+    (description
+     "Backport of @code{funcsigs} which was introduced in Python 3.3.")
+    (license license:asl2.0)))
+
 (define-public python-linecache2
   (package
     (name "python-linecache2")
-- 
2.13.3

[bug#27888] [PATCH 17/18] gnu: python-mock: Make 2.0 the default package.

[Marius Bakke]

* gnu/packages/python.scm (python-mock): Update to 2.0.0.
[native-inputs]: Add PYTHON-UNITTEST2.
[propagated-inputs]: Add PYTHON-PBR and PYTHON-SIX.
[arguments]: Override 'check' phase.
[properties]: Declare python2 variant.
(python2-mock)[propagated-inputs]: Add PYTHON2-FUNCTOOLS32 and PYTHON2-FUNCSIGS.
(python-mock-2): Remove variable.
* gnu/packages/python.scm (python-flake8)[native-inputs]: Adjust accordingly.
* gnu/packages/tls.scm (python-acme, certbot)[native-inputs]: Likewise.
---
 gnu/packages/python.scm | 47 ++++++++++++++++++++++-------------------------
 gnu/packages/tls.scm    |  4 ++--
 2 files changed, 24 insertions(+), 27 deletions(-)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index e1bc3905a..9a7f88c30 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -932,46 +932,43 @@ API for locking files.")
 (define-public python-mock
   (package
     (name "python-mock")
-    (version "1.0.1")
+    (version "2.0.0")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "mock" version))
        (sha256
         (base32
-         "0kzlsbki6q0awf89rc287f3aj8x431lrajf160a70z0ikhnxsfdq"))))
+         "1flbpksir5sqrvq2z0dp8sl4bzbadg21sj4d42w3klpdfvgvcn5i"))))
+    (propagated-inputs
+     `(("python-pbr" ,python-pbr-minimal)
+       ("python-six" ,python-six)))
     (build-system python-build-system)
-    (arguments '(#:test-target "check"))
+    (native-inputs
+     `(("python-unittest2" ,python-unittest2)))
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (replace 'check
+           (lambda _
+             (zero? (system* "unit2")))))))
     (home-page "https://github.com/testing-cabal/mock")
     (synopsis "Python mocking and patching library for testing")
     (description
      "Mock is a library for testing in Python.  It allows you to replace parts
 of your system under test with mock objects and make assertions about how they
 have been used.")
+    (properties `((python2-variant . ,(delay python2-mock))))
     (license license:expat)))
 
 (define-public python2-mock
-  (package-with-python2 python-mock))
-
-;;; Some packages (notably, certbot and python-acme) rely on this newer version
-;;; of python-mock. However, a large number of packages fail to build with
-;;; mock@2, so we add a new variable for now. Also, there may be a dependency
-;;; cycle between mock and six, so we avoid creating python2-mock@2 for now.
-(define-public python-mock-2
-  (package
-    (inherit python-mock)
-    (version "2.0.0")
-    (source
-      (origin
-        (method url-fetch)
-        (uri (pypi-uri "mock" version))
-        (sha256
-         (base32
-          "1flbpksir5sqrvq2z0dp8sl4bzbadg21sj4d42w3klpdfvgvcn5i"))))
-    (propagated-inputs
-     `(("python-pbr" ,python-pbr-minimal)
-       ("python-six" ,python-six)
-       ,@(package-propagated-inputs python-mock)))))
+  (let ((base (package-with-python2
+               (strip-python2-variant python-mock))))
+    (package (inherit base)
+      (propagated-inputs
+       `(("python2-functools32" ,python2-functools32)
+         ("python2-funcsigs" ,python2-funcsigs)
+         ,@(package-propagated-inputs base))))))
 
 (define-public python-setuptools
   (package
@@ -6927,7 +6924,7 @@ complexity of Python source code.")
         ("python-setuptools" ,python-setuptools)
         ("python-mccabe" ,python-mccabe)))
     (native-inputs
-      `(("python-mock" ,python-mock-2) ; TODO: only required for < 3.3
+      `(("python-mock" ,python-mock) ; TODO: only required for < 3.3
         ("python-pytest" ,python-pytest-bootstrap)
         ("python-pytest-runner" ,python-pytest-runner)))
     (home-page "https://gitlab.com/pycqa/flake8")
diff --git a/gnu/packages/tls.scm b/gnu/packages/tls.scm
index 5a94ac619..4c64a0b0a 100644
--- a/gnu/packages/tls.scm
+++ b/gnu/packages/tls.scm
@@ -502,7 +502,7 @@ netcat implementation that supports TLS.")
                #t))))))
     ;; TODO: Add optional inputs for testing.
     (native-inputs
-     `(("python-mock" ,python-mock-2)
+     `(("python-mock" ,python-mock)
        ;; For documentation
        ("python-sphinx" ,python-sphinx)
        ("python-sphinxcontrib-programoutput" ,python-sphinxcontrib-programoutput)
@@ -554,7 +554,7 @@ netcat implementation that supports TLS.")
     ;; TODO: Add optional inputs for testing.
     (native-inputs
      `(("python-nose" ,python-nose)
-       ("python-mock" ,python-mock-2)
+       ("python-mock" ,python-mock)
        ;; For documentation
        ("python-sphinx" ,python-sphinx)
        ("python-sphinx-rtd-theme" ,python-sphinx-rtd-theme)
-- 
2.13.3

[bug#27888] [PATCH 17/18] gnu: python-mock: Make 2.0 the default package.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 104 bytes --]

I've retitled this to 'gnu: Remove python-mock@1.' to be consistent with
patch 04/18. Also +2 accuracy.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#27888] [PATCH 18/18] gnu: python-hypothesis: Update to 3.14.0.

[Marius Bakke]

* gnu/packages/python.scm (python-hypothesis): Update to 3.14.0.
[native-inputs]: Remove PYTHON-FLAKE8 and PYTHON-PYTEST.
---
 gnu/packages/python.scm | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 9a7f88c30..b32065f80 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -9878,17 +9878,15 @@ Amazon Web Services (AWS) API.")
 (define-public python-hypothesis
   (package
     (name "python-hypothesis")
-    (version "3.1.0")
+    (version "3.14.0")
     (source (origin
               (method url-fetch)
               (uri (pypi-uri "hypothesis" version))
               (sha256
                (base32
-                "0qyqq9akm4vshhn8cngjc1qykcvsn7cz6dlm6njfsgpbraqrmbbw"))))
+                "11q26a0j49hqw233gnvphlkin6piw2bpd0kji5zxr0980vl5rxvi"))))
     (build-system python-build-system)
-    (native-inputs
-     `(("python-flake8" ,python-flake8)
-       ("python-pytest" ,python-pytest-bootstrap)))
+    ;; FIXME: The PyPi release tarball does not contain tests.
     (synopsis "Library for property based testing")
     (description "Hypothesis is a library for testing your Python code against a
 much larger range of examples than you would ever want to write by hand.  It’s
-- 
2.13.3

[bug#27888] [PATCH 01/18] gnu: python@3: Update to 3.6.2.

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 623 bytes --]

On Mon, Jul 31, 2017 at 10:07:18PM +0200, Marius Bakke wrote:
> * gnu/packages/patches/python-3.5-fix-tests.patch: Rename to ...
> * gnu/packages/patches/python-3-fix-tests.patch: ... this. Adjust and disable
> more tests.
> * gnu/packages/patches/python-3.5-getentropy-on-old-kernels.patch: Delete file.

I'm curious, do you know if they handled this issue upstream in a
reliable way, or is getentropy() simply unused in Python 3.6?

Also, do you have any idea about the state of the bytecode timestamp
issue, which is related to the problem with the bytecode cache? I can
work on that on python-updates when you push it.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#27888] [PATCH 01/18] gnu: python@3: Update to 3.6.2.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1347 bytes --]

Leo Famulari <leo@famulari.name> writes:

> On Mon, Jul 31, 2017 at 10:07:18PM +0200, Marius Bakke wrote:
>> * gnu/packages/patches/python-3.5-fix-tests.patch: Rename to ...
>> * gnu/packages/patches/python-3-fix-tests.patch: ... this. Adjust and disable
>> more tests.
>> * gnu/packages/patches/python-3.5-getentropy-on-old-kernels.patch: Delete file.
>
> I'm curious, do you know if they handled this issue upstream in a
> reliable way, or is getentropy() simply unused in Python 3.6?

The getentropy fix for 3.5 was a port of "random.c" from 3.6 + the fix %
3.5 specific behaviour. Since 3.6.2 was just released it should have
it. Most hunks are there, but didn't verify all.

> Also, do you have any idea about the state of the bytecode timestamp
> issue, which is related to the problem with the bytecode cache? I can
> work on that on python-updates when you push it.

Yes. You might have seen this:

https://github.com/NixOS/nixpkgs/pull/25916

NixOS patches the interpreter to unconditionally set bytecode timestamps
to epoch 1. Now I also found this:

https://github.com/python/cpython/pull/296

So I think we only need that patch and to set PYTHONHASHSEED to some
deterministic value in python-build-system. 

I can give it a go next weekend if you're busy, but more than happy to
share the workload :-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#27888] [PATCH 01/18] gnu: python@3: Update to 3.6.2.

[Ricardo Wurmus]

Hi Marius and Leo,

> Leo Famulari <leo@famulari.name> writes:
>
>> On Mon, Jul 31, 2017 at 10:07:18PM +0200, Marius Bakke wrote:
>>> * gnu/packages/patches/python-3.5-fix-tests.patch: Rename to ...
>>> * gnu/packages/patches/python-3-fix-tests.patch: ... this. Adjust and disable
>>> more tests.
>>> * gnu/packages/patches/python-3.5-getentropy-on-old-kernels.patch: Delete file.
>>
>> I'm curious, do you know if they handled this issue upstream in a
>> reliable way, or is getentropy() simply unused in Python 3.6?
>
> The getentropy fix for 3.5 was a port of "random.c" from 3.6 + the fix %
> 3.5 specific behaviour. Since 3.6.2 was just released it should have
> it. Most hunks are there, but didn't verify all.
>
>> Also, do you have any idea about the state of the bytecode timestamp
>> issue, which is related to the problem with the bytecode cache? I can
>> work on that on python-updates when you push it.
>
> Yes. You might have seen this:
>
> https://github.com/NixOS/nixpkgs/pull/25916
>
> NixOS patches the interpreter to unconditionally set bytecode timestamps
> to epoch 1. Now I also found this:
>
> https://github.com/python/cpython/pull/296
>
> So I think we only need that patch and to set PYTHONHASHSEED to some
> deterministic value in python-build-system. 
>
> I can give it a go next weekend if you're busy, but more than happy to
> share the workload :-)

I hope you don’t mind me asking: is there any progress here?  Anything
blocking that I could help with?

-- 
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

bug#27888: [PATCH 01/18] gnu: python@3: Update to 3.6.2.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 2294 bytes --]

Ricardo Wurmus <rekado@elephly.net> writes:

> Hi Marius and Leo,
>
>> Leo Famulari <leo@famulari.name> writes:
>>
>>> On Mon, Jul 31, 2017 at 10:07:18PM +0200, Marius Bakke wrote:
>>>> * gnu/packages/patches/python-3.5-fix-tests.patch: Rename to ...
>>>> * gnu/packages/patches/python-3-fix-tests.patch: ... this. Adjust and disable
>>>> more tests.
>>>> * gnu/packages/patches/python-3.5-getentropy-on-old-kernels.patch: Delete file.
>>>
>>> I'm curious, do you know if they handled this issue upstream in a
>>> reliable way, or is getentropy() simply unused in Python 3.6?
>>
>> The getentropy fix for 3.5 was a port of "random.c" from 3.6 + the fix %
>> 3.5 specific behaviour. Since 3.6.2 was just released it should have
>> it. Most hunks are there, but didn't verify all.
>>
>>> Also, do you have any idea about the state of the bytecode timestamp
>>> issue, which is related to the problem with the bytecode cache? I can
>>> work on that on python-updates when you push it.
>>
>> Yes. You might have seen this:
>>
>> https://github.com/NixOS/nixpkgs/pull/25916
>>
>> NixOS patches the interpreter to unconditionally set bytecode timestamps
>> to epoch 1. Now I also found this:
>>
>> https://github.com/python/cpython/pull/296
>>
>> So I think we only need that patch and to set PYTHONHASHSEED to some
>> deterministic value in python-build-system. 
>>
>> I can give it a go next weekend if you're busy, but more than happy to
>> share the workload :-)
>
> I hope you don’t mind me asking: is there any progress here?  Anything
> blocking that I could help with?

Hello!

I just pushed a rebased version of this branch to a new 'python-updates'
branch on Savannah, with Python 3.6.3 and 2.7.14.  I also have an update
for 'python-hypothesis' coming, probably this weekend.

Wrt the reproducibility patch, I've tried to cherry-pick
<https://github.com/python/cpython/pull/296> but got a couple of new
test failures.  I have been really busy lately and got an injury to
boot, so my work capacity is limited and any help very welcome.

Closing this bug since the patches are "out there", but feel free to
continue the discussion.  I don't expect a lot of problems due to this
update, so we can probably handle it on 'core-updates'.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#27888] [PATCH 01/18] gnu: python@3: Update to 3.6.2.

[Cyril Roelandt]

[-- Attachment #1.1: Type: text/plain, Size: 1075 bytes --]

On 10/19/2017 12:30 AM, Marius Bakke wrote:
> I just pushed a rebased version of this branch to a new 'python-updates'
> branch on Savannah, with Python 3.6.3 and 2.7.14.  I also have an update
> for 'python-hypothesis' coming, probably this weekend.

Do you wish to push the python-hypothesis update to core-updates as
well? I have the following patch ready to be pushed:

--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -10150,9 +10150,9 @@ seamlessly into your existing Python unit
testing work flow.")
   (let ((hypothesis (package-with-python2
                      (strip-python2-variant python-hypothesis))))
     (package (inherit hypothesis)
-      (native-inputs
+      (propagated-inputs
        `(("python2-enum34" ,python2-enum34)
-         ,@(package-native-inputs hypothesis))))))
+         ,@(package-propagated-inputs hypothesis))))))

 (define-public python-pytest-subtesthack
   (package


I need this to be able to upgrade python-chardet. Do you mind if I push
this patch to master first?


Regards,
Cyril.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 862 bytes --]

[bug#27888] [PATCH 01/18] gnu: python@3: Update to 3.6.2.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 1182 bytes --]

Cyril Roelandt <tipecaml@gmail.com> writes:

> On 10/19/2017 12:30 AM, Marius Bakke wrote:
>> I just pushed a rebased version of this branch to a new 'python-updates'
>> branch on Savannah, with Python 3.6.3 and 2.7.14.  I also have an update
>> for 'python-hypothesis' coming, probably this weekend.
>
> Do you wish to push the python-hypothesis update to core-updates as
> well? I have the following patch ready to be pushed:
>
> --- a/gnu/packages/python.scm
> +++ b/gnu/packages/python.scm
> @@ -10150,9 +10150,9 @@ seamlessly into your existing Python unit
> testing work flow.")
>    (let ((hypothesis (package-with-python2
>                       (strip-python2-variant python-hypothesis))))
>      (package (inherit hypothesis)
> -      (native-inputs
> +      (propagated-inputs
>         `(("python2-enum34" ,python2-enum34)
> -         ,@(package-native-inputs hypothesis))))))
> +         ,@(package-propagated-inputs hypothesis))))))
>
>  (define-public python-pytest-subtesthack
>    (package
>
>
> I need this to be able to upgrade python-chardet. Do you mind if I push
> this patch to master first?

Not at all; go ahead :) I have this in my queue as well, so LGTM!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#27888] [PATCH 01/18] gnu: python@3: Update to 3.6.2.

[Ricardo Wurmus]

Hi Marius,

> I just pushed a rebased version of this branch to a new 'python-updates'
> branch on Savannah, with Python 3.6.3 and 2.7.14.  I also have an update
> for 'python-hypothesis' coming, probably this weekend.

That’s great!  Thank you!

> Wrt the reproducibility patch, I've tried to cherry-pick
> <https://github.com/python/cpython/pull/296> but got a couple of new
> test failures.  I have been really busy lately and got an injury to
> boot, so my work capacity is limited and any help very welcome.

I’ll take a look at this.  I wish you a speedy recovery.  Take care!

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net