Re: Python 3.5 start of update

[Leo Famulari <leo@famulari.name>]

[-- Attachment #1.1: Type: text/plain, Size: 2141 bytes --]

On Wed, Jun 22, 2016 at 08:15:02AM -0500, Christopher Allan Webber wrote:
> Ludovic Courtès writes:
> >> From bbc1f68204318d79ece3fa46ed71d7c21d6ec8a2 Mon Sep 17 00:00:00 2001
> >> From: Christopher Allan Webber <cwebber@dustycloud.org>
> 
> >> diff --git a/gnu/packages/patches/python-fix-tests-py3.5.patch b/gnu/packages/patches/python-fix-tests-py3.5.patch
> >> new file mode 100644
> >> index 0000000..47dded4
> >> --- /dev/null
> >> +++ b/gnu/packages/patches/python-fix-tests-py3.5.patch
> >> @@ -0,0 +1,35 @@
> >> +Additional test fixes which affect Python 3.5 (and presumably later) but not
> >> +prior revisions of Python.
> >> +
> >> +--- Lib/test/test_pathlib.py     2014-03-01 03:02:36.088311000 +0100
> >> ++++ Lib/test/test_pathlib.py     2014-03-01 04:56:37.768311000 +0100
> >> +@@ -1986,8 +1986,9 @@
> >> +         expect = set() if not support.fs_is_case_insensitive(BASE) else given
> >> +         self.assertEqual(given, expect)
> >> +         self.assertEqual(set(p.rglob("FILEd*")), set())
> >> + 
> >> ++    @unittest.skipIf(True, "Guix builder home is '/' which causes trouble for these tests")
> >
> > Could it be that adding a phase that does (setenv "HOME" (getcwd)) would
> > help?

IIUC, we already do that in python-2, and python-3 inherits from
python-2. So, it seems that it's not enough.

> I'll try to get the rest of this done this afternoon, but it might
> happen sometime later this week.

;)

I picked this up. I've attached two patches. They upgrade python-3.4 to
3.4.5, and then upgrade python-3 to 3.5.2 while preserving a python-3.4.
These are the latest releases in their respective series.

I had to split part of the python-fix-tests.patch into 3.4 and 3.5
specific patches, since those packages' codebases have diverged.

python@3.4.5 and python@3.5.2 both build on master and wip-python.
python-setuptools builds with python@3.5.2 on master.

I'd like to put these on core-updates. The wip-python branch will take a
while to get ready. I found this saga about what Nix had to do to
upgrade setuptools:
https://github.com/NixOS/nixpkgs/pull/12552

[-- Attachment #1.2: 0001-gnu-python-3.4-Update-to-3.4.5.patch --]
[-- Type: text/plain, Size: 4185 bytes --]

From 34f52581bed88368396490d6e42fd1cb50e0a3f1 Mon Sep 17 00:00:00 2001
From: Christopher Allan Webber <cwebber@dustycloud.org>
Date: Mon, 20 Jun 2016 14:05:20 -0500
Subject: [PATCH 1/2] gnu: python-3.4: Update to 3.4.5.

* gnu/packages/patches/python-disable-ssl-test.patch: Delete file.
* gnu/local.mk (dist_patch_DATA): Remove it.
* gnu/packages/python.scm (python-3.4): Update to 3.4.5.
[source]: Remove patch.
[arguments]: Remove field.
(python-minimal)[inputs]: Add libffi.
---
 gnu/local.mk                                       |  1 -
 gnu/packages/patches/python-disable-ssl-test.patch | 12 ------------
 gnu/packages/python.scm                            | 14 +++++---------
 3 files changed, 5 insertions(+), 22 deletions(-)
 delete mode 100644 gnu/packages/patches/python-disable-ssl-test.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index 3444c20..f201232 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -734,7 +734,6 @@ dist_patch_DATA =						\
   %D%/packages/patches/python-3-deterministic-build-info.patch	\
   %D%/packages/patches/python-3-search-paths.patch		\
   %D%/packages/patches/python-dendropy-exclude-failing-tests.patch \
-  %D%/packages/patches/python-disable-ssl-test.patch		\
   %D%/packages/patches/python-fix-tests.patch			\
   %D%/packages/patches/python-ipython-inputhook-ctype.patch	\
   %D%/packages/patches/python-rarfile-fix-tests.patch		\
diff --git a/gnu/packages/patches/python-disable-ssl-test.patch b/gnu/packages/patches/python-disable-ssl-test.patch
deleted file mode 100644
index e351c77..0000000
--- a/gnu/packages/patches/python-disable-ssl-test.patch
+++ /dev/null
@@ -1,12 +0,0 @@
-Disable a test that fails with openssl-1.0.2b.
-
---- Lib/test/test_ssl.py.orig	2015-02-25 06:27:45.000000000 -0500
-+++ Lib/test/test_ssl.py	2015-06-12 03:14:09.395212502 -0400
-@@ -2718,6 +2718,7 @@
-                                        chatty=True, connectionchatty=True)
-             self.assertIs(stats['compression'], None)
- 
-+        @unittest.skipIf(True, "openssl 1.0.2b complains: dh key too small")
-         def test_dh_params(self):
-             # Check we can get a connection with ephemeral Diffie-Hellman
-             context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index a83e4ee..5d529b0 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -301,21 +301,19 @@ data types.")
 
 (define-public python-3.4
   (package (inherit python-2)
-    (version "3.4.3")
+    (version "3.4.5")
     (source (origin
               (method url-fetch)
               (uri (string-append "https://www.python.org/ftp/python/"
                                   version "/Python-" version ".tar.xz"))
               (patches (search-patches
                         "python-fix-tests.patch"
-                        ;; XXX Try removing this patch for python > 3.4.3
-                        "python-disable-ssl-test.patch"
                         "python-3-deterministic-build-info.patch"
                         "python-3-search-paths.patch"))
               (patch-flags '("-p0"))
               (sha256
                (base32
-                "1f4nm4z08sy0kqwisvv95l02crv6dyysdmx44p1mz3bn6csrdcxm"))))
+                "12l9klp778wklxmckhghniy5hklss8r26995pyd00qbllk4b2r7f"))))
     (arguments (substitute-keyword-arguments (package-arguments python-2)
                  ((#:tests? _) #t)))
     (native-search-paths
@@ -348,14 +346,12 @@ data types.")
   (package (inherit python)
     (name "python-minimal")
     (outputs '("out"))
-    (arguments
-     (substitute-keyword-arguments (package-arguments python)
-       ((#:configure-flags cf)
-        `(append ,cf '("--without-system-ffi")))))
 
+    ;; Build fails due to missing ctypes without libffi.
     ;; OpenSSL is a mandatory dependency of Python 3.x, for urllib;
     ;; zlib is required by 'zipimport', used by pip.
-    (inputs `(("openssl" ,openssl)
+    (inputs `(("libffi" ,libffi)
+              ("openssl" ,openssl)
               ("zlib" ,zlib)))))
 
 (define* (wrap-python3 python
-- 
2.9.3


[-- Attachment #1.3: 0002-gnu-python-3-Update-to-python-3.5.2.patch --]
[-- Type: text/plain, Size: 8863 bytes --]

From d4b9e30eae7c230a93d9510d5680e7ef4ae6b9d6 Mon Sep 17 00:00:00 2001
From: Leo Famulari <leo@famulari.name>
Date: Sun, 21 Aug 2016 00:51:38 -0400
Subject: [PATCH 2/2] gnu: python-3: Update to python-3.5.2.

* gnu/packages/patches/python-3.4-fix-tests.patch,
gnu/packages/patches/python-3.5-fix-tests.patch: New files.
* gnu/local.mk (dist_patch_DATA): Add them.
* gnu/packages/patches/python-fix-tests.patch: Move python-3.4 specific
hunk to 'gnu/packages/patches/python-3.4-fix-tests.patch'.
* gnu/packages/python.scm (python-3.5): New variable.
(python-3.4): Inherit from python-3.5.
[source]: Use 'gnu/packages/patches/python-3.4-fix-tests.patch'.
(python-3): Point to python-3.5.

Co-authored-by: Diane Trout <diane@ghic.org>
Co-authored-by: Christopher Allan Webber <cwebber@dustycloud.org>
---
 gnu/local.mk                                    |  2 ++
 gnu/packages/patches/python-3.4-fix-tests.patch | 12 +++++++
 gnu/packages/patches/python-3.5-fix-tests.patch | 46 +++++++++++++++++++++++++
 gnu/packages/patches/python-fix-tests.patch     | 15 --------
 gnu/packages/python.scm                         | 26 +++++++++++---
 5 files changed, 82 insertions(+), 19 deletions(-)
 create mode 100644 gnu/packages/patches/python-3.4-fix-tests.patch
 create mode 100644 gnu/packages/patches/python-3.5-fix-tests.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index f201232..2b129a6 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -733,6 +733,8 @@ dist_patch_DATA =						\
   %D%/packages/patches/python-2.7-source-date-epoch.patch	\
   %D%/packages/patches/python-3-deterministic-build-info.patch	\
   %D%/packages/patches/python-3-search-paths.patch		\
+  %D%/packages/patches/python-3.4-fix-tests.patch		\
+  %D%/packages/patches/python-3.5-fix-tests.patch		\
   %D%/packages/patches/python-dendropy-exclude-failing-tests.patch \
   %D%/packages/patches/python-fix-tests.patch			\
   %D%/packages/patches/python-ipython-inputhook-ctype.patch	\
diff --git a/gnu/packages/patches/python-3.4-fix-tests.patch b/gnu/packages/patches/python-3.4-fix-tests.patch
new file mode 100644
index 0000000..d1f8138
--- /dev/null
+++ b/gnu/packages/patches/python-3.4-fix-tests.patch
@@ -0,0 +1,12 @@
+--- Lib/test/test_posixpath.py  2014-03-01 05:46:56.984311000 +0100
++++ Lib/test/test_posixpath.py  2014-03-07 00:59:20.888311000 +0100
+@@ -319,7 +319,11 @@
+                 del env['HOME']
+                 home = pwd.getpwuid(os.getuid()).pw_dir
+                 # $HOME can end with a trailing /, so strip it (see #17809)
+-                self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
++                # The Guix builders have '/' as a home directory, so
++                # home.rstrip("/") will be an empty string and the test will
++                # fail. Let's just disable it since it does not really make
++                # sense with such a bizarre setup.
++                # self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
diff --git a/gnu/packages/patches/python-3.5-fix-tests.patch b/gnu/packages/patches/python-3.5-fix-tests.patch
new file mode 100644
index 0000000..46d2a84
--- /dev/null
+++ b/gnu/packages/patches/python-3.5-fix-tests.patch
@@ -0,0 +1,46 @@
+Additional test fixes which affect Python 3.5 (and presumably later) but not
+prior revisions of Python.
+
+--- Lib/test/test_pathlib.py     2014-03-01 03:02:36.088311000 +0100
++++ Lib/test/test_pathlib.py     2014-03-01 04:56:37.768311000 +0100
+@@ -1986,8 +1986,9 @@
+         expect = set() if not support.fs_is_case_insensitive(BASE) else given
+         self.assertEqual(given, expect)
+         self.assertEqual(set(p.rglob("FILEd*")), set())
+ 
++    @unittest.skipIf(True, "Guix builder home is '/' which causes trouble for these tests")
+     def test_expanduser(self):
+         P = self.cls
+         support.import_module('pwd')
+         import pwd
+--- Lib/test/test_tarfile.py        2016-02-24 19:22:52.597208055 +0000
++++ Lib/test/test_tarfile.py     2016-02-24 20:50:48.941950135 +0000
+@@ -2305,11 +2305,14 @@
+     try:
+         import pwd, grp
+     except ImportError:
+         return False
+-    if pwd.getpwuid(0)[0] != 'root':
+-        return False
+-    if grp.getgrgid(0)[0] != 'root':
++    try:
++        if pwd.getpwuid(0)[0] != 'root':
++            return False
++        if grp.getgrgid(0)[0] != 'root':
++            return False
++    except KeyError:
+         return False
+     return True
+
+
+--- Lib/test/test_asyncio/test_base_events.py
++++ Lib/test/test_asyncio/test_base_events.py
+@@ -142,6 +142,8 @@ class BaseEventTests(test_utils.TestCase):
+             (INET, STREAM, TCP, '', ('1.2.3.4', 1)),
+             base_events._ipaddr_info('1.2.3.4', b'1', INET, STREAM, TCP))
+ 
++    @unittest.skipUnless(support.is_resource_enabled('network'),
++                         'network is not enabled')
+     def test_getaddrinfo_servname(self):
+         INET = socket.AF_INET
+         STREAM = socket.SOCK_STREAM
diff --git a/gnu/packages/patches/python-fix-tests.patch b/gnu/packages/patches/python-fix-tests.patch
index 82c1998..e093307 100644
--- a/gnu/packages/patches/python-fix-tests.patch
+++ b/gnu/packages/patches/python-fix-tests.patch
@@ -20,21 +20,6 @@ http://bugs.python.org/issue20868 .
      def test_tarfile_root_owner(self):
          tmpdir, tmpdir2, base_name =  self._create_files()
 
---- Lib/test/test_posixpath.py	2014-03-01 05:46:56.984311000 +0100
-+++ Lib/test/test_posixpath.py	2014-03-07 00:59:20.888311000 +0100
-@@ -319,7 +319,11 @@
-                 del env['HOME']
-                 home = pwd.getpwuid(os.getuid()).pw_dir
-                 # $HOME can end with a trailing /, so strip it (see #17809)
--                self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
-+                # The Guix builders have '/' as a home directory, so
-+                # home.rstrip("/") will be an empty string and the test will
-+                # fail. Let's just disable it since it does not really make
-+                # sense with such a bizarre setup.
-+                # self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
- 
-     def test_normpath(self):
-         self.assertEqual(posixpath.normpath(""), ".")
 --- Lib/test/test_socket.py.orig	2014-03-02 22:14:12.264311000 +0100
 +++ Lib/test/test_socket.py	2014-03-21 03:50:45.660311000 +0100
 @@ -819,6 +819,8 @@
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 5d529b0..975459e 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -299,21 +299,22 @@ data types.")
 ;; Current 2.x version.
 (define-public python-2 python-2.7)
 
-(define-public python-3.4
+(define-public python-3.5
   (package (inherit python-2)
-    (version "3.4.5")
+    (version "3.5.2")
     (source (origin
               (method url-fetch)
               (uri (string-append "https://www.python.org/ftp/python/"
                                   version "/Python-" version ".tar.xz"))
               (patches (search-patches
                         "python-fix-tests.patch"
+                        "python-3.5-fix-tests.patch"
                         "python-3-deterministic-build-info.patch"
                         "python-3-search-paths.patch"))
               (patch-flags '("-p0"))
               (sha256
                (base32
-                "12l9klp778wklxmckhghniy5hklss8r26995pyd00qbllk4b2r7f"))))
+                "0h6a5fr7ram2s483lh0pnmc4ncijb8llnpfdxdcl5dxr01hza400"))))
     (arguments (substitute-keyword-arguments (package-arguments python-2)
                  ((#:tests? _) #t)))
     (native-search-paths
@@ -323,8 +324,25 @@ data types.")
                                         (version-major+minor version)
                                         "/site-packages"))))))))
 
+(define-public python-3.4
+  (package (inherit python-3.5)
+    (version "3.4.5")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append "https://www.python.org/ftp/python/"
+                                  version "/Python-" version ".tar.xz"))
+              (patches (search-patches
+                        "python-fix-tests.patch"
+                        "python-3.4-fix-tests.patch"
+                        "python-3-deterministic-build-info.patch"
+                        "python-3-search-paths.patch"))
+              (patch-flags '("-p0"))
+              (sha256
+               (base32
+                "12l9klp778wklxmckhghniy5hklss8r26995pyd00qbllk4b2r7f"))))))
+
 ;; Current 3.x version.
-(define-public python-3 python-3.4)
+(define-public python-3 python-3.5)
 
 ;; Current major version.
 (define-public python python-3)
-- 
2.9.3


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]