* [PATCH 0/5] Python improvements.
@ 2014-03-07 2:47 Cyril Roelandt
2014-03-07 2:47 ` [PATCH 1/5] gnu: Enable the 'ctypes' module in Python Cyril Roelandt
` (4 more replies)
0 siblings, 5 replies; 21+ messages in thread
From: Cyril Roelandt @ 2014-03-07 2:47 UTC (permalink / raw)
To: guix-devel
This patch series, meant to be applied on the core-updates branch, brings a
bunch of improvements to the Python package.
The first one fixes bug #16569. The second one uses /nix/.../sh in the
subprocess module if /bin/sh is not available. I know Mark is not too found of
using /bin/sh if it is available, but I think Python users are willing to use
/bin/sh, that might be a symlink to whatever shell they may want to use. I don't
really feel too strongly about it, so if people agree on always using
/nix/.../sh, let's go with it, just let me know what you think. The last three
patches are (almost) trivial.
Regards,
Cyril.
---
Cyril Roelandt (5):
gnu: Enable the 'ctypes' module in Python.
gnu: Python: use /nix/.../sh if /bin/sh cannot be found
gnu: Enable tests in Python 3.
gnu: Python: bump to 3.3.4
gnu: remove python-fix-dbm.patch
gnu-system.am | 2 +-
gnu/packages/patches/python-fix-dbm.patch | 20 ----------
gnu/packages/patches/python-fix-tests.patch | 62 +++++++++++++++++++++++++++++
gnu/packages/python.scm | 51 ++++++++++++++++++++++--
4 files changed, 111 insertions(+), 24 deletions(-)
delete mode 100644 gnu/packages/patches/python-fix-dbm.patch
create mode 100644 gnu/packages/patches/python-fix-tests.patch
--
1.8.4.rc3
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 1/5] gnu: Enable the 'ctypes' module in Python.
2014-03-07 2:47 [PATCH 0/5] Python improvements Cyril Roelandt
@ 2014-03-07 2:47 ` Cyril Roelandt
2014-03-07 9:28 ` Ludovic Courtès
2014-03-07 2:47 ` [PATCH 2/5] gnu: Python: use /nix/.../sh if /bin/sh cannot be found Cyril Roelandt
` (3 subsequent siblings)
4 siblings, 1 reply; 21+ messages in thread
From: Cyril Roelandt @ 2014-03-07 2:47 UTC (permalink / raw)
To: guix-devel
* gnu/packages/python.scm (python-2): add libffi to the inputs and use it to
build the ctypes module.
---
gnu/packages/python.scm | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 44e3c14..8f92dc2 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -27,9 +27,11 @@
#:use-module (gnu packages compression)
#:use-module (gnu packages gdbm)
#:use-module (gnu packages icu4c)
+ #:use-module (gnu packages libffi)
#:use-module (gnu packages readline)
#:use-module (gnu packages openssl)
#:use-module (gnu packages elf)
+ #:use-module (gnu packages pkg-config)
#:use-module (gnu packages sqlite)
#:use-module (guix packages)
#:use-module (guix download)
@@ -98,10 +100,12 @@
#:configure-flags
(let ((bz2 (assoc-ref %build-inputs "bzip2"))
(gdbm (assoc-ref %build-inputs "gdbm"))
+ (libffi (assoc-ref %build-inputs "libffi"))
(openssl (assoc-ref %build-inputs "openssl"))
(readline (assoc-ref %build-inputs "readline"))
(zlib (assoc-ref %build-inputs "zlib")))
(list "--enable-shared" ; allow embedding
+ "--with-system-ffi" ; build ctypes
(string-append "CPPFLAGS="
"-I" bz2 "/include "
"-I" gdbm "/include "
@@ -111,6 +115,7 @@
(string-append "LDFLAGS="
"-L" bz2 "/lib "
"-L" gdbm "/lib "
+ "-L" libffi "/lib "
"-L" openssl "/lib "
"-L" readline "/lib "
"-L" zlib "/lib")))
@@ -137,10 +142,13 @@
(inputs
`(("bzip2" ,bzip2)
("gdbm" ,gdbm)
+ ("libffi" ,libffi) ; for ctypes
("openssl" ,openssl)
("readline" ,readline)
("zlib" ,zlib)
("patchelf" ,patchelf))) ; for (guix build rpath)
+ (native-inputs
+ `(("pkg-config" ,pkg-config)))
(native-search-paths
(list (search-path-specification
(variable "PYTHONPATH")
--
1.8.4.rc3
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 2/5] gnu: Python: use /nix/.../sh if /bin/sh cannot be found
2014-03-07 2:47 [PATCH 0/5] Python improvements Cyril Roelandt
2014-03-07 2:47 ` [PATCH 1/5] gnu: Enable the 'ctypes' module in Python Cyril Roelandt
@ 2014-03-07 2:47 ` Cyril Roelandt
2014-03-07 9:31 ` Ludovic Courtès
2014-03-07 2:47 ` [PATCH 3/5] gnu: Enable tests in Python 3 Cyril Roelandt
` (2 subsequent siblings)
4 siblings, 1 reply; 21+ messages in thread
From: Cyril Roelandt @ 2014-03-07 2:47 UTC (permalink / raw)
To: guix-devel
* gnu/packages/python.scm (python-2): patch Lib/subprocess.py to use
/nix/.../sh when /bin/sh is not available (most of the time, in Guix chroot)
---
gnu/packages/python.scm | 19 ++++++++++++++++++-
1 file changed, 18 insertions(+), 1 deletion(-)
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 8f92dc2..a663b5e 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -138,7 +138,24 @@
(with-directory-excursion out
(for-each (cut augment-rpath <> lib)
(find-files "bin" ".*")))))
- %standard-phases)))
+ (alist-replace
+ 'configure
+ (lambda* (#:key outputs #:allow-other-keys #:rest args)
+ (let ((configure (assoc-ref %standard-phases 'configure)))
+ ;; Some libraries try to call subprocess.Popen(), which uses
+ ;; /bin/sh, in their tests. Since /bin/sh is not available in the
+ ;; chroot, the tests fail. Instead of disabling them, add
+ ;; '/nix/.../sh' as an alternative shell. The default shell is
+ ;; still /bin/sh, so the subprocess module works exactly as
+ ;; expected. The extra 'if' condition will slow down things, but
+ ;; not enough to be noticed.
+ (substitute* "Lib/subprocess.py"
+ (("args = \\[\"/bin/sh")
+ (string-append
+ "args = [\"/bin/sh\" if os.path.exists(\"/bin/sh\") else "
+ "\"" (which "sh"))))
+ (apply configure args)))
+ %standard-phases))))
(inputs
`(("bzip2" ,bzip2)
("gdbm" ,gdbm)
--
1.8.4.rc3
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 3/5] gnu: Enable tests in Python 3.
2014-03-07 2:47 [PATCH 0/5] Python improvements Cyril Roelandt
2014-03-07 2:47 ` [PATCH 1/5] gnu: Enable the 'ctypes' module in Python Cyril Roelandt
2014-03-07 2:47 ` [PATCH 2/5] gnu: Python: use /nix/.../sh if /bin/sh cannot be found Cyril Roelandt
@ 2014-03-07 2:47 ` Cyril Roelandt
2014-03-07 9:40 ` Ludovic Courtès
2014-03-07 2:47 ` [PATCH 4/5] gnu: Python: bump to 3.3.4 Cyril Roelandt
2014-03-07 2:47 ` [PATCH 5/5] gnu: remove python-fix-dbm.patch Cyril Roelandt
4 siblings, 1 reply; 21+ messages in thread
From: Cyril Roelandt @ 2014-03-07 2:47 UTC (permalink / raw)
To: guix-devel
* gnu/packages/python.scm: enable tests for Python 3
* gnu/packages/python-fix-tests.patch: New file.
* gnu/packages/gnu-sysem.am (dist_patch_DATA): add it.
---
gnu-system.am | 1 +
gnu/packages/patches/python-fix-tests.patch | 62 +++++++++++++++++++++++++++++
gnu/packages/python.scm | 22 +++++++++-
3 files changed, 84 insertions(+), 1 deletion(-)
create mode 100644 gnu/packages/patches/python-fix-tests.patch
diff --git a/gnu-system.am b/gnu-system.am
index b5be893..6ef6edf 100644
--- a/gnu-system.am
+++ b/gnu-system.am
@@ -307,6 +307,7 @@ dist_patch_DATA = \
gnu/packages/patches/pulseaudio-test-timeouts.patch \
gnu/packages/patches/pulseaudio-volume-test.patch \
gnu/packages/patches/python-fix-dbm.patch \
+ gnu/packages/patches/python-fix-tests.patch \
gnu/packages/patches/qemu-make-4.0.patch \
gnu/packages/patches/qemu-multiple-smb-shares.patch \
gnu/packages/patches/qt4-tests.patch \
diff --git a/gnu/packages/patches/python-fix-tests.patch b/gnu/packages/patches/python-fix-tests.patch
new file mode 100644
index 0000000..23ef17f
--- /dev/null
+++ b/gnu/packages/patches/python-fix-tests.patch
@@ -0,0 +1,62 @@
+--- Lib/test/test_shutil.py 2014-03-01 04:56:37.768311000 +0100
++++ Lib/test/test_shutil.py 2014-03-01 03:02:36.088311000 +0100
+@@ -1053,7 +1053,6 @@
+ self.assertRaises(ValueError, make_archive, base_name, 'xxx')
+
+ @requires_zlib
+- @unittest.skipIf(True, "getgrgid(0)[0] raises a KeyError on Guix")
+ def test_make_archive_owner_group(self):
+ # testing make_archive with owner and group, with various combinations
+ # this works even if there's not gid/uid support
+@@ -1082,7 +1081,6 @@
+
+
+ @requires_zlib
+- @unittest.skipIf(True, "getgrgid(0)[0] raises a KeyError on Guix")
+ @unittest.skipUnless(UID_GID_SUPPORT, "Requires grp and pwd support")
+ def test_tarfile_root_owner(self):
+ tmpdir, tmpdir2, base_name = self._create_files()
+--- Lib/test/test_posixpath.py 2014-03-01 05:46:56.984311000 +0100
++++ Lib/test/test_posixpath.py 2014-03-01 06:20:50.704311000 +0100
+@@ -319,7 +319,11 @@
+ del env['HOME']
+ home = pwd.getpwuid(os.getuid()).pw_dir
+ # $HOME can end with a trailing /, so strip it (see #17809)
+- self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
++ # The Guix builders have '/' as a home directory, so
++ # home.rstrip("/") will be an empty string and the test will
++ # fail. Let's just disable it since it does not really make
++ # sense with such a bizarre setup.
++ # self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
+
+ def test_normpath(self):
+ self.assertEqual(posixpath.normpath(""), ".")
+--- Lib/test/test_socket.py 2014-03-02 22:14:12.264311000 +0100
++++ Lib/test/test_socket.py 2014-03-03 01:12:21.360311000 +0100
+@@ -819,6 +819,8 @@
+ self.assertRaises(OverflowError, socket.htonl, k)
+ self.assertRaises(OverflowError, socket.htons, k)
+
++ @unittest.skipUnless(os.path.exists("/etc/services"),
++ "getservbyname uses /etc/services, which is not in the chroot")
+ def testGetServBy(self):
+ eq = self.assertEqual
+ # Find one service that exists, then check all the related interfaces.
+@@ -1104,6 +1106,8 @@
+ self.assertRaises(ValueError, s.ioctl, -1, None)
+ s.ioctl(socket.SIO_KEEPALIVE_VALS, (1, 100, 100))
+
++ @unittest.skipUnless(os.path.exists("/etc/gai.conf"),
++ "getaddrinfo uses /etc/gai.conf, which is not in the chroot")
+ def testGetaddrinfo(self):
+ try:
+ socket.getaddrinfo('localhost', 80)
+@@ -1174,6 +1178,8 @@
+ # only IP addresses are allowed
+ self.assertRaises(socket.error, socket.getnameinfo, ('mail.python.org',0), 0)
+
++ @unittest.skipUnless(os.path.exists("/etc/gai.conf"),
++ "getaddrinfo uses /etc/gai.conf, which is not in the chroot")
+ @unittest.skipUnless(support.is_resource_enabled('network'),
+ 'network is not enabled')
+ def test_idna(self):
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index a663b5e..f58e6db 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -154,8 +154,17 @@
(string-append
"args = [\"/bin/sh\" if os.path.exists(\"/bin/sh\") else "
"\"" (which "sh"))))
+ (substitute*
+ '("Lib/distutils/tests/test_spawn.py"
+ "Lib/test/test_subprocess.py")
+ (("/bin/sh") (which "sh")))
(apply configure args)))
- %standard-phases))))
+ (alist-cons-before
+ 'check 'pre-check
+ (lambda _
+ ;; 'Lib/test/test_site.py' needs a valid $HOME
+ (setenv "HOME" (getcwd)))
+ %standard-phases)))))
(inputs
`(("bzip2" ,bzip2)
("gdbm" ,gdbm)
@@ -191,9 +200,20 @@ data types.")
(method url-fetch)
(uri (string-append "http://www.python.org/ftp/python/"
version "/Python-" version ".tar.xz"))
+ (patches (list (search-patch "python-fix-tests.patch")))
+ (patch-flags '("-p0"))
(sha256
(base32
"11f6hg9wdhm6hyzj49gxlvvp1s0l5hqgcsq1i4ayygqs1arpb4ik"))))
+ (arguments
+ (let ((args `(#:modules ((guix build gnu-build-system)
+ (guix build utils)
+ (srfi srfi-1)
+ (srfi srfi-26))
+ ,@(package-arguments python-2))))
+ (substitute-keyword-arguments args
+ ((#:tests? _)
+ `(list #t)))))
(native-search-paths
(list (search-path-specification
(variable "PYTHONPATH")
--
1.8.4.rc3
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 4/5] gnu: Python: bump to 3.3.4
2014-03-07 2:47 [PATCH 0/5] Python improvements Cyril Roelandt
` (2 preceding siblings ...)
2014-03-07 2:47 ` [PATCH 3/5] gnu: Enable tests in Python 3 Cyril Roelandt
@ 2014-03-07 2:47 ` Cyril Roelandt
2014-03-07 9:31 ` Ludovic Courtès
2014-03-07 2:47 ` [PATCH 5/5] gnu: remove python-fix-dbm.patch Cyril Roelandt
4 siblings, 1 reply; 21+ messages in thread
From: Cyril Roelandt @ 2014-03-07 2:47 UTC (permalink / raw)
To: guix-devel
* gnu/packages/python.scm (python): bump to 3.3.4
---
gnu/packages/python.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index f58e6db..df53b18 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -194,7 +194,7 @@ data types.")
(define-public python
(package (inherit python-2)
- (version "3.3.3")
+ (version "3.3.4")
(source
(origin
(method url-fetch)
@@ -204,7 +204,7 @@ data types.")
(patch-flags '("-p0"))
(sha256
(base32
- "11f6hg9wdhm6hyzj49gxlvvp1s0l5hqgcsq1i4ayygqs1arpb4ik"))))
+ "12ank7in8xyncim3yyn3mi84wkc4g9nx7yrci1406kn0j5ni5k66"))))
(arguments
(let ((args `(#:modules ((guix build gnu-build-system)
(guix build utils)
--
1.8.4.rc3
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH 5/5] gnu: remove python-fix-dbm.patch
2014-03-07 2:47 [PATCH 0/5] Python improvements Cyril Roelandt
` (3 preceding siblings ...)
2014-03-07 2:47 ` [PATCH 4/5] gnu: Python: bump to 3.3.4 Cyril Roelandt
@ 2014-03-07 2:47 ` Cyril Roelandt
2014-03-07 9:32 ` Ludovic Courtès
4 siblings, 1 reply; 21+ messages in thread
From: Cyril Roelandt @ 2014-03-07 2:47 UTC (permalink / raw)
To: guix-devel
* gnu/packages/patches/python-fix-dbm.patch: remove file. It is not needed
anymore, and is probably a left-over of a failed merge.
* gnu-system.am: remove gnu/packages/patches/python-fix-dbm.patch
---
gnu-system.am | 1 -
gnu/packages/patches/python-fix-dbm.patch | 20 --------------------
2 files changed, 21 deletions(-)
delete mode 100644 gnu/packages/patches/python-fix-dbm.patch
diff --git a/gnu-system.am b/gnu-system.am
index 6ef6edf..2bd7656 100644
--- a/gnu-system.am
+++ b/gnu-system.am
@@ -306,7 +306,6 @@ dist_patch_DATA = \
gnu/packages/patches/procps-make-3.82.patch \
gnu/packages/patches/pulseaudio-test-timeouts.patch \
gnu/packages/patches/pulseaudio-volume-test.patch \
- gnu/packages/patches/python-fix-dbm.patch \
gnu/packages/patches/python-fix-tests.patch \
gnu/packages/patches/qemu-make-4.0.patch \
gnu/packages/patches/qemu-multiple-smb-shares.patch \
diff --git a/gnu/packages/patches/python-fix-dbm.patch b/gnu/packages/patches/python-fix-dbm.patch
deleted file mode 100644
index 29e4521..0000000
--- a/gnu/packages/patches/python-fix-dbm.patch
+++ /dev/null
@@ -1,20 +0,0 @@
-This patch allows the dbm module to be built using the compatibility mode of
-gdbm. It will not be needed any more with Python 2.7.4.
---- setup.py 2013-04-06 00:53:37.000000000 +0200
-+++ setup.py.new 2013-04-06 19:55:05.000000000 +0200
-@@ -1158,10 +1158,14 @@
- for cand in dbm_order:
- if cand == "ndbm":
- if find_file("ndbm.h", inc_dirs, []) is not None:
-- # Some systems have -lndbm, others don't
-+ # Some systems have -lndbm, some have -lgdbm_compat,
-+ # others have no particular linker flags.
- if self.compiler.find_library_file(lib_dirs,
- 'ndbm'):
- ndbm_libs = ['ndbm']
-+ elif self.compiler.find_library_file(lib_dirs,
-+ 'gdbm_compat'):
-+ ndbm_libs = ['gdbm_compat']
- else:
- ndbm_libs = []
- print "building dbm using ndbm"
--
1.8.4.rc3
^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [PATCH 1/5] gnu: Enable the 'ctypes' module in Python.
2014-03-07 2:47 ` [PATCH 1/5] gnu: Enable the 'ctypes' module in Python Cyril Roelandt
@ 2014-03-07 9:28 ` Ludovic Courtès
0 siblings, 0 replies; 21+ messages in thread
From: Ludovic Courtès @ 2014-03-07 9:28 UTC (permalink / raw)
To: Cyril Roelandt; +Cc: guix-devel
Cyril Roelandt <tipecaml@gmail.com> skribis:
> * gnu/packages/python.scm (python-2): add libffi to the inputs and use it to
> build the ctypes module.
OK to push.
Ludo’.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 2/5] gnu: Python: use /nix/.../sh if /bin/sh cannot be found
2014-03-07 2:47 ` [PATCH 2/5] gnu: Python: use /nix/.../sh if /bin/sh cannot be found Cyril Roelandt
@ 2014-03-07 9:31 ` Ludovic Courtès
2014-03-22 3:36 ` [PATCH 2/4 v2] gnu: Python: use /nix/.../sh instead of /bin/sh in the subprocess module Cyril Roelandt
0 siblings, 1 reply; 21+ messages in thread
From: Ludovic Courtès @ 2014-03-07 9:31 UTC (permalink / raw)
To: Cyril Roelandt; +Cc: guix-devel
Cyril Roelandt <tipecaml@gmail.com> skribis:
> * gnu/packages/python.scm (python-2): patch Lib/subprocess.py to use
> /nix/.../sh when /bin/sh is not available (most of the time, in Guix chroot)
> ---
> gnu/packages/python.scm | 19 ++++++++++++++++++-
> 1 file changed, 18 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
> index 8f92dc2..a663b5e 100644
> --- a/gnu/packages/python.scm
> +++ b/gnu/packages/python.scm
> @@ -138,7 +138,24 @@
> (with-directory-excursion out
> (for-each (cut augment-rpath <> lib)
> (find-files "bin" ".*")))))
> - %standard-phases)))
> + (alist-replace
> + 'configure
> + (lambda* (#:key outputs #:allow-other-keys #:rest args)
> + (let ((configure (assoc-ref %standard-phases 'configure)))
> + ;; Some libraries try to call subprocess.Popen(), which uses
> + ;; /bin/sh, in their tests. Since /bin/sh is not available in the
> + ;; chroot, the tests fail. Instead of disabling them, add
> + ;; '/nix/.../sh' as an alternative shell. The default shell is
> + ;; still /bin/sh, so the subprocess module works exactly as
> + ;; expected. The extra 'if' condition will slow down things, but
> + ;; not enough to be noticed.
> + (substitute* "Lib/subprocess.py"
> + (("args = \\[\"/bin/sh")
> + (string-append
> + "args = [\"/bin/sh\" if os.path.exists(\"/bin/sh\") else "
> + "\"" (which "sh"))))
> + (apply configure args)))
I’m in favor of just (which "sh").
As discussed on IRC, all one can expect from /bin/sh is to point to a
Bourne-compatible shell, and that’s what we’d guarantee by explicitly
using the build-time (which "sh").
Fine with you?
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 4/5] gnu: Python: bump to 3.3.4
2014-03-07 2:47 ` [PATCH 4/5] gnu: Python: bump to 3.3.4 Cyril Roelandt
@ 2014-03-07 9:31 ` Ludovic Courtès
2014-03-21 4:07 ` Cyril Roelandt
0 siblings, 1 reply; 21+ messages in thread
From: Ludovic Courtès @ 2014-03-07 9:31 UTC (permalink / raw)
To: Cyril Roelandt; +Cc: guix-devel
Cyril Roelandt <tipecaml@gmail.com> skribis:
> * gnu/packages/python.scm (python): bump to 3.3.4
OK to push.
Ludo’.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 5/5] gnu: remove python-fix-dbm.patch
2014-03-07 2:47 ` [PATCH 5/5] gnu: remove python-fix-dbm.patch Cyril Roelandt
@ 2014-03-07 9:32 ` Ludovic Courtès
0 siblings, 0 replies; 21+ messages in thread
From: Ludovic Courtès @ 2014-03-07 9:32 UTC (permalink / raw)
To: Cyril Roelandt; +Cc: guix-devel
Cyril Roelandt <tipecaml@gmail.com> skribis:
> * gnu/packages/patches/python-fix-dbm.patch: remove file. It is not needed
> anymore, and is probably a left-over of a failed merge.
> * gnu-system.am: remove gnu/packages/patches/python-fix-dbm.patch
OK to push.
Ludo’.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 3/5] gnu: Enable tests in Python 3.
2014-03-07 2:47 ` [PATCH 3/5] gnu: Enable tests in Python 3 Cyril Roelandt
@ 2014-03-07 9:40 ` Ludovic Courtès
2014-03-21 4:12 ` [PATCH 3/4 v2] " Cyril Roelandt
2014-03-21 4:14 ` [PATCH 3/5] " Cyril Roelandt
0 siblings, 2 replies; 21+ messages in thread
From: Ludovic Courtès @ 2014-03-07 9:40 UTC (permalink / raw)
To: Cyril Roelandt; +Cc: guix-devel
Cyril Roelandt <tipecaml@gmail.com> skribis:
> * gnu/packages/python.scm: enable tests for Python 3
> * gnu/packages/python-fix-tests.patch: New file.
> * gnu/packages/gnu-sysem.am (dist_patch_DATA): add it.
Typo here.
> diff --git a/gnu/packages/patches/python-fix-tests.patch b/gnu/packages/patches/python-fix-tests.patch
> new file mode 100644
> index 0000000..23ef17f
> --- /dev/null
> +++ b/gnu/packages/patches/python-fix-tests.patch
> @@ -0,0 +1,62 @@
> +--- Lib/test/test_shutil.py 2014-03-01 04:56:37.768311000 +0100
> ++++ Lib/test/test_shutil.py 2014-03-01 03:02:36.088311000 +0100
> +@@ -1053,7 +1053,6 @@
> + self.assertRaises(ValueError, make_archive, base_name, 'xxx')
> +
Could you add a couple of lines of summary and (possibly) and a link to
the upstream report/discussion (or a statement on the upstream status)?
> + @requires_zlib
> +- @unittest.skipIf(True, "getgrgid(0)[0] raises a KeyError on Guix")
It seems that this part of the patch is reversed, no?
> +--- Lib/test/test_posixpath.py 2014-03-01 05:46:56.984311000 +0100
> ++++ Lib/test/test_posixpath.py 2014-03-01 06:20:50.704311000 +0100
> +@@ -319,7 +319,11 @@
> + del env['HOME']
> + home = pwd.getpwuid(os.getuid()).pw_dir
> + # $HOME can end with a trailing /, so strip it (see #17809)
> +- self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
> ++ # The Guix builders have '/' as a home directory, so
> ++ # home.rstrip("/") will be an empty string and the test will
> ++ # fail. Let's just disable it since it does not really make
> ++ # sense with such a bizarre setup.
> ++ # self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
I see that the recipe’s ‘pre-check’ phase does:
(setenv "HOME" (getcwd))
so $HOME should actually be /tmp/nix-build-xxx, not just /, no?
> +--- Lib/test/test_socket.py 2014-03-02 22:14:12.264311000 +0100
> ++++ Lib/test/test_socket.py 2014-03-03 01:12:21.360311000 +0100
> +@@ -819,6 +819,8 @@
> + self.assertRaises(OverflowError, socket.htonl, k)
> + self.assertRaises(OverflowError, socket.htons, k)
> +
> ++ @unittest.skipUnless(os.path.exists("/etc/services"),
> ++ "getservbyname uses /etc/services, which is not in the chroot")
(I think that should go upstream.)
> ++ @unittest.skipUnless(os.path.exists("/etc/gai.conf"),
> ++ "getaddrinfo uses /etc/gai.conf, which is not in the chroot")
I don’t think gai.conf is the problem. Instead, the problem is that one
cannot rely on name lookups in the build environment at all, and even in
general.
So this patch may be OK for ourselves (except the comment), but upstream
Python should really just gracefully skip the test if name lookup
fails.
> + (arguments
> + (let ((args `(#:modules ((guix build gnu-build-system)
> + (guix build utils)
> + (srfi srfi-1)
> + (srfi srfi-26))
> + ,@(package-arguments python-2))))
> + (substitute-keyword-arguments args
> + ((#:tests? _)
> + `(list #t)))))
#t instead of `(list #t) should do the job. :-)
Thanks for going through all this!
Ludo’.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 4/5] gnu: Python: bump to 3.3.4
2014-03-07 9:31 ` Ludovic Courtès
@ 2014-03-21 4:07 ` Cyril Roelandt
2014-03-21 13:02 ` Ludovic Courtès
0 siblings, 1 reply; 21+ messages in thread
From: Cyril Roelandt @ 2014-03-21 4:07 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: guix-devel
On 03/07/2014 10:31 AM, Ludovic Courtès wrote:
> Cyril Roelandt <tipecaml@gmail.com> skribis:
>
>> * gnu/packages/python.scm (python): bump to 3.3.4
>
> OK to push.
>
> Ludo’.
>
I'll drop this one. Since I've written this patch, Python 3.4 came out,
and it brings more test failures :)
Cyril.
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 3/4 v2] gnu: Enable tests in Python 3.
2014-03-07 9:40 ` Ludovic Courtès
@ 2014-03-21 4:12 ` Cyril Roelandt
2014-03-22 14:32 ` Ludovic Courtès
2014-03-21 4:14 ` [PATCH 3/5] " Cyril Roelandt
1 sibling, 1 reply; 21+ messages in thread
From: Cyril Roelandt @ 2014-03-21 4:12 UTC (permalink / raw)
To: guix-devel
* gnu/packages/python.scm: enable tests for Python 3
* gnu/packages/python-fix-tests.patch: New file.
* gnu/packages/gnu-system.am (dist_patch_DATA): add it.
---
gnu-system.am | 1 +
gnu/packages/patches/python-fix-tests.patch | 66 +++++++++++++++++++++++++++++
gnu/packages/python.scm | 21 ++++++++-
3 files changed, 87 insertions(+), 1 deletion(-)
create mode 100644 gnu/packages/patches/python-fix-tests.patch
diff --git a/gnu-system.am b/gnu-system.am
index 52c58d8..91896ba 100644
--- a/gnu-system.am
+++ b/gnu-system.am
@@ -311,6 +311,7 @@ dist_patch_DATA = \
gnu/packages/patches/plotutils-libpng-jmpbuf.patch \
gnu/packages/patches/procps-make-3.82.patch \
gnu/packages/patches/python-fix-dbm.patch \
+ gnu/packages/patches/python-fix-tests.patch \
gnu/packages/patches/qemu-make-4.0.patch \
gnu/packages/patches/qemu-multiple-smb-shares.patch \
gnu/packages/patches/qt4-tests.patch \
diff --git a/gnu/packages/patches/python-fix-tests.patch b/gnu/packages/patches/python-fix-tests.patch
new file mode 100644
index 0000000..fecebda
--- /dev/null
+++ b/gnu/packages/patches/python-fix-tests.patch
@@ -0,0 +1,66 @@
+See the discussion about the issues fixed here at:
+http://bugs.python.org/issue20868 .
+
+--- Lib/test/test_shutil.py 2014-03-01 03:02:36.088311000 +0100
++++ Lib/test/test_shutil.py 2014-03-01 04:56:37.768311000 +0100
+@@ -1053,6 +1053,7 @@
+ self.assertRaises(ValueError, make_archive, base_name, 'xxx')
+
+ @requires_zlib
++ @unittest.skipIf(True, "getgrgid(0)[0] raises a KeyError on Guix")
+ def test_make_archive_owner_group(self):
+ # testing make_archive with owner and group, with various combinations
+ # this works even if there's not gid/uid support
+@@ -1081,6 +1082,7 @@
+
+
+ @requires_zlib
++ @unittest.skipIf(True, "getgrgid(0)[0] raises a KeyError on Guix")
+ @unittest.skipUnless(UID_GID_SUPPORT, "Requires grp and pwd support")
+ def test_tarfile_root_owner(self):
+ tmpdir, tmpdir2, base_name = self._create_files()
+
+--- Lib/test/test_posixpath.py 2014-03-01 05:46:56.984311000 +0100
++++ Lib/test/test_posixpath.py 2014-03-07 00:59:20.888311000 +0100
+@@ -319,7 +319,11 @@
+ del env['HOME']
+ home = pwd.getpwuid(os.getuid()).pw_dir
+ # $HOME can end with a trailing /, so strip it (see #17809)
+- self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
++ # The Guix builders have '/' as a home directory, so
++ # home.rstrip("/") will be an empty string and the test will
++ # fail. Let's just disable it since it does not really make
++ # sense with such a bizarre setup.
++ # self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
+
+ def test_normpath(self):
+ self.assertEqual(posixpath.normpath(""), ".")
+--- Lib/test/test_socket.py.orig 2014-03-02 22:14:12.264311000 +0100
++++ Lib/test/test_socket.py 2014-03-21 03:50:45.660311000 +0100
+@@ -819,6 +819,8 @@
+ self.assertRaises(OverflowError, socket.htonl, k)
+ self.assertRaises(OverflowError, socket.htons, k)
+
++ @unittest.skipUnless(os.path.exists("/etc/services"),
++ "getservbyname uses /etc/services, which is not in the chroot")
+ def testGetServBy(self):
+ eq = self.assertEqual
+ # Find one service that exists, then check all the related interfaces.
+@@ -1104,6 +1106,8 @@
+ self.assertRaises(ValueError, s.ioctl, -1, None)
+ s.ioctl(socket.SIO_KEEPALIVE_VALS, (1, 100, 100))
+
++ @unittest.skipUnless(os.path.exists("/etc/gai.conf"),
++ "getaddrinfo() will fail")
+ def testGetaddrinfo(self):
+ try:
+ socket.getaddrinfo('localhost', 80)
+@@ -1174,6 +1178,8 @@
+ # only IP addresses are allowed
+ self.assertRaises(socket.error, socket.getnameinfo, ('mail.python.org',0), 0)
+
++ @unittest.skipUnless(os.path.exists("/etc/gai.conf"),
++ "getaddrinfo() will fail")
+ @unittest.skipUnless(support.is_resource_enabled('network'),
+ 'network is not enabled')
+ def test_idna(self):
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 2c69926..01de2f6 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -146,8 +146,17 @@
(substitute* "Lib/subprocess.py"
(("args = \\[\"/bin/sh")
(string-append "args = [\"" (which "sh"))))
+ (substitute*
+ '("Lib/distutils/tests/test_spawn.py"
+ "Lib/test/test_subprocess.py")
+ (("/bin/sh") (which "sh")))
(apply configure args)))
- %standard-phases))))
+ (alist-cons-before
+ 'check 'pre-check
+ (lambda _
+ ;; 'Lib/test/test_site.py' needs a valid $HOME
+ (setenv "HOME" (getcwd)))
+ %standard-phases)))))
(inputs
`(("bzip2" ,bzip2)
("gdbm" ,gdbm)
@@ -183,9 +192,19 @@ data types.")
(method url-fetch)
(uri (string-append "https://www.python.org/ftp/python/"
version "/Python-" version ".tar.xz"))
+ (patches (list (search-patch "python-fix-tests.patch")))
+ (patch-flags '("-p0"))
(sha256
(base32
"11f6hg9wdhm6hyzj49gxlvvp1s0l5hqgcsq1i4ayygqs1arpb4ik"))))
+ (arguments
+ (let ((args `(#:modules ((guix build gnu-build-system)
+ (guix build utils)
+ (srfi srfi-1)
+ (srfi srfi-26))
+ ,@(package-arguments python-2))))
+ (substitute-keyword-arguments args
+ ((#:tests? _) #t))))
(native-search-paths
(list (search-path-specification
(variable "PYTHONPATH")
--
1.8.4.rc3
^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [PATCH 3/5] gnu: Enable tests in Python 3.
2014-03-07 9:40 ` Ludovic Courtès
2014-03-21 4:12 ` [PATCH 3/4 v2] " Cyril Roelandt
@ 2014-03-21 4:14 ` Cyril Roelandt
2014-03-21 13:15 ` Ludovic Courtès
1 sibling, 1 reply; 21+ messages in thread
From: Cyril Roelandt @ 2014-03-21 4:14 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: guix-devel
On 03/07/2014 10:40 AM, Ludovic Courtès wrote:
> Cyril Roelandt <tipecaml@gmail.com> skribis:
>
>> * gnu/packages/python.scm: enable tests for Python 3
>> * gnu/packages/python-fix-tests.patch: New file.
>> * gnu/packages/gnu-sysem.am (dist_patch_DATA): add it.
>
> Typo here.
>
Fixed.
>> diff --git a/gnu/packages/patches/python-fix-tests.patch b/gnu/packages/patches/python-fix-tests.patch
>> new file mode 100644
>> index 0000000..23ef17f
>> --- /dev/null
>> +++ b/gnu/packages/patches/python-fix-tests.patch
>> @@ -0,0 +1,62 @@
>> +--- Lib/test/test_shutil.py 2014-03-01 04:56:37.768311000 +0100
>> ++++ Lib/test/test_shutil.py 2014-03-01 03:02:36.088311000 +0100
>> +@@ -1053,7 +1053,6 @@
>> + self.assertRaises(ValueError, make_archive, base_name, 'xxx')
>> +
>
> Could you add a couple of lines of summary and (possibly) and a link to
> the upstream report/discussion (or a statement on the upstream status)?
Done.
>
>> + @requires_zlib
>> +- @unittest.skipIf(True, "getgrgid(0)[0] raises a KeyError on Guix")
>
> It seems that this part of the patch is reversed, no?
>
Oops. Fixed.
>> +--- Lib/test/test_posixpath.py 2014-03-01 05:46:56.984311000 +0100
>> ++++ Lib/test/test_posixpath.py 2014-03-01 06:20:50.704311000 +0100
>> +@@ -319,7 +319,11 @@
>> + del env['HOME']
>> + home = pwd.getpwuid(os.getuid()).pw_dir
>> + # $HOME can end with a trailing /, so strip it (see #17809)
>> +- self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
>> ++ # The Guix builders have '/' as a home directory, so
>> ++ # home.rstrip("/") will be an empty string and the test will
>> ++ # fail. Let's just disable it since it does not really make
>> ++ # sense with such a bizarre setup.
>> ++ # self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
>
> I see that the recipe’s ‘pre-check’ phase does:
>
> (setenv "HOME" (getcwd))
>
> so $HOME should actually be /tmp/nix-build-xxx, not just /, no?
>
Yes, but I think it uses getpwent() or something that does not use $HOME
to retrieve the home directory in this case.
>> +--- Lib/test/test_socket.py 2014-03-02 22:14:12.264311000 +0100
>> ++++ Lib/test/test_socket.py 2014-03-03 01:12:21.360311000 +0100
>> +@@ -819,6 +819,8 @@
>> + self.assertRaises(OverflowError, socket.htonl, k)
>> + self.assertRaises(OverflowError, socket.htons, k)
>> +
>> ++ @unittest.skipUnless(os.path.exists("/etc/services"),
>> ++ "getservbyname uses /etc/services, which is not in the chroot")
>
> (I think that should go upstream.)
>
>> ++ @unittest.skipUnless(os.path.exists("/etc/gai.conf"),
>> ++ "getaddrinfo uses /etc/gai.conf, which is not in the chroot")
>
> I don’t think gai.conf is the problem. Instead, the problem is that one
> cannot rely on name lookups in the build environment at all, and even in
> general.
>
> So this patch may be OK for ourselves (except the comment), but upstream
> Python should really just gracefully skip the test if name lookup
> fails.
>
>> + (arguments
>> + (let ((args `(#:modules ((guix build gnu-build-system)
>> + (guix build utils)
>> + (srfi srfi-1)
>> + (srfi srfi-26))
>> + ,@(package-arguments python-2))))
>> + (substitute-keyword-arguments args
>> + ((#:tests? _)
>> + `(list #t)))))
>
> #t instead of `(list #t) should do the job. :-)
>
Fixed.
Cyril.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 4/5] gnu: Python: bump to 3.3.4
2014-03-21 4:07 ` Cyril Roelandt
@ 2014-03-21 13:02 ` Ludovic Courtès
0 siblings, 0 replies; 21+ messages in thread
From: Ludovic Courtès @ 2014-03-21 13:02 UTC (permalink / raw)
To: Cyril Roelandt; +Cc: guix-devel
Cyril Roelandt <tipecaml@gmail.com> skribis:
> On 03/07/2014 10:31 AM, Ludovic Courtès wrote:
>> Cyril Roelandt <tipecaml@gmail.com> skribis:
>>
>>> * gnu/packages/python.scm (python): bump to 3.3.4
>>
>> OK to push.
>>
>> Ludo’.
>>
>
> I'll drop this one. Since I've written this patch, Python 3.4 came
> out, and it brings more test failures :)
Interesting. You’ll have to keep up with the release rate. :-)
Ludo’.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 3/5] gnu: Enable tests in Python 3.
2014-03-21 4:14 ` [PATCH 3/5] " Cyril Roelandt
@ 2014-03-21 13:15 ` Ludovic Courtès
2014-03-22 23:13 ` Cyril Roelandt
0 siblings, 1 reply; 21+ messages in thread
From: Ludovic Courtès @ 2014-03-21 13:15 UTC (permalink / raw)
To: Cyril Roelandt; +Cc: guix-devel
Cyril Roelandt <tipecaml@gmail.com> skribis:
> On 03/07/2014 10:40 AM, Ludovic Courtès wrote:
>> Cyril Roelandt <tipecaml@gmail.com> skribis:
[...]
>>> +--- Lib/test/test_posixpath.py 2014-03-01 05:46:56.984311000 +0100
>>> ++++ Lib/test/test_posixpath.py 2014-03-01 06:20:50.704311000 +0100
>>> +@@ -319,7 +319,11 @@
>>> + del env['HOME']
>>> + home = pwd.getpwuid(os.getuid()).pw_dir
>>> + # $HOME can end with a trailing /, so strip it (see #17809)
>>> +- self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
>>> ++ # The Guix builders have '/' as a home directory, so
>>> ++ # home.rstrip("/") will be an empty string and the test will
>>> ++ # fail. Let's just disable it since it does not really make
>>> ++ # sense with such a bizarre setup.
>>> ++ # self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
>>
>> I see that the recipe’s ‘pre-check’ phase does:
>>
>> (setenv "HOME" (getcwd))
>>
>> so $HOME should actually be /tmp/nix-build-xxx, not just /, no?
>>
>
> Yes, but I think it uses getpwent() or something that does not use
> $HOME to retrieve the home directory in this case.
Here’s what I see, with nscd turned off (this uses getpwnam(2)):
--8<---------------cut here---------------start------------->8---
$ strace guile -c '(getpw "ludo")'
[...]
open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=2072, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f264dea5000
read(5, "root:x:0:0:System administrator:"..., 4096) = 2072
--8<---------------cut here---------------end--------------->8---
Normally the chroot environment has all that too. And indeed, the
equivalent of the above Python snippet seems to work:
--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(guix)
scheme@(guile-user)> ,use(guix monads)
scheme@(guile-user)> (define s (open-connection))
scheme@(guile-user)> (derivation-expression "getpw" '(pk (getpw (getuid))))
$11 = #<procedure 3da30f0 at guix/monads.scm:297:6 (store)>
scheme@(guile-user)> (run-with-store s $11)
$12 = #<derivation /gnu/store/bhmd5bg9s64an3s8ssmwx6l90dkr56hx-getpw.drv => /gnu/store/z68rzri3myxx1dg931fb1ksxdg7c29zg-getpw 33dfa00>
scheme@(guile-user)> (build-derivations s (list $12))
building path(s) `/gnu/store/z68rzri3myxx1dg931fb1ksxdg7c29zg-getpw'
;;; (#("nixbld" "x" 30001 30000 "Nix build user" "/" "/noshell"))
builder for `/gnu/store/bhmd5bg9s64an3s8ssmwx6l90dkr56hx-getpw.drv' failed to produce output path `/gnu/store/z68rzri3myxx1dg931fb1ksxdg7c29zg-getpw'
--8<---------------cut here---------------end--------------->8---
Libguile implements this using getpwuid(3). I wonder if CPython is
doing something else.
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH 2/4 v2] gnu: Python: use /nix/.../sh instead of /bin/sh in the subprocess module
2014-03-07 9:31 ` Ludovic Courtès
@ 2014-03-22 3:36 ` Cyril Roelandt
2014-03-22 14:26 ` Ludovic Courtès
0 siblings, 1 reply; 21+ messages in thread
From: Cyril Roelandt @ 2014-03-22 3:36 UTC (permalink / raw)
To: guix-devel
* gnu/packages/python.scm (python-2): patch Lib/subprocess.py to use
/nix/.../sh.
---
gnu/packages/python.scm | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 056956e..2c69926 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -139,7 +139,15 @@
(with-directory-excursion out
(for-each (cut augment-rpath <> lib)
(find-files "bin" ".*")))))
- %standard-phases)))
+ (alist-replace
+ 'configure
+ (lambda* (#:key outputs #:allow-other-keys #:rest args)
+ (let ((configure (assoc-ref %standard-phases 'configure)))
+ (substitute* "Lib/subprocess.py"
+ (("args = \\[\"/bin/sh")
+ (string-append "args = [\"" (which "sh"))))
+ (apply configure args)))
+ %standard-phases))))
(inputs
`(("bzip2" ,bzip2)
("gdbm" ,gdbm)
--
1.8.4.rc3
^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [PATCH 2/4 v2] gnu: Python: use /nix/.../sh instead of /bin/sh in the subprocess module
2014-03-22 3:36 ` [PATCH 2/4 v2] gnu: Python: use /nix/.../sh instead of /bin/sh in the subprocess module Cyril Roelandt
@ 2014-03-22 14:26 ` Ludovic Courtès
0 siblings, 0 replies; 21+ messages in thread
From: Ludovic Courtès @ 2014-03-22 14:26 UTC (permalink / raw)
To: Cyril Roelandt; +Cc: guix-devel
Cyril Roelandt <tipecaml@gmail.com> skribis:
> * gnu/packages/python.scm (python-2): patch Lib/subprocess.py to use
> /nix/.../sh.
OK for core-updates.
Ludo'.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 3/4 v2] gnu: Enable tests in Python 3.
2014-03-21 4:12 ` [PATCH 3/4 v2] " Cyril Roelandt
@ 2014-03-22 14:32 ` Ludovic Courtès
0 siblings, 0 replies; 21+ messages in thread
From: Ludovic Courtès @ 2014-03-22 14:32 UTC (permalink / raw)
To: Cyril Roelandt; +Cc: guix-devel
Cyril Roelandt <tipecaml@gmail.com> skribis:
> * gnu/packages/python.scm: enable tests for Python 3
> * gnu/packages/python-fix-tests.patch: New file.
> * gnu/packages/gnu-system.am (dist_patch_DATA): add it.
OK, with one detail:
> ++ @unittest.skipUnless(os.path.exists("/etc/gai.conf"),
> ++ "getaddrinfo() will fail")
> + def testGetaddrinfo(self):
Is this an upstream patch? If yes, then fine, otherwise I think we
should not mention gai.conf since it’s really not the issue.
So @unittest.skipUnless(false, "getaddrinfo() will fail") would be good
enough IMO.
OK to push with these last thing addressed. :-)
Ludo’.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 3/5] gnu: Enable tests in Python 3.
2014-03-21 13:15 ` Ludovic Courtès
@ 2014-03-22 23:13 ` Cyril Roelandt
2014-03-23 10:43 ` Ludovic Courtès
0 siblings, 1 reply; 21+ messages in thread
From: Cyril Roelandt @ 2014-03-22 23:13 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: guix-devel
On 03/21/2014 02:15 PM, Ludovic Courtès wrote:
> Cyril Roelandt <tipecaml@gmail.com> skribis:
>
>> On 03/07/2014 10:40 AM, Ludovic Courtès wrote:
>>> Cyril Roelandt <tipecaml@gmail.com> skribis:
>
> [...]
>
>>>> +--- Lib/test/test_posixpath.py 2014-03-01 05:46:56.984311000 +0100
>>>> ++++ Lib/test/test_posixpath.py 2014-03-01 06:20:50.704311000 +0100
>>>> +@@ -319,7 +319,11 @@
>>>> + del env['HOME']
>>>> + home = pwd.getpwuid(os.getuid()).pw_dir
>>>> + # $HOME can end with a trailing /, so strip it (see #17809)
>>>> +- self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
>>>> ++ # The Guix builders have '/' as a home directory, so
>>>> ++ # home.rstrip("/") will be an empty string and the test will
>>>> ++ # fail. Let's just disable it since it does not really make
>>>> ++ # sense with such a bizarre setup.
>>>> ++ # self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
>>>
>>> I see that the recipe’s ‘pre-check’ phase does:
>>>
>>> (setenv "HOME" (getcwd))
>>>
>>> so $HOME should actually be /tmp/nix-build-xxx, not just /, no?
>>>
>>
>> Yes, but I think it uses getpwent() or something that does not use
>> $HOME to retrieve the home directory in this case.
>
> Here’s what I see, with nscd turned off (this uses getpwnam(2)):
>
> --8<---------------cut here---------------start------------->8---
> $ strace guile -c '(getpw "ludo")'
> [...]
> open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 5
> fstat(5, {st_mode=S_IFREG|0644, st_size=2072, ...}) = 0
> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f264dea5000
> read(5, "root:x:0:0:System administrator:"..., 4096) = 2072
> --8<---------------cut here---------------end--------------->8---
>
> Normally the chroot environment has all that too. And indeed, the
> equivalent of the above Python snippet seems to work:
>
> --8<---------------cut here---------------start------------->8---
> scheme@(guile-user)> ,use(guix)
> scheme@(guile-user)> ,use(guix monads)
> scheme@(guile-user)> (define s (open-connection))
> scheme@(guile-user)> (derivation-expression "getpw" '(pk (getpw (getuid))))
> $11 = #<procedure 3da30f0 at guix/monads.scm:297:6 (store)>
> scheme@(guile-user)> (run-with-store s $11)
> $12 = #<derivation /gnu/store/bhmd5bg9s64an3s8ssmwx6l90dkr56hx-getpw.drv => /gnu/store/z68rzri3myxx1dg931fb1ksxdg7c29zg-getpw 33dfa00>
> scheme@(guile-user)> (build-derivations s (list $12))
> building path(s) `/gnu/store/z68rzri3myxx1dg931fb1ksxdg7c29zg-getpw'
>
> ;;; (#("nixbld" "x" 30001 30000 "Nix build user" "/" "/noshell"))
Am I misteading this, or is the home directory '/' for the nixbld user ?
That's exactly what happens in Python (that uses getwpent). Hence the
bug. I think skipping this assertion is the right thing to do here.
Cyril.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH 3/5] gnu: Enable tests in Python 3.
2014-03-22 23:13 ` Cyril Roelandt
@ 2014-03-23 10:43 ` Ludovic Courtès
0 siblings, 0 replies; 21+ messages in thread
From: Ludovic Courtès @ 2014-03-23 10:43 UTC (permalink / raw)
To: Cyril Roelandt; +Cc: guix-devel
Cyril Roelandt <tipecaml@gmail.com> skribis:
> On 03/21/2014 02:15 PM, Ludovic Courtès wrote:
>> Cyril Roelandt <tipecaml@gmail.com> skribis:
>>
>>> On 03/07/2014 10:40 AM, Ludovic Courtès wrote:
>>>> Cyril Roelandt <tipecaml@gmail.com> skribis:
>>
>> [...]
>>
>>>>> +--- Lib/test/test_posixpath.py 2014-03-01 05:46:56.984311000 +0100
>>>>> ++++ Lib/test/test_posixpath.py 2014-03-01 06:20:50.704311000 +0100
>>>>> +@@ -319,7 +319,11 @@
>>>>> + del env['HOME']
>>>>> + home = pwd.getpwuid(os.getuid()).pw_dir
>>>>> + # $HOME can end with a trailing /, so strip it (see #17809)
>>>>> +- self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
>>>>> ++ # The Guix builders have '/' as a home directory, so
>>>>> ++ # home.rstrip("/") will be an empty string and the test will
>>>>> ++ # fail. Let's just disable it since it does not really make
>>>>> ++ # sense with such a bizarre setup.
>>>>> ++ # self.assertEqual(posixpath.expanduser("~"), home.rstrip("/"))
>>>>
>>>> I see that the recipe’s ‘pre-check’ phase does:
>>>>
>>>> (setenv "HOME" (getcwd))
>>>>
>>>> so $HOME should actually be /tmp/nix-build-xxx, not just /, no?
>>>>
>>>
>>> Yes, but I think it uses getpwent() or something that does not use
>>> $HOME to retrieve the home directory in this case.
>>
>> Here’s what I see, with nscd turned off (this uses getpwnam(2)):
>>
>> --8<---------------cut here---------------start------------->8---
>> $ strace guile -c '(getpw "ludo")'
>> [...]
>> open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 5
>> fstat(5, {st_mode=S_IFREG|0644, st_size=2072, ...}) = 0
>> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f264dea5000
>> read(5, "root:x:0:0:System administrator:"..., 4096) = 2072
>> --8<---------------cut here---------------end--------------->8---
>>
>> Normally the chroot environment has all that too. And indeed, the
>> equivalent of the above Python snippet seems to work:
>>
>> --8<---------------cut here---------------start------------->8---
>> scheme@(guile-user)> ,use(guix)
>> scheme@(guile-user)> ,use(guix monads)
>> scheme@(guile-user)> (define s (open-connection))
>> scheme@(guile-user)> (derivation-expression "getpw" '(pk (getpw (getuid))))
>> $11 = #<procedure 3da30f0 at guix/monads.scm:297:6 (store)>
>> scheme@(guile-user)> (run-with-store s $11)
>> $12 = #<derivation /gnu/store/bhmd5bg9s64an3s8ssmwx6l90dkr56hx-getpw.drv => /gnu/store/z68rzri3myxx1dg931fb1ksxdg7c29zg-getpw 33dfa00>
>> scheme@(guile-user)> (build-derivations s (list $12))
>> building path(s) `/gnu/store/z68rzri3myxx1dg931fb1ksxdg7c29zg-getpw'
>>
>> ;;; (#("nixbld" "x" 30001 30000 "Nix build user" "/" "/noshell"))
>
> Am I misteading this, or is the home directory '/' for the nixbld user
> ?
Correct, as per build.cc:
--8<---------------cut here---------------start------------->8---
/* Create a /etc/passwd with entries for the build user and the
nobody account. The latter is kind of a hack to support
Samba-in-QEMU. */
createDirs(chrootRootDir + "/etc");
writeFile(chrootRootDir + "/etc/passwd",
(format(
"nixbld:x:%1%:%2%:Nix build user:/:/noshell\n"
"nobody:x:65534:65534:Nobody:/:/noshell\n")
% (buildUser.enabled() ? buildUser.getUID() : getuid())
% (buildUser.enabled() ? buildUser.getGID() : getgid())).str());
--8<---------------cut here---------------end--------------->8---
> That's exactly what happens in Python (that uses getwpent). Hence the
> bug. I think skipping this assertion is the right thing to do here.
Oh right. (Somehow I ended up thinking that getpwent was not working or
something like that; sorry for the misunderstanding!)
So OK to push as is!
Thanks,
Ludo’.
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2014-03-23 10:43 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-03-07 2:47 [PATCH 0/5] Python improvements Cyril Roelandt
2014-03-07 2:47 ` [PATCH 1/5] gnu: Enable the 'ctypes' module in Python Cyril Roelandt
2014-03-07 9:28 ` Ludovic Courtès
2014-03-07 2:47 ` [PATCH 2/5] gnu: Python: use /nix/.../sh if /bin/sh cannot be found Cyril Roelandt
2014-03-07 9:31 ` Ludovic Courtès
2014-03-22 3:36 ` [PATCH 2/4 v2] gnu: Python: use /nix/.../sh instead of /bin/sh in the subprocess module Cyril Roelandt
2014-03-22 14:26 ` Ludovic Courtès
2014-03-07 2:47 ` [PATCH 3/5] gnu: Enable tests in Python 3 Cyril Roelandt
2014-03-07 9:40 ` Ludovic Courtès
2014-03-21 4:12 ` [PATCH 3/4 v2] " Cyril Roelandt
2014-03-22 14:32 ` Ludovic Courtès
2014-03-21 4:14 ` [PATCH 3/5] " Cyril Roelandt
2014-03-21 13:15 ` Ludovic Courtès
2014-03-22 23:13 ` Cyril Roelandt
2014-03-23 10:43 ` Ludovic Courtès
2014-03-07 2:47 ` [PATCH 4/5] gnu: Python: bump to 3.3.4 Cyril Roelandt
2014-03-07 9:31 ` Ludovic Courtès
2014-03-21 4:07 ` Cyril Roelandt
2014-03-21 13:02 ` Ludovic Courtès
2014-03-07 2:47 ` [PATCH 5/5] gnu: remove python-fix-dbm.patch Cyril Roelandt
2014-03-07 9:32 ` Ludovic Courtès
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).