bug#16569: Python package doesn't install ctypes module

bug#16569: Python package doesn't install ctypes module

[Sree Harsha Totakura]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Python (both versions 2.7.6 and 3.3) installed through Guix do not
contain/install ctypes module.
> $ guix package --list-installed ;;; note: source file
> /usr/local/share/guile/site/2.0/srfi/srfi-37.scm ;;;       newer
> than compiled /usr/local/share/guile/site/2.0/srfi/srfi-37.go hello
> 2.9	out	/nix/store/jgvkvznlsvwax0hc29f5br508ivphks1-hello-2.9 
> gnutls	3.2.4	out
> /nix/store/r64154hy14lzdxgp8gy8kk99kh4mw7hf-gnutls-3.2.4 libgcrypt
> 1.6.0	out
> /nix/store/5ad1sv3fr8bjh1db9jwf0z130gbb584a-libgcrypt-1.6.0 ncdc
> 1.18.1	out	/nix/store/95i40hzr1scgszdsqa9xc38mighczx5g-ncdc-1.18.1 
> $ guix package -i python ;;; note: source file
> /usr/local/share/guile/site/2.0/srfi/srfi-37.scm ;;;       newer
> than compiled /usr/local/share/guile/site/2.0/srfi/srfi-37.go The
> following package will be installed: python-3.3.3	out
> /nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3
> 
> substitute-binary: ;;; note: source file
> /usr/local/share/guile/site/2.0/srfi/srfi-37.scm substitute-binary:
> ;;;       newer than compiled
> /usr/local/share/guile/site/2.0/srfi/srfi-37.go 5 packages in
> profile The following environment variable definitions may be
> needed: export
> PYTHONPATH="/home/totakura/.guix-profile/lib/python3.3/site-packages"
>
> 
$ export
PYTHONPATH="/home/totakura/.guix-profile/lib/python3.3/site-packages"
> $ which python3.3 /home/totakura/.guix-profile/bin/python3.3 $
> python3.3 Python 3.3.3 (default, Dec 30 2013, 01:48:16) [GCC 4.8.2]
> on linux Type "help", "copyright", "credits" or "license" for more
> information.
>>>> import ctypes
> Traceback (most recent call last): File "<stdin>", line 1, in
> <module> File
> "/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/ctypes/__init__.py",
> line 7, in <module> from _ctypes import Union, Structure, Array 
> ImportError: No module named '_ctypes'
>>>> 
> $ cd /nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3 $
> pwd /nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3 $ find
> ./ -name "*ctypes*" 
> ./lib/python3.3/test/__pycache__/test_ctypes.cpython-33.pyo 
> ./lib/python3.3/test/__pycache__/test_ctypes.cpython-33.pyc 
> ./lib/python3.3/test/test_ctypes.py ./lib/python3.3/ctypes 
> ./lib/python3.3/ctypes/macholib/README.ctypes 
> ./lib/python3.3/lib-dynload/_ctypes_test.cpython-33m.so 
> ./lib/python3.3/multiprocessing/__pycache__/sharedctypes.cpython-33.pyc
>
> 
./lib/python3.3/multiprocessing/__pycache__/sharedctypes.cpython-33.pyo
> ./lib/python3.3/multiprocessing/sharedctypes.py $

On my debian system, python was able to find ctypes module after
searching through these paths:
> stat("/usr/lib/python2.7/ctypes", {st_mode=S_IFDIR|0755,
> st_size=4096, ...}) = 0 stat("/usr/lib/python2.7/ctypes",
> {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 
> stat("/usr/lib/python2.7/ctypes/os", 0x7fff3be9e430) = -1 ENOENT
> (No such file or directory) open("/usr/lib/python2.7/ctypes/os.so",
> O_RDONLY) = -1 ENOENT (No such file or directory) 
> open("/usr/lib/python2.7/ctypes/osmodule.so", O_RDONLY) = -1 ENOENT
> (No such file or directory) open("/usr/lib/python2.7/ctypes/os.py",
> O_RDONLY) = -1 ENOENT (No such file or directory) 
> open("/usr/lib/python2.7/ctypes/os.pyc", O_RDONLY) = -1 ENOENT (No
> such file or directory) stat("/usr/lib/python2.7/ctypes/sys",
> 0x7fff3be9e430) = -1 ENOENT (No such file or directory) 
> open("/usr/lib/python2.7/ctypes/sys.so", O_RDONLY) = -1 ENOENT (No
> such file or directory) 
> open("/usr/lib/python2.7/ctypes/sysmodule.so", O_RDONLY) = -1
> ENOENT (No such file or directory) 
> open("/usr/lib/python2.7/ctypes/sys.py", O_RDONLY) = -1 ENOENT (No
> such file or directory) open("/usr/lib/python2.7/ctypes/sys.pyc",
> O_RDONLY) = -1 ENOENT (No such file or directory) 
> stat("/usr/lib/python2.7/ctypes/_ctypes", 0x7fff3be9e430) = -1
> ENOENT (No such file or directory) 
> open("/usr/lib/python2.7/ctypes/_ctypes.so", O_RDONLY) = -1 ENOENT
> (No such file or directory) 
> open("/usr/lib/python2.7/ctypes/_ctypesmodule.so", O_RDONLY) = -1
> ENOENT (No such file or directory) 
> open("/usr/lib/python2.7/ctypes/_ctypes.py", O_RDONLY) = -1 ENOENT
> (No such file or directory) 
> open("/usr/lib/python2.7/ctypes/_ctypes.pyc", O_RDONLY) = -1 ENOENT
> (No such file or directory) stat("_ctypes", 0x7fff3be9e430)
> = -1 ENOENT (No such file or directory) open("_ctypes.so",
> O_RDONLY)            = -1 ENOENT (No such file or directory) 
> open("_ctypesmodule.so", O_RDONLY)      = -1 ENOENT (No such file
> or directory) open("_ctypes.py", O_RDONLY)            = -1 ENOENT
> (No such file or directory) open("_ctypes.pyc", O_RDONLY)
> = -1 ENOENT (No such file or directory) 
> stat("/usr/lib/python2.7/_ctypes", 0x7fff3be9e430) = -1 ENOENT (No
> such file or directory) open("/usr/lib/python2.7/_ctypes.so",
> O_RDONLY) = -1 ENOENT (No such file or directory) 
> open("/usr/lib/python2.7/_ctypesmodule.so", O_RDONLY) = -1 ENOENT
> (No such file or directory) open("/usr/lib/python2.7/_ctypes.py",
> O_RDONLY) = -1 ENOENT (No such file or directory) 
> open("/usr/lib/python2.7/_ctypes.pyc", O_RDONLY) = -1 ENOENT (No
> such file or directory) 
> stat("/usr/lib/python2.7/plat-linux2/_ctypes", 0x7fff3be9e430) = -1
> ENOENT (No such file or directory) 
> open("/usr/lib/python2.7/plat-linux2/_ctypes.so", O_RDONLY) = -1
> ENOENT (No such file or directory) 
> open("/usr/lib/python2.7/plat-linux2/_ctypesmodule.so", O_RDONLY) =
> -1 ENOENT (No such file or directory) 
> open("/usr/lib/python2.7/plat-linux2/_ctypes.py", O_RDONLY) = -1
> ENOENT (No such file or directory) 
> open("/usr/lib/python2.7/plat-linux2/_ctypes.pyc", O_RDONLY) = -1
> ENOENT (No such file or directory) 
> stat("/usr/lib/python2.7/lib-tk/_ctypes", 0x7fff3be9e430) = -1
> ENOENT (No such file or directory) 
> open("/usr/lib/python2.7/lib-tk/_ctypes.so", O_RDONLY) = -1 ENOENT
> (No such file or directory) 
> open("/usr/lib/python2.7/lib-tk/_ctypesmodule.so", O_RDONLY) = -1
> ENOENT (No such file or directory) 
> open("/usr/lib/python2.7/lib-tk/_ctypes.py", O_RDONLY) = -1 ENOENT
> (No such file or directory) 
> open("/usr/lib/python2.7/lib-tk/_ctypes.pyc", O_RDONLY) = -1 ENOENT
> (No such file or directory) 
> stat("/usr/lib/python2.7/lib-dynload/_ctypes", 0x7fff3be9e430) = -1
> ENOENT (No such file or directory) 
> open("/usr/lib/python2.7/lib-dynload/_ctypes.so", O_RDONLY) = 4

With the Guix installed python I observe this through strace:
> rt_sigprocmask(SIG_BLOCK, [INT], [], 8) = 0 ioctl(0,
> SNDCTL_TMR_STOP or TCSETSW, {B38400 opost isig icanon echo ...}) =
> 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 
> rt_sigaction(SIGWINCH, {SIG_DFL, [], SA_RESTORER, 0x7fb581e85870},
> {0x7fb57a520760, [], SA_RESTORER|SA_RESTART, 0x7fb581e85870}, 8) =
> 0 stat(".", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0 
> stat("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3",
> {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0 
> stat("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/ctypes",
> {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0 
> stat("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/ctypes/__init__.cpython-33m.so",
> 0x7fff05ac1650) = -1 ENOENT (No such file or directory) 
> stat("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/ctypes/__init__.abi3.so",
> 0x7fff05ac1650) = -1 ENOENT (No such file or directory) 
> stat("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/ctypes/__init__.so",
> 0x7fff05ac1650) = -1 ENOENT (No such file or directory) 
> stat("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/ctypes/__init__.py",
> {st_mode=S_IFREG|0444, st_size=17043, ...}) = 0 
> stat("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/ctypes/__init__.py",
> {st_mode=S_IFREG|0444, st_size=17043, ...}) = 0 
> open("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/ctypes/__pycache__/__init__.cpython-33.pyc",
> O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0444, st_size=29315, ...})
> = 0 lseek(3, 0, SEEK_CUR)                   = 0 fstat(3,
> {st_mode=S_IFREG|0444, st_size=29315, ...}) = 0 read(3,
> "\236\f\r\nh\321\300R\223B\0\0c\0\0\0\0\0\0\0\0\0\0\0\0\r\0\0\0@\0\0"...,
> 29316) = 29315 read(3, "", 1)                          = 0 close(3)
> = 0 
> open("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/ctypes/__init__.py",
> O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0444, st_size=17043, ...})
> = 0 lseek(3, 0, SEEK_CUR)                   = 0 fstat(3,
> {st_mode=S_IFREG|0444, st_size=17043, ...}) = 0 read(3,
> "\"\"\"create and manipulate C data "..., 17044) = 17043 read(3,
> "", 1)                          = 0 close(3)
> = 0 
> stat("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/ctypes/__init__.py",
> {st_mode=S_IFREG|0444, st_size=17043, ...}) = 0 
> stat("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/ctypes/__pycache__",
> {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0 
> open("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/ctypes/__pycache__/__init__.cpython-33.pyc.140417554480296",
> O_WRONLY|O_CREAT|O_EXCL, 0644) = -1 EACCES (Permission denied) 
> stat(".", {st_mode=S_IFDIR|0750, st_size=4096, ...}) = 0 
> stat("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3",
> {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0 
> stat("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/plat-linux",
> {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0 
> stat("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/lib-dynload",
> {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0 
> stat("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/site-packages",
> {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0 write(2, "Traceback
> (most recent call last"..., 35Traceback (most recent call last): )
> = 35 write(2, "  File \"<stdin>\", line 1, in <mo"..., 38  File
> "<stdin>", line 1, in <module> ) = 38 open("<stdin>", O_RDONLY)
> = -1 ENOENT (No such file or directory) open("<stdin>", O_RDONLY)
> = -1 ENOENT (No such file or directory) 
> open("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python33.zip/<stdin>",
> O_RDONLY) = -1 ENOENT (No such file or directory) 
> open("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/<stdin>",
> O_RDONLY) = -1 ENOENT (No such file or directory) 
> open("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/plat-linux/<stdin>",
> O_RDONLY) = -1 ENOENT (No such file or directory) 
> open("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/lib-dynload/<stdin>",
> O_RDONLY) = -1 ENOENT (No such file or directory) 
> open("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/site-packages/<stdin>",
> O_RDONLY) = -1 ENOENT (No such file or directory) write(2, "  File
> \"/nix/store/b50aj7l8qhaxy"..., 120  File
> "/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/ctypes/__init__.py",
> line 7, in <module> ) = 120 
> open("/nix/store/b50aj7l8qhaxys65z57wcqhkyi22ni2i-python-3.3.3/lib/python3.3/ctypes/__init__.py",
> O_RDONLY) = 3 fstat(3, {st_mode=S_IFREG|0444, st_size=17043, ...})
> = 0 ioctl(3, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff05ac1220) = -1
> ENOTTY (Inappropriate ioctl for device) fstat(3,
> {st_mode=S_IFREG|0444, st_size=17043, ...}) = 0 lseek(3, 0,
> SEEK_CUR)                   = 0 dup(3)
> = 4 fcntl(4, F_GETFL)                       = 0x8000 (flags
> O_RDONLY|O_LARGEFILE) fstat(4, {st_mode=S_IFREG|0444,
> st_size=17043, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb58257d000 lseek(4, 0,
> SEEK_CUR)                   = 0 read(4, "\"\"\"create and
> manipulate C data "..., 4096) = 4096 close(4)
> = 0 munmap(0x7fb58257d000, 4096)            = 0 lseek(3, 0,
> SEEK_SET)                   = 0 lseek(3, 0, SEEK_CUR)
> = 0 read(3, "\"\"\"create and manipulate C data "..., 8192) = 8192 
> close(3)                                = 0 write(2, "    from
> _ctypes import Union, S"..., 48    from _ctypes import Union,
> Structure, Array ) = 48 write(2, "ImportError: No module named
> '_c"..., 39ImportError: No module named '_ctypes' ) = 39

Sree
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Icedove - http://www.enigmail.net/

iEYEARECAAYFAlLmRNkACgkQO2+K8UPCHztJRgCeIQzRwGkGRAYGq3xENXDDhdac
rtcAn2HjBpRCg535Mhy/ry9uIsA2JsNm
=UhMJ
-----END PGP SIGNATURE-----

bug#16569: Python package doesn't install ctypes module

[Ludovic Courtès]

Sree Harsha Totakura <sreeharsha@totakura.in> skribis:

> Python (both versions 2.7.6 and 3.3) installed through Guix do not
> contain/install ctypes module.

Is this fixed with the libffi change you proposed (commit
d43547f12d4eae19845f3bf13a53a050af965063)?

Ludo’.

bug#16569: Python package doesn't install ctypes module

[Sree Harsha Totakura]

On 01/27/2014 07:06 PM, Ludovic Courtès wrote:
> Is this fixed with the libffi change you proposed (commit
> d43547f12d4eae19845f3bf13a53a050af965063)?

I haven't tested it, but I don't think the libffi change I proposed
would fix this, because python source comes with libffi.

What I could not confirm is whether the ctypes module is getting built
or not.  Maybe knowing that can be helpful.

Sree

bug#16569: Python package doesn't install ctypes module

[Cyril Roelandt]

On 01/27/2014 08:17 PM, Sree Harsha Totakura wrote:
> On 01/27/2014 07:06 PM, Ludovic Courtès wrote:
>> Is this fixed with the libffi change you proposed (commit
>> d43547f12d4eae19845f3bf13a53a050af965063)?
>
> I haven't tested it, but I don't think the libffi change I proposed
> would fix this, because python source comes with libffi.
>

Actually, it helps! Python only uses its own version of libffi if no 
libffi can be found on the system. This is what it tries to do in Guix, 
but it fails because it uses commands that ultimately use /bin/sh, which 
is not available in the chroot.


I'll send a patch that forces Python to use the system libffi. It's 
against core-updates, since your patch that symlinks stuff in libffi is 
needed; however, I've only tested it against master after manually 
'backporting' it. Could you tell us whether it fixes your issue ?

Regards,
Cyril.

bug#16569: [PATCH] gnu: Enable the 'ctypes' module in Python.

[Cyril Roelandt]

* gnu/packages/python.scm (python-2): add libffi to the inputs and use it to
  build the ctypes module.
---
 gnu/packages/python.scm | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 6d2c940..260bace 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -27,9 +27,11 @@
   #:use-module (gnu packages compression)
   #:use-module (gnu packages gdbm)
   #:use-module (gnu packages icu4c)
+  #:use-module (gnu packages libffi)
   #:use-module (gnu packages readline)
   #:use-module (gnu packages openssl)
   #:use-module (gnu packages patchelf)
+  #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages sqlite)
   #:use-module (guix packages)
   #:use-module (guix download)
@@ -98,10 +100,12 @@
        #:configure-flags
         (let ((bz2 (assoc-ref %build-inputs "bzip2"))
               (gdbm (assoc-ref %build-inputs "gdbm"))
+              (libffi (assoc-ref %build-inputs "libffi"))
               (openssl (assoc-ref %build-inputs "openssl"))
               (readline (assoc-ref %build-inputs "readline"))
               (zlib (assoc-ref %build-inputs "zlib")))
          (list "--enable-shared"                  ; allow embedding
+               "--with-system-ffi"                ; build ctypes
                (string-append "CPPFLAGS="
                 "-I" bz2 "/include "
                 "-I" gdbm "/include "
@@ -111,6 +115,7 @@
                (string-append "LDFLAGS="
                 "-L" bz2 "/lib "
                 "-L" gdbm "/lib "
+                "-L" libffi "/lib "
                 "-L" openssl "/lib "
                 "-L" readline "/lib "
                 "-L" zlib "/lib")))
@@ -137,7 +142,9 @@
     (inputs
      `(("bzip2" ,bzip2)
        ("gdbm" ,gdbm)
+       ("libffi" ,libffi)                         ; for ctypes
        ("openssl" ,openssl)
+       ("pkg-config" ,pkg-config)
        ("readline" ,readline)
        ("zlib" ,zlib)
        ("patchelf" ,patchelf)))                   ; for (guix build rpath)
-- 
1.8.4.rc3

bug#16569: Python package doesn't install ctypes module

[Ludovic Courtès]

Cyril Roelandt <tipecaml@gmail.com> skribis:

> On 01/27/2014 08:17 PM, Sree Harsha Totakura wrote:
>> On 01/27/2014 07:06 PM, Ludovic Courtès wrote:
>>> Is this fixed with the libffi change you proposed (commit
>>> d43547f12d4eae19845f3bf13a53a050af965063)?
>>
>> I haven't tested it, but I don't think the libffi change I proposed
>> would fix this, because python source comes with libffi.
>>
>
> Actually, it helps! Python only uses its own version of libffi if no
> libffi can be found on the system.

So the change makes it use the system’s libffi, but it doesn’t make it
build ctype, does it?

> I'll send a patch that forces Python to use the system libffi.

But if the libffi change above solves this problem, what else is needed?

Thanks,
Ludo’.

bug#16569: [PATCH] gnu: Enable the 'ctypes' module in Python.

[Ludovic Courtès]

Cyril Roelandt <tipecaml@gmail.com> skribis:

> * gnu/packages/python.scm (python-2): add libffi to the inputs and use it to
>   build the ctypes module.

Oh I see now, nice!

>      (inputs
>       `(("bzip2" ,bzip2)
>         ("gdbm" ,gdbm)
> +       ("libffi" ,libffi)                         ; for ctypes
>         ("openssl" ,openssl)
> +       ("pkg-config" ,pkg-config)

‘pkg-config’ should be in ‘native-inputs’.

OK to commit with this change!

Thanks,
Ludo’.

bug#16569: [PATCH] gnu: Enable the 'ctypes' module in Python.

[Sree Harsha Totakura]

Hi!

This patch solved this bug.  Thanks!

Sree

On 02/01/2014 01:46 AM, Cyril Roelandt wrote:
> * gnu/packages/python.scm (python-2): add libffi to the inputs and use it to
>   build the ctypes module.

bug#16569: [PATCH] gnu: Enable the 'ctypes' module in Python.

[Ludovic Courtès]

Fixed by b10ab72, now in ‘master’.  Thanks to both of you!

Ludo’.