tk-update update

tk-update update

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 2223 bytes --]

Hello!

As you may have noticed, the ‘tk-update’ branch is not merged yet.

There’s one problem, manifesting itself in webkitgtk-2.4
(http://hydra.gnu.org/build/831334):

--8<---------------cut here---------------start------------->8---
  GEN      DerivedSources/WebCore/CommandLineAPIModuleSource.h
Traceback (most recent call last):
  File "/gnu/store/7lqv9lk5j0kg6yrhwqh5cbwqbibzmxgl-glib-2.46.1-bin/bin/gdbus-codegen", line 37, in <module>
    from codegen import codegen_main
  File "/gnu/store/fvg98wicda2ia6wxx90gfbs8ydy27qdi-glib-2.46.1/share/glib-2.0/codegen/codegen_main.py", line 28, in <module>
    from . import parser
  File "/gnu/store/fvg98wicda2ia6wxx90gfbs8ydy27qdi-glib-2.46.1/share/glib-2.0/codegen/parser.py", line 23, in <module>
    import xml.parsers.expat
  File "/gnu/store/5v7xrwnzz0d1h9qfwwq5aaj33a6d7xm1-python-3.4.3/lib/python3.4/xml/parsers/expat.py", line 4, in <module>
    from pyexpat import *
ImportError: dynamic module does not define init function (PyInit_pyexpat)
--8<---------------cut here---------------end--------------->8---

The problem can be reproduced this way:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix environment --ad-hoc python-2 python-3 -- python3
Python 3.4.3 (default, Jan  1 1970, 00:00:01) 
[GCC 4.9.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.environ['PYTHONPATH']
'/gnu/store/q405y94pbnx028mygranaww5pnbqhnq1-python-2.7.10/lib/python2.7/site-packages:/gnu/store/q405y94pbnx028mygranaww5pnbqhnq1-python-2.7.10/lib/python2.7/lib-dynload:/gnu/store/5v7xrwnzz0d1h9qfwwq5aaj33a6d7xm1-python-3.4.3/lib/python3.4/site-packages:/gnu/store/5v7xrwnzz0d1h9qfwwq5aaj33a6d7xm1-python-3.4.3/lib/python3.4/lib-dynload:/home/ludo/.guix-profile/lib/python2.7/site-packages:/run/current-system/profile/lib/python2.7/site-packages'
>>> import pyexpat
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dynamic module does not define init function (PyInit_pyexpat)
--8<---------------cut here---------------end--------------->8---

What happens is that as a consequence of d7572b4, which does:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 410 bytes --]

@@ -215,7 +233,8 @@
     (native-search-paths
      (list (search-path-specification
             (variable "PYTHONPATH")
-            (files '("lib/python2.7/site-packages")))))
+            (files '("lib/python2.7/site-packages"
+                     "lib/python2.7/lib-dynload")))))
     (home-page "http://python.org")
     (synopsis "High-level, dynamically-typed programming language")
     (description

[-- Attachment #3: Type: text/plain, Size: 676 bytes --]


… we end up with python2’s pyexpat.so in the search path, and python3
fails when it tries to load it.  (This change was done so that .so files
such as tkinter.so can be found even when they don’t lie under Python’s
prefix.)

My analysis is that it’s “reasonable” to expect breakage when mixing
python2 and python3 in the same environment, and that the problem here
is more that standalone programs like ‘gdbus-codegen’ should be wrapped
to have their PYTHONPATH set in stone.

What do people think?

If there’s consensus, we need to fix GLib in that branch, which means
rebuilding a lot of things again (604 packages.)

Thanks,
Ludo’.

Re: tk-update update

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> … we end up with python2’s pyexpat.so in the search path, and python3
> fails when it tries to load it.  (This change was done so that .so files
> such as tkinter.so can be found even when they don’t lie under Python’s
> prefix.)
>
> My analysis is that it’s “reasonable” to expect breakage when mixing
> python2 and python3 in the same environment, and that the problem here
> is more that standalone programs like ‘gdbus-codegen’ should be wrapped
> to have their PYTHONPATH set in stone.

This is the correct approach in my opinion.  Standalone Python
programmes should really be wrapped so that all required paths are
defined.

~~ Ricardo

Re: tk-update update

[Ludovic Courtès]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> … we end up with python2’s pyexpat.so in the search path, and python3
>> fails when it tries to load it.  (This change was done so that .so files
>> such as tkinter.so can be found even when they don’t lie under Python’s
>> prefix.)
>>
>> My analysis is that it’s “reasonable” to expect breakage when mixing
>> python2 and python3 in the same environment, and that the problem here
>> is more that standalone programs like ‘gdbus-codegen’ should be wrapped
>> to have their PYTHONPATH set in stone.
>
> This is the correct approach in my opinion.  Standalone Python
> programmes should really be wrapped so that all required paths are
> defined.

Well, ‘gdbus-codegen’ normally has no problem finding its modules,
because it uses only modules provided by Python itself.

The problem here is that setting PYTHONPATH interferes with that.

So if we wrap ‘gdbus-codegen’, we do it such that the wrapper unsets
or clears PYTHONPATH altogether.

Ludo’.

Re: tk-update update

[Ludovic Courtès]

ludo@gnu.org (Ludovic Courtès) skribis:

> What happens is that as a consequence of d7572b4, which does:
>
> @@ -215,7 +233,8 @@
>      (native-search-paths
>       (list (search-path-specification
>              (variable "PYTHONPATH")
> -            (files '("lib/python2.7/site-packages")))))
> +            (files '("lib/python2.7/site-packages"
> +                     "lib/python2.7/lib-dynload")))))
>      (home-page "http://python.org")
>      (synopsis "High-level, dynamically-typed programming language")
>      (description
>
> … we end up with python2’s pyexpat.so in the search path, and python3
> fails when it tries to load it.  (This change was done so that .so files
> such as tkinter.so can be found even when they don’t lie under Python’s
> prefix.)

I realized it’s safer to avoid adding lib-dynload to the search path and
simply install tkinter under lib/python2.7/site-packages.

I’ve done that in tk-update, rebased on top of master.

Ludo’.

Re: tk-update update

[Ludovic Courtès]

I’ve finally merged the branch.

There are a few new failures:

  http://hydra.gnu.org/eval/108145?compare=master

In particular the python2-pathpy and python2-statsmodels failure might
be related.  Kudos to anyone who fixes those.  :-)

Ludo’.