bug#20889: Tkinter moved to separate output

bug#20889: Tkinter moved to separate output

[Ludovic Courtès]

In ‘tk-update’ the ‘python’ packages now have a separate ‘tk’ output for
Tkinter (adding ~50 MiB to the closure of Python.)  I’ve tested it along
these lines, for 2.x and 3.x:

  guix environment --ad-hoc python python:tk --pure -- python
  > import Tkinter
  > root = Tkinter.Tk()
  
One thing I haven’t checked is whether some packages need to have an
additional python:tk input.  Matplotlib maybe?  Federico?

I plan to have Hydra build that branch once we’ve checked that.

Thanks,
Ludo’.

bug#20889: Tkinter moved to separate output

[Federico Beffa]

On Mon, Nov 16, 2015 at 9:42 AM, Ludovic Courtès <ludo@gnu.org> wrote:
> In ‘tk-update’ the ‘python’ packages now have a separate ‘tk’ output for
> Tkinter (adding ~50 MiB to the closure of Python.)  I’ve tested it along
> these lines, for 2.x and 3.x:
>
>   guix environment --ad-hoc python python:tk --pure -- python
>   > import Tkinter
>   > root = Tkinter.Tk()
>
> One thing I haven’t checked is whether some packages need to have an
> additional python:tk input.  Matplotlib maybe?  Federico?

Yes, given that the back-end currently used by matploblib doesn't
work, I would change that by using 'Tkinter'.

Since you are at it, I would also change the default back-end by
changing 'GTK3Agg' to 'TkAgg' in the following snippet:

(lambda (port)
  (format port "[rc_options]~%
backend = GTK3Agg~%")))))

I'm not convinced that having 2 versions of packages (a "normal" and a
"minimal" one) is a good idea. The reason is that you often end up
downloading both instead of just one. As one example, I just
downloaded 'vlc' and I noticed that it downloaded both 'cups' and
'cups-minimal'. Similarly, for python, anyone needing 'Tkinter' will
end up with 2 versions in the store.

Regards,
Fede

bug#20889: Tkinter moved to separate output

[Ludovic Courtès]

Federico Beffa <beffa@ieee.org> skribis:

> On Mon, Nov 16, 2015 at 9:42 AM, Ludovic Courtès <ludo@gnu.org> wrote:
>> In ‘tk-update’ the ‘python’ packages now have a separate ‘tk’ output for
>> Tkinter (adding ~50 MiB to the closure of Python.)  I’ve tested it along
>> these lines, for 2.x and 3.x:
>>
>>   guix environment --ad-hoc python python:tk --pure -- python
>>   > import Tkinter
>>   > root = Tkinter.Tk()
>>
>> One thing I haven’t checked is whether some packages need to have an
>> additional python:tk input.  Matplotlib maybe?  Federico?
>
> Yes, given that the back-end currently used by matploblib doesn't
> work, I would change that by using 'Tkinter'.
>
> Since you are at it, I would also change the default back-end by
> changing 'GTK3Agg' to 'TkAgg' in the following snippet:
>
> (lambda (port)
>   (format port "[rc_options]~%
> backend = GTK3Agg~%")))))

OTOH I see that Matploblib already depends on GTK+ 3.x and
python-pygobject, so maybe it’s best to stick to GTK+ by default?

> I'm not convinced that having 2 versions of packages (a "normal" and a
> "minimal" one) is a good idea. The reason is that you often end up
> downloading both instead of just one. As one example, I just
> downloaded 'vlc' and I noticed that it downloaded both 'cups' and
> 'cups-minimal'. Similarly, for python, anyone needing 'Tkinter' will
> end up with 2 versions in the store.

The ‘minimal’ variants are there to break cyclic dependencies most of
the time.  This is the case for cups/cups-filters and Python/Tk.

At run time, cups indeed depends on cups-filters, which depends on
cups-minimal.  In that case, as you say, we end up downloading those two
variants.  But there’s not much of a choice, as was discussed during the
initial review:

  https://lists.gnu.org/archive/html/guix-devel/2015-01/msg00057.html

Conversely, python does not depend on python-minimal.  So python-minimal
is here just to break the cycle with Tk/libxcb, as noted in python.scm
and discussed in:

  http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20889

As for Tkinter, it is in a separate “tk” output, which users can choose
to install or not.  The “tk” output contains a single file, but it pulls
in Tcl/Tk.

I hope this clarifies things.

Thanks for your feedback,
Ludo’.

bug#20889: Tkinter moved to separate output

[Federico Beffa]

On Mon, Nov 16, 2015 at 1:57 PM, Ludovic Courtès <ludo@gnu.org> wrote:
> Federico Beffa <beffa@ieee.org> skribis:
>> Yes, given that the back-end currently used by matploblib doesn't
>> work, I would change that by using 'Tkinter'.
>>
>> Since you are at it, I would also change the default back-end by
>> changing 'GTK3Agg' to 'TkAgg' in the following snippet:
>>
>> (lambda (port)
>>   (format port "[rc_options]~%
>> backend = GTK3Agg~%")))))
>
> OTOH I see that Matploblib already depends on GTK+ 3.x and
> python-pygobject, so maybe it’s best to stick to GTK+ by default?

Unfortunately, the back-ends based on GTK+ are broken and I didn't
manage to find how to fix the problem.
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20888

That's why I'm looking forward to get Tkinter :-)

>> I'm not convinced that having 2 versions of packages (a "normal" and a
>> "minimal" one) is a good idea. The reason is that you often end up
>> downloading both instead of just one. As one example, I just
>> downloaded 'vlc' and I noticed that it downloaded both 'cups' and
>> 'cups-minimal'. Similarly, for python, anyone needing 'Tkinter' will
>> end up with 2 versions in the store.
>
> The ‘minimal’ variants are there to break cyclic dependencies most of
> the time.  This is the case for cups/cups-filters and Python/Tk.
>
> At run time, cups indeed depends on cups-filters, which depends on
> cups-minimal.  In that case, as you say, we end up downloading those two
> variants.  But there’s not much of a choice, as was discussed during the
> initial review:
>
>   https://lists.gnu.org/archive/html/guix-devel/2015-01/msg00057.html
>
> Conversely, python does not depend on python-minimal.  So python-minimal
> is here just to break the cycle with Tk/libxcb, as noted in python.scm
> and discussed in:
>
>   http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20889
>
> As for Tkinter, it is in a separate “tk” output, which users can choose
> to install or not.  The “tk” output contains a single file, but it pulls
> in Tcl/Tk.

OK. Thanks for the explanation!

Regards,
Fede

bug#20889: Tkinter moved to separate output

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 978 bytes --]

Federico Beffa <beffa@ieee.org> skribis:

> On Mon, Nov 16, 2015 at 1:57 PM, Ludovic Courtès <ludo@gnu.org> wrote:
>> Federico Beffa <beffa@ieee.org> skribis:
>>> Yes, given that the back-end currently used by matploblib doesn't
>>> work, I would change that by using 'Tkinter'.
>>>
>>> Since you are at it, I would also change the default back-end by
>>> changing 'GTK3Agg' to 'TkAgg' in the following snippet:
>>>
>>> (lambda (port)
>>>   (format port "[rc_options]~%
>>> backend = GTK3Agg~%")))))
>>
>> OTOH I see that Matploblib already depends on GTK+ 3.x and
>> python-pygobject, so maybe it’s best to stick to GTK+ by default?
>
> Unfortunately, the back-ends based on GTK+ are broken and I didn't
> manage to find how to fix the problem.
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20888
>
> That's why I'm looking forward to get Tkinter :-)

Could you or someone else test this patch on current master?

Thanks in advance!

Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 963 bytes --]

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 39d0751..76bda96 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -3190,6 +3190,7 @@ transcendental functions).")
      `(("python-pyparsing" ,python-pyparsing)
        ("python-pygobject" ,python-pygobject)
        ("gobject-introspection" ,gobject-introspection)
+       ("python-tkinter" ,python "tk")
        ;; The 'gtk+' package (and 'gdk-pixbuf', 'atk' and 'pango' propagated
        ;; from 'gtk+') provides the required 'typelib' files used by
        ;; 'gobject-introspection'. The location of these files is set with the
@@ -3244,7 +3245,7 @@ transcendental functions).")
             (call-with-output-file "setup.cfg"
               (lambda (port)
                 (format port "[rc_options]~%
-backend = GTK3Agg~%")))))
+backend = TkAgg~%")))))
         (alist-cons-after
          'install 'install-doc
          (lambda* (#:key outputs #:allow-other-keys)

bug#20889: Tkinter moved to separate output

[Federico Beffa]

[-- Attachment #1: Type: text/plain, Size: 2994 bytes --]

On Sun, Nov 29, 2015 at 11:09 PM, Ludovic Courtès <ludo@gnu.org> wrote:
> Federico Beffa <beffa@ieee.org> skribis:
>
>> On Mon, Nov 16, 2015 at 1:57 PM, Ludovic Courtès <ludo@gnu.org> wrote:
>>> Federico Beffa <beffa@ieee.org> skribis:
>>>> Yes, given that the back-end currently used by matploblib doesn't
>>>> work, I would change that by using 'Tkinter'.
>>>>
>>>> Since you are at it, I would also change the default back-end by
>>>> changing 'GTK3Agg' to 'TkAgg' in the following snippet:
>>>>
>>>> (lambda (port)
>>>>   (format port "[rc_options]~%
>>>> backend = GTK3Agg~%")))))
>>>
>>> OTOH I see that Matploblib already depends on GTK+ 3.x and
>>> python-pygobject, so maybe it’s best to stick to GTK+ by default?
>>
>> Unfortunately, the back-ends based on GTK+ are broken and I didn't
>> manage to find how to fix the problem.
>> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=20888
>>
>> That's why I'm looking forward to get Tkinter :-)
>
> Could you or someone else test this patch on current master?

It does not work. It can't find the tcl/tk header files:

...
                 tkagg: no  [The C/C++ header for Tk (tk.h) could not be
                        found.  You may need to install the development
                        package.]
...

With the attached patch it does find the headers, but then it fails at
link time.

...
g++ -pthread -shared
-Wl,-rpath=/gnu/store/cxrqlhbr6xqgi08x3amd0lvl6m7sxx57-python-3.4.3/lib
build/temp.linux-x86_64-3.4/src/agg_py_transforms.o
build/temp.linux-x86_64-3.4/src/_tkagg.o
build/temp.linux-x86_64-3.4/extern/agg24/src/agg_bezier_arc.o
build/temp.linux-x86_64-3.4/extern/agg24/src/agg_curves.o
build/temp.linux-x86_64-3.4/extern/agg24/src/agg_image_filters.o
build/temp.linux-x86_64-3.4/extern/agg24/src/agg_trans_affine.o
build/temp.linux-x86_64-3.4/extern/agg24/src/agg_vcgen_contour.o
build/temp.linux-x86_64-3.4/extern/agg24/src/agg_vcgen_dash.o
build/temp.linux-x86_64-3.4/extern/agg24/src/agg_vcgen_stroke.o
build/temp.linux-x86_64-3.4/extern/agg24/src/agg_vpgen_segmentator.o
build/temp.linux-x86_64-3.4/extern/CXX/IndirectPythonInterface.o
build/temp.linux-x86_64-3.4/extern/CXX/cxxsupport.o
build/temp.linux-x86_64-3.4/extern/CXX/cxx_extensions.o
build/temp.linux-x86_64-3.4/extern/CXX/cxxextensions.o
-L/gnu/store/ipsq9hlk6k0w1qfiw0y0a03bxkp2clhl-tcl-8.6.4/lib
-L/gnu/store/my9wgryzpfvksng6k7h87xz9rphdwddw-tk-8.6.4/lib
-L/usr/local/lib -L/usr/local/lib
-L/gnu/store/cxrqlhbr6xqgi08x3amd0lvl6m7sxx57-python-3.4.3/lib -ltcl
-ltk -lstdc++ -lm -lpython3.4m -o
build/lib.linux-x86_64-3.4/matplotlib/backends/_tkagg.cpython-34m.so
ld: cannot find -ltcl
ld: cannot find -ltk
collect2: error: ld returned 1 exit status
error: command 'g++' failed with exit status 1
phase `build' failed after 94.8 seconds
...

Looking into the failed build directory, it seems that PKG_CONFIG_PATH
is set correctly. Still, it doesn't get the correct flags.

Regards,
Fede

[-- Attachment #2: mpl.patch --]
[-- Type: text/x-diff, Size: 1510 bytes --]

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 39d0751..4dfe557 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -3190,6 +3190,7 @@ transcendental functions).")
      `(("python-pyparsing" ,python-pyparsing)
        ("python-pygobject" ,python-pygobject)
        ("gobject-introspection" ,gobject-introspection)
+       ("python-tkinter" ,python "tk")
        ;; The 'gtk+' package (and 'gdk-pixbuf', 'atk' and 'pango' propagated
        ;; from 'gtk+') provides the required 'typelib' files used by
        ;; 'gobject-introspection'. The location of these files is set with the
@@ -3224,7 +3225,8 @@ transcendental functions).")
        ;; FIXME: Add backends when available.
        ;("python-wxpython" ,python-wxpython)
        ;("python-pyqt" ,python-pyqt)
-       ))
+       ("tcl" ,tcl)
+       ("tk" ,tk)))
     (native-inputs
      `(("pkg-config" ,pkg-config)
        ("texlive" ,texlive)
@@ -3243,8 +3245,13 @@ transcendental functions).")
             (setenv "HOME" (getcwd))
             (call-with-output-file "setup.cfg"
               (lambda (port)
-                (format port "[rc_options]~%
-backend = GTK3Agg~%")))))
+                (format port "[directories]~%
+basedirlist = ~a,~a~%
+~%
+[rc_options]~%
+backend = TkAgg~%"
+                        (assoc-ref inputs "tcl")
+                        (assoc-ref inputs "tk"))))))
         (alist-cons-after
          'install 'install-doc
          (lambda* (#:key outputs #:allow-other-keys)

bug#20889: Tkinter moved to separate output

[Federico Beffa]

[-- Attachment #1: Type: text/plain, Size: 422 bytes --]

Federico Beffa <beffa@ieee.org> writes:

> On Sun, Nov 29, 2015 at 11:09 PM, Ludovic Courtès <ludo@gnu.org> wrote:
>> Federico Beffa <beffa@ieee.org> skribis:

> Looking into the failed build directory, it seems that PKG_CONFIG_PATH
> is set correctly. Still, it doesn't get the correct flags.

The attached patch fixes the problem and I can now plot with TkAgg :-)

Thanks for making Tkinter available!
Fede

[-- Attachment #2: 0001-gnu-python-matplotlib-Add-TkAgg-backend-and-update-t.patch --]
[-- Type: text/x-diff, Size: 4427 bytes --]

From b40cf5522bcc15166ca07dfbae50167203d29e2d Mon Sep 17 00:00:00 2001
From: Federico Beffa <beffa@fbengineering.ch>
Date: Tue, 1 Dec 2015 17:20:59 +0100
Subject: [PATCH 1/2] gnu: python-matplotlib: Add 'TkAgg' backend and update to
 version '1.4.3'.

* gnu/packages/python.scm (python-matplotlib): Do it.
* gnu/packages/patches/matplotlib-setupext-tk.patch: New file.
---
 gnu/packages/patches/matplotlib-setupext-tk.patch | 30 +++++++++++++++++++++++
 gnu/packages/python.scm                           | 17 +++++++++----
 2 files changed, 42 insertions(+), 5 deletions(-)
 create mode 100644 gnu/packages/patches/matplotlib-setupext-tk.patch

diff --git a/gnu/packages/patches/matplotlib-setupext-tk.patch b/gnu/packages/patches/matplotlib-setupext-tk.patch
new file mode 100644
index 0000000..cd0332e
--- /dev/null
+++ b/gnu/packages/patches/matplotlib-setupext-tk.patch
@@ -0,0 +1,30 @@
+Use 'pkg-config' instead of heuristics to find 'tk' flags.
+
+--- matplotlib-1.4.3/setupext.py.orig	2015-12-01 14:21:19.554417453 +0100
++++ matplotlib-1.4.3/setupext.py	2015-12-01 14:35:51.999928797 +0100
+@@ -1457,7 +1457,7 @@
+             p = subprocess.Popen(
+                 '. %s ; eval echo ${%s}' % (file, varname),
+                 shell=True,
+-                executable="/bin/sh",
++                executable="sh",
+                 stdout=subprocess.PIPE)
+             result = p.communicate()[0]
+             return result.decode('ascii')
+@@ -1601,8 +1601,15 @@
+             #      of distros.
+ 
+             # Query Tcl/Tk system for library paths and version string
++            def getoutput(s):
++                ret = os.popen(s).read().strip()
++                return ret
+             try:
+-                tcl_lib_dir, tk_lib_dir, tk_ver = self.query_tcltk()
++                #tcl_lib_dir, tk_lib_dir, tk_ver = self.query_tcltk()
++                pkg_config_res = getoutput('pkg-config --libs tk').split()
++                tk_ver = pkg_config_res[-1][-3:]
++                tcl_lib_dir = pkg_config_res[0][2:] + '/tcl' + tk_ver
++                tk_lib_dir = pkg_config_res[1][2:] + '/tk' + tk_ver
+             except:
+                 tk_ver = ''
+                 result = self.hardcoded_tcl_config()
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 39d0751..5e23bab 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -3175,7 +3175,7 @@ transcendental functions).")
 (define-public python-matplotlib
   (package
     (name "python-matplotlib")
-    (version "1.4.2")
+    (version "1.4.3")
     (source
      (origin
        (method url-fetch)
@@ -3183,13 +3183,15 @@ transcendental functions).")
                            "/matplotlib-" version ".tar.gz"))
        (sha256
         (base32
-         "0m6v9nwdldlwk22gcd339zg6mny5m301fxgks7z8sb8m9wawg8qp"))))
+         "1dn05cvd0g984lzhh72wa0z93psgwshbbg93fkab6slx5m3l95av"))
+       (patches (list (search-patch "matplotlib-setupext-tk.patch")))))
     (build-system python-build-system)
     (outputs '("out" "doc"))
     (propagated-inputs ; the following packages are all needed at run time
      `(("python-pyparsing" ,python-pyparsing)
        ("python-pygobject" ,python-pygobject)
        ("gobject-introspection" ,gobject-introspection)
+       ("python" ,python "tk")
        ;; The 'gtk+' package (and 'gdk-pixbuf', 'atk' and 'pango' propagated
        ;; from 'gtk+') provides the required 'typelib' files used by
        ;; 'gobject-introspection'. The location of these files is set with the
@@ -3224,7 +3226,8 @@ transcendental functions).")
        ;; FIXME: Add backends when available.
        ;("python-wxpython" ,python-wxpython)
        ;("python-pyqt" ,python-pyqt)
-       ))
+       ("tcl" ,tcl)
+       ("tk" ,tk)))
     (native-inputs
      `(("pkg-config" ,pkg-config)
        ("texlive" ,texlive)
@@ -3243,8 +3246,12 @@ transcendental functions).")
             (setenv "HOME" (getcwd))
             (call-with-output-file "setup.cfg"
               (lambda (port)
-                (format port "[rc_options]~%
-backend = GTK3Agg~%")))))
+                (format port "[directories]~%
+basedirlist = ~a,~a~%
+[rc_options]~%
+backend = TkAgg~%"
+                        (assoc-ref inputs "tcl")
+                        (assoc-ref inputs "tk"))))))
         (alist-cons-after
          'install 'install-doc
          (lambda* (#:key outputs #:allow-other-keys)
-- 
2.4.3

bug#20889: Tkinter moved to separate output

[Ludovic Courtès]

Federico Beffa <beffa@ieee.org> skribis:

> The attached patch fixes the problem and I can now plot with TkAgg :-)
>
> Thanks for making Tkinter available!
> Fede
>
> From b40cf5522bcc15166ca07dfbae50167203d29e2d Mon Sep 17 00:00:00 2001
> From: Federico Beffa <beffa@fbengineering.ch>
> Date: Tue, 1 Dec 2015 17:20:59 +0100
> Subject: [PATCH 1/2] gnu: python-matplotlib: Add 'TkAgg' backend and update to
>  version '1.4.3'.
>
> * gnu/packages/python.scm (python-matplotlib): Do it.
> * gnu/packages/patches/matplotlib-setupext-tk.patch: New file.

Nice!

Please make sure to add the patch to gnu-system.am, but otherwise looks
great!

Thanks for fixing it!

I gather this addresses http://bugs.gnu.org/20888, right?

Ludo’.

bug#20889: Tkinter moved to separate output

[Federico Beffa]

On Tue, Dec 1, 2015 at 11:00 PM, Ludovic Courtès <ludo@gnu.org> wrote:
> I gather this addresses http://bugs.gnu.org/20888, right?

Well, it partially solves it:
Now there is a working backend suitable for interactive plotting which
is enabled by default. However, the 'GTK3Agg' backend is still there
and still not working. At some point it would be nice to have it
working as well.

Regards,
Fede