[PATCH 4/4] gnu: matplotlib: Add gtk3 backends.

[PATCH 4/4] gnu: matplotlib: Add gtk3 backends.

[Federico Beffa]

[-- Attachment #1: Type: text/plain, Size: 794 bytes --]

This one is problematic: for the package to work the user has to define

export GI_TYPELIB_PATH=~/.guix-profile/lib/girepository-1.0

and

export LD_LIBRARY_PATH=~/.guix-profile/lib

The former is required to instruct 'gobject-introspection' where to
find typelib files and should be fine, but I'm not sure how to display
a message telling this at the end of the installation.

The latter is a hack. As discussed on another thread, the typelib
files have references to dynamic libraries and they do not include the
full path.  We are looking into a patch to add the full path into the
typelib files, however, the patch only works on newer versions of
'gobject-introspection' which breaks several other packages.

Setting this environment variable should be a temporary solution.

Regards,
Fede

[-- Attachment #2: 0004-gnu-matplotlib-Add-gtk3-backends.patch --]
[-- Type: text/x-patch, Size: 9122 bytes --]

From e1e3df22ea06bd7325bc2a1fd389c89f13ccf752 Mon Sep 17 00:00:00 2001
From: Federico Beffa <beffa@fbengineering.ch>
Date: Thu, 11 Dec 2014 14:26:13 +0100
Subject: [PATCH 4/4] gnu: matplotlib: Add gtk3 backends.

* gnu/packages/python.scm (python-matplotlib, python2-matplotlib): Add
  necessary inputs and 'configure-environment' phase.
---
 gnu/packages/python.scm | 130 +++++++++++++++++++++++++++++++++---------------
 1 file changed, 89 insertions(+), 41 deletions(-)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index 9b3d721..bd5c519 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -46,11 +46,14 @@
   #:use-module (gnu packages texlive)
   #:use-module (gnu packages texinfo)
   #:use-module (gnu packages image)
+  #:use-module (gnu packages ghostscript)
   #:use-module (gnu packages imagemagick)
   #:use-module (gnu packages fontutils)
   #:use-module (gnu packages which)
   #:use-module (gnu packages perl)
   #:use-module (gnu packages xorg)
+  #:use-module (gnu packages glib)
+  #:use-module (gnu packages gtk)
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
@@ -2117,10 +2120,35 @@ that client code uses to construct the grammar directly in Python code.")
          "0m6v9nwdldlwk22gcd339zg6mny5m301fxgks7z8sb8m9wawg8qp"))))
     (build-system python-build-system)
     (outputs '("out" "doc"))
+    (propagated-inputs
+     `(("python-pyparsing" ,python-pyparsing)
+       ("python-pygobject" ,python-pygobject)
+       ("gobject-introspection" ,gobject-introspection)
+       ;; The 'gtk+' package (and 'gdk-pixbuf', 'atk' and 'pango' propagated
+       ;; from 'gtk+') provides the required 'typelib' files used by
+       ;; 'gobject-introspection'. The location of these files is set with the
+       ;; help of the environment variable GI_TYPELIB_PATH. At build time this
+       ;; is done automatically by a 'native-search-path' procedure. However,
+       ;; at run-time the user must set this variable as follows:
+       ;;
+       ;; export GI_TYPELIB_PATH=~/.guix-profile/lib/girepository-1.0
+       ;;
+       ;; 'typelib' files include references to dynamic libraries. Currently
+       ;; the references do not include the full path to the libraries. For
+       ;; this reason the user must set the LD_LIBRARY_PATH to the location of 
+       ;; 'libgtk-3.so.0', 'libgdk-3.so.0' and 'libatk-1.0.so.0':
+       ;;
+       ;; export LD_LIBRARY_PATH=~/.guix-profile/lib
+       ("gtk+" ,gtk+)
+       ;; From version 1.4.0 'matplotlib' makes use of 'cairocffi' instead of
+       ;; 'pycairo'. However, 'pygobject' makes use of a 'pycairo' 'context'
+       ;; object. For this reason we need to import both libraries.
+       ;; https://pythonhosted.org/cairocffi/cffi_api.html#converting-pycairo
+       ("python-pycairo" ,python-pycairo)
+       ("python-cairocffi" ,python-cairocffi)))
     (inputs
      `(("python-setuptools" ,python-setuptools)
        ("python-dateutil" ,python-dateutil-2)
-       ("python-pyparsing" ,python-pyparsing)
        ("python-six" ,python-six)
        ("python-pytz" ,python-pytz)
        ("python-numpy" ,python-numpy-bootstrap)
@@ -2131,53 +2159,65 @@ that client code uses to construct the grammar directly in Python code.")
        ("libpng" ,libpng)
        ("imagemagick" ,imagemagick)
        ("freetype" ,freetype)
+       ("cairo" ,cairo)
+       ("glib" ,glib)
+       ("python-pillow" ,python-pillow)
        ;; FIXME: Add backends when available.
-       ;("python-pygtk" ,python-pygtk)
-       ;("python-pycairo" ,python-pycairo)
-       ;("python-pygobject" ,python-pygobject)
        ;("python-wxpython" ,python-wxpython)
        ;("python-pyqt" ,python-pyqt)
        ))
     (native-inputs
      `(("pkg-config" ,pkg-config)
        ("texlive" ,texlive)
+       ("ghostscript" ,ghostscript)
        ("texinfo" ,texinfo)))
     (arguments
      `(#:phases
-       (alist-cons-after
-        'install 'install-doc
-        (lambda* (#:key outputs #:allow-other-keys)
-          (let* ((data (string-append (assoc-ref outputs "doc") "/share"))
-                 (doc (string-append data "/doc/" ,name "-" ,version))
-                 (info (string-append data "/info"))
-                 (html (string-append doc "/html")))
-            (with-directory-excursion "doc"
-              ;; Without setting this variable we get an encoding error.
-              (setenv "LANG" "en_US.UTF-8")
-              ;; Produce pdf in 'A4' format.
-              (substitute* (find-files "." "conf\\.py")
-                (("latex_paper_size = 'letter'")
-                 "latex_paper_size = 'a4'"))
-              (mkdir-p html)
-              (mkdir-p info)
-              ;; The doc recommends to run the 'html' target twice.
-              (system* "python" "make.py" "html")
-              (system* "python" "make.py" "html")
-              (system* "python" "make.py" "latex")
-              (system* "python" "make.py" "texinfo")
-              (copy-file "build/texinfo/matplotlib.info"
-                         (string-append info "/matplotlib.info"))
-              (copy-file "build/latex/Matplotlib.pdf"
-                         (string-append doc "/Matplotlib.pdf"))
-              (with-directory-excursion "build/html"
-                (map (lambda (file)
-                       (let* ((dir (dirname file))
-                              (tgt-dir (string-append html "/" dir)))
-                         (unless (equal? "." dir)
-                           (mkdir-p tgt-dir))
-                         (copy-file file (string-append html "/" file))))
-                     (find-files "." ".*"))))))
-        %standard-phases)))
+       (alist-cons-before
+        'build 'configure-environment
+        (lambda* (#:key outputs inputs #:allow-other-keys)
+          (let ((cairo (assoc-ref inputs "cairo"))
+                (gtk+ (assoc-ref inputs "gtk+")))
+            ;; Setting these directories in the 'basedirlist' of 'setup.cfg'
+            ;; has not effect.
+            ;;
+            ;; FIXME: setting LD_LIBRARY_PATH should be removed once we patch
+            ;; gobject-introspection to include the full path of shared
+            ;; libraries in 'typelib' files.
+            (setenv "LD_LIBRARY_PATH"
+                    (string-append cairo "/lib:" gtk+ "/lib"))
+            (setenv "HOME" (getcwd))
+            (call-with-output-file "setup.cfg"
+              (lambda (port)
+                (format port "[rc_options]~%
+backend = GTK3Agg~%")))))
+        (alist-cons-after
+         'install 'install-doc
+         (lambda* (#:key outputs #:allow-other-keys)
+           (let* ((data (string-append (assoc-ref outputs "doc") "/share"))
+                  (doc (string-append data "/doc/" ,name "-" ,version))
+                  (info (string-append data "/info"))
+                  (html (string-append doc "/html")))
+             (with-directory-excursion "doc"
+               ;; Without setting this variable we get an encoding error.
+               (setenv "LANG" "en_US.UTF-8")
+               ;; Produce pdf in 'A4' format.
+               (substitute* (find-files "." "conf\\.py")
+                 (("latex_paper_size = 'letter'")
+                  "latex_paper_size = 'a4'"))
+               (mkdir-p html)
+               (mkdir-p info)
+               ;; The doc recommends to run the 'html' target twice.
+               (system* "python" "make.py" "html")
+               (system* "python" "make.py" "html")
+               (system* "python" "make.py" "latex")
+               (system* "python" "make.py" "texinfo")
+               (copy-file "build/texinfo/matplotlib.info"
+                          (string-append info "/matplotlib.info"))
+               (copy-file "build/latex/Matplotlib.pdf"
+                          (string-append doc "/Matplotlib.pdf"))
+               (copy-recursively "build/html" html))))
+        %standard-phases))))
     (home-page "http://matplotlib.org")
     (synopsis "2D plotting library for Python")
     (description
@@ -2193,9 +2233,17 @@ toolkits.")
     (package (inherit matplotlib)
       ;; Make sure we use exactly PYTHON2-NUMPYDOC, which is
       ;; customized for Python 2.
-      (inputs `(("python2-numpydoc" ,python2-numpydoc)
-                ,@(alist-delete "python-numpydoc" 
-                                (package-inputs matplotlib)))))))
+      (propagated-inputs 
+       `(("python2-py2cairo" ,python2-py2cairo)
+         ("python2-pygobject-2" ,python2-pygobject-2)
+         ,@(alist-delete "python-pycairo"
+                         (alist-delete "python-pygobject"
+                                       (package-propagated-inputs 
+                                        matplotlib)))))
+      (inputs 
+       `(("python2-numpydoc" ,python2-numpydoc)
+         ,@(alist-delete "python-numpydoc" 
+                         (package-inputs matplotlib)))))))
 
 ;; Scipy 0.14.0 with Numpy 0.19.X fails several tests.  This is known and
 ;; planned to be fixed in 0.14.1.  It is claimed that the failures can safely
-- 
1.8.4

Re: [PATCH 4/4] gnu: matplotlib: Add gtk3 backends.

[Ludovic Courtès]

Federico Beffa <beffa@ieee.org> skribis:

> This one is problematic: for the package to work the user has to define
>
> export GI_TYPELIB_PATH=~/.guix-profile/lib/girepository-1.0
>
> and
>
> export LD_LIBRARY_PATH=~/.guix-profile/lib
>
> The former is required to instruct 'gobject-introspection' where to
> find typelib files and should be fine, but I'm not sure how to display
> a message telling this at the end of the installation.

One should get a message about GI_TYPELIB_PATH, but only if
gobject-introspection is also installed.  Not sure if that helps.

> The latter is a hack. As discussed on another thread, the typelib
> files have references to dynamic libraries and they do not include the
> full path.  We are looking into a patch to add the full path into the
> typelib files, however, the patch only works on newer versions of
> 'gobject-introspection' which breaks several other packages.

Yes, right.

Speaking of which, could you (or Mark?) create a branch with all the
updates that you did, so we keep track of it?

> From e1e3df22ea06bd7325bc2a1fd389c89f13ccf752 Mon Sep 17 00:00:00 2001
> From: Federico Beffa <beffa@fbengineering.ch>
> Date: Thu, 11 Dec 2014 14:26:13 +0100
> Subject: [PATCH 4/4] gnu: matplotlib: Add gtk3 backends.
>
> * gnu/packages/python.scm (python-matplotlib, python2-matplotlib): Add
>   necessary inputs and 'configure-environment' phase.

The patch also move python-pyparsing from ‘inputs’ to
‘propagated-inputs’, so it would be good to mention it.

> +    (propagated-inputs
> +     `(("python-pyparsing" ,python-pyparsing)
> +       ("python-pygobject" ,python-pygobject)

Could you add a comment saying why they’re here?

>      (native-inputs
>       `(("pkg-config" ,pkg-config)
>         ("texlive" ,texlive)
> +       ("ghostscript" ,ghostscript)

This doesn’t sound GTK-related, no?

Thanks,
Ludo’.

Re: [PATCH 4/4] gnu: matplotlib: Add gtk3 backends.

[Federico Beffa]

[-- Attachment #1: Type: text/plain, Size: 2157 bytes --]

On Mon, Dec 15, 2014 at 11:37 PM, Ludovic Courtès <ludo@gnu.org> wrote:
> One should get a message about GI_TYPELIB_PATH, but only if
> gobject-introspection is also installed.  Not sure if that helps.

My question was actually how to produce the message. Is that
automatically handled by the 'search-path' directive in the package?
If not, could you point me an example?

>> From e1e3df22ea06bd7325bc2a1fd389c89f13ccf752 Mon Sep 17 00:00:00 2001
>> From: Federico Beffa <beffa@fbengineering.ch>
>> Date: Thu, 11 Dec 2014 14:26:13 +0100
>> Subject: [PATCH 4/4] gnu: matplotlib: Add gtk3 backends.
>>
>> * gnu/packages/python.scm (python-matplotlib, python2-matplotlib): Add
>>   necessary inputs and 'configure-environment' phase.
>
> The patch also move python-pyparsing from ‘inputs’ to
> ‘propagated-inputs’, so it would be good to mention it.

Yes, I forgot to mention this. I've expanded the commit log.

>
>> +    (propagated-inputs
>> +     `(("python-pyparsing" ,python-pyparsing)
>> +       ("python-pygobject" ,python-pygobject)
>
> Could you add a comment saying why they’re here?

Hope my comment is enough :-)

>
>>      (native-inputs
>>       `(("pkg-config" ,pkg-config)
>>         ("texlive" ,texlive)
>> +       ("ghostscript" ,ghostscript)
>
> This doesn’t sound GTK-related, no?

This is an optional dependency and I initially thought to add it to
make the package more "complete". After your comment I decided to look
after what it is actually used for. I learned that it is "only" needed
as one component necessary to make matplotlib able to render text
through LaTeX. So, if we do not make texlive a propagated-input of
matplotlib, then ghostscript is not really needed and I've removed it
again.

Given the size of texlive I'm hesitant in making it a
propagated-input. However, it would make it possible to produce
pictures with the same fonts as the ones used in a latex document,
which is quite nice.

What do you think?

I've removed ghostscript, but I've added another optional dependency:
pillow. This time I mention it in the commit log :-)

Regards,
Fede

[-- Attachment #2: 0004-gnu-matplotlib-Add-gtk3-backends-and-optional-depend.patch --]
[-- Type: text/x-patch, Size: 9116 bytes --]

From 4eb7c8dff92f641391c0d902e1ab43536ce0d763 Mon Sep 17 00:00:00 2001
From: Federico Beffa <beffa@fbengineering.ch>
Date: Thu, 11 Dec 2014 14:26:13 +0100
Subject: [PATCH 4/4] gnu: matplotlib: Add gtk3 backends and optional
 dependency.  Adjust inputs.

* gnu/packages/python.scm (python-matplotlib, python2-matplotlib): Add gtk3
  backends with the necessary inputs and a 'configure-environment' phase.  Add
  the optional 'python-pillow' dependency.  Move 'python-pyparsing' from
  'inputs' to 'propagated-inputs' as it is required at run time.
---
 gnu/packages/python.scm | 128 ++++++++++++++++++++++++++++++++----------------
 1 file changed, 87 insertions(+), 41 deletions(-)

diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index c2657f2..90d0a8a 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -51,6 +51,8 @@
   #:use-module (gnu packages which)
   #:use-module (gnu packages perl)
   #:use-module (gnu packages xorg)
+  #:use-module (gnu packages glib)
+  #:use-module (gnu packages gtk)
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
@@ -2117,10 +2119,35 @@ that client code uses to construct the grammar directly in Python code.")
          "0m6v9nwdldlwk22gcd339zg6mny5m301fxgks7z8sb8m9wawg8qp"))))
     (build-system python-build-system)
     (outputs '("out" "doc"))
+    (propagated-inputs ; the following packages are all needed at run time
+     `(("python-pyparsing" ,python-pyparsing)
+       ("python-pygobject" ,python-pygobject)
+       ("gobject-introspection" ,gobject-introspection)
+       ;; The 'gtk+' package (and 'gdk-pixbuf', 'atk' and 'pango' propagated
+       ;; from 'gtk+') provides the required 'typelib' files used by
+       ;; 'gobject-introspection'. The location of these files is set with the
+       ;; help of the environment variable GI_TYPELIB_PATH. At build time this
+       ;; is done automatically by a 'native-search-path' procedure. However,
+       ;; at run-time the user must set this variable as follows:
+       ;;
+       ;; export GI_TYPELIB_PATH=~/.guix-profile/lib/girepository-1.0
+       ;;
+       ;; 'typelib' files include references to dynamic libraries. Currently
+       ;; the references do not include the full path to the libraries. For
+       ;; this reason the user must set the LD_LIBRARY_PATH to the location of 
+       ;; 'libgtk-3.so.0', 'libgdk-3.so.0' and 'libatk-1.0.so.0':
+       ;;
+       ;; export LD_LIBRARY_PATH=~/.guix-profile/lib
+       ("gtk+" ,gtk+)
+       ;; From version 1.4.0 'matplotlib' makes use of 'cairocffi' instead of
+       ;; 'pycairo'. However, 'pygobject' makes use of a 'pycairo' 'context'
+       ;; object. For this reason we need to import both libraries.
+       ;; https://pythonhosted.org/cairocffi/cffi_api.html#converting-pycairo
+       ("python-pycairo" ,python-pycairo)
+       ("python-cairocffi" ,python-cairocffi)))
     (inputs
      `(("python-setuptools" ,python-setuptools)
        ("python-dateutil" ,python-dateutil-2)
-       ("python-pyparsing" ,python-pyparsing)
        ("python-six" ,python-six)
        ("python-pytz" ,python-pytz)
        ("python-numpy" ,python-numpy-bootstrap)
@@ -2131,10 +2158,10 @@ that client code uses to construct the grammar directly in Python code.")
        ("libpng" ,libpng)
        ("imagemagick" ,imagemagick)
        ("freetype" ,freetype)
+       ("cairo" ,cairo)
+       ("glib" ,glib)
+       ("python-pillow" ,python-pillow)
        ;; FIXME: Add backends when available.
-       ;("python-pygtk" ,python-pygtk)
-       ;("python-pycairo" ,python-pycairo)
-       ;("python-pygobject" ,python-pygobject)
        ;("python-wxpython" ,python-wxpython)
        ;("python-pyqt" ,python-pyqt)
        ))
@@ -2144,40 +2171,51 @@ that client code uses to construct the grammar directly in Python code.")
        ("texinfo" ,texinfo)))
     (arguments
      `(#:phases
-       (alist-cons-after
-        'install 'install-doc
-        (lambda* (#:key outputs #:allow-other-keys)
-          (let* ((data (string-append (assoc-ref outputs "doc") "/share"))
-                 (doc (string-append data "/doc/" ,name "-" ,version))
-                 (info (string-append data "/info"))
-                 (html (string-append doc "/html")))
-            (with-directory-excursion "doc"
-              ;; Without setting this variable we get an encoding error.
-              (setenv "LANG" "en_US.UTF-8")
-              ;; Produce pdf in 'A4' format.
-              (substitute* (find-files "." "conf\\.py")
-                (("latex_paper_size = 'letter'")
-                 "latex_paper_size = 'a4'"))
-              (mkdir-p html)
-              (mkdir-p info)
-              ;; The doc recommends to run the 'html' target twice.
-              (system* "python" "make.py" "html")
-              (system* "python" "make.py" "html")
-              (system* "python" "make.py" "latex")
-              (system* "python" "make.py" "texinfo")
-              (copy-file "build/texinfo/matplotlib.info"
-                         (string-append info "/matplotlib.info"))
-              (copy-file "build/latex/Matplotlib.pdf"
-                         (string-append doc "/Matplotlib.pdf"))
-              (with-directory-excursion "build/html"
-                (map (lambda (file)
-                       (let* ((dir (dirname file))
-                              (tgt-dir (string-append html "/" dir)))
-                         (unless (equal? "." dir)
-                           (mkdir-p tgt-dir))
-                         (copy-file file (string-append html "/" file))))
-                     (find-files "." ".*"))))))
-        %standard-phases)))
+       (alist-cons-before
+        'build 'configure-environment
+        (lambda* (#:key outputs inputs #:allow-other-keys)
+          (let ((cairo (assoc-ref inputs "cairo"))
+                (gtk+ (assoc-ref inputs "gtk+")))
+            ;; Setting these directories in the 'basedirlist' of 'setup.cfg'
+            ;; has not effect.
+            ;;
+            ;; FIXME: setting LD_LIBRARY_PATH should be removed once we patch
+            ;; gobject-introspection to include the full path of shared
+            ;; libraries in 'typelib' files.
+            (setenv "LD_LIBRARY_PATH"
+                    (string-append cairo "/lib:" gtk+ "/lib"))
+            (setenv "HOME" (getcwd))
+            (call-with-output-file "setup.cfg"
+              (lambda (port)
+                (format port "[rc_options]~%
+backend = GTK3Agg~%")))))
+        (alist-cons-after
+         'install 'install-doc
+         (lambda* (#:key outputs #:allow-other-keys)
+           (let* ((data (string-append (assoc-ref outputs "doc") "/share"))
+                  (doc (string-append data "/doc/" ,name "-" ,version))
+                  (info (string-append data "/info"))
+                  (html (string-append doc "/html")))
+             (with-directory-excursion "doc"
+               ;; Without setting this variable we get an encoding error.
+               (setenv "LANG" "en_US.UTF-8")
+               ;; Produce pdf in 'A4' format.
+               (substitute* (find-files "." "conf\\.py")
+                 (("latex_paper_size = 'letter'")
+                  "latex_paper_size = 'a4'"))
+               (mkdir-p html)
+               (mkdir-p info)
+               ;; The doc recommends to run the 'html' target twice.
+               (system* "python" "make.py" "html")
+               (system* "python" "make.py" "html")
+               (system* "python" "make.py" "latex")
+               (system* "python" "make.py" "texinfo")
+               (copy-file "build/texinfo/matplotlib.info"
+                          (string-append info "/matplotlib.info"))
+               (copy-file "build/latex/Matplotlib.pdf"
+                          (string-append doc "/Matplotlib.pdf"))
+               (copy-recursively "build/html" html))))
+        %standard-phases))))
     (home-page "http://matplotlib.org")
     (synopsis "2D plotting library for Python")
     (description
@@ -2193,9 +2231,17 @@ toolkits.")
     (package (inherit matplotlib)
       ;; Make sure we use exactly PYTHON2-NUMPYDOC, which is
       ;; customized for Python 2.
-      (inputs `(("python2-numpydoc" ,python2-numpydoc)
-                ,@(alist-delete "python-numpydoc" 
-                                (package-inputs matplotlib)))))))
+      (propagated-inputs 
+       `(("python2-py2cairo" ,python2-py2cairo)
+         ("python2-pygobject-2" ,python2-pygobject-2)
+         ,@(alist-delete "python-pycairo"
+                         (alist-delete "python-pygobject"
+                                       (package-propagated-inputs 
+                                        matplotlib)))))
+      (inputs 
+       `(("python2-numpydoc" ,python2-numpydoc)
+         ,@(alist-delete "python-numpydoc" 
+                         (package-inputs matplotlib)))))))
 
 ;; Scipy 0.14.0 with Numpy 0.19.X fails several tests.  This is known and
 ;; planned to be fixed in 0.14.1.  It is claimed that the failures can safely
-- 
1.8.4

Re: [PATCH 4/4] gnu: matplotlib: Add gtk3 backends.

[Ludovic Courtès]

Federico Beffa <beffa@ieee.org> skribis:

> On Mon, Dec 15, 2014 at 11:37 PM, Ludovic Courtès <ludo@gnu.org> wrote:
>> One should get a message about GI_TYPELIB_PATH, but only if
>> gobject-introspection is also installed.  Not sure if that helps.
>
> My question was actually how to produce the message. Is that
> automatically handled by the 'search-path' directive in the package?

Yes, exactly.

As an example, if you install both Guile and Guile-JSON, ‘guix package’
will invite you to define GUILE_LOAD_PATH if it’s not already defined.
And then, ‘guix package --search-paths’ will display all the environment
variables that should be defined.

>>> +    (propagated-inputs
>>> +     `(("python-pyparsing" ,python-pyparsing)
>>> +       ("python-pygobject" ,python-pygobject)
>>
>> Could you add a comment saying why they’re here?
>
> Hope my comment is enough :-)

It’s perfect!

>>
>>>      (native-inputs
>>>       `(("pkg-config" ,pkg-config)
>>>         ("texlive" ,texlive)
>>> +       ("ghostscript" ,ghostscript)
>>
>> This doesn’t sound GTK-related, no?
>
> This is an optional dependency and I initially thought to add it to
> make the package more "complete". After your comment I decided to look
> after what it is actually used for. I learned that it is "only" needed
> as one component necessary to make matplotlib able to render text
> through LaTeX. So, if we do not make texlive a propagated-input of
> matplotlib, then ghostscript is not really needed and I've removed it
> again.
>
> Given the size of texlive I'm hesitant in making it a
> propagated-input. However, it would make it possible to produce
> pictures with the same fonts as the ones used in a latex document,
> which is quite nice.
>
> What do you think?

I think TeXLive shouldn’t be a propagated input, nor even something
referenced by the package output.

Perhaps the LaTeX functionality magically becomes available when
‘pdflatex’ or similar is found in $PATH, in which case there’s nothing
to do: users who care would explicitly install TeX Live in their profile
and get that functionality.

WDYT?

> From 4eb7c8dff92f641391c0d902e1ab43536ce0d763 Mon Sep 17 00:00:00 2001
> From: Federico Beffa <beffa@fbengineering.ch>
> Date: Thu, 11 Dec 2014 14:26:13 +0100
> Subject: [PATCH 4/4] gnu: matplotlib: Add gtk3 backends and optional
>  dependency.  Adjust inputs.
>
> * gnu/packages/python.scm (python-matplotlib, python2-matplotlib): Add gtk3
>   backends with the necessary inputs and a 'configure-environment' phase.  Add
>   the optional 'python-pillow' dependency.  Move 'python-pyparsing' from
>   'inputs' to 'propagated-inputs' as it is required at run time.

LGTM, thanks!

Ludo’.

Re: [PATCH 4/4] gnu: matplotlib: Add gtk3 backends.

[Federico Beffa]

On Wed, Dec 17, 2014 at 9:34 PM, Ludovic Courtès <ludo@gnu.org> wrote:
> Perhaps the LaTeX functionality magically becomes available when
> ‘pdflatex’ or similar is found in $PATH, in which case there’s nothing
> to do: users who care would explicitly install TeX Live in their profile
> and get that functionality.

You are absolutely right. I tested it this way and it works :-)

Thanks,
Fede

Re: [PATCH 4/4] gnu: matplotlib: Add gtk3 backends.

[Ludovic Courtès]

I found that “guix build python2-matplotlib -n” triggers a seemingly
infinite loop (either uses of ‘package-with-python2’ somehow introduce
cycles, or they lead to very large DAGs), which is what is causing Hydra
evaluation failures.

I tried reverting 25f9a0 but it doesn’t help.

Could you try to bisect it and revert or fix the problem?  If it turns
out to require more time, could you just comment out the offending parts
of python.scm so Hydra can resume?

Thanks in advance, and apologies for not noticing earlier!

Ludo’.

Re: [PATCH 4/4] gnu: matplotlib: Add gtk3 backends.

[Federico Beffa]

On Tue, Dec 23, 2014 at 1:17 AM, Ludovic Courtès <ludo@gnu.org> wrote:
> I found that “guix build python2-matplotlib -n” triggers a seemingly
> infinite loop (either uses of ‘package-with-python2’ somehow introduce
> cycles, or they lead to very large DAGs), which is what is causing Hydra
> evaluation failures.
>
> I tried reverting 25f9a0 but it doesn’t help.
>
> Could you try to bisect it and revert or fix the problem?  If it turns
> out to require more time, could you just comment out the offending parts
> of python.scm so Hydra can resume?

The build does finish as I've tested it on my machine before
committing the package. However, for some reason, guix needs very long
to generate the derivation. On my machine (quad-core Xeon E5520)
python2-matplotlib takes ca. 31 mins to start building.
Python2-scipy, which includes python2-matplotlib, takes ca. 80 mins.!
Currently I do not understand the reason, but I note that matplotlib,
numpy and scipy are the only python packages with a large number of
inputs.

I've tried to use trace in the REPL and I see that there is a very
large activity about inputs and their origins. However, the buffer
just becomes too large to handle. It needs more sophisticated
debugging methods.

For the moment I've commented out python2-matplotlib and the python-2
versions of the packages making use of it: scipy and numpy.

Sorry for the inconvenience!  I did not expect that this would cause
troubles to hydra.
Fede

Re: [PATCH 4/4] gnu: matplotlib: Add gtk3 backends.

[Eric Bavier]

Federico Beffa writes:

> On Tue, Dec 23, 2014 at 1:17 AM, Ludovic Courtès <ludo@gnu.org> wrote:
>> I found that “guix build python2-matplotlib -n” triggers a seemingly
>> infinite loop (either uses of ‘package-with-python2’ somehow introduce
>> cycles, or they lead to very large DAGs), which is what is causing Hydra
>> evaluation failures.
>>
>> I tried reverting 25f9a0 but it doesn’t help.
>>
>> Could you try to bisect it and revert or fix the problem?  If it turns
>> out to require more time, could you just comment out the offending parts
>> of python.scm so Hydra can resume?
>
> The build does finish as I've tested it on my machine before
> committing the package. However, for some reason, guix needs very long
> to generate the derivation. On my machine (quad-core Xeon E5520)
> python2-matplotlib takes ca. 31 mins to start building.
> Python2-scipy, which includes python2-matplotlib, takes ca. 80 mins.!
> Currently I do not understand the reason, but I note that matplotlib,
> numpy and scipy are the only python packages with a large number of
> inputs.

I wonder if this could this be a result of package-with-python2's
behavior of recursively creating new package objects for each input in a
package.  As Ludovic suggested, this could easily create very large DAGs
that then need to be processed.

This behavior of package-with-python2 was giving me a nightmare when
trying to code some improvements to `guix refresh -l`, since there would
easily become tens or hundreds of logically identical python package
objects floating through the package dependency DAG.  The reason I
haven't pushed any of those `guix refresh -l` changes is because I
haven't yet figure out a way to get output from it in under ~10 minutes.

-- 
Eric Bavier

Please avoid sending me Word or PowerPoint attachments.
See http://www.gnu.org/philosophy/no-word-attachments.html

Re: [PATCH 4/4] gnu: matplotlib: Add gtk3 backends.

[Ludovic Courtès]

Federico Beffa <beffa@ieee.org> skribis:

> The build does finish as I've tested it on my machine before
> committing the package. However, for some reason, guix needs very long
> to generate the derivation. On my machine (quad-core Xeon E5520)
> python2-matplotlib takes ca. 31 mins to start building.
> Python2-scipy, which includes python2-matplotlib, takes ca. 80 mins.!

Eric Bavier <ericbavier@gmail.com> skribis:

> I wonder if this could this be a result of package-with-python2's
> behavior of recursively creating new package objects for each input in a
> package.  As Ludovic suggested, this could easily create very large DAGs
> that then need to be processed.
>
> This behavior of package-with-python2 was giving me a nightmare when
> trying to code some improvements to `guix refresh -l`, since there would
> easily become tens or hundreds of logically identical python package
> objects floating through the package dependency DAG.

Ouch, thanks for putting me on the right track.  Basically
‘package-with-explicit-python’ was recursing way too much, leading to
hundreds of unique package objects, and thus defeating memoization of
‘package-derivation’.

Commit 78a2745 fixes that.  ‘guix build python2-matplotlib -n
--no-substitutes’ takes ~5 seconds on my laptop (which is still too much
IMO, but that’s another story.)  I’ve thus reinstated numpy, scipy, etc.


In the future, don’t hesitate to report a bug when computing a
derivation takes more than a few seconds.

Eric: the good news is that you can now resume work on ‘guix refresh
-l’.  ;-)

Ludo’.

Re: [PATCH 4/4] gnu: matplotlib: Add gtk3 backends.

[Eric Bavier]

Ludovic Courtès writes:

> Federico Beffa <beffa@ieee.org> skribis:
>
>> The build does finish as I've tested it on my machine before
>> committing the package. However, for some reason, guix needs very long
>> to generate the derivation. On my machine (quad-core Xeon E5520)
>> python2-matplotlib takes ca. 31 mins to start building.
>> Python2-scipy, which includes python2-matplotlib, takes ca. 80 mins.!
>
> Eric Bavier <ericbavier@gmail.com> skribis:
>
>> I wonder if this could this be a result of package-with-python2's
>> behavior of recursively creating new package objects for each input in a
>> package.  As Ludovic suggested, this could easily create very large DAGs
>> that then need to be processed.
>>
>> This behavior of package-with-python2 was giving me a nightmare when
>> trying to code some improvements to `guix refresh -l`, since there would
>> easily become tens or hundreds of logically identical python package
>> objects floating through the package dependency DAG.
>
> Ouch, thanks for putting me on the right track.  Basically
> ‘package-with-explicit-python’ was recursing way too much, leading to
> hundreds of unique package objects, and thus defeating memoization of
> ‘package-derivation’.
>
> Commit 78a2745 fixes that.  ‘guix build python2-matplotlib -n
> --no-substitutes’ takes ~5 seconds on my laptop (which is still too much
> IMO, but that’s another story.)  I’ve thus reinstated numpy, scipy,
> etc.

Great.

> Eric: the good news is that you can now resume work on ‘guix refresh
> -l’.  ;-)

Will do.  Thanks for the help!

-- 
Eric Bavier

Please avoid sending me Word or PowerPoint attachments.
See http://www.gnu.org/philosophy/no-word-attachments.html

Re: [PATCH 4/4] gnu: matplotlib: Add gtk3 backends.

[Mark H Weaver]

ludo@gnu.org (Ludovic Courtès) writes:

> Federico Beffa <beffa@ieee.org> skribis:
>
>> The build does finish as I've tested it on my machine before
>> committing the package. However, for some reason, guix needs very long
>> to generate the derivation. On my machine (quad-core Xeon E5520)
>> python2-matplotlib takes ca. 31 mins to start building.
>> Python2-scipy, which includes python2-matplotlib, takes ca. 80 mins.!
>
> Eric Bavier <ericbavier@gmail.com> skribis:
>
>> I wonder if this could this be a result of package-with-python2's
>> behavior of recursively creating new package objects for each input in a
>> package.  As Ludovic suggested, this could easily create very large DAGs
>> that then need to be processed.
>>
>> This behavior of package-with-python2 was giving me a nightmare when
>> trying to code some improvements to `guix refresh -l`, since there would
>> easily become tens or hundreds of logically identical python package
>> objects floating through the package dependency DAG.
>
> Ouch, thanks for putting me on the right track.  Basically
> ‘package-with-explicit-python’ was recursing way too much, leading to
> hundreds of unique package objects, and thus defeating memoization of
> ‘package-derivation’.
>
> Commit 78a2745 fixes that.  ‘guix build python2-matplotlib -n
> --no-substitutes’ takes ~5 seconds on my laptop (which is still too much
> IMO, but that’s another story.)  I’ve thus reinstated numpy, scipy, etc.

This seems to have fixed Hydra as well.  Previously, it was timing out
(after 3 hours) while trying to evaluate jobsets.  Hydra is now building
the xorg-updates branch.

     Thanks!
       Mark