libxml2-python

libxml2-python

[Andreas Enge]

xorg-server requires mesa, and mesa requires python and libxml2, more 
exactly libxml2-python.

This is supposed to be set up the python way using
   python setup.py build install
which already does not work manually:
   failed to find headers for libxml2: update includes_dir
Supposedly, one needs to specify the path to libxml2.

Alternatively, one can run setup.py during the compilation of libxml2. 
Trying this by hand results in
   copying build/lib.linux-x86_64-2.7/libxml2mod.so -> 
/usr/local/lib/python2.7/dist-packages
   error: /usr/local/lib/python2.7/dist-packages/libxml2mod.so: Permission 
denied
Clearly, we do not want python modules to end up in /usr/local.

Do you see a solution?

Andreas

Re: libxml2-python

[Cyril Roelandt]

On 03/06/2013 11:43 PM, Andreas Enge wrote:
> xorg-server requires mesa, and mesa requires python and libxml2, more
> exactly libxml2-python.
>
> This is supposed to be set up the python way using
>     python setup.py build install
> which already does not work manually:
>     failed to find headers for libxml2: update includes_dir
> Supposedly, one needs to specify the path to libxml2.
>

 From setup.py:

# those are examined to find
# - libxml2/libxml/tree.h
# - iconv.h
# - libxslt/xsltconfig.h
includes_dir = [
"/usr/include",
"/usr/local/include",
"/opt/include",
os.path.join(ROOT,'include'),
HOME
];

You could patch setup.py to add the right path. Or you could run 
"configure" in libxml2 with "--prefix=<yourprefix>", then cd into the 
"python" directory, and ROOT will be set to <youprefix> in setup.py.

> Alternatively, one can run setup.py during the compilation of libxml2.
> Trying this by hand results in
>     copying build/lib.linux-x86_64-2.7/libxml2mod.so ->
> /usr/local/lib/python2.7/dist-packages
>     error: /usr/local/lib/python2.7/dist-packages/libxml2mod.so: Permission
> denied
> Clearly, we do not want python modules to end up in /usr/local.
>

Try:

$ python setup.py install --prefix=/path/to/foo/bar

You may have to adjust $PYTHONPATH to something like:

/path/to/foo/bar:$PYTHONPATH

for this to work.


Cyril.

Re: libxml2-python

[Andreas Enge]

[-- Attachment #1: Type: Text/Plain, Size: 1886 bytes --]

Am Donnerstag, 7. März 2013 schrieb Cyril Roelandt:
>  From setup.py:
> # those are examined to find
> # - libxml2/libxml/tree.h
> # - iconv.h
> # - libxslt/xsltconfig.h
> includes_dir = [
> "/usr/include",
> "/usr/local/include",
> "/opt/include",
> os.path.join(ROOT,'include'),
> HOME
> ];
> 
> You could patch setup.py to add the right path. Or you could run
> "configure" in libxml2 with "--prefix=<yourprefix>", then cd into the
> "python" directory, and ROOT will be set to <youprefix> in setup.py.

Thanks for your help! Indeed, when running "setup.py install" from the guix 
builder, libxml2 is found after installing it, so the "--prefix" is 
honoured. However, then iconv.h is searched for in the given list of paths, 
which does not include /nix/store/...glibc.../include, and the CPATH is not 
taken into account, so iconv.h is not found.

I tried adding glibc as an explicit input and to overwrite the path in 
setup.py. However, that does not use the glibc of the CPATH, but builds 
another one on top of it. Do I understand it correctly that I need to use 
glibc-final instead?

> Try:
> $ python setup.py install --prefix=/path/to/foo/bar
> You may have to adjust $PYTHONPATH to something like:
> /path/to/foo/bar:$PYTHONPATH
> for this to work.

Specifying no --prefix, it now tries to install a file as
   /nix/store/j8f28wavspyic8g9didl1ninaz48vdq0-
python-2.7.3/lib/python2.7/site-packages/libxml2mod.so
which fails since this is another package, not libxml2.
Giving as --prefix the output path, files are instead installed into
   /nix/store/svnw8fq87qrfrpcs7grx5nram558ahkp-
libxml2-2.9.0/lib/python2.7/site-packages/
which looks reasonable.

I am attaching the resulting patch. Is it reasonable to apply it to the 
libxml2 package, or would it be preferable to create a separate libxml2-
python package?

Andreas

[-- Attachment #2: libxml2.patch --]
[-- Type: text/x-patch, Size: 1501 bytes --]

diff --git a/gnu/packages/xml.scm b/gnu/packages/xml.scm
index b3c5f7d..0a47f35 100644
--- a/gnu/packages/xml.scm
+++ b/gnu/packages/xml.scm
@@ -19,6 +19,8 @@
 
 (define-module (gnu packages xml)
   #:use-module (gnu packages)
+  #:use-module (gnu packages base)
+  #:use-module (gnu packages compression)
   #:use-module (gnu packages gnupg)
   #:use-module (gnu packages perl)
   #:use-module (gnu packages python)
@@ -64,7 +66,22 @@ things the parser might find in the XML document (like start tags).")
     (home-page "http://www.xmlsoft.org/")
     (synopsis "libxml2, a C parser for XML")
     (inputs `(("perl" ,perl)
-              ("python" ,python)))
+              ("python" ,python)
+              ("zlib" ,zlib)
+              ("glibc" ,glibc-final)))
+    (arguments
+     `(#:phases
+        (alist-replace
+         'install
+         (lambda* (#:key inputs outputs #:allow-other-keys #:rest args)
+          (let ((install (assoc-ref %standard-phases 'install))
+                (glibc (assoc-ref inputs "glibc"))
+                (out (assoc-ref outputs "out")))
+            (apply install args)
+            (chdir "python")
+            (substitute* "setup.py" (("/opt/include") (string-append glibc "/include")))
+            (system* "python" "setup.py" "install" (string-append "--prefix=" out))))
+        %standard-phases)))
     (description
      "Libxml2 is the XML C parser and toolkit developed for the Gnome project
 (but it is usable outside of the Gnome platform).")

Re: libxml2-python

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> I tried adding glibc as an explicit input and to overwrite the path in 
> setup.py. However, that does not use the glibc of the CPATH, but builds 
> another one on top of it. Do I understand it correctly that I need to use 
> glibc-final instead?

Glibc is automatically added as an input, under the name “glibc” (see
build-system/gnu.scm).

So you can just do something like:

  (lambda* (#:key inputs #:allow-other-keys)
    (substitute* "setup.py"
      (("/usr/include")
       (string-append (assoc-ref inputs "glibc") "/include"))))

HTH,
Ludo’.

Re: libxml2-python

[Andreas Enge]

Am Freitag, 8. März 2013 schrieb Ludovic Courtès:
> Glibc is automatically added as an input, under the name “glibc” (see
> build-system/gnu.scm).
> 
> So you can just do something like:
> 
>   (lambda* (#:key inputs #:allow-other-keys)
>     (substitute* "setup.py"
>       (("/usr/include")
>        (string-append (assoc-ref inputs "glibc") "/include"))))

That is what I tried first, but it does not work:
    (arguments
     `(#:phases
        (alist-replace
         'install
         (lambda* (#:key inputs outputs #:allow-other-keys #:rest args)
          (let ((install (assoc-ref %standard-phases 'install))
                (glibc (assoc-ref inputs "glibc"))
                (out (assoc-ref outputs "out")))
            (apply install args)
            (chdir "python")
            (substitute* "setup.py" (("/opt/include") (string-append glibc 
"/include")))
            (system* "python" "setup.py" "install" (string-append "--
prefix=" out))))
        %standard-phases)))

yields
...
In unknown file:
   ?: 1 [string-append #f "/include"]
In ice-9/boot-9.scm:
 106: 0 [#<procedure 2347b40 at ice-9/boot-9.scm:97:6 (thrown-k . args)> 
wrong-type-arg ...]

Indeed, the key of the hash table is "libc" and not "glibc". I think it 
would be better to change it to "glibc" (which is what both of us thought 
naturally that it was already). I think this occurrence needs to be 
changed:
  ./gnu/packages/base.scm:1091:      ("libc" ,glibc-final))))
in %final-inputs, and probably other places to be coherent.

Andreas

Re: libxml2-python

[Ludovic Courtès]

Hello!

Andreas Enge <andreas@enge.fr> skribis:

> Indeed, the key of the hash table is "libc" and not "glibc".

Ah, right, sorry.

> I think it would be better to change it to "glibc" (which is what both
> of us thought naturally that it was already). I think this occurrence
> needs to be changed: ./gnu/packages/base.scm:1091: ("libc"
> ,glibc-final)))) in %final-inputs, and probably other places to be
> coherent.

Well, I’d rather keep it as is to avoid needless churn, and because the
‘g’ is really implicit anyway.

Ludo’.

Re: libxml2-python

[Andreas Enge]

Am Donnerstag, 14. März 2013 schrieb Ludovic Courtès:
> Andreas Enge <andreas@enge.fr> skribis:
> > I think it would be better to change it to "glibc" (which is what both
> > of us thought naturally that it was already). I think this occurrence
> > needs to be changed: ./gnu/packages/base.scm:1091: ("libc"
> > ,glibc-final)))) in %final-inputs, and probably other places to be
> > coherent.
> Well, I’d rather keep it as is to avoid needless churn, and because the
> ‘g’ is really implicit anyway.

It is not very intuitive, since all the packages are called "glibc*", and 
we usually use the package name as key in the hash tables. So one should be 
coherent in making the "g" explicit or not. In any case, this is not a big 
problem.

Andreas

Re: libxml2-python

[Cyril Roelandt]

On 03/07/2013 11:38 PM, Andreas Enge wrote:
> Am Donnerstag, 7. März 2013 schrieb Cyril Roelandt:
>>    From setup.py:
>> # those are examined to find
>> # - libxml2/libxml/tree.h
>> # - iconv.h
>> # - libxslt/xsltconfig.h
>> includes_dir = [
>> "/usr/include",
>> "/usr/local/include",
>> "/opt/include",
>> os.path.join(ROOT,'include'),
>> HOME
>> ];
>>
>> You could patch setup.py to add the right path. Or you could run
>> "configure" in libxml2 with "--prefix=<yourprefix>", then cd into the
>> "python" directory, and ROOT will be set to<youprefix>  in setup.py.
>
> Thanks for your help! Indeed, when running "setup.py install" from the guix
> builder, libxml2 is found after installing it, so the "--prefix" is
> honoured. However, then iconv.h is searched for in the given list of paths,
> which does not include /nix/store/...glibc.../include, and the CPATH is not
> taken into account, so iconv.h is not found.
>
> I tried adding glibc as an explicit input and to overwrite the path in
> setup.py. However, that does not use the glibc of the CPATH, but builds
> another one on top of it. Do I understand it correctly that I need to use
> glibc-final instead?
>
>> Try:
>> $ python setup.py install --prefix=/path/to/foo/bar
>> You may have to adjust $PYTHONPATH to something like:
>> /path/to/foo/bar:$PYTHONPATH
>> for this to work.
>
> Specifying no --prefix, it now tries to install a file as
>     /nix/store/j8f28wavspyic8g9didl1ninaz48vdq0-
> python-2.7.3/lib/python2.7/site-packages/libxml2mod.so
> which fails since this is another package, not libxml2.
> Giving as --prefix the output path, files are instead installed into
>     /nix/store/svnw8fq87qrfrpcs7grx5nram558ahkp-
> libxml2-2.9.0/lib/python2.7/site-packages/
> which looks reasonable.
>
> I am attaching the resulting patch. Is it reasonable to apply it to the
> libxml2 package, or would it be preferable to create a separate libxml2-
> python package?
>

I'm not sure it is a good idea to automatically install bindings along 
with the library, so I'd rather have a package per binding.

On an unrelated note, the PYTHONPATH envrionment variable must be 
adjusted in order to use the library, since it is not installed in a 
"standard" location, but I guess there is not much we can do about it.

Cyril.

Re: libxml2-python

[Andreas Enge]

Am Sonntag, 10. März 2013 schrieb Cyril Roelandt:
> On 03/07/2013 11:38 PM, Andreas Enge wrote:
> > I am attaching the resulting patch. Is it reasonable to apply it to
> > the libxml2 package, or would it be preferable to create a separate
> > libxml2- python package?
> I'm not sure it is a good idea to automatically install bindings along
> with the library, so I'd rather have a package per binding.

I would rather say "one tarball, one package". But this case is borderline: 
The python bindings are not installed automatically, but need an additonal 
"python setup.py" step in a subdirectory, which even compiles a shared 
library. And they can be downloaded in a separate tarball.

Are there any other opinions? So far, I modified the existing package to 
also install the bindings, but they could easily be split out.

> On an unrelated note, the PYTHONPATH envrionment variable must be
> adjusted in order to use the library, since it is not installed in a
> "standard" location, but I guess there is not much we can do about it.

Yes, I needed to add
   (setenv "PYTHONPATH" (string-append libxml2 "/lib/python2.7/site-
packages"))

as well as in some other place
   (setenv "PERL5LIB" (string-append perl-xml-parser 
"/lib/perl5/site_perl"))

This had better be handled centrally. One could do it in the same way as 
CPATH currently, but Ludovic had some ideas how to generalise it even 
further. I added a "FIXME" comment in the build recipe.

Andreas

Re: libxml2-python

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> Yes, I needed to add
>    (setenv "PYTHONPATH" (string-append libxml2 "/lib/python2.7/site-
> packages"))
>
> as well as in some other place
>    (setenv "PERL5LIB" (string-append perl-xml-parser 
> "/lib/perl5/site_perl"))
>
> This had better be handled centrally. One could do it in the same way as 
> CPATH currently, but Ludovic had some ideas how to generalise it even 
> further.

Yes, this is the intended purpose of the currently-unused ‘search-paths’
field of packages.

Ludo’.