Python applications that are also libraries

Python applications that are also libraries

[Ricardo Wurmus]

Hi Guix,

we have a bunch of packages that are used both as applications and as
Python libraries.  An example is “deeptools”.

As a library we need to propagate other Python inputs; as an application
this is not necessary because we have wrappers.

I wonder how to deal with this.  Should we assume that these packages
are used as libraries and default to propagating all Python inputs?  Or
should we have package variants (or outputs?) that propagate inputs as a
side-effect?

--
Ricardo

Re: Python applications that are also libraries

[Hartmut Goebel]

Am 04.04.2018 um 11:36 schrieb Ricardo Wurmus:
> I wonder how to deal with this.  Should we assume that these packages
> are used as libraries and default to propagating all Python inputs?  Or
> should we have package variants (or outputs?) that propagate inputs as a
> side-effect?

If this is a "pure" application, I'd install it with*out* propagated
inputs. This might not be easy to determine, though.

https://lists.gnu.org/archive/html/guix-devel/2018-02/msg00456.html


-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Python applications that are also libraries

[Ricardo Wurmus]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Am 04.04.2018 um 11:36 schrieb Ricardo Wurmus:
>> I wonder how to deal with this.  Should we assume that these packages
>> are used as libraries and default to propagating all Python inputs?  Or
>> should we have package variants (or outputs?) that propagate inputs as a
>> side-effect?
>
> If this is a "pure" application, I'd install it with*out* propagated
> inputs. This might not be easy to determine, though.

It is both.  It is often used just as an application, but the procedures
that make up the application are just as often used in an interactive
Python session or as a library.

> https://lists.gnu.org/archive/html/guix-devel/2018-02/msg00456.html

This is about wrapping (or not) using virtual envs.  I don’t really see
how this relates to this problem, but maybe I’m missing something
obvious.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Python applications that are also libraries

[Hartmut Goebel]

[-- Attachment #1: Type: text/plain, Size: 1478 bytes --]

Am 04.04.2018 um 22:13 schrieb Ricardo Wurmus:
>> If this is a "pure" application, I'd install it with*out* propagated
>> inputs. This might not be easy to determine, though.
> It is both.  It is often used just as an application, but the procedures
> that make up the application are just as often used in an interactive
> Python session or as a library.

So I'm afraid, we need to install it with propagated inputs.

Stimulated by your question I rethought whether we might come around
propagated inputs, and I did not find a solution. There must only be one
version of a library in each profile, otherwise we'd get conflicts. We
could provide our own implementation of site.py (or site-customize.py)
to avoid *propagating*, but this would not avoid the *conflicts* but
only hide the cause of the conflicts and make them hard to find (as we
already discusses a year or two ago).


>> https://lists.gnu.org/archive/html/guix-devel/2018-02/msg00456.html
> This is about wrapping (or not) using virtual envs.  I don’t really see
> how this relates to this problem, but maybe I’m missing something
> obvious.

Sorry for the confusion, this link shouldn't have been there. I had
pasted it in since I thought it is related and I'm going to refer to is,
but it is not.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |


[-- Attachment #2: Type: text/html, Size: 2558 bytes --]

Re: Python applications that are also libraries

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 451 bytes --]

Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:

> we have a bunch of packages that are used both as applications and as
> Python libraries.  An example is “deeptools”.

I took a brief peek at deeptools.  It looks like there are programs in
bin, and libraries in lib.  Why can't we just split them into two
outputs?  For example, put the libraries into the default "out" output
and the programs into the "bin" output.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Python applications that are also libraries

[Hartmut Goebel]

Am 06.04.2018 um 11:12 schrieb Chris Marusich:
> Why can't we just split them into two
> outputs?  For example, put the libraries into the default "out" output
> and the programs into the "bin" output.

For consistence, we should then do this for all other python packages
including a script, e.g. sphinx and a lot others.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Python applications that are also libraries

[Ricardo Wurmus]

Chris Marusich <cmmarusich@gmail.com> writes:

> Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de> writes:
>
>> we have a bunch of packages that are used both as applications and as
>> Python libraries.  An example is “deeptools”.
>
> I took a brief peek at deeptools.  It looks like there are programs in
> bin, and libraries in lib.  Why can't we just split them into two
> outputs?  For example, put the libraries into the default "out" output
> and the programs into the "bin" output.

The programs in “bin” use the libraries in “lib”.  The binaries aren’t
big, so separating them from the lib output provides hardly any benefit.

The point here is that in the library case inputs must be propagated,
whereas in the case where only the executables are used the inputs don’t
have to be propagated.

Separating them in two outputs doesn’t address this problem as far as I
can see.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Python applications that are also libraries

[Hartmut Goebel]

Am 06.04.2018 um 14:12 schrieb Ricardo Wurmus:
> The point here is that in the library case inputs must be propagated,
> whereas in the case where only the executables are used the inputs don’t
> have to be propagated.
>
> Separating them in two outputs doesn’t address this problem as far as I
> can see.
Using the venv-based wrapper I describe in
<https://lists.gnu.org/archive/html/guix-devel/2018-02/msg00456.html
could avoid propagating for the scripts - but not for the libs.

IMHO it does not make much sense having "script" packages without
propagation, as we still need "libs" packages with propagation. Even if
"deeptools:script" would not propagate its inputs, "somepackage:lib"
(required by "deeptools:script") would still need to propagate its
inputs. Thus nothing gained.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |