Re: proper Python inputs (was: gnu: Add python-pytest-pep8.)

[Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>]

Hartmut Goebel <h.goebel@crazy-compilers.com> writes:

> Am 25.04.2016 um 20:19 schrieb Leo Famulari:
>> This seems like an unusual case, since the package in question appears
>> to be a build-time tool. So, it would make sense to have things like
>> python-pytest available during python-pytest-pep8's run-time.
>
>> If we have a use case for python-pytest-pep8, it could be worthwhile
>> to see if it will work without propagating these anyways; I've noticed
>> some Python packages have a way to link to non-propagated-inputs. 
>
> Reading this makes me *totally* confused about how to package Python for
> Guix. I think, we should enhance the Guix Python Packaging guide to
> clarify thinks for all.
>
> 1. Fact: If some Python package lists another package in
> "install_requires", then this other Python package is required to run.
> In a non Guix-environment, pip would install these other package, too.
>
> 2. Until 5 minutes ago I thought, that adding these other packages as
> "normal" inputs is enough. But then I found that quite a bunch of
> packages are using "propagated-inputs". So I tried to find out, what the
> difference is:
>
>   - "normal" inputs get added to PYTHONPATH via the .pth mechanism.
>
>      Example: guix package -i python-execnet puts only
> "execnet-1.4.1-py3.4.egg" into site-packages, plus
> "python-execnet-1.4.1.pth", which adds the required package via a link
> to "/gnu/store/....python-apipkg-1.4/..."
>
>  - "Propagated" inputs are all linked into site-packages
>
>     Example: guix package -i python-zope-interface puts zope-interface
> and zope-event into site-packages.
>
>  So this is not much of a difference: In both cases all required inputs
> will be available in PYTHONPATH at run-time and can be imported.
>
> 3. But anyhow: Trouble is ahead: If some package aaa has a normal input
> "zzz@1.2", while package bbb as normal input "zzz@1.1" and package ccc
> as propagated input "zzz@1.3", we will end up with this in site-packages:
>
>    - aaa.pth adding path to zzz@1.2
>    - bbb.pth adding path to zzz@1.1
>    - zzz-1.3.egg
>
>   Which package "zzz" gets imported is somewhat arbitrary [2].
>
> Is this intended? Did I miss something?

It’s not so easy as “pth” files aren’t necessarily evaluated.  Usually
we propagate inputs that need to be available at runtime.

This often causes problems here at work when people installed
“python2-numpy” a couple of months ago and then install “miso” which
propagates “python2-numpy”.  There are then at least two variants of
numpy in the profile leading to many collisions.

We’ve previously discussed alternatives to propagation:

    http://lists.gnu.org/archive/html/guix-devel/2016-02/msg00810.html

But so far there haven’t been any patch-ready solutions.  I’d really
like to fix this, though, as it’s uncomfortable to have to tell users to
either use manifests (thereby upgrading all of their applications in
their profile before installing something new) or to use separate
profiles for Python stuff.

~~ Ricardo