Re: PYTHONPATH woes

[Ricardo Wurmus <rekado@elephly.net>]

Hi Pjotr,

> On Tue, Feb 20, 2018 at 11:53:54AM +0100, Ricardo Wurmus wrote:
>> Would it be good to make the wrappers for Python scripts stricter and
>> not accept any user-set PYTHONPATH?
>
> I think that is a bad idea. You need to be able to opt out. Also
> people need to experiment with modules without understanding Guix per
> se. In my upcomping blog I would emphasize packaging at the point you
> become a serious user.
>
> That should come with a health warning ;). Similarly we should allow
> for LD_LIBRARY_PATH etc. It is what they exist for, even if it is
> dangerous.

While I agree that it must be *possible* to opt out, I think the
defaults are wrong here.  We don’t ever set LD_LIBRARY_PATH when
building a profile, but we do set PYTHONPATH.  Having PYTHONPATH set
(even without the knowledge of the user) leads to problems here when it
shouldn’t.

I’d argue that the number of users who need to be able to override
individual Python packages for a tool like Ribodiff is way lower than
the number of those who accidentally get into a situation where
PYTHONPATH is set purely because of the contents of their profile,
leading to breakage of unrelated packages that just happen to use Python
modules.

>> How do we approach the problem of having both Python 2 modules and
>> Python 3 modules in the same profile?  PYTHONPATH will be set to refer
>> to the site-packages directories of both versions, which is never good.
>> Does Python offer us a way to do better?  Can we make use of pth files
>> to get around this problem somehow?
>
> Python should have created PYTHONPATH2 to split them out. We could
> patch python2 to do just that.

I think there’s an alternative, but I don’t know it well.  The official
way is to use “.pth” files instead of setting PYTHONPATH.  Maybe there’s
a way that doesn’t involve setting PYTHONPATH.  Instead we could nudge
Python towards reading the profile’s “.pth” file and read the package
locations from there.

This would require a new profile hook and possibly a patch to Python to
add a way to tell it to read the “.pth” file from a location provided by
a Guix environment variable.

> Even so, the real solution is separate profiles. I get that with
> versions of Ruby too.

Right, but it is not obvious what packages must be treated with extra
care.  When I install the Ribodiff package I don’t know or care that
it’s written in Python.  That shouldn’t matter at all.  But now I
actually have to pay attention to this and install Ribodiff in a profile
that doesn’t contain Python 3 things.

That’s rather complicated and unfriendly for users.  I don’t want the
users here to be anxious about installing software, just because a new
tool may be using Python and thus might break when installed to a
profile containing Python things.  We made these wrappers precisely to
isolate the tools from the current environment.  Allowing them to be
disturbed so easily is making wrappers much less useful.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net