Re: Guix's python has pip's user dir in its loadpath

[Wojtek Kosior via "Development of GNU Guix and the GNU System distribution." <guix-devel@gnu.org>]

[-- Attachment #1: Type: text/plain, Size: 4605 bytes --]

> > But we'll be rebuilding the Python world anyway, so now is a chance to
> > try out some changes like that, though maybe it is a bit much with
> > what we are trying already. See <https://issues.guix.gnu.org/63139>  
> 
> It's a simple change, I guess we could try it at the same time, if
> someone volunteers to do it!

I just saw this message and hurried myself up to test the patch to
python-build-system that I made. Unfortunately, it turns out the
"PYTHONNOUSERSITE=1" env var breaks pip which tries to install wheels to
the system site directory and fails due to a read-only filesystem.

It seems we need to have this configurable on per-package basis, after
all. Should I try to make a patch which adds a build system argument
that controls this?

Also, the PYTHONNOUSERSITE variable only affects loading from the
actual site dir, not from virtualenvs (which rely on PYTHONPATH IIRC).
Should we try to add extra hardening to packages so that they are not
affected by virtualenvs either?

Best,
Wojtek

-- (sig_start)
website: https://koszko.org/koszko.html
fingerprint: E972 7060 E3C5 637C 8A4F  4B42 4BC5 221C 5A79 FD1A
follow me on Fediverse: https://friendica.me/profile/koszko/profile

♥ R29kIGlzIHRoZXJlIGFuZCBsb3ZlcyBtZQ== | ÷ c2luIHNlcGFyYXRlZCBtZSBmcm9tIEhpbQ==
✝ YnV0IEplc3VzIGRpZWQgdG8gc2F2ZSBtZQ== | ? U2hhbGwgSSBiZWNvbWUgSGlzIGZyaWVuZD8=
-- (sig_end)


On Thu, 06 Jul 2023 11:35:15 -0400 Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

> Hi John,
> 
> John Kehayias <john.kehayias@protonmail.com> writes:
> 
> > Hi,
> >
> > On Sat, Jul 01, 2023 at 11:57 AM, Maxim Cournoyer wrote:
> >  
> >> Hi,
> >>
> >> Wojtek Kosior <koszko@koszko.org> writes:
> >>  
> >>> The precedence of local, pip-installed Python libraries over Guix ones
> >>> has already been a source of bugs. And these can be hard to diagnose.  
> >>  
> >>> I imagine an optimal solution would be to configure this behavior on
> >>> per-package basis. The vast majority of applications does not need to
> >>> load local libraries. There are just a few exceptions like
> >>> `python-virtualenv`.
> >>>
> >>> Once I did write a package definition that deliberately disabled user
> >>> site dir package loading. I used code similar to what's below.
> >>>  
> >>>> (modify-phases %standard-phases
> >>>>   (add-after 'wrap 'prevent-local-package-interference
> >>>>     (lambda* (#:key outputs #:allow-other-keys)
> >>>>       (substitute* (string-append (assoc-ref outputs "out")
> >>>>                                   "/bin/<program-name>")
> >>>>         (("^#!/.*$" shabang)
> >>>>          (string-append shabang
> >>>>                         "export PYTHONNOUSERSITE=1\n"))))))  
> >>
> >> That is indeed a simple thing we could do to harden Python binaries from
> >> picking up user pip-installed dependencies potentially causing
> >> problems.  I would welcome such a patch.
> >>  
> >
> > Perhaps, but if this is expected (and known) upstream behavior, I'm
> > wary of deviating from these expectations. This general area does seem
> > tricky and no simple best answer I guess.  
> 
> While it's true that it's an intended upstream behavior,  I think in the
> context of Guix users packages to be self-contained or in some case be
> able to load Guix-installed plugins or extensions, but here it seems
> reasonable that a Guix-packaged Python binary prefers loading Python
> libraries from Guix rather that from the Python user site.
> 
> >>> Of course, it makes no sense to add such snippet to all definitions.
> >>> Instead, we could modify python-build-system to allow doing a similar
> >>> thing based on a flag passed in package's `(arguments)`.  
> >>
> >> I think it need not be made configurable but just applied
> >> indiscriminately to the wrap phase used in the python-build-system.  
> >
> > And this is part of the same question then, we should try to be
> > consistent, yes. I don't see a clear right path, but I haven't thought
> > much about this area. I think it comes down to a current
> > issue/limitation/quirk of Python from upstream and packaging for our
> > distro puts us in between what comes from them and how to take care of
> > our users.
> >
> > But we'll be rebuilding the Python world anyway, so now is a chance to
> > try out some changes like that, though maybe it is a bit much with
> > what we are trying already. See <https://issues.guix.gnu.org/63139>  
> 
> It's a simple change, I guess we could try it at the same time, if
> someone volunteers to do it!
> 

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 228 bytes --]