bug#52269: [PATCH core-updates-frozen] sitecustomize does not honor .pth files

[Maxim Cournoyer <maxim.cournoyer@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1834 bytes --]

tags 52269 patch
thanks

Hi!

The following patch fixes it.  I used site.addsitedir but ensured the
correct ordering of sys.path (we need to make the Guix-installed
packages appear before Python's own site-packages directory otherwise we
wouldn't be able to override its bundled packages such as 'pip').

Here's how I tested:

Copy the sitecustomize.py file to the current directory, then:

--8<---------------cut here---------------start------------->8---
$ pip --version
pip 21.1.3 from $HOME/.guix-profile/lib/python3.9/site-packages/pip (python 3.9)

$ guix show python-pip | recsel -p version
version: 20.2.4
--8<---------------cut here---------------end--------------->8---

Ensure installed pip still overrides Python's own.  PYTHONPATH=. forces
the sitecustomize.py file in the CWD to take precedence over the one
currently installed along Python.

--8<---------------cut here---------------start------------->8---
$ guix shell --pure python python-pip python-pdbpp
[env]$ PYTHONPATH=. python3 -c 'import pip; print(pip.__version__)'
20.2.4
--8<---------------cut here---------------end--------------->8---

Next I created a dummy script to trigger run pdb:

#file: test.py
print('hello')
import pdb; pdb.set_trace()

--8<---------------cut here---------------start------------->8---
$ guix shell --pure python python-pip python-pdbpp
[env]$ python3 test.py
hello
> /tmp/toto/test.py(7)<module>()
-> exit(1)
(Pdb)
--8<---------------cut here---------------end--------------->8---

This is the current bug; this is the regular Pdb, not Pdbpp.  Let's
force our revised sitecustomize.py file:

--8<---------------cut here---------------start------------->8---
$ PYTHONPATH=. python3 test.py
hello
[0] > /tmp/toto/test.py(7)<module>()
-> exit(1)
(Pdb++)
--8<---------------cut here---------------end--------------->8---

Better!


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-sitecustomize.py-Honor-.pth-files.patch --]
[-- Type: text/x-patch, Size: 2208 bytes --]

From 762357609270ab016236d22999ae5cfc3fe4ff28 Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Date: Fri, 3 Dec 2021 22:36:26 -0500
Subject: [PATCH] sitecustomize.py: Honor .pth files.

Fixes <https://issues.guix.gnu.org/52269>.

* gnu/packages/aux-files/python/sitecustomize.py: Use site.addsitedirs to add
the site directories; this takes care of the .pth files.  Make sure the added
items still appear before Python's own 'site-packages' directory.
---
 .../aux-files/python/sitecustomize.py         | 24 ++++++++++++++-----
 1 file changed, 18 insertions(+), 6 deletions(-)

diff --git a/gnu/packages/aux-files/python/sitecustomize.py b/gnu/packages/aux-files/python/sitecustomize.py
index 71e328b9ac..bdaaa8e9e2 100644
--- a/gnu/packages/aux-files/python/sitecustomize.py
+++ b/gnu/packages/aux-files/python/sitecustomize.py
@@ -18,6 +18,7 @@
 # along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
 
 import os
+import site
 import sys
 
 # Commentary:
@@ -47,9 +48,20 @@ all_sites_norm = [os.path.normpath(p) for p in all_sites_raw]
 matching_sites = [p for p in all_sites_norm
                   if p.endswith(site_packages_prefix)]
 
-# Insert sites matching the current version into sys.path, right before
-# Python's own site.  This way, the user can override the libraries provided
-# by Python itself.
-sys_path_absolute = [os.path.realpath(p) for p in sys.path]
-index = sys_path_absolute.index(python_site)
-sys.path[index:index] = matching_sites
+if not matching_sites:
+    exit(0)
+
+# Deduplicate the entries, append them to sys.path, and handle any .pth files
+# they contain.
+for s in matching_sites:
+    site.addsitedir(s)
+
+# Move the entries that were appended to sys.path in front of Python's own
+# site-packages directory.  This enables Guix packages to override Python's
+# bundled packages, such as 'pip'.
+python_site_index = sys.path.index(python_site)
+new_site_start_index = sys.path.index(matching_sites[0])
+if python_site_index < new_site_start_index:
+    sys.path = (sys.path[:python_site_index]
+                + sys.path[new_site_start_index:]
+                + sys.path[python_site_index:new_site_start_index])
-- 
2.34.0