From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
To: Lars-Dominik Braun <lars@6xq.net>
Cc: 52269@debbugs.gnu.org
Subject: bug#52269: [core-updates-frozen] sitecustomize.py does not honor .pth files
Date: Mon, 13 Dec 2021 14:04:49 -0500 [thread overview]
Message-ID: <87ee6gcomm.fsf_-_@gmail.com> (raw)
In-Reply-To: <Ya3NEAZXT5U0bPSH@noor.fritz.box> (Lars-Dominik Braun's message of "Mon, 6 Dec 2021 09:42:56 +0100")
[-- Attachment #1: Type: text/plain, Size: 3518 bytes --]
Hello,
Lars-Dominik Braun <lars@6xq.net> writes:
> Hi Maxim,
>
>> +if not matching_sites:
>> + exit(0)
> are you sure about using `exit()` here? sitecustomize.py is imported
> during startup and this would simply quit the Python interpreter if
> GUIX_PYTHONPATH is not set, wouldn’t it? (Can’t test the change
> unfortunately, because it’s a massive rebuild.)
You can test it by placing the new sitecustomize.py file in the current
directory, and then:
$ guix shell python-wrapper python-pdbpp
[env]$ $ PYTHONPATH=. GUIX_PYTHONPATH= python sample.py
where sample.py contains something like:
--8<---------------cut here---------------start------------->8---
__import__("pdb").set_trace()
print('hello')
--8<---------------cut here---------------end--------------->8---
Indeed, when GUIX_PYTHONPATH is unset or matching_sites is empty, it
exit with 0 as you expected:
--8<---------------cut here---------------start------------->8---
$ PYTHONPATH=. GUIX_PYTHONPATH= python sample.py
Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
File "/gnu/store/p5fgysbcnnp8b1d91mrvjvababmczga0-python-3.9.6/lib/python3.9/site.py", line 589, in <module>
main()
File "/gnu/store/p5fgysbcnnp8b1d91mrvjvababmczga0-python-3.9.6/lib/python3.9/site.py", line 582, in main
execsitecustomize()
File "/gnu/store/p5fgysbcnnp8b1d91mrvjvababmczga0-python-3.9.6/lib/python3.9/site.py", line 521, in execsitecustomize
import sitecustomize
File "/home/maxim/proj/kinova/kts_robot/sitecustomize.py", line 52, in <module>
exit(0)
File "/gnu/store/p5fgysbcnnp8b1d91mrvjvababmczga0-python-3.9.6/lib/python3.9/_sitebuiltins.py", line 26, in __call__
raise SystemExit(code)
SystemExit: 0
--8<---------------cut here---------------end--------------->8---
After the proposed change:
--8<---------------cut here---------------start------------->8---
[env]$ PYTHONPATH=. GUIX_PYTHONPATH= python sample.py
> /home/maxim/proj/kinova/kts_robot/sample.py(5)<module>()
-> print('hello')
--8<---------------cut here---------------end--------------->8---
There's no longer pdbpp because of clearing GUIX_PYTHONPATH but at least
it doesn't crash :-).
>> +# Move the entries that were appended to sys.path in front of Python's own
>> +# site-packages directory. This enables Guix packages to override Python's
>> +# bundled packages, such as 'pip'.
>> +python_site_index = sys.path.index(python_site)
>> +new_site_start_index = sys.path.index(matching_sites[0])
>> +if python_site_index < new_site_start_index:
>> + sys.path = (sys.path[:python_site_index]
>> + + sys.path[new_site_start_index:]
>> + + sys.path[python_site_index:new_site_start_index])
> This is unrelated to the pdb issue, right? I see that it’s necessary
> right now, but as suggested in #46848 I’d prefer unbundling
> setuptools/pip from python. (I’ll send a v3 of the patchset at some
> point.)
Previously the Guix-provided paths were directly spliced at the right
location; now using 'site.addsitedir' simply appends them, which
requires manual fiddling afterward.
I agree that after it's un-bundled it shouldn't be necessary anymore, but
let's keep this change for core-updates along work on the 517
python-build-system (I'll try having a look to it after the next release
it out -- ping me otherwise).
Thank you,
Maxim
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-sitecustomize.py-Honor-.pth-files.patch --]
[-- Type: text/x-patch, Size: 2241 bytes --]
From 49f0d2a493b868b9414ea10c7a676cf8404e1bca Mon Sep 17 00:00:00 2001
From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Date: Fri, 3 Dec 2021 22:36:26 -0500
Subject: [PATCH] sitecustomize.py: Honor .pth files.
Fixes <https://issues.guix.gnu.org/52269>.
* gnu/packages/aux-files/python/sitecustomize.py: Use site.addsitedirs to add
the site directories; this takes care of the .pth files. Make sure the added
items still appear before Python's own 'site-packages' directory.
---
.../aux-files/python/sitecustomize.py | 22 ++++++++++++++-----
1 file changed, 16 insertions(+), 6 deletions(-)
diff --git a/gnu/packages/aux-files/python/sitecustomize.py b/gnu/packages/aux-files/python/sitecustomize.py
index 71e328b9ac..e2348e0356 100644
--- a/gnu/packages/aux-files/python/sitecustomize.py
+++ b/gnu/packages/aux-files/python/sitecustomize.py
@@ -18,6 +18,7 @@
# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
import os
+import site
import sys
# Commentary:
@@ -47,9 +48,18 @@ all_sites_norm = [os.path.normpath(p) for p in all_sites_raw]
matching_sites = [p for p in all_sites_norm
if p.endswith(site_packages_prefix)]
-# Insert sites matching the current version into sys.path, right before
-# Python's own site. This way, the user can override the libraries provided
-# by Python itself.
-sys_path_absolute = [os.path.realpath(p) for p in sys.path]
-index = sys_path_absolute.index(python_site)
-sys.path[index:index] = matching_sites
+if matching_sites:
+ # Deduplicate the entries, append them to sys.path, and handle any
+ # .pth files they contain.
+ for s in matching_sites:
+ site.addsitedir(s)
+
+ # Move the entries that were appended to sys.path in front of
+ # Python's own site-packages directory. This enables Guix
+ # packages to override Python's bundled packages, such as 'pip'.
+ python_site_index = sys.path.index(python_site)
+ new_site_start_index = sys.path.index(matching_sites[0])
+ if python_site_index < new_site_start_index:
+ sys.path = (sys.path[:python_site_index]
+ + sys.path[new_site_start_index:]
+ + sys.path[python_site_index:new_site_start_index])
--
2.34.0
next prev parent reply other threads:[~2021-12-13 19:06 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-12-04 2:59 bug#52269: [core-updates-frozen] Some Python packages relying on .pth are broken Maxim Cournoyer
2021-12-04 5:36 ` bug#52269: [PATCH core-updates-frozen] sitecustomize does not honor .pth files Maxim Cournoyer
2021-12-13 10:12 ` bug#52269: [core-updates-frozen] sitecustomize.py " Ludovic Courtès
2021-12-13 14:10 ` Maxim Cournoyer
2021-12-06 8:42 ` bug#52269: [PATCH] sitecustomize.py: Honor " Lars-Dominik Braun
2021-12-13 19:04 ` Maxim Cournoyer [this message]
2021-12-16 9:48 ` bug#52269: [core-updates-frozen] sitecustomize.py does not honor " Lars-Dominik Braun
2021-12-17 14:41 ` Maxim Cournoyer
2021-12-17 15:02 ` Maxim Cournoyer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87ee6gcomm.fsf_-_@gmail.com \
--to=maxim.cournoyer@gmail.com \
--cc=52269@debbugs.gnu.org \
--cc=lars@6xq.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.