From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp1 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 4Ay+BLOZt2HYiAAAgWs5BA (envelope-from ) for ; Mon, 13 Dec 2021 20:06:27 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp1 with LMTPS id sIRwALOZt2GCRAAAbx9fmQ (envelope-from ) for ; Mon, 13 Dec 2021 19:06:27 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id AC7204C80 for ; Mon, 13 Dec 2021 20:06:26 +0100 (CET) Received: from localhost ([::1]:42394 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mwqeX-0001ug-Dh for larch@yhetil.org; Mon, 13 Dec 2021 14:06:25 -0500 Received: from eggs.gnu.org ([209.51.188.92]:60154) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mwqeA-0001tZ-Fw for bug-guix@gnu.org; Mon, 13 Dec 2021 14:06:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:45275) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mwqeA-0003Qd-75 for bug-guix@gnu.org; Mon, 13 Dec 2021 14:06:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mwqe9-0002Ax-O6 for bug-guix@gnu.org; Mon, 13 Dec 2021 14:06:01 -0500 X-Loop: help-debbugs@gnu.org Subject: bug#52269: [core-updates-frozen] sitecustomize.py does not honor .pth files Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Mon, 13 Dec 2021 19:06:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 52269 X-GNU-PR-Package: guix X-GNU-PR-Keywords: patch To: Lars-Dominik Braun Received: via spool by 52269-submit@debbugs.gnu.org id=B52269.16394223048289 (code B ref 52269); Mon, 13 Dec 2021 19:06:01 +0000 Received: (at 52269) by debbugs.gnu.org; 13 Dec 2021 19:05:04 +0000 Received: from localhost ([127.0.0.1]:56821 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwqd9-00029E-Q0 for submit@debbugs.gnu.org; Mon, 13 Dec 2021 14:05:04 -0500 Received: from mail-qt1-f172.google.com ([209.85.160.172]:44721) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mwqd6-00028y-Ss for 52269@debbugs.gnu.org; Mon, 13 Dec 2021 14:04:58 -0500 Received: by mail-qt1-f172.google.com with SMTP id a2so16174502qtx.11 for <52269@debbugs.gnu.org>; Mon, 13 Dec 2021 11:04:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=BkP3HtSfRPRcr9kVkNx14s6LxaiftGVYYsrHZLa98fw=; b=XxeQl3q08xMAqpjAik27wbEH5KHHvA6s4f0eLo3Efn6PtcfiS94P6D8dms/H3DIPtl he/aGjvNPIxBw6Gt+sKFLnp7N8evi1oupzatUothzZnuW27MTcQpPRlKaygy2OVJ8K3l APtVyqQtNBQxuCn+KEEseSFtaGnIBYKirgzkRdnxWoSvhbeuER7Lg62D0wR9n4C9BJOn 7xnb/RPcvPeMdQangRzCOLadU6PdO8y9S47v56FOAvKLU5Wu4ZZwCYvYjexLPY6IWzez RFdfCLTkbpw7LVCwUgDXym9ORmo6h5GvV+MvYzEn8wdNB6K/VMCxA+9FTvF/WeyAT3ho Ct/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=BkP3HtSfRPRcr9kVkNx14s6LxaiftGVYYsrHZLa98fw=; b=cpiz9XfuHSIAi0m1DdlHZWRjCaMgB3fvLJIKw/IvPIpSWYRkb+6yKXBM6zCX995lhs 5J1yLSuAB0cP2FG4LwsrkWLAR21UWYe8Odqd9bS5CbU4TcARhBdKjh5mFuPGpqqjokUR jeQHiqyuwUtWym0/OLbn8JXt64frcK9Npc28b+hrw+mlfiT3ivoOYXR6KfMJHzlpMepg 0lnJ/Y90Pp54VeqwFNKCPmajKsAIOO88geFTK5UIJE2FABvdBlXXXI5wEpnoE+Dylaj1 +E/1AHFt2fftSz1OWQHY8JRi7Jj6oJXPGD+J6z9lhCMSw8nrYCeB65sokiujdDwjTPGr 5mtA== X-Gm-Message-State: AOAM532VFNYl0JyrtEHLgf7LVXkJA2DXJ9q0xw+xBNLUKba5D+hhy6X8 IWJcCluqP//TFwIJeJ9k9PDFk/D8ZBc= X-Google-Smtp-Source: ABdhPJxJ7k0OUmhny+UM75FSVWoHaoukZYSrV/i5njqh97uNMNCVw0xT10JKbRIv9QqKyS+AyN9W0g== X-Received: by 2002:ac8:57ce:: with SMTP id w14mr261142qta.252.1639422291055; Mon, 13 Dec 2021 11:04:51 -0800 (PST) Received: from hurd (dsl-10-141-188.b2b2c.ca. [72.10.141.188]) by smtp.gmail.com with ESMTPSA id f11sm6428957qko.84.2021.12.13.11.04.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Dec 2021 11:04:50 -0800 (PST) From: Maxim Cournoyer References: <871r2taxcm.fsf@gmail.com> Date: Mon, 13 Dec 2021 14:04:49 -0500 In-Reply-To: (Lars-Dominik Braun's message of "Mon, 6 Dec 2021 09:42:56 +0100") Message-ID: <87ee6gcomm.fsf_-_@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 52269@debbugs.gnu.org Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1639422386; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=BkP3HtSfRPRcr9kVkNx14s6LxaiftGVYYsrHZLa98fw=; b=GjWO42giqftHWBWuCUDit+8dYakWbP1MQsxzWOLiIJPYNXLfjlImZfgO/U90QbULDi0gU4 MPnu10yFQhhs7TDZ92h1N5OXfatOeWx9mWcKQTrknw3PolCueHU614YleVrhv1giXewv/Y ZdplcrhSZ4MI2vNeGADzeYs1l597JEDvtsJrlW+flmJcGXNfZKDjG58288LfGpvdF4GSXB UbPQ+4NYB0il4jLjye6CS8vRuWn+WkMbDF4Mp0DGJCy5Sy9YRNW5S2o9UL93kxpi0F7unz x9jEcnqQVFyCjTmKEWlrqj+u5jqOMRUCDf9LFjHWz5pPmV6w1P52w4ko9Q8sdA== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1639422386; a=rsa-sha256; cv=none; b=GvKohbkEv8zHFxqgzhLvxyR4JGziujOiH+oBs31lq4GFXoBwWE/Fe+SZiDJPBQwwY/Fqfh 5dTuF0L+OJg60c5FlCBzUla39UYRV/o/jadxD5hjyKMqq64HxgV9yno2s2SHGdYw4iLN1C Bt0Zvy7qqx/a5HBlzeCyC9JJbnaASDMoMG9vFipH0dLLY5ZxOqJ+Hx/kOfJ2O2Uv9tsZqC ONWQ7fRduXtlblK2hq11t8auudOJZ/+omsNojSUUb5X3aPP4mCmg/UBfPUs1+pP6zTF5kM wgTlgZTkdhSjnz0+5ms6Ctul3/DDxaoSfUbPy/SrIbrT9RN4GCTRquDludTAgA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=XxeQl3q0; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -1.87 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=XxeQl3q0; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: AC7204C80 X-Spam-Score: -1.87 X-Migadu-Scanner: scn1.migadu.com X-TUID: GiptT7TI+10G --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello, Lars-Dominik Braun writes: > Hi Maxim, > >> +if not matching_sites: >> + exit(0) > are you sure about using `exit()` here? sitecustomize.py is imported > during startup and this would simply quit the Python interpreter if > GUIX_PYTHONPATH is not set, wouldn=E2=80=99t it? (Can=E2=80=99t test the = change > unfortunately, because it=E2=80=99s a massive rebuild.) You can test it by placing the new sitecustomize.py file in the current directory, and then: $ guix shell python-wrapper python-pdbpp [env]$ $ PYTHONPATH=3D. GUIX_PYTHONPATH=3D python sample.py where sample.py contains something like: --8<---------------cut here---------------start------------->8--- __import__("pdb").set_trace() print('hello') --8<---------------cut here---------------end--------------->8--- Indeed, when GUIX_PYTHONPATH is unset or matching_sites is empty, it exit with 0 as you expected: --8<---------------cut here---------------start------------->8--- $ PYTHONPATH=3D. GUIX_PYTHONPATH=3D python sample.py Fatal Python error: init_import_site: Failed to import the site module Python runtime state: initialized Traceback (most recent call last): File "/gnu/store/p5fgysbcnnp8b1d91mrvjvababmczga0-python-3.9.6/lib/python= 3.9/site.py", line 589, in main() File "/gnu/store/p5fgysbcnnp8b1d91mrvjvababmczga0-python-3.9.6/lib/python= 3.9/site.py", line 582, in main execsitecustomize() File "/gnu/store/p5fgysbcnnp8b1d91mrvjvababmczga0-python-3.9.6/lib/python= 3.9/site.py", line 521, in execsitecustomize import sitecustomize File "/home/maxim/proj/kinova/kts_robot/sitecustomize.py", line 52, in exit(0) File "/gnu/store/p5fgysbcnnp8b1d91mrvjvababmczga0-python-3.9.6/lib/python= 3.9/_sitebuiltins.py", line 26, in __call__ raise SystemExit(code) SystemExit: 0 --8<---------------cut here---------------end--------------->8--- After the proposed change: --8<---------------cut here---------------start------------->8--- [env]$ PYTHONPATH=3D. GUIX_PYTHONPATH=3D python sample.py > /home/maxim/proj/kinova/kts_robot/sample.py(5)() -> print('hello') --8<---------------cut here---------------end--------------->8--- There's no longer pdbpp because of clearing GUIX_PYTHONPATH but at least it doesn't crash :-). >> +# Move the entries that were appended to sys.path in front of Python's = own >> +# site-packages directory. This enables Guix packages to override Pyth= on's >> +# bundled packages, such as 'pip'. >> +python_site_index =3D sys.path.index(python_site) >> +new_site_start_index =3D sys.path.index(matching_sites[0]) >> +if python_site_index < new_site_start_index: >> + sys.path =3D (sys.path[:python_site_index] >> + + sys.path[new_site_start_index:] >> + + sys.path[python_site_index:new_site_start_index]) > This is unrelated to the pdb issue, right? I see that it=E2=80=99s necess= ary > right now, but as suggested in #46848 I=E2=80=99d prefer unbundling > setuptools/pip from python. (I=E2=80=99ll send a v3 of the patchset at so= me > point.) Previously the Guix-provided paths were directly spliced at the right location; now using 'site.addsitedir' simply appends them, which requires manual fiddling afterward. I agree that after it's un-bundled it shouldn't be necessary anymore, but let's keep this change for core-updates along work on the 517 python-build-system (I'll try having a look to it after the next release it out -- ping me otherwise). Thank you, Maxim --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-sitecustomize.py-Honor-.pth-files.patch >From 49f0d2a493b868b9414ea10c7a676cf8404e1bca Mon Sep 17 00:00:00 2001 From: Maxim Cournoyer Date: Fri, 3 Dec 2021 22:36:26 -0500 Subject: [PATCH] sitecustomize.py: Honor .pth files. Fixes . * gnu/packages/aux-files/python/sitecustomize.py: Use site.addsitedirs to add the site directories; this takes care of the .pth files. Make sure the added items still appear before Python's own 'site-packages' directory. --- .../aux-files/python/sitecustomize.py | 22 ++++++++++++++----- 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/gnu/packages/aux-files/python/sitecustomize.py b/gnu/packages/aux-files/python/sitecustomize.py index 71e328b9ac..e2348e0356 100644 --- a/gnu/packages/aux-files/python/sitecustomize.py +++ b/gnu/packages/aux-files/python/sitecustomize.py @@ -18,6 +18,7 @@ # along with GNU Guix. If not, see . import os +import site import sys # Commentary: @@ -47,9 +48,18 @@ all_sites_norm = [os.path.normpath(p) for p in all_sites_raw] matching_sites = [p for p in all_sites_norm if p.endswith(site_packages_prefix)] -# Insert sites matching the current version into sys.path, right before -# Python's own site. This way, the user can override the libraries provided -# by Python itself. -sys_path_absolute = [os.path.realpath(p) for p in sys.path] -index = sys_path_absolute.index(python_site) -sys.path[index:index] = matching_sites +if matching_sites: + # Deduplicate the entries, append them to sys.path, and handle any + # .pth files they contain. + for s in matching_sites: + site.addsitedir(s) + + # Move the entries that were appended to sys.path in front of + # Python's own site-packages directory. This enables Guix + # packages to override Python's bundled packages, such as 'pip'. + python_site_index = sys.path.index(python_site) + new_site_start_index = sys.path.index(matching_sites[0]) + if python_site_index < new_site_start_index: + sys.path = (sys.path[:python_site_index] + + sys.path[new_site_start_index:] + + sys.path[python_site_index:new_site_start_index]) -- 2.34.0 --=-=-=--