From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id 1DZBFKT+qmHsAgEAgWs5BA (envelope-from ) for ; Sat, 04 Dec 2021 06:37:40 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id mGhFD6T+qmHnCAAAB5/wlQ (envelope-from ) for ; Sat, 04 Dec 2021 05:37:40 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id D8F812D2FB for ; Sat, 4 Dec 2021 06:37:39 +0100 (CET) Received: from localhost ([::1]:37496 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mtNju-0006r6-Ip for larch@yhetil.org; Sat, 04 Dec 2021 00:37:38 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33656) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mtNjK-0006q8-Gv for bug-guix@gnu.org; Sat, 04 Dec 2021 00:37:02 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:41235) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mtNjK-0002fR-8Z for bug-guix@gnu.org; Sat, 04 Dec 2021 00:37:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mtNjK-0004JH-4I for bug-guix@gnu.org; Sat, 04 Dec 2021 00:37:02 -0500 X-Loop: help-debbugs@gnu.org Subject: bug#52269: [PATCH core-updates-frozen] sitecustomize does not honor .pth files Resent-From: Maxim Cournoyer Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Sat, 04 Dec 2021 05:37:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 52269 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 52269@debbugs.gnu.org, GNU Debbugs Received: via spool by 52269-submit@debbugs.gnu.org id=B52269.163859619616530 (code B ref 52269); Sat, 04 Dec 2021 05:37:02 +0000 Received: (at 52269) by debbugs.gnu.org; 4 Dec 2021 05:36:36 +0000 Received: from localhost ([127.0.0.1]:52781 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mtNiu-0004IS-9W for submit@debbugs.gnu.org; Sat, 04 Dec 2021 00:36:36 -0500 Received: from mail-qk1-f170.google.com ([209.85.222.170]:41646) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mtNip-0004I7-7N; Sat, 04 Dec 2021 00:36:34 -0500 Received: by mail-qk1-f170.google.com with SMTP id t83so5770293qke.8; Fri, 03 Dec 2021 21:36:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:subject:references:date:in-reply-to:message-id:user-agent :mime-version; bh=Trx/zUMN1VTEHVYsdG43n1GeNumg8RVRJvaJfSjTjDA=; b=dIwPD4TpGZhXnTUVV8JG7ZTXjKiuQWL9ppU8WnAppby+f0gXRlSKo3QpuuT0Qk2Ed5 nHTTzgBTpzfBJxnMUYJNr2JEe0/BnL0j14rSQ/oWxv/2zJ1UD7L5H+g7Dp2zhAm9whTS smXxLKG1GBF/YSG+/LbIKZ1AxwB3sB9IYKdzLYSbGuhlTsA86i7Sbv6zXrTX33/7rlRy kb3zlqoi5nXoYrnyI6Mdg2tR2fJvkmOJm5BlrcUEtaOukQJVTnkAIDQmBDL1UNor5miC CLtkpkJh5kPFS5MYiXah03s7zIAsbSXroqS95mFvKUGY6kzIUXR5qrR/dcw9asUh+EM6 Ohrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:subject:references:date:in-reply-to :message-id:user-agent:mime-version; bh=Trx/zUMN1VTEHVYsdG43n1GeNumg8RVRJvaJfSjTjDA=; b=Gs0dvt0+EpTCfrQD+bDnP56SckTdHOsa/rFB8TQAhXzQoOTq5kAZ0Vp76V1TY/vac+ S/YyxCCVUvkgIH9XohnR7WnmxxXXH+9jRM7GVdMtkKFg2sYsWb1h0pFih98KO7nyIo9M RAH9GE/Bnlf5N9OxxYSLVO0EvipjptSqyWj4Njf3QYEUEReWKp9tYSVXBs218YZYzVgT fnGkFZZJkL67Irtoh8+DS94X0nV81UV+5sv/OqmDkHjB5TgqVNgyrWnrwQPIk8cDr9st Dcfc6eW9uVPuLdEOfVzKRywVH3UCsOnkYZryRCiy20d7zApsFdoZb/KCD04QimaqoiIA A7nQ== X-Gm-Message-State: AOAM5302RHFN7uUh0olgWB2yYYvJ0Tc3t4ay1j3Fg0wRzzezerco6+X4 Ao9D1qPzl0ljzCtOIsw4wrHmZfJaYic= X-Google-Smtp-Source: ABdhPJxPvf5N1gq4TsAS59rhjKh2KXokKyXMK5inMLwh6lPJtnH/BtGrha+TuFXEUpjDZyKKls+IAw== X-Received: by 2002:ae9:f716:: with SMTP id s22mr21573976qkg.212.1638596185333; Fri, 03 Dec 2021 21:36:25 -0800 (PST) Received: from hurd (dsl-156-29.b2b2c.ca. [66.158.156.29]) by smtp.gmail.com with ESMTPSA id e20sm4023223qty.14.2021.12.03.21.36.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 03 Dec 2021 21:36:24 -0800 (PST) From: Maxim Cournoyer References: <871r2taxcm.fsf@gmail.com> Date: Sat, 04 Dec 2021 00:36:23 -0500 In-Reply-To: <871r2taxcm.fsf@gmail.com> (Maxim Cournoyer's message of "Fri, 03 Dec 2021 21:59:53 -0500") Message-ID: <87mtlh9bjc.fsf@gmail.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1638596260; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:resent-cc:resent-from:resent-sender: resent-message-id:in-reply-to:in-reply-to:references:references: list-id:list-help:list-unsubscribe:list-subscribe:list-post: dkim-signature; bh=Trx/zUMN1VTEHVYsdG43n1GeNumg8RVRJvaJfSjTjDA=; b=Oe+o6T/sm9dkBsgVP7jA/3plAbx6TkYUm8Y3M7sMy8pvZdwVnHxEc8WYSrXCPMK/Yg4VeI IWPxjg0571/GoDJevpybr02A7y2nsOau/SQrT6Qp9Fe8gYL/CR/b+Ay83Gm4/bCouyQqzy uKL5D8253DkXphVWmK01/Jbr3N5Loc9MdYnsotTi8DrckBtdKYEZ1xvRSOnQT62vu2W5X4 zwAKMmeBBxW/+xA3qBcSXKpKuQgF+6N0nLnKj8SBzmqyJm8BRDj3o0y0z8oEA42ZOMp1qo yW696VhSPql+ie26vkZDx/Bw7R4YcEi/YfD1JnNqaSS5RVGbr/F7sI8osVyOqg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1638596260; a=rsa-sha256; cv=none; b=Rq+tGKA3IqXu45wd9LxFSxvI/APCdEKY3ws/adgzjVtgEUk4DfC8zyvsQM8fT+hAYXthZR ghqYoG2gFdPdfHMhzrc11y320y9UWkcqGo4HJAJkpFic3dXLx3bCQ7lgAeIYBOrW2Ns6Tl Du2gH1yoX4KHpFeKc5c9XFStW2VW/ayLnTDCy8lOri3+ARLctJLAk1dUK6z3QJBy/pkLHa nBPsVYyF1b3ylsC0kL0y4T2kV+dXlBrEy4faiE9rdqp4vCQoRB6k9n76weZRAxpHD4uLbm np6JRTPhxzCRtc9PXCEttDW5Xpt/+G6DsZ7gu64mBnKqzgfVPRB0b/3IKZgKkg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=dIwPD4Tp; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -1.83 Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=dIwPD4Tp; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none); spf=pass (aspmx1.migadu.com: domain of "bug-guix-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="bug-guix-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: D8F812D2FB X-Spam-Score: -1.83 X-Migadu-Scanner: scn0.migadu.com X-TUID: 2v3clJ3i7OVq --=-=-= Content-Type: text/plain tags 52269 patch thanks Hi! The following patch fixes it. I used site.addsitedir but ensured the correct ordering of sys.path (we need to make the Guix-installed packages appear before Python's own site-packages directory otherwise we wouldn't be able to override its bundled packages such as 'pip'). Here's how I tested: Copy the sitecustomize.py file to the current directory, then: --8<---------------cut here---------------start------------->8--- $ pip --version pip 21.1.3 from $HOME/.guix-profile/lib/python3.9/site-packages/pip (python 3.9) $ guix show python-pip | recsel -p version version: 20.2.4 --8<---------------cut here---------------end--------------->8--- Ensure installed pip still overrides Python's own. PYTHONPATH=. forces the sitecustomize.py file in the CWD to take precedence over the one currently installed along Python. --8<---------------cut here---------------start------------->8--- $ guix shell --pure python python-pip python-pdbpp [env]$ PYTHONPATH=. python3 -c 'import pip; print(pip.__version__)' 20.2.4 --8<---------------cut here---------------end--------------->8--- Next I created a dummy script to trigger run pdb: #file: test.py print('hello') import pdb; pdb.set_trace() --8<---------------cut here---------------start------------->8--- $ guix shell --pure python python-pip python-pdbpp [env]$ python3 test.py hello > /tmp/toto/test.py(7)() -> exit(1) (Pdb) --8<---------------cut here---------------end--------------->8--- This is the current bug; this is the regular Pdb, not Pdbpp. Let's force our revised sitecustomize.py file: --8<---------------cut here---------------start------------->8--- $ PYTHONPATH=. python3 test.py hello [0] > /tmp/toto/test.py(7)() -> exit(1) (Pdb++) --8<---------------cut here---------------end--------------->8--- Better! --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=0001-sitecustomize.py-Honor-.pth-files.patch >From 762357609270ab016236d22999ae5cfc3fe4ff28 Mon Sep 17 00:00:00 2001 From: Maxim Cournoyer Date: Fri, 3 Dec 2021 22:36:26 -0500 Subject: [PATCH] sitecustomize.py: Honor .pth files. Fixes . * gnu/packages/aux-files/python/sitecustomize.py: Use site.addsitedirs to add the site directories; this takes care of the .pth files. Make sure the added items still appear before Python's own 'site-packages' directory. --- .../aux-files/python/sitecustomize.py | 24 ++++++++++++++----- 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/gnu/packages/aux-files/python/sitecustomize.py b/gnu/packages/aux-files/python/sitecustomize.py index 71e328b9ac..bdaaa8e9e2 100644 --- a/gnu/packages/aux-files/python/sitecustomize.py +++ b/gnu/packages/aux-files/python/sitecustomize.py @@ -18,6 +18,7 @@ # along with GNU Guix. If not, see . import os +import site import sys # Commentary: @@ -47,9 +48,20 @@ all_sites_norm = [os.path.normpath(p) for p in all_sites_raw] matching_sites = [p for p in all_sites_norm if p.endswith(site_packages_prefix)] -# Insert sites matching the current version into sys.path, right before -# Python's own site. This way, the user can override the libraries provided -# by Python itself. -sys_path_absolute = [os.path.realpath(p) for p in sys.path] -index = sys_path_absolute.index(python_site) -sys.path[index:index] = matching_sites +if not matching_sites: + exit(0) + +# Deduplicate the entries, append them to sys.path, and handle any .pth files +# they contain. +for s in matching_sites: + site.addsitedir(s) + +# Move the entries that were appended to sys.path in front of Python's own +# site-packages directory. This enables Guix packages to override Python's +# bundled packages, such as 'pip'. +python_site_index = sys.path.index(python_site) +new_site_start_index = sys.path.index(matching_sites[0]) +if python_site_index < new_site_start_index: + sys.path = (sys.path[:python_site_index] + + sys.path[new_site_start_index:] + + sys.path[python_site_index:new_site_start_index]) -- 2.34.0 --=-=-=--