From mboxrd@z Thu Jan 1 00:00:00 1970 From: Hartmut Goebel Subject: PYTHONPATH issue analysis - part 1 (was: PYTHONPATH woes) Date: Tue, 27 Feb 2018 12:43:18 +0100 Message-ID: <533d1b89-377f-d026-df90-b81b9b7b3f0c@crazy-compilers.com> References: <87371tqbyb.fsf@elephly.net> <20180223165953.GA6088@thebird.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:40876) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eqdfM-0000jh-LY for guix-devel@gnu.org; Tue, 27 Feb 2018 06:43:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eqdfJ-00038K-96 for guix-devel@gnu.org; Tue, 27 Feb 2018 06:43:28 -0500 Received: from mail-out.m-online.net ([212.18.0.10]:56942) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eqdfI-00037l-UD for guix-devel@gnu.org; Tue, 27 Feb 2018 06:43:25 -0500 In-Reply-To: Content-Language: de-DE List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Pjotr Prins Cc: guix-devel@gnu.org Hi, I'm going to analyse the PYTHONPATH issues systematically. This is part 1= of the analysis. Result =3D=3D=3D=3D=3D=3D=3D PYTHONPATH intented (by upstream) for adding site-packages.=C2=A0 PYTHONP= ATH elements are added in front of essential packages, while site-packages ar= e added behind. This means the way we are using PYTHONPATH should be changed. Preliminary Proposal =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D This proposal is under the limitation of the current state of the analysis. Further parts of the analysis may yield more insights, and propose different solutions. 1. The search-path-specification PYTHONPATH should be replaced by =C2=A0=C2=A0 GUIX-PYTHON2-SITE-PACKAGES resp. GUIX-PYTHON3-SITE-PACKAGES = (names =C2=A0=C2=A0 tbd). 2. 'site.py' will be patched to add theses entries to sys.path. Limitations: This only solves the issue related to version-specific site-packages. pyt= hon still thinks its "home" is in /gnu/store while it actually should be in t= he guix-profile. Rational =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D According to the documentation [1,2,PEP370], sys.path is composed as follows: =C2=A0 $PWD resp. dir containing the script =C2=A0 $PYTHONPATH elements =C2=A0 default search path PREFIX/lib/pythonX.Y =C2=A0 user site-packages=C2=A0 ~/.local/lib/pythonX.Y/site-packages =C2=A0 system-site packages PREFIX/lib/python2.6/site-packages The .pth-files are processed in user and system site-packages only, but not in $PYTHONPATH. [PEP370] explicitly says: "The [=E2=80=A6] site directory is added [=E2=80= =A6] after Python's search paths and PYTHONPATH. This setup [=E2=80=A6] prevents [=E2= =80=A6] overwriting a stdlib module. Stdlib modules can still be overwritten with PYTHONPATH." This means: PYTHONPATH is not intented to set system site-packages. Example: $ cd ~ $ PYTHONPATH=3D/tmp/aaa:/tmp/bbb /usr/bin/python -m site sys.path =3D [ =C2=A0=C2=A0=C2=A0 '/home/htgoebel, =C2=A0=C2=A0=C2=A0 '/tmp/aaa', =C2=A0=C2=A0=C2=A0 '/tmp/bbb', =C2=A0=C2=A0=C2=A0 '/usr/lib/python27.zip', =C2=A0=C2=A0=C2=A0 '/usr/lib64/python2.7', =C2=A0=C2=A0=C2=A0 '/usr/lib64/python2.7/plat-linux2', =C2=A0=C2=A0=C2=A0 '/usr/lib64/python2.7/lib-tk', =C2=A0=C2=A0=C2=A0 '/usr/lib64/python2.7/lib-old', =C2=A0=C2=A0=C2=A0 '/usr/lib64/python2.7/lib-dynload', =C2=A0=C2=A0=C2=A0 '/home/htgoebel/.local/lib/python2.7/site-packages', =C2=A0=C2=A0=C2=A0 '/usr/lib64/python2.7/site-packages', =C2=A0=C2=A0=C2=A0 '/usr/lib64/python2.7/site-packages/gtk-2.0', =C2=A0=C2=A0=C2=A0 '/usr/lib64/python2.7/site-packages/wx-3.0-gtk2', =C2=A0=C2=A0=C2=A0 '/usr/lib/python2.7/site-packages', ] [1] https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPATH [2] https://docs.python.org/3/library/site.html#module-site [PEP370] https://www.python.org/dev/peps/pep-0370/#implementation --=20 Regards Hartmut Goebel | Hartmut Goebel | h.goebel@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible |