PYTHONPATH issue analysis - part 1 (was: PYTHONPATH woes)

[Hartmut Goebel <h.goebel@crazy-compilers.com>]

Hi,


I'm going to analyse the PYTHONPATH issues systematically. This is part 1 of
the analysis.


Result
=======

PYTHONPATH intented (by upstream) for adding site-packages.  PYTHONPATH
elements are added in front of essential packages, while site-packages are
added behind.

This means the way we are using PYTHONPATH should be changed.


Preliminary Proposal
=======================

This proposal is under the limitation of the current state of the
analysis. Further parts of the analysis may yield more
insights, and propose different solutions.

1. The search-path-specification PYTHONPATH should be replaced by
   GUIX-PYTHON2-SITE-PACKAGES resp. GUIX-PYTHON3-SITE-PACKAGES (names
   tbd).

2. 'site.py' will be patched to add theses entries to sys.path.

Limitations:

This only solves the issue related to version-specific site-packages. python
still thinks its "home" is in /gnu/store while it actually should be in the
guix-profile.


Rational
=================

According to the documentation [1,2,PEP370], sys.path is composed as
follows:

  $PWD resp. dir containing the script
  $PYTHONPATH elements
  default search path PREFIX/lib/pythonX.Y
  user site-packages  ~/.local/lib/pythonX.Y/site-packages
  system-site packages PREFIX/lib/python2.6/site-packages

The .pth-files are processed in user and system site-packages only,
but not in $PYTHONPATH.

[PEP370] explicitly says: "The […] site directory is added […] after
Python's search paths and PYTHONPATH. This setup […] prevents […]
overwriting a stdlib module. Stdlib modules can still be overwritten
with PYTHONPATH."

This means: PYTHONPATH is not intented to set system site-packages.

Example:

$ cd ~
$ PYTHONPATH=/tmp/aaa:/tmp/bbb /usr/bin/python -m site
sys.path = [
    '/home/htgoebel,
    '/tmp/aaa',
    '/tmp/bbb',
    '/usr/lib/python27.zip',
    '/usr/lib64/python2.7',
    '/usr/lib64/python2.7/plat-linux2',
    '/usr/lib64/python2.7/lib-tk',
    '/usr/lib64/python2.7/lib-old',
    '/usr/lib64/python2.7/lib-dynload',
    '/home/htgoebel/.local/lib/python2.7/site-packages',
    '/usr/lib64/python2.7/site-packages',
    '/usr/lib64/python2.7/site-packages/gtk-2.0',
    '/usr/lib64/python2.7/site-packages/wx-3.0-gtk2',
    '/usr/lib/python2.7/site-packages',
]


[1] https://docs.python.org/3/using/cmdline.html#envvar-PYTHONPATH
[2] https://docs.python.org/3/library/site.html#module-site
[PEP370] https://www.python.org/dev/peps/pep-0370/#implementation

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |