Avoiding PYTHONPATH - latest?

[Phil <phil@beadling.co.uk>]

Hi all,

I've been having an argument with myself over the last 4 days about if
Guix's use of PYTHONPATH is a necessary evil or avoidable on a foreign OS.

I've found references to a similar discussion last year, and reference
to using a 'fake virtual environment' (does anyone have a reference to
the other thread referenced where fake venvs are demonstrated as not
working):

https://lists.gnu.org/archive/html/guix-devel/2019-06/msg00204.html
https://lists.gnu.org/archive/html/guix-devel/2019-06/msg00221.html

I had what I think is a similar idea over the weekend and tried it out.

Reading the mechanics of venvs as per PEP:
https://www.python.org/dev/peps/pep-0405

I decided to see if I added a fake pyvenv.cfg to the python.scm package
could allow for us to drop the use of PYTHONPATH completely.

Now the results are very preliminary, and I am essentially replacing
one hack with another (I'm not creating a venv after-all) - it does seem
to work just adding this to the python-3.8's phases: 

(add-after 'install 'add-pyvenv-cfg
           (lambda* (#:key outputs #:allow-other-keys)
                     ;; guix-daemon --build-users-group=guixbuild --chroot-directory=/path/to/guix_chroot_extra
                     (let ((out (assoc-ref outputs "out")))
                       (display (string-append "\nOutput Directory: \n" out "\n"))
                       (install-file "/path/to/guix_chroot_extra/pyvenv.cfg" out)
                       (substitute* (string-append out "/pyvenv.cfg")
                                    (("^home.*") (string-append "home = " out "\n"))
                                    (("^version.*") (string-append "version = 3.8.2\n"))))))

The native-search-paths can then have PYTHONPATH removed (I emulated
this by unset'ing it after creating a profile with the modified version
of the python package).

Starting python and looking at sys.path I could see this worked, both in
the cases where lib and bin symlink their counterpart
directories in the store, and also work when real lib and bin
directories in the profile contain symlinks to the store at the file level.

It seems so simple I'm sure it's been tried and found to misbehave, as I
hands-up accept it's misuse venv's funtionality (using pyvenv.cfg
without setting VIRTUAL_ENV is not defined behaviour).

But the current PYTHONPATH implementation immediately runs into problems
when you use venvs for development, via the cannonical "pip install -e
.".  The use of PYTHONPATH means that venvs are not created pure, they
inherit Guix's python env which is the exact opposite to what any
non-Guix python developer would expect.

Ironically when sharing environment setup responsibilities between
profiles and venvs like this, inheriting the profile's PYTHONPATH is
often useful - but it's still unexpected.

The PYTHONPATH, if I've understood correctly, is for user-level tweaks,
so hijacking it for system use may cause issues.

I was wondering what the current thoughts were on how Guix could
workaround using PYTHONPATH to set base system paths?

Thanks,
Phil.