From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id 4P7FCnO+xF/uFAAA0tVLHw (envelope-from ) for ; Mon, 30 Nov 2020 09:42:11 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id eKiNBnO+xF8PNAAAB5/wlQ (envelope-from ) for ; Mon, 30 Nov 2020 09:42:11 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 173BF9404C7 for ; Mon, 30 Nov 2020 09:42:10 +0000 (UTC) Received: from localhost ([::1]:45100 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kjfhB-0004SY-3W for larch@yhetil.org; Mon, 30 Nov 2020 04:42:09 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:42166) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kjfgw-0004SG-BT for help-guix@gnu.org; Mon, 30 Nov 2020 04:41:54 -0500 Received: from mail-wm1-x332.google.com ([2a00:1450:4864:20::332]:51338) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kjfgt-0005PY-5r for help-guix@gnu.org; Mon, 30 Nov 2020 04:41:54 -0500 Received: by mail-wm1-x332.google.com with SMTP id v14so12041534wml.1 for ; Mon, 30 Nov 2020 01:41:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=beadling-co-uk.20150623.gappssmtp.com; s=20150623; h=references:user-agent:from:to:subject:in-reply-to:date:message-id :mime-version; bh=uBaPTr50yWdt/imhX1jmYvBWo1sjS4uMXY6KRq0AJ9k=; b=CJ5CNXBQtceXNjGqeD2+1NEzOQghJdg+DQ6VrvBV5dzbBZzzdsC0qKSHoHq8smzI0Z P96/4BEzYNrayNlD/teMeTsBIU+J3tQ2xybMh8bmHrLlND4Da1d355MKLhY9vPUQm4W+ HguWgoS26GDXXRLdPNxNiY9b30ZKcOXckBELZgCQadDs1CD0CYKnDYrkLmwrPxmmePjh bwkojDGaG2Ff2otCxoP9xtwpLNH3VoX8UeM/hM/4UWWTakMAx6+oX8WrW+F8DCS0zYSu f7O7KJqspTti8QdrABxXYTNSWBmUQ6/jBVco0VUGr5cCWvC4zVemiOqHEZQeJpq/uPtT Fxug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:references:user-agent:from:to:subject :in-reply-to:date:message-id:mime-version; bh=uBaPTr50yWdt/imhX1jmYvBWo1sjS4uMXY6KRq0AJ9k=; b=GHXZwDSj7VzTjIM87X8w1mMlGuQ6Mc8XoYXhV7riI/reRgt9AesUENXWBJZZz9NjLQ JGGOzuVHXTj1IIBsJ2JQLFW6V0pA5PNcQgJTMDJ4i070zhjoymz0nGJLOJo9JFEP3o/U EYU7L5zd/s7esYzrA9huAMiGdFa0ZRBIrwTQHw7GXgOSNSQ2Y3qLFUjm6sm0xC3W+AVa OCXafDIel0LKIch0mGohj7vxdlhIfWu7gcVhmJMgI6D1GRxZssJwdnfgMMgHjqrOsNrm ARQfngNrJKdm6owUjkA3yZx0RcH/gGsqiXtUa3v7OX3/uD7wrEoGsD3X8BcY4ETmQ8T/ b9AA== X-Gm-Message-State: AOAM5334OkhJ49vetmY2vsZy6ViKP+Sc61KaR/FRiQ+QvHz6ekDfJKBL URBgKkCCEcTuP8wighPIEjrMocZahBB/D3Dwcao= X-Google-Smtp-Source: ABdhPJzx0M76A2K/ykGt7WU15aqMebliawFfra3CjZ+bReYF6iuPHobOIj5N5khpluYJi9lIBVwJuQ== X-Received: by 2002:a1c:f203:: with SMTP id s3mr11736648wmc.14.1606729308131; Mon, 30 Nov 2020 01:41:48 -0800 (PST) Received: from phil-XPS-13-9360 (88-111-129-212.dynamic.dsl.as9105.com. [88.111.129.212]) by smtp.gmail.com with ESMTPSA id e1sm4901799wma.17.2020.11.30.01.41.47 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 30 Nov 2020 01:41:47 -0800 (PST) References: <85360rr8v5.fsf@beadling.co.uk> User-agent: mu4e 1.2.0; emacs 26.3 From: Phil To: help-guix@gnu.org Subject: Re: Avoiding PYTHONPATH - latest? In-reply-to: <85360rr8v5.fsf@beadling.co.uk> Date: Mon, 30 Nov 2020 09:41:46 +0000 Message-ID: <851rgbqizp.fsf@beadling.co.uk> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: none client-ip=2a00:1450:4864:20::332; envelope-from=phil@beadling.co.uk; helo=mail-wm1-x332.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: "Help-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -1.48 X-Scanner: ns3122888.ip-94-23-21.eu Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=beadling-co-uk.20150623.gappssmtp.com header.s=20150623 header.b=CJ5CNXBQ; dmarc=none; spf=pass (aspmx1.migadu.com: domain of help-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=help-guix-bounces@gnu.org X-TUID: 6wCnz8GUu/nx I should have added the pyvenv.cfg file should look like this: $ cat pyvenv.cfg home = /replace/with/out/variable/in/guix include-system-site-packages = true version = replace.with.guix.version.variable We include system packages to ensure we take site-packages from both the Python 3.8 store's and the profile's site-packages. ---- One thing I have noticed is that if you want to further nest a venv (using the regular: python3 -m venv) inside the setup below - that will NOT work if you include --system-site-packages in the nested venv. The profile's site-packages are not included, because it would seem that python does not respect the outer pyvenv.cfg in nested venvs. If this can't be solved it pours cold water on the idea somewhat. Phil writes: > Hi all, > > I've been having an argument with myself over the last 4 days about if > Guix's use of PYTHONPATH is a necessary evil or avoidable on a foreign OS. > > I've found references to a similar discussion last year, and reference > to using a 'fake virtual environment' (does anyone have a reference to > the other thread referenced where fake venvs are demonstrated as not > working): > > https://lists.gnu.org/archive/html/guix-devel/2019-06/msg00204.html > https://lists.gnu.org/archive/html/guix-devel/2019-06/msg00221.html > > I had what I think is a similar idea over the weekend and tried it out. > > Reading the mechanics of venvs as per PEP: > https://www.python.org/dev/peps/pep-0405 > > I decided to see if I added a fake pyvenv.cfg to the python.scm package > could allow for us to drop the use of PYTHONPATH completely. > > Now the results are very preliminary, and I am essentially replacing > one hack with another (I'm not creating a venv after-all) - it does seem > to work just adding this to the python-3.8's phases: > > (add-after 'install 'add-pyvenv-cfg > (lambda* (#:key outputs #:allow-other-keys) > ;; guix-daemon --build-users-group=guixbuild --chroot-directory=/path/to/guix_chroot_extra > (let ((out (assoc-ref outputs "out"))) > (display (string-append "\nOutput Directory: \n" out "\n")) > (install-file "/path/to/guix_chroot_extra/pyvenv.cfg" out) > (substitute* (string-append out "/pyvenv.cfg") > (("^home.*") (string-append "home = " out "\n")) > (("^version.*") (string-append "version = 3.8.2\n")))))) > > The native-search-paths can then have PYTHONPATH removed (I emulated > this by unset'ing it after creating a profile with the modified version > of the python package). > > Starting python and looking at sys.path I could see this worked, both in > the cases where lib and bin symlink their counterpart > directories in the store, and also work when real lib and bin > directories in the profile contain symlinks to the store at the file level. > > It seems so simple I'm sure it's been tried and found to misbehave, as I > hands-up accept it's misuse venv's funtionality (using pyvenv.cfg > without setting VIRTUAL_ENV is not defined behaviour). > > But the current PYTHONPATH implementation immediately runs into problems > when you use venvs for development, via the cannonical "pip install -e > .". The use of PYTHONPATH means that venvs are not created pure, they > inherit Guix's python env which is the exact opposite to what any > non-Guix python developer would expect. > > Ironically when sharing environment setup responsibilities between > profiles and venvs like this, inheriting the profile's PYTHONPATH is > often useful - but it's still unexpected. > > The PYTHONPATH, if I've understood correctly, is for user-level tweaks, > so hijacking it for system use may cause issues. > > I was wondering what the current thoughts were on how Guix could > workaround using PYTHONPATH to set base system paths? > > Thanks, > Phil.