From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id dnnwE1dSvV88egAA0tVLHw (envelope-from ) for ; Tue, 24 Nov 2020 18:35:03 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id yMpdD1dSvV8cdAAA1q6Kng (envelope-from ) for ; Tue, 24 Nov 2020 18:35:03 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 7BA189401BC for ; Tue, 24 Nov 2020 18:35:02 +0000 (UTC) Received: from localhost ([::1]:37088 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1khd9Y-0002gN-Bp for larch@yhetil.org; Tue, 24 Nov 2020 13:35:00 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:33382) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1khc1X-0004y6-SF for help-guix@gnu.org; Tue, 24 Nov 2020 12:22:39 -0500 Received: from mail-wm1-x334.google.com ([2a00:1450:4864:20::334]:36875) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1khc1U-0002mY-53 for help-guix@gnu.org; Tue, 24 Nov 2020 12:22:39 -0500 Received: by mail-wm1-x334.google.com with SMTP id h21so3700738wmb.2 for ; Tue, 24 Nov 2020 09:22:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=beadling-co-uk.20150623.gappssmtp.com; s=20150623; h=user-agent:from:to:subject:date:message-id:mime-version; bh=nANLp2DLCYQvsqbEGBV8LZF+3Di+WhO/jIQxrjOe3xE=; b=MyWELkpZi+gRtoQzZWz7Xb9HxOttzZjFsGBzJP9SbhPe+51LubTr9UeUw7QiShd/li VASdzUauZ4Wi6lcuJkRbh4FLXXQKYsgu/0DvSZBabeJ6lZpUQFl6VOkWrY7CKvfzaU06 5d1odjQ9vvCkeUaPWxvmd2NNfdMZS7PHw7fJ6fIQl48Oy0Ndg++nnEK8stqugvdLd7iy 4EuRGyXp/4b6fUSyju+lBpIUZzHBp8vbpqREGFXKVhBrHv7cno7Pa56NUsb+IIOHUjRn w4wXiJnbdj6ll016/Z174iRGFyjOcYnpz1VvVFLNvfd4EuhwRxhIMp2LlCQL6c0HB5/a Atsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:user-agent:from:to:subject:date:message-id :mime-version; bh=nANLp2DLCYQvsqbEGBV8LZF+3Di+WhO/jIQxrjOe3xE=; b=tF2fy35Rmm75aklCvk45NfA24uILFa9nD9Zhqn1dxo5cOjqZAnFSQnTcNZuaAeMySd gibWxFMj7ZfZThUBA3StVJIkqy+v8zAWbF7JtFGTuCECFWc5qCBXh9cwzqzn8I3Agawu rOpJ7KX8uERAvdSpey1AQ3xaL8krEEoPkeweW5ukprAI4m6z3Yo7beRTuRgDZ5MK9ht+ jWpYrdN4iprTv4cr61sBwP1GIaDkqGwLjHoPGCSu74oc9w66PvVEleni8bdF/xfQX3e6 w1BjKU9TAouRwX9lOlbRUBN5/Ny5q+QFvnSoTP5vxFJhi1IXqwH6i9ecnrFWel26/yPg HLNg== X-Gm-Message-State: AOAM530eW1JbzfMPIh0soL9dfVQ3DyFz0/+nbY8UZOH7no121JCDKQ7a UcFKl1ysSYekAFNqJSoLleOS76bY++nVX31M1pc= X-Google-Smtp-Source: ABdhPJzAGgd/41W5bBGBO/uc4HfVrKY/7gdT4NSMGKiXgCg2Nc2R7Wi0VhwJQkrQYop9Oo0rq7BYxA== X-Received: by 2002:a05:600c:214f:: with SMTP id v15mr5759487wml.5.1606238552287; Tue, 24 Nov 2020 09:22:32 -0800 (PST) Received: from phil-XPS-13-9360 (88-111-129-212.dynamic.dsl.as9105.com. [88.111.129.212]) by smtp.gmail.com with ESMTPSA id m7sm6495160wmc.22.2020.11.24.09.22.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 24 Nov 2020 09:22:31 -0800 (PST) User-agent: mu4e 1.2.0; emacs 26.3 From: Phil To: help-guix@gnu.org Subject: Replacing python venv - environment or profile? Date: Tue, 24 Nov 2020 17:22:29 +0000 Message-ID: <85r1oifz56.fsf@beadling.co.uk> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: none client-ip=2a00:1450:4864:20::334; envelope-from=phil@beadling.co.uk; helo=mail-wm1-x334.google.com X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Tue, 24 Nov 2020 13:34:51 -0500 X-BeenThere: help-guix@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-guix-bounces+larch=yhetil.org@gnu.org Sender: "Help-Guix" X-Scanner: ns3122888.ip-94-23-21.eu Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=beadling-co-uk.20150623.gappssmtp.com header.s=20150623 header.b=MyWELkpZ; dmarc=none; spf=pass (aspmx1.migadu.com: domain of help-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=help-guix-bounces@gnu.org X-Spam-Score: -0.21 X-TUID: ylMBQpZf5c/+ Hi all, Apologies this is a bit longer than I anticipated, but I wanted to record all the steps I'd taken to explain my reasoning - in case it's wrong! I'm new the Guix and considering adopting it as an expansion to my current use of Python virtual environments in development and production, to include other non-Python packages too. The potential of the software is very exciting! I've read the manual/cookbook and I get the gist of the tool. One thing I'm getting a bit stuck on is the appropriate use of environment vs profile; different sources give slightly different takes on the theme, and I want to make sure my use is correct from the get-go. I'll be using Guix on a foreign OS, I suspect the answers might be slightly different if I was able to adopt the Guix System OS, but for now I can't. As well as the manual cookbook I came across this blog: https://trivialfis.github.io/linux/2018/06/10/Using-guix-for-development.html This is describing the use of 'guix environment' and manifests to create a structure very similar to a python virtual environment, but including the python package itself too. Which is exactly the starting point I had in mind. I understand that I can use the --pure switch to ensure no pollution from the foreign OS programs - just like the default behaviour of python venv, eg: ubuntu@primary:~$ guix environment --ad-hoc coreutils --pure ubuntu@primary:~$ ls dev ubuntu@primary:~$ nano Command 'nano' is available in the following places * /bin/nano * /usr/bin/nano The command could not be located because '/bin:/usr/bin' is not included in the PATH environment variable. nano: command not found This is great - but the manual points out that none of the installed packages are marked such that they avoid garbage collection. So I came up with something like the below - manifests do seem to be an exact analogy to Python requirements.txt files: $ cat manifest.scm (specifications->manifest '("coreutils" "emacs@27.1" "python" "python-pytest" "python-coverage" "python-pytest-cov" "python-black" "python-mypy" "python-flake8")) $ guix environment --pure --manifest=manifest.scm --root=./test-profile This creates a profile as a side-effect, I think. This would be all well and good but both the manual and the cookbook, and a few other sources I've found online seem to equate a *profile* with a virtualenv rather than use of the *guix environment* command. To test this as an alternative, I source the profile created by the manifest above: $ export GUIX_PROFILE=/home/ubuntu/dev/test-profile $ . $GUIX_PROFILE/etc/profile I note that unlike the environment which creates what I assume is its own temporary profile, eg: $ echo $PATH /gnu/store/hchmga9ybpdc4zph9cs8jr7m1k8gxw9f-profile/bin Sourcing the created profile references both it, the default profile, and guix's profile, followed by the foreign OS PATH variables: $ echo $PATH /home/ubuntu/dev/test-profile/bin:/home/ubuntu/.guix-profile/bin:/home/ubuntu/.config/guix/current/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin I'm keen to avoid accidentally calling anything in the foreign OS, something the 'guix environment' command gives me, but obviously the foreign OS will serve as a fallback given the above PATH construction under a sourced profile. So if my profile doesn't have python installed, but it is part of the foreign OS, I would silently pick that up, which would be bad. As per 4.1.1 in the cookbook I can avoid setting other profiles using: $ env -i $(which bash) --login --noprofile --norc $ export GUIX_PROFILE=/home/ubuntu/dev/test-profile $ . $GUIX_PROFILE/etc/profile But the foreign OS variables remain: $ echo $PATH /home/ubuntu/dev/test-profile/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin $ So to my mind Guix profiles are more like Python virtual environments with the with the non-standard --system-site-packages switch than the classic use of Python virtual environment? Useful, but different. The 'guix environment' is accidentally providing a closer parallel. I'm guessing that if I used the Guix System OS the profile analogy would be much more accurate as there would be no foreign OS to fallback onto? Finally.... I get to my question! So in light of the above (assuming I haven't missed the point completely!), what is the canonical way of isolating a virtual environment using Guix on a foreign OS installation? Is the use of 'guix environment' as per the blog referenced above considered good practice, or is this as I now suspect, inappropriate re-purposing of a feature designed to create transient environments for building/debugging specific programs packaged in Guix? Links such as this suggest this might be the case: https://yhetil.org/guix-user/1700d451826.11317682011034.4058430466276292306@zoho.com/T/ Of course I could probably write a script that scrubs my own environment variables of all references to the foreign OS, but this feels like I'm doing what I thought I'd get for free with Guix in the first place? Last point - ultimately as well as using Guix to provide a consistent profile/environment to do Python work in, I'd ultimately like to hook Guix up to Jenkins so that it can package and deploy a repo to a production server. This probably involves setting-up a private Guix Channel and importing built wheels, and to then pull the new Guix package from the production server. Ultimately tho I'd want the same thing - a pure environment in production which has access to exactly and only the software as described in the manifest. This feels like it should be possible too, but again, just looking for any initial guidance on which features of Guix to use to do this. Sorry for the long e-mail, and thanks in advance for any guidance or advice on the matter! Cheers, Phil.