Replacing python venv - environment or profile?

Replacing python venv - environment or profile?

[Phil]

Hi all,

Apologies this is a bit longer than I anticipated, but I wanted to record
all the steps I'd taken to explain my reasoning - in case it's wrong!

I'm new the Guix and considering adopting it as an expansion to my
current use of Python virtual environments in development and
production, to include other non-Python packages too.

The potential of the software is very exciting!

I've read the manual/cookbook and I get the gist of the tool.  One thing
I'm getting a bit stuck on is the appropriate use of environment vs
profile; different sources give slightly different takes on the theme, and I
want to make sure my use is correct from the get-go.

I'll be using Guix on a foreign OS, I suspect the answers might be
slightly different if I was able to adopt the Guix System OS, but for
now I can't.

As well as the manual cookbook I came across this blog:
https://trivialfis.github.io/linux/2018/06/10/Using-guix-for-development.html

This is describing the use of 'guix environment' and manifests to create a
structure very similar to a python virtual environment, but including
the python package itself too.  Which is exactly the starting point I had in mind.

I understand that I can use the --pure switch to ensure no pollution
from the foreign OS programs - just like the default behaviour of python venv, eg:

ubuntu@primary:~$ guix environment --ad-hoc coreutils --pure

ubuntu@primary:~$ ls
dev
ubuntu@primary:~$ nano
Command 'nano' is available in the following places
 * /bin/nano
 * /usr/bin/nano
The command could not be located because '/bin:/usr/bin' is not included in the PATH environment variable.
nano: command not found


This is great - but the manual points out that none of the
installed packages are marked such that they avoid garbage collection.

So I came up with something like the below - manifests do seem to be an
exact analogy to Python requirements.txt files:

$ cat manifest.scm 
(specifications->manifest
  '("coreutils"
    "emacs@27.1"
    "python"
    "python-pytest"
    "python-coverage"
    "python-pytest-cov"
    "python-black"
    "python-mypy"
    "python-flake8"))
$ guix environment --pure --manifest=manifest.scm --root=./test-profile

This creates a profile as a side-effect, I think.

This would be all well and good but both the manual and the cookbook,
and a few other sources I've found online seem to equate a *profile* with
a virtualenv rather than use of the *guix environment* command.

To test this as an alternative, I source the profile created by the manifest above:

$ export GUIX_PROFILE=/home/ubuntu/dev/test-profile
$ . $GUIX_PROFILE/etc/profile

I note that unlike the environment which creates what I assume is its
own temporary profile, eg:

$ echo $PATH
/gnu/store/hchmga9ybpdc4zph9cs8jr7m1k8gxw9f-profile/bin

Sourcing the created profile references both it, the default profile,
and guix's profile, followed by the foreign OS PATH variables:

$ echo $PATH
/home/ubuntu/dev/test-profile/bin:/home/ubuntu/.guix-profile/bin:/home/ubuntu/.config/guix/current/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin


I'm keen to avoid accidentally calling anything in the foreign OS,
something the 'guix environment' command gives me, but obviously the foreign
OS will serve as a fallback given the above PATH construction under a sourced
profile.  So if my profile doesn't have python installed, but it is part
of the foreign OS, I would silently pick that up, which would be bad.


As per 4.1.1 in the cookbook I can avoid setting other profiles using:

$ env -i $(which bash) --login --noprofile --norc
$ export GUIX_PROFILE=/home/ubuntu/dev/test-profile
$ . $GUIX_PROFILE/etc/profile

But the foreign OS variables remain:

$ echo $PATH
/home/ubuntu/dev/test-profile/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
$ 

So to my mind Guix profiles are more like Python virtual environments with
the with the non-standard --system-site-packages switch than the classic
use of Python virtual environment?  Useful, but different.  The 'guix
environment' is accidentally providing a closer parallel.

I'm guessing that if I used the Guix System OS the profile analogy would be much
more accurate as there would be no foreign OS to fallback onto?

Finally.... I get to my question!  So in light of the above (assuming I
haven't missed the point completely!), what is the canonical way of
isolating a virtual environment using Guix on a foreign OS installation?

Is the use of 'guix environment' as per the blog referenced above
considered good practice, or is this as I now suspect, inappropriate re-purposing of a
feature designed to create transient environments for building/debugging specific
programs packaged in Guix?

Links such as this suggest this might be the case:
https://yhetil.org/guix-user/1700d451826.11317682011034.4058430466276292306@zoho.com/T/

Of course I could probably write a script that scrubs my own
environment variables of all references to the foreign OS, but this feels
like I'm doing what I thought I'd get for free with Guix in the first
place?

Last point - ultimately as well as using Guix to provide a consistent
profile/environment to do Python work in, I'd ultimately
like to hook Guix up to Jenkins so that it can package and deploy a repo to a
production server.  This probably involves setting-up a private Guix
Channel and importing built wheels, and to then pull the new Guix package from the
production server.  Ultimately tho I'd want the same thing - a pure
environment in production which has access to exactly and only the
software as described in the manifest.  This feels like it should be
possible too, but again, just looking for any initial guidance on which features
of Guix to use to do this.

Sorry for the long e-mail, and thanks in advance for any guidance
or advice on the matter!


Cheers,
Phil.

Re: Replacing python venv - environment or profile?

[zimoun]

Hi Phil,

On Tue, 24 Nov 2020 at 17:22, Phil <phil@beadling.co.uk> wrote:

> I'm new the Guix and considering adopting it as an expansion to my
> current use of Python virtual environments in development and
> production, to include other non-Python packages too.

Nice!


> I've read the manual/cookbook and I get the gist of the tool.  One thing
> I'm getting a bit stuck on is the appropriate use of environment vs
> profile; different sources give slightly different takes on the theme, and I
> want to make sure my use is correct from the get-go.

Initially, coming from Python terminology, I was confused too.

Well roughly, an environment is a temporary profile.  So, in the Python
terminology, the “environment” corresponds to the Guix profile.  And the
Guix environment does not have an equivalent in the Python terminology,
AFAIK.


> I'll be using Guix on a foreign OS, I suspect the answers might be
> slightly different if I was able to adopt the Guix System OS, but for
> now I can't.

No, it should change nothing.  IIUC.

>
> As well as the manual cookbook I came across this blog:
> https://trivialfis.github.io/linux/2018/06/10/Using-guix-for-development.html

Thanks for the pointer.  I have not read it yet.


> I understand that I can use the --pure switch to ensure no pollution

To understand the difference between ’--pure’ or not:

       $ guix environment --ad-hoc python
 [env] $ echo $PATH
vs
       $ guix environment --ad-hoc python --pure
 [env] $ echo $PATH

And this is done for all the environment variables.


> This is great - but the manual points out that none of the
> installed packages are marked such that they avoid garbage collection.

The Guix environment are temporary.  Compare:

a  $ guix environment --ad-hoc hello -- hello
b  $ guix gc --list-dead | grep hello
c  $ guix gc --list-live | grep hello

d  $ guix install hello
e  $ guix gc --list-dead | grep hello
f  $ guix gc --list-live | grep hello

The a create a temporary profile (environment) containing the package
hello and then the program hello is run inside.

The b lists all the items in the store that are not used by any profile,
so they could be garbage collected because nothing need them.  The c
lists all the live items, i.e, all the item used by one or more profile,
so they cannot be garbage collected.

The d installs hello in the default profile (~/.guix-profile), so now
the e shows nothing because one profile uses it and the f shows.

Does it make sense?


The next point is that you can create any profile (as the Python
environment).  For example,


  guix install python python-mypy -p ~/my-first-profile


The big difference is “guix pull”.  Well, imagine this sequence:

  guix environment --ad-hoc python
  [..work..]
  guix pull
  guix environment --ad-hoc python

Now, the first Python is not necessary the exact same than the second
one.  Considering the sequence,

  guix install python -p /my-profile
  [..work..]
  guix pull
  eval $(guix package --search-paths=prefix -p ~/my-profile)

The Python inside your profile ~/my-profile has not changed.


> So I came up with something like the below - manifests do seem to be an
> exact analogy to Python requirements.txt files:

Yes, manifest.scm is equivalent to requirements.txt.  The only
difference is that power because it is possible to write Scheme inside
this file and so apply package transformation, create on-the-fly package
variant, etc.  Well, another story. :-)


> $ cat manifest.scm 
> (specifications->manifest
>   '("coreutils"
>     "emacs@27.1"
>     "python"
>     "python-pytest"
>     "python-coverage"
>     "python-pytest-cov"
>     "python-black"
>     "python-mypy"
>     "python-flake8"))
> $ guix environment --pure --manifest=manifest.scm --root=./test-profile

I am not convinced you need the --root at first.


> This creates a profile as a side-effect, I think.

This create a environment, not a profile.  For a profile, you should do:

  guix package -m manifest.scm -p ./test-profile


> So to my mind Guix profiles are more like Python virtual environments with
> the with the non-standard --system-site-packages switch than the
> classic

Yes.

> use of Python virtual environment?  Useful, but different.  The 'guix
> environment' is accidentally providing a closer parallel.

AFAIK, the Guix environment has no equivalent.  For example, you can
create a Guix environment in a container; --pure on steroids. :-)


> Finally.... I get to my question!  So in light of the above (assuming I
> haven't missed the point completely!), what is the canonical way of
> isolating a virtual environment using Guix on a foreign OS
> installation?

Profile and then the correct --search-paths.

But if you want crazy isolation with container, you can do:

  guix describe -f channels > my-channels.scm

then any point in time, you will have the exact same environment with:

  guix time-machine -C my-channels -- environment -C -m my-manifest.scm


> Last point - ultimately as well as using Guix to provide a consistent
> profile/environment to do Python work in, I'd ultimately
> like to hook Guix up to Jenkins so that it can package and deploy a
> repo to a

That’s a bigger story…

> production server.  This probably involves setting-up a private Guix
> Channel and importing built wheels, and to then pull the new Guix package from the
> production server.  Ultimately tho I'd want the same thing - a pure
> environment in production which has access to exactly and only the
> software as described in the manifest.  This feels like it should be
> possible too, but again, just looking for any initial guidance on which features
> of Guix to use to do this.

…so let’s discuss it once you are comfortable with Guix on your local
machine. ;-)


All the best,
simon

Re: Replacing python venv - environment or profile?

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1470 bytes --]


Phil <phil@beadling.co.uk> writes:

> I'm new the Guix and considering adopting it as an expansion to my
> current use of Python virtual environments in development and
> production, to include other non-Python packages too.

As far as I see it, guix environment is what you get from a virtualenv,
plus a good deal more. I start more complex programs nowadays by pulling
in their dependencies in a small startup file. The following is my
actual command for running IntelliJ with custom libraries and the
correct libstdc++:

exec -a "$0" guix environment --ad-hoc openjdk@14 openjdk@14:jdk gcc-toolchain@10.2 -- bash -c "LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/home/arne/.guix-profile/lib:$(realpath ~/.guix-profile/lib/libsqlite3.so):/gnu/store/qj38f3vi4q1d7z30hkpaxyajv49rwamb-gcc-10.2.0-lib/lib/:."' IDEA_JDK="$(dirname $(dirname $(realpath $(which java))))" exec -a "$0" bash -x idea-IU-193.6911.18/bin/idea.sh'


I used the environment for something closer to your example for actual
day-work to get a local environment with java 8 and documented that
briefly:

https://www.draketo.de/software/guix-work.html#org07ebedd

    # download and unpack tomcat and enter the folder, then call
    guix environment --ad-hoc icedtea@3.7.0
    bin/startup.sh

I know that this 3-liner does not look so awesome, but it is crazy how
many problems it can solve :-)


Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein
ohne es zu merken

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: Replacing python venv - environment or profile?

[Christopher Lam]

No constructive comments here, except a thank you for sharing an
incremental tutorial on using guix.

Sorely needed:
- similar incremental tutorial for C based projects.
- similar tutorial for other text editors eg vscode


On Tue, 24 Nov 2020 at 18:35, Phil <phil@beadling.co.uk> wrote:

> Hi all,
>
> Apologies this is a bit longer than I anticipated, but I wanted to record
> all the steps I'd taken to explain my reasoning - in case it's wrong!
>
> I'm new the Guix and considering adopting it as an expansion to my
> current use of Python virtual environments in development and
> production, to include other non-Python packages too.
>
> The potential of the software is very exciting!
>
> I've read the manual/cookbook and I get the gist of the tool.  One thing
> I'm getting a bit stuck on is the appropriate use of environment vs
> profile; different sources give slightly different takes on the theme, and
> I
> want to make sure my use is correct from the get-go.
>
> I'll be using Guix on a foreign OS, I suspect the answers might be
> slightly different if I was able to adopt the Guix System OS, but for
> now I can't.
>
> As well as the manual cookbook I came across this blog:
>
> https://trivialfis.github.io/linux/2018/06/10/Using-guix-for-development.html
>
> This is describing the use of 'guix environment' and manifests to create a
> structure very similar to a python virtual environment, but including
> the python package itself too.  Which is exactly the starting point I had
> in mind.
>
> I understand that I can use the --pure switch to ensure no pollution
> from the foreign OS programs - just like the default behaviour of python
> venv, eg:
>
> ubuntu@primary:~$ guix environment --ad-hoc coreutils --pure
>
> ubuntu@primary:~$ ls
> dev
> ubuntu@primary:~$ nano
> Command 'nano' is available in the following places
>  * /bin/nano
>  * /usr/bin/nano
> The command could not be located because '/bin:/usr/bin' is not included
> in the PATH environment variable.
> nano: command not found
>
>
> This is great - but the manual points out that none of the
> installed packages are marked such that they avoid garbage collection.
>
> So I came up with something like the below - manifests do seem to be an
> exact analogy to Python requirements.txt files:
>
> $ cat manifest.scm
> (specifications->manifest
>   '("coreutils"
>     "emacs@27.1"
>     "python"
>     "python-pytest"
>     "python-coverage"
>     "python-pytest-cov"
>     "python-black"
>     "python-mypy"
>     "python-flake8"))
> $ guix environment --pure --manifest=manifest.scm --root=./test-profile
>
> This creates a profile as a side-effect, I think.
>
> This would be all well and good but both the manual and the cookbook,
> and a few other sources I've found online seem to equate a *profile* with
> a virtualenv rather than use of the *guix environment* command.
>
> To test this as an alternative, I source the profile created by the
> manifest above:
>
> $ export GUIX_PROFILE=/home/ubuntu/dev/test-profile
> $ . $GUIX_PROFILE/etc/profile
>
> I note that unlike the environment which creates what I assume is its
> own temporary profile, eg:
>
> $ echo $PATH
> /gnu/store/hchmga9ybpdc4zph9cs8jr7m1k8gxw9f-profile/bin
>
> Sourcing the created profile references both it, the default profile,
> and guix's profile, followed by the foreign OS PATH variables:
>
> $ echo $PATH
>
> /home/ubuntu/dev/test-profile/bin:/home/ubuntu/.guix-profile/bin:/home/ubuntu/.config/guix/current/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
>
>
> I'm keen to avoid accidentally calling anything in the foreign OS,
> something the 'guix environment' command gives me, but obviously the
> foreign
> OS will serve as a fallback given the above PATH construction under a
> sourced
> profile.  So if my profile doesn't have python installed, but it is part
> of the foreign OS, I would silently pick that up, which would be bad.
>
>
> As per 4.1.1 in the cookbook I can avoid setting other profiles using:
>
> $ env -i $(which bash) --login --noprofile --norc
> $ export GUIX_PROFILE=/home/ubuntu/dev/test-profile
> $ . $GUIX_PROFILE/etc/profile
>
> But the foreign OS variables remain:
>
> $ echo $PATH
>
> /home/ubuntu/dev/test-profile/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
> $
>
> So to my mind Guix profiles are more like Python virtual environments with
> the with the non-standard --system-site-packages switch than the classic
> use of Python virtual environment?  Useful, but different.  The 'guix
> environment' is accidentally providing a closer parallel.
>
> I'm guessing that if I used the Guix System OS the profile analogy would
> be much
> more accurate as there would be no foreign OS to fallback onto?
>
> Finally.... I get to my question!  So in light of the above (assuming I
> haven't missed the point completely!), what is the canonical way of
> isolating a virtual environment using Guix on a foreign OS installation?
>
> Is the use of 'guix environment' as per the blog referenced above
> considered good practice, or is this as I now suspect, inappropriate
> re-purposing of a
> feature designed to create transient environments for building/debugging
> specific
> programs packaged in Guix?
>
> Links such as this suggest this might be the case:
>
> https://yhetil.org/guix-user/1700d451826.11317682011034.4058430466276292306@zoho.com/T/
>
> Of course I could probably write a script that scrubs my own
> environment variables of all references to the foreign OS, but this feels
> like I'm doing what I thought I'd get for free with Guix in the first
> place?
>
> Last point - ultimately as well as using Guix to provide a consistent
> profile/environment to do Python work in, I'd ultimately
> like to hook Guix up to Jenkins so that it can package and deploy a repo
> to a
> production server.  This probably involves setting-up a private Guix
> Channel and importing built wheels, and to then pull the new Guix package
> from the
> production server.  Ultimately tho I'd want the same thing - a pure
> environment in production which has access to exactly and only the
> software as described in the manifest.  This feels like it should be
> possible too, but again, just looking for any initial guidance on which
> features
> of Guix to use to do this.
>
> Sorry for the long e-mail, and thanks in advance for any guidance
> or advice on the matter!
>
>
> Cheers,
> Phil.
>
>

Re: Replacing python venv - environment or profile?

[zimoun]

Hi,

On Wed, 25 Nov 2020 at 14:37, Christopher Lam <christopher.lck@gmail.com> wrote:

> - similar tutorial for other text editors eg vscode

What do you mean?

About Emacs, the package Emacs-Guix (which needs love!) helps to
manage profiles, environments, etc.  Well, when it is not broken [1].

Since I have never used VSCode (and I will never do, almost surely),
could you indicate what/how VSCode does with Python tools (virtualenv,
conda, etc.)?  And about C tools since you seems interested in?


1: <https://yhetil.org/guix-devel/87sg9dxnvr.fsf@asu.edu>

Thanks,
simon

Re: Replacing python venv - environment or profile?

[Phil]

Thanks Simon for the helpful comments - everything's hopefully clicked now - some
comments inline.


zimoun writes:


> Well roughly, an environment is a temporary profile.  So, in the Python
> terminology, the “environment” corresponds to the Guix profile.  And the
> Guix environment does not have an equivalent in the Python terminology,
> AFAIK.

Yes - I think it was the fact 'guix environment' had the '--pure' flag
that initially drew me in to consider this to be the parallel with
Python's virtual environment (and the original blog nudged me in that
direction too).

> To understand the difference between ’--pure’ or not:
>
>        $ guix environment --ad-hoc python
>  [env] $ echo $PATH
> vs
>        $ guix environment --ad-hoc python --pure
>  [env] $ echo $PATH
>
> And this is done for all the environment variables.

Yes this makes perfect sense to me, I guess what I was missing was that
a 'guix environment' is just a temporary and special case of sourcing a
Guix profile.

However the "--pure" switch means it has a useful feature that
originally I didn't appreciate was available when sourcing a profile by
using "--search-paths=exact" - note: the penny dropped for me below when reading your
e-mail, I've removed some of my own comments here about "--pure" now that I
understand it is not the only way to get a clean environment from a profile!

>   guix install python python-mypy -p ~/my-first-profile
>
>
> The big difference is “guix pull”.  Well, imagine this sequence:
>
>   guix environment --ad-hoc python
>   [..work..]
>   guix pull
>   guix environment --ad-hoc python
>
> Now, the first Python is not necessary the exact same than the second
> one.  Considering the sequence,

Yep you're creating two separate environments each taken from the
current 'latest' python version.  Which may have been updated by guix
pull for example.

>
>   guix install python -p /my-profile
>   [..work..]
>   guix pull
>   eval $(guix package --search-paths=prefix -p ~/my-profile)
>
> The Python inside your profile ~/my-profile has not changed.

Yes - I get this now.  The eval trick is nice.

>> $ guix environment --pure --manifest=manifest.scm --root=./test-profile
>
> I am not convinced you need the --root at first.

This is purely a way of creating a pure environment from a manifest, and
creating a profile from that same manifest as a one-liner (to pin the
packages in the store).  If you'd previously created a profile with the
same manifest this would be unnecessary.

> This create a environment, not a profile.  For a profile, you should do:
>
>   guix package -m manifest.scm -p ./test-profile

Yep - so this does the same thing as the above, but doesn't then put me
into a 'guix environment' referencing the same pacakges as the
test-profile does.

>
>> Finally.... I get to my question!  So in light of the above (assuming I
>> haven't missed the point completely!), what is the canonical way of
>> isolating a virtual environment using Guix on a foreign OS
>> installation?
>
> Profile and then the correct --search-paths.

Ah-ha! Now the penny has dropped - this was the piece of the puzzle I
had missed!

$ guix package --search-paths=exact -p ./test-profile
export PATH="./test-profile/bin"
export EMACSLOADPATH="./test-profile/share/emacs/site-lisp:./test-profile/share/emacs/27.1/lisp"
export INFOPATH="./test-profile/share/info"
export PYTHONPATH="./test-profile/lib/python3.8/site-packages"

So, if I instantiate the environment using "--search-paths=exact" then I
get *pure* environment variables, rather than appending or prepending to
the current variables.

So using this I don't need "guix environment --pure" anymore!  Which was
what I wanted.


> But if you want crazy isolation with container, you can do:
>
>   guix describe -f channels > my-channels.scm
>
> then any point in time, you will have the exact same environment with:
>
>   guix time-machine -C my-channels -- environment -C -m my-manifest.scm

This is snapshotting not just my manifest, but also the exact state of
the Guix commands themselves - so the environment and the tooling for
that environment are both replicated?


>
>> Last point - ultimately as well as using Guix to provide a consistent
>> profile/environment to do Python work in, I'd ultimately
>> like to hook Guix up to Jenkins so that it can package and deploy a
>> repo to a
>
> That’s a bigger story…

Yes agreed - let's discuss later, I just wanted to give the group a
vague idea of where I'm hopefully ultimately heading with my Guix use,
but I must walk before I run!

Re: Replacing python venv - environment or profile?

[Phil]

Much appreciated Arne - lots of interesting stuff on your website - a few
comments below too.

Dr. Arne Babenhauserheide writes:


> As far as I see it, guix environment is what you get from a virtualenv,
> plus a good deal more. I start more complex programs nowadays by pulling
> in their dependencies in a small startup file. The following is my
> actual command for running IntelliJ with custom libraries and the
> correct libstdc++:
>
> exec -a "$0" guix environment --ad-hoc openjdk@14 openjdk@14:jdk gcc-toolchain@10.2 -- bash -c "LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/home/arne/.guix-profile/lib:$(realpath ~/.guix-profile/lib/libsqlite3.so):/gnu/store/qj38f3vi4q1d7z30hkpaxyajv49rwamb-gcc-10.2.0-lib/lib/:."' IDEA_JDK="$(dirname $(dirname $(realpath $(which java))))" exec -a "$0" bash -x idea-IU-193.6911.18/bin/idea.sh'

Cool so Intellij starts up inside it's own environment with specific
toolchain and JDK requirements.

One question - is there any reason to use 'guix environment' rather than
have a profile setup with packages you need that you could then source
and start idea.sh from?

Re: Replacing python venv - environment or profile?

[zimoun]

Hi,

On Wed, 25 Nov 2020 at 20:21, Phil <phil@beadling.co.uk> wrote:

> Yes this makes perfect sense to me, I guess what I was missing was that
> a 'guix environment' is just a temporary and special case of sourcing a
> Guix profile.

Yes, if you are curious by the internal machinery.  The default profile
~/.guix-profile is a symlink

  $ readlink -f ~/.guix-profile
  /gnu/store/z32aid8bjbz7ylwbjfh5x6lbg2g7fiv0-profile

Now compare:

  $ guix environment --ad-hoc hello
  [env]$ echo $GUIX_ENVIRONMENT
  /gnu/store/hgh67ilwsbx6v2irc7vgrnv354cv4h8a-profile

but no symlink points to the environment temporary profile, so once out,
the GC can do its job.

To end with the internals, give a look to the file:

  /gnu/store/…-profile/etc/profile

it describes how the environment variables are set.


>>   guix environment --ad-hoc python
>>   [..work..]
>>   guix pull
>>   guix environment --ad-hoc python
>>
>> Now, the first Python is not necessary the exact same than the second
>> one.  Considering the sequence,
>
> Yep you're creating two separate environments each taken from the
> current 'latest' python version.  Which may have been updated by guix
> pull for example.

Yes, exactly.


>>> $ guix environment --pure --manifest=manifest.scm --root=./test-profile
>>
>> I am not convinced you need the --root at first.
>
> This is purely a way of creating a pure environment from a manifest, and
> creating a profile from that same manifest as a one-liner (to pin the
> packages in the store).  If you'd previously created a profile with the
> same manifest this would be unnecessary.

Yes.  My point was to have at first an explanation about environment vs
profile, trying to avoid to mix all the concept in the same time. :-)


>> This create a environment, not a profile.  For a profile, you should do:
>>
>>   guix package -m manifest.scm -p ./test-profile
>
> Yep - so this does the same thing as the above, but doesn't then put me
> into a 'guix environment' referencing the same pacakges as the
> test-profile does.

I am not sure that understand what you mean.

  $ guix environment --pure --manifest=manifest.scm --root=./test-profile
<=>
  $ guix package -m manifest.scm -p ./test-profile
  $ eval $(guix package –search-paths=exact)

I almost never use the former, i.e., options -m and -r in the same
line.  And I prefer the latter.  Just a matter of taste.

However, I use “guix environment hello -r /tmp/guix”.  I let you notice
the difference. ;-)


> So, if I instantiate the environment using "--search-paths=exact" then I
> get *pure* environment variables, rather than appending or prepending to
> the current variables.

Yes, even if I do not know if “instantiate the environment” is the
correct terminology here. :-)


> So using this I don't need "guix environment --pure" anymore!  Which was
> what I wanted.

It depends.  Sometimes, you want to temporary extend a profile so “guix
environment” is really useful.  Or run in a container.  Or have all the
dependencies necessary of one package, for example:

  guix environment python-mypy --ad-hoc python-numpy


>> But if you want crazy isolation with container, you can do:
>>
>>   guix describe -f channels > my-channels.scm

Run

  guix help
  guix describe

and then the commit capture everything of Guix.  So at any point in
time, you can still run:

   guix time-machine –commit=123456 – help

and it will be the exact same code running “help” that the one running
now.  And any command works: install, environment, pack, system, etc.


>> then any point in time, you will have the exact same environment with:
>>
>>   guix time-machine -C my-channels -- environment -C -m my-manifest.scm
>
> This is snapshotting not just my manifest, but also the exact state of
> the Guix commands themselves - so the environment and the tooling for
> that environment are both replicated?

Yes.

In a repo, you have to track 2 things: the manifest.scm about the
packages you need **and** the channels.scm file which is the correct
description of their versions (and the versions of all the machinery
used to bootstrap, compile or byte-compile them).

This way, you can regenerate the exact same “environment” at any time
and any where, for example:

 guix time-machine -C my-channels -- pack -f docker -m my-manifest.scm


> Yes agreed - let's discuss later, I just wanted to give the group a
> vague idea of where I'm hopefully ultimately heading with my Guix use,
> but I must walk before I run!

Sound a cool goal!


All the best,
simon

Re: Replacing python venv - environment or profile?

[Christopher Lam]

I'd like to reproduce my current dev environment.

Dependencies are https://wiki.gnucash.org/wiki/Installing_Dependencies
My dev environment is emacs + magit ivy etc

I could reuse the recipe in `guix edit gnucash` but instead to build from
scratch to gain a deeper understanding. The guix reference is too
interconnected.

On Wed, 25 Nov 2020 at 15:42, zimoun <zimon.toutoune@gmail.com> wrote:

> Hi,
>
> On Wed, 25 Nov 2020 at 14:37, Christopher Lam <christopher.lck@gmail.com>
> wrote:
>
> > - similar tutorial for other text editors eg vscode
>
> What do you mean?
>
> About Emacs, the package Emacs-Guix (which needs love!) helps to
> manage profiles, environments, etc.  Well, when it is not broken [1].
>
> Since I have never used VSCode (and I will never do, almost surely),
> could you indicate what/how VSCode does with Python tools (virtualenv,
> conda, etc.)?  And about C tools since you seems interested in?
>
>
> 1: <https://yhetil.org/guix-devel/87sg9dxnvr.fsf@asu.edu>
>
> Thanks,
> simon
>

Re: Replacing python venv - environment or profile?

[zimoun]

Dear,


On Wed, 25 Nov 2020 at 22:54, Christopher Lam <christopher.lck@gmail.com> wrote:
> I'd like to reproduce my current dev environment.

Cool!  Please report if you hit troubles.


> Dependencies are https://wiki.gnucash.org/wiki/Installing_Dependencies

The command

   guix environment gnucash

instantiates an environment with all the inputs necessary to build it;
namely, all the inputs that Guix uses to build it.  And if you want to
create a persistent profile with these dependencies:

  guix environment gnucash -r ~/gnucash-profile


> My dev environment is emacs + magit ivy etc

I miss your point and what you have tried to mean with:

    - similar tutorial for other text editors eg vscode


All the best,
simon

Re: Replacing python venv - environment or profile?

[Maxim Cournoyer]

Hi Phil,

Phil <phil@beadling.co.uk> writes:

> Hi all,
>
> Apologies this is a bit longer than I anticipated, but I wanted to record
> all the steps I'd taken to explain my reasoning - in case it's wrong!
>
> I'm new the Guix and considering adopting it as an expansion to my
> current use of Python virtual environments in development and
> production, to include other non-Python packages too.
>
> The potential of the software is very exciting!
>
> I've read the manual/cookbook and I get the gist of the tool.  One thing
> I'm getting a bit stuck on is the appropriate use of environment vs
> profile; different sources give slightly different takes on the theme, and I
> want to make sure my use is correct from the get-go.

It seems you've done your homework :-).

> I'll be using Guix on a foreign OS, I suspect the answers might be
> slightly different if I was able to adopt the Guix System OS, but for
> now I can't.
>

[...]

> I'm keen to avoid accidentally calling anything in the foreign OS,
> something the 'guix environment' command gives me, but obviously the foreign
> OS will serve as a fallback given the above PATH construction under a sourced
> profile.  So if my profile doesn't have python installed, but it is part
> of the foreign OS, I would silently pick that up, which would be bad.
>
>
> As per 4.1.1 in the cookbook I can avoid setting other profiles using:
>
> $ env -i $(which bash) --login --noprofile --norc
> $ export GUIX_PROFILE=/home/ubuntu/dev/test-profile
> $ . $GUIX_PROFILE/etc/profile
>
> But the foreign OS variables remain:
>
> $ echo $PATH
> /home/ubuntu/dev/test-profile/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
> $ 

I could reproduce on a foreign distro but not on Guix System:

--8<---------------cut here---------------start------------->8---
$ env -i $(which bash) -c 'echo $PATH'
/no-such-path
--8<---------------cut here---------------end--------------->8---

It appears this is caused by the foreign Bash, which hard-codes a
default value for PATH
(/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin in your
case).

The Bash package on Guix (you can go to its definition using 'guix edit
bash') has the following compilation flags:

  (let* ((cppflags (string-join '("-DDEFAULT_PATH_VALUE='\"/no-such-path\"'"
                                  "-DSTANDARD_UTILS_PATH='\"/no-such-path\"'"
                                  "-DNON_INTERACTIVE_LOGIN_SHELLS"
                                  "-DSSH_SOURCE_BASHRC")
                                " "))

This is where that non-existing (/no-such-path) value comes from.

As a workaround, you could use Bash from Guix after installing it with 'guix install bash'.

HTH!

Thanks,

Maxim