How can I replace Python venv and pip with Guix?

How can I replace Python venv and pip with Guix?

[sirgazil]

Hi,

I'm currently using two package managers when working on my projects written in Python. I use Guix to install Python, and Python pip to install Python packages in project environments created with Python venv. So, for working on a specific project, I do the following:


#+BEGIN_EXAMPLE
# Create an environment (one-time operation)
$ python3 -m venv ~/.virtualenvs/my-project

# Activate the environment
$ source ~/.virtualenvs/my-project/bin/activate
(my-project) $

# Install packages in the environment
(my-project) $ pip install -r packages.txt

# Deactivate the environment (Ctrl+D)
$ deactivate
#+END_EXAMPLE


Emacs Elpy detects the virtual environments I create in this way, so I can activate and deactivate them from Emacs with an easy to use interface and have code completion that depends on the packages available in the active environment.

I'd like to use Guix and its environments to replace this functionality and to make my development environments easier to reproduce, but I don't understand how to that exactly.

So far I have tried the "guix environment" command as follows, trying to create a persistent environment I can activate/deactivate whenever I want:

#+BEGIN_EXAMPLE
$ guix environment --pure --manifest=guix.scm --root=/path/to/my-guix-envs/my-project
#+END_EXAMPLE

After running that command, I can see the prompt adds a "[env]" label indicating that I'm in the environment, and I can use the packages specified in the manifest file. But

• How could one make the prompt add "[my-project]" instead of the generic "[env]"?
• Once I deactivate this environment (Ctrl+D), how can I activate it again?


Thanks,



---
https://sirgazil.bitbucket.io/

Re: How can I replace Python venv and pip with Guix?

[Zelphir Kaltstahl]

Hi,

I know you asked something else, but want to mention this: You could use
Poetry to make reproducible environments for Python. Only yesterday I
have been through the process of building Python 3.8.1 on Trisquel and
getting Poetry installed through Pip.

Also if it is any help, I can share my scripts for building Python with
everyone. I am planning to upload them to some repo later anyway.

Regards,
Zelphir

On 2/3/20 6:00 PM, help-guix-request@gnu.org wrote:
> Hi,
>
> I'm currently using two package managers when working on my projects written in Python. I use Guix to install Python, and Python pip to install Python packages in project environments created with Python venv. So, for working on a specific project, I do the following:
>
>
> #+BEGIN_EXAMPLE
> # Create an environment (one-time operation)
> $ python3 -m venv ~/.virtualenvs/my-project
>
> # Activate the environment
> $ source ~/.virtualenvs/my-project/bin/activate
> (my-project) $
>
> # Install packages in the environment
> (my-project) $ pip install -r packages.txt
>
> # Deactivate the environment (Ctrl+D)
> $ deactivate
> #+END_EXAMPLE
>
>
> Emacs Elpy detects the virtual environments I create in this way, so I can activate and deactivate them from Emacs with an easy to use interface and have code completion that depends on the packages available in the active environment.
>
> I'd like to use Guix and its environments to replace this functionality and to make my development environments easier to reproduce, but I don't understand how to that exactly.
>
> So far I have tried the "guix environment" command as follows, trying to create a persistent environment I can activate/deactivate whenever I want:
>
> #+BEGIN_EXAMPLE
> $ guix environment --pure --manifest=guix.scm --root=/path/to/my-guix-envs/my-project
> #+END_EXAMPLE
>
> After running that command, I can see the prompt adds a "[env]" label indicating that I'm in the environment, and I can use the packages specified in the manifest file. But
>
> • How could one make the prompt add "[my-project]" instead of the generic "[env]"?
> • Once I deactivate this environment (Ctrl+D), how can I activate it again?
>
>
> Thanks,
>
>
>
> ---
> https://sirgazil.bitbucket.io/

Re: How can I replace Python venv and pip with Guix?

[sirgazil]

 ---- On Mon, 03 Feb 2020 16:09:19 -0500 Zelphir Kaltstahl <zelphirkaltstahl@gmail.com> wrote ----
 > Hi,
 > 
 > I know you asked something else, but want to mention this: You could use
 > Poetry to make reproducible environments for Python. Only yesterday I
 > have been through the process of building Python 3.8.1 on Trisquel and
 > getting Poetry installed through Pip.
 > 
 > Also if it is any help, I can share my scripts for building Python with
 > everyone. I am planning to upload them to some repo later anyway.


Thanks for the information, Zelphir, I didn't know about Poetry. But yeah, I want to use Guix for package management and for dealing with development environments, and for distribution, and for deployment, and for whatever else it can help me with :)

Re: How can I replace Python venv and pip with Guix?

[Moritz Lell]

Hello!


> • Once I deactivate this environment (Ctrl+D), how can I activate it
>again?

You've already achieved that, as far as I understand. Because you use the --root option, the profile is registered in GUIX and will not be garbage collected. See

    guix gc --list-roots


Cheers,
Moritz

Re: How can I replace Python venv and pip with Guix?

[Tanguy Le Carrour]

Hi,

Le 02/02, sirgazil a écrit :
> I'm currently using two package managers when working on my projects
> written in Python. I use Guix to install Python,

Depending on your OS, you can also use Pyenv [1][] to install what
ever version of Python you like. I haven't figured out how to package it
for Guix, though.

[1]: https://github.com/pyenv/pyenv


> […] and Python pip to install

As Zelphir mentioned, you should definitively use Poetry to manage your
Python dependencies! Poetry 0.12.17 is available in Guix and version 1.0.3 is on
it's way!
Poetry manages the dependencies and the packaging. It will create the
`setup.py` with the appropriate `install_requires`.


> […]
> Emacs Elpy detects the virtual environments I create in this way, so I can
> activate and deactivate them from Emacs with an easy to use interface and
> have code completion that depends on the packages available in the active
> environment.
> 
> I'd like to use Guix and its environments to replace this functionality and
> to make my development environments easier to reproduce, but I don't
> understand how to that exactly.
> 
> So far I have tried the "guix environment" command as follows, trying to
> create a persistent environment I can activate/deactivate whenever I want:
> 
> #+BEGIN_EXAMPLE
> $ guix environment --pure --manifest=guix.scm --root=/path/to/my-guix-envs/my-project
> #+END_EXAMPLE

If you want to create a persistent environment, you should use a profile,
not the `environment` command.
There's a good cookbook entry for that [2][].

[2]: https://guix.gnu.org/cookbook/en/html_node/Guix-Profiles-in-Practice.html#Guix-Profiles-in-Practice


> After running that command, I can see the prompt adds a "[env]" label
> indicating that I'm in the environment, and I can use the packages specified
> in the manifest file. But
> 
> • How could one make the prompt add "[my-project]" instead of the generic
>   "[env]"?

I would say this depends on the shell you are using. If you are a Bash
user, you have to set the PS1 variable. If you use Fish [3][], you have to set your
prompt in a config file: `~/.config/fish/functions/fish_prompt.fish`
and/or `~/.config/fish/functions/fish_right_prompt.fish`.

[3]: https://fishshell.com/docs/current/tutorial.html#tut_prompt


> • Once I deactivate this environment (Ctrl+D), how can I activate it again?

I personally use Direnv [4][] to manage this. Whenever I `cd` to a
project folder, it creates and/or activates the venv and deactivates it
when you `cd` out of it.

[4]: https://github.com/direnv/direnv/wiki/Python


Using poetry + pyenv + direnv might be a bit confusing at first, but it's
really worth a try!

Regards

-- 
Tanguy

Re: How can I replace Python venv and pip with Guix?

[sirgazil]

---- On Tue, 04 Feb 2020 02:36:39 -0500 Moritz Lell <mlell@posteo.de> wrote ----
 > Hello!

Hi, Moritz.

 > > • Once I deactivate this environment (Ctrl+D), how can I activate it
 > >again?
 > 
 > You've already achieved that, as far as I understand. Because you use the --root option, the profile is registered in GUIX and will not be garbage collected. See

Well, the profile is created and persists, but, after deactivating the environment, how do I start an environment that uses that profile? 

I'll take a look at what tanguy says, though (not using the environment command for this at all).

Re: How can I replace Python venv and pip with Guix?

[Ricardo Wurmus]

sirgazil <sirgazil@zoho.com> writes:

> • Once I deactivate this environment (Ctrl+D), how can I activate it again?

It’s a matter of evaluating the etc/profile file that is generated for
every Guix profile.  Since you passed

    --root=/path/to/my-guix-envs/my-project

to “guix environment” there will be a file

    /path/to/my-guix-envs/my-project/etc/profile

You can set all the environment variables it defines with

    GUIX_PROFILE=/path/to/my-guix-envs/my-project
    source $GUIX_PROFILE/etc/profile

If you want a pure environment you can use “env”.  On our HPC cluster I
provide “activate” scripts like this one:

--8<---------------cut here---------------start------------->8---
#!/bin/bash

profile=/path/to/.guix-profile
prompt='\u@\h:\W \[\e[31;47;1m\][pigx]\[\e[m\] $ '

REAL_HOME=$HOME
exec /bin/env - PS1="$prompt" \
                HOME=$REAL_HOME \
                LANG=en_US.UTF-8 \
                GUIX_LOCPATH="$profile/lib/locale" \
                CURL_CA_BUNDLE="/etc/ssl/certs/ca-bundle.crt" \
                /bin/bash --init-file "$profile/etc/profile"
--8<---------------cut here---------------end--------------->8---

It sets a few environment variables, recovers some variables that are
important, and then starts a sub-shell that uses the generated
etc/profile file as its init file.

Something like that might work for you.

--
Ricardo

Re: How can I replace Python venv and pip with Guix?

[sirgazil]

 ---- On Tue, 04 Feb 2020 02:59:25 -0500 Tanguy Le Carrour <tanguy@bioneland.org> wrote ----
 > Hi,
 > 
 > Le 02/02, sirgazil a écrit :
 > > I'm currently using two package managers when working on my projects
 > > written in Python. I use Guix to install Python,
 >
 > Depending on your OS, you can also use Pyenv [1][] to install what
 > ever version of Python you like. I haven't figured out how to package it
 > for Guix, though.
 > 
 > [1]: https://github.com/pyenv/pyenv

I only mentioned Python here as something I install with the OS package manager, but I actually install other things that are not packaged for the language package manager. And I wanted to see if I can use Guix instead of several package managers.


 > > […]
 > > Emacs Elpy detects the virtual environments I create in this way, so I can
 > > activate and deactivate them from Emacs with an easy to use interface and
 > > have code completion that depends on the packages available in the active
 > > environment.
 > > 
 > > I'd like to use Guix and its environments to replace this functionality and
 > > to make my development environments easier to reproduce, but I don't
 > > understand how to that exactly.
 > > 
 > > So far I have tried the "guix environment" command as follows, trying to
 > > create a persistent environment I can activate/deactivate whenever I want:
 > > 
 > > #+BEGIN_EXAMPLE
 > > $ guix environment --pure --manifest=guix.scm --root=/path/to/my-guix-envs/my-project
 > > #+END_EXAMPLE
 > 
 > If you want to create a persistent environment, you should use a profile,
 > not the `environment` command.
 > There's a good cookbook entry for that [2][].
 > 
 > [2]: https://guix.gnu.org/cookbook/en/html_node/Guix-Profiles-in-Practice.html#Guix-Profiles-in-Practice


I don't know how I missed this information. I'll take a look, thanks :)


 > > After running that command, I can see the prompt adds a "[env]" label
 > > indicating that I'm in the environment, and I can use the packages specified
 > > in the manifest file. But
 > > 
 > > • How could one make the prompt add "[my-project]" instead of the generic
 > >   "[env]"?
 > 
 > I would say this depends on the shell you are using. If you are a Bash
 > user, you have to set the PS1 variable. If you use Fish [3][], you have to set your
 > prompt in a config file: `~/.config/fish/functions/fish_prompt.fish`
 > and/or `~/.config/fish/functions/fish_right_prompt.fish`.


Sorry, what I wanted to say is that I would expect that label to change automatically to "[my-project]" when I use the "--root" option.


 > > • Once I deactivate this environment (Ctrl+D), how can I activate it again?
 > 
 > I personally use Direnv [4][] to manage this. Whenever I `cd` to a
 > project folder, it creates and/or activates the venv and deactivates it
 > when you `cd` out of it.
 > 
 > [4]: https://github.com/direnv/direnv/wiki/Python
 > 
 > 
 > Using poetry + pyenv + direnv might be a bit confusing at first, but it's
 > really worth a try!

I will give all that a try if Guix doesn't satisfy my needs.

Thanks, Tanguy :)

Re: How can I replace Python venv and pip with Guix?

[mlell]

Hi sirgazil,

> Well, the profile is created and persists, but, after deactivating the
> environment, how do I start an environment that uses that profile?
> 


If you want to use the isolation features of `guix environment`, like 
running inside a container,
you can just use the same command that you used for creating the 
profile.  Guix will not rebuild
anything because all derivations are already in the store from the first 
time you executed this,
so

     guix environment --pure --manifest=guix.scm 
--root=/path/to/my-guix-envs/my-project

Otherwise, you can use the standard way that you can use to load any 
GUIX profile: You source
the <profile>/etc/profile file into your shell:

      GUIX_PROFILE="/path/to/my-guix-envs/my-project"; . 
"$GUIX_PROFILE"/etc/profile

The dot is the shell command `source`. It defines the nessecary 
environment variables like PATH.
Defining the variable GUIX_PROFILE has the effect thatyou always access 
the newest generation of the
profile. If you update your profile while your shell is running, you 
automatically access the files
of the new generation. Read the source code of $GUIX_PROFILE/etc/profile 
for more info.

As a general note, the use case of `guix environment` is to debug how a 
specific program is built
by GUIX. For example, if you want to enter the environment that GUIX 
creates for the building of
GNU hello, you call `guix environment hello`. You enter a profile with 
the *dependencies* of hello
which you can then try to build yourself. If you just want to create a 
profile following a manifest
file, just use

    guix package --manifest="your-manifest.scm" -p 
/path/to/profile-folder

That said, the containerization features of `guix environment` can be 
handy in other cases, too.

Cheers, Moritz

Re: How can I replace Python venv and pip with Guix?

[sirgazil]

 ---- On Tue, 04 Feb 2020 09:31:07 -0500  <mlell@posteo.de> wrote ----
 > Hi sirgazil,
 > 
 > > Well, the profile is created and persists, but, after deactivating the
 > > environment, how do I start an environment that uses that profile?
 > > 
 >
 > If you just want to create a profile following a manifest file, just use
 > 
 >     guix package --manifest="your-manifest.scm" -p 
 > /path/to/profile-folder
 > 
 > That said, the containerization features of `guix environment` can be 
 > handy in other cases, too.


Yes, I just skimmed through the "Guix Profiles in Practice" recipe in the Cookbook, and that seems to be what I should be using.

«[...] you can think of a profile as a kind of universal ‘virtualenv’ that can hold any kind of software whatsoever, not just Python software.»

I'll give it a try.

In my humble opinion, the "Guix Profiles in Practice" recipe should be converted into a section in the manual (called "Profiles"). I thought it was a nice overview.

Thanks again, Moritz.

Re: How can I replace Python venv and pip with Guix?

[sirgazil]

 ---- On Tue, 04 Feb 2020 09:02:36 -0500 Ricardo Wurmus <rekado@elephly.net> wrote ----
 > 
 > sirgazil <sirgazil@zoho.com> writes:
 > 
 > > • Once I deactivate this environment (Ctrl+D), how can I activate it again?
 > 
 > It’s a matter of evaluating the etc/profile file that is generated for
 > every Guix profile.  Since you passed
 > 
 >     --root=/path/to/my-guix-envs/my-project
 > 
 > to “guix environment” there will be a file
 > 
 >     /path/to/my-guix-envs/my-project/etc/profile
 > 
 > You can set all the environment variables it defines with
 > 
 >     GUIX_PROFILE=/path/to/my-guix-envs/my-project
 >     source $GUIX_PROFILE/etc/profile
 > 
 > If you want a pure environment you can use “env”.  On our HPC cluster I
 > provide “activate” scripts like this one:
 > 
 > --8<---------------cut here---------------start------------->8---
 > #!/bin/bash
 > 
 > profile=/path/to/.guix-profile
 > prompt='\u@\h:\W \[\e[31;47;1m\][pigx]\[\e[m\] $ '
 > 
 > REAL_HOME=$HOME
 > exec /bin/env - PS1="$prompt" \
 >                 HOME=$REAL_HOME \
 >                 LANG=en_US.UTF-8 \
 >                 GUIX_LOCPATH="$profile/lib/locale" \
 >                 CURL_CA_BUNDLE="/etc/ssl/certs/ca-bundle.crt" \
 >                 /bin/bash --init-file "$profile/etc/profile"
 > --8<---------------cut here---------------end--------------->8---
 > 
 > It sets a few environment variables, recovers some variables that are
 > important, and then starts a sub-shell that uses the generated
 > etc/profile file as its init file.
 > 
 > Something like that might work for you.


Thank you, Ricardo :)