Language library packages should require language package

Language library packages should require language package

[Hartmut Goebel]

Hi,

playing around with guix environment, I stepped over some weirdness The
weirdest of is (e.g.):

guix environment  --ad-hoc python-requests

will give you . . . . nothing.

Since python-requests does not have python as input, the package is
installed somewhere out of reach. This is even true when using a python
virtualenv prior to entering the guix environment (see below for details).

I assume, this is the same for perl and other languages.

So I propose to add python, perl, ... as an import for the "library"
packages of this language. This would be in the same line as .rpm and
.deb packages work.


Now the weirdness analysis:

$ cat echo_paths.sh
echo PATH=$PATH
echo PYTHONPATH=$PYTHONPATH
type python

Now let's set up "--ad-hoc python-requests" (but no python):

$ guix environment --pure --ad-hoc python-requests -- ./echo_paths.sh
PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
PYTHONPATH=
python is /usr/bin/python

Wow, the system python is used, but where is python-requests? Maybe not
installed, since it already is in the system installation. Let's try a
virtual environment:

$ pyvenv /tmp/foo
$ source /tmp/foo/bin/activate
(foo)$ type python
python ist /tmp/foo/bin/python

(foo)$ guix environment --pure --ad-hoc python-requests -- ./echo_paths.sh
PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
PYTHONPATH=
python is /usr/bin/python

Huh? What? The virtual env was ignored (or dropped). Why this? Looks
like --pure resets the environment to something is thinks is correct,
even if if ist not adding anything to the environments.

Let's try without --pure:

(foo)$ guix environment  --ad-hoc python-requests -- ./echo_paths.sh
PATH=/tmp/foo/bin:/usr/local/bin:/usr/bin:/usr/local/games:/usr/games:/usr/lib64/qt4/bin:/usr/lib64/qt5/bin:/home/hartmut/bin:/home/hartmut/.local/bin
PYTHONPATH=/home/hartmut/lib/python
python is /tmp/foo/bin/python

Okay, this time I get the python within the virtual env. But still:
where is python-requests?

(foo)$ guix environment  --ad-hoc python-requests -- ls
/tmp/foo/lib/python3.4/site-packages/
easy_install.py  pip                  pkg_resources.py  setuptools
_markerlib       pip-6.1.1.dist-info  __pycache__      
setuptools-3.6.dist-info

It is not installed in the virtual environment either. Where is it?

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Language library packages should require language package

[Thompson, David]

On Sun, Apr 24, 2016 at 8:24 AM, Hartmut Goebel
<h.goebel@crazy-compilers.com> wrote:
> Hi,
>
> playing around with guix environment, I stepped over some weirdness The
> weirdest of is (e.g.):
>
> guix environment  --ad-hoc python-requests
>
> will give you . . . . nothing.

That's not quite right.  It will give you exactly what you asked for:
an environment with only python-requests used in it.  python-requests
defines no search paths, so no environment variables are set.

> Since python-requests does not have python as input, the package is
> installed somewhere out of reach. This is even true when using a python
> virtualenv prior to entering the guix environment (see below for details).
>
> I assume, this is the same for perl and other languages.
>
> So I propose to add python, perl, ... as an import for the "library"
> packages of this language. This would be in the same line as .rpm and
> .deb packages work.

No, this wouldn't be a good idea.  You used the --ad-hoc flag, which
means "create an environment with *only* the direct packages I
specified."  This behavior is correct and will not be changed.  If you
want an ad-hoc python environment, you should add a python package to
your environment:

    guix environment --ad-hoc python python-requests

>
> Now the weirdness analysis:
>
> $ cat echo_paths.sh
> echo PATH=$PATH
> echo PYTHONPATH=$PYTHONPATH
> type python
>
> Now let's set up "--ad-hoc python-requests" (but no python):
>
> $ guix environment --pure --ad-hoc python-requests -- ./echo_paths.sh
> PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
> PYTHONPATH=
> python is /usr/bin/python
>
> Wow, the system python is used, but where is python-requests? Maybe not
> installed, since it already is in the system installation. Let's try a
> virtual environment:
>
> $ pyvenv /tmp/foo
> $ source /tmp/foo/bin/activate
> (foo)$ type python
> python ist /tmp/foo/bin/python
>
> (foo)$ guix environment --pure --ad-hoc python-requests -- ./echo_paths.sh
> PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
> PYTHONPATH=
> python is /usr/bin/python
>
> Huh? What? The virtual env was ignored (or dropped). Why this? Looks
> like --pure resets the environment to something is thinks is correct,
> even if if ist not adding anything to the environments.
>
> Let's try without --pure:
>
> (foo)$ guix environment  --ad-hoc python-requests -- ./echo_paths.sh
> PATH=/tmp/foo/bin:/usr/local/bin:/usr/bin:/usr/local/games:/usr/games:/usr/lib64/qt4/bin:/usr/lib64/qt5/bin:/home/hartmut/bin:/home/hartmut/.local/bin
> PYTHONPATH=/home/hartmut/lib/python
> python is /tmp/foo/bin/python
>
> Okay, this time I get the python within the virtual env. But still:
> where is python-requests?
>
> (foo)$ guix environment  --ad-hoc python-requests -- ls
> /tmp/foo/lib/python3.4/site-packages/
> easy_install.py  pip                  pkg_resources.py  setuptools
> _markerlib       pip-6.1.1.dist-info  __pycache__
> setuptools-3.6.dist-info
>
> It is not installed in the virtual environment either. Where is it?
>
> --
> Regards
> Hartmut Goebel
>
> | Hartmut Goebel          | h.goebel@crazy-compilers.com               |
> | www.crazy-compilers.com | compilers which you thought are impossible |
>
>
>

Re: Language library packages should require language package

[Thompson, David]

Sorry, I accidentally sent the previous email before I was finished
writing.  The rest of the response is below.

On Sun, Apr 24, 2016 at 8:24 AM, Hartmut Goebel
<h.goebel@crazy-compilers.com> wrote:

[...]

> Now the weirdness analysis:
>
> $ cat echo_paths.sh
> echo PATH=$PATH
> echo PYTHONPATH=$PYTHONPATH
> type python
>
> Now let's set up "--ad-hoc python-requests" (but no python):
>
> $ guix environment --pure --ad-hoc python-requests -- ./echo_paths.sh
> PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
> PYTHONPATH=
> python is /usr/bin/python
>
> Wow, the system python is used, but where is python-requests? Maybe not
> installed, since it already is in the system installation.
> Let's try a virtual environment:

> $ pyvenv /tmp/foo
> $ source /tmp/foo/bin/activate
> (foo)$ type python
> python ist /tmp/foo/bin/python
>
> (foo)$ guix environment --pure --ad-hoc python-requests -- ./echo_paths.sh
> PATH=/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:.
> PYTHONPATH=
> python is /usr/bin/python
>
> Huh? What? The virtual env was ignored (or dropped). Why this? Looks
> like --pure resets the environment to something is thinks is correct,
> even if if ist not adding anything to the environments.

This is because your .bashrc is broken and sets $PATH and other
environment variables.  The --pure flag clears the environment
variables first, so the only way you can get stuff like this back is
if your .bashrc is messed up.  Environment variables should only be
set in the .bash_profile, which is sourced in *login* shells only.

See the manual for further explanation:

https://gnu.org/software/guix/manual/html_node/Invoking-guix-environment.html#FOOT13

> Let's try without --pure:
>
> (foo)$ guix environment  --ad-hoc python-requests -- ./echo_paths.sh
> PATH=/tmp/foo/bin:/usr/local/bin:/usr/bin:/usr/local/games:/usr/games:/usr/lib64/qt4/bin:/usr/lib64/qt5/bin:/home/hartmut/bin:/home/hartmut/.local/bin
> PYTHONPATH=/home/hartmut/lib/python
> python is /tmp/foo/bin/python
>
> Okay, this time I get the python within the virtual env. But still:
> where is python-requests?

Python-requests, by itself, specifies no search paths, so 'guix
environment' doesn't set anything.  It just makes a profile that
doesn't end up being referenced in the new environment.

> (foo)$ guix environment  --ad-hoc python-requests -- ls
> /tmp/foo/lib/python3.4/site-packages/
> easy_install.py  pip                  pkg_resources.py  setuptools
> _markerlib       pip-6.1.1.dist-info  __pycache__
> setuptools-3.6.dist-info
>
> It is not installed in the virtual environment either. Where is it?

Why would you think it would be here?  In Guix, everything is kept in
/gnu/store.

Please re-read the documentation for 'guix environment'.  I think
things will become more clear to you afterwards.

Thanks,

- Dave

Re: Language library packages should require language package

[Hartmut Goebel]

Am 24.04.2016 um 15:09 schrieb Thompson, David:
> That's not quite right. It will give you exactly what you asked for:
> an environment with only python-requests used in it. python-requests
> defines no search paths, so no environment variables are set. 


But where is `python-requests` then?


> No, this wouldn't be a good idea.  You used the --ad-hoc flag, which
> means "create an environment with *only* the direct packages I
> specified."  This behavior is correct and will not be changed. 

But this is description is at least misleading, if not wrong. `guix
environment --ad-hoc python python-requests` does not only install
python and python-requests, but also pytest, pytest-coverage, wheel (and
some more).

But you are missing my main point: While `guix environment --ad-hoc
python-requests` installs some depended python packages, it does not
install `python`. This is asymmetric and quite different than users are
used from rpm and deb based systems.

To take this one step further: Assume in GuixSD 1.3 some user is told
"install python-bla-blub vai the Guix Package Manager GUI", then she
ends up with an incomplete environment. She would have to figure out
what other dependencies (esp. the language interpreter) are required.

This is why I'm proposing to add the dependency to the language interpreter.

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Language library packages should require language package

[Ludovic Courtès]

Hartmut Goebel <h.goebel@crazy-compilers.com> skribis:

> But you are missing my main point: While `guix environment --ad-hoc
> python-requests` installs some depended python packages, it does not
> install `python`. This is asymmetric and quite different than users are
> used from rpm and deb based systems.

Indeed.  I understand this is surprising.  It stems from the fact that
Guix doesn’t know about the relation between ‘python-requests’ and
‘python’.

Even if it did, it’s not clear to me that automatically installing both
would be a good idea.  There are cases where one may want to choose a
particular variant of the ‘python’ package rather than have an
arbitrarily chosen default.

Does it make sense?

> To take this one step further: Assume in GuixSD 1.3 some user is told
> "install python-bla-blub vai the Guix Package Manager GUI", then she
> ends up with an incomplete environment. She would have to figure out
> what other dependencies (esp. the language interpreter) are required.

Most likely one would provide a file that can be passed to ‘guix
environment -l’ instead of writing down the instructions.  :-)

Thanks,
Ludo’.