unofficial mirror of bug-guix@gnu.org 
 help / color / mirror / code / Atom feed
* bug#46569: python can't find internal pip modules in environment from manifest
@ 2021-02-16 20:58 Jelle Licht
  2021-02-18 13:49 ` Jelle Licht
                   ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Jelle Licht @ 2021-02-16 20:58 UTC (permalink / raw)
  To: 46569

Python can not load all pip-related python modules when loaded from a
manifest file, yet seems to work fine when loaded 'directly' as part of
a `guix environment'-invocation.


Provided that we have a file in `hi.py':
--8<---------------cut here---------------start------------->8---
import pip
print("halfwaypoint")
import pip._internal.index.package_finder
print("I made it!")
--8<---------------cut here---------------end--------------->8---

and a `manifest.scm' with:

--8<---------------cut here---------------start------------->8---
(use-modules (gnu) (guix packages)
	     (gnu packages python)
	     (gnu packages python-xyz))

(packages->manifest (list python python-pip))
--8<---------------cut here---------------end--------------->8---

I can get the expected output by running:
`guix environment --pure --ad-hoc python python-pip -- python3 hi.py' =>
> halfwaypoint
> I made it!

Yet, when I run the following:
`guix environment --pure -m manifest.scm -- python3 hi.py' =>

--8<---------------cut here---------------start------------->8---
halfwaypoint
Traceback (most recent call last):
  File "hi.py", line 4, in <module>
    import pip._internal.index.package_finder
  File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8/site-packages/pip/_internal/__init__.py", line 40, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py", line 8, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py", line 11, in <module>
    from pip._internal.commands import (
  File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8/site-packages/pip/_internal/commands/__init__.py", line 6, in <module>
    from pip._internal.commands.completion import CompletionCommand
  File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8/site-packages/pip/_internal/commands/completion.py", line 6, in <module>
    from pip._internal.cli.base_command import Command
  File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 26, in <module>
    from pip._internal.index import PackageFinder
ImportError: cannot import name 'PackageFinder' from 'pip._internal.index' (/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8/site-packages/pip/_internal/index/__init__.py)
--8<---------------cut here---------------end--------------->8---

Why is there a difference in the first place? Shouldn't both approaches work?

- Jelle




^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#46569: python can't find internal pip modules in environment from manifest
  2021-02-16 20:58 bug#46569: python can't find internal pip modules in environment from manifest Jelle Licht
@ 2021-02-18 13:49 ` Jelle Licht
  2021-02-18 16:37   ` Jelle Licht
  2021-02-18 18:05 ` Maxim Cournoyer
  2021-02-18 21:18 ` Maxim Cournoyer
  2 siblings, 1 reply; 6+ messages in thread
From: Jelle Licht @ 2021-02-18 13:49 UTC (permalink / raw)
  To: 46569

Jelle Licht <jlicht@fsfe.org> writes:

> Python can not load all pip-related python modules when loaded from a
> manifest file, yet seems to work fine when loaded 'directly' as part of
> a `guix environment'-invocation.
>
>
> Provided that we have a file in `hi.py':
> --8<---------------cut here---------------start------------->8---
> import pip
> print("halfwaypoint")
> import pip._internal.index.package_finder
> print("I made it!")
> --8<---------------cut here---------------end--------------->8---
>
> and a `manifest.scm' with:
>
> --8<---------------cut here---------------start------------->8---
> (use-modules (gnu) (guix packages)
> 	     (gnu packages python)
> 	     (gnu packages python-xyz))
>
> (packages->manifest (list python python-pip))
> --8<---------------cut here---------------end--------------->8---
>
> I can get the expected output by running:
> `guix environment --pure --ad-hoc python python-pip -- python3 hi.py' =>
>> halfwaypoint
>> I made it!
>
> Yet, when I run the following:
> `guix environment --pure -m manifest.scm -- python3 hi.py' =>
>
> --8<---------------cut here---------------start------------->8---
> halfwaypoint
> Traceback (most recent call last):
>   File "hi.py", line 4, in <module>
>     import pip._internal.index.package_finder
>   File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8/site-packages/pip/_internal/__init__.py", line 40, in <module>
>     from pip._internal.cli.autocompletion import autocomplete
>   File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8/site-packages/pip/_internal/cli/autocompletion.py", line 8, in <module>
>     from pip._internal.cli.main_parser import create_main_parser
>   File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8/site-packages/pip/_internal/cli/main_parser.py", line 11, in <module>
>     from pip._internal.commands import (
>   File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8/site-packages/pip/_internal/commands/__init__.py", line 6, in <module>
>     from pip._internal.commands.completion import CompletionCommand
>   File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8/site-packages/pip/_internal/commands/completion.py", line 6, in <module>
>     from pip._internal.cli.base_command import Command
>   File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 26, in <module>
>     from pip._internal.index import PackageFinder
> ImportError: cannot import name 'PackageFinder' from 'pip._internal.index' (/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8/site-packages/pip/_internal/index/__init__.py)
> --8<---------------cut here---------------end--------------->8---
>
> Why is there a difference in the first place? Shouldn't both approaches work?

Using diffoscope, I found the (practical) difference:

$GUIX_ENVIRONMENT/bin/pip3 is
│ │ -destination: /gnu/store/rz42ba0my9vrgbkjpkzr2drmnjk5ah50-python-3.8.2/bin/pip3
│ │ +destination: /gnu/store/d75bwzbla5ybhs0mdw80qy94mawnhhsw-python-pip-20.2.4/bin/pip3

If you make sure that `python-pip' precedes the `python' package in the
manifest, things work out fine. Likewise for the following --ad-hoc
example:

This fails:
guix environment --ad-hoc python-pip python -- python3 hi.py

This works:
guix environment --ad-hoc python python-pip -- python3 hi.py

So it seems the order in which guix sees packages when building profiles
determines how collisions are resolved, good to know :-).

Is the python-included pip package supposed to be broken like this, though?
 - Jelle




^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#46569: python can't find internal pip modules in environment from manifest
  2021-02-18 13:49 ` Jelle Licht
@ 2021-02-18 16:37   ` Jelle Licht
  0 siblings, 0 replies; 6+ messages in thread
From: Jelle Licht @ 2021-02-18 16:37 UTC (permalink / raw)
  To: 46569

Jelle Licht <jlicht@fsfe.org> writes:

> Jelle Licht <jlicht@fsfe.org> writes:
> This fails:
> guix environment --ad-hoc python-pip python -- python3 hi.py
>
> This works:
> guix environment --ad-hoc python python-pip -- python3 hi.py

This should be the other way around, pasting error on my end.




^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#46569: python can't find internal pip modules in environment from manifest
  2021-02-16 20:58 bug#46569: python can't find internal pip modules in environment from manifest Jelle Licht
  2021-02-18 13:49 ` Jelle Licht
@ 2021-02-18 18:05 ` Maxim Cournoyer
  2021-02-18 19:45   ` Jelle Licht
  2021-02-18 21:18 ` Maxim Cournoyer
  2 siblings, 1 reply; 6+ messages in thread
From: Maxim Cournoyer @ 2021-02-18 18:05 UTC (permalink / raw)
  To: Jelle Licht; +Cc: 46569

Hi Jelle,

Jelle Licht <jlicht@fsfe.org> writes:

> Python can not load all pip-related python modules when loaded from a
> manifest file, yet seems to work fine when loaded 'directly' as part of
> a `guix environment'-invocation.
>
>
> Provided that we have a file in `hi.py':
>
> import pip
> print("halfwaypoint")
> import pip._internal.index.package_finder
> print("I made it!")
>
>
> and a `manifest.scm' with:
>
> (use-modules (gnu) (guix packages)
> 	     (gnu packages python)
> 	     (gnu packages python-xyz))
>
> (packages->manifest (list python python-pip))
>
>
> I can get the expected output by running:
> `guix environment --pure --ad-hoc python python-pip -- python3 hi.py' =>
>> halfwaypoint
>> I made it!

It seems that two pip versions don't cohabit very well.  The same
happens if you install pip via 'pip install --user pip' (e.g., to
~/.local/bin) and try to use it.

You could try using the pip version that comes bundled with Python
itself (e.g, not installing python-pip in the same profile); if it's the
only one, it should work.

HTH,

Maxim




^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#46569: python can't find internal pip modules in environment from manifest
  2021-02-18 18:05 ` Maxim Cournoyer
@ 2021-02-18 19:45   ` Jelle Licht
  0 siblings, 0 replies; 6+ messages in thread
From: Jelle Licht @ 2021-02-18 19:45 UTC (permalink / raw)
  To: Maxim Cournoyer; +Cc: 46569

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> Hi Jelle,
>
> Jelle Licht <jlicht@fsfe.org> writes:
>
>> Python can not load all pip-related python modules when loaded from a
>> manifest file, yet seems to work fine when loaded 'directly' as part of
>> a `guix environment'-invocation.
>>
>>
>> Provided that we have a file in `hi.py':
>>
>> import pip
>> print("halfwaypoint")
>> import pip._internal.index.package_finder
>> print("I made it!")
>>
>>
>> and a `manifest.scm' with:
>>
>> (use-modules (gnu) (guix packages)
>> 	     (gnu packages python)
>> 	     (gnu packages python-xyz))
>>
>> (packages->manifest (list python python-pip))
>>
>>
>> I can get the expected output by running:
>> `guix environment --pure --ad-hoc python python-pip -- python3 hi.py' =>
>>> halfwaypoint
>>> I made it!
>
> It seems that two pip versions don't cohabit very well.  The same
> happens if you install pip via 'pip install --user pip' (e.g., to
> ~/.local/bin) and try to use it.
>
> You could try using the pip version that comes bundled with Python
> itself (e.g, not installing python-pip in the same profile); if it's the
> only one, it should work.

It doesn't, though, with the exact same error message as above.

Does my example (hi.py) work for you?





^ permalink raw reply	[flat|nested] 6+ messages in thread

* bug#46569: python can't find internal pip modules in environment from manifest
  2021-02-16 20:58 bug#46569: python can't find internal pip modules in environment from manifest Jelle Licht
  2021-02-18 13:49 ` Jelle Licht
  2021-02-18 18:05 ` Maxim Cournoyer
@ 2021-02-18 21:18 ` Maxim Cournoyer
  2 siblings, 0 replies; 6+ messages in thread
From: Maxim Cournoyer @ 2021-02-18 21:18 UTC (permalink / raw)
  To: Jelle Licht; +Cc: 46569-done

Hello Jelle,

Jelle Licht <jlicht@fsfe.org> writes:

> Python can not load all pip-related python modules when loaded from a
> manifest file, yet seems to work fine when loaded 'directly' as part of
> a `guix environment'-invocation.
>
>
> Provided that we have a file in `hi.py':
>
> import pip
> print("halfwaypoint")
> import pip._internal.index.package_finder
> print("I made it!")

The problem seems to be related to the version of pip used.  The one
bundled with our Python on master is:

$ guix environment --pure --ad-hoc python -- pip3 --version
pip 19.2.3 from
/gnu/store/pw65c67dmzr2m9wbkj5mskhm6ljk6xal-profile/lib/python3.8/site-packages/pip
(python 3.8)

and

$ guix environment --pure --ad-hoc python -- python3 -c 'import pip._internal.index.package_finder'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named
'pip._internal.index.package_finder'; 'pip._internal.index' is not a
package

but

$ guix environment --pure --ad-hoc python python-pip -- pip3 --version
pip 20.2.4 from
/gnu/store/d75bwzbla5ybhs0mdw80qy94mawnhhsw-python-pip-20.2.4/lib/python3.8/site-packages/pip
(python 3.8)

and

$ guix environment --pure --ad-hoc python python-pip -- python3 -c 'import pip._internal.index.package_finder'
$ echo $?
0

HTH!

Closing.

Maxim




^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-02-18 21:20 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-16 20:58 bug#46569: python can't find internal pip modules in environment from manifest Jelle Licht
2021-02-18 13:49 ` Jelle Licht
2021-02-18 16:37   ` Jelle Licht
2021-02-18 18:05 ` Maxim Cournoyer
2021-02-18 19:45   ` Jelle Licht
2021-02-18 21:18 ` Maxim Cournoyer

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).