From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id P4WMMqNwLmDaUwAA0tVLHw (envelope-from ) for ; Thu, 18 Feb 2021 13:50:27 +0000 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id KA4oLqNwLmDRHQAA1q6Kng (envelope-from ) for ; Thu, 18 Feb 2021 13:50:27 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 00D3A251FF for ; Thu, 18 Feb 2021 14:50:27 +0100 (CET) Received: from localhost ([::1]:46392 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lCjhJ-0003Nd-0j for larch@yhetil.org; Thu, 18 Feb 2021 08:50:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:49116) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lCjgw-0003Lf-ID for bug-guix@gnu.org; Thu, 18 Feb 2021 08:50:04 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:34880) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lCjgw-0005L7-2q for bug-guix@gnu.org; Thu, 18 Feb 2021 08:50:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lCjgv-00071P-V0 for bug-guix@gnu.org; Thu, 18 Feb 2021 08:50:01 -0500 X-Loop: help-debbugs@gnu.org Subject: bug#46569: python can't find internal pip modules in environment from manifest Resent-From: Jelle Licht Original-Sender: "Debbugs-submit" Resent-CC: bug-guix@gnu.org Resent-Date: Thu, 18 Feb 2021 13:50:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 46569 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 46569@debbugs.gnu.org Received: via spool by 46569-submit@debbugs.gnu.org id=B46569.161365617726950 (code B ref 46569); Thu, 18 Feb 2021 13:50:01 +0000 Received: (at 46569) by debbugs.gnu.org; 18 Feb 2021 13:49:37 +0000 Received: from localhost ([127.0.0.1]:46426 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lCjgS-00070V-5f for submit@debbugs.gnu.org; Thu, 18 Feb 2021 08:49:37 -0500 Received: from mail1.fsfe.org ([217.69.89.151]:57068) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lCjgQ-00070O-N0 for 46569@debbugs.gnu.org; Thu, 18 Feb 2021 08:49:31 -0500 From: Jelle Licht In-Reply-To: <86czwz7ms3.fsf@fsfe.org> References: <86czwz7ms3.fsf@fsfe.org> Date: Thu, 18 Feb 2021 14:49:28 +0100 Message-ID: <86zh015vvb.fsf@fsfe.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guix@gnu.org List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+larch=yhetil.org@gnu.org Sender: "bug-Guix" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -2.27 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=fsfe.org (policy=none); spf=pass (aspmx1.migadu.com: domain of bug-guix-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=bug-guix-bounces@gnu.org X-Migadu-Queue-Id: 00D3A251FF X-Spam-Score: -2.27 X-Migadu-Scanner: scn0.migadu.com X-TUID: 8lldu3Mf9D0z Jelle Licht writes: > Python can not load all pip-related python modules when loaded from a > manifest file, yet seems to work fine when loaded 'directly' as part of > a `guix environment'-invocation. > > > Provided that we have a file in `hi.py': > --8<---------------cut here---------------start------------->8--- > import pip > print("halfwaypoint") > import pip._internal.index.package_finder > print("I made it!") > --8<---------------cut here---------------end--------------->8--- > > and a `manifest.scm' with: > > --8<---------------cut here---------------start------------->8--- > (use-modules (gnu) (guix packages) > (gnu packages python) > (gnu packages python-xyz)) > > (packages->manifest (list python python-pip)) > --8<---------------cut here---------------end--------------->8--- > > I can get the expected output by running: > `guix environment --pure --ad-hoc python python-pip -- python3 hi.py' =3D> >> halfwaypoint >> I made it! > > Yet, when I run the following: > `guix environment --pure -m manifest.scm -- python3 hi.py' =3D> > > --8<---------------cut here---------------start------------->8--- > halfwaypoint > Traceback (most recent call last): > File "hi.py", line 4, in > import pip._internal.index.package_finder > File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8= /site-packages/pip/_internal/__init__.py", line 40, in > from pip._internal.cli.autocompletion import autocomplete > File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8= /site-packages/pip/_internal/cli/autocompletion.py", line 8, in > from pip._internal.cli.main_parser import create_main_parser > File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8= /site-packages/pip/_internal/cli/main_parser.py", line 11, in > from pip._internal.commands import ( > File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8= /site-packages/pip/_internal/commands/__init__.py", line 6, in > from pip._internal.commands.completion import CompletionCommand > File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8= /site-packages/pip/_internal/commands/completion.py", line 6, in > from pip._internal.cli.base_command import Command > File "/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8= /site-packages/pip/_internal/cli/base_command.py", line 26, in > from pip._internal.index import PackageFinder > ImportError: cannot import name 'PackageFinder' from 'pip._internal.index= ' (/gnu/store/4m5vhlq61qnj36rq60l83xcmgj3mx92j-profile/lib/python3.8/site-p= ackages/pip/_internal/index/__init__.py) > --8<---------------cut here---------------end--------------->8--- > > Why is there a difference in the first place? Shouldn't both approaches w= ork? Using diffoscope, I found the (practical) difference: $GUIX_ENVIRONMENT/bin/pip3 is =E2=94=82 =E2=94=82 -destination: /gnu/store/rz42ba0my9vrgbkjpkzr2drmnjk5ah= 50-python-3.8.2/bin/pip3 =E2=94=82 =E2=94=82 +destination: /gnu/store/d75bwzbla5ybhs0mdw80qy94mawnhh= sw-python-pip-20.2.4/bin/pip3 If you make sure that `python-pip' precedes the `python' package in the manifest, things work out fine. Likewise for the following --ad-hoc example: This fails: guix environment --ad-hoc python-pip python -- python3 hi.py This works: guix environment --ad-hoc python python-pip -- python3 hi.py So it seems the order in which guix sees packages when building profiles determines how collisions are resolved, good to know :-). Is the python-included pip package supposed to be broken like this, though? - Jelle