From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp10.migadu.com ([2001:41d0:403:4789::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id GOk5JEMyp2SpNAAASxT56A (envelope-from ) for ; Thu, 06 Jul 2023 23:29:39 +0200 Received: from aspmx1.migadu.com ([2001:41d0:403:4789::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp10.migadu.com with LMTPS id cBA+I0Myp2TOFAAAG6o9tA (envelope-from ) for ; Thu, 06 Jul 2023 23:29:39 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id E1B75381CD for ; Thu, 6 Jul 2023 23:29:38 +0200 (CEST) Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=koszko.org header.s=mail header.b=cvdUO7Ab; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1688678979; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=wh4cwmav049fX19qaNYL1Xd9yYnmPPmtl9BXK7S8LN8=; b=B0bZfRsWRmw4G1VVfqJEncaUt0ifv+kMhgFGb3NcpWP7+nSptoOnxZNZVOII5JgbjJF59N r4OH3S0tZIbsNidcd6LFmC9CnnNdqoaYYY/2I3R6uZeb8Xg+v/zHPhzid0sGKaSheUbGbD XIyjsrX8CK6oopFFxVsgZPN7ykQl6COmG73HjVy3jVocJGd2aAiIrc0+IaV2QXw03Lacgc hlu8axkauD2acoe9fTyOrTFtlpZbNLu8jZjP13YKRV4R6zWHr7urUeV1YAenAsu3lftJeu dEFGCBOsy+Sm6AP+AKPVoQUSWOyPQx7/gHvXb5X5AHVfxCkF9ygsbTS6NiMhYg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=koszko.org header.s=mail header.b=cvdUO7Ab; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Seal: i=1; s=key1; d=yhetil.org; t=1688678979; a=rsa-sha256; cv=none; b=Xgks/OfzlQINYPBN+zXoEbIFMN0WYU5OJI4lyBAVjSRGXZyWobGSEmXZJDeyPhgocV8PsO rnzUZpbYTcC/9F7rR6rVvAf8MgGttoNno1atjkZRg98MDaFHPWMM8fiapAyErEijbncn0a 1iRsBzpFYEIeiVYV4EkuQbZVyK8fYprgAmJwHzMAhE0SaEy/qveGnoml3fgYGqe84Iqg6h JI5DXKxRz10x/fNUh2bT9Q0Kc2TFInsHUpHdbYui3COG/zP4FWSAuDVlTIct2mKPGcL1uR lvRJPlEpwXNfRwDXMWzwv5A4ixRkJctRiX9WhUcUtClxjyMfe0bBOfPGQoymTQ== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qHWXF-0002lx-Vt; Thu, 06 Jul 2023 17:29:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qHWXC-0002lj-9S for guix-devel@gnu.org; Thu, 06 Jul 2023 17:29:06 -0400 Received: from koszko.org ([93.95.227.159]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qHWX9-0000Is-5u for guix-devel@gnu.org; Thu, 06 Jul 2023 17:29:06 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=koszko.org; s=mail; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID: Subject:Cc:To:From:Date:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=wh4cwmav049fX19qaNYL1Xd9yYnmPPmtl9BXK7S8LN8=; b=cvdUO7AblWo931Q2PICJ0A4EFP +pdMf1q9gKDnUKhZ6R6FlbvL2Zvkv/mwyj+m6Il2VrWNJ+U0irIff4MRMP9FGDK4brHRHmou38xlj CJxqZLZXybmISADFYN2LyVnsFX63T9qaVEBN+A1bDV87cwWDa0vZNA66tiKCckIgITqSPUynuwwy3 8GoOkvlCeSVMBuwDrPESMKVRd2OxwWQJ0qSpZyZmzAt+vbsYzpNrJkUMriwQK+TsGIMsgSseCZuQ9 tg4X2nAwZgOvCYX9A6EUZfq60WiqlKkxoV3Ej78FZeTEhO16Qta7sOShoGdiL1cw10fOhPdvR51PO LYEqbq/ppOQhun0kpxLWG6oWoRT3USoOVmcamkipA+NzcQ985frIs1zdk67+rIfLoM5Yko1Hc7lOF xH+FstnSC/Ib+FBQZoVhKPPcHi/EHKtQvmNnAEOwpC+lBuRtJWcm+tGgX+XHwQxXGuGU56iwNFe+e xF+iaoXGQjPdnNxTSxFn2OmNocuN7hffho6fwRMJFNXkD56/1AEEkvFN2mOwrqfQAOI2t7BB2NYqu hd8bXVc0GqdDwTzw6V+uatgDSmfv3JVYJqCWnzY+SNINJvxSB2I2N5C9gfzRZwbt17UvOR/Hdbvd8 oxeOeSRB0yjMFLHtdRpoFE4aZL39X9JcDUMEBZ7xo=; Received: from 77-252-46-118.static.ip.netia.com.pl ([77.252.46.118] helo=localhost) by koszko.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qHWWj-0007fg-KH; Thu, 06 Jul 2023 23:28:37 +0200 Date: Thu, 6 Jul 2023 23:28:35 +0200 To: Maxim Cournoyer Cc: John Kehayias , =?UTF-8?B?5a6L5paH5q2m?= , edk@beaver-labs.com, guix-devel@gnu.org Subject: Re: Guix's python has pip's user dir in its loadpath Message-ID: <20230706232835.343c6cfe.koszko@koszko.org> In-Reply-To: <87o7kpyrws.fsf@gmail.com> References: <87edmey1wg.fsf@rdklein.fr> <877crma7qe.fsf@envs.net> <87edls1fyk.fsf@gmail.com> <20230701133257.6ada1e94.koszko@koszko.org> <871qhr1v6y.fsf@gmail.com> <87cz16kspd.fsf@protonmail.com> <87o7kpyrws.fsf@gmail.com> X-Mailer: Claws Mail 4.1.1 (GTK 3.24.37; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="Sig_/c.8jpGVrrp5DyPrHF6i=hh+"; protocol="application/pgp-signature"; micalg=pgp-sha256 Received-SPF: pass client-ip=93.95.227.159; envelope-from=koszko@koszko.org; helo=koszko.org X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Wojtek Kosior From: Wojtek Kosior via "Development of GNU Guix and the GNU System distribution." Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Spam-Score: -5.67 X-Migadu-Queue-Id: E1B75381CD X-Migadu-Scanner: mx1.migadu.com X-Migadu-Spam-Score: -5.67 X-TUID: 40SkOiEHZWuR --Sig_/c.8jpGVrrp5DyPrHF6i=hh+ Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable > > But we'll be rebuilding the Python world anyway, so now is a chance to > > try out some changes like that, though maybe it is a bit much with > > what we are trying already. See =20 >=20 > It's a simple change, I guess we could try it at the same time, if > someone volunteers to do it! I just saw this message and hurried myself up to test the patch to python-build-system that I made. Unfortunately, it turns out the "PYTHONNOUSERSITE=3D1" env var breaks pip which tries to install wheels to the system site directory and fails due to a read-only filesystem. It seems we need to have this configurable on per-package basis, after all. Should I try to make a patch which adds a build system argument that controls this? Also, the PYTHONNOUSERSITE variable only affects loading from the actual site dir, not from virtualenvs (which rely on PYTHONPATH IIRC). Should we try to add extra hardening to packages so that they are not affected by virtualenvs either? Best, Wojtek -- (sig_start) website: https://koszko.org/koszko.html fingerprint: E972 7060 E3C5 637C 8A4F 4B42 4BC5 221C 5A79 FD1A follow me on Fediverse: https://friendica.me/profile/koszko/profile =E2=99=A5 R29kIGlzIHRoZXJlIGFuZCBsb3ZlcyBtZQ=3D=3D | =C3=B7 c2luIHNlcGFyYXR= lZCBtZSBmcm9tIEhpbQ=3D=3D =E2=9C=9D YnV0IEplc3VzIGRpZWQgdG8gc2F2ZSBtZQ=3D=3D | ? U2hhbGwgSSBiZWNvbWUg= SGlzIGZyaWVuZD8=3D -- (sig_end) On Thu, 06 Jul 2023 11:35:15 -0400 Maxim Cournoyer wrote: > Hi John, >=20 > John Kehayias writes: >=20 > > Hi, > > > > On Sat, Jul 01, 2023 at 11:57 AM, Maxim Cournoyer wrote: > > =20 > >> Hi, > >> > >> Wojtek Kosior writes: > >> =20 > >>> The precedence of local, pip-installed Python libraries over Guix ones > >>> has already been a source of bugs. And these can be hard to diagnose.= =20 > >> =20 > >>> I imagine an optimal solution would be to configure this behavior on > >>> per-package basis. The vast majority of applications does not need to > >>> load local libraries. There are just a few exceptions like > >>> `python-virtualenv`. > >>> > >>> Once I did write a package definition that deliberately disabled user > >>> site dir package loading. I used code similar to what's below. > >>> =20 > >>>> (modify-phases %standard-phases > >>>> (add-after 'wrap 'prevent-local-package-interference > >>>> (lambda* (#:key outputs #:allow-other-keys) > >>>> (substitute* (string-append (assoc-ref outputs "out") > >>>> "/bin/") > >>>> (("^#!/.*$" shabang) > >>>> (string-append shabang > >>>> "export PYTHONNOUSERSITE=3D1\n")))))) =20 > >> > >> That is indeed a simple thing we could do to harden Python binaries fr= om > >> picking up user pip-installed dependencies potentially causing > >> problems. I would welcome such a patch. > >> =20 > > > > Perhaps, but if this is expected (and known) upstream behavior, I'm > > wary of deviating from these expectations. This general area does seem > > tricky and no simple best answer I guess. =20 >=20 > While it's true that it's an intended upstream behavior, I think in the > context of Guix users packages to be self-contained or in some case be > able to load Guix-installed plugins or extensions, but here it seems > reasonable that a Guix-packaged Python binary prefers loading Python > libraries from Guix rather that from the Python user site. >=20 > >>> Of course, it makes no sense to add such snippet to all definitions. > >>> Instead, we could modify python-build-system to allow doing a similar > >>> thing based on a flag passed in package's `(arguments)`. =20 > >> > >> I think it need not be made configurable but just applied > >> indiscriminately to the wrap phase used in the python-build-system. =20 > > > > And this is part of the same question then, we should try to be > > consistent, yes. I don't see a clear right path, but I haven't thought > > much about this area. I think it comes down to a current > > issue/limitation/quirk of Python from upstream and packaging for our > > distro puts us in between what comes from them and how to take care of > > our users. > > > > But we'll be rebuilding the Python world anyway, so now is a chance to > > try out some changes like that, though maybe it is a bit much with > > what we are trying already. See =20 >=20 > It's a simple change, I guess we could try it at the same time, if > someone volunteers to do it! >=20 --Sig_/c.8jpGVrrp5DyPrHF6i=hh+ Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iHUEARYIAB0WIQTpcnBg48VjfIpPS0JLxSIcWnn9GgUCZKcyAwAKCRBLxSIcWnn9 GpeFAQCrMCLPNUlRrEu2LU1Rsky7JyIphmnH3Bf55H6Mb5Ia4wEAvWtilDUvY39f 4r/NPjti1rueIPSCWPDZjIzxe0wSZAM= =i02o -----END PGP SIGNATURE----- --Sig_/c.8jpGVrrp5DyPrHF6i=hh+--