From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2 ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms11 with LMTPS id wFzpMIAdHWCvHwAA0tVLHw (envelope-from ) for ; Fri, 05 Feb 2021 10:27:12 +0000 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2 with LMTPS id OJnKLIAdHWBOPwAAB5/wlQ (envelope-from ) for ; Fri, 05 Feb 2021 10:27:12 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 8347B9403EA for ; Fri, 5 Feb 2021 10:27:12 +0000 (UTC) Received: from localhost ([::1]:46374 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1l7yKV-0000ih-6Y for larch@yhetil.org; Fri, 05 Feb 2021 05:27:11 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:48252) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l7yJs-0000ex-Du for guix-devel@gnu.org; Fri, 05 Feb 2021 05:26:34 -0500 Received: from mail-out.m-online.net ([212.18.0.9]:41134) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1l7yJp-0004Qo-6S; Fri, 05 Feb 2021 05:26:32 -0500 Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4DXBR44cyBz1qtf4; Fri, 5 Feb 2021 11:26:24 +0100 (CET) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4DXBR44Dypz1tYTb; Fri, 5 Feb 2021 11:26:24 +0100 (CET) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id boMMBQZFoajx; Fri, 5 Feb 2021 11:26:23 +0100 (CET) Received: from hermia.goebel-consult.de (ppp-188-174-60-247.dynamic.mnet-online.de [188.174.60.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPS; Fri, 5 Feb 2021 11:26:23 +0100 (CET) Received: from lenashee.goebel-consult.de (lenashee.goebel-consult.de [192.168.110.2]) by hermia.goebel-consult.de (Postfix) with ESMTP id DBB08602A2; Fri, 5 Feb 2021 11:26:20 +0100 (CET) Subject: Re: 03/163: build/python: Add a new guix-pythonpath procedure. From: Hartmut Goebel To: Maxim Cournoyer , =?UTF-8?Q?Ludovic_Court=c3=a8s?= References: <20210125070022.22870.17321@vcs0.savannah.gnu.org> <20210125070114.03C0B20E1C@vcs0.savannah.gnu.org> <87czxs3d1i.fsf@gnu.org> <878s8g3bxa.fsf@gmail.com> <87lfcdm9jb.fsf@gnu.org> <87im7fyg5c.fsf@gmail.com> Organization: crazy-compilers.com Message-ID: Date: Fri, 5 Feb 2021 11:26:20 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: <87im7fyg5c.fsf@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US Received-SPF: none client-ip=212.18.0.9; envelope-from=h.goebel@crazy-compilers.com; helo=mail-out.m-online.net X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, NICE_REPLY_A=-0.182, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: guix-devel@gnu.org Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-Spam-Score: -2.35 Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 8347B9403EA X-Spam-Score: -2.35 X-Migadu-Scanner: scn1.migadu.com X-TUID: awXc4hL8ag/H Hi Maxim, many thanks for picking up this issue. > Indeed, I thought about the possibility to filter the GUIX_PYTHONPATH > entries based on their version at runtime after I wrote my initial > reply. It makes life easier. I've updated the > cu/farewell-to-pythonpath branch with this new way of doing things. I had a look at the first changes (only) and have some remarks: 1) Did I understand this correctly: `sitecustomize.py` is filtering GUIX_PYTHONPATH for all parts containing "/lib/pythonX.Y/site-packages" (with X.Y being Major.Minor)? 2) This does not remove duplicates and does not honor .pth files in the respective directories - which might still be used. Thus site.addsitedir() should be called for adding the paths. This also takes care about duplicates. 3) Empty part (…::…) are not handled. 4) Since PYTHONPATH is evaluated prior to importing sitecustomize, any sitecustominze.py in the user's path will overwrite our file, thus inhibiting our paths to be added. Not sure this is what we want in Guix. 5) When implementing the logic into site.py, the code could be simplified, Eg. You could patch a "getguixsitepackages" (based on getsitepackages) and a "addguixsitepackages" (based on addguixsitepackages) into site.py. Downside is that maybe we need different patches for different Python versions. Benefit is simpler code - no need to search the correct position in the list. 6) Please add some more comments to the code explaining the idea. Some nitpicking: > python_root = os.path.realpath(sys.executable).split('/bin/')[0] There already is sys.prefix, which is also the base for the entry in sys.path (see top of site.py > major_minor = '{}.{}'.format(sys.version_info[0], sys.version_info[1]) major_minor = '{}.{}'.format(*sys.version_info) >sys.path = sys.path[:index] + matching_sites + sys.path[index:] sys.path[index:index] = matching_sites I suggest using os.path.join(), os.path.split(), os.pathsep, etc. to be forward-compatible. Imagine we want to port Guix to another platform with different separators. -- Regards Hartmut Goebel | Hartmut Goebel | h.goebel@crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible |