From mboxrd@z Thu Jan 1 00:00:00 1970 From: Christopher Allan Webber Subject: Re: Python and propagation Date: Thu, 18 Feb 2016 14:38:46 -0800 Message-ID: <87fuwpprfx.fsf@dustycloud.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:58174) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aWXLU-00074n-LT for guix-devel@gnu.org; Thu, 18 Feb 2016 17:46:49 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aWXLT-0000rc-Go for guix-devel@gnu.org; Thu, 18 Feb 2016 17:46:48 -0500 Received: from dustycloud.org ([2600:3c02::f03c:91ff:feae:cb51]:47844) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aWXLT-0000r9-9v for guix-devel@gnu.org; Thu, 18 Feb 2016 17:46:47 -0500 In-reply-to: List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: Ricardo Wurmus Cc: guix-devel Ricardo Wurmus writes: > 1) print a warning when a collision is expected to happen, not when a > collision has happened. > > Guix knows what is currently installed in a profile, so it knows that, > say, =E2=80=9Cpython2-numpy=E2=80=9D is installed. It also knows that = installing > =E2=80=9Chtseq=E2=80=9D is also going to install =E2=80=9Cpython2-numpy= =E2=80=9D into that profile. It > also knows that the propagated =E2=80=9Cpython2-numpy=E2=80=9D is diffe= rent from the > installed =E2=80=9Cpython2-numpy=E2=80=9D. Knowing all that, it should= also know that > there are going to be file conflicts, no? > > If that=E2=80=99s all true, could we make Guix print a warning about im= pending > doom *before* it creates a new profile generation? It=E2=80=99s a much= nicer > user experience (at least to me) when I=E2=80=99m being told about poss= ible > conflicts (at the package level) before Guix creates a new profile and > nonchalantly informs me about arbitrarily resolving conflicts (at the > file level). The difference is in the amount of warnings I get (packag= e > vs individual files) and about the time I would otherwise have to waste > to roll back the profile and upgrade already installed libraries or > choose to install =E2=80=9Chtseq=E2=80=9D into a new profile. This would be nice, or at least, is there some kind of "guix package --show-collisions" or something? > 2) avoid PYTHONPATH, patch all Python files invasively! > > Python does not have any feature that is comparable to RUNPATH. It is > only concerned with finding libraries/modules by *name* in one of the > directories specified by the PYTHONPATH environment variable. > > But actually the PYTHONPATH variable is not the only means to affect th= e > search path for modules. It is possible to change the search path > programmatically: > > import sys > sys.path.append("/gnu/store/cabba9e...-numpy.../lib/...") > import numpy > > The first two lines add an explicit store item path to the search path; > the third line just imports the numpy module by name (as usual). Even > without setting the PYTHONPATH to include the numpy in the profile the > third line won=E2=80=99t fail. > > I wonder if we could design a phase that =E2=80=94 very much like the > =E2=80=9Cwrap-program=E2=80=9D phase =E2=80=94 modifies *every* Python = file and injects lines > like the first two in the example above, appending explicit store item > paths, so that all dependent Python libraries can be found without the > need to have them installed in the same profile and without the need to > set PYTHONPATH. > > Maybe this is crazy, and maybe this causes other annoying problems, but > I think it is the closest we can get to a RUNPATH-like feature in > Python. > > What do you think? Do I need a long vacation or is this something we > might realistically do in an automated fashion? > > ~~ Ricardo So... that may work! Except... well, I've raised this on another thread, but I worry about late-binding based plugins or programs that themselves call other python scripts (same with Guile, etc). The previous example I gave was, what if I write a Haunt extension library, and I need to call "haunt.scm" and pull in that stuff? http://lists.gnu.org/archive/html/help-guix/2016-02/msg00032.html What also about MediaGoblin plugins? MediaGoblin allows you to define plugins for different media types, and etc. If you're adding a "gmg_frobnicator" plugin, can MediaGoblin find gmg_frobnicator on its PYTHONPATH when run? And the reverse question: can gmg_frobnicator import back into MediaGoblin and its dependencies? I'm not sure...