Re: Python and propagation

[Ricardo Wurmus <ricardo.wurmus@mdc-berlin.de>]

Christopher Allan Webber <cwebber@dustycloud.org> writes:

> Ricardo Wurmus writes:
>> 2) avoid PYTHONPATH, patch all Python files invasively!

[...]

> So... that may work!  Except... well, I've raised this on another
> thread, but I worry about late-binding based plugins or programs that
> themselves call other python scripts (same with Guile, etc).  The
> previous example I gave was, what if I write a Haunt extension library,
> and I need to call "haunt.scm" and pull in that stuff?
>
>   http://lists.gnu.org/archive/html/help-guix/2016-02/msg00032.html
>
> What also about MediaGoblin plugins?  MediaGoblin allows you to define
> plugins for different media types, and etc.

I don’t understand the issues, primarily because I don’t understand if
the load path is a global mutable variable for the Python process, or if
it is local to the file, or if maybe it’s something else entirely.

I only proposed manipulating the load path before an “import” statement
because that’s how we can make the “import” statement succeed even if
the PYTHONPATH at programme start does not contain the directory holding
a needed library’s files.  “import” doesn’t allow us to use full paths,
so the idea was to “wrap” the “import” statement in an environment in
which the load path has been sufficiently augmented.  It would not be
feasible to replace every “import foo” statement with something like
this:

    old_load_path = sys.path
    sys.path.append("/gnu/store/cabba9e...-foo.../lib/...")
    import foo
    sys.path = old_load_path

so sys.path is instead just changed once per file.  It would be worth
figuring out how far these changes to sys.path reach — do they have a
global, process-wide effect?  (That would be inconvenient as it’s hard
to reason about a change like this, because ordering becomes important.)

How exactly to manipulate the load path — I don’t know.  We could indeed
recursively load .pth files if they exist, thereby adding things to the
search path.  (I did this in “(@@ (gnu packages music) solfege)” to make
pygtk work, for example.)  Or we could create a list of paths at build
time.

> If you're adding a "gmg_frobnicator" plugin, can MediaGoblin find
> gmg_frobnicator on its PYTHONPATH when run?

It would not affect the loading of plugins, as far as I imagine it,
because plugins would deal with their search paths by themselves (e.g. a
user adds them to the PYTHONPATH).  Since we only augment the load path
instead of replacing it, $PYTHONPATH will always be available.  So the
answer to this question is “yes”, I think.

> And the reverse question: can gmg_frobnicator import back into
> MediaGoblin and its dependencies?

I don’t know what “import back into” means.

I’m not a Python programmer, so I cannot come up with a simple example
project that would allow us to test such an invasive substitution
scheme.  Is anyone else here willing to give this a try?

~~ Ricardo