Re: Python and propagation

[Danny Milosavljevic <dannym@scratchpost.org>]

Or:

3) Use Python importlib[1] metapath functionality[2] in order to register your own module finder early[3].

You can override the entire module loading functionality like this, both the finding of modules and the loading them (off a stone tablet for example ^^).
You can even override builtin (!!) module loaders that way.

The default sys.metapath is [<class '_frozen_importlib.BuiltinImporter'>, <class '_frozen_importlib.FrozenImporter'>, <class '_frozen_importlib.PathFinder'>].

(All these classes do not need to be instantiated since you use them via classmethods)

The frozen importer is used in order to find dependencies in order to build a self-contained special python interpreter which includes all the dependencies you use (see Tools/freeze/ [4]).

So while I'm not sure what you want to achieve in detail (I've read some of the E-Mails in this thread but not every word), I think you can either use "freeze" or modify sitecustomize.py in order to make it magically modify metapath so it does what you want - without patching any of the other files.

(For comparison, check out what pip freeze does (despite the name, it just determines dependencies - a la ldd - and prints them))

On the other hand, virtualenv also builds a special Python interpreter.

Note that if you want to do something equivalent to 

  sys.path = ["a"]
  import x
  sys.path = ["b"]
  import x

if both a/x.py and b/x.py exist (and are different) and have the latter "import" import another x (after the former "import" succeeded), that's impossible and Python's semantics forbid that that works in any case. The latter import must use the already-loaded module (because of the same name). Also, the name "x" is visible in the remainder of the script that did the import, so no renaming either. Since Python does dynamic binding, you also can't just have the first "x" be visible in the first half and the second "x" in the second half - although (only) Python3 changed some stuff to allow lexical binding in some special cases.

Also not:

  sys.path = ["a"]
  import x
  sys.path = ["c"]
  import d

where
  c/d.py:
    sys.path = ["../b"]
    import x # nope! It's still the old one!

Also, it's customary to do this:

  a/__init__.py:
    x = 3
    import b
    y = 4
  a/b.py:
    import a
    print(a.x)
  main.py:
    import a
    print(a.y)

and it should work fine, printing 3 and 4. So while a module is being loaded, it should be able to import modules. (here, a/__init__.py is running, importing b, and b is using "a" although it's only half-there - b at this point in time also only sees the first half - i.e. it doesn't see y yet).

[1] https://docs.python.org/3/library/importlib.html
[2] https://www.python.org/dev/peps/pep-0302/ (search for "metapath")
[3] in lib/python.../site-packages/sitecustomize.py
[4] https://wiki.python.org/moin/Freeze