bug#10194: 24.0.91; Python scripts run by python-load-file cannot use eggs installed in home directory

bug#10194: 24.0.91; Python scripts run by python-load-file cannot use eggs installed in home directory

[Reuben Thomas]

When I C-c C-l a file in python-mode which contains:

import sys
print (sys.path)

the output does not include eggs in ~/.local:

['/home/rrt/Software/gnat/', '/usr/local/lib/python2.7/dist-packages/Sphinx-1.1.2-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/Jinja2-2.6-py2.7.egg', '/usr/local/share/emacs/24.0.91/etc', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/home/rrt/.local/lib/python2.7/site-packages', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PIL', '/usr/lib/python2.7/dist-packages/gst-0.10', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7', '/usr/lib/python2.7/dist-packages/ubuntu-sso-client']

and hence modules installed as eggs in my home directory are not found if I try to import them. If from within the session in the "*Python*" buffer I "print (sys.path)", it has changed:

['/home/rrt/.local/lib/python2.7/site-packages/pymeta2-0.0.0-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/Sphinx-1.1.2-py2.7.egg', '/usr/local/lib/python2.7/dist-packages/Jinja2-2.6-py2.7.egg', '/usr/local/share/emacs/24.0.91/etc', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/home/rrt/.local/lib/python2.7/site-packages', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PIL', '/usr/lib/python2.7/dist-packages/gst-0.10', '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7', '/usr/lib/python2.7/dist-packages/ubuntu-sso-client']

i.e. it seems that /home/rrt/.local/lib/python2.7/site-packages/easy-install.pth seems to be run after the python-mode buffer has already been loaded into the python process, but before control is returned to the user.


In GNU Emacs 24.0.91.3 (i686-pc-linux-gnu, GTK+ Version 2.24.6)
 of 2011-11-25 on mord
Windowing system distributor `The X.Org Foundation', version 11.0.11004000
Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: en_GB.UTF-8
  value of $LC_CTYPE: en_GB.UTF-8
  value of $LC_MESSAGES: en_GB.UTF-8
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_GB.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Text

Minor modes in effect:
  buffer-face-mode: t
  flyspell-mode: t
  smart-quotes-mode: t
  recentf-mode: t
  show-paren-mode: t
  server-mode: t
  savehist-mode: t
  minibuffer-electric-default-mode: t
  iswitchb-mode: t
  icomplete-mode: t
  global-whitespace-mode: t
  global-auto-revert-mode: t
  desktop-save-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <right> <right> <right> <right> <left> 
<M-backspace> C-x C-s m u s t M-d <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<left> <left> <left> <right> <left> <left> <left> <left> 
<left> <left> <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> t h 
e SPC p r o j e c t s SPC i t SPC <backspace> SPC f 
u n d s SPC w i l l SPC b e M-d M-d M-d M-d M-d M-d 
M-d M-d M-d M-d M-d M-d M-d M-d <right> <right> <right> 
<left> M-d M-d b y SPC m e a n s SPC o f SPC M-d M-d 
t h e i r <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> C-x C-s C-x C-f 
B r i t t e n <return> W o l c u m SPC 2 : 1 8 <return> 
T h a t SPC Y o n g e SPC C h i l d e SPC <backspace> 
: S-SPC 6 4 <backspace> : <backspace> <backspace> <backspace> 
<backspace> SPC 6 : 4 6 C-a C-k O h SPC m y SPC d e 
r e SPC h e r t S-SPC 8 : 3 5 <return> C-x C-s <left> 
SPC 9 : 5 <left> <left> <left> <backspace> - C-e 5 
C-x C-s <help-echo> M-x r e p o r t - e m a c s - e 
<backspace> b <tab> <return>

Recent messages:
byte-code: End of buffer [20 times]
Saving file /home/rrt/FSFE Horizon 2020.new...
Wrote /home/rrt/FSFE Horizon 2020.new
(New file)
Auto-saving...done
Auto-saving...done
Saving file /home/rrt/Britten...
Wrote /home/rrt/Britten
Saving file /home/rrt/Britten...
Wrote /home/rrt/Britten

Load-path shadows:
/home/rrt/local/share/emacs/site-lisp/dict hides /usr/local/share/emacs/24.0.91/site-lisp/emacs-goodies-el/dict
/home/rrt/local/share/emacs/site-lisp/gforth hides /usr/local/share/emacs/24.0.91/site-lisp/gforth/gforth
/usr/local/share/emacs/24.0.91/site-lisp/auctex/tex-style hides /usr/share/emacs/site-lisp/auctex/tex-style
/usr/local/share/emacs/24.0.91/site-lisp/auctex/tex-buf hides /usr/share/emacs/site-lisp/auctex/tex-buf
/usr/local/share/emacs/24.0.91/site-lisp/auctex/context hides /usr/share/emacs/site-lisp/auctex/context
/usr/local/share/emacs/24.0.91/site-lisp/auctex/bib-cite hides /usr/share/emacs/site-lisp/auctex/bib-cite
/usr/local/share/emacs/24.0.91/site-lisp/auctex/tex-fold hides /usr/share/emacs/site-lisp/auctex/tex-fold
/usr/local/share/emacs/24.0.91/site-lisp/auctex/tex-jp hides /usr/share/emacs/site-lisp/auctex/tex-jp
/usr/local/share/emacs/24.0.91/site-lisp/auctex/context-nl hides /usr/share/emacs/site-lisp/auctex/context-nl
/usr/local/share/emacs/24.0.91/site-lisp/auctex/toolbar-x hides /usr/share/emacs/site-lisp/auctex/toolbar-x
/usr/local/share/emacs/24.0.91/site-lisp/auctex/tex-mik hides /usr/share/emacs/site-lisp/auctex/tex-mik
/usr/local/share/emacs/24.0.91/site-lisp/auctex/context-en hides /usr/share/emacs/site-lisp/auctex/context-en
/usr/local/share/emacs/24.0.91/site-lisp/auctex/texmathp hides /usr/share/emacs/site-lisp/auctex/texmathp
/usr/local/share/emacs/24.0.91/site-lisp/auctex/tex-info hides /usr/share/emacs/site-lisp/auctex/tex-info
/usr/local/share/emacs/24.0.91/site-lisp/auctex/tex-fptex hides /usr/share/emacs/site-lisp/auctex/tex-fptex
/usr/local/share/emacs/24.0.91/site-lisp/auctex/tex-font hides /usr/share/emacs/site-lisp/auctex/tex-font
/usr/local/share/emacs/24.0.91/site-lisp/auctex/latex hides /usr/share/emacs/site-lisp/auctex/latex
/usr/local/share/emacs/24.0.91/site-lisp/auctex/font-latex hides /usr/share/emacs/site-lisp/auctex/font-latex
/usr/local/share/emacs/24.0.91/site-lisp/auctex/tex-bar hides /usr/share/emacs/site-lisp/auctex/tex-bar
/usr/local/share/emacs/24.0.91/site-lisp/auctex/multi-prompt hides /usr/share/emacs/site-lisp/auctex/multi-prompt
/usr/local/share/emacs/24.0.91/site-lisp/auctex/tex hides /usr/share/emacs/site-lisp/auctex/tex

Features:
(shadow sort gnus-util mail-extr message format-spec rfc822 mml mml-sec
mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mailabbrev mail-utils gmm-utils mailheader
emacsbug find-func help-mode view cus-edit multi-isearch vc-git
cperl-mode ansi-color info python-21 python flymake compile comint ring
face-remap regexp-opt flyspell smart-quotes jka-compr
auto-dictionary-autoloads c-eldoc-autoloads dictionary-autoloads
diff-git-autoloads dired-isearch-autoloads full-ack-autoloads
guess-style-autoloads kill-ring-search-autoloads magit-autoloads
mv-shell-autoloads tumble-autoloads http-post-simple-autoloads package
tabulated-list completing-help recentf tree-widget wid-edit uniquify
paren server savehist minibuf-eldef iswitchb icomplete whitespace
autorevert desktop cus-start cus-load ropemacs pymacs go-mode-load ffap
ispell advice advice-preload yasnippet help-fns derived edmacro kmacro
easymenu assoc cl muse-autoloads emacs-goodies-el emacs-goodies-custom
emacs-goodies-loaddefs easy-mmode devhelp preview-latex tex-site
auto-loads user-site-loaddefs time-date tooltip ediff-hook vc-hooks
lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image fringe
lisp-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer loaddefs button faces cus-face files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dbusbind
dynamic-setting system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty emacs)

-- 
http://rrt.sc3d.org/

bug#10194: Acknowledgement (24.0.91; Python scripts run by python-load-file cannot use eggs installed in home directory)

[Reuben Thomas]

I think I've found the bug: emacs2.py's eimport overwrites sys.path[0]
while loading the given file. sys.path[0] is normally '' (the current
directory), but by default Emacs removes this entry when starting up
Python (python-remove-cwd-from-path), and so instead sys.path[0] is
some non-empty path element, and modules on it cannot be found when
loaded via eimport.

A workaround is to set python-remove-cwd-from-path to nil, but to work
properly, eimport should prepend the file's path to sys.path, thereby
allowing modules to be loaded from the current directory if
python-remove-cwd-from-path is nil, as currently that won't work
either.

-- 
http://rrt.sc3d.org

bug#10194: 24.0.91; Python scripts run by python-load-file cannot use eggs installed in home directory

[Glenn Morris]

This may be fixed in the current Emacs trunk, where there is a new
python.el. I'm afraid I cannot tell.

bug#10194: 24.0.91; Python scripts run by python-load-file cannot use eggs installed in home directory

[Fabián Ezequiel Gallina]

This is indeed fixed in current trunk. The new python.el allows changing 
the environment of the running shell completely and that covers this 
particular use case.