python-try-complete

python-try-complete

[Julian Scheid]

Hi,

first of all I am writing a lot of Python code these days and 
python-mode is indispensable.  Thanks for this great tool.

I take issue with one of its design decisions however: sneaking in 
python-try-complete into hippie-expand-try-functions-list.  From 
python.el (found in both my 22.1.2 and in HEAD):

   (if (featurep 'hippie-exp)
       (set (make-local-variable 'hippie-expand-try-functions-list)
	   (cons 'python-try-complete hippie-expand-try-functions-list)))

The main reason why I use hippie-expand (and similar completion 
facilities in Emacs) is simply raw speed.  I'll take lightweight, 
context-insensitive completion over heavyweight, database-querying or 
external-process-launching context-sensitive completion any day.

I am defining hippie-expand-try-functions-list explicitly in my .emacs 
and until recently I was under the impression that this way I get to 
choose which route hippie-expand takes to try and complete.  But the 
other day I learned that python-try-complete gets inserted into that 
list without my consent, a function that launches an external process 
and which I would thus consider quite heavyweight.

I understand that this has been done with only the best intentions but I 
would like to kindly ask the maintainers of python-mode to remove this 
and instead advertise the existence of python-try-complete and leave it 
up to hippie-expand users to insert or not insert it into their list of 
functions to try.

Alternatively, there could be a customization setting for this which 
defaults to off (no automatic registration of python-try-complete).  I 
think that the lo-fi approach of having people add it to their list 
explicitly would be superior, though, because this way users get more 
control over the order in which the functions are invoked.

FWIW my current unsophisticated workaround looks like this:

   (require 'python)
   (defun python-try-complete (old) nil)


The second problem I noticed with python-try-complete, and this is what 
made me aware of it being used in the first place, is that it will 
apparently launch the Python interpreter on a remote host if the file in 
question is remote, opened using Tramp in my case. This remote 
invocation often (always?) fails, apparently due to shell quoting issues.

That wouldn't be a big problem if the completion attempt would simply be 
ignored in case of failure; however it seems as if python-try-complete 
would get stuck this way (I think it's waiting indefinitely for a reply 
from the remote Python script), forcing me to C-g out of it which is 
quite a nuisance in interactive usage when all you wanted is get an 
expansion of the word at point.

I got the following information from Michael, the tramp maintainer, 
about this situation - and I'm not quoting directly here, any errors in 
the following are mine and not his:

Either python-try-complete should refrain from launching the process 
remotely by looking at (file-remote-p default-directory), or the scripts 
it launches need to be prepared (e.g. in terms of quoting) for being run 
remotely via ssh.

If you need backtraces or similar for tracking this down I'm happy to 
provide them the next time I'm in a situation to reproduce the issue.

Let me know in case my explanation of these problems was insufficient.

I'm using Emacs 22.1.2, hippie-expand 1.6, tramp 2.1.13, and python.el 
that ships with my Emacs.

-Julian

Re: python-try-complete

[Stefan Monnier]

> I take issue with one of its design decisions however: sneaking in
> python-try-complete into hippie-expand-try-functions-list.

Agreed, it should be fixed on the EMACS_22_BASE branch now,


        Stefan

Re: python-try-complete

[Julian Scheid]

>> I take issue with one of its design decisions however: sneaking in
>> python-try-complete into hippie-expand-try-functions-list.
> 
> Agreed, it should be fixed on the EMACS_22_BASE branch now,

Thanks for that, Stefan.

Did you have any thoughts about the second problem I described in the 
same mail - python-try-complete locking up when used in a remote buffer? 
To quote from my previous mail,

   Either python-try-complete should refrain from launching the process
   remotely by looking at (file-remote-p default-directory), or the
   scripts it launches need to be prepared (e.g. in terms of quoting) for
   being run remotely via ssh.

I am not using python-try-complete anymore but this is a bug and I would 
be happy to help fix it.

Julian

Re: python-try-complete

[Julian Scheid]

> Did you have any thoughts about the second problem I described in the 
> same mail - python-try-complete locking up when used in a remote buffer? 
> To quote from my previous mail,
> 
>   Either python-try-complete should refrain from launching the process
>   remotely by looking at (file-remote-p default-directory), or the
>   scripts it launches need to be prepared (e.g. in terms of quoting) for
>   being run remotely via ssh.

I had a chance to narrow it down a bit more. Here is the stack trace I 
get when I quit while python-try-complete blocks:

Debugger entered--Lisp error: (quit)
   accept-process-output(#<process Python> 5)
   python-send-receive("import emacs; print '_emacs_out ()'")
   run-python(nil t)
   python-proc()
   python-send-string("emacs.complete(\"exitCode\",\"import os\\nimport 
logging\\nimport Queue\\nimport threading\\n\")")
   python-send-receive("emacs.complete(\"exitCode\",\"import os\\nimport 
logging\\nimport Queue\\nimport threading\\n\")")
   byte-code("..." [symbol python-imports read-from-string 
python-send-receive format "emacs.complete(%S,%s)"] 6)
   python-symbol-completions("exitCode")
   (and symbol (python-symbol-completions symbol))
   (setq he-expand-list (and symbol (python-symbol-completions symbol)))
   (let ((symbol ...)) (he-init-string (- ... ...) (point)) (if (not 
...) (push he-search-string he-tried-table)) (setq he-expand-list (and 
symbol ...)))
   (if old nil (let (...) (he-init-string ... ...) (if ... ...) (setq 
he-expand-list ...)))
   (unless old (let (...) (he-init-string ... ...) (if ... ...) (setq 
he-expand-list ...)))
   (progn (unless old (let ... ... ... ...)) (while (and he-expand-list 
...) (pop he-expand-list)) (if he-expand-list (progn ... t) (if old ...) 
nil))
   (if (derived-mode-p (quote python-mode)) (progn (unless old ...) 
(while ... ...) (if he-expand-list ... ... nil)))
   (when (derived-mode-p (quote python-mode)) (unless old (let ... ... 
... ...)) (while (and he-expand-list ...) (pop he-expand-list)) (if 
he-expand-list (progn ... t) (if old ...) nil))
   python-try-complete(nil)
   apply(python-try-complete nil)
   hippie-expand(nil)
   call-interactively(hippie-expand)

And in *Python* I have:

Python 2.3.4 (#1, Feb  2 2005, 12:11:53)
[GCC 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> Traceback (most recent call last):
   File "<stdin>", line 1, in ?
ImportError: No module named emacs
 >>>

Now the interesting thing with the second line of this output is that my 
Emacs is running on top of OS X; Linux is running on the box on which 
the Python file lives that I've opened using Tramp (that was visited 
when I ran hippie-expand.)

So apparently Python gets run remotely (by the magic of Tramp?) but the 
"emacs" package which I suspect python-mode uses for helping with 
completion isn't available on the remote host; thus the whole completion 
attempt fails, but instead of gracefully failing python-try-complete 
sits there indefinitely.

There seem to be two problems here:

1) Proper exception and/or exit code handling is missing which would 
allow the Python script to exit with an error without locking up Emacs.

2) Either the external Python process should be forced to run locally 
even if the file is remote; or python-try-complete should refrain from 
launching a Python process at all if the file is remote; or 
python-try-complete should be prepared for its external process to run 
remotely and set up its environment properly (by uploading the emacs 
module and setting PYTHONPATH accordingly.)

As I said earlier I don't use python-try-complete anymore but FWIW 
simply not running Python when file-remote-p looks to me to be the best 
option.

I'm using Emacs 22.1.2 and tramp 2.1.13.

Hope this helps,

Julian

bug#140: python-try-complete

[Chong Yidong]

Julian Scheid <julians37@googlemail.com> writes:

> Did you have any thoughts about the second problem I described in the
> same mail - python-try-complete locking up when used in a remote
> buffer? To quote from my previous mail,
>
>   Either python-try-complete should refrain from launching the process
>   remotely by looking at (file-remote-p default-directory), or the
>   scripts it launches need to be prepared (e.g. in terms of quoting) for
>   being run remotely via ssh.

I think Michael Albinus finally fixed this in June (r108500).  Closing.