bug#23609: 25.0.92; Python eldoc freeze

bug#23609: 25.0.92; Python eldoc freeze

[Jules Tamagnan]

This is an issue in relation to python and the interpetter shell. If
the python shell is running something and it is not done then when the
user goes back the the document that was being editted emacs will
freeze. If the user hits C-g then a keyboard interupt is sent to the
interpeter and the action that was being run is canceled. If the user
has the misfortune of using the mouse to click into another window then
emacs will not be able to be recovered (haven't tested that last part
entirely but the easiest way is to get back to working is to 'sudo
killall emacs')

The issue seems to be with eldoc mode. If eldoc mode (or
global-eldoc-mode) is on then python.el tries to send something to the
interepter to get the definition of the symbol at the point and then
since something is running in the interpetter emacs will hang while
waiting for the interpetter to repond. This issue is most obvious in
emacs 25 because global-eldoc-mode is on by default. This causes loss of
data.

This bug is easily reproducable from emacs -Q.
 - run emacs -Q
 - Open a file /tmp/test.py (or any other python file)
 - C-c C-p to start the interpetter (it does not matter if the interpetter
   is dedicated or not)
 - in the interpetter write an expression that will not terminate
   e.g. while True: 2+2
   e.g. (anything that plots in matplotlib)
 - go back to /tmp/test.py and type anything.
 - you will now be frozen
 - type C-g to send a keyboard interupt to the interpetter

repeating these steps without global-eldoc-mode does not cause any
issues

The most obvious fix that I have found and use is to set
global-eldoc-mode to -1 on init.



In GNU Emacs 25.0.92.1 (x86_64-pc-linux-gnu, GTK+ Version 3.18.9)
 of 2016-04-07 built on bandersnatch
Repository revision: 6287381da495ac5cc2a2a4acb7f49825677a44c8
System Description:	Debian GNU/Linux testing (stretch)

Configured using:
 'configure --with-x-toolkit=gtk3 --with-xwidgets CC=gcc'

Configured features:
XPM JPEG TIFF GIF PNG SOUND DBUS GSETTINGS NOTIFY LIBXML2 FREETYPE
LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XWIDGETS

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Python

Minor modes in effect:
  shell-dirtrack-mode: t
  gpm-mouse-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
Shell native completion is disabled, using fallback
p is undefined
y is undefined
t is undefined
Type "q" in help window to restore its previous buffer, C-M-v to scroll help.
user-error: No cross-reference here
Quit
eldoc error: (wrong-type-argument arrayp nil)
Quit
Making completion list... [4 times]

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message dired rfc822 mml mml-sec epg
epg-config mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mail-utils
thingatpt warnings compile tramp-cache python tramp-sh tramp
tramp-compat auth-source cl-seq eieio eieio-core cl-macs gnus-util
mm-util help-fns mail-prsvr password-cache tramp-loaddefs trampver
ucs-normalize shell pcomplete format-spec advice json map seq byte-opt
gv bytecomp byte-compile cconv cl-extra help-mode easymenu comint ring
cl-loaddefs pcase cl-lib ansi-color t-mouse time-date mule-util tooltip
eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list newcomment elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help
simple abbrev minibuffer cl-preloaded nadvice loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
dbusbind inotify dynamic-setting system-font-setting font-render-setting
xwidget-internal move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
conses 16 237883 7352)
 (symbols 48 23012 0)
 (miscs 40 116 165)
 (strings 32 25679 7181)
 (string-bytes 1 1156766)
 (vectors 16 35555)
 (vector-slots 8 664891 2342)
 (floats 8 238 254)
 (intervals 56 1567 1139)
 (buffers 976 19)
 (heap 1024 23963 1702))

bug#23609: 25.0.92; Python eldoc freeze

[Andreas Röhler]

On 24.05.2016 17:34, Jules Tamagnan wrote:
> This is an issue in relation to python and the interpetter shell. If
> the python shell is running something and it is not done then when the
> user goes back the the document that was being editted emacs will
> freeze. If the user hits C-g then a keyboard interupt is sent to the
> interpeter and the action that was being run is canceled. If the user
> has the misfortune of using the mouse to click into another window then
> emacs will not be able to be recovered (haven't tested that last part
> entirely but the easiest way is to get back to working is to 'sudo
> killall emacs')
>
> The issue seems to be with eldoc mode. If eldoc mode (or
> global-eldoc-mode) is on then python.el tries to send something to the
> interepter to get the definition of the symbol at the point and then
> since something is running in the interpetter emacs will hang while
> waiting for the interpetter to repond. This issue is most obvious in
> emacs 25 because global-eldoc-mode is on by default. This causes loss of
> data.
>
> This bug is easily reproducable from emacs -Q.
>   - run emacs -Q
>   - Open a file /tmp/test.py (or any other python file)
>   - C-c C-p to start the interpetter (it does not matter if the interpetter
>     is dedicated or not)
>   - in the interpetter write an expression that will not terminate
>     e.g. while True: 2+2
>     e.g. (anything that plots in matplotlib)
>   - go back to /tmp/test.py and type anything.
>   - you will now be frozen
>   - type C-g to send a keyboard interupt to the interpetter
>
> repeating these steps without global-eldoc-mode does not cause any
> issues
>
> The most obvious fix that I have found and use is to set
> global-eldoc-mode to -1 on init.
>
>

May reproduce this with current pretest

GNU Emacs 25.0.94.1 (i686-pc-linux-gnu, GTK+ Version 2.24.25) of 2016-05-18

bug#23609: 25.0.92; Python eldoc freeze

[Jules Tamagnan]

Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
> May reproduce this with current pretest
>
> GNU Emacs 25.0.94.1 (i686-pc-linux-gnu, GTK+ Version 2.24.25) of 2016-05-18

Are you saying that it can be reproduced or that I should try to
reproduce it?

This seems like a big issue before release because it breaks many python
workflows, it was not an issue in 24.x because global-eldoc mode was not
on by default. Until this bug is worked out I believe that global-eldoc
mode should not be on by default at the very least it should not be on
in python.

-- message originally, accidentally, sent only to Andreas

bug#23609: 25.0.92; Python eldoc freeze

[Andreas Röhler]

On 25.05.2016 17:18, Jules Tamagnan wrote:
> Andreas Röhler <andreas.roehler@easy-emacs.de> writes:
>> May reproduce this with current pretest
>>
>> GNU Emacs 25.0.94.1 (i686-pc-linux-gnu, GTK+ Version 2.24.25) of 2016-05-18
> Are you saying that it can be reproduced

Yes. Confirmed your report.

>   or that I should try to
> reproduce it?
>
> This seems like a big issue before release because it breaks many python
> workflows, it was not an issue in 24.x because global-eldoc mode was not
> on by default. Until this bug is worked out I believe that global-eldoc
> mode should not be on by default at the very least it should not be on
> in python.
>
> -- message originally, accidentally, sent only to Andreas
>
>
>

bug#23609: 25.0.92; Python eldoc freeze

[Jules Tamagnan]

Should this issue be fixed before the release, if so should the urgency
of this issue be increased. It seems like a pretty large regression from
emacs 24.

bug#23609: 25.0.92; Python eldoc freeze

[Eli Zaretskii]

> From: Jules Tamagnan <jtamagnan@gmail.com>
> Date: Thu, 26 May 2016 11:52:50 -0400
> 
> Should this issue be fixed before the release, if so should the urgency
> of this issue be increased. It seems like a pretty large regression from
> emacs 24.

Not IMO, since an easy workaround is available.

bug#23609: 25.0.92; Python eldoc freeze

[Dmitry Gutov]

On 05/26/2016 07:31 PM, Eli Zaretskii wrote:

> Not IMO, since an easy workaround is available.

Disabling eldoc-mode? It's cutting off a very handy feature.

bug#23609: 25.0.92; Python eldoc freeze

[Eli Zaretskii]

> Cc: 23609@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Thu, 26 May 2016 19:37:57 +0300
> 
> On 05/26/2016 07:31 PM, Eli Zaretskii wrote:
> 
> > Not IMO, since an easy workaround is available.
> 
> Disabling eldoc-mode? It's cutting off a very handy feature.

A feature that wasn't turned on in 24.5, and whose turning on exposed
this bug.  At least that's my understanding.

If I'm right, then either this isn't a regression, or a regression
with an easy workaround.

Which doesn't mean we shouldn't try fixing this bug, just that it is
not critical enough to delay the release if not fixed.  Motivated
volunteers are very much encouraged to work on a fix; if that fix is
safe, I see no reason not to accept it on the release branch.

Thanks.

bug#23609: 25.0.92; Python eldoc freeze

[Jules Tamagnan]

Eli Zaretskii <eliz@gnu.org> writes:

>> Cc: 23609@debbugs.gnu.org
>> From: Dmitry Gutov <dgutov@yandex.ru>
>> Date: Thu, 26 May 2016 19:37:57 +0300
>> 
>> On 05/26/2016 07:31 PM, Eli Zaretskii wrote:
>> 
>> > Not IMO, since an easy workaround is available.
>> 
>> Disabling eldoc-mode? It's cutting off a very handy feature.
>
> A feature that wasn't turned on in 24.5, and whose turning on exposed
> this bug.  At least that's my understanding.
>
> If I'm right, then either this isn't a regression, or a regression
> with an easy workaround.
>
> Which doesn't mean we shouldn't try fixing this bug, just that it is
> not critical enough to delay the release if not fixed.  Motivated
> volunteers are very much encouraged to work on a fix; if that fix is
> safe, I see no reason not to accept it on the release branch.
>
> Thanks.

I've already filled out the proper paperwork and wouldn't mind doing
it. Would a timeout, of one second seem acceptable? If getting the
documentation takes longer than 1 second then it should timeout and we
should then locally set global-eldoc and eldoc mode to -1.

Thanks

bug#23609: 25.0.92; Python eldoc freeze

[Andreas Röhler]

On 26.05.2016 18:37, Dmitry Gutov wrote:
> On 05/26/2016 07:31 PM, Eli Zaretskii wrote:
>
>> Not IMO, since an easy workaround is available.
>
> Disabling eldoc-mode? It's cutting off a very handy feature.
>
>
>

Not enabling by default doesn't mean cutting it off.

For several reasons eldoc-mode is brittle.
Don't assume python.el being the only one with such a bug.

When keeping on by default, there is a risk users of the release will 
face a fair number of so far unknown bugs.

bug#23609: 25.0.92; Python eldoc freeze

[Eli Zaretskii]

> From: Jules Tamagnan <jtamagnan@gmail.com>
> Cc: Dmitry Gutov <dgutov@yandex.ru>,  23609@debbugs.gnu.org
> Date: Thu, 26 May 2016 13:13:27 -0400
> 
> I've already filled out the proper paperwork and wouldn't mind doing
> it. Would a timeout, of one second seem acceptable? If getting the
> documentation takes longer than 1 second then it should timeout and we
> should then locally set global-eldoc and eldoc mode to -1.

Sorry, I'm not sure I'm following: what timeout are you talking about?
(I've re-read the bug discussion, and there's no reference to any
timeouts there, either.)

Perhaps show a patch you had in mind, then your intent will be clear?

Thanks.

bug#23609: 25.0.92; Python eldoc freeze

[Eli Zaretskii]

Please don't add this bug to the list of those which block 25.1
without discussing that here first.  I explained why I think it
shouldn't block the release; if you have counter-arguments, let's
talk.

Thanks.

bug#23609: 25.0.92; Python eldoc freeze

[Jules Tamagnan]

Eli Zaretskii <eliz@gnu.org> writes:

> Please don't add this bug to the list of those which block 25.1
> without discussing that here first.  I explained why I think it
> shouldn't block the release; if you have counter-arguments, let's
> talk.
>
> Thanks.

I didn't, but I do think that something should be done before the
release. If somebody opens up emacs for the first time in 25.1 and
decides to try using python and then it doesn't work and keeps freezing,
I don't think many people would be willing to debug their issue in a
completely different environment than what they're used to when they
could simply use a different editor that works out of the box. Emacs is
already seen as not very user friendly compared to other (inferior)
editors such as atom and sublime and If we have such a glaring bug in
one of the most popular language modes then that reflects badly on emacs
as a whole.

I'll try and fix this bug suitably beforehand because I do think that it
should be blocking.

Why was global-eldoc mode turned on by default anyways?

thanks

bug#23609: 25.0.92; Python eldoc freeze

[Glenn Morris]

Eli Zaretskii wrote:

> Please don't add this bug to the list of those which block 25.1
> without discussing that here first.

That's a new rule.

It's a bad regression that could easily be worked around by defaulting
eldoc mode to off in python mode. That could be done very easily.

bug#23609: 25.0.92; Python eldoc freeze

[Jules Tamagnan]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Jules Tamagnan <jtamagnan@gmail.com>
>> Cc: Dmitry Gutov <dgutov@yandex.ru>,  23609@debbugs.gnu.org
>> Date: Thu, 26 May 2016 13:13:27 -0400
>> 
>> I've already filled out the proper paperwork and wouldn't mind doing
>> it. Would a timeout, of one second seem acceptable? If getting the
>> documentation takes longer than 1 second then it should timeout and we
>> should then locally set global-eldoc and eldoc mode to -1.
>
> Sorry, I'm not sure I'm following: what timeout are you talking about?
> (I've re-read the bug discussion, and there's no reference to any
> timeouts there, either.)
>
> Perhaps show a patch you had in mind, then your intent will be clear?
>
> Thanks.

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 5aeefe4..8f01ac9 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -4293,12 +4293,18 @@ python-eldoc--get-doc-at-point
         (unless (zerop (length docstring))
           docstring)))))
 
+(defvar-local python-eldoc-get-doc t
+  "If eldoc should be used to get the documentation automatically")
+
 (defun python-eldoc-function ()
   "`eldoc-documentation-function' for Python.
 For this to work as best as possible you should call
 `python-shell-send-buffer' from time to time so context in
 inferior Python process is updated properly."
-  (python-eldoc--get-doc-at-point))
+  (when python-eldoc-get-doc
+    (with-timeout (1
+                   (setq python-eldoc-get-doc nil))
+      (python-eldoc--get-doc-at-point))))
 
 (defun python-eldoc-at-point (symbol)
   "Get help on SYMBOL using `help'.

This patch adds the buffer local variable 'python-eldoc-get-doc which is
intially set to t but if eldoc takes longer than one second to fetch the
documentation then it will be set to nil. With the variable being nil,
eldoc will no longer try to automatically fetch the documentation. The
documentation can still be fetched with "C-cC-f". This avoids removing
global-eldoc mode and allows the user to manually fetch the
documentation while disabling the automatic fetching which causes the
unexpected freezing. If the user calls eldoc themselves then it can
still freeze but then thats their fault and they can press "C-g" to send
a keyboard interupt to their process. At least it won't freeze
spontaneously.

bug#23609: 25.0.92; Python eldoc freeze

[Dmitry Gutov]

On 05/27/2016 07:12 PM, Jules Tamagnan wrote:

> +(defvar-local python-eldoc-get-doc t
> +  "If eldoc should be used to get the documentation automatically")

Why not unset/un-advise eldoc-documentation-function instead?

A new variable doesn't seem necessary. Having python-eldoc-function do 
nothing in certain cases, on the other hand, might look puzzling.

bug#23609: 25.0.92; Python eldoc freeze

[Jules Tamagnan]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 05/27/2016 07:12 PM, Jules Tamagnan wrote:
>
>> +(defvar-local python-eldoc-get-doc t
>> +  "If eldoc should be used to get the documentation automatically")
>
> Why not unset/un-advise eldoc-documentation-function instead?
>
> A new variable doesn't seem necessary. Having python-eldoc-function do
> nothing in certain cases, on the other hand, might look puzzling.

I thought that this would make it clearer and easier to check if the
automatic eldoc check was disabled. Also easier to reset if somebody
wanted to. Python-eldoc-function isn't interactive and is not used
anywhere else so I didn't think it would be a big issue if it were
simply disabled

bug#23609: 25.0.92; Python eldoc freeze

[Dmitry Gutov]

On 05/27/2016 08:12 PM, Jules Tamagnan wrote:

> I thought that this would make it clearer and easier to check if the
> automatic eldoc check was disabled. Also easier to reset if somebody
> wanted to. Python-eldoc-function isn't interactive and is not used
> anywhere else so I didn't think it would be a big issue if it were
> simply disabled

OK, that makes sense. I think it'd be better to make that variable 
private, though. So we don't have to carry it on after the bug is fixed 
properly.

bug#23609: 25.0.92; Python eldoc freeze

[Jules Tamagnan]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 05/27/2016 08:12 PM, Jules Tamagnan wrote:
>
>> I thought that this would make it clearer and easier to check if the
>> automatic eldoc check was disabled. Also easier to reset if somebody
>> wanted to. Python-eldoc-function isn't interactive and is not used
>> anywhere else so I didn't think it would be a big issue if it were
>> simply disabled
>
> OK, that makes sense. I think it'd be better to make that variable
> private, though. So we don't have to carry it on after the bug is
> fixed properly.

What do you mean by private?

Thanks

bug#23609: 25.0.92; Python eldoc freeze

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: jtamagnan@gmail.com,  dgutov@yandex.ru,  23609@debbugs.gnu.org
> Date: Fri, 27 May 2016 11:42:22 -0400
> 
> Eli Zaretskii wrote:
> 
> > Please don't add this bug to the list of those which block 25.1
> > without discussing that here first.
> 
> That's a new rule.

No, it isn't.  I was asked whether the bug should block, and answered
that it shouldn't, providing my rationale.  Given that answer, common
sense would have it that adding the bug to the list without discussion
is not TRT.

> It's a bad regression that could easily be worked around by defaulting
> eldoc mode to off in python mode. That could be done very easily.

That a bug doesn't block the release doesn't mean it cannot be easily
fixed, or that a fix won't be accepted on the release branch.  I said
that as well back when I was asked.

bug#23609: 25.0.92; Python eldoc freeze

[Dmitry Gutov]

On 05/27/2016 09:39 PM, Jules Tamagnan wrote:

> What do you mean by private?

It's when a variable or a function has "--" in its name. This functions 
purely as a documentation that external code should not rely on it (or 
refer to it, really).

E.g. python-eldoc--get-doc.

bug#23609: 25.0.92; Python eldoc freeze

[Jules Tamagnan]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 05/27/2016 09:39 PM, Jules Tamagnan wrote:
>
>> What do you mean by private?
>
> It's when a variable or a function has "--" in its name. This
> functions purely as a documentation that external code should not rely
> on it (or refer to it, really).
>
> E.g. python-eldoc--get-doc.

Oh yes, well here it is with the different naming convention:

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 5aeefe4..2da72fd 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -4293,12 +4293,18 @@ python-eldoc--get-doc-at-point
         (unless (zerop (length docstring))
           docstring)))))
 
+(defvar-local python-eldoc--get-doc t
+  "If eldoc should be used to get the documentation automatically")
+
 (defun python-eldoc-function ()
   "`eldoc-documentation-function' for Python.
 For this to work as best as possible you should call
 `python-shell-send-buffer' from time to time so context in
 inferior Python process is updated properly."
-  (python-eldoc--get-doc-at-point))
+  (when python-eldoc--get-doc
+    (with-timeout (1
+                   (setq python-eldoc--get-doc nil))
+      (python-eldoc--get-doc-at-point))))
 
 (defun python-eldoc-at-point (symbol)
   "Get help on SYMBOL using `help'.

bug#23609: 25.0.92; Python eldoc freeze

[Eli Zaretskii]

> From: Jules Tamagnan <jtamagnan@gmail.com>
> Cc: 23609@debbugs.gnu.org,  dgutov@yandex.ru
> Date: Fri, 27 May 2016 12:12:26 -0400
> 
> diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
> index 5aeefe4..8f01ac9 100644
> --- a/lisp/progmodes/python.el
> +++ b/lisp/progmodes/python.el
> @@ -4293,12 +4293,18 @@ python-eldoc--get-doc-at-point
>          (unless (zerop (length docstring))
>            docstring)))))
>  
> +(defvar-local python-eldoc-get-doc t
> +  "If eldoc should be used to get the documentation automatically")
> +
>  (defun python-eldoc-function ()
>    "`eldoc-documentation-function' for Python.
>  For this to work as best as possible you should call
>  `python-shell-send-buffer' from time to time so context in
>  inferior Python process is updated properly."
> -  (python-eldoc--get-doc-at-point))
> +  (when python-eldoc-get-doc
> +    (with-timeout (1
> +                   (setq python-eldoc-get-doc nil))
> +      (python-eldoc--get-doc-at-point))))
>  
>  (defun python-eldoc-at-point (symbol)
>    "Get help on SYMBOL using `help'.
> 
> This patch adds the buffer local variable 'python-eldoc-get-doc which is
> intially set to t but if eldoc takes longer than one second to fetch the
> documentation then it will be set to nil. With the variable being nil,
> eldoc will no longer try to automatically fetch the documentation. The
> documentation can still be fetched with "C-cC-f". This avoids removing
> global-eldoc mode and allows the user to manually fetch the
> documentation while disabling the automatic fetching which causes the
> unexpected freezing. If the user calls eldoc themselves then it can
> still freeze but then thats their fault and they can press "C-g" to send
> a keyboard interupt to their process. At least it won't freeze
> spontaneously.

Thanks, this is now clear.  A coupe of comments:

  . Should 1 be a defcustom?  IOW, could it be that different users
    have different annoyance thresholds, and are willing to wait
    longer before turning eldoc-mode off?

  . The problem happens only when the interpreter is busy doing
    something, is that right?  If so, perhaps we shouldn't turn
    eldoc-mode off, but instead simply return?

  . If we do turn eldoc-mode off, then I think a message to that
    effect is in order, to let the user know.

Thanks for working on this.

bug#23609: 25.0.92; Python eldoc freeze

[Eli Zaretskii]

> From: Jules Tamagnan <jtamagnan@gmail.com>
> Cc: Glenn Morris <rgm@gnu.org>,  23609@debbugs.gnu.org,  dgutov@yandex.ru
> Date: Fri, 27 May 2016 10:52:43 -0400
> 
> Why was global-eldoc mode turned on by default anyways?

It was deemed a very useful feature.

bug#23609: 25.0.92; Python eldoc freeze

[Jules Tamagnan]

Eli Zaretskii <eliz@gnu.org> writes:

> Thanks, this is now clear.  A coupe of comments:
>
>   . Should 1 be a defcustom?  IOW, could it be that different users
>     have different annoyance thresholds, and are willing to wait
>     longer before turning eldoc-mode off?

Well eldoc mode remains on it simply won't do the automatic check
anymore when hovering over a function, you can still request the
documenation though the keybinding.

And yes the threshhold can be different for different people if that is
an option that you think would be good to add its pretty simple to add.

>   . The problem happens only when the interpreter is busy doing
>     something, is that right?  If so, perhaps we shouldn't turn
>     eldoc-mode off, but instead simply return?

The problem with simply returning instead of canceling the behavior for
the buffer is that a second (or so) later eldoc will once again try to
fetch the documentation and will be frozen until the timeout. This means
that you will be repeatidly be getting frozen as you type. It won't be
a complete freeze that crashes emacs but it won't be pleasant.

>   . If we do turn eldoc-mode off, then I think a message to that
>     effect is in order, to let the user know.

We're not turning it off but yes a warning would definitely be nice.

Thanks for hearing me out

bug#23609: 25.0.92; Python eldoc freeze

[Eli Zaretskii]

> From: Jules Tamagnan <jtamagnan@gmail.com>
> Cc: 23609@debbugs.gnu.org,  dgutov@yandex.ru
> Date: Fri, 27 May 2016 15:13:42 -0400
> 
> >   . The problem happens only when the interpreter is busy doing
> >     something, is that right?  If so, perhaps we shouldn't turn
> >     eldoc-mode off, but instead simply return?
> 
> The problem with simply returning instead of canceling the behavior for
> the buffer is that a second (or so) later eldoc will once again try to
> fetch the documentation and will be frozen until the timeout. This means
> that you will be repeatidly be getting frozen as you type. It won't be
> a complete freeze that crashes emacs but it won't be pleasant.

Once again, perhaps I misunderstand the nature of the problem, but
doesn't it happen only when the Python interpreter is busy doing
something when the eldoc function is invoked?  If so, then the next
time eldoc is invoked, the Python interpreter might not be busy, and
the feature will work without hanging, right?  Or am I missing
something?

> 
> >   . If we do turn eldoc-mode off, then I think a message to that
> >     effect is in order, to let the user know.
> 
> We're not turning it off but yes a warning would definitely be nice.

Well, we turn off the automatic display of the documentation in the
echo area, right?

Thanks.

bug#23609: 25.0.92; Python eldoc freeze

[Jules Tamagnan]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Jules Tamagnan <jtamagnan@gmail.com>
>> Cc: 23609@debbugs.gnu.org,  dgutov@yandex.ru
>> Date: Fri, 27 May 2016 15:13:42 -0400
>>
>> >   . The problem happens only when the interpreter is busy doing
>> >     something, is that right?  If so, perhaps we shouldn't turn
>> >     eldoc-mode off, but instead simply return?
>>
>> The problem with simply returning instead of canceling the behavior for
>> the buffer is that a second (or so) later eldoc will once again try to
>> fetch the documentation and will be frozen until the timeout. This means
>> that you will be repeatidly be getting frozen as you type. It won't be
>> a complete freeze that crashes emacs but it won't be pleasant.
>
> Once again, perhaps I misunderstand the nature of the problem, but
> doesn't it happen only when the Python interpreter is busy doing
> something when the eldoc function is invoked?  If so, then the next
> time eldoc is invoked, the Python interpreter might not be busy, and
> the feature will work without hanging, right?  Or am I missing
> something?

No that's exactly correct. I'm just thinking of it from the point of
view that perhaps somebody is plotting a graph from the interpreter
which causes the interpreter to be busy. Then while having that graph
open they would still like to be able to edit their code (I have been in
this situation dozens of times). Or if the code they are running takes a
a minute or two there is a good chance that they would want to edit some
of the code while it is running. In those scenarios I feel like it makes
sense to stop eldoc from doing the automatic checking permanently for
that buffer because otherwise my work would be constantly interrupted.

If we plan on the process only being busy for a few seconds at a time then I
understand where you are coming from.

I have also found for python at least that there are very good tools out
there that work better than eldoc and which use a separate process so
they never hang up.

>> >   . If we do turn eldoc-mode off, then I think a message to that
>> >     effect is in order, to let the user know.
>>
>> We're not turning it off but yes a warning would definitely be nice.
>
> Well, we turn off the automatic display of the documentation in the
> echo area, right?

Yep but eldoc mode is still on

Thanks.

bug#23609: 25.0.92; Python eldoc freeze

[Eli Zaretskii]

> From: Jules Tamagnan <jtamagnan@gmail.com>
> Cc: 23609@debbugs.gnu.org,  dgutov@yandex.ru
> Date: Fri, 27 May 2016 15:43:18 -0400
> 
> > Once again, perhaps I misunderstand the nature of the problem, but
> > doesn't it happen only when the Python interpreter is busy doing
> > something when the eldoc function is invoked?  If so, then the next
> > time eldoc is invoked, the Python interpreter might not be busy, and
> > the feature will work without hanging, right?  Or am I missing
> > something?
> 
> No that's exactly correct. I'm just thinking of it from the point of
> view that perhaps somebody is plotting a graph from the interpreter
> which causes the interpreter to be busy. Then while having that graph
> open they would still like to be able to edit their code (I have been in
> this situation dozens of times). Or if the code they are running takes a
> a minute or two there is a good chance that they would want to edit some
> of the code while it is running. In those scenarios I feel like it makes
> sense to stop eldoc from doing the automatic checking permanently for
> that buffer because otherwise my work would be constantly interrupted.

OK, so let's go with your suggestion, but make that message we agreed
should be added say something like

  ElDoc mode echo-area display turned off; type .... to turn on

where "..." stands for whatever needs to be done to turn it back on
again.

Also, I do think a defcustom is due instead of a hardwired constant of
1 sec.

Could you please augment your patch along these lines?

Thanks.

bug#23609: 25.0.92; Python eldoc freeze

[Jules Tamagnan]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Jules Tamagnan <jtamagnan@gmail.com>
>> Cc: 23609@debbugs.gnu.org,  dgutov@yandex.ru
>> Date: Fri, 27 May 2016 15:43:18 -0400
>>
>> > Once again, perhaps I misunderstand the nature of the problem, but
>> > doesn't it happen only when the Python interpreter is busy doing
>> > something when the eldoc function is invoked?  If so, then the next
>> > time eldoc is invoked, the Python interpreter might not be busy, and
>> > the feature will work without hanging, right?  Or am I missing
>> > something?
>>
>> No that's exactly correct. I'm just thinking of it from the point of
>> view that perhaps somebody is plotting a graph from the interpreter
>> which causes the interpreter to be busy. Then while having that graph
>> open they would still like to be able to edit their code (I have been in
>> this situation dozens of times). Or if the code they are running takes a
>> a minute or two there is a good chance that they would want to edit some
>> of the code while it is running. In those scenarios I feel like it makes
>> sense to stop eldoc from doing the automatic checking permanently for
>> that buffer because otherwise my work would be constantly interrupted.
>
> OK, so let's go with your suggestion, but make that message we agreed
> should be added say something like
>
>   ElDoc mode echo-area display turned off; type .... to turn on
>
> where "..." stands for whatever needs to be done to turn it back on
> again.
>
> Also, I do think a defcustom is due instead of a hardwired constant of
> 1 sec.
>
> Could you please augment your patch along these lines?
>
> Thanks.

diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
index 5aeefe4..46c3bd9 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -4293,12 +4293,29 @@ python-eldoc--get-doc-at-point
         (unless (zerop (length docstring))
           docstring)))))

+(defvar-local python-eldoc--get-doc t
+  "If eldoc should be used to get the documentation automatically")
+
+(defcustom python-eldoc-function-timeout 1
+  "Timeout for python-eldoc-function in seconds."
+  :group 'python
+  :type 'integer)
+
 (defun python-eldoc-function ()
   "`eldoc-documentation-function' for Python.
 For this to work as best as possible you should call
 `python-shell-send-buffer' from time to time so context in
-inferior Python process is updated properly."
-  (python-eldoc--get-doc-at-point))
+inferior Python process is updated properly.
+
+If `python-eldoc-function-timeout' seconds elapse before this
+function returns then `python-eldoc--get-doc' will be set to nil
+and eldoc will no longer return the documentation at the point
+automatically."
+  (when python-eldoc--get-doc
+    (with-timeout (python-eldoc-function-timeout
+                   (message "Eldoc mode echo-area display turned off in current buffer; eval (setq python-eldoc--get-doc t) to turn on")
+                   (setq python-eldoc--get-doc nil))
+      (python-eldoc--get-doc-at-point))))

 (defun python-eldoc-at-point (symbol)
   "Get help on SYMBOL using `help'.

Here was my rendition of the task at hand.

We could also perhaps make it such that if the timeout variable were
negative then instead of permanently stopping eldoc it would only
return, like you had suggested. A boolean defcustom could also be
used for that if we want.

Also 'python-eldoc--get-doc' could be renamed to not be private if we
expect somebody to want to change it

Thanks

bug#23609: 25.0.92; Python eldoc freeze

[Eli Zaretskii]

> From: Jules Tamagnan <jtamagnan@gmail.com>
> Cc: 23609@debbugs.gnu.org,  dgutov@yandex.ru
> Date: Fri, 27 May 2016 16:19:57 -0400
> 
> diff --git a/lisp/progmodes/python.el b/lisp/progmodes/python.el
> index 5aeefe4..46c3bd9 100644
> --- a/lisp/progmodes/python.el
> +++ b/lisp/progmodes/python.el
> @@ -4293,12 +4293,29 @@ python-eldoc--get-doc-at-point
>          (unless (zerop (length docstring))
>            docstring)))))
> 
> +(defvar-local python-eldoc--get-doc t
> +  "If eldoc should be used to get the documentation automatically")
> +
> +(defcustom python-eldoc-function-timeout 1
> +  "Timeout for python-eldoc-function in seconds."
> +  :group 'python
> +  :type 'integer)
> +
>  (defun python-eldoc-function ()
>    "`eldoc-documentation-function' for Python.
>  For this to work as best as possible you should call
>  `python-shell-send-buffer' from time to time so context in
> -inferior Python process is updated properly."
> -  (python-eldoc--get-doc-at-point))
> +inferior Python process is updated properly.
> +
> +If `python-eldoc-function-timeout' seconds elapse before this
> +function returns then `python-eldoc--get-doc' will be set to nil
> +and eldoc will no longer return the documentation at the point
> +automatically."
> +  (when python-eldoc--get-doc
> +    (with-timeout (python-eldoc-function-timeout
> +                   (message "Eldoc mode echo-area display turned off in current buffer; eval (setq python-eldoc--get-doc t) to turn on")
> +                   (setq python-eldoc--get-doc nil))
> +      (python-eldoc--get-doc-at-point))))
> 
>  (defun python-eldoc-at-point (symbol)
>    "Get help on SYMBOL using `help'.
> 
> Here was my rendition of the task at hand.

Thanks.  We should try making the message text shorter, so it fits on
a single line.

> We could also perhaps make it such that if the timeout variable were
> negative then instead of permanently stopping eldoc it would only
> return, like you had suggested. A boolean defcustom could also be
> used for that if we want.

I don't see any significant advantage either way.

> Also 'python-eldoc--get-doc' could be renamed to not be private if we
> expect somebody to want to change it

Yes, that variable should not be private if users are expected to set
it.  Alternatively, we could add a simple command to set the variable
non-nil, which might also help making the message text shorter.

bug#23609: 25.0.92; Python eldoc freeze

[Jules Tamagnan]

Eli Zaretskii <eliz@gnu.org> writes:

> Thanks.  We should try making the message text shorter, so it fits on
> a single line.
>
>> We could also perhaps make it such that if the timeout variable were
>> negative then instead of permanently stopping eldoc it would only
>> return, like you had suggested. A boolean defcustom could also be
>> used for that if we want.
>
> I don't see any significant advantage either way.
>
>> Also 'python-eldoc--get-doc' could be renamed to not be private if we
>> expect somebody to want to change it
>
> Yes, that variable should not be private if users are expected to set
> it.  Alternatively, we could add a simple command to set the variable
> non-nil, which might also help making the message text shorter.

I've improved the documentation, added the boolean to make the timeout
permanent and shortened the messages

index 5aeefe4..d555e21 100644
--- a/lisp/progmodes/python.el
+++ b/lisp/progmodes/python.el
@@ -4293,12 +4293,45 @@ python-eldoc--get-doc-at-point
         (unless (zerop (length docstring))
           docstring)))))

+(defvar-local python-eldoc-get-doc t
+  "Non-nil means eldoc should fetch the documentation
+  automatically. Set to nil by `python-eldoc-function' if
+  `python-eldoc-function-timeout-permanent' is non-nil and
+  `python-eldoc-function' times out.")
+
+(defcustom python-eldoc-function-timeout 1
+  "Timeout for `python-eldoc-function' in seconds."
+  :group 'python
+  :type 'integer)
+
+(defcustom python-eldoc-function-timeout-permanent t
+  "Non-nil means that when `python-eldoc-function' times out
+`python-eldoc-get-doc' will be set to nil"
+  :group 'python
+  :type 'boolean)
+
 (defun python-eldoc-function ()
   "`eldoc-documentation-function' for Python.
 For this to work as best as possible you should call
 `python-shell-send-buffer' from time to time so context in
-inferior Python process is updated properly."
-  (python-eldoc--get-doc-at-point))
+inferior Python process is updated properly.
+
+If `python-eldoc-function-timeout' seconds elapse before this
+function returns then if
+`python-eldoc-function-timeout-permanent' is non-nil
+`python-eldoc-get-doc' will be set to nil and eldoc will no
+longer return the documentation at the point automatically.
+
+Set `python-eldoc-get-doc' to t to reenable eldoc documentation
+fetching"
+  (when python-eldoc-get-doc
+    (with-timeout (python-eldoc-function-timeout
+                   (if python-eldoc-function-timeout-permanent
+                       (progn
+                         (message "Eldoc echo-area display muted in current buffer, see `python-eldoc-function'")
+                         (setq python-eldoc-get-doc nil))
+                     (message "`python-eldoc-function' timed out, see `python-eldoc-function-timeout'")))
+      (python-eldoc--get-doc-at-point))))

 (defun python-eldoc-at-point (symbol)
   "Get help on SYMBOL using `help'.

bug#23609: 25.0.92; Python eldoc freeze

[Jules Tamagnan]

I'd like to bump this thread to try to finish fix this issue once and
for all. Especially since there is already a patch

bug#23609: 25.0.92; Python eldoc freeze

[Eli Zaretskii]

> From: Jules Tamagnan <jtamagnan@gmail.com>
> Cc: 23609@debbugs.gnu.org
> Date: Mon, 06 Jun 2016 10:08:51 -0400
> 
> I'd like to bump this thread to try to finish fix this issue once and
> for all. Especially since there is already a patch

It's near the top of my todo, so I haven't forgotten.  But if someone
else can push it, no need to wait for me.

Thanks.

bug#23609: 25.0.92; Python eldoc freeze

[Eli Zaretskii]

> From: Jules Tamagnan <jtamagnan@gmail.com>
> Cc: 23609@debbugs.gnu.org
> Date: Mon, 06 Jun 2016 10:08:51 -0400
> 
> I'd like to bump this thread to try to finish fix this issue once and
> for all. Especially since there is already a patch

I've committed your patch to the release branch.

Sorry for the long delay, and thank you for your efforts and patience.