bug#18052: 24.3; python.el and ipython shell

bug#18052: 24.3; python.el and ipython shell

[Federico Beffa]

I'm trying to use the ipython shell from within emacs. To do so I've
followed the instructions at the top of python.el and added the
following code to my .emacs:

(setq
 python-shell-interpreter "C:/Anaconda/python.exe"
 python-shell-interpreter-args
   "-i C:/Anaconda/Scripts/ipython-script.py console --matplotlib=qt"
 python-shell-prompt-regexp "In \\[[0-9]+\\]: "
 python-shell-prompt-output-regexp "Out\\[[0-9]+\\]: "
 python-shell-completion-setup-code
   "from IPython.core.completerlib import module_completion"
 python-shell-completion-module-string-code
   "';'.join(module_completion('''%s'''))\n"
 python-shell-completion-string-code
   "';'.join(get_ipython().Completer.all_completions('''%s'''))\n")

When I execute (M-x) run-python, SOMETIMES the session does not
initialize properly and the new buffer looks like this:

-----------------------------------------------------------
Python 2.7.6 |Anaconda 2.0.1 (64-bit)| (default, Nov 11 2013,
10:49:15) [MSC v.1500 64 bit (AMD64)]
Type "copyright", "credits" or "license" for more information.

IPython 2.1.0 -- An enhanced Interactive Python.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]:
-----------------------------------------------------------

The difference compared to the situation when everything is well, is
that here I only see "In [1]:", while normally I should see 4 empty
input prompts, with the last one reading "In [4]". When I execute any
command such as "1+1" and press RET, nothing happens.

This happens starting emacs with the -Q option and executing the ipython
customization in the *scratch* buffer.

By running the debugger I've found out that emacs sends some commands to
the inferior python shell with (comint-send-string ...) and tries to
hide them from the user. That appears to be the reason for the four
empty prompts.

Looking at the running processes I've noted that there are 2 python
processes running. The task manager reports one as:
c:\Anaconda\python.exe -c"from IPython.kernel.zmq.kernelapp import main;
main()" -f
C:\Users\beffa\.ipython\profile_default\security\kernel-976.json
--matplotlib=qt --IPKernelApp.parent_appname='ipython-console'
--interrupt=692 --parent=696

(Obviously the value of the last two arguments change everytime.)
If I kill this process, the inferior python shell asks:

"kernel died, restart ([y]/n)? kernel died, restart ([y]/n)? kernel
died, restart ([y]/n)?"

If I press y the process is restarted and the inferior python shell
works fine (or at least responds to the commands like 1+1 that I've
tried).

I would greatly appreciate any suggestion on how to further dig into
debugging and solve this intermittent problem.

Thank you in advance for your help.
Fede


In GNU Emacs 24.3.1 (i386-mingw-nt6.1.7601)
 of 2013-03-17 on MARVIN
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
 `configure --with-gcc (4.7) --cflags
 -ID:/devel/emacs/libs/libXpm-3.5.8/include
 -ID:/devel/emacs/libs/libXpm-3.5.8/src
 -ID:/devel/emacs/libs/libpng-dev_1.4.3-1/include
 -ID:/devel/emacs/libs/zlib-dev_1.2.5-2/include
 -ID:/devel/emacs/libs/giflib-4.1.4-1/include
 -ID:/devel/emacs/libs/jpeg-6b-4/include
 -ID:/devel/emacs/libs/tiff-3.8.2-1/include
 -ID:/devel/emacs/libs/gnutls-3.0.9/include
 -ID:/devel/emacs/libs/libiconv-1.13.1-1-dev/include
 -ID:/devel/emacs/libs/libxml2-2.7.8/include/libxml2'

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1252
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  show-paren-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<help-echo> C-x C-r C-SPC C-a C-w c : / p r o <tab>
SPC <tab> SPC <tab> e m <tab> 3 <tab> s i <tab> s i
<tab> d <tab> f <tab> <return> <next> <next> <down>
<down> <down> <down> <down> <down> <down> <down> <down>
<down> <down> <down> <down> <down> <down> <down> <down>
<down> <down> <down> <down> <down> <down> <down> <down>
<down> <down> <down> <down> <down> <down> <down> <down>
<down> <down> <down> <down> <down> <down> <down> C-SPC
<down> <down> <down> <down> <down> <down> <down> <down>
<down> <down> <down> <down> <left> M-w C-x b <return>
C-y M-x s h o w - p a <tab> <return> C-x C-e M-x r
u n - p y <tab> <return> C-x k <return> y e s <return>
C-x 0 M-x r u n - p <tab> y <tab> <return> <help-echo>
<help-echo> C-x o M-x r e p o <tab> r <tab> <retur
n>

Recent messages:
"';'.join(get_ipython().Completer.all_completions('''%s'''))
"
Sent python-shell-completion-setup-code
Sent python-ffap-setup-code
Sent python-eldoc-setup-code
Making completion list...
Sent python-shell-completion-setup-code
Sent python-ffap-setup-code
Sent python-eldoc-setup-code
Making completion list...

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils help-mode compile python rx easymenu comint ring
ansi-color cus-start cus-load paren time-date tooltip ediff-hook
vc-hooks lisp-float-type mwheel dos-w32 ls-lisp w32-common-fns
disp-table w32-win w32-vars tool-bar dnd fontset image regexp-opt fringe
tabulated-list newcomment lisp-mode register page menu-bar rfn-eshadow
timer select scroll-bar mouse jit-lock font-lock syntax facemenu
font-core frame cham georgian utf-8-lang misc-lang vietnamese tibetan
thai tai-viet lao korean japanese hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
make-network-process w32 multi-tty emacs)

bug#18052: 24.3; python.el and ipython shell

[Eli Zaretskii]

> Date: Fri, 18 Jul 2014 15:51:47 +0200
> From: Federico Beffa <beffa@ieee.org>
> 
> I'm trying to use the ipython shell from within emacs. To do so I've
> followed the instructions at the top of python.el and added the
> following code to my .emacs:
> 
> (setq
>  python-shell-interpreter "C:/Anaconda/python.exe"
>  python-shell-interpreter-args
>    "-i C:/Anaconda/Scripts/ipython-script.py console --matplotlib=qt"
>  python-shell-prompt-regexp "In \\[[0-9]+\\]: "
>  python-shell-prompt-output-regexp "Out\\[[0-9]+\\]: "
>  python-shell-completion-setup-code
>    "from IPython.core.completerlib import module_completion"
>  python-shell-completion-module-string-code
>    "';'.join(module_completion('''%s'''))\n"
>  python-shell-completion-string-code
>    "';'.join(get_ipython().Completer.all_completions('''%s'''))\n")
> 
> When I execute (M-x) run-python, SOMETIMES the session does not
> initialize properly and the new buffer looks like this:
> 
> -----------------------------------------------------------
> Python 2.7.6 |Anaconda 2.0.1 (64-bit)| (default, Nov 11 2013,
> 10:49:15) [MSC v.1500 64 bit (AMD64)]
> Type "copyright", "credits" or "license" for more information.
> 
> IPython 2.1.0 -- An enhanced Interactive Python.
> Anaconda is brought to you by Continuum Analytics.
> Please check out: http://continuum.io/thanks and https://binstar.org
> ?         -> Introduction and overview of IPython's features.
> %quickref -> Quick reference.
> help      -> Python's own help system.
> object?   -> Details about 'object', use 'object??' for extra details.
> 
> In [1]:
> -----------------------------------------------------------
> 
> The difference compared to the situation when everything is well, is
> that here I only see "In [1]:", while normally I should see 4 empty
> input prompts, with the last one reading "In [4]". When I execute any
> command such as "1+1" and press RET, nothing happens.
> 
> This happens starting emacs with the -Q option and executing the ipython
> customization in the *scratch* buffer.

Did you try to explore the possibility that this is caused by
buffering?  Pipes on Windows are fully buffered, not line-buffered as
on Posix systems.  If there's some way to control buffering from the
Python side, I suggest to try that, perhaps it will fix the problem.

bug#18052: 24.3; python.el and ipython shell

[Federico Beffa]

I've looked in the ipython configuration file, but have not seen anything.

Do you have any suggestion on where to look further?

On Fri, Jul 18, 2014 at 4:07 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>> Date: Fri, 18 Jul 2014 15:51:47 +0200
>> From: Federico Beffa <beffa@ieee.org>
>>
>> I'm trying to use the ipython shell from within emacs. To do so I've
>> followed the instructions at the top of python.el and added the
>> following code to my .emacs:
>>
>> (setq
>>  python-shell-interpreter "C:/Anaconda/python.exe"
>>  python-shell-interpreter-args
>>    "-i C:/Anaconda/Scripts/ipython-script.py console --matplotlib=qt"
>>  python-shell-prompt-regexp "In \\[[0-9]+\\]: "
>>  python-shell-prompt-output-regexp "Out\\[[0-9]+\\]: "
>>  python-shell-completion-setup-code
>>    "from IPython.core.completerlib import module_completion"
>>  python-shell-completion-module-string-code
>>    "';'.join(module_completion('''%s'''))\n"
>>  python-shell-completion-string-code
>>    "';'.join(get_ipython().Completer.all_completions('''%s'''))\n")
>>
>> When I execute (M-x) run-python, SOMETIMES the session does not
>> initialize properly and the new buffer looks like this:
>>
>> -----------------------------------------------------------
>> Python 2.7.6 |Anaconda 2.0.1 (64-bit)| (default, Nov 11 2013,
>> 10:49:15) [MSC v.1500 64 bit (AMD64)]
>> Type "copyright", "credits" or "license" for more information.
>>
>> IPython 2.1.0 -- An enhanced Interactive Python.
>> Anaconda is brought to you by Continuum Analytics.
>> Please check out: http://continuum.io/thanks and https://binstar.org
>> ?         -> Introduction and overview of IPython's features.
>> %quickref -> Quick reference.
>> help      -> Python's own help system.
>> object?   -> Details about 'object', use 'object??' for extra details.
>>
>> In [1]:
>> -----------------------------------------------------------
>>
>> The difference compared to the situation when everything is well, is
>> that here I only see "In [1]:", while normally I should see 4 empty
>> input prompts, with the last one reading "In [4]". When I execute any
>> command such as "1+1" and press RET, nothing happens.
>>
>> This happens starting emacs with the -Q option and executing the ipython
>> customization in the *scratch* buffer.
>
> Did you try to explore the possibility that this is caused by
> buffering?  Pipes on Windows are fully buffered, not line-buffered as
> on Posix systems.  If there's some way to control buffering from the
> Python side, I suggest to try that, perhaps it will fix the problem.

bug#18052: 24.3; python.el and ipython shell

[Eli Zaretskii]

> Date: Fri, 18 Jul 2014 18:38:11 +0200
> From: Federico Beffa <beffa@ieee.org>
> Cc: 18052@debbugs.gnu.org
> 
> I've looked in the ipython configuration file, but have not seen anything.
> 
> Do you have any suggestion on where to look further?

Sorry, I don't know enough Python to help you there.

bug#18052: 24.3; python.el and ipython shell

[Fabián Ezequiel Gallina]

Does adding the -u switch for python help?

(setq  python-shell-interpreter-args
       "-u -i C:/Anaconda/Scripts/ipython-script.py console --matplotlib=qt")



Regards,
Fabián

bug#18052: 24.3; python.el and ipython shell

[Federico Beffa]

Unfortunately it does not. I see the same intermittent problem as
without the "-u" option.

bug#18052: 24.3; python.el and ipython shell

[Fabián Ezequiel Gallina]

Federico Beffa <beffa@ieee.org> writes:

> Unfortunately it does not. I see the same intermittent problem as
> without the "-u" option.
>

So I tracked down what's going on.  Removing "console" from your
`python-shell-interpreter-args' should do the trick.

The "console" subcommand, starts an iPython server (AKA kernel) and then
connects the client console to it (that's the reason you see two
processes).  Removing the "console" subcommand just starts a standalone
iPython shell that doesn't suffer from this intermittent initialization
problem.

Now the reason why the console subcommand fails seems to be an iPython
bug itself.  What happens is that python.el shell setup codes are sent
quickly as soon as the inferior process starts, but it seems that
sometimes the iPython kernel startup doesn't expect that and fails to
start properly, thus causing the resulting iPython shell to be non
responsive.

This is not particularly a problem of python.el, in fact the issue can
be replicated in the commandline by spawning processes like so:

    echo "print ('yes')" | ipython console

With that, I get the same intermittent behavior observed previously.  So
my suggestion is to report this bug upstream.

In the meantime, I'll include some notes about this in the header of
python.el.


PS: As you may expect, when an iPython kernel is already started, using
"console --existing" works reliably, this may be useful in case you
really needed the client-server mode.



Cheers,
Fabián

bug#18052: closed (Re: bug#18052: 24.3; python.el and ipython shell)

[Federico Beffa]

I can confirm that removing "console" resolves the problem.

Thank you very much for the help!
Best Regards,
Fede

On Sun, Jul 20, 2014 at 8:03 AM, GNU bug Tracking System
<help-debbugs@gnu.org> wrote:
> Your bug report
>
> #18052: 24.3; python.el and ipython shell
>
> which was filed against the emacs package, has been closed.
>
> The explanation is attached below, along with your original report.
> If you require more details, please reply to 18052@debbugs.gnu.org.
>
> --
> 18052: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=18052
> GNU Bug Tracking System
> Contact help-debbugs@gnu.org with problems
>
>
> ---------- Forwarded message ----------
> From: fgallina@gnu.org (Fabián Ezequiel Gallina)
> To: Federico Beffa <beffa@ieee.org>
> Cc: 18052-done@debbugs.gnu.org
> Date: Sun, 20 Jul 2014 03:02:06 -0300
> Subject: Re: bug#18052: 24.3; python.el and ipython shell
> Federico Beffa <beffa@ieee.org> writes:
>
>> Unfortunately it does not. I see the same intermittent problem as
>> without the "-u" option.
>>
>
> So I tracked down what's going on.  Removing "console" from your
> `python-shell-interpreter-args' should do the trick.
>
> The "console" subcommand, starts an iPython server (AKA kernel) and then
> connects the client console to it (that's the reason you see two
> processes).  Removing the "console" subcommand just starts a standalone
> iPython shell that doesn't suffer from this intermittent initialization
> problem.
>
> Now the reason why the console subcommand fails seems to be an iPython
> bug itself.  What happens is that python.el shell setup codes are sent
> quickly as soon as the inferior process starts, but it seems that
> sometimes the iPython kernel startup doesn't expect that and fails to
> start properly, thus causing the resulting iPython shell to be non
> responsive.
>
> This is not particularly a problem of python.el, in fact the issue can
> be replicated in the commandline by spawning processes like so:
>
>     echo "print ('yes')" | ipython console
>
> With that, I get the same intermittent behavior observed previously.  So
> my suggestion is to report this bug upstream.
>
> In the meantime, I'll include some notes about this in the header of
> python.el.
>
>
> PS: As you may expect, when an iPython kernel is already started, using
> "console --existing" works reliably, this may be useful in case you
> really needed the client-server mode.
>
>
>
> Cheers,
> Fabián
>
>
>
> ---------- Forwarded message ----------
> From: Federico Beffa <beffa@ieee.org>
> To: bug-gnu-emacs@gnu.org
> Cc:
> Date: Fri, 18 Jul 2014 15:51:47 +0200
> Subject: 24.3; python.el and ipython shell
> I'm trying to use the ipython shell from within emacs. To do so I've
> followed the instructions at the top of python.el and added the
> following code to my .emacs:
>
> (setq
>  python-shell-interpreter "C:/Anaconda/python.exe"
>  python-shell-interpreter-args
>    "-i C:/Anaconda/Scripts/ipython-script.py console --matplotlib=qt"
>  python-shell-prompt-regexp "In \\[[0-9]+\\]: "
>  python-shell-prompt-output-regexp "Out\\[[0-9]+\\]: "
>  python-shell-completion-setup-code
>    "from IPython.core.completerlib import module_completion"
>  python-shell-completion-module-string-code
>    "';'.join(module_completion('''%s'''))\n"
>  python-shell-completion-string-code
>    "';'.join(get_ipython().Completer.all_completions('''%s'''))\n")
>
> When I execute (M-x) run-python, SOMETIMES the session does not
> initialize properly and the new buffer looks like this:
>
> -----------------------------------------------------------
> Python 2.7.6 |Anaconda 2.0.1 (64-bit)| (default, Nov 11 2013,
> 10:49:15) [MSC v.1500 64 bit (AMD64)]
> Type "copyright", "credits" or "license" for more information.
>
> IPython 2.1.0 -- An enhanced Interactive Python.
> Anaconda is brought to you by Continuum Analytics.
> Please check out: http://continuum.io/thanks and https://binstar.org
> ?         -> Introduction and overview of IPython's features.
> %quickref -> Quick reference.
> help      -> Python's own help system.
> object?   -> Details about 'object', use 'object??' for extra details.
>
> In [1]:
> -----------------------------------------------------------
>
> The difference compared to the situation when everything is well, is
> that here I only see "In [1]:", while normally I should see 4 empty
> input prompts, with the last one reading "In [4]". When I execute any
> command such as "1+1" and press RET, nothing happens.
>
> This happens starting emacs with the -Q option and executing the ipython
> customization in the *scratch* buffer.
>
> By running the debugger I've found out that emacs sends some commands to
> the inferior python shell with (comint-send-string ...) and tries to
> hide them from the user. That appears to be the reason for the four
> empty prompts.
>
> Looking at the running processes I've noted that there are 2 python
> processes running. The task manager reports one as:
> c:\Anaconda\python.exe -c"from IPython.kernel.zmq.kernelapp import main;
> main()" -f
> C:\Users\beffa\.ipython\profile_default\security\kernel-976.json
> --matplotlib=qt --IPKernelApp.parent_appname='ipython-console'
> --interrupt=692 --parent=696
>
> (Obviously the value of the last two arguments change everytime.)
> If I kill this process, the inferior python shell asks:
>
> "kernel died, restart ([y]/n)? kernel died, restart ([y]/n)? kernel
> died, restart ([y]/n)?"
>
> If I press y the process is restarted and the inferior python shell
> works fine (or at least responds to the commands like 1+1 that I've
> tried).
>
> I would greatly appreciate any suggestion on how to further dig into
> debugging and solve this intermittent problem.
>
> Thank you in advance for your help.
> Fede
>
>
> In GNU Emacs 24.3.1 (i386-mingw-nt6.1.7601)
>  of 2013-03-17 on MARVIN
> Windowing system distributor `Microsoft Corp.', version 6.1.7601
> Configured using:
>  `configure --with-gcc (4.7) --cflags
>  -ID:/devel/emacs/libs/libXpm-3.5.8/include
>  -ID:/devel/emacs/libs/libXpm-3.5.8/src
>  -ID:/devel/emacs/libs/libpng-dev_1.4.3-1/include
>  -ID:/devel/emacs/libs/zlib-dev_1.2.5-2/include
>  -ID:/devel/emacs/libs/giflib-4.1.4-1/include
>  -ID:/devel/emacs/libs/jpeg-6b-4/include
>  -ID:/devel/emacs/libs/tiff-3.8.2-1/include
>  -ID:/devel/emacs/libs/gnutls-3.0.9/include
>  -ID:/devel/emacs/libs/libiconv-1.13.1-1-dev/include
>  -ID:/devel/emacs/libs/libxml2-2.7.8/include/libxml2'
>
> Important settings:
>   value of $LANG: ENU
>   locale-coding-system: cp1252
>   default enable-multibyte-characters: t
>
> Major mode: Lisp Interaction
>
> Minor modes in effect:
>   show-paren-mode: t
>   tooltip-mode: t
>   mouse-wheel-mode: t
>   tool-bar-mode: t
>   menu-bar-mode: t
>   file-name-shadow-mode: t
>   global-font-lock-mode: t
>   font-lock-mode: t
>   blink-cursor-mode: t
>   auto-composition-mode: t
>   auto-encryption-mode: t
>   auto-compression-mode: t
>   line-number-mode: t
>   transient-mark-mode: t
>
> Recent input:
> <help-echo> C-x C-r C-SPC C-a C-w c : / p r o <tab>
> SPC <tab> SPC <tab> e m <tab> 3 <tab> s i <tab> s i
> <tab> d <tab> f <tab> <return> <next> <next> <down>
> <down> <down> <down> <down> <down> <down> <down> <down>
> <down> <down> <down> <down> <down> <down> <down> <down>
> <down> <down> <down> <down> <down> <down> <down> <down>
> <down> <down> <down> <down> <down> <down> <down> <down>
> <down> <down> <down> <down> <down> <down> <down> C-SPC
> <down> <down> <down> <down> <down> <down> <down> <down>
> <down> <down> <down> <down> <left> M-w C-x b <return>
> C-y M-x s h o w - p a <tab> <return> C-x C-e M-x r
> u n - p y <tab> <return> C-x k <return> y e s <return>
> C-x 0 M-x r u n - p <tab> y <tab> <return> <help-echo>
> <help-echo> C-x o M-x r e p o <tab> r <tab> <retur
> n>
>
> Recent messages:
> "';'.join(get_ipython().Completer.all_completions('''%s'''))
> "
> Sent python-shell-completion-setup-code
> Sent python-ffap-setup-code
> Sent python-eldoc-setup-code
> Making completion list...
> Sent python-shell-completion-setup-code
> Sent python-ffap-setup-code
> Sent python-eldoc-setup-code
> Making completion list...
>
> Load-path shadows:
> None found.
>
> Features:
> (shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
> mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
> gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
> mail-prsvr mail-utils help-mode compile python rx easymenu comint ring
> ansi-color cus-start cus-load paren time-date tooltip ediff-hook
> vc-hooks lisp-float-type mwheel dos-w32 ls-lisp w32-common-fns
> disp-table w32-win w32-vars tool-bar dnd fontset image regexp-opt fringe
> tabulated-list newcomment lisp-mode register page menu-bar rfn-eshadow
> timer select scroll-bar mouse jit-lock font-lock syntax facemenu
> font-core frame cham georgian utf-8-lang misc-lang vietnamese tibetan
> thai tai-viet lao korean japanese hebrew greek romanian slovak czech
> european ethiopic indian cyrillic chinese case-table epa-hook
> jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
> cus-face macroexp files text-properties overlay sha1 md5 base64 format
> env code-pages mule custom widget hashtable-print-readable backquote
> make-network-process w32 multi-tty emacs)
>
>
>

bug#18052: closed (Re: bug#18052: 24.3; python.el and ipython shell)

[Federico Beffa]

Hi,

I would like to add that without the "console" argument, the
interactive matplotlib plotting capabilities do not work.

Regards,
Fede

On Sun, Jul 20, 2014 at 3:49 PM, Federico Beffa <beffa@ieee.org> wrote:
> I can confirm that removing "console" resolves the problem.
>
> Thank you very much for the help!
> Best Regards,
> Fede
>
> On Sun, Jul 20, 2014 at 8:03 AM, GNU bug Tracking System
> <help-debbugs@gnu.org> wrote:
>> Your bug report
>>
>> #18052: 24.3; python.el and ipython shell
>>
>> which was filed against the emacs package, has been closed.
>>
>> The explanation is attached below, along with your original report.
>> If you require more details, please reply to 18052@debbugs.gnu.org.
>>
>> --
>> 18052: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=18052
>> GNU Bug Tracking System
>> Contact help-debbugs@gnu.org with problems
>>
>>
>> ---------- Forwarded message ----------
>> From: fgallina@gnu.org (Fabián Ezequiel Gallina)
>> To: Federico Beffa <beffa@ieee.org>
>> Cc: 18052-done@debbugs.gnu.org
>> Date: Sun, 20 Jul 2014 03:02:06 -0300
>> Subject: Re: bug#18052: 24.3; python.el and ipython shell
>> Federico Beffa <beffa@ieee.org> writes:
>>
>>> Unfortunately it does not. I see the same intermittent problem as
>>> without the "-u" option.
>>>
>>
>> So I tracked down what's going on.  Removing "console" from your
>> `python-shell-interpreter-args' should do the trick.
>>
>> The "console" subcommand, starts an iPython server (AKA kernel) and then
>> connects the client console to it (that's the reason you see two
>> processes).  Removing the "console" subcommand just starts a standalone
>> iPython shell that doesn't suffer from this intermittent initialization
>> problem.
>>
>> Now the reason why the console subcommand fails seems to be an iPython
>> bug itself.  What happens is that python.el shell setup codes are sent
>> quickly as soon as the inferior process starts, but it seems that
>> sometimes the iPython kernel startup doesn't expect that and fails to
>> start properly, thus causing the resulting iPython shell to be non
>> responsive.
>>
>> This is not particularly a problem of python.el, in fact the issue can
>> be replicated in the commandline by spawning processes like so:
>>
>>     echo "print ('yes')" | ipython console
>>
>> With that, I get the same intermittent behavior observed previously.  So
>> my suggestion is to report this bug upstream.
>>
>> In the meantime, I'll include some notes about this in the header of
>> python.el.
>>
>>
>> PS: As you may expect, when an iPython kernel is already started, using
>> "console --existing" works reliably, this may be useful in case you
>> really needed the client-server mode.
>>
>>
>>
>> Cheers,
>> Fabián
>>
>>
>>
>> ---------- Forwarded message ----------
>> From: Federico Beffa <beffa@ieee.org>
>> To: bug-gnu-emacs@gnu.org
>> Cc:
>> Date: Fri, 18 Jul 2014 15:51:47 +0200
>> Subject: 24.3; python.el and ipython shell
>> I'm trying to use the ipython shell from within emacs. To do so I've
>> followed the instructions at the top of python.el and added the
>> following code to my .emacs:
>>
>> (setq
>>  python-shell-interpreter "C:/Anaconda/python.exe"
>>  python-shell-interpreter-args
>>    "-i C:/Anaconda/Scripts/ipython-script.py console --matplotlib=qt"
>>  python-shell-prompt-regexp "In \\[[0-9]+\\]: "
>>  python-shell-prompt-output-regexp "Out\\[[0-9]+\\]: "
>>  python-shell-completion-setup-code
>>    "from IPython.core.completerlib import module_completion"
>>  python-shell-completion-module-string-code
>>    "';'.join(module_completion('''%s'''))\n"
>>  python-shell-completion-string-code
>>    "';'.join(get_ipython().Completer.all_completions('''%s'''))\n")
>>
>> When I execute (M-x) run-python, SOMETIMES the session does not
>> initialize properly and the new buffer looks like this:
>>
>> -----------------------------------------------------------
>> Python 2.7.6 |Anaconda 2.0.1 (64-bit)| (default, Nov 11 2013,
>> 10:49:15) [MSC v.1500 64 bit (AMD64)]
>> Type "copyright", "credits" or "license" for more information.
>>
>> IPython 2.1.0 -- An enhanced Interactive Python.
>> Anaconda is brought to you by Continuum Analytics.
>> Please check out: http://continuum.io/thanks and https://binstar.org
>> ?         -> Introduction and overview of IPython's features.
>> %quickref -> Quick reference.
>> help      -> Python's own help system.
>> object?   -> Details about 'object', use 'object??' for extra details.
>>
>> In [1]:
>> -----------------------------------------------------------
>>
>> The difference compared to the situation when everything is well, is
>> that here I only see "In [1]:", while normally I should see 4 empty
>> input prompts, with the last one reading "In [4]". When I execute any
>> command such as "1+1" and press RET, nothing happens.
>>
>> This happens starting emacs with the -Q option and executing the ipython
>> customization in the *scratch* buffer.
>>
>> By running the debugger I've found out that emacs sends some commands to
>> the inferior python shell with (comint-send-string ...) and tries to
>> hide them from the user. That appears to be the reason for the four
>> empty prompts.
>>
>> Looking at the running processes I've noted that there are 2 python
>> processes running. The task manager reports one as:
>> c:\Anaconda\python.exe -c"from IPython.kernel.zmq.kernelapp import main;
>> main()" -f
>> C:\Users\beffa\.ipython\profile_default\security\kernel-976.json
>> --matplotlib=qt --IPKernelApp.parent_appname='ipython-console'
>> --interrupt=692 --parent=696
>>
>> (Obviously the value of the last two arguments change everytime.)
>> If I kill this process, the inferior python shell asks:
>>
>> "kernel died, restart ([y]/n)? kernel died, restart ([y]/n)? kernel
>> died, restart ([y]/n)?"
>>
>> If I press y the process is restarted and the inferior python shell
>> works fine (or at least responds to the commands like 1+1 that I've
>> tried).
>>
>> I would greatly appreciate any suggestion on how to further dig into
>> debugging and solve this intermittent problem.
>>
>> Thank you in advance for your help.
>> Fede
>>
>>
>> In GNU Emacs 24.3.1 (i386-mingw-nt6.1.7601)
>>  of 2013-03-17 on MARVIN
>> Windowing system distributor `Microsoft Corp.', version 6.1.7601
>> Configured using:
>>  `configure --with-gcc (4.7) --cflags
>>  -ID:/devel/emacs/libs/libXpm-3.5.8/include
>>  -ID:/devel/emacs/libs/libXpm-3.5.8/src
>>  -ID:/devel/emacs/libs/libpng-dev_1.4.3-1/include
>>  -ID:/devel/emacs/libs/zlib-dev_1.2.5-2/include
>>  -ID:/devel/emacs/libs/giflib-4.1.4-1/include
>>  -ID:/devel/emacs/libs/jpeg-6b-4/include
>>  -ID:/devel/emacs/libs/tiff-3.8.2-1/include
>>  -ID:/devel/emacs/libs/gnutls-3.0.9/include
>>  -ID:/devel/emacs/libs/libiconv-1.13.1-1-dev/include
>>  -ID:/devel/emacs/libs/libxml2-2.7.8/include/libxml2'
>>
>> Important settings:
>>   value of $LANG: ENU
>>   locale-coding-system: cp1252
>>   default enable-multibyte-characters: t
>>
>> Major mode: Lisp Interaction
>>
>> Minor modes in effect:
>>   show-paren-mode: t
>>   tooltip-mode: t
>>   mouse-wheel-mode: t
>>   tool-bar-mode: t
>>   menu-bar-mode: t
>>   file-name-shadow-mode: t
>>   global-font-lock-mode: t
>>   font-lock-mode: t
>>   blink-cursor-mode: t
>>   auto-composition-mode: t
>>   auto-encryption-mode: t
>>   auto-compression-mode: t
>>   line-number-mode: t
>>   transient-mark-mode: t
>>
>> Recent input:
>> <help-echo> C-x C-r C-SPC C-a C-w c : / p r o <tab>
>> SPC <tab> SPC <tab> e m <tab> 3 <tab> s i <tab> s i
>> <tab> d <tab> f <tab> <return> <next> <next> <down>
>> <down> <down> <down> <down> <down> <down> <down> <down>
>> <down> <down> <down> <down> <down> <down> <down> <down>
>> <down> <down> <down> <down> <down> <down> <down> <down>
>> <down> <down> <down> <down> <down> <down> <down> <down>
>> <down> <down> <down> <down> <down> <down> <down> C-SPC
>> <down> <down> <down> <down> <down> <down> <down> <down>
>> <down> <down> <down> <down> <left> M-w C-x b <return>
>> C-y M-x s h o w - p a <tab> <return> C-x C-e M-x r
>> u n - p y <tab> <return> C-x k <return> y e s <return>
>> C-x 0 M-x r u n - p <tab> y <tab> <return> <help-echo>
>> <help-echo> C-x o M-x r e p o <tab> r <tab> <retur
>> n>
>>
>> Recent messages:
>> "';'.join(get_ipython().Completer.all_completions('''%s'''))
>> "
>> Sent python-shell-completion-setup-code
>> Sent python-ffap-setup-code
>> Sent python-eldoc-setup-code
>> Making completion list...
>> Sent python-shell-completion-setup-code
>> Sent python-ffap-setup-code
>> Sent python-eldoc-setup-code
>> Making completion list...
>>
>> Load-path shadows:
>> None found.
>>
>> Features:
>> (shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
>> mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
>> gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
>> mail-prsvr mail-utils help-mode compile python rx easymenu comint ring
>> ansi-color cus-start cus-load paren time-date tooltip ediff-hook
>> vc-hooks lisp-float-type mwheel dos-w32 ls-lisp w32-common-fns
>> disp-table w32-win w32-vars tool-bar dnd fontset image regexp-opt fringe
>> tabulated-list newcomment lisp-mode register page menu-bar rfn-eshadow
>> timer select scroll-bar mouse jit-lock font-lock syntax facemenu
>> font-core frame cham georgian utf-8-lang misc-lang vietnamese tibetan
>> thai tai-viet lao korean japanese hebrew greek romanian slovak czech
>> european ethiopic indian cyrillic chinese case-table epa-hook
>> jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
>> cus-face macroexp files text-properties overlay sha1 md5 base64 format
>> env code-pages mule custom widget hashtable-print-readable backquote
>> make-network-process w32 multi-tty emacs)
>>
>>
>>