bug#18396: 24.3.1; On windows, process-send-string can freeze Emacs

[Jorgen Schaefer <forcer@forcix.cx>]

On Wed, 03 Sep 2014 22:01:18 +0300
Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Wed, 3 Sep 2014 20:43:07 +0200
> > From: Jorgen Schaefer <forcer@forcix.cx>
> > Cc: 18396@debbugs.gnu.org
> > 
> > > Using "/" as the default directory on Windows is a bad idea, as
> > > that is not a fully-qualified absolute file name.
> > 
> > What would be the equivalent for "out of the way and not blocking
> > any mount point" (or equivalent) on Windows?
> 
> I might have a suggestion, if you explain what "out of the way" and
> "not blocking any mount points" mean.

The primary reason Elpy starts the process in "/" is to avoid
accidental imports of Python modules. As the process is started from a
Python buffer, there often are Python files in the current directory,
which can accidentally be imported. "/" is unlikely to have Python
modules.

Also, a current directory of "/" means the process won't
accidentally block a mount point, much like why daemons chdir to /. I'm
not sure if this concept makes sense in Windows.

> > > Looks like the write to the pipe never returned.  This could be
> > > because the pipe is full and is not being read from the other end
> > > (Windows pipes have 4K buffers, and you show above more than 6K of
> > > data).
> > 
> > That is quite likely the explanation. The Python process does the
> > equivalent of a REPL, reading one RPC call, evaluating it, and
> > writing the response. If in the duration of that evaluation Emacs
> > sends more than 4k of data, it will hang. If the response is larger
> > than 4k, Python in turn will hang. Resulting in a deadlock.
> > 
> > Am I missing something?
> 
> I'd expect Python to continue reading from the pipe once it evaluated
> one call and sent back the response.  It should see that more input is
> available and continue reading.

But if the sending of the response runs into the same problem? The
response can contain docstrings and can easily be larger than 4k, so
it's conceivable that Python sends more than 4k of data as well, which
would block the Python process, too? And thus prevent it from reading,
which keeps Emacs blocked?

> Could this be an end-of-line format issue?  Are you sure the commands
> used from Emacs side produce Windows-style CRLF EOLs?  Or maybe they
> do, but Python expects Unix-style newline-only EOLs (maybe it's a
> Cygwin or MSYS Python, for example)?  A wrong EOL format might cause
> Python to fail to realize it was handed a full line of input.

Unlikely. The RPC calls work perfectly fine most of the time, so the
line ending convention does not cause any confusion. Also, changing
the line ending convention on the Emacs side caused an error, so I
assume it's working fine as is.

> > Does Emacs have a chance to check for a pipe to be writable before
> > doing so? The whole process blocking like this feels a bit weird.
> 
> I don't know how to do such a check with pipes on Windows.  More
> importantly, how would that help?  The pipe will fill up anyway, and
> the communications with Python will stop.  Being able to interrupt
> with C-g vs killing the subprocess is not such a big win, IMO.

Well, if the deadlock hypothesis is correct, Emacs would check if the
pipe is writable, notice that it isn't and keep checking, to notice
that the pipe is readable, read data, and thus break the deadlock. That
of course requires that the deadlock hypothesis is indeed correct. :-)

> > > The obvious way: attach a debugger to Emacs and see where it is
> > > hung or waiting.  It is important to ask the user to produce
> > > backtraces from all the threads, because at least 2 threads are
> > > involved in interaction with a subprocess on MS-Windows.
> > 
> > Thanks. I'll ask, though I'm not sure if the user has a debugger
> > available.
> 
> If the user doesn't have GDB, he/she can download one from the MinGW
> site.  I think using a debugger is the only way to understand what
> happens here.

I'll pass it on, thanks.

Jorgen