bug#18396: 24.3.1; On windows, process-send-string can freeze Emacs

[Eli Zaretskii <eliz@gnu.org>]

> Date: Wed, 3 Sep 2014 21:28:33 +0200
> From: Jorgen Schaefer <forcer@forcix.cx>
> Cc: 18396@debbugs.gnu.org
> 
> On Wed, 03 Sep 2014 22:01:18 +0300
> Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > > Date: Wed, 3 Sep 2014 20:43:07 +0200
> > > From: Jorgen Schaefer <forcer@forcix.cx>
> > > Cc: 18396@debbugs.gnu.org
> > > 
> > > > Using "/" as the default directory on Windows is a bad idea, as
> > > > that is not a fully-qualified absolute file name.
> > > 
> > > What would be the equivalent for "out of the way and not blocking
> > > any mount point" (or equivalent) on Windows?
> > 
> > I might have a suggestion, if you explain what "out of the way" and
> > "not blocking any mount points" mean.
> 
> The primary reason Elpy starts the process in "/" is to avoid
> accidental imports of Python modules. As the process is started from a
> Python buffer, there often are Python files in the current directory,
> which can accidentally be imported. "/" is unlikely to have Python
> modules.

Why not use ~/, then?

> Also, a current directory of "/" means the process won't
> accidentally block a mount point, much like why daemons chdir to /. I'm
> not sure if this concept makes sense in Windows.

(expand-file-name "/") should do what you want, I think.

> > > > Looks like the write to the pipe never returned.  This could be
> > > > because the pipe is full and is not being read from the other end
> > > > (Windows pipes have 4K buffers, and you show above more than 6K of
> > > > data).
> > > 
> > > That is quite likely the explanation. The Python process does the
> > > equivalent of a REPL, reading one RPC call, evaluating it, and
> > > writing the response. If in the duration of that evaluation Emacs
> > > sends more than 4k of data, it will hang. If the response is larger
> > > than 4k, Python in turn will hang. Resulting in a deadlock.
> > > 
> > > Am I missing something?
> > 
> > I'd expect Python to continue reading from the pipe once it evaluated
> > one call and sent back the response.  It should see that more input is
> > available and continue reading.
> 
> But if the sending of the response runs into the same problem?

Each direction of the pipe has its own separate buffering, so this is
unlikely.

> The response can contain docstrings and can easily be larger than
> 4k, so it's conceivable that Python sends more than 4k of data as
> well, which would block the Python process, too? And thus prevent it
> from reading, which keeps Emacs blocked?

Emacs reads in a separate thread, so again, unlikely.

> > Could this be an end-of-line format issue?  Are you sure the commands
> > used from Emacs side produce Windows-style CRLF EOLs?  Or maybe they
> > do, but Python expects Unix-style newline-only EOLs (maybe it's a
> > Cygwin or MSYS Python, for example)?  A wrong EOL format might cause
> > Python to fail to realize it was handed a full line of input.
> 
> Unlikely. The RPC calls work perfectly fine most of the time

Perhaps this user has a different kind of Python than others.

> > > Does Emacs have a chance to check for a pipe to be writable before
> > > doing so? The whole process blocking like this feels a bit weird.
> > 
> > I don't know how to do such a check with pipes on Windows.  More
> > importantly, how would that help?  The pipe will fill up anyway, and
> > the communications with Python will stop.  Being able to interrupt
> > with C-g vs killing the subprocess is not such a big win, IMO.
> 
> Well, if the deadlock hypothesis is correct, Emacs would check if the
> pipe is writable, notice that it isn't and keep checking, to notice
> that the pipe is readable, read data, and thus break the deadlock.

See above: there's no interconnection between reading and writing, so
that's not the problem.