Re: Calling 'select' from emacs_gnutls_pull

[Ted Zlatanov <tzz@lifelogs.com>]

On Sat, 16 Feb 2013 13:38:43 +0200 Eli Zaretskii <eliz@gnu.org> wrote: 

EZ> I need your help in understanding the implementation of
EZ> emacs_gnutls_pull.  Specifically, why does it need to call 'select'
EZ> when 'sys_read' returns EWOULDBLOCK?  The file descriptor should
EZ> already be watched by the call to 'select' in
EZ> 'wait_reading_process_output', so it seems like a call to 'sys_read'
EZ> is all that's needed here.  Am I missing something?

Well, I *think* the function `emacs_gnutls_pull' is called in a
different context from most of Emacs.  In gnutls.c we set that function
as the transport pull at the GnuTLS level:

fn_gnutls_transport_set_push_function (state, &emacs_gnutls_push);
fn_gnutls_transport_set_pull_function (state, &emacs_gnutls_pull);

...and there are comments accumulated over the years in that
neighborhood.  Claudio Bley contributed much of the original code here,
including the piece you're referring to.  The history is at

http://comments.gmane.org/gmane.emacs.devel/136816

EZ> The reason I ask is that someone reported seeing these messages when
EZ> running Emacs on Windows under a debugger:

EZ>   warning: sys_read called when read is in progress

EZ> This comes from 'sys_read', and it means that the reader thread (used
EZ> by Emacs to read from a socket) is already blocked inside a call to
EZ> 'recv' on behalf of this file descriptor, when 'sys_read' is called.

OK.

EZ> I think that the sequence of events which leads to this is as follows:

EZ>   . emacs_gnutls_pull calls sys_read, which returns EWOULDBLOCK,
EZ>     because there's no data ready to be read

EZ>   . emacs_gnutls_pull then calls select, which wakes up the reader
EZ>     thread and tells it to try reading from the socket

EZ>   . the reader thread blocks inside the call to recv (since no data is
EZ>     ready)

EZ>   . the call to select times out and returns EWOULDBLOCK, which causes
EZ>     emacs_gnutls_pull to break from the loop and return EAGAIN

EZ>   . after some time, emacs_gnutls_pull is called again, and calls
EZ>     sys_read while the reader thread is still blocked in recv, so we
EZ>     get the above message

EZ> Is this a plausible description of what might happen?  If so, would it
EZ> make sense to avoid calling 'select' in emacs_gnutls_pull, and instead
EZ> let 'wait_reading_process_output' call 'select', as it does for all
EZ> the other sockets open in Emacs?  Or am I missing something?

I think you're correct, but remember the GnuTLS library is calling these
functions.  So I don't know if it would make sense to let the Emacs
reader thread handle the socket management, because we could end up with
poor performance or even contention.  I just don't know the intricacies
of the W32 I/O system and how it interacts with GnuTLS well enough.

The best way is to ask the GnuTLS developers and to test it, I think.
There are some hints at

http://www.gnu.org/software/gnutls/manual/html_node/Asynchronous-operation.html
http://stackoverflow.com/questions/911565/select-recv-and-ewouldblock-on-non-blocking-sockets

Sorry this is all I can suggest right now, but if I can help simulate
this condition or test some possible solutions, I'll be glad to assist.

Ted