Re: MPS: Win64 testers?

[Eli Zaretskii <eliz@gnu.org>]

> Date: Thu, 22 Aug 2024 19:14:51 +0000
> From: Pip Cet <pipcet@protonmail.com>
> Cc: sebastian@sebasmonia.com, kien.n.quang@gmail.com, emacs-devel@gnu.org, yantar92@posteo.net
> 
> "Eli Zaretskii" <eliz@gnu.org> writes:
> 
> I meant this code in gnutls.c:
> 
>   if (proc->gnutls_initstage < GNUTLS_STAGE_TRANSPORT_POINTERS_SET)
>     {
> # ifdef WINDOWSNT
>       /* On W32 we cannot transfer socket handles between different runtime
> 	 libraries, so we tell GnuTLS to use our special push/pull
> 	 functions.  */
>       gnutls_transport_set_ptr2 (state,
> 				 (gnutls_transport_ptr_t) proc,
> 				 (gnutls_transport_ptr_t) proc);
>       gnutls_transport_set_push_function (state, &emacs_gnutls_push);
>       gnutls_transport_set_pull_function (state, &emacs_gnutls_pull);
> # else
>       /* This is how GnuTLS takes sockets: as file descriptors passed
> 	 in.  For an Emacs process socket, infd and outfd are the
> 	 same but we use this two-argument version for clarity.  */
>       gnutls_transport_set_ptr2 (state,
> 				 (void *) (intptr_t) proc->infd,
> 				 (void *) (intptr_t) proc->outfd);
>       if (proc->is_non_blocking_client)
> 	gnutls_transport_set_errno_function (state,
> 					     emacs_gnutls_nonblock_errno);
> # endif
> 
>       proc->gnutls_initstage = GNUTLS_STAGE_TRANSPORT_POINTERS_SET;
>     }
> 
> 
> That hands a pointer to 'proc' to gnutls, and if 'proc' subsequently
> moves, emacs_gnutls_push or emacs_gnutls_pull will try to dereference a
> stale pointer.

But the pointer to 'proc' is passed to GnuTLS for the sole purpose
that GnuTLS passes it back to our callback, emacs_gnutls_pull.  GnuTLS
itself does not (and cannot) use that pointer, for the obvious reasons
that it knows nothing about our objects.

> > You are talking about this line in emacs_gnutls_pull:
> >
> >   emacs_gnutls_transport_set_errno (process->gnutls_state,
> >                                     errno == EWOULDBLOCK ? EAGAIN : errno);
> >
> > which causes us to call GnuTLS thusly:
> >
> >   gnutls_transport_set_errno (state, err);
> >
> > is that right?
> 
> > Does this mean we will have to audit all our calls to
> > external libraries and make sure we never pass to them any part of any
> > Lisp object?  That's a tough requirement; it's very easy to miss some
> > such use, which then triggers segfaults.  Am I missing something?
> 
> Synchronous calls do not need to be changed, only asynchronous callbacks
> that currently use a pointer to a Lisp object as their cookie.

But emacs_gnutls_pull is not an asynchronous call, it is called when
Emacs calls GnuTLS to read from the connection.  So how this could be
a problem?

And what do you mean by "asynchronous calls"?  For example, you can
see in w32font.c that we call certain Windows APIs which work by
enumerating objects of some kind and calling our callback for each
such object (the callback that does with each object what it needs to
do).  Is this an "asynchronous call"?  If not, what is?

> > Btw, I tried using EWW in my MinGW MPS build, and it didn't crash.  Is
> > it possible to have a reproducible recipe for this?
> 
> I'll try coming up with one, though that's probably just going to mean
> adding (igc-collect) after the gnutls handshake somewhere in the Lisp
> code.

Thanks.

But a clear understanding of what causes these segfaults is also
needed, because Emacs is full of similar callbacks, especially on
Windows, where many APIs we use work with callbacks.