From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Lars Ingebrigtsen Newsgroups: gmane.emacs.bugs Subject: bug#22493: 25.1.50; open-gnutls-stream doesn't respect :nowait, so the connections are synchronous Date: Thu, 04 Feb 2016 13:47:05 +1100 Message-ID: <87a8nhnqdy.fsf@gnus.org> References: <87mvrnzpge.fsf@gnus.org> <878u37zndq.fsf@gnus.org> <83r3gzwhg8.fsf@gnu.org> <87fuxebrsy.fsf@gnus.org> <878u36fung.fsf@gnus.org> <8360y93fka.fsf@gnu.org> <87wpqpwd8p.fsf@gnus.org> <83d1sh14is.fsf@gnu.org> <87egcx13kc.fsf@gnus.org> <83bn801d06.fsf@gnu.org> <87bn7znazd.fsf@gnus.org> <83lh73ytxo.fsf@gnu.org> <87a8nizjia.fsf@gnus.org> <8760y6zha1.fsf@gnus.org> <83fux9yend.fsf@gnu.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1454554103 18147 80.91.229.3 (4 Feb 2016 02:48:23 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 4 Feb 2016 02:48:23 +0000 (UTC) Cc: 22493@debbugs.gnu.org To: Eli Zaretskii Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Thu Feb 04 03:48:12 2016 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1aR9xr-0001g2-8l for geb-bug-gnu-emacs@m.gmane.org; Thu, 04 Feb 2016 03:48:11 +0100 Original-Received: from localhost ([::1]:39115 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aR9xq-0002OH-4M for geb-bug-gnu-emacs@m.gmane.org; Wed, 03 Feb 2016 21:48:10 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:33241) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aR9xl-0002Ny-6d for bug-gnu-emacs@gnu.org; Wed, 03 Feb 2016 21:48:06 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aR9xh-00037w-VW for bug-gnu-emacs@gnu.org; Wed, 03 Feb 2016 21:48:05 -0500 Original-Received: from debbugs.gnu.org ([208.118.235.43]:49984) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aR9xh-00037s-Ry for bug-gnu-emacs@gnu.org; Wed, 03 Feb 2016 21:48:01 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84) (envelope-from ) id 1aR9xh-00081N-N1 for bug-gnu-emacs@gnu.org; Wed, 03 Feb 2016 21:48:01 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Lars Ingebrigtsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 04 Feb 2016 02:48:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 22493 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: Original-Received: via spool by 22493-submit@debbugs.gnu.org id=B22493.145455405430797 (code B ref 22493); Thu, 04 Feb 2016 02:48:01 +0000 Original-Received: (at 22493) by debbugs.gnu.org; 4 Feb 2016 02:47:34 +0000 Original-Received: from localhost ([127.0.0.1]:58573 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aR9xG-00080f-59 for submit@debbugs.gnu.org; Wed, 03 Feb 2016 21:47:34 -0500 Original-Received: from hermes.netfonds.no ([80.91.224.195]:55209) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aR9xE-00080X-L8 for 22493@debbugs.gnu.org; Wed, 03 Feb 2016 21:47:33 -0500 Original-Received: from cpe-60-225-211-161.nsw.bigpond.net.au ([60.225.211.161] helo=mouse) by hermes.netfonds.no with esmtpsa (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.72) (envelope-from ) id 1aR9ws-0005tO-3E; Thu, 04 Feb 2016 03:47:10 +0100 In-Reply-To: <83fux9yend.fsf@gnu.org> (Eli Zaretskii's message of "Wed, 03 Feb 2016 17:52:54 +0200") User-Agent: Gnus/5.130014 (Ma Gnus v0.14) Emacs/25.1.50 (gnu/linux) X-MailScanner-ID: 1aR9ws-0005tO-3E MailScanner-NULL-Check: 1455158831.14842@DFmUK58xXDk9BpPI2vRXUw X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.bugs:112366 Archived-At: Eli Zaretskii writes: >> With async DNS, this will fail, because the process-send-string happens >> before the connection had completed. (And this isn't a TLS socket.) So >> I think that (like I said on the emacs-devel threads) we may have to >> change the :nowait stuff to allow a more fine-grained control. > > If you make process-send-string wait until DNS completes, the problem > will disappear. Yes. That was basically what the #ifdef HAVE_GNUTLS /* The TLS connection hasn't been set up yet, so we can't write anything on the socket. */ if (!NILP (p->gnutls_boot_parameters)) return; #endif in send_process achieves (in the TLS case). If we amend that to if (!NILP (p->gnutls_boot_parameters) || p->outfd == 0) return; then that will have the same effect on non-TLS sockets. But as noted in the previous email, set-process-coding-system would also have to be changed to remove these outfd guards... >> make-network-process gives us a process in "open", and then when it >> changes to "connected" (after connecting the socket) we can't start >> talking. We have to wait until the TLS has been negotiated. So perhaps >> it should only move to the "connected" state after the negotiation has >> finished? Or introduce more states? > > Like with process-send-string, we could make gnutls-boot wait until > DNS resolution completes (and error out if DNS fails). That's true... Hm... > As for making gnutls-boot run in the background, I'm still not > convinced it buys us enough advantages (or any advantages) to justify > the changes. Let's continue discussing that, OK? I think we've been talking a bit past each other on the TLS issue, so I'll just write out the scenario as I see it, and why I think it has to be implemented the way I did it, and you can correct me if you see other options. OK: eww has displayed a web page, and now wants to fetch an image over https. Our goal is to have Emacs stop as little as possible while this is happening in the background, so that the user can scroll around or do whatever the user wants to do while eww is working on getting the image. So eww issues a url-retrieve command in the background, which ends up calling make-network-process :nowait t. This returns immediately, with no user-noticeable hangs. getaddrinfo_a has been issued, and after a while a response is received and is noticed by the idle loop. Emacs then calls connect_network_process. This basically is a wrapper around the connect(2), and it's pretty async on a non-blocking socket, so this should not be noticeable to the user. Everything up until now has happened identically for TLS and non-TLS sockets. Now, for a TLS socket, we have to do the negotiating. This is currently a blocking process, but it doesn't have to be. The GnuTLS library in itself is non-blocking, but our interaction with it currently is. So we call gnutls_boot after the connection has happened. I originally did this with a sentinel on a process, but that doesn't really work, because the callers of make_network_process want their own sentinels on the process. So I call gnutls_boot from the C layer instead of from a sentinel. No matter how we call gnutls_boot, it will currently hang Emacs while it's transferring all those certificates back and forth. After that, the url.el sentinel will fire with a "connect" state, and it'll say "GET /foo.png" to the server, and the server will output the image (non-blocking). And then the image will be parsed and inserted into the buffer (blocking and noticeable for large images). *phew* :-) > Running gnutls-boot from a separate thread on the C level is also > possible, but AFAIU that requires a total rewrite of the current C > implementation, as it does a lot of stuff that cannot be done from a > non-main thread. Yup. > Either way, if we decide that making gnutls-boot run in the background > is a Good Thing, then I feel that doing so requires code restructuring > that must involve application-level considerations. Actually, almost all of the major network usages in Emacs today require no changes. The only one I've found is erc. url.el, Gnus, smtpmail, imap.el, nntp.el all work without any changes with the current implementation. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no