From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Eli Zaretskii Newsgroups: gmane.emacs.devel Subject: Re: Asynchronous DNS Date: Sun, 07 Feb 2016 17:55:37 +0200 Message-ID: <83mvrcsefa.fsf@gnu.org> References: <87si1gx6wz.fsf@gnus.org> <86y4b5zvzt.fsf@gmail.com> <8760y9kwrk.fsf@gnus.org> <8760y7nag7.fsf@gnus.org> <83oabzzsjq.fsf@gnu.org> <87fuxazkfe.fsf@gnus.org> <83io25yeqk.fsf@gnu.org> <87h9hpnreg.fsf@gnus.org> <83y4b0wi7m.fsf@gnu.org> <87si17evk6.fsf@gnus.org> <83twlnvcz2.fsf@gnu.org> <87vb63obm3.fsf@gnus.org> <87r3gqmg6g.fsf@gnus.org> <83egcqtfnm.fsf@gnu.org> <86mvrdmk8p.fsf@realize.ch> Reply-To: Eli Zaretskii NNTP-Posting-Host: plane.gmane.org X-Trace: ger.gmane.org 1454860570 30824 80.91.229.3 (7 Feb 2016 15:56:10 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 7 Feb 2016 15:56:10 +0000 (UTC) Cc: larsi@gnus.org, emacs-devel@gnu.org To: Alain Schneble Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sun Feb 07 16:56:09 2016 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1aSRh3-00024x-Jc for ged-emacs-devel@m.gmane.org; Sun, 07 Feb 2016 16:56:09 +0100 Original-Received: from localhost ([::1]:35835 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aSRh3-0006jG-1z for ged-emacs-devel@m.gmane.org; Sun, 07 Feb 2016 10:56:09 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:46342) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aSRgv-0006id-Nc for emacs-devel@gnu.org; Sun, 07 Feb 2016 10:56:05 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aSRgr-0000rL-5q for emacs-devel@gnu.org; Sun, 07 Feb 2016 10:56:01 -0500 Original-Received: from fencepost.gnu.org ([2001:4830:134:3::e]:57169) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aSRgr-0000rH-2G; Sun, 07 Feb 2016 10:55:57 -0500 Original-Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:4039 helo=home-c4e4a596f7) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1aSRgq-00040M-8U; Sun, 07 Feb 2016 10:55:56 -0500 In-reply-to: <86mvrdmk8p.fsf@realize.ch> (message from Alain Schneble on Sun, 7 Feb 2016 01:34:14 +0100) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:199444 Archived-At: > From: Alain Schneble > CC: Lars Ingebrigtsen , > Date: Sun, 7 Feb 2016 01:34:14 +0100 > > >> I kinda think we've gotten as far as we're going to get in this > >> discussion, and that it's time to merge. > > > > I'm sorry, but I disagree. I think the APIs that expect a fully > > functional process object should wait for the DNS resolution to > > complete, before they do anything. > > FWIW, the process-send-* functions do not wait for an async connection > to complete either, even without async-DNS-feature turned on. They > throw an error if called before the async connection completed, IIUC. That's not my reading of the code. Perhaps I'm missing something. Could you tell the details -- where do you see this? What I see is this: process-send-string calls send_process, which signals an error if either the status of the process object is other than 'run' or its output file descriptor is invalid. But make-network-process takes care to arrange for both of these: it calls make_process, which sets the process status to 'run' and sets up its output file descriptor to a valid value (the socket we got from the call to 'socket'). It does that even if the connection is asynchronous and has not yet completed. So I don't expect an error in this scenario with the existing code. If the connection is asynchronous, and is not yet completed, then send_process will wait in a loop, because the write to the socket will return EWOULDBLOCK or EAGAIN. Did I miss something? > And the documentation does not mention anything about this behavior. Well, if I'm right, it shouldn't ;-) > It seems to me like it is a matter of definition to say which functions > should work properly also before an async connection is established and > which not. I definitely hope that is not true. We couldn't have reliable network connections if it were true. > If backward compatibility were not an issue, we could also adopt a more > restrictive approach where errors would be thrown in many of the > functions accepting a (async socket) Lisp process and requiring a valid > infd/outfd before async connection completion. I think this would make network programming in Emacs a nuisance. > Documentation of `make-network-process' could say something like "if > :nowait is t, process returned shall not be used in any call before > async connection has completed. Consider performing any required > process configuration withing the SENTINEL". Oops, just realized > that this is what Lars more or less put in the doc string for the > 'dns case... Please consider the plight of a programmer who needs to implement such a connection. It would be much more complex and hard to get right than what we have today. E.g., look at smtpmail.el: it opens the network stream, sets up the process filter and encoding, and immediately proceeds to sending the HELO command to the server. Imagine how much more complex this simple job would be if it needed to jump through hoops as you described above. > Just as an overview, here is an extract of functions relying on a valid > infd/outfd and hence would be candidates that could either throw errors > or, as you suggested, synchronously wait for the DNS resolution to > complete. There are quite a few though: They all should call a single function that waits calling gai_error until the connection completes, and signals an error if it completes with a failure indication. That's a very simple and straightforward addition to the preamble of all those functions, and it should fix all those problems in a way that is 100% backward compatible. > >> There should be no impact on any existing use. It should be 100% > >> backward compatible. > > > > But it isn't, as was demonstrated by ERC. And it cannot be 100% > > backward compatible, unless the above-mentioned waiting is added to > > the affected APIs. > > I think what Lars meant was that with :nowait to t, client code should > be 100% backward compatible, as async connection establishment is still > handled the same way internally. Yes, I understand. But I don't think this is a proper way to be backward compatible. With my suggestion, code remains backward compatible even if it uses :nowait. This is better, IMO, because it leaves more applications backward compatible. For example, imagine that :nowait comes from some higher level, so the level that arranges the connection has no idea whether it needs to do the simple thing or the complex one. What Lars suggested means that every application that doesn't pass literal parameters to make-network-process will now have to analyze the parameters passed by the caller, to see if :nowait is there and what is its value. I don't expect package maintainers to love this. > Only when :nowait is 'dns, the new async DNS resolution is activated. > And only in this case, we might run into /new/ issues with calls to > functions accepting a Lisp process, such as the issue revealed by ERC > before Lars introduced 'dns. I think the solution I suggest will provide backward compatibility even if :nowait is 'dns. So the new issues are avoided. > On one hand, I see the practical approach with the 'dns async mode to > circumvent these compatibility issues. On the other hand, I don't think > it's a good choice from a interface/usage point of view. I think it is > not clear for a user why she has to care about whether both async DNS > and async socket connection or only the latter shall be used and why the > process behavior is different. This distinction only makes sense with > the knowledge of how it's implemented internally, I think. Yes, and so my suggestion is better, I think, as it avoids all of that. An application that didn't make the necessary changes will not be able to take the full advantage of the async DNS, but it will still work. > I guess changing the semantics of :nowait t, to say that clients shall > not use the returned process before connection is established, is out of > discussion, right? Or do you think that this would even be a ridiculous > restriction (even without the backward compatibility issue)? Yes, that'd be a terrible nuisance, IMO. Users and Lisp programmers will hate us if we do that.