[RFC] automatically retrying network connections

[RFC] automatically retrying network connections

[Robert Pluim]

Hi,

one of the consequences of asking the user questions during network
connection setup is that the server they're trying to connect to might
have decided to close the connection by the time they've finished
answering.

This suggests that there should be an option to retry the connection
in such a situation. I have a trial implementation involving a new
parameter to open-network-stream to request retrying, but was
wondering if it might not be better if it was a globally enabled
default. Iʼll note that various browsers retry failing connections
automatically.

Thanks

Robert

Re: [RFC] automatically retrying network connections

[Jimmy Yuen Ho Wong]

On Fri, Jul 20, 2018 at 9:06 PM, Robert Pluim <rpluim@gmail.com> wrote:
> Hi,
>
> one of the consequences of asking the user questions during network
> connection setup is that the server they're trying to connect to might
> have decided to close the connection by the time they've finished
> answering.
>
> This suggests that there should be an option to retry the connection
> in such a situation. I have a trial implementation involving a new
> parameter to open-network-stream to request retrying, but was
> wondering if it might not be better if it was a globally enabled
> default. Iʼll note that various browsers retry failing connections
> automatically.
>
> Thanks
>
> Robert
>

I have a related but different issue, and I think I'm looking at a
different solution from an option that automatically retries at layer
4.

The OCSP RFC I'm implementing specifies a response status type
`tryLater`, this is on layer 7. What I really need is some kind of
promise/future primitive or await async built into Emacs so I don't
have to block the other OCSP requests that I need to make while I'm
waiting for a good answer for this one, or worse, block the UI. I'm
currently using generators to alleviate UI blocking a bit, but I
haven't found a good answer to making async network processes and then
joining their results later. (short of writing a giant block of
callback spagetti anyway...)

P.S. Thanks a lot for exposing `getaddrinfo`, I haven't tried it out
yet. I'll get it ASAP.

Re: [RFC] automatically retrying network connections

[Lars Ingebrigtsen]

Jimmy Yuen Ho Wong <wyuenho@gmail.com> writes:

> I'm currently using generators to alleviate UI blocking a bit, but I
> haven't found a good answer to making async network processes and then
> joining their results later. (short of writing a giant block of
> callback spagetti anyway...)

You want to do a bunch of async HTTP calls, and when they're all
complete (including possibly some tryLaters), then you finish the entire
thing?  I don't really see the problem...  can't you just create a queue
(a la url-queue), or if you want to do all the calls in parallel, just
create a structure all the threads fill in and the last one that
finishes does the "complete" action?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [RFC] automatically retrying network connections

[Jimmy Yuen Ho Wong]

On Sun, Jul 22, 2018 at 11:28 AM, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> Jimmy Yuen Ho Wong <wyuenho@gmail.com> writes:
>
>> I'm currently using generators to alleviate UI blocking a bit, but I
>> haven't found a good answer to making async network processes and then
>> joining their results later. (short of writing a giant block of
>> callback spagetti anyway...)
>
> You want to do a bunch of async HTTP calls, and when they're all
> complete (including possibly some tryLaters), then you finish the entire
> thing?  I don't really see the problem...  can't you just create a queue
> (a la url-queue), or if you want to do all the calls in parallel, just
> create a structure all the threads fill in and the last one that
> finishes does the "complete" action?
>

The latter, parellel requests, and then join the results when they all
come back. I've come up with some bastardization of MapReduce now, but
this would be a lot nicer if Emacs had something like
https://github.com/chuntaro/emacs-promise built in, so I can combine
it with a generator to emulate async/await semantics.

Re: [RFC] automatically retrying network connections

[Michael Albinus]

Jimmy Yuen Ho Wong <wyuenho@gmail.com> writes:

> The latter, parellel requests, and then join the results when they all
> come back. I've come up with some bastardization of MapReduce now, but
> this would be a lot nicer if Emacs had something like
> https://github.com/chuntaro/emacs-promise built in, so I can combine
> it with a generator to emulate async/await semantics.

Emacs has threads. So you might do something like

(let (threads result)
  ;; Create a thread per request.
  (setq threads
        (mapcar
         (lambda (request)
           (make-thread
            (lambda () <do your request>))))
         <all-requests>)
  ;; Collect the results.
  (thread-yield)
  (dolist (thread threads result)
    (setq result (cons (thread-join thread) result))))

Best regards, Michael.

Re: [RFC] automatically retrying network connections

[Jimmy Yuen Ho Wong]

On Sun, Jul 22, 2018 at 5:44 PM, Michael Albinus <michael.albinus@gmx.de> wrote:
> Jimmy Yuen Ho Wong <wyuenho@gmail.com> writes:
>
>> The latter, parellel requests, and then join the results when they all
>> come back. I've come up with some bastardization of MapReduce now, but
>> this would be a lot nicer if Emacs had something like
>> https://github.com/chuntaro/emacs-promise built in, so I can combine
>> it with a generator to emulate async/await semantics.
>
> Emacs has threads. So you might do something like
>
> (let (threads result)
>   ;; Create a thread per request.
>   (setq threads
>         (mapcar
>          (lambda (request)
>            (make-thread
>             (lambda () <do your request>))))
>          <all-requests>)
>   ;; Collect the results.
>   (thread-yield)
>   (dolist (thread threads result)
>     (setq result (cons (thread-join thread) result))))
>
> Best regards, Michael.

Every `thread-join` blocks, isn't it basically the same as a generator?

If nobody objects, I'm just going to use a generator for now, and see
if I can get the emacs-promise author to donate it to the core.

Re: [RFC] automatically retrying network connections

[Robert Pluim]

Jimmy Yuen Ho Wong <wyuenho@gmail.com> writes:

>
> P.S. Thanks a lot for exposing `getaddrinfo`, I haven't tried it out
> yet. I'll get it ASAP.

Of course, emacs being emacs:

    dns-query is a compiled Lisp function in `dns.el'.

    (dns-query NAME &optional TYPE FULLP REVERSEP)

    Query a DNS server for NAME of TYPE.
    If FULLP, return the entire record returned.
    If REVERSEP, look up an IP address.

which you could use instead of 'nslookup' or 'getaddrinfo' (although
'dns-query' can also punt to 'nslookup', so maybe getaddrinfo is
better).

Robert

Re: [RFC] automatically retrying network connections

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

>     (dns-query NAME &optional TYPE FULLP REVERSEP)
>
>     Query a DNS server for NAME of TYPE.
>     If FULLP, return the entire record returned.
>     If REVERSEP, look up an IP address.
>
> which you could use instead of 'nslookup' or 'getaddrinfo' (although
> 'dns-query' can also punt to 'nslookup', so maybe getaddrinfo is
> better).

getaddrinfo is probably a lot better.  `dns-query' isn't that good at
retries and the like, if I remember correctly.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [RFC] automatically retrying network connections

[Jimmy Yuen Ho Wong]

On Sun, Jul 22, 2018 at 12:43 PM, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> Robert Pluim <rpluim@gmail.com> writes:
>
>>     (dns-query NAME &optional TYPE FULLP REVERSEP)
>>
>>     Query a DNS server for NAME of TYPE.
>>     If FULLP, return the entire record returned.
>>     If REVERSEP, look up an IP address.
>>
>> which you could use instead of 'nslookup' or 'getaddrinfo' (although
>> 'dns-query' can also punt to 'nslookup', so maybe getaddrinfo is
>> better).
>
> getaddrinfo is probably a lot better.  `dns-query' isn't that good at
> retries and the like, if I remember correctly.
>

He's just pointing out one of the MANY functions in emacs that allows
you to look up an IP address. I've found at least 4, and we now
introducing the 5th and 6th...

Re: [RFC] automatically retrying network connections

[Andreas Schwab]

On Jul 23 2018, Jimmy Yuen Ho Wong <wyuenho@gmail.com> wrote:

> On Sun, Jul 22, 2018 at 12:43 PM, Lars Ingebrigtsen <larsi@gnus.org> wrote:
>> Robert Pluim <rpluim@gmail.com> writes:
>>
>>>     (dns-query NAME &optional TYPE FULLP REVERSEP)
>>>
>>>     Query a DNS server for NAME of TYPE.
>>>     If FULLP, return the entire record returned.
>>>     If REVERSEP, look up an IP address.
>>>
>>> which you could use instead of 'nslookup' or 'getaddrinfo' (although
>>> 'dns-query' can also punt to 'nslookup', so maybe getaddrinfo is
>>> better).
>>
>> getaddrinfo is probably a lot better.  `dns-query' isn't that good at
>> retries and the like, if I remember correctly.

dns-query can only do a single DNS query, which is unusable for proper
address resolution.

> He's just pointing out one of the MANY functions in emacs that allows
> you to look up an IP address. I've found at least 4, and we now
> introducing the 5th and 6th...

Address resolution isn't trivial, that's why getaddrinfo exists.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

Re: [RFC] automatically retrying network connections

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> one of the consequences of asking the user questions during network
> connection setup is that the server they're trying to connect to might
> have decided to close the connection by the time they've finished
> answering.

Yes, the NSM should reconnect if the user says "go ahead" to the NSM
warning and the server has closed the connection.  The reason that
landed on the back burner is that servers seem to mostly have long
timeouts, so this turned out to be less of a problem in practice than I
expected.  (I.e., I don't recall seeing a bug report about this, which
is just plain weird.)

We could also reverse the logic a bit and have the NSM always shut down
the connection before prompting.  This will make network connections
(that prompt an NSM warning) be somewhat slower, but it shouldn't be a
big deal.

In any case, when reconnecting we have to consider whether this would
trigger extra auth-source prompts, which would be annoying, but I have a
feeling like these are mostly done later in the connection process
usually, so it shouldn't be an issue.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [RFC] automatically retrying network connections

[Robert Pluim]

[-- Attachment #1: Type: text/plain, Size: 1927 bytes --]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Robert Pluim <rpluim@gmail.com> writes:
>
>> one of the consequences of asking the user questions during network
>> connection setup is that the server they're trying to connect to might
>> have decided to close the connection by the time they've finished
>> answering.
>
> Yes, the NSM should reconnect if the user says "go ahead" to the NSM
> warning and the server has closed the connection.  The reason that
> landed on the back burner is that servers seem to mostly have long
> timeouts, so this turned out to be less of a problem in practice than I
> expected.  (I.e., I don't recall seeing a bug report about this, which
> is just plain weird.)

Thereʼs this little-known news server called news.gmane.org that has a
10 second timeout :-) Reconnecting immediately works, so itʼs not a
big deal.

So you'd put this in nsm-verify-connection, without a user option? I
donʼt think nsm currently has access to all the connection setup
parameters, which is why I put the logic in open-network-stream.
Proof-of-concept patch attached (it should really do more checking of
what nsm returned).

> We could also reverse the logic a bit and have the NSM always shut down
> the connection before prompting.  This will make network connections
> (that prompt an NSM warning) be somewhat slower, but it shouldn't be a
> big deal.

That seems wasteful. In most cases the connection will complete OK, so
why add an extra connection setup for all cases?

> In any case, when reconnecting we have to consider whether this would
> trigger extra auth-source prompts, which would be annoying, but I have a
> feeling like these are mostly done later in the connection process
> usually, so it shouldn't be an issue.

If youʼre thinking of authentication for SMTP sessions and the like,
they'd all arrive after completion of the TLS setup.

Robert


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Allow-retrying-of-network-connections-on-failure.patch --]
[-- Type: text/x-diff, Size: 5647 bytes --]

From 99ba6345bf38305b5480d07851bc5d64fd3e9dcd Mon Sep 17 00:00:00 2001
From: Robert Pluim <rpluim@gmail.com>
Date: Fri, 20 Jul 2018 18:35:22 +0200
Subject: [PATCH] Allow retrying of network connections on failure
To: emacs-devel@gnu.org

With network-security-manager, there are cases where the user is
expected to answer one or more questions before a network connection
is finally established, which can result in the other end having timed
out.  Allow specifying :retry-on-fail t to open-network-stream to
retry the connection, which should now succeed.

* src/process.c (syms_of_process): Define Qprocess_not_running_error
symbol.
(send_process): Signal Qprocess_not_running_error for the specific
case of the process having died only.

* lisp/net/network-stream.el (open-network-stream): Add :retry-on-fail
parameter, and retry network connection if it is set and we receive a
Qprocess_not_running_error.
---
 lisp/net/network-stream.el | 42 +++++++++++++++++++++++++++-----------
 src/process.c              | 15 +++++++++++++-
 2 files changed, 44 insertions(+), 13 deletions(-)

diff --git a/lisp/net/network-stream.el b/lisp/net/network-stream.el
index a0589e25a4..3ecd8c2c64 100644
--- a/lisp/net/network-stream.el
+++ b/lisp/net/network-stream.el
@@ -153,22 +153,34 @@ open-network-stream
 opening a TLS connection.  The first element is the TLS
 type (either `gnutls-x509pki' or `gnutls-anon'), and the
 remaining elements should be a keyword list accepted by
-gnutls-boot (as returned by `gnutls-boot-parameters')."
+gnutls-boot (as returned by `gnutls-boot-parameters').
+
+:retry-on-fail, if non-nil, means attempt the connection again,
+once, if it initially fails.  This is to cover the case where
+answering network-security-manager questions causes the remote
+server to timeout the connection."
   (unless (featurep 'make-network-process)
     (error "Emacs was compiled without networking support"))
   (let ((type (plist-get parameters :type))
-	(return-list (plist-get parameters :return-list)))
+	(return-list (plist-get parameters :return-list))
+        (fun 'make-network-process)
+        (args (list :name name :buffer buffer
+                    :host (puny-encode-domain host) :service service
+                    :nowait (plist-get parameters :nowait)
+                    :tls-parameters
+                    (plist-get parameters :tls-parameters))))
     (if (and (not return-list)
 	     (or (eq type 'plain)
 		 (and (memq type '(nil network))
 		      (not (and (plist-get parameters :success)
 				(plist-get parameters :capability-command))))))
 	;; The simplest case: wrapper around `make-network-process'.
-	(make-network-process :name name :buffer buffer
-			      :host (puny-encode-domain host) :service service
-			      :nowait (plist-get parameters :nowait)
-                              :tls-parameters
-                              (plist-get parameters :tls-parameters))
+        (condition-case err
+            (apply fun args)
+          (process-not-running-error
+           (when (plist-get parameters :retry-on-fail)
+             (delete-process (cdr err))
+             (apply fun args))))
       (let ((work-buffer (or buffer
 			     (generate-new-buffer " *stream buffer*")))
 	    (fun (cond ((and (eq type 'plain)
@@ -181,12 +193,18 @@ open-network-stream
 		       ((eq type 'shell) 'network-stream-open-shell)
 		       (t (error "Invalid connection type %s" type))))
 	    result)
-	(unwind-protect
+	(condition-case err
 	    (setq result (funcall fun name work-buffer host service parameters))
-	  (unless buffer
-	    (and (processp (car result))
-		 (set-process-buffer (car result) nil))
-	    (kill-buffer work-buffer)))
+          (process-not-running-error
+           (when (plist-get parameters :retry-on-fail)
+            (delete-process (cdr err))
+            (setq result (funcall fun name work-buffer host service parameters))))
+          (error
+	   (unless buffer
+	     (and (processp (car result))
+	          (set-process-buffer (car result) nil))
+	     (kill-buffer work-buffer))
+           (signal (car err) (cdr err))))
 	(if return-list
 	    (list (car result)
 		  :greeting     (nth 1 result)
diff --git a/src/process.c b/src/process.c
index 279b74bc66..222bd9fcff 100644
--- a/src/process.c
+++ b/src/process.c
@@ -6233,8 +6233,11 @@ send_process (Lisp_Object proc, const char *buf, ptrdiff_t len,
 
   if (p->raw_status_new)
     update_status (p);
+  /* Process might have died whilst we were waiting for the user to
+     answer nsm questions.  Signal a specific error in that case so
+     higher levels can retry if they want.  */
   if (! EQ (p->status, Qrun))
-    error ("Process %s not running", SDATA (p->name));
+    xsignal (Qprocess_not_running_error, proc);
   if (p->outfd < 0)
     error ("Output file descriptor of %s is closed", SDATA (p->name));
 
@@ -8091,6 +8094,8 @@ syms_of_process (void)
 {
 #ifdef subprocesses
 
+  Lisp_Object error_tail;
+
   DEFSYM (Qprocessp, "processp");
   DEFSYM (Qrun, "run");
   DEFSYM (Qstop, "stop");
@@ -8241,6 +8246,14 @@ returns non-`nil'.  */);
 	  "internal-default-interrupt-process");
   DEFSYM (Qinterrupt_process_functions, "interrupt-process-functions");
 
+  DEFSYM (Qprocess_not_running_error, "process-not-running-error");
+  error_tail = pure_cons (Qprocess_not_running_error, Qnil);
+
+  Fput (Qprocess_not_running_error, Qerror_conditions,
+	error_tail);
+  Fput (Qprocess_not_running_error, Qerror_message,
+	build_pure_c_string ("process not running error"));
+
   defsubr (&Sprocessp);
   defsubr (&Sget_process);
   defsubr (&Sdelete_process);
-- 
2.18.0.129.ge3331758f1

Re: [RFC] automatically retrying network connections

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> Thereʼs this little-known news server called news.gmane.org that has a
> 10 second timeout :-) Reconnecting immediately works, so itʼs not a
> big deal.

Never heard about it.  :-)

> So you'd put this in nsm-verify-connection, without a user option? I
> donʼt think nsm currently has access to all the connection setup
> parameters, which is why I put the logic in open-network-stream.
> Proof-of-concept patch attached (it should really do more checking of
> what nsm returned).

I was thinking that the `nsm-verify-connection' call sites would
reconnect that function said "go ahead" and the process was dead.  Which
is basically if it returns non-nil, I guess, so the return value of that
function doesn't have to change.

:retry-on-fail is too general -- it's not really useful to reestablish
the connection if the server closes the connection for other reasons
than an NSM-related timeout.

>> We could also reverse the logic a bit and have the NSM always shut down
>> the connection before prompting.  This will make network connections
>> (that prompt an NSM warning) be somewhat slower, but it shouldn't be a
>> big deal.
>
> That seems wasteful. In most cases the connection will complete OK, so
> why add an extra connection setup for all cases?

An NSM warning isn't the common case, and reestablishing the connection
would be a minuscule extra delay.  It would just be more...  consistent?
But little real-world impact.

>> In any case, when reconnecting we have to consider whether this would
>> trigger extra auth-source prompts, which would be annoying, but I have a
>> feeling like these are mostly done later in the connection process
>> usually, so it shouldn't be an issue.
>
> If youʼre thinking of authentication for SMTP sessions and the like,
> they'd all arrive after completion of the TLS setup.

I think so, too, but we should audit the call sites and ensure that
nobody does stuff like

(open-network-stream ... :starttls-function () (auth-source-search ...))

Seems unlikely, though.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [RFC] automatically retrying network connections

[Robert Pluim]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I was thinking that the `nsm-verify-connection' call sites would
> reconnect that function said "go ahead" and the process was dead.  Which
> is basically if it returns non-nil, I guess, so the return value of that
> function doesn't have to change.

Is this the kind of change youʼre thinking of?

diff --git i/lisp/net/network-stream.el w/lisp/net/network-stream.el
index a0589e25a4..dbaf719f41 100644
--- i/lisp/net/network-stream.el
+++ w/lisp/net/network-stream.el
@@ -383,7 +383,7 @@ network-stream-get-response
 
 (declare-function open-tls-stream "tls" (name buffer host port))
 
-(defun network-stream-open-tls (name buffer host service parameters)
+(defun network-stream-open-tls (name buffer host service parameters &optional retry)
   (with-current-buffer buffer
     (let* ((start (point-max))
 	   (stream
@@ -398,10 +398,14 @@ network-stream-open-tls
         ;; Check certificate validity etc.
         (when (and (gnutls-available-p) stream)
           (setq stream (nsm-verify-connection stream host service)))
-        (if (null stream)
-            (list nil nil nil 'plain)
+        (cond ((null stream)
+               (list nil nil nil 'plain))
               ;; If we're using tls.el, we have to delete the output from
               ;; openssl/gnutls-cli.
+              ((and (not retry) (not (process-live-p)))
+               (message "Initial connect failed, retrying")
+               (network-stream-open-tls name buffer host service parameters t))
+              (t
                (when (and (not (gnutls-available-p))
                           eoc)
                  (network-stream-get-response stream start eoc)
@@ -415,7 +419,7 @@ network-stream-open-tls
                  (list stream
                        (network-stream-get-response stream start eoc)
                        (network-stream-command stream capability-command eo-capa)
-                  'tls)))))))
+                  'tls))))))))
 
 (declare-function format-spec "format-spec" (format spec))
 (declare-function format-spec-make "format-spec" (&rest pairs))

Re: [RFC] automatically retrying network connections

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> Is this the kind of change youʼre thinking of?

Yup.  Or at a layer higher up, if that makes sense (to avoid code
duplication).  The async NSM check would need some more scaffolding,
though...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [RFC] automatically retrying network connections

[Andy Moreton]

On Sun 22 Jul 2018, Lars Ingebrigtsen wrote:

> Robert Pluim <rpluim@gmail.com> writes:
>
>> one of the consequences of asking the user questions during network
>> connection setup is that the server they're trying to connect to might
>> have decided to close the connection by the time they've finished
>> answering.
>
> Yes, the NSM should reconnect if the user says "go ahead" to the NSM
> warning and the server has closed the connection.  The reason that
> landed on the back burner is that servers seem to mostly have long
> timeouts, so this turned out to be less of a problem in practice than I
> expected.  (I.e., I don't recall seeing a bug report about this, which
> is just plain weird.)
>
> We could also reverse the logic a bit and have the NSM always shut down
> the connection before prompting.  This will make network connections
> (that prompt an NSM warning) be somewhat slower, but it shouldn't be a
> big deal.

Won't that cause problems for users of sites that use fail2ban to lock
people out after failed login attempts ? It might be convenient, but
would need a user option so it could be disabled.

    AndyM

Re: [RFC] automatically retrying network connections

[Lars Ingebrigtsen]

Andy Moreton <andrewjmoreton@gmail.com> writes:

> Won't that cause problems for users of sites that use fail2ban to lock
> people out after failed login attempts ? It might be convenient, but
> would need a user option so it could be disabled.

The disconnect/reconnect always happens before a password is presented,
I think, but I guess somebody could have a fail2ban rule that would ban
a host if it connects, negotiates TLS, and then shuts down the
connection without doing anything else.  Seems a bit odd, though.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [RFC] automatically retrying network connections

[Jimmy Yuen Ho Wong]

On Sun, Jul 22, 2018 at 2:32 PM, Andy Moreton <andrewjmoreton@gmail.com> wrote:
> On Sun 22 Jul 2018, Lars Ingebrigtsen wrote:
>
>> Robert Pluim <rpluim@gmail.com> writes:
>>
>>> one of the consequences of asking the user questions during network
>>> connection setup is that the server they're trying to connect to might
>>> have decided to close the connection by the time they've finished
>>> answering.
>>
>> Yes, the NSM should reconnect if the user says "go ahead" to the NSM
>> warning and the server has closed the connection.  The reason that
>> landed on the back burner is that servers seem to mostly have long
>> timeouts, so this turned out to be less of a problem in practice than I
>> expected.  (I.e., I don't recall seeing a bug report about this, which
>> is just plain weird.)
>>
>> We could also reverse the logic a bit and have the NSM always shut down
>> the connection before prompting.  This will make network connections
>> (that prompt an NSM warning) be somewhat slower, but it shouldn't be a
>> big deal.
>
> Won't that cause problems for users of sites that use fail2ban to lock
> people out after failed login attempts ? It might be convenient, but
> would need a user option so it could be disabled.
>

I think exponential backoff and a max-retry is in order.

Re: [RFC] automatically retrying network connections

[Robert Pluim]

Jimmy Yuen Ho Wong <wyuenho@gmail.com> writes:

>> Won't that cause problems for users of sites that use fail2ban to lock
>> people out after failed login attempts ? It might be convenient, but
>> would need a user option so it could be disabled.
>>
>
> I think exponential backoff and a max-retry is in order.

Iʼd argue for simplicity: try the connection, if it fails because too
much time was spent answering the nsm questions, retry once. That way
any true failure gets quickly communicated to the user.

Robert

Re: [RFC] automatically retrying network connections

[Jimmy Yuen Ho Wong]

On Mon, Jul 23, 2018 at 9:23 AM, Robert Pluim <rpluim@gmail.com> wrote:
> Jimmy Yuen Ho Wong <wyuenho@gmail.com> writes:
>
>>> Won't that cause problems for users of sites that use fail2ban to lock
>>> people out after failed login attempts ? It might be convenient, but
>>> would need a user option so it could be disabled.
>>>
>>
>> I think exponential backoff and a max-retry is in order.
>
> Iʼd argue for simplicity: try the connection, if it fails because too
> much time was spent answering the nsm questions, retry once. That way
> any true failure gets quickly communicated to the user.
>
> Robert

Sounds good to me too.

Re: [RFC] automatically retrying network connections

[Noam Postavsky]

On 22 July 2018 at 06:24, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> Robert Pluim <rpluim@gmail.com> writes:
>
>> one of the consequences of asking the user questions during network
>> connection setup is that the server they're trying to connect to might
>> have decided to close the connection by the time they've finished
>> answering.

> (I.e., I don't recall seeing a bug report about this, which
> is just plain weird.)

Maybe because NSM currently does very few checks by default, and so is
rarely triggered at all (i.e., the reason this whole thread started in
the first place)?

Re: [RFC] automatically retrying network connections

[Lars Ingebrigtsen]

Noam Postavsky <npostavs@gmail.com> writes:

> Maybe because NSM currently does very few checks by default, and so is
> rarely triggered at all (i.e., the reason this whole thread started in
> the first place)?

Well, it does certificate checks, which in practice will represent 99.7%
of all user interactions.  The TLS protocol checks check for things that
barely exist in real life.  (Except as test suites and at villainous
sites, and it's the latter we want to protect against.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: [RFC] automatically retrying network connections

[Jimmy Yuen Ho Wong]

On Sun, Jul 22, 2018 at 2:41 PM, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> Noam Postavsky <npostavs@gmail.com> writes:
>
>> Maybe because NSM currently does very few checks by default, and so is
>> rarely triggered at all (i.e., the reason this whole thread started in
>> the first place)?
>
> Well, it does certificate checks, which in practice will represent 99.7%
> of all user interactions.  The TLS protocol checks check for things that
> barely exist in real life.  (Except as test suites and at villainous
> sites, and it's the latter we want to protect against.)
>

Or that both of you have over-estimated people's willingness and
ability to report bugs to emacs :)