bug#37221: 27.0.50; gnus fails to open connection post-NSM update

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Alex Branham]

Hi -

After the recent NSM update gnus fails to open my mail with this message:

Warning: Opening nnimap server on LocalMail...failed: ; Unable to open server nnimap+LocalMail due to: Buffer  *nnimap localhost nil  *nntpd** has no process

My setup is that I use mbsync/isync to download the mail and run dovecot
read the maildir and host it as an imap server. Since it's local it's
not encrypted. Setting network-security-protocol-checks to nil does not
change anything. I've only recently started using dovecot/gnus so it's
possible I've done something silly but it was working before the NSM
update (at commit ef8531d262081d91ecf2a4f349bc63a0fede90d4) and isn't
working as of 4b87169d113a151e5d9d6cf7b0d7cb4fb1d3d2d7.

Thanks,
Alex

In GNU Emacs 27.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.10)
Windowing system distributor 'The X.Org Foundation', version 11.0.12004000
System Description: NixOS 19.03.173394.147bd882fc6 (Koi)

Recent messages:
Warning: Opening nnimap server on LocalMail...failed: ; Unable to open server nnimap+LocalMail due to: Buffer  *nnimap localhost nil  *nntpd** has no process
Saving file /home/alex/Sync/emacs/gnus/.newsrc-dribble...
Wrote /home/alex/Sync/emacs/gnus/.newsrc-dribble
(Saved .newsrc-dribble)
Mark set
Are you sure you want to quit reading news? (y or n) y
Saving /home/alex/Sync/emacs/gnus/.newsrc.eld...
Saving file /home/alex/Sync/emacs/gnus/.newsrc.eld...
Wrote /home/alex/Sync/emacs/gnus/.newsrc.eld
Saving /home/alex/Sync/emacs/gnus/.newsrc.eld...done

Configured using:
 'configure
 --prefix=/nix/store/7rfv0dv583css6s7drg71bj4p3gv77dj-emacs-27.0.50
 --disable-build-details --with-modules --with-x-toolkit=gtk3 --with-xft
 CFLAGS=-DMAC_OS_X_VERSION_MAX_ALLOWED=101200'

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND DBUS GSETTINGS GLIB NOTIFY INOTIFY
LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON
PDUMPER GMP

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Memory information:
((conses 16 436220 242939)
 (symbols 48 42227 20)
 (strings 32 127301 28815)
 (string-bytes 1 4640702)
 (vectors 16 72982)
 (vector-slots 8 1109088 233166)
 (floats 8 581 295)
 (intervals 56 651 192)
 (buffers 992 24))

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Robert Pluim]

>>>>> On Thu, 29 Aug 2019 11:56:27 -0500, Alex Branham <alex.branham@gmail.com> said:

    Alex> Hi -
    Alex> After the recent NSM update gnus fails to open my mail with this message:

    Alex> Warning: Opening nnimap server on LocalMail...failed: ; Unable to open server nnimap+LocalMail due to: Buffer  *nnimap localhost nil  *nntpd** has no process

    Alex> My setup is that I use mbsync/isync to download the mail and run dovecot
    Alex> read the maildir and host it as an imap server. Since it's local it's
    Alex> not encrypted. Setting network-security-protocol-checks to nil does not
    Alex> change anything. I've only recently started using dovecot/gnus so it's
    Alex> possible I've done something silly but it was working before the NSM
    Alex> update (at commit ef8531d262081d91ecf2a4f349bc63a0fede90d4) and isn't
    Alex> working as of 4b87169d113a151e5d9d6cf7b0d7cb4fb1d3d2d7.

Can you show us your Gnus configuration? Also, what is your
'network-security-level' set to?

Robert

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Alex Branham]

On Mon 02 Sep 2019 at 11:33, Robert Pluim <rpluim@gmail.com> wrote:

>>>>>> On Thu, 29 Aug 2019 11:56:27 -0500, Alex Branham <alex.branham@gmail.com> said:
>
>     Alex> Hi -
>     Alex> After the recent NSM update gnus fails to open my mail with this message:
>
>     Alex> Warning: Opening nnimap server on LocalMail...failed: ;
>     Alex> Unable to open server nnimap+LocalMail due to: Buffer
>     Alex> *nnimap localhost nil *nntpd** has no process
>
>     Alex> My setup is that I use mbsync/isync to download the mail and run dovecot
>     Alex> read the maildir and host it as an imap server. Since it's local it's
>     Alex> not encrypted. Setting network-security-protocol-checks to nil does not
>     Alex> change anything. I've only recently started using dovecot/gnus so it's
>     Alex> possible I've done something silly but it was working before the NSM
>     Alex> update (at commit ef8531d262081d91ecf2a4f349bc63a0fede90d4) and isn't
>     Alex> working as of 4b87169d113a151e5d9d6cf7b0d7cb4fb1d3d2d7.
>
> Can you show us your Gnus configuration? Also, what is your
> 'network-security-level' set to?

Sure thing, it's below. network-security-level is medium. If I do (setq
nsm-trust-local-network t) then all seems to work as expected.

Thanks,
Alex


(use-package gnus
  :defer
  :custom
  (gnus-home-directory (file-name-directory (expand-file-name "~/Sync/emacs/gnus/")))
  (gnus-save-killed-list nil)
  (gnus-save-newsrc-file nil)
  (gnus-group-line-format "%M%S%L%P     %y:%B%G\n")
  (gnus-thread-sort-functions '(gnus-thread-sort-by-most-recent-number))
  (gnus-summary-line-format "%U%R%z%&user-date; %-15,15f: %B%s%)\n")
  (gnus-user-date-format-alist '((t . "%Y-%m-%d %H:%M")))
  (gnus-sum-thread-tree-false-root "")
  (gnus-sum-thread-tree-indent " ")
  (gnus-sum-thread-tree-leaf-with-other "├► ")
  (gnus-sum-thread-tree-root "")
  (gnus-sum-thread-tree-single-leaf "╰► ")
  (gnus-sum-thread-tree-vertical "│")
  :init
  (setq gnus-select-method '(nnnil "")
        gnus-secondary-select-methods '((nnimap "LocalMail"
                                                (nnimap-address "localhost")
	                                        (nnimap-stream network))
                                        (nntp "gnus.user"
                                              (nntp-address "news.gmane.org"))))
  (defun my/email-update ()
    (make-process
     :name "mbsync" :buffer nil
     :command '("mbsync" "-a")
     :noquery t))
  (when (executable-find "mbsync")
    (run-with-timer 10 60 #'my/email-update))
  :config
  (setq gnus-parameters
        '((".*"
           (name "Alex Branham")
           (posting-style
            (address "alex.branham@gmail.com")
            (x-message-smtp-method "smtp smtp.gmail.com 587 alex.branham@gmail.com")))
          ("INBOX"
           (display . all))
           ;; <here's where my work email is configured>
          ("gmane"
           (auto-expire . t))))
  ;; can't be set in gnus-parameters:
  (setq gnus-permanently-visible-groups "INBOX")
  (setq gnus-treat-from-gravatar 'head)
  ;; discourage html
  (setq mm-discouraged-alternatives '("text/html" "text/richtext"))
  (setq gnus-message-archive-group nil)
  (use-package gnus-topic
    :hook (gnus-group-mode . gnus-topic-mode)))

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Robert Pluim]

>>>>> On Tue, 03 Sep 2019 13:26:27 -0500, Alex Branham <alex.branham@gmail.com> said:
    >> Can you show us your Gnus configuration? Also, what is your
    >> 'network-security-level' set to?

    Alex> Sure thing, it's below. network-security-level is medium. If I do (setq
    Alex> nsm-trust-local-network t) then all seems to work as expected.

Oh, thatʼs interesting. Youʼre using (nnimap-stream network), which
will attempt STARTTLS with localhost, unless nsm-trust-local-network
is t. (nnimap-stream plain) will turn off the attempted STARTTLS
upgrade.

Could you try with nsm-trust-local-network nil, and
nnimap-record-commands t ? That should produce an "*imap log*" buffer
that might tell us more, perhaps with (setq debug-on-error t) for good
measure.

Iʼll admit Iʼm at a loss as to why the recent nsm changes would cause
this to start failing. Iʼm assuming thereʼs nothing interesting in
"*Messages*", and youʼre not prompted for anything?

Robert

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Alex Branham]

On Tue 03 Sep 2019 at 22:16, Robert Pluim <rpluim@gmail.com> wrote:

>     Alex> Sure thing, it's below. network-security-level is medium. If I do (setq
>     Alex> nsm-trust-local-network t) then all seems to work as expected.
>
> Oh, thatʼs interesting. Youʼre using (nnimap-stream network), which
> will attempt STARTTLS with localhost, unless nsm-trust-local-network
> is t. (nnimap-stream plain) will turn off the attempted STARTTLS
> upgrade.

I tried chancing it to plain and get the same error with the same message.

> Could you try with nsm-trust-local-network nil, and
> nnimap-record-commands t ? That should produce an "*imap log*" buffer
> that might tell us more, perhaps with (setq debug-on-error t) for good
> measure.

When I tried this there was not an *imap log* buffer. When I set
nsm-trust-local-network to t there's an *imap log* buffer, though that's
not as useful I suppose. But at least the logging is working?

> Iʼll admit Iʼm at a loss as to why the recent nsm changes would cause
> this to start failing. Iʼm assuming thereʼs nothing interesting in
> "*Messages*", and youʼre not prompted for anything?

Correct, other than the message I originally reported (Warning: Opening
nnimap server on LocalMail...failed: ; Unable to open server
nnimap+LocalMail due to: Buffer *nnimap localhost nil *nntpd** has no
process)

Thanks,
Alex

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Lars Ingebrigtsen]

Alex Branham <alex.branham@gmail.com> writes:

>> Could you try with nsm-trust-local-network nil, and
>> nnimap-record-commands t ? That should produce an "*imap log*" buffer
>> that might tell us more, perhaps with (setq debug-on-error t) for good
>> measure.
>
> When I tried this there was not an *imap log* buffer. When I set
> nsm-trust-local-network to t there's an *imap log* buffer, though that's
> not as useful I suppose. But at least the logging is working?

The setting of nsm-trust-local-network should not affect whether the
*imap log* buffer is created or not, so there's something odd going on
in your Emacs, I think.

>> Iʼll admit Iʼm at a loss as to why the recent nsm changes would cause
>> this to start failing. Iʼm assuming thereʼs nothing interesting in
>> "*Messages*", and youʼre not prompted for anything?
>
> Correct, other than the message I originally reported (Warning: Opening
> nnimap server on LocalMail...failed: ; Unable to open server
> nnimap+LocalMail due to: Buffer *nnimap localhost nil *nntpd** has no
> process)

I can't see any reason why the NSM changes should affect your local
connection to your IMAP server, either.

Does your setup work if you revert Emacs back to before the NSM changes?
And what is the contents of your *imap log* buffer?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Alex Branham]

On Wed 04 Sep 2019 at 15:37, Lars Ingebrigtsen <larsi@gnus.org> wrote:

> Alex Branham <alex.branham@gmail.com> writes:
>
>>> Could you try with nsm-trust-local-network nil, and
>>> nnimap-record-commands t ? That should produce an "*imap log*" buffer
>>> that might tell us more, perhaps with (setq debug-on-error t) for good
>>> measure.
>>
>> When I tried this there was not an *imap log* buffer. When I set
>> nsm-trust-local-network to t there's an *imap log* buffer, though that's
>> not as useful I suppose. But at least the logging is working?
>
> The setting of nsm-trust-local-network should not affect whether the
> *imap log* buffer is created or not, so there's something odd going on
> in your Emacs, I think.

I'm not at all familiar with this part of Emacs, but could it be the
case that the NSM is somehow killing the nnimap process before it has a
chance to do anything, including logging?

> Does your setup work if you revert Emacs back to before the NSM changes?
> And what is the contents of your *imap log* buffer?

Yes, it does. After reverting to Emacs built August 18th, here's the *imap log* buffer:

08:56:15 [localhost] (inhibited)
08:56:15 [localhost] 26 CAPABILITY.
08:56:15 [localhost] 27 ENABLE QRESYNC.
08:56:15 [localhost] 28 LIST "" "*".
08:56:19 [localhost] 29 EXAMINE "utexas/[Gmail]/.Sent Mail" (QRESYNC (1565742123 3)).
08:56:19 [localhost] 30 EXAMINE "utexas/[Gmail]/.Drafts" (QRESYNC (1565742121 2)).
08:56:19 [localhost] 31 EXAMINE "utexas/[Gmail]/.All Mail" (QRESYNC (1565742120 16)).
08:56:19 [localhost] 32 EXAMINE "gmail/[Gmail]/.Sent Mail" (QRESYNC (1565742118 21)).
08:56:19 [localhost] 33 SELECT "gmail/[Gmail]/.Drafts".
08:56:19 [localhost] 34 UID FETCH 1:* FLAGS.
08:56:19 [localhost] 35 EXAMINE "gmail/[Gmail]/.All Mail" (QRESYNC (1565742116 48)).
08:56:19 [localhost] 36 EXAMINE "utexas/[Gmail]/.Trash" (QRESYNC (1565742122 1)).
08:56:19 [localhost] 37 SELECT "INBOX".
08:56:19 [localhost] 38 UID FETCH 1:* FLAGS.
08:56:19 [localhost] 39 EXAMINE "gmail/emacs-orgmode" (QRESYNC (1565742113 2)).
08:56:19 [localhost] 40 EXAMINE "gmail/emacs-devel" (QRESYNC (1565742114 2)).
08:56:19 [localhost] 41 EXAMINE "gmail/[Gmail]/.Trash" (QRESYNC (1565742117 1)).
08:56:19 [localhost] 42 EXAMINE "utexas/INBOX" (QRESYNC (1565742119 36)).
08:56:19 [localhost] 43 EXAMINE "gmail/INBOX" (QRESYNC (1565742115 101)).

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Lars Ingebrigtsen]

Alex Branham <alex.branham@gmail.com> writes:

> I'm not at all familiar with this part of Emacs, but could it be the
> case that the NSM is somehow killing the nnimap process before it has a
> chance to do anything, including logging?

No, the NSM just pups up a warning (and will then kill the connection if
you tell it to).

There were other changes in the patch set that could possibly affect
something, though.

>> Does your setup work if you revert Emacs back to before the NSM changes?
>> And what is the contents of your *imap log* buffer?
>
> Yes, it does. After reverting to Emacs built August 18th, here's the
> *imap log* buffer:

But what does the *imap log* buffer look like in the current trunk?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Alex Branham]

On Wed 04 Sep 2019 at 16:16, Lars Ingebrigtsen <larsi@gnus.org> wrote:

> Alex Branham <alex.branham@gmail.com> writes:
>
>> I'm not at all familiar with this part of Emacs, but could it be the
>> case that the NSM is somehow killing the nnimap process before it has a
>> chance to do anything, including logging?
>
> No, the NSM just pups up a warning (and will then kill the connection if
> you tell it to).
>
> There were other changes in the patch set that could possibly affect
> something, though.
>
>>> Does your setup work if you revert Emacs back to before the NSM changes?
>>> And what is the contents of your *imap log* buffer?
>>
>> Yes, it does. After reverting to Emacs built August 18th, here's the
>> *imap log* buffer:
>
> But what does the *imap log* buffer look like in the current trunk?

Not quite "current" trunk (commit
42ba6200af10c00c72ac13912d6fb42a7af88058 from August 26th), but with
nsm-trust-local-network set to nil (the default), the *imap log* buffer
doesn't get created. If I set it to t, then *imap log* looks like this:

09:21:46 [localhost] (inhibited)
09:21:46 [localhost] 501 CAPABILITY.
09:21:46 [localhost] 502 ENABLE QRESYNC.
09:21:46 [localhost] 503 LIST "" "*".
09:21:49 [localhost] 504 EXAMINE "utexas/[Gmail]/.Sent Mail" (QRESYNC (1565742123 3)).
09:21:49 [localhost] 505 EXAMINE "utexas/[Gmail]/.Drafts" (QRESYNC (1565742121 2)).
09:21:49 [localhost] 506 EXAMINE "utexas/[Gmail]/.All Mail" (QRESYNC (1565742120 17)).
09:21:49 [localhost] 507 EXAMINE "gmail/[Gmail]/.Sent Mail" (QRESYNC (1565742118 23)).
09:21:49 [localhost] 508 SELECT "gmail/[Gmail]/.Drafts".
09:21:49 [localhost] 509 UID FETCH 1:* FLAGS.
09:21:49 [localhost] 510 EXAMINE "gmail/[Gmail]/.All Mail" (QRESYNC (1565742116 52)).
09:21:49 [localhost] 511 EXAMINE "utexas/[Gmail]/.Trash" (QRESYNC (1565742122 1)).
09:21:49 [localhost] 512 SELECT "INBOX".
09:21:49 [localhost] 513 UID FETCH 1:* FLAGS.
09:21:49 [localhost] 514 EXAMINE "gmail/emacs-orgmode" (QRESYNC (1565742113 2)).
09:21:49 [localhost] 515 EXAMINE "gmail/emacs-devel" (QRESYNC (1565742114 2)).
09:21:49 [localhost] 516 EXAMINE "gmail/[Gmail]/.Trash" (QRESYNC (1565742117 1)).
09:21:49 [localhost] 517 EXAMINE "utexas/INBOX" (QRESYNC (1565742119 39)).
09:21:49 [localhost] 518 EXAMINE "gmail/INBOX" (QRESYNC (1565742115 107)).

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Lars Ingebrigtsen]

Alex Branham <alex.branham@gmail.com> writes:

> Not quite "current" trunk (commit
> 42ba6200af10c00c72ac13912d6fb42a7af88058 from August 26th), but with
> nsm-trust-local-network set to nil (the default), the *imap log* buffer
> doesn't get created. If I set it to t, then *imap log* looks like this:

Oh, I see...  When nsm-trust-local-network is nil, then we don't even
get as far as starting the IMAP session.

Are there any messages in *Messages* that could indicate where it's
bugging out?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Alex Branham]

On Wed 04 Sep 2019 at 16:35, Lars Ingebrigtsen <larsi@gnus.org> wrote:

> Are there any messages in *Messages* that could indicate where it's
> bugging out?

No, nothing other than the original report:

Warning: Opening nnimap server on LocalMail...failed: ; Unable to open
server nnimap+LocalMail due to: Buffer  *nnimap localhost nil  *nntpd**
has no process

Thanks,
Alex

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Robert Pluim]

>>>>> On Wed, 04 Sep 2019 09:00:39 -0500, Alex Branham <alex.branham@gmail.com> said:
    >> The setting of nsm-trust-local-network should not affect whether the
    >> *imap log* buffer is created or not, so there's something odd going on
    >> in your Emacs, I think.

    Alex> I'm not at all familiar with this part of Emacs, but could it be the
    Alex> case that the NSM is somehow killing the nnimap process before it has a
    Alex> chance to do anything, including logging?

Yes, thatʼs what it looks like

    >> Does your setup work if you revert Emacs back to before the NSM changes?
    >> And what is the contents of your *imap log* buffer?

    Alex> Yes, it does. After reverting to Emacs built August 18th, here's the *imap log* buffer:

    Alex> 08:56:15 [localhost] (inhibited)

Iʼve spotted one possible problem: nsm-query runs even when the
connection is unencrypted, which ends up doing (insert nil), which
aborts the connection. Does the following help? (Lars, why is nsm
prompting for unencrypted connections?)

diff --git a/lisp/net/nsm.el b/lisp/net/nsm.el
index 5e8381075b..7243ae8822 100644
--- a/lisp/net/nsm.el
+++ b/lisp/net/nsm.el
@@ -788,7 +788,8 @@ nsm-check-plain-connection
 (defun nsm-query (host port status what problems message)
   ;; If there is no user to answer queries, then say `no' to everything.
   (if (or noninteractive
-	  nsm-noninteractive)
+	  nsm-noninteractive
+          (not status))
       nil
     (let ((response
 	   (condition-case nil

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> Iʼve spotted one possible problem: nsm-query runs even when the
> connection is unencrypted, which ends up doing (insert nil), which
> aborts the connection. Does the following help? (Lars, why is nsm
> prompting for unencrypted connections?)

It runs for all kinds of connections -- a non-encrypted one could, for
instance, be one that's a result of a downgrade attack.

So if it now assumes that the connection is always encrypted, that's a
new bug.  Hm...  anybody know of a server that absolutely not doesn't
have STARTTLS that I can test with?  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Alex Branham]

On Wed 04 Sep 2019 at 16:56, Robert Pluim <rpluim@gmail.com> wrote:

> Iʼve spotted one possible problem: nsm-query runs even when the
> connection is unencrypted, which ends up doing (insert nil), which
> aborts the connection. Does the following help? (Lars, why is nsm
> prompting for unencrypted connections?)
>
> diff --git a/lisp/net/nsm.el b/lisp/net/nsm.el
> index 5e8381075b..7243ae8822 100644
> --- a/lisp/net/nsm.el
> +++ b/lisp/net/nsm.el
> @@ -788,7 +788,8 @@ nsm-check-plain-connection
>  (defun nsm-query (host port status what problems message)
>    ;; If there is no user to answer queries, then say `no' to everything.
>    (if (or noninteractive
> -	  nsm-noninteractive)
> +	  nsm-noninteractive
> +          (not status))
>        nil
>      (let ((response
>  	   (condition-case nil

nsm-query gets skipped now but the process is still killed (same
message, etc).

Thanks,
Alex

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Robert Pluim]

>>>>> On Wed, 04 Sep 2019 16:59:35 +0200, Lars Ingebrigtsen <larsi@gnus.org> said:

    Lars> Robert Pluim <rpluim@gmail.com> writes:
    >> Iʼve spotted one possible problem: nsm-query runs even when the
    >> connection is unencrypted, which ends up doing (insert nil), which
    >> aborts the connection. Does the following help? (Lars, why is nsm
    >> prompting for unencrypted connections?)

    Lars> It runs for all kinds of connections -- a non-encrypted one could, for
    Lars> instance, be one that's a result of a downgrade attack.

    Lars> So if it now assumes that the connection is always encrypted, that's a
    Lars> new bug.  Hm...  anybody know of a server that absolutely not doesn't
    Lars> have STARTTLS that I can test with?  :-)

Not quite. nsm-query-user assumes that status is non-nil, which is not
the case in my test, which was imap to localhost. It never gets the
chance to emit STARTTLS, since nsm has already killed the connection:

(defun nsm-query-user (message status)
  (let ((buffer (get-buffer-create "*Network Security Manager*"))
        (cert-buffer (get-buffer-create "*Certificate Details*"))
        (certs (plist-get status :certificates)))
    (save-window-excursion
      ;; First format the certificate and warnings.
      (with-current-buffer-window
       buffer nil nil
       (insert (nsm-format-certificate status)) <= status is nil here,
  which gives us "(wrong-type-argument char-or-string-p nil)", which
  causes nsm-query to kill the connection.

Robert

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Robert Pluim]

>>>>> On Wed, 04 Sep 2019 10:04:11 -0500, Alex Branham <alex.branham@gmail.com> said:

    Alex> On Wed 04 Sep 2019 at 16:56, Robert Pluim <rpluim@gmail.com> wrote:
    >> Iʼve spotted one possible problem: nsm-query runs even when the
    >> connection is unencrypted, which ends up doing (insert nil), which
    >> aborts the connection. Does the following help? (Lars, why is nsm
    >> prompting for unencrypted connections?)
    >> 
    >> diff --git a/lisp/net/nsm.el b/lisp/net/nsm.el
    >> index 5e8381075b..7243ae8822 100644
    >> --- a/lisp/net/nsm.el
    >> +++ b/lisp/net/nsm.el
    >> @@ -788,7 +788,8 @@ nsm-check-plain-connection
    >> (defun nsm-query (host port status what problems message)
    >> ;; If there is no user to answer queries, then say `no' to everything.
    >> (if (or noninteractive
    >> -	  nsm-noninteractive)
    >> +	  nsm-noninteractive
    >> +          (not status))
    >> nil
    >> (let ((response
    >> (condition-case nil

    Alex> nsm-query gets skipped now but the process is still killed (same
    Alex> message, etc).

Oh right, because now nsm-save-host doesnʼt get called. This should
work instead of the previous patch (but is not the right
solution, I think). You should get prompted by nsm now.

diff --git a/lisp/net/nsm.el b/lisp/net/nsm.el
index 5e8381075b..8750c19267 100644
--- a/lisp/net/nsm.el
+++ b/lisp/net/nsm.el
@@ -823,7 +823,7 @@ nsm-query-user
       ;; First format the certificate and warnings.
       (with-current-buffer-window
        buffer nil nil
-       (insert (nsm-format-certificate status))
+       (when status (insert (nsm-format-certificate status)))
        (insert message)
        (goto-char (point-min))
        ;; Fill the first line of the message, which usually

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Alex Branham]

On Wed 04 Sep 2019 at 17:25, Robert Pluim <rpluim@gmail.com> wrote:

> Oh right, because now nsm-save-host doesnʼt get called. This should
> work instead of the previous patch (but is not the right
> solution, I think). You should get prompted by nsm now.

Yes, now I get prompted about the unencrypted connection and it works.

Thanks,
Alex

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Robert Pluim]

>>>>> On Wed, 04 Sep 2019 10:55:46 -0500, Alex Branham <alex.branham@gmail.com> said:

    Alex> On Wed 04 Sep 2019 at 17:25, Robert Pluim <rpluim@gmail.com> wrote:
    >> Oh right, because now nsm-save-host doesnʼt get called. This should
    >> work instead of the previous patch (but is not the right
    >> solution, I think). You should get prompted by nsm now.

    Alex> Yes, now I get prompted about the unencrypted connection and it works.

Thanks for testing. Lars, I donʼt think nsm should be prompting for a
never-seen-before unencrypted connection, only for a previously
encrypted downgraded connection?

Robert

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> Thanks for testing. Lars, I donʼt think nsm should be prompting for a
> never-seen-before unencrypted connection, only for a previously
> encrypted downgraded connection?

That depends on the settings, I think?  In `paranoid' it probably
should, and if it's an IMAP connection, it definitely should even on
`medium'.

Or is taking care of not sending passwords over non-encrypted
connections done somewhere else now?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Robert Pluim]

>>>>> On Wed, 04 Sep 2019 19:06:24 +0200, Lars Ingebrigtsen <larsi@gnus.org> said:

    Lars> Robert Pluim <rpluim@gmail.com> writes:
    >> Thanks for testing. Lars, I donʼt think nsm should be prompting for a
    >> never-seen-before unencrypted connection, only for a previously
    >> encrypted downgraded connection?

    Lars> That depends on the settings, I think?  In `paranoid' it probably
    Lars> should, and if it's an IMAP connection, it definitely should even on
    Lars> `medium'.

Even when the user has said '(nnimap-stream plain)'? Thatʼs a very
explicit request not to do STARTTLS.

    Lars> Or is taking care of not sending passwords over non-encrypted
    Lars> connections done somewhere else now?

Not that I know of.

Robert

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> Even when the user has said '(nnimap-stream plain)'? Thatʼs a very
> explicit request not to do STARTTLS.

Let's see...

	      (open-network-stream
	       "*nnimap*" (current-buffer) nnimap-address
	       (nnimap-map-port (car ports))
	       :type nnimap-stream
	       :warn-unless-encrypted t

So it should always warn, no matter what the type is.  Here's the call
back to the NSM:

(defun network-stream-open-plain (name buffer host service parameters)

[...]

    (when (plist-get parameters :warn-unless-encrypted)
      (setq stream (nsm-verify-connection stream host service nil t)))

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Robert Pluim]

>>>>> On Wed, 04 Sep 2019 19:24:28 +0200, Lars Ingebrigtsen <larsi@gnus.org> said:

    Lars> Robert Pluim <rpluim@gmail.com> writes:
    >> Even when the user has said '(nnimap-stream plain)'? Thatʼs a very
    >> explicit request not to do STARTTLS.

    Lars> Let's see...

    Lars> 	      (open-network-stream
    Lars> 	       "*nnimap*" (current-buffer) nnimap-address
    Lars> 	       (nnimap-map-port (car ports))
    Lars> 	       :type nnimap-stream
    Lars> 	       :warn-unless-encrypted t

    Lars> So it should always warn, no matter what the type is.  Here's the call
    Lars> back to the NSM:

    Lars> (defun network-stream-open-plain (name buffer host service parameters)

    Lars> [...]

    Lars>     (when (plist-get parameters :warn-unless-encrypted)
    Lars>       (setq stream (nsm-verify-connection stream host service nil t)))

OK. In that case perhaps I should push the following:

From 2972228d4761a1bf5682dfd361c5f916704c9222 Mon Sep 17 00:00:00 2001
From: Robert Pluim <rpluim@gmail.com>
Date: Thu, 31 Jan 2019 14:25:32 +0100
Subject: [PATCH] Make sure nntp connections always use TLS
To: emacs-devel@gnu.org

---
 lisp/gnus/nntp.el | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lisp/gnus/nntp.el b/lisp/gnus/nntp.el
index 49aa6ab144..09512488fb 100644
--- a/lisp/gnus/nntp.el
+++ b/lisp/gnus/nntp.el
@@ -1263,6 +1263,7 @@ nntp-open-connection
 		     :type (cadr (assoc nntp-open-connection-function map))
 		     :end-of-command "^\\([2345]\\|[.]\\).*\n"
 		     :capability-command "HELP\r\n"
+                     :warn-unless-encrypted t
 		     :success "^3"
 		     :starttls-function
 		     (lambda (capabilities)
-- 
2.23.0

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> OK. In that case perhaps I should push the following:
>
>>From 2972228d4761a1bf5682dfd361c5f916704c9222 Mon Sep 17 00:00:00 2001
> From: Robert Pluim <rpluim@gmail.com>
> Date: Thu, 31 Jan 2019 14:25:32 +0100
> Subject: [PATCH] Make sure nntp connections always use TLS

[...]

>  		     :type (cadr (assoc nntp-open-connection-function map))
>  		     :end-of-command "^\\([2345]\\|[.]\\).*\n"
>  		     :capability-command "HELP\r\n"
> +                     :warn-unless-encrypted t

No, nntp connections are commonly without passwords, so warning about
them being non-encrypted usually isn't helpful.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Robert Pluim]

>>>>> On Wed, 04 Sep 2019 21:37:04 +0200, Lars Ingebrigtsen <larsi@gnus.org> said:

    Lars> Robert Pluim <rpluim@gmail.com> writes:
    >> OK. In that case perhaps I should push the following:
    >> 
    >>> From 2972228d4761a1bf5682dfd361c5f916704c9222 Mon Sep 17 00:00:00 2001
    >> From: Robert Pluim <rpluim@gmail.com>
    >> Date: Thu, 31 Jan 2019 14:25:32 +0100
    >> Subject: [PATCH] Make sure nntp connections always use TLS

    Lars> [...]

    >> :type (cadr (assoc nntp-open-connection-function map))
    >> :end-of-command "^\\([2345]\\|[.]\\).*\n"
    >> :capability-command "HELP\r\n"
    >> +                     :warn-unless-encrypted t

    Lars> No, nntp connections are commonly without passwords, so warning about
    Lars> them being non-encrypted usually isn't helpful.

Speak for yourself :-) Iʼll keep it locally.

I guess we should just add the check for status being nil in
nsm-query-user then.

Robert

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> I guess we should just add the check for status being nil in
> nsm-query-user then.

I think so.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#37221: 27.0.50; gnus fails to open connection post-NSM update

[Robert Pluim]

tags 37221 fixed
close 37221 27.1
quit

>>>>> On Wed, 04 Sep 2019 22:41:49 +0200, Lars Ingebrigtsen <larsi@gnus.org> said:

    Lars> Robert Pluim <rpluim@gmail.com> writes:
    >> I guess we should just add the check for status being nil in
    >> nsm-query-user then.

    Lars> I think so.

Done. Closing.
Committed as fc9ed61a43

Robert