Emacs gnus (tls) IMAP connection problems with Google

Emacs gnus (tls) IMAP connection problems with Google

[Kostas Zorbadelos]

Hello all,

I use emacs gnus for all my mail handling mostly over IMAP.
A recent upgrade in my Arch system left me with a very strange
problem. My software versions:

GNU Emacs 26.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.30)
 of 2018-07-05

gnutls 3.6.5-4

My .gnus.el hasn't changed for a very long time, looking like this: 

(add-to-list 'gnus-secondary-select-methods '(nnimap "gmail"
                                  (nnimap-address "imap.gmail.com")
                                  (nnimap-server-port 993)
                                  (nnimap-stream ssl)
				  (nnimap-authinfo-file "~/.authinfo.gpg")))

Everything worked very well until I got the following errors connecting
to Google's IMAP service (could be that a recent gnutls upgrade is
responsible, older versions of emacs and gnutls I use in other Linux
systems have no such issue)  

*Messages*
Opening nnimap server on gmail...
Opening connection to imap.gmail.com via tls...
Unable to open server nnimap+gmail due to: Process *nnimap*<1> not running
Opening nnimap server on gmail...failed:

Sometimes however (rarely) the connections are established OK. 

Unfortunately, I could not debug this, I am at a loss. Trying to get
more info, I found a workaround by setting

(setq gnutls-log-level 2)

in .gnus.el. So, when gnutls debugging is turned on, seems the problem
is hiding (timing issue?). Since mail handling is a core activity for me
I would highly appreciate any help in debugging and solving the
issue. Let me know if and how I can provide any logs / further info. I
run a current version of Arch Linux (updated today).
If you think this list is not appropriate for the issue, feel free to
suggest a more proper place for the discussion.

Regards,
Kostas Zorbadelos

Re: Emacs gnus (tls) IMAP connection problems with Google

[Alex Kost]

Kostas Zorbadelos (2019-01-26 13:13 +0200) wrote:

> Hello all,

Hello,

> I use emacs gnus for all my mail handling mostly over IMAP.
> A recent upgrade in my Arch system left me with a very strange
> problem. My software versions:
>
> GNU Emacs 26.1 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.22.30)
>  of 2018-07-05
>
> gnutls 3.6.5-4
>
> My .gnus.el hasn't changed for a very long time, looking like this:
>
> (add-to-list 'gnus-secondary-select-methods '(nnimap "gmail"
>                                   (nnimap-address "imap.gmail.com")
>                                   (nnimap-server-port 993)
>                                   (nnimap-stream ssl)
> 				  (nnimap-authinfo-file "~/.authinfo.gpg")))
>
> Everything worked very well until I got the following errors connecting
> to Google's IMAP service (could be that a recent gnutls upgrade is
> responsible, older versions of emacs and gnutls I use in other Linux
> systems have no such issue)
>
> *Messages*
> Opening nnimap server on gmail...
> Opening connection to imap.gmail.com via tls...
> Unable to open server nnimap+gmail due to: Process *nnimap*<1> not running
> Opening nnimap server on gmail...failed:
>
> Sometimes however (rarely) the connections are established OK.

I think I have a similar problem: whenever I try to enter my gmail group
for the first time, I get the following error:

  gnus-select-newsgroup: Couldn’t request group nnimap+gmail:INBOX:

But if I enter it again immediately, it opens successfully.

Sorry, I don't have more information, it is just a "me too" message :-)

-- 
Alex

Re: Emacs gnus (tls) IMAP connection problems with Google

[Van L]

> But if I enter it again immediately, it opens successfully.

It's possible the first attempt fell in a black hole on the 'net.
The second attempt went a different circuitous route.

Re: Emacs gnus (tls) IMAP connection problems with Google

[Kostas Zorbadelos]

On Κυρ, Ιαν 27 2019 at 01:55:03 μμ, Van L <van@scratch.space> wrote:

>> But if I enter it again immediately, it opens successfully.
>
> It's possible the first attempt fell in a black hole on the 'net.
> The second attempt went a different circuitous route.
>

In my case, the problem is consistent. Hopefully someone could give a
clue on how to debug the issue and get more information.

-- 
Kostas Zorbadelos

Re: Emacs gnus (tls) IMAP connection problems with Google

[Van L]

>> It's possible the first attempt fell in a black hole on the 'net. 
>> The second attempt went a different circuitous route. 

> In my case, the problem is consistent. 
> Hopefully someone could give a clue on how to debug the issue and get more information.

The Taipei Times reported yesterday that the link between Fiji and Tonga has been cut the last five days. Tonga is isolated in the main other than satlinks for priority traffic, which does not include Facebook for holiday makers. Traffic won’t get through there, for example.

One way to sussout what’s going for you is to see the raw packet exchange and interpret that with capture tools. (Wireshark? Kali?)

Re: Emacs gnus (tls) IMAP connection problems with Google

[Grant Rettke]

On Sun, Jan 27, 2019 at 11:39 AM Kostas Zorbadelos <kzorba@nixly.net> wrote:
> In my case, the problem is consistent. Hopefully someone could give a

A few weeks ago I ran into a problem with TLS. The context was using
Org2Block. Org2Blog
runs in Emacs and talks to WordPress via XML-RPC over TLS.

Short story of my solution is that I had to configure GnuTLS to
not-use TLSv1.3. Here is the explanation:
https://github.com/org2blog/org2blog/wiki/Debugging-Stories#inexplicably-cant-login-or-do-anything
.

Here is the config, which you can only set up once, and now every tls
connection Emacs makes will use this config:

(setq gnutls-algorithm-priority "NORMAL:-VERS-TLS1.3")

The long version is how I figured it out:

https://github.com/org2blog/org2blog/wiki/Debugging#when-things-go-wrong-how-to-debug

Re: Emacs gnus (tls) IMAP connection problems with Google

[Emanuel Berg]

Grant Rettke wrote:

> A few weeks ago I ran into a problem with
> TLS. The context was using Org2Block.
> Org2Blog runs in Emacs and talks to WordPress
> via XML-RPC over TLS.

I don't know if it is related, but I get this
error

    gnutls.c: [0] (Emacs) fatal error: Error in the pull function.

very often with Gnus and TLS.

I then do C-g and try to get mail/news again,
and then it most often works. It seems to be
the previous connection that has somehow, for
some reason, failed. (Sometimes I do not get
the message, Emacs just stalls forever, and
this is also remedied with C-g.)

I reported it as a Gnus bug but apparently it
isn't in Gnus but somewhere else, which perhaps
the error message itself is indicative of.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Emacs gnus (tls) IMAP connection problems with Google

[Kostas Zorbadelos]

Thank you so much about this, works for me too. To be honest I got lazy
since I found the workaround and didn't trace using wireshark. One
question now is whether this is an issue with Google's IMAP tls 1.3
implementation or it's a gnutls issue.

Best regards,
Kostas

On Sun, Feb 10 2019 at 03:54:49 PM, Grant Rettke <grant@wisdomandwonder.com> wrote:

> On Sun, Jan 27, 2019 at 11:39 AM Kostas Zorbadelos <kzorba@nixly.net> wrote:
>> In my case, the problem is consistent. Hopefully someone could give a
>
> A few weeks ago I ran into a problem with TLS. The context was using
> Org2Block. Org2Blog
> runs in Emacs and talks to WordPress via XML-RPC over TLS.
>
> Short story of my solution is that I had to configure GnuTLS to
> not-use TLSv1.3. Here is the explanation:
> https://github.com/org2blog/org2blog/wiki/Debugging-Stories#inexplicably-cant-login-or-do-anything
> .
>
> Here is the config, which you can only set up once, and now every tls
> connection Emacs makes will use this config:
>
> (setq gnutls-algorithm-priority "NORMAL:-VERS-TLS1.3")
>
> The long version is how I figured it out:
>
> https://github.com/org2blog/org2blog/wiki/Debugging#when-things-go-wrong-how-to-debug

-- 
Kostas Zorbadelos

Re: Emacs gnus (tls) IMAP connection problems with Google

[Robert Pluim]

Kostas Zorbadelos <kzorba@nixly.net> writes:

> Thank you so much about this, works for me too. To be honest I got lazy
> since I found the workaround and didn't trace using wireshark. One
> question now is whether this is an issue with Google's IMAP tls 1.3
> implementation or it's a gnutls issue.

I think Google's TLS 1.3 implementation is fine. The way emacs 26.1
uses the GnuTLS api is slightly wrong for tls1.3. It looks like Paul
Eggert has already fixed this for the unreleased emacs-27.1, maybe
that fix can be applied to emacs-26 as well.

Robert

Re: Emacs gnus (tls) IMAP connection problems with Google

[Emanuel Berg]

> I don't know if it is related, but I get this
> error
>
>     gnutls.c: [0] (Emacs) fatal error: Error in the pull function.
>
> very often with Gnus and TLS.
>
> I then do C-g and try to get mail/news again,
> and then it most often works. It seems to be
> the previous connection that has somehow, for
> some reason, failed. (Sometimes I do not get
> the message, Emacs just stalls forever, and
> this is also remedied with C-g.)
>
> I reported it as a Gnus bug but apparently it
> isn't in Gnus but somewhere else, which perhaps
> the error message itself is indicative of.

I think I has been able to solve this issue
after several years of getting that error.
The solution:

    (require 'tls)

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Emacs gnus (tls) IMAP connection problems with Google

[Emanuel Berg]

> I think I has been able to solve this issue
> after several years of getting that error.
> The solution:
>
>     (require 'tls)

I spoke to soon. I still get the error message.

The error this solved was the one that didn't
come with a message, the one when the
`gnus-group-get-new-news' would stall
indefinitely and one would have to do C-g
(somtimes twice, or even a couple of times)
before one try finally went thru.

On the positive side, that error was ten times
as frequent and annoying, so I suppose one
shouldn't complain.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Emacs gnus (tls) IMAP connection problems with Google

[Emanuel Berg]

>> I think I has been able to solve this issue
>> after several years of getting that error.
>> The solution:
>>
>>     (require 'tls)
>
> I spoke to soon. I still get the
> error message.
>
> The error this solved was the one that didn't
> come with a message, the one when the
> `gnus-group-get-new-news' would stall
> indefinitely and one would have to do C-g
> (somtimes twice, or even a couple of times)
> before one try finally went thru.
>
> On the positive side, that error was ten
> times as frequent and annoying, so I suppose
> one shouldn't complain.

OK, after working great yesterday (yesternight)
when I today tried to get my mail it wouldn't
work *at all*! "Authentication failed"!
This has never happened before! As has been
evident from square one, I have absolutely no
clue what's going on.

After messing around with the files I got it to
work again, *but* in the old state where I get
to "get mail" repeatedly before it works, and
I do get the old error message [1] now
and then.

All of this is probably of no value to anyone
because I have an old system and an old Emacs.
I should upgrade it all but I only have
everything on a single SD card, 10+ years of
work on my very own all but complete system
(WRT to the interface and UX), and I don't dare
upgrade anything before I clone the system,
which I can't as I have only one computer,
which, of course, runs the very system!

Well, hell, I'll solve it someday, God willing.

As for a new Emacs, I did manually install it
from source (if "manually" is the correct word
since it is super-automated and requires no
skills whatsoever, compliments to the chef) but
all my 100+ Elisp files were a complete mess on
the new Emacs, and in particular, I couldn't
get Emacs-w3m to work.

Anyway I look forward to solving all those
issues one day, God willing.

PS. If this Manny's Computer News broadcast
    wasn't interesting for you to watch, then
    why didn't you simply switch off your
    terminal emulator to do some push-ups
    instead? DS.


[1] gnutls.c: [0] (Emacs) fatal error: Error in the pull function.

-- 
underground experts united
http://user.it.uu.se/~embe8573