bug#35443: 27.0.50; Gnus (nnimap) shows "ghost" messages in summary buffer

[Eric Abrahamsen <eric@ericabrahamsen.net>]

Ulrich Mueller <ulm@gentoo.org> writes:

> In GNU Emacs 27.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit)
>  of 2019-04-21 built on a1i15
> Windowing system distributor 'The X.Org Foundation', version 11.0.11905000
> System Description: Gentoo/Linux
>
> When fetching e-mail from a dovecot-2.3.5.1 IMAP server, Gnus/nnimap
> gets confused and displays ghost messages with address "nobody" and
> subject "(none)" in the summary buffer, like in this example:
>
> *Summary nnimap+dev.gentoo.org:INBOX*
> ----------------------------------------------------------------------
> R. [   1: Ulrich Mueller         ] test 1
>  . [   ?: nobody                 ] (none)
>  . [   1: Ulrich Mueller         ] test 2
> ----------------------------------------------------------------------

[...]

> Buffer " *nnimap dev.gentoo.org nil *nntpd**" looks like this, upon
> entering nnimap-transform-headers:
>----------------------------------------------------------------------
> * 583 FETCH (UID 32409 RFC822.SIZE 698 BODYSTRUCTURE ("text" "plain" ("charset" "us-ascii") NIL NIL "7bit" 8 1 NIL NIL NIL NIL) BODY[HEADER.FIELDS (SUBJECT FROM DATE MESSAGE-ID REFERENCES IN-REPLY-TO XREF X-DIARY-TIME-ZONE X-DIARY-DOW X-DIARY-YEAR X-DIARY-MONTH X-DIARY-DOM X-DIARY-HOUR X-DIARY-MINUTE TO NEWSGROUPS CC)] {165}
> From: Ulrich Mueller <ulm@gentoo.org>
> To: ulm@gentoo.org
> Subject: test 1
> Date: Sat, 27 Apr 2019 07:39:56 +0200
> Message-ID: <w6gd0l8vwer.fsf@kph.uni-mainz.de>
>
> )
> * 584 FETCH (UID 32410 RFC822.SIZE 698 BODYSTRUCTURE ("text" "plain" ("charset" "us-ascii") NIL NIL "7bit" 8 1 NIL NIL NIL NIL) BODY[HEADER.FIELDS (SUBJECT FROM DATE MESSAGE-ID REFERENCES IN-REPLY-TO XREF X-DIARY-TIME-ZONE X-DIARY-DOW X-DIARY-YEAR X-DIARY-MONTH X-DIARY-DOM X-DIARY-HOUR X-DIARY-MINUTE TO NEWSGROUPS CC)] {165}
> From: Ulrich Mueller <ulm@gentoo.org>
> To: ulm@gentoo.org
> Subject: test 2
> Date: Sat, 27 Apr 2019 07:40:15 +0200
> Message-ID: <w6g8svwvwe8.fsf@kph.uni-mainz.de>
>
> )
> * 583 FETCH (UID 32409 MODSEQ (63364) FLAGS ($HasNoAttachment))
> * 584 FETCH (UID 32410 MODSEQ (63364) FLAGS ($HasNoAttachment))
> 10194 OK Fetch completed (0.003 + 0.000 + 0.002 secs).
> ----------------------------------------------------------------------

Okay, I've made a bit of progress on this.

Locally I'm using Dovecot 2.3.6, and it does not output those last two
lines before the OK.

`nnimap-transform-headers' is not expecting those two lines -- it
deletes all but the last, leaving a buffer that looks like:

211 32409 Article retrieved.
Chars: 698
Lines: 1
From: Ulrich Mueller <ulm@gentoo.org>
To: ulm@gentoo.org
Subject: test 1
Date: Sat, 27 Apr 2019 07:39:56 +0200
Message-ID: <w6gd0l8vwer.fsf@kph.uni-mainz.de>

.
211 32410 Article retrieved.
Chars: 698
Lines: 1
From: Ulrich Mueller <ulm@gentoo.org>
To: ulm@gentoo.org
Subject: test 2
Date: Sat, 27 Apr 2019 07:40:15 +0200
Message-ID: <w6g8svwvwe8.fsf@kph.uni-mainz.de>

.
211 32409 Article retrieved.
* 584 FETCH (UID 32410 MODSEQ (63364) FLAGS ($HasNoAttachment))
10194 OK Fetch completed (0.003 + 0.000 + 0.002 secs).
.

Which Gnus then parses as an _extra_ article 32409, but then there's no
header data for it, which is why you get all the "nobody" "none"
nonsense.

Essentially, we're not set up to parse this particular return value.

I guess what I'll do is ask on the dovecot mailing list under what
circumstances/versions we'd get a string like that, and try to help Gnus
parse it correctly.

Thanks for your patience,
Eric