From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Eric Abrahamsen Newsgroups: gmane.emacs.bugs Subject: bug#35443: 27.0.50; Gnus (nnimap) shows "ghost" messages in summary buffer Date: Thu, 09 May 2019 10:27:07 -0700 Message-ID: <871s1735ec.fsf@ericabrahamsen.net> References: <87wojfjxry.fsf@ericabrahamsen.net> <87h8ajjhux.fsf@ericabrahamsen.net> <87lfztiqu3.fsf@ericabrahamsen.net> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="93679"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) Cc: 35443@debbugs.gnu.org To: Ulrich Mueller Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Thu May 09 19:31:13 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1hOmsy-000OFB-IA for geb-bug-gnu-emacs@m.gmane.org; Thu, 09 May 2019 19:31:12 +0200 Original-Received: from localhost ([127.0.0.1]:58544 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hOmsx-00008n-Ka for geb-bug-gnu-emacs@m.gmane.org; Thu, 09 May 2019 13:31:11 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]:58128) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hOmpv-0006Q4-1J for bug-gnu-emacs@gnu.org; Thu, 09 May 2019 13:28:04 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hOmpt-0000cx-Us for bug-gnu-emacs@gnu.org; Thu, 09 May 2019 13:28:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:50984) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hOmpt-0000cs-Rr for bug-gnu-emacs@gnu.org; Thu, 09 May 2019 13:28:01 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hOmpt-0006YV-N6 for bug-gnu-emacs@gnu.org; Thu, 09 May 2019 13:28:01 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eric Abrahamsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Thu, 09 May 2019 17:28:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 35443 X-GNU-PR-Package: emacs Original-Received: via spool by 35443-submit@debbugs.gnu.org id=B35443.155742283725121 (code B ref 35443); Thu, 09 May 2019 17:28:01 +0000 Original-Received: (at 35443) by debbugs.gnu.org; 9 May 2019 17:27:17 +0000 Original-Received: from localhost ([127.0.0.1]:36286 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hOmpB-0006X6-68 for submit@debbugs.gnu.org; Thu, 09 May 2019 13:27:17 -0400 Original-Received: from ericabrahamsen.net ([52.70.2.18]:48370 helo=mail.ericabrahamsen.net) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hOmp9-0006Wo-Hq for 35443@debbugs.gnu.org; Thu, 09 May 2019 13:27:16 -0400 Original-Received: from localhost (97-113-41-132.tukw.qwest.net [97.113.41.132]) (Authenticated sender: eric@ericabrahamsen.net) by mail.ericabrahamsen.net (Postfix) with ESMTPSA id D8B55FA621; Thu, 9 May 2019 17:27:08 +0000 (UTC) In-Reply-To: <87lfztiqu3.fsf@ericabrahamsen.net> (Eric Abrahamsen's message of "Sun, 28 Apr 2019 17:43:16 -0700") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:158993 Archived-At: Eric Abrahamsen writes: --8<---------------cut here---------------start------------->8--- > Ulrich Mueller writes: > >>>>>>> On Sat, 27 Apr 2019, Eric Abrahamsen wrote: >> >>> Huh. Can you show us the value of gnus-newsgroup-data and >>> gnus-newsgroup-headers in this group? >> >> See below, for the current group, whose summary buffer shows your >> article and a dummy article. >> >>> Does the dummy article always show up? >> >> It always shows up when new articles have been fetched. It doesn't >> show up when entering a group that doesn't have new mail (i.e., if a >> group previously had a dummy article, it will be gone when reentering >> that group). >> >>> Only in this group? >> >> All groups for that particular IMAP server. >> >>> Is this a newly-added IMAP server, and did it display this problem >>> right from the beginning? >> >> AFAICS, the problem appeared after dovecot on the server side was >> upgraded from version 2.2.34 to 2.3.5.1. But I see it only with Gnus. >> I don't have any issues with Mozilla Thunderbird (on GNU/Linux) nor with >> K-9 Mail (on Android/Linux). > > Thanks, this should be enough to make some progress in tracking the > problem down. Give me a couple of days... Okay, here's what I was told on irc: --8<---------------cut here---------------start------------->8--- looks like it's the new feature that adds $HasAttachment or $HasNoAttachment flags to mails. they're actually supposed to be added only during mail delivery, and there's a setting needed to enable them: mail_attachment_detection_options = add-flags-on-save but .. there's an "unintentional feature" :) that they also get added during some FETCH commands if they're not already there but even without this, imap protocol allows sending FETCH FLAGS updates (or any updates really) as a response to any command. and with concurrent imap access dovecot does this. for example if another client adds a \Seen flag, that same thing could happen either way, a client should really be able to process these "unexpeceted" FETCH updates, that's what the standard is about --8<---------------cut here---------------end--------------->8--- Ultimately, the "real" fix would be to teach Gnus to handle all possible responses from IMAP servers, probably using a proper parser. But if you have access to the dovecot config in your case, you might try removing the mail_attachment_detection_options setting above, if it's set. If it's not set, then you might be getting the "unintentional feature". I suppose in the interim I could mess with `nnimap-transform-headers' and try to add some bookkeeping so that multiple FETCH responses for the same article UID get merged together. On the other hand, this feature only seems to relate to has/hasnoattachment flags, which Gnus doesn't handle anyway -- we could safely drop those lines altogether. (Though it sure would be nice to handle hasattachment flags, that's something that users have requested in the past...) Well, that's where we're at. Eric