From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Eric Abrahamsen Newsgroups: gmane.emacs.bugs Subject: bug#35443: 27.0.50; Gnus (nnimap) shows "ghost" messages in summary buffer Date: Sat, 22 Jun 2019 14:36:13 -0700 Message-ID: <87pnn5wbea.fsf@ericabrahamsen.net> References: <87wojfjxry.fsf@ericabrahamsen.net> <87h8ajjhux.fsf@ericabrahamsen.net> <87lfztiqu3.fsf@ericabrahamsen.net> <871s1735ec.fsf@ericabrahamsen.net> <87ftpn75bv.fsf@ericabrahamsen.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="245398"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) Cc: 35443@debbugs.gnu.org, Ulrich Mueller To: Lars Ingebrigtsen Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Sat Jun 22 23:37:11 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1henh7-0011eU-Dg for geb-bug-gnu-emacs@m.gmane.org; Sat, 22 Jun 2019 23:37:09 +0200 Original-Received: from localhost ([::1]:42494 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1henh6-0001N8-FF for geb-bug-gnu-emacs@m.gmane.org; Sat, 22 Jun 2019 17:37:08 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:51486) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1henh1-0001Mq-Cy for bug-gnu-emacs@gnu.org; Sat, 22 Jun 2019 17:37:04 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1henh0-0005pB-0C for bug-gnu-emacs@gnu.org; Sat, 22 Jun 2019 17:37:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:39069) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hengz-0005oj-R4 for bug-gnu-emacs@gnu.org; Sat, 22 Jun 2019 17:37:01 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hengz-0005p0-Mt for bug-gnu-emacs@gnu.org; Sat, 22 Jun 2019 17:37:01 -0400 X-Loop: help-debbugs@gnu.org In-Reply-To: Resent-From: Eric Abrahamsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 22 Jun 2019 21:37:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 35443 X-GNU-PR-Package: emacs Original-Received: via spool by 35443-submit@debbugs.gnu.org id=B35443.156123939222341 (code B ref 35443); Sat, 22 Jun 2019 21:37:01 +0000 Original-Received: (at 35443) by debbugs.gnu.org; 22 Jun 2019 21:36:32 +0000 Original-Received: from localhost ([127.0.0.1]:52613 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hengV-0005oG-N8 for submit@debbugs.gnu.org; Sat, 22 Jun 2019 17:36:32 -0400 Original-Received: from ericabrahamsen.net ([52.70.2.18]:50910 helo=mail.ericabrahamsen.net) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hengR-0005o1-K0 for 35443@debbugs.gnu.org; Sat, 22 Jun 2019 17:36:29 -0400 Original-Received: from localhost (unknown [172.92.212.120]) (Authenticated sender: eric@ericabrahamsen.net) by mail.ericabrahamsen.net (Postfix) with ESMTPSA id 9A456FA05D; Sat, 22 Jun 2019 21:36:19 +0000 (UTC) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:161064 Archived-At: --=-=-= Content-Type: text/plain Lars Ingebrigtsen writes: > Eric Abrahamsen writes: > >>> Maybe drop the extra FETCH responses for now, and add proper bookkeeping >>> later when implementing the requested feature? >> >> Okay, I'll work something up in the next week that goes halfway with >> this. > > Did you get any further in fixing this nnimap parsing bug? Here's a whack at it. I tried to make sure that it would handle unwanted FETCH responses whether they came before or after (or in the middle of) the wanted FETCH responses, but I'm not in love with checking the header regexp this way. Because this IMAP server feature is very closely focused on adding a flag in case of attachment (and because Gnus never explicitly requests this flag, though I'd sure like to in the future), another more targeted approach would be to simply delete any lines containing $Has\(No\)?Attachment, assuming that these FETCH responses will only take up one line. I've configured my local Dovecot with the offending setting, and will test it out for a bit. Eric --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=imap-header-transform-fix.diff diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el index 9e52abc1ca..7478fcb3e5 100644 --- a/lisp/gnus/nnimap.el +++ b/lisp/gnus/nnimap.el @@ -232,7 +232,7 @@ nnimap-retrieve-headers (defun nnimap-transform-headers () (goto-char (point-min)) - (let (article lines size string labels) + (let (seen-articles article lines size string labels) (cl-block nil (while (not (eobp)) (while (not (looking-at "\\* [0-9]+ FETCH")) @@ -261,45 +261,56 @@ nnimap-transform-headers (and (re-search-forward "UID \\([0-9]+\\)" (line-end-position) t) (match-string 1))) - (setq lines nil) - (beginning-of-line) - (setq size - (and (re-search-forward "RFC822.SIZE \\([0-9]+\\)" - (line-end-position) - t) - (match-string 1))) - (beginning-of-line) - (when (search-forward "X-GM-LABELS" (line-end-position) t) - (setq labels (ignore-errors (read (current-buffer))))) - (beginning-of-line) - (when (search-forward "BODYSTRUCTURE" (line-end-position) t) - (let ((structure (ignore-errors - (read (current-buffer))))) - (while (and (consp structure) - (not (atom (car structure)))) - (setq structure (car structure))) - (setq lines (if (and - (stringp (car structure)) - (equal (upcase (nth 0 structure)) "MESSAGE") - (equal (upcase (nth 1 structure)) "RFC822")) - (nth 9 structure) - (nth 7 structure))))) - (delete-region (line-beginning-position) (line-end-position)) - (insert (format "211 %s Article retrieved." article)) - (forward-line 1) - (when size - (insert (format "Chars: %s\n" size))) - (when lines - (insert (format "Lines: %s\n" lines))) - (when labels - (insert (format "X-GM-LABELS: %s\n" labels))) - ;; Most servers have a blank line after the headers, but - ;; Davmail doesn't. - (unless (re-search-forward "^\r$\\|^)\r?$" nil t) - (goto-char (point-max))) - (delete-region (line-beginning-position) (line-end-position)) - (insert ".") - (forward-line 1))))) + ;; If we've already got headers for this article, or this + ;; FETCH line doesn't provide headers for the article, skip + ;; it. See bug#35433. + (if (or (member article seen-articles) + (save-excursion + (forward-line) + (null (looking-at-p + "\\(From\\|To\\|Subject\\|Date\\|Message-ID\\)")))) + (delete-region (line-beginning-position) + (1+ (line-end-position))) + (setq lines nil) + (beginning-of-line) + (setq size + (and (re-search-forward "RFC822.SIZE \\([0-9]+\\)" + (line-end-position) + t) + (match-string 1))) + (beginning-of-line) + (when (search-forward "X-GM-LABELS" (line-end-position) t) + (setq labels (ignore-errors (read (current-buffer))))) + (beginning-of-line) + (when (search-forward "BODYSTRUCTURE" (line-end-position) t) + (let ((structure (ignore-errors + (read (current-buffer))))) + (while (and (consp structure) + (not (atom (car structure)))) + (setq structure (car structure))) + (setq lines (if (and + (stringp (car structure)) + (equal (upcase (nth 0 structure)) "MESSAGE") + (equal (upcase (nth 1 structure)) "RFC822")) + (nth 9 structure) + (nth 7 structure))))) + (delete-region (line-beginning-position) (line-end-position)) + (insert (format "211 %s Article retrieved." article)) + (forward-line 1) + (when size + (insert (format "Chars: %s\n" size))) + (when lines + (insert (format "Lines: %s\n" lines))) + (when labels + (insert (format "X-GM-LABELS: %s\n" labels))) + ;; Most servers have a blank line after the headers, but + ;; Davmail doesn't. + (unless (re-search-forward "^\r$\\|^)\r?$" nil t) + (goto-char (point-max))) + (delete-region (line-beginning-position) (line-end-position)) + (insert ".") + (forward-line 1) + (push article seen-articles)))))) (defun nnimap-unfold-quoted-lines () ;; Unfold quoted {number} strings. --=-=-=--