From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Eric Abrahamsen Newsgroups: gmane.emacs.bugs Subject: bug#35443: 27.0.50; Gnus (nnimap) shows "ghost" messages in summary buffer Date: Fri, 12 Jul 2019 09:58:49 -0700 Message-ID: <871ryvw5o6.fsf@ericabrahamsen.net> References: <87wojfjxry.fsf@ericabrahamsen.net> <87h8ajjhux.fsf@ericabrahamsen.net> <87lfztiqu3.fsf@ericabrahamsen.net> <871s1735ec.fsf@ericabrahamsen.net> <87ftpn75bv.fsf@ericabrahamsen.net> <87pnn5wbea.fsf@ericabrahamsen.net> <87a7dkv5cx.fsf@ericabrahamsen.net> <87k1cotix6.fsf@ericabrahamsen.net> <875zo7w62u.fsf@ericabrahamsen.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="110939"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) To: 35443@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Fri Jul 12 19:00:10 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1hlyu1-000SeO-QH for geb-bug-gnu-emacs@m.gmane.org; Fri, 12 Jul 2019 19:00:09 +0200 Original-Received: from localhost ([::1]:51268 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hlyu0-0005xi-PN for geb-bug-gnu-emacs@m.gmane.org; Fri, 12 Jul 2019 13:00:08 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:57688) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hlytx-0005wB-SG for bug-gnu-emacs@gnu.org; Fri, 12 Jul 2019 13:00:07 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hlytw-0007EH-I8 for bug-gnu-emacs@gnu.org; Fri, 12 Jul 2019 13:00:05 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:60043) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hlytv-0007Ch-AF for bug-gnu-emacs@gnu.org; Fri, 12 Jul 2019 13:00:04 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hlytu-0007Wa-8B for bug-gnu-emacs@gnu.org; Fri, 12 Jul 2019 13:00:03 -0400 X-Loop: help-debbugs@gnu.org In-Reply-To: Resent-From: Eric Abrahamsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 12 Jul 2019 17:00:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 35443 X-GNU-PR-Package: emacs X-Debbugs-Original-To: bug-gnu-emacs@gnu.org Original-Received: via spool by submit@debbugs.gnu.org id=B.156295075028812 (code B ref -1); Fri, 12 Jul 2019 17:00:02 +0000 Original-Received: (at submit) by debbugs.gnu.org; 12 Jul 2019 16:59:10 +0000 Original-Received: from localhost ([127.0.0.1]:40631 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hlyt4-0007Ue-1a for submit@debbugs.gnu.org; Fri, 12 Jul 2019 12:59:10 -0400 Original-Received: from lists.gnu.org ([209.51.188.17]:34497) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hlyt2-0007UT-Cg for submit@debbugs.gnu.org; Fri, 12 Jul 2019 12:59:08 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:57405) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hlyt0-0005Uu-S5 for bug-gnu-emacs@gnu.org; Fri, 12 Jul 2019 12:59:08 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hlysz-0006bZ-KH for bug-gnu-emacs@gnu.org; Fri, 12 Jul 2019 12:59:06 -0400 Original-Received: from 195-159-176-226.customer.powertech.no ([195.159.176.226]:57448 helo=blaine.gmane.org) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hlysz-0006aT-BU for bug-gnu-emacs@gnu.org; Fri, 12 Jul 2019 12:59:05 -0400 Original-Received: from list by blaine.gmane.org with local (Exim 4.89) (envelope-from ) id 1hlysv-000RAZ-So for bug-gnu-emacs@gnu.org; Fri, 12 Jul 2019 18:59:01 +0200 X-Injected-Via-Gmane: http://gmane.org/ Cancel-Lock: sha1:9/RfmDyhE4dmf5P54g4qcguQf5M= X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:162793 Archived-At: --=-=-= Content-Type: text/plain Eric Abrahamsen writes: > Lars Ingebrigtsen writes: > >> Eric Abrahamsen writes: >> >>> Which is obviously insufficient. I've since changed it to: >>> >>> "\\([A-Z][[:ascii:]-]+: \\)" >>> >>> Which works better, but also seems susceptible to breakage due to >>> ignorance. A quick scan doesn't turn up any `mail-header-regexp' >>> variables or `this-is-a-mail-header-p' functions -- does the above >>> regexp seem reasonable? >> >> Headers are case insensitive... but not all characters in the [:ascii:] >> set are allowed; far from it. [:alnum:] is closer, but with - and... >> er... stuff... One of the RFCs probably has the definition. :-) > > Oh, good point. RFC2822 says chars 33 to 126, inclusive, minus the > colon. So I guess it's "[!-9,;-~]+: ", unlikely as some of those > characters seem. > > Now I'm working with the attached diff. Sorry, this one... --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=imap-header-transform-fix.diff diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el index 9e52abc1ca..1fe6d32c4f 100644 --- a/lisp/gnus/nnimap.el +++ b/lisp/gnus/nnimap.el @@ -232,7 +232,7 @@ nnimap-retrieve-headers (defun nnimap-transform-headers () (goto-char (point-min)) - (let (article lines size string labels) + (let (seen-articles article lines size string labels) (cl-block nil (while (not (eobp)) (while (not (looking-at "\\* [0-9]+ FETCH")) @@ -261,45 +261,56 @@ nnimap-transform-headers (and (re-search-forward "UID \\([0-9]+\\)" (line-end-position) t) (match-string 1))) - (setq lines nil) - (beginning-of-line) - (setq size - (and (re-search-forward "RFC822.SIZE \\([0-9]+\\)" - (line-end-position) - t) - (match-string 1))) - (beginning-of-line) - (when (search-forward "X-GM-LABELS" (line-end-position) t) - (setq labels (ignore-errors (read (current-buffer))))) - (beginning-of-line) - (when (search-forward "BODYSTRUCTURE" (line-end-position) t) - (let ((structure (ignore-errors - (read (current-buffer))))) - (while (and (consp structure) - (not (atom (car structure)))) - (setq structure (car structure))) - (setq lines (if (and - (stringp (car structure)) - (equal (upcase (nth 0 structure)) "MESSAGE") - (equal (upcase (nth 1 structure)) "RFC822")) - (nth 9 structure) - (nth 7 structure))))) - (delete-region (line-beginning-position) (line-end-position)) - (insert (format "211 %s Article retrieved." article)) - (forward-line 1) - (when size - (insert (format "Chars: %s\n" size))) - (when lines - (insert (format "Lines: %s\n" lines))) - (when labels - (insert (format "X-GM-LABELS: %s\n" labels))) - ;; Most servers have a blank line after the headers, but - ;; Davmail doesn't. - (unless (re-search-forward "^\r$\\|^)\r?$" nil t) - (goto-char (point-max))) - (delete-region (line-beginning-position) (line-end-position)) - (insert ".") - (forward-line 1))))) + ;; If we've already got headers for this article, or this + ;; FETCH line doesn't provide headers for the article, skip + ;; it. See bug#35433. + (if (or (member article seen-articles) + (save-excursion + (forward-line) + (null (looking-at-p + "^[!-9;-~]+: ")))) + (delete-region (line-beginning-position) + (1+ (line-end-position))) + (setq lines nil) + (beginning-of-line) + (setq size + (and (re-search-forward "RFC822.SIZE \\([0-9]+\\)" + (line-end-position) + t) + (match-string 1))) + (beginning-of-line) + (when (search-forward "X-GM-LABELS" (line-end-position) t) + (setq labels (ignore-errors (read (current-buffer))))) + (beginning-of-line) + (when (search-forward "BODYSTRUCTURE" (line-end-position) t) + (let ((structure (ignore-errors + (read (current-buffer))))) + (while (and (consp structure) + (not (atom (car structure)))) + (setq structure (car structure))) + (setq lines (if (and + (stringp (car structure)) + (equal (upcase (nth 0 structure)) "MESSAGE") + (equal (upcase (nth 1 structure)) "RFC822")) + (nth 9 structure) + (nth 7 structure))))) + (delete-region (line-beginning-position) (line-end-position)) + (insert (format "211 %s Article retrieved." article)) + (forward-line 1) + (when size + (insert (format "Chars: %s\n" size))) + (when lines + (insert (format "Lines: %s\n" lines))) + (when labels + (insert (format "X-GM-LABELS: %s\n" labels))) + ;; Most servers have a blank line after the headers, but + ;; Davmail doesn't. + (unless (re-search-forward "^\r$\\|^)\r?$" nil t) + (goto-char (point-max))) + (delete-region (line-beginning-position) (line-end-position)) + (insert ".") + (forward-line 1) + (push article seen-articles)))))) (defun nnimap-unfold-quoted-lines () ;; Unfold quoted {number} strings. --=-=-=--