From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Eric Abrahamsen Newsgroups: gmane.emacs.bugs Subject: bug#35443: 27.0.50; Gnus (nnimap) shows "ghost" messages in summary buffer Date: Fri, 12 Jul 2019 09:50:01 -0700 Message-ID: <875zo7w62u.fsf@ericabrahamsen.net> References: <87wojfjxry.fsf@ericabrahamsen.net> <87h8ajjhux.fsf@ericabrahamsen.net> <87lfztiqu3.fsf@ericabrahamsen.net> <871s1735ec.fsf@ericabrahamsen.net> <87ftpn75bv.fsf@ericabrahamsen.net> <87pnn5wbea.fsf@ericabrahamsen.net> <87a7dkv5cx.fsf@ericabrahamsen.net> <87k1cotix6.fsf@ericabrahamsen.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="68345"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.50 (gnu/linux) Cc: 35443@debbugs.gnu.org To: Lars Ingebrigtsen Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Fri Jul 12 18:51:16 2019 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1hlylN-000Hd1-RD for geb-bug-gnu-emacs@m.gmane.org; Fri, 12 Jul 2019 18:51:13 +0200 Original-Received: from localhost ([::1]:51192 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hlylM-0002GG-TK for geb-bug-gnu-emacs@m.gmane.org; Fri, 12 Jul 2019 12:51:12 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:55384) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hlylJ-0002G7-VQ for bug-gnu-emacs@gnu.org; Fri, 12 Jul 2019 12:51:11 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hlylE-00008l-7F for bug-gnu-emacs@gnu.org; Fri, 12 Jul 2019 12:51:06 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:60035) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hlylC-00007z-FR for bug-gnu-emacs@gnu.org; Fri, 12 Jul 2019 12:51:04 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hlylC-0007E8-E8 for bug-gnu-emacs@gnu.org; Fri, 12 Jul 2019 12:51:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Eric Abrahamsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 12 Jul 2019 16:51:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 35443 X-GNU-PR-Package: emacs Original-Received: via spool by 35443-submit@debbugs.gnu.org id=B35443.156295021627711 (code B ref 35443); Fri, 12 Jul 2019 16:51:02 +0000 Original-Received: (at 35443) by debbugs.gnu.org; 12 Jul 2019 16:50:16 +0000 Original-Received: from localhost ([127.0.0.1]:40623 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hlykR-0007Ct-N0 for submit@debbugs.gnu.org; Fri, 12 Jul 2019 12:50:16 -0400 Original-Received: from ericabrahamsen.net ([52.70.2.18]:58878 helo=mail.ericabrahamsen.net) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hlykP-0007CY-C2 for 35443@debbugs.gnu.org; Fri, 12 Jul 2019 12:50:14 -0400 Original-Received: from localhost (c-73-254-86-141.hsd1.wa.comcast.net [73.254.86.141]) (Authenticated sender: eric@ericabrahamsen.net) by mail.ericabrahamsen.net (Postfix) with ESMTPSA id 40F6CFA02D; Fri, 12 Jul 2019 16:50:03 +0000 (UTC) In-Reply-To: (Lars Ingebrigtsen's message of "Fri, 12 Jul 2019 16:18:02 +0200") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.org gmane.emacs.bugs:162791 Archived-At: --=-=-= Content-Type: text/plain Lars Ingebrigtsen writes: > Eric Abrahamsen writes: > >> Which is obviously insufficient. I've since changed it to: >> >> "\\([A-Z][[:ascii:]-]+: \\)" >> >> Which works better, but also seems susceptible to breakage due to >> ignorance. A quick scan doesn't turn up any `mail-header-regexp' >> variables or `this-is-a-mail-header-p' functions -- does the above >> regexp seem reasonable? > > Headers are case insensitive... but not all characters in the [:ascii:] > set are allowed; far from it. [:alnum:] is closer, but with - and... > er... stuff... One of the RFCs probably has the definition. :-) Oh, good point. RFC2822 says chars 33 to 126, inclusive, minus the colon. So I guess it's "[!-9,;-~]+: ", unlikely as some of those characters seem. Now I'm working with the attached diff. Eric --=-=-= Content-Type: text/x-patch Content-Disposition: attachment; filename=imap-header-transform-fix.diff diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el index 9e52abc1ca..802254fc14 100644 --- a/lisp/gnus/nnimap.el +++ b/lisp/gnus/nnimap.el @@ -232,7 +232,7 @@ nnimap-retrieve-headers (defun nnimap-transform-headers () (goto-char (point-min)) - (let (article lines size string labels) + (let (seen-articles article lines size string labels) (cl-block nil (while (not (eobp)) (while (not (looking-at "\\* [0-9]+ FETCH")) @@ -261,45 +261,56 @@ nnimap-transform-headers (and (re-search-forward "UID \\([0-9]+\\)" (line-end-position) t) (match-string 1))) - (setq lines nil) - (beginning-of-line) - (setq size - (and (re-search-forward "RFC822.SIZE \\([0-9]+\\)" - (line-end-position) - t) - (match-string 1))) - (beginning-of-line) - (when (search-forward "X-GM-LABELS" (line-end-position) t) - (setq labels (ignore-errors (read (current-buffer))))) - (beginning-of-line) - (when (search-forward "BODYSTRUCTURE" (line-end-position) t) - (let ((structure (ignore-errors - (read (current-buffer))))) - (while (and (consp structure) - (not (atom (car structure)))) - (setq structure (car structure))) - (setq lines (if (and - (stringp (car structure)) - (equal (upcase (nth 0 structure)) "MESSAGE") - (equal (upcase (nth 1 structure)) "RFC822")) - (nth 9 structure) - (nth 7 structure))))) - (delete-region (line-beginning-position) (line-end-position)) - (insert (format "211 %s Article retrieved." article)) - (forward-line 1) - (when size - (insert (format "Chars: %s\n" size))) - (when lines - (insert (format "Lines: %s\n" lines))) - (when labels - (insert (format "X-GM-LABELS: %s\n" labels))) - ;; Most servers have a blank line after the headers, but - ;; Davmail doesn't. - (unless (re-search-forward "^\r$\\|^)\r?$" nil t) - (goto-char (point-max))) - (delete-region (line-beginning-position) (line-end-position)) - (insert ".") - (forward-line 1))))) + ;; If we've already got headers for this article, or this + ;; FETCH line doesn't provide headers for the article, skip + ;; it. See bug#35433. + (if (or (member article seen-articles) + (save-excursion + (forward-line) + (null (looking-at-p + "[!-9,;-~]+: ")))) + (delete-region (line-beginning-position) + (1+ (line-end-position))) + (setq lines nil) + (beginning-of-line) + (setq size + (and (re-search-forward "RFC822.SIZE \\([0-9]+\\)" + (line-end-position) + t) + (match-string 1))) + (beginning-of-line) + (when (search-forward "X-GM-LABELS" (line-end-position) t) + (setq labels (ignore-errors (read (current-buffer))))) + (beginning-of-line) + (when (search-forward "BODYSTRUCTURE" (line-end-position) t) + (let ((structure (ignore-errors + (read (current-buffer))))) + (while (and (consp structure) + (not (atom (car structure)))) + (setq structure (car structure))) + (setq lines (if (and + (stringp (car structure)) + (equal (upcase (nth 0 structure)) "MESSAGE") + (equal (upcase (nth 1 structure)) "RFC822")) + (nth 9 structure) + (nth 7 structure))))) + (delete-region (line-beginning-position) (line-end-position)) + (insert (format "211 %s Article retrieved." article)) + (forward-line 1) + (when size + (insert (format "Chars: %s\n" size))) + (when lines + (insert (format "Lines: %s\n" lines))) + (when labels + (insert (format "X-GM-LABELS: %s\n" labels))) + ;; Most servers have a blank line after the headers, but + ;; Davmail doesn't. + (unless (re-search-forward "^\r$\\|^)\r?$" nil t) + (goto-char (point-max))) + (delete-region (line-beginning-position) (line-end-position)) + (insert ".") + (forward-line 1) + (push article seen-articles)))))) (defun nnimap-unfold-quoted-lines () ;; Unfold quoted {number} strings. --=-=-=--