* bug#21182: 25.0.50; gnus: every other unread message is marked as read on each nnimap group refresh @ 2015-08-03 13:01 Nick Alcock [not found] ` <handler.21182.B.14386068914118.ack@debbugs.gnu.org> ` (2 more replies) 0 siblings, 3 replies; 11+ messages in thread From: Nick Alcock @ 2015-08-03 13:01 UTC (permalink / raw) To: 21182; +Cc: Lars Magne Ingebrigtsen The bug is fairly easily characterized, and easy to spot in a group with a great many consecutive unread messages: every time the group is refreshed, half the unread messages are marked as read, in an alternating pattern, so after one refresh the result runs (... unread, read, unread, read...): after more than one refresh the result is harder to see. After the first instance of this problem, we also see valid messages apparently originating from nobody (their name appears as '(none)', and other similarly bizarre messups indicating something very wrong with IMAP communication. (Cc:ing Drew because the thing I was communicating with was an SSL channel to the very same Beehive mailserver he uses -- I know Drew doesn't use Gnus to post here, but maybe he sometimes looks at his work mailbox via Gnus, in which case he might appreciate a warning!) This happened after my first Emacs update for nearly a year (a bit of a 'big bang'), so it could have come from almost any change to nnimap since September last year. I'll do a bisection and hunt for it. (It's not like I can do any more damage to my imap read marks than I already have! I wanted to get to 'inbox zero', but not this way...) In GNU Emacs 25.0.50.1 (x86_64-unknown-linux-gnu, X toolkit, Xaw3d scroll bars) of 2015-08-01 on spindle Repository revision: 08d65696c947b09d4c32606f279c3c2594e99e16 Windowing system distributor `The X.Org Foundation', version 11.0.11702000 System LSB Version: :core-4.1-amd64:core-4.1-noarch Configured using: `configure --without-pop --without-kerberos --without-hesiod --with-x-toolkit=lucid --with-wide-int --with-file-notification=inotify NO_FAST_MATH=t 'CFLAGS=-Wstrict-overflow=1 -g -gdwarf-4 -feliminate-unused-debug-types -fvar-tracking-assignments -feliminate-dwarf2-dups -O2 -pipe -D__NO_STRING_INLINES -D__NO_MATH_INLINES -D_LARGEFILE64_SOURCE' 'CPPFLAGS=-D__NO_STRING_INLINES -D__NO_MATH_INLINES -D_LARGEFILE64_SOURCE' 'LDFLAGS=-Wl,--enable-new-dtags -Wl,--hash-style=gnu -Wl,-O1 -Wl,-z,relro -pipe'' Configured features: XAW3D XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND DBUS GCONF GSETTINGS NOTIFY ACL GNUTLS LIBXML2 FREETYPE LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS LUCID X11 Important settings: value of $LC_COLLATE: C value of $LANG: en_GB.UTF-8 locale-coding-system: utf-8-unix Major mode: Summary Minor modes in effect: timeclock-mode-line-display: t TeX-PDF-mode: t magit-wip-after-apply-mode: t global-undo-tree-mode: t undo-tree-mode: t diff-auto-refine-mode: t global-semantic-mru-bookmark-mode: t global-semanticdb-minor-mode: t global-semantic-decoration-mode: t global-semantic-idle-scheduler-mode: t global-semantic-idle-local-symbol-highlight-mode: t global-semantic-idle-summary-mode: t global-srecode-minor-mode: t global-ede-mode: t global-semantic-highlight-func-mode: t global-semantic-stickyfunc-mode: t semantic-mode: t global-git-gutter-mode: t global-auto-complete-mode: t global-cwarn-mode: t which-function-mode: t mv-shell-mode: t desktop-save-mode: t savehist-mode: t winner-mode: t Info-breadcrumbs-in-mode-line-mode: t shell-dirtrack-mode: t recentf-mode: t display-time-mode: t override-global-mode: t image-diredx-async-mode: t tooltip-mode: t global-eldoc-mode: t electric-indent-mode: t mouse-wheel-mode: t menu-bar-mode: t file-name-shadow-mode: t global-font-lock-mode: t font-lock-mode: t blink-cursor-mode: t auto-composition-mode: t auto-encryption-mode: t auto-compression-mode: t buffer-read-only: t column-number-mode: t line-number-mode: t Recent messages: Auto-saving...done Auto-saving...done Quit Mark set previous-line: Beginning of buffer [4 times] Mark set C-x C-g is undefined C-M-g is undefined Load-path shadows: /home/nix/lisp/defaults hides /usr/share/emacs/site-lisp/defaults /home/nix/lisp/emacs/scroll-in-place hides /usr/share/emacs/site-lisp/scroll-in-place /usr/share/emacs/site-lisp/emms/tq hides /usr/share/emacs/25.0.50/lisp/emacs-lisp/tq Features: (shadow emacsbug canlock gnus-html flow-fill mm-archive eieio-opt mailalias misearch multi-isearch nnir gnus-picon gnus-cite qp gnus-async gnus-bcklg gnus-salt gnus-dup gnus-ml disp-table semantic/tag-write gnus-topic nndraft gnus-agent gnus-srvr gnus-score score-mode nnvirtual gnus-msg utf-7 gnutls nnmh nnfolder gnus-cache bbdb-gnus bbdb-snarf gnus-demon nntp gnus/mail gnus-registry registry eieio-compat gnus/splits nnmairix nnml smtpmail gnus-art mm-uu mml2015 mm-view mml-smime smime dig gnus/articles init/time-tracking timeclock auto-edit-substitute gnus/sa background gnus-sum gnus-group gnus-undo gnus/bbdb gnus/colourization tc mail-extr gnus-start gnus-cloud nnimap nnmail mail-source utf7 netrc nnoo parse-time gnus-spec gnus-int gnus-range gnus-win face-remap add-log shr-color color network-stream nsm starttls url-http tls url-gw url-cache url-auth eww mm-url gnus gnus-ems nnheader url-queue shr dom subr-x browse-url epa-file epa epg semantic/tag-file semantic/bovine/c hideif semantic/bovine/c-by semantic/lex-spp semantic/bovine/gcc semantic/bovine semantic/analyze/refs semantic/analyze semantic/scope semantic/analyze/fcn vc-filewise vc-rcs generic tex-buf semantic/texi tex-info tex dbus xml texinfo semantic/imenu semantic/sb semantic/sort semantic/db-file data-debug cedet-files semantic/wisent/python semantic/decorate/include semantic/dep semantic/wisent/python-wy rx python json ede/dired jka-compr magit-wip magit-core magit-process with-editor server magit-popup magit-mode magit-git crm magit-section magit-utils dash undo-tree diff vc vc-dispatcher vc-git diff-mode flymake checkdoc hideshow elisp-slime-nav init/emacs init/prog-modes filecache inversion semantic/db-global semantic/symref/global semantic/symref semantic/db-find semantic/db-ref cedet-global semantic/mru-bookmark semantic/db-mode semantic/db semantic/decorate/mode semantic/idle srecode/mode semantic/senator semantic/format semantic/ctxt semantic/wisent semantic/wisent/wisent semantic/decorate pulse srecode/insert srecode/filters srecode/args srecode/find srecode/map srecode/ctxt semantic/tag-ls semantic/find srecode/compile srecode/dictionary srecode/table ede/speedbar ede/files ede ede/detect ede/base ede/auto ede/source eieio-base eieio-speedbar speedbar sb-image ezimage dframe eieio-custom srecode semantic/util-modes semantic/util semantic semantic/tag semantic/lex semantic/fw mode-local cedet git-gutter-fringe fringe-helper git-gutter auto-complete-config auto-complete popup cwarn gtags filladapt cc-mode cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs which-func generic-x init/processes vlf-setup mv-shell init/music emms-volume emms-volume-amixer emms-history emms-bookmarks emms-metaplaylist-mode emms-browser sort emms-playlist-sort emms-last-played emms-playing-time emms-stream-info emms-streams emms-mode-line emms-playlist-limit emms-playlist-mode emms-player-mpd emms-url url url-proxy url-privacy url-expand url-methods url-history url-cookie url-domsuf url-util mailcap emms-cache emms-info later-do tq emms-player-simple emms-source-playlist emms-source-file locate emms emms-compat emms-auto init/messaging bbdb-expire bbdb-hooks silly-mail sendmail message rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mail-utils gmm-utils mailheader bbdb-autoloads init/state desktop frameset savehist saveplace init/editing gpicker finder find-func lisp-mnt windmove framemove miniedit winner scroll-in-place flash-paren init/faces nix-dark-theme init/drew two-column icicles icicles-mode dired+ image-file dired-aux mb-depth+ mb-depth icicles-cmd2 compile imenu completion icicles-cmd1 bbdb-com warnings mailabbrev bbdb timezone yow etags xref project cookie1 icicles-mcmd filesets icicles-fn icicles-var icicles-opt ffap url-parse url-vars fuzzy-match el-swank-fuzzy levenshtein fuzzy cus-theme cus-edit cus-start cus-load hexrgb icicles-face icicles-mac bookmark+ bookmark+-key derived dired-x crosshairs col-highlight vline hl-line+ hl-line bookmark+-1 bookmark+-bmu bookmark+-lit bookmark menu-bar+ linkd doremi apropos-fn+var help-fns+ help+ frame-fns avoid info+ thingatpt+ thingatpt mouse3 naked apropos+ apropos init/appearance tramp-cache tramp-sh tramp tramp-compat auth-source eieio byte-opt eieio-core gnus-util mm-util help-fns mail-prsvr password-cache tramp-loaddefs trampver shell pcomplete comint ansi-color ring recentf tree-widget wid-edit printing ps-print ps-def lpr time-date init/modelines time uptimes pp init/options xemacs-compat misc gawd-lists use-package-external use-package diminish bytecomp byte-compile cl-extra help-mode seq cconv bind-key easy-mmode finder-inf tex-site iedit-autoloads image-dired+ image-dired format-spec dired info yasnippet-bundle cl-macs cl-seq dropdown-list advice yasnippet edmacro kmacro easymenu assoc cl gv cl-loaddefs pcase cl-lib package epg-config mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european ethiopic indian cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice loaddefs button faces cus-face macroexp files text-properties overlay sha1 md5 base64 format env code-pages mule custom widget hashtable-print-readable backquote dbusbind inotify dynamic-setting system-font-setting font-render-setting x-toolkit x multi-tty make-network-process emacs) Memory information: ((conses 16 2245900 1795918) (symbols 48 141793 238) (miscs 40 6910 9463) (strings 32 782235 1143702) (string-bytes 1 34552691) (vectors 16 100813) (vector-slots 8 3950744 1469730) (floats 8 2600 6311) (intervals 56 50025 16112) (buffers 976 145) (heap 1024 313480 457073)) ^ permalink raw reply [flat|nested] 11+ messages in thread
[parent not found: <handler.21182.B.14386068914118.ack@debbugs.gnu.org>]
* bug#21182: 25.0.50; gnus: every other unread message is marked as read on each nnimap group refresh [not found] ` <handler.21182.B.14386068914118.ack@debbugs.gnu.org> @ 2015-08-03 15:04 ` Nix 2016-02-07 5:56 ` Lars Ingebrigtsen 0 siblings, 1 reply; 11+ messages in thread From: Nix @ 2015-08-03 15:04 UTC (permalink / raw) To: 21182; +Cc: Trevor Murphy, Lars Magne Ingebrigtsen I reported: > This happened after my first Emacs update for nearly a year (a bit of a > 'big bang'), so it could have come from almost any change to nnimap > since September last year. I'll do a bisection and hunt for it. The first bad commit is commit 242354a23acf214ad06d4e3e7e5f5580c8b21d4a (HEAD, refs/bisect/bad) Author: Trevor Murphy <trevor.m.murphy@gmail.com> Date: Mon Jan 26 07:56:37 2015 +0000 lisp/gnus/nnimap.el Allow using the Google X-GM-LABELS, if present and indeed reverting this causes the devastating automatically-mark- every-other-item-as-read behaviour to go away. I am *not* using gmail, but Beehive, so clearly this is negatively affecting other imap servers, when it should obviously have no effect on them. I'm frankly surprised a bug this serious has persisted for half a year. It makes me wonder if I'm the only person using trunk Emacs with non- Google imap servers, or if this is something specific to the (rare) Beehive server. Now, if you'll excuse me, I have a huge inbox full of wrongly-marked-as-read messages to sort through :( -- NULL && (void) ^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#21182: 25.0.50; gnus: every other unread message is marked as read on each nnimap group refresh 2015-08-03 15:04 ` Nix @ 2016-02-07 5:56 ` Lars Ingebrigtsen 2017-01-26 21:07 ` Lars Ingebrigtsen 0 siblings, 1 reply; 11+ messages in thread From: Lars Ingebrigtsen @ 2016-02-07 5:56 UTC (permalink / raw) To: Nix; +Cc: Trevor Murphy, 21182 Nix <nix@esperi.org.uk> writes: > The first bad commit is > > commit 242354a23acf214ad06d4e3e7e5f5580c8b21d4a (HEAD, refs/bisect/bad) > Author: Trevor Murphy <trevor.m.murphy@gmail.com> > Date: Mon Jan 26 07:56:37 2015 +0000 > > lisp/gnus/nnimap.el Allow using the Google X-GM-LABELS, if present > > and indeed reverting this causes the devastating automatically-mark- > every-other-item-as-read behaviour to go away. I am *not* using gmail, > but Beehive, so clearly this is negatively affecting other imap servers, > when it should obviously have no effect on them. Huh. That patch should have no effect on any servers that say nil to (nnimap-capability "X-GM-EXT-1"), unless I'm reading it wrong. And nnimap is not behaving on this strange way for me, and I don't use Gmail, so I don't quite understand this. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#21182: 25.0.50; gnus: every other unread message is marked as read on each nnimap group refresh 2016-02-07 5:56 ` Lars Ingebrigtsen @ 2017-01-26 21:07 ` Lars Ingebrigtsen 2017-01-26 22:32 ` Nix 0 siblings, 1 reply; 11+ messages in thread From: Lars Ingebrigtsen @ 2017-01-26 21:07 UTC (permalink / raw) To: Nix; +Cc: Trevor Murphy, 21182 Lars Ingebrigtsen <larsi@gnus.org> writes: > Nix <nix@esperi.org.uk> writes: > >> The first bad commit is >> >> commit 242354a23acf214ad06d4e3e7e5f5580c8b21d4a (HEAD, refs/bisect/bad) >> Author: Trevor Murphy <trevor.m.murphy@gmail.com> >> Date: Mon Jan 26 07:56:37 2015 +0000 >> >> lisp/gnus/nnimap.el Allow using the Google X-GM-LABELS, if present >> >> and indeed reverting this causes the devastating automatically-mark- >> every-other-item-as-read behaviour to go away. I am *not* using gmail, >> but Beehive, so clearly this is negatively affecting other imap servers, >> when it should obviously have no effect on them. > > Huh. That patch should have no effect on any servers that say nil to > (nnimap-capability "X-GM-EXT-1"), unless I'm reading it wrong. > > And nnimap is not behaving on this strange way for me, and I don't use > Gmail, so I don't quite understand this. Are you still seeing this? Nobody else has reported anything similar (I think), so I suspect it's something particular with your setup or server... -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#21182: 25.0.50; gnus: every other unread message is marked as read on each nnimap group refresh 2017-01-26 21:07 ` Lars Ingebrigtsen @ 2017-01-26 22:32 ` Nix 0 siblings, 0 replies; 11+ messages in thread From: Nix @ 2017-01-26 22:32 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: Trevor Murphy, 21182 On 26 Jan 2017, Lars Ingebrigtsen spake thusly: > Lars Ingebrigtsen <larsi@gnus.org> writes: > >> Nix <nix@esperi.org.uk> writes: >> >>> The first bad commit is >>> >>> commit 242354a23acf214ad06d4e3e7e5f5580c8b21d4a (HEAD, refs/bisect/bad) >>> Author: Trevor Murphy <trevor.m.murphy@gmail.com> >>> Date: Mon Jan 26 07:56:37 2015 +0000 >>> >>> lisp/gnus/nnimap.el Allow using the Google X-GM-LABELS, if present >>> >>> and indeed reverting this causes the devastating automatically-mark- >>> every-other-item-as-read behaviour to go away. I am *not* using gmail, >>> but Beehive, so clearly this is negatively affecting other imap servers, >>> when it should obviously have no effect on them. >> >> Huh. That patch should have no effect on any servers that say nil to >> (nnimap-capability "X-GM-EXT-1"), unless I'm reading it wrong. >> >> And nnimap is not behaving on this strange way for me, and I don't use >> Gmail, so I don't quite understand this. > > Are you still seeing this? Nobody else has reported anything similar (I > think), so I suspect it's something particular with your setup or server... I haven't retried for a few months: I'm running with that patch backed out. I'll check again, but I think it's quite unlikely that the problem has gone away. It was still present when I last checked, and nothing has changed in this part of Gnus at all since then, at least not on the Emacs 25 branch where I'm living. I did a bunch more tracking down in the bug back in June last year: see <https://debbugs.gnu.org/cgi/bugreport.cgi?bug=21182#30>. I would be utterly, cosmically unsurprised to find that the problem was specific to the format of the IMAP sent back by the crawling horror which is Beehive. (I could see if I could convince Drew Adams to set up Gnus to talk to the Oracle external IMAP server, but I suspect that if I tell him that this is to try to replicate a bug that makes every other mail get marked as read he'll tell me to bugger off, he likes his read states just the way they are!) -- NULL && (void) ^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#21182: 25.0.50; gnus: every other unread message is marked as read on each nnimap group refresh 2015-08-03 13:01 bug#21182: 25.0.50; gnus: every other unread message is marked as read on each nnimap group refresh Nick Alcock [not found] ` <handler.21182.B.14386068914118.ack@debbugs.gnu.org> @ 2016-02-07 5:57 ` Lars Ingebrigtsen 2016-02-08 19:05 ` Nix 2016-06-11 16:16 ` Nix 2016-05-22 19:26 ` Paul Eggert 2 siblings, 2 replies; 11+ messages in thread From: Lars Ingebrigtsen @ 2016-02-07 5:57 UTC (permalink / raw) To: Nick Alcock; +Cc: 21182 Nick Alcock <nix@esperi.org.uk> writes: > This happened after my first Emacs update for nearly a year (a bit of a > 'big bang'), so it could have come from almost any change to nnimap > since September last year. I'll do a bisection and hunt for it. (It's > not like I can do any more damage to my imap read marks than I already > have! I wanted to get to 'inbox zero', but not this way...) Are you still seeing this problem? -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#21182: 25.0.50; gnus: every other unread message is marked as read on each nnimap group refresh 2016-02-07 5:57 ` Lars Ingebrigtsen @ 2016-02-08 19:05 ` Nix 2016-02-09 0:57 ` Lars Ingebrigtsen 2016-06-11 16:16 ` Nix 1 sibling, 1 reply; 11+ messages in thread From: Nix @ 2016-02-08 19:05 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: 21182 On 7 Feb 2016, Lars Ingebrigtsen told this: > Nick Alcock <nix@esperi.org.uk> writes: > >> This happened after my first Emacs update for nearly a year (a bit of a >> 'big bang'), so it could have come from almost any change to nnimap >> since September last year. I'll do a bisection and hunt for it. (It's >> not like I can do any more damage to my imap read marks than I already >> have! I wanted to get to 'inbox zero', but not this way...) > > Are you still seeing this problem? It's hard to say -- I reverted the patch locally immediately. The problem is that whenever this goes wrong, 50% of my unread email gets marked as read, with no obvious way to reverse this. This is rather a high price to pay for replicating a bug :) I have some sacrificial folders I could give up to the cause -- I'll see if I can make things go wrong with them in the next day or so. -- NULL && (void) ^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#21182: 25.0.50; gnus: every other unread message is marked as read on each nnimap group refresh 2016-02-08 19:05 ` Nix @ 2016-02-09 0:57 ` Lars Ingebrigtsen 0 siblings, 0 replies; 11+ messages in thread From: Lars Ingebrigtsen @ 2016-02-09 0:57 UTC (permalink / raw) To: Nix; +Cc: 21182 Nix <nix@esperi.org.uk> writes: > It's hard to say -- I reverted the patch locally immediately. The > problem is that whenever this goes wrong, 50% of my unread email gets > marked as read, with no obvious way to reverse this. This is rather a > high price to pay for replicating a bug :) > > I have some sacrificial folders I could give up to the cause -- I'll see > if I can make things go wrong with them in the next day or so. I just don't understand how the following patch could have this effect... diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el index ced5561..8e81abc 100644 --- a/lisp/gnus/nnimap.el +++ b/lisp/gnus/nnimap.el @@ -166,14 +166,21 @@ nnimap-buffer (nnimap-find-process-buffer nntp-server-buffer)) (defun nnimap-header-parameters () - (format "(UID RFC822.SIZE BODYSTRUCTURE %s)" - (format + (let (params) + (push "UID" params) + (push "RFC822.SIZE" params) + (when (nnimap-capability "X-GM-EXT-1") + (push "X-GM-LABELS" params)) So it only adds that to the params if the server supports it... + (push "BODYSTRUCTURE" params) + (push (format (if (nnimap-ver4-p) "BODY.PEEK[HEADER.FIELDS %s]" "RFC822.HEADER.LINES %s") (append '(Subject From Date Message-Id References In-Reply-To Xref) - nnmail-extra-headers)))) + nnmail-extra-headers)) + params) + (format "%s" (nreverse params)))) So on non-Gmail servers the result should be identical... - (let (article lines size string) + (let (article lines size string labels) (block nil (while (not (eobp)) (while (not (looking-at "\\* [0-9]+ FETCH")) @@ -232,6 +239,9 @@ nnimap-transform-headers t) (match-string 1))) (beginning-of-line) + (when (search-forward "X-GM-LABELS" (line-end-position) t) + (setq labels (ignore-errors (read (current-buffer))))) + (beginning-of-line) Again, it should have no effect on non-Gmail... (when lines (insert (format "Lines: %s\n" lines))) + (when labels + (insert (format "X-GM-LABELS: %s\n" labels))) ;; Most servers have a blank line after the headers, but ;; Davmail doesn't. (unless (re-search-forward "^\r$\\|^)\r?$" nil t) And ditto. So it's a mystery to me why that patch should affect anything unless you're using Gmail.. -- (domestic pets only, the antidote for overdose, milk.) bloggy blog: http://lars.ingebrigtsen.no ^ permalink raw reply related [flat|nested] 11+ messages in thread
* bug#21182: 25.0.50; gnus: every other unread message is marked as read on each nnimap group refresh 2016-02-07 5:57 ` Lars Ingebrigtsen 2016-02-08 19:05 ` Nix @ 2016-06-11 16:16 ` Nix 1 sibling, 0 replies; 11+ messages in thread From: Nix @ 2016-06-11 16:16 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: 21182 On 7 Feb 2016, Lars Ingebrigtsen told this: > Nick Alcock <nix@esperi.org.uk> writes: > >> This happened after my first Emacs update for nearly a year (a bit of a >> 'big bang'), so it could have come from almost any change to nnimap >> since September last year. I'll do a bisection and hunt for it. (It's >> not like I can do any more damage to my imap read marks than I already >> have! I wanted to get to 'inbox zero', but not this way...) > > Are you still seeing this problem? Just retested with trunk. It still happens. Here's a smoking gun (on a folder where I don't care about the read marks!). I marked no mails as read in this time, just went into the group and out of it again. Have a pile of debugging on the off-chance that you'll be able to figure out the problem before I can even finish analyzing it :) Enter group: 16:18:00 [stbeehive.oracle.com] 67 SELECT "Unrelated/Bugs" 16:18:01 [stbeehive.oracle.com] 68 SELECT "Unrelated/Bugs" 16:18:01 [stbeehive.oracle.com] 69 UID FETCH 4380,4910,5279,5788,6273,6790,7314,7610,7615,7619,7623,7629,7633,7637,7641,7645,7650,7653,7657,7661,7665,7669,7673,7677,7681,7685,7689,7695,7699,7703,7707,7711,7715,7720:7721,7725,7729,7733,7737,7741,7744,7749,7753,7757,7761,7765,7768:7769,7773,7777,7781:7782,7786,7790,7795,7802,7807,7812,7819,7824,7828,7832,7836,7841,7845,7849,7853,7857,7861,7865:7866,7869,7873,7877,7881,7885,7889,7894,7900,7904,7908,7911,7916,7920,7924,7928,7933,7937,7941,7945,7949,7954,7958,7962,7966,7971,7975,7979,7983,7987,7992,7996,8001,8005,8009,8013,8018,8022,8028,8032,8036,8040,8044,8048,8052,8059,8063,8067,8071,8075,8079,8085,8089,8093,8098,8102,8106,8110,8114,8118,8122,8126,8130,8134,8138,8142,8146,8150,8152,8156,8160,8164,8168,8172,8176,8180,8184:8185,8189,8194,8198,8202,8205,8212,8216,8220,8224,8228,8232,8237,8242,8248,8253:8254,8258,8262,8269,8275,8279,8283,8288,8294,8300,8304,8310,8315,8319,8323,8329,8333,8338,8342,8348,8352,8355,8360,8365,8367,8373,8377,8381,8386,8390,8395,8399,8404:8405,8409,8414,8419,8423,8427,8431,8436,8441,8445,8451,8455,8461:8462,8467,8471,8475,8479,8484,8488,8494,8498,8502,8506,8510,8514,8518,8522,8527,8531,8535,8540,8544,8548,8552,8556,8560,8564,8568,8572:8573,8577,8581,8584,8588,8591:8592,8596:8597,8601,8606,8610,8615,8619,8623,8628,8632,8636,8642,8646,8650,8655,8659,8669,8673,8677,8681,8685,8689,8694,8700,8705,8713,8717 (UID RFC822.SIZE BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (Subject From Date Message-Id References In-Reply-To Xref X-Spam-Status To Cc Keywords Gcc Newsgroups X-GM-LABELS)]) Exit group, having read not a single article: 16:18:26 [stbeehive.oracle.com] 70 SELECT "Unrelated/Bugs" 16:18:26 [stbeehive.oracle.com] 71 UID STORE 4910,5788,6790,7610,7619,7629,7637,7645,7653,7661,7669,7677,7685,7695,7703,7711,7720,7725,7733,7741,7749,7757,7765,7773,7781,7786,7795,7807,7819,7828,7836,7845,7853,7861,7869,7877,7885,7894,7904,7911,7920,7928,7937,7945,7954,7962,7971,7979,7987,7996,8005,8013,8022,8032,8040,8048,8059,8067,8075,8085,8093,8102,8110,8118,8126,8134,8142,8150,8156,8164,8172,8180,8189,8198,8205,8216,8224,8232,8242,8253,8258,8269,8279,8288,8300,8310,8319,8329,8338,8348,8355,8365,8373,8381,8390,8399,8409,8419,8427,8436,8445,8455,8467,8475,8484,8494,8502,8510,8518,8527,8535,8544,8552,8560,8568,8577,8584,8591,8596,8601,8610,8619,8628,8636,8646,8655,8669,8677,8685,8694,8705,8717 +FLAGS.SILENT (\Seen) Note that it appears to believe the thing has X-GM-LABELS. This is unfortunate, given that (nnimap-capabilities nnimap-object) returns ("IMAP4REV1" "IDLE" "AUTH=PLAIN"). No X-GM-EXT1. The articles received, according to the " *nnimap ..." buffer (which I had to extract from nnimap-process-buffers, I couldn't see it in the buffer list, what's up with that?) all have initial headers that look like this, and oh look X-GM-LABELS is there: Chars: 3533 Lines: 39 X-GM-LABELS: )] {313} and that latter header frankly looks like something very fubared that Gnus has added itself. In the absence of the buggy patch, we see these headers (content redacted): Chars: 8196 Lines: 148 Subject: [...] From: [...] Date: [...] Message-id: <E1aOM3t-0004jM-O0@verbose.us.oracle.com> To: [...] In its presence, we see this: Chars: 8196 Lines: 179 X-GM-LABELS: )] {278} Subject: [...] From: [...] Date: [...] Message-id: <E1aOM3t-0004jM-O0@verbose.us.oracle.com> To: [...] ) Note the trailing ), extra X-GM-LABELS, and adjusted lines and length: note also the unchanged Message-ID. nnimap-header-parameters returns the exact same content before and after the patch, so that can't be the problem. nnimap-transform-headers is clearly fubared. If I add a message call into nnimap-transform-headers: (when (search-forward "X-GM-LABELS" (line-end-position) t) (setq labels (ignore-errors (read (current-buffer)))) (message "set labels to %s" labels)) what do I see but set labels to )] {313} Oops. If I dump the line it's searching in, it becomes clear what's going on: labels: * 4358 FETCH (UID 4380 RFC822.SIZE 3533 BODYSTRUCTURE ("TEXT" "PLAIN" NIL NIL NIL "8BIT" 1869 46 NIL ("INLINE" NIL) NIL NIL) BODY[HEADER.FIELDS ("Subject" "From" "Date" "Message-Id" "References" "In-Reply-To" "Xref" "X-Spam-Status" "To" "Cc" "Keywords" "Gcc" "Newsgroups" "X-GM-LABELS")] {313} Youwhat? What's that even doing there? That's the header section: it should have been erased! Let's look at the *nnimap* buffer for that message right after the first entry into the group with the buggy patch in place: * 8196 FETCH (UID 8238 RFC822.SIZE 18125 BODYSTRUCTURE ("TEXT" "PLAIN" NIL NIL NIL "8BIT" 16480 378 NIL ("INLINE" NIL) NIL NIL) BODY[HEADER.FIELDS ("Subject" "From" "Date" "Message-Id" "References" "In-Reply-To" "Xref" "X-Spam-Status" "To" "Cc" "Keywords" "Gcc" "Newsgroups" "X-GM-LABELS")] {303} Subject: [...] From: [...] Date: [...] Message-id: <E1aOM3t-0004jM-O0@verbose.us.oracle.com> To: linuxrep_ww@oracle.com, linux_tools_us@oracle.com ) Not erased. It's clear where the the Chars: 8196 above came from now, and the X-GM-LABELS header is coming from the same place. The nnimap buffer after transformation displays an alternating, on-again, off-again character, which matches the "every other marked as read" symptoms: 211 4380 Article retrieved. Chars: 3533 Lines: 14 X-GM-LABELS: )] {313} Subject: [...] From: [...] Date: [...] Message-id: <E1VjhEK-0003wo-Dk@verbose.us.oracle.com> To: [...] ) * 7655 FETCH (UID 7697 RFC822.SIZE 2045 BODYSTRUCTURE ( Subject: [...] From: [...] Date: [...] Message-id: <E1ZRnyM-0002bO-BJ@verbose.us.oracle.com> To: linuxrep_ww@oracle.com, linux_tools_us@oracle.com . * 7656 FETCH (UID 7698 RFC822.SIZE 2492 BODYSTRUCTURE ("TEXT" "PLAIN" NIL NIL NIL "8BIT" 851 30 NIL ("INLINE" NIL) NIL NIL) BODY[HEADER.FIELDS ("Subject" "From" "Date" "Message-Id" "References" "In-Reply-To" "Xref" "X-Spam-Status" "To" "Cc" "Keywords" "Gcc" "Newsgroups" "X-GM-LABELS")] {302} Subject: [...] From: [...] Date: [...] Message-id: <E1ZRxL0-0004SK-8R@verbose.us.oracle.com> To: linuxrep_ww@oracle.com, linux_tools_us@oracle.com ) Note that the brackets in the FETCH are not balanced: the open bracket after HEADER.FIELDS is closed, but never the BODY[] brackets nor the FETCH (... list bracket. -- NULL && (void) ^ permalink raw reply [flat|nested] 11+ messages in thread
* bug#21182: 25.0.50; gnus: every other unread message is marked as read on each nnimap group refresh 2015-08-03 13:01 bug#21182: 25.0.50; gnus: every other unread message is marked as read on each nnimap group refresh Nick Alcock [not found] ` <handler.21182.B.14386068914118.ack@debbugs.gnu.org> 2016-02-07 5:57 ` Lars Ingebrigtsen @ 2016-05-22 19:26 ` Paul Eggert 2 siblings, 0 replies; 11+ messages in thread From: Paul Eggert @ 2016-05-22 19:26 UTC (permalink / raw) To: Lars Ingebrigtsen; +Cc: 21182 From: Lars Ingebrigtsen <larsi@gnus.org> > + (when (nnimap-capability "X-GM-EXT-1") > + (push "X-GM-LABELS" params)) > > So it only adds that to the params if the server supports it... Suppose the newly-introduced call to nnimap-capability throws an exception or signals an error? Could that introduce the symptoms in question? (I don't really understand the code, just trying to help move this along...) ^ permalink raw reply [flat|nested] 11+ messages in thread
[parent not found: <mailman.7787.1438606938.904.bug-gnu-emacs@gnu.org>]
* Re: bug#21182: 25.0.50; gnus: every other unread message is marked as read on each nnimap group refresh [not found] <mailman.7787.1438606938.904.bug-gnu-emacs@gnu.org> @ 2016-08-20 2:15 ` joyperkins07 0 siblings, 0 replies; 11+ messages in thread From: joyperkins07 @ 2016-08-20 2:15 UTC (permalink / raw) To: bug-gnu-emacs On Monday, August 3, 2015 at 8:02:19 AM UTC-5, Nick Alcock wrote: > The bug is fairly easily characterized, and easy to spot in a group with > a great many consecutive unread messages: every time the group is > refreshed, half the unread messages are marked as read, in an > alternating pattern, so after one refresh the result runs (... unread, > read, unread, read...): after more than one refresh the result is harder > to see. > > After the first instance of this problem, we also see valid messages > apparently originating from nobody (their name appears as '(none)', and > other similarly bizarre messups indicating something very wrong with > IMAP communication. (Cc:ing Drew because the thing I was communicating > with was an SSL channel to the very same Beehive mailserver he uses -- I > know Drew doesn't use Gnus to post here, but maybe he sometimes looks at > his work mailbox via Gnus, in which case he might appreciate a warning!) > > This happened after my first Emacs update for nearly a year (a bit of a > 'big bang'), so it could have come from almost any change to nnimap > since September last year. I'll do a bisection and hunt for it. (It's > not like I can do any more damage to my imap read marks than I already > have! I wanted to get to 'inbox zero', but not this way...) > > In GNU Emacs 25.0.50.1 (x86_64-unknown-linux-gnu, X toolkit, Xaw3d scroll bars) > of 2015-08-01 on spindle > Repository revision: 08d65696c947b09d4c32606f279c3c2594e99e16 > Windowing system distributor `The X.Org Foundation', version 11.0.11702000 > System LSB Version: :core-4.1-amd64:core-4.1-noarch > > Configured using: > `configure --without-pop --without-kerberos --without-hesiod > --with-x-toolkit=lucid --with-wide-int --with-file-notification=inotify > NO_FAST_MATH=t 'CFLAGS=-Wstrict-overflow=1 -g -gdwarf-4 > -feliminate-unused-debug-types -fvar-tracking-assignments > -feliminate-dwarf2-dups -O2 -pipe -D__NO_STRING_INLINES > -D__NO_MATH_INLINES -D_LARGEFILE64_SOURCE' > 'CPPFLAGS=-D__NO_STRING_INLINES -D__NO_MATH_INLINES > -D_LARGEFILE64_SOURCE' 'LDFLAGS=-Wl,--enable-new-dtags > -Wl,--hash-style=gnu -Wl,-O1 -Wl,-z,relro -pipe'' > > Configured features: > XAW3D XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND DBUS GCONF GSETTINGS > NOTIFY ACL GNUTLS LIBXML2 FREETYPE LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS > LUCID X11 > > Important settings: > value of $LC_COLLATE: C > value of $LANG: en_GB.UTF-8 > locale-coding-system: utf-8-unix > > Major mode: Summary > > Minor modes in effect: > timeclock-mode-line-display: t > TeX-PDF-mode: t > magit-wip-after-apply-mode: t > global-undo-tree-mode: t > undo-tree-mode: t > diff-auto-refine-mode: t > global-semantic-mru-bookmark-mode: t > global-semanticdb-minor-mode: t > global-semantic-decoration-mode: t > global-semantic-idle-scheduler-mode: t > global-semantic-idle-local-symbol-highlight-mode: t > global-semantic-idle-summary-mode: t > global-srecode-minor-mode: t > global-ede-mode: t > global-semantic-highlight-func-mode: t > global-semantic-stickyfunc-mode: t > semantic-mode: t > global-git-gutter-mode: t > global-auto-complete-mode: t > global-cwarn-mode: t > which-function-mode: t > mv-shell-mode: t > desktop-save-mode: t > savehist-mode: t > winner-mode: t > Info-breadcrumbs-in-mode-line-mode: t > shell-dirtrack-mode: t > recentf-mode: t > display-time-mode: t > override-global-mode: t > image-diredx-async-mode: t > tooltip-mode: t > global-eldoc-mode: t > electric-indent-mode: t > mouse-wheel-mode: t > menu-bar-mode: t > file-name-shadow-mode: t > global-font-lock-mode: t > font-lock-mode: t > blink-cursor-mode: t > auto-composition-mode: t > auto-encryption-mode: t > auto-compression-mode: t > buffer-read-only: t > column-number-mode: t > line-number-mode: t > > Recent messages: > Auto-saving...done > Auto-saving...done > Quit > Mark set > previous-line: Beginning of buffer [4 times] > Mark set > C-x C-g is undefined > C-M-g is undefined > > Load-path shadows: > /home/nix/lisp/defaults hides /usr/share/emacs/site-lisp/defaults > /home/nix/lisp/emacs/scroll-in-place hides /usr/share/emacs/site-lisp/scroll-in-place > /usr/share/emacs/site-lisp/emms/tq hides /usr/share/emacs/25.0.50/lisp/emacs-lisp/tq > > Features: > (shadow emacsbug canlock gnus-html flow-fill mm-archive eieio-opt > mailalias misearch multi-isearch nnir gnus-picon gnus-cite qp gnus-async > gnus-bcklg gnus-salt gnus-dup gnus-ml disp-table semantic/tag-write > gnus-topic nndraft gnus-agent gnus-srvr gnus-score score-mode nnvirtual > gnus-msg utf-7 gnutls nnmh nnfolder gnus-cache bbdb-gnus bbdb-snarf > gnus-demon nntp gnus/mail gnus-registry registry eieio-compat > gnus/splits nnmairix nnml smtpmail gnus-art mm-uu mml2015 mm-view > mml-smime smime dig gnus/articles init/time-tracking timeclock > auto-edit-substitute gnus/sa background gnus-sum gnus-group gnus-undo > gnus/bbdb gnus/colourization tc mail-extr gnus-start gnus-cloud nnimap > nnmail mail-source utf7 netrc nnoo parse-time gnus-spec gnus-int > gnus-range gnus-win face-remap add-log shr-color color network-stream > nsm starttls url-http tls url-gw url-cache url-auth eww mm-url gnus > gnus-ems nnheader url-queue shr dom subr-x browse-url epa-file epa epg > semantic/tag-file semantic/bovine/c hideif semantic/bovine/c-by > semantic/lex-spp semantic/bovine/gcc semantic/bovine > semantic/analyze/refs semantic/analyze semantic/scope > semantic/analyze/fcn vc-filewise vc-rcs generic tex-buf semantic/texi > tex-info tex dbus xml texinfo semantic/imenu semantic/sb semantic/sort > semantic/db-file data-debug cedet-files semantic/wisent/python > semantic/decorate/include semantic/dep semantic/wisent/python-wy rx > python json ede/dired jka-compr magit-wip magit-core magit-process > with-editor server magit-popup magit-mode magit-git crm magit-section > magit-utils dash undo-tree diff vc vc-dispatcher vc-git diff-mode > flymake checkdoc hideshow elisp-slime-nav init/emacs init/prog-modes > filecache inversion semantic/db-global semantic/symref/global > semantic/symref semantic/db-find semantic/db-ref cedet-global > semantic/mru-bookmark semantic/db-mode semantic/db > semantic/decorate/mode semantic/idle srecode/mode semantic/senator > semantic/format semantic/ctxt semantic/wisent semantic/wisent/wisent > semantic/decorate pulse srecode/insert srecode/filters srecode/args > srecode/find srecode/map srecode/ctxt semantic/tag-ls semantic/find > srecode/compile srecode/dictionary srecode/table ede/speedbar ede/files > ede ede/detect ede/base ede/auto ede/source eieio-base eieio-speedbar > speedbar sb-image ezimage dframe eieio-custom srecode > semantic/util-modes semantic/util semantic semantic/tag semantic/lex > semantic/fw mode-local cedet git-gutter-fringe fringe-helper git-gutter > auto-complete-config auto-complete popup cwarn gtags filladapt cc-mode > cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars > cc-defs which-func generic-x init/processes vlf-setup mv-shell > init/music emms-volume emms-volume-amixer emms-history emms-bookmarks > emms-metaplaylist-mode emms-browser sort emms-playlist-sort > emms-last-played emms-playing-time emms-stream-info emms-streams > emms-mode-line emms-playlist-limit emms-playlist-mode emms-player-mpd > emms-url url url-proxy url-privacy url-expand url-methods url-history > url-cookie url-domsuf url-util mailcap emms-cache emms-info later-do tq > emms-player-simple emms-source-playlist emms-source-file locate emms > emms-compat emms-auto init/messaging bbdb-expire bbdb-hooks silly-mail > sendmail message rfc822 mml mml-sec mm-decode mm-bodies mm-encode > mail-parse rfc2231 rfc2047 rfc2045 ietf-drums mail-utils gmm-utils > mailheader bbdb-autoloads init/state desktop frameset savehist saveplace > init/editing gpicker finder find-func lisp-mnt windmove framemove > miniedit winner scroll-in-place flash-paren init/faces nix-dark-theme > init/drew two-column icicles icicles-mode dired+ image-file dired-aux > mb-depth+ mb-depth icicles-cmd2 compile imenu completion icicles-cmd1 > bbdb-com warnings mailabbrev bbdb timezone yow etags xref project > cookie1 icicles-mcmd filesets icicles-fn icicles-var icicles-opt ffap > url-parse url-vars fuzzy-match el-swank-fuzzy levenshtein fuzzy > cus-theme cus-edit cus-start cus-load hexrgb icicles-face icicles-mac > bookmark+ bookmark+-key derived dired-x crosshairs col-highlight vline > hl-line+ hl-line bookmark+-1 bookmark+-bmu bookmark+-lit bookmark > menu-bar+ linkd doremi apropos-fn+var help-fns+ help+ frame-fns avoid > info+ thingatpt+ thingatpt mouse3 naked apropos+ apropos init/appearance > tramp-cache tramp-sh tramp tramp-compat auth-source eieio byte-opt > eieio-core gnus-util mm-util help-fns mail-prsvr password-cache > tramp-loaddefs trampver shell pcomplete comint ansi-color ring recentf > tree-widget wid-edit printing ps-print ps-def lpr time-date > init/modelines time uptimes pp init/options xemacs-compat misc > gawd-lists use-package-external use-package diminish bytecomp > byte-compile cl-extra help-mode seq cconv bind-key easy-mmode finder-inf > tex-site iedit-autoloads image-dired+ image-dired format-spec dired info > yasnippet-bundle cl-macs cl-seq dropdown-list advice yasnippet edmacro > kmacro easymenu assoc cl gv cl-loaddefs pcase cl-lib package epg-config > mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks > lisp-float-type mwheel x-win term/common-win x-dnd tool-bar dnd fontset > image regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode > prog-mode register page menu-bar rfn-eshadow timer select scroll-bar > mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham > georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao > korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech > european ethiopic indian cyrillic chinese charscript case-table epa-hook > jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice > loaddefs button faces cus-face macroexp files text-properties overlay > sha1 md5 base64 format env code-pages mule custom widget > hashtable-print-readable backquote dbusbind inotify dynamic-setting > system-font-setting font-render-setting x-toolkit x multi-tty > make-network-process emacs) > > Memory information: > ((conses 16 2245900 1795918) > (symbols 48 141793 238) > (miscs 40 6910 9463) > (strings 32 782235 1143702) > (string-bytes 1 34552691) > (vectors 16 100813) > (vector-slots 8 3950744 1469730) > (floats 8 2600 6311) > (intervals 56 50025 16112) > (buffers 976 145) > (heap 1024 313480 457073)) ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2017-01-26 22:32 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2015-08-03 13:01 bug#21182: 25.0.50; gnus: every other unread message is marked as read on each nnimap group refresh Nick Alcock [not found] ` <handler.21182.B.14386068914118.ack@debbugs.gnu.org> 2015-08-03 15:04 ` Nix 2016-02-07 5:56 ` Lars Ingebrigtsen 2017-01-26 21:07 ` Lars Ingebrigtsen 2017-01-26 22:32 ` Nix 2016-02-07 5:57 ` Lars Ingebrigtsen 2016-02-08 19:05 ` Nix 2016-02-09 0:57 ` Lars Ingebrigtsen 2016-06-11 16:16 ` Nix 2016-05-22 19:26 ` Paul Eggert [not found] <mailman.7787.1438606938.904.bug-gnu-emacs@gnu.org> 2016-08-20 2:15 ` joyperkins07
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).