bug#25126: 26.0.50; Gnus reacts poorly to email flags being changed behind its back

bug#25126: 26.0.50; Gnus reacts poorly to email flags being changed behind its back

[Daniel Colascione]

I'm trying an email setup that involves pointing Gnus as a maildir tree
and having mbsync run from cron and, well, sync with an IMAP server.
Sometimes, if I open a mail group in Gnus, read a message, and before
leaving that mail group, go read that message on another client, the
cron job will change the message-file's flags behind Gnus's back,
causing Gnus to raise an error when I try to leave the group,
complaining that a message file it thought existed no longer exists
(because it's been renamed).

Can Gnus cope better with other programs modifying messages concurrently?


In GNU Emacs 26.0.50.12 (x86_64-unknown-linux-gnu, GTK+ Version 3.10.8)
 of 2016-11-08 built on REDACTED
Repository revision: 06cfaa3dfa1888b55df437a16ced6f718678bc56
Windowing system distributor 'The X.Org Foundation', version 11.0.11702000
System Description:	Ubuntu 14.04.5 LTS

Recent messages:
gnus-group-read-group: No group on current line
Running mbsync to fetch news...
mbsync succeeded in 9.972675 seconds
Checking new news...
Reading active file via nnnil...done
nnimap read 0k from dancol.org
Reading active file from archive via nnfolder...done
Reading active file via nndraft...done
Checking new news...done
nnimap read 0k from dancol.org

Configured using:
 'configure --with-x-toolkit=gtk3 --without-toolkit-scroll-bars
 --with-modules --with-dbus 'CFLAGS=-O2 -march=native -g3 -flto'
 RANLIB=gcc-ranlib AR=gcc-ar'

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND DBUS GCONF GSETTINGS NOTIFY LIBSELINUX
GNUTLS LIBXML2 FREETYPE XFT ZLIB GTK3 X11 MODULES

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Group

Minor modes in effect:
  hl-line-mode: t
  gnus-undo-mode: t
  save-place-mode: t
  which-function-mode: t
  winner-mode: t
  savehist-mode: t
  show-paren-mode: t
  global-hi-lock-mode: t
  hi-lock-mode: t
  iswitchb-mode: t
  global-subword-mode: t
  subword-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Load-path shadows:
~/emacs/override/iswitchb hides /home/dancol/edev/trunk/lisp/obsolete/iswitchb
/home/dancol/edev/trunk/lisp/linum hides ~/emacs/misc/linum
/home/dancol/edev/trunk/lisp/textmodes/css-mode hides ~/emacs/misc/css-mode
~/emacs/override/autoloads hides ~/emacs/misc/autoloads
~/emacs/override/autoloads hides ~/emacs/s/autoloads
~/emacs/override/autoloads hides ~/emacs/pcre2el/autoloads
~/emacs/override/autoloads hides ~/emacs/ws-butler/autoloads
~/emacs/override/autoloads hides ~/emacs/logcat/autoloads
~/emacs/override/autoloads hides ~/emacs/gnuplot/autoloads
~/emacs/override/autoloads hides ~/emacs/go-mode/autoloads
~/emacs/override/autoloads hides ~/emacs/groovy-emacs-modes/autoloads
~/emacs/override/autoloads hides ~/emacs/gnus-alias/autoloads
~/emacs/override/autoloads hides ~/emacs/elpa-used/ack/autoloads
~/emacs/override/autoloads hides ~/emacs/elpa-used/company/autoloads
~/emacs/override/autoloads hides ~/emacs/elpa-used/nlinum/autoloads
~/emacs/override/autoloads hides ~/emacs/elpa-used/num3-mode/autoloads
~/emacs/override/autoloads hides ~/emacs/elpa-used/register-list/autoloads
~/emacs/override/autoloads hides ~/emacs/elpa-used/temp-buffer-browse/autoloads
~/emacs/override/autoloads hides ~/emacs/elpa-used/undo-tree/autoloads
~/emacs/override/autoloads hides ~/emacs/elpa-used/yasnippet/autoloads

Features:
(shadow flyspell ispell emacsbug sendmail shr-color color gnus-cite
smiley ansi-color url-util url-parse url-vars qp shr svg xml dom
browse-url mm-archive mail-extr gnus-bcklg sort gnus-ml disp-table
hl-line nndraft nnmh nnmaildir utf-7 network-stream nsm auth-source
cl-seq eieio eieio-core cl-macs eieio-loaddefs starttls nnfolder nnnil
gnus-agent gnus-srvr gnus-score score-mode nnvirtual gnus-cache
gnus-demon gnus-msg gnus-art mm-uu mml2015 mm-view mml-smime smime dig
mailcap gnus-alias advice gnus-async nntp gnus-sum gnus-group gnus-undo
gnus-start gnus-cloud nnimap nnmail mail-source tls gnutls utf7 netrc
nnoo parse-time gnus-spec gnus-int gnus-range message puny seq byte-opt
bytecomp byte-compile cl-extra help-mode cconv dired dired-loaddefs
format-spec rfc822 mml mml-sec password-cache epa derived epg epg-config
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader gnus-win gnus nnheader subr-x gnus-util rmail rmail-loaddefs
rfc2047 rfc2045 ietf-drums mail-utils mm-util mail-prsvr autoloads
dabbrev saveplace which-func imenu server temp-buffer-browse winner ring
savehist paren hi-lock iswitchb bar-cursor uptime env-shell-command
cap-words superword subword gv time-date mule-util cus-edit easymenu
cus-start cus-load wid-edit cl-loaddefs pcase cl-lib tooltip eldoc
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel term/x-win
x-win term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list newcomment elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese charscript case-table epa-hook
jka-cmpr-hook help simple abbrev obarray minibuffer cl-preloaded nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote dbusbind inotify dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty make-network-process emacs)

Memory information:
((conses 16 721145 128352)
 (symbols 48 153426 193)
 (miscs 40 650 1698)
 (strings 32 421493 44995)
 (string-bytes 1 41518202)
 (vectors 16 148737)
 (vector-slots 8 1630936 87147)
 (floats 8 644 1631)
 (intervals 56 1822 775)
 (buffers 976 48)
 (heap 1024 134554 8463))

bug#25126: 26.0.50; Gnus reacts poorly to email flags being changed behind its back

[Lars Ingebrigtsen]

Daniel Colascione <dancol@dancol.org> writes:

> I'm trying an email setup that involves pointing Gnus as a maildir tree
> and having mbsync run from cron and, well, sync with an IMAP server.
> Sometimes, if I open a mail group in Gnus, read a message, and before
> leaving that mail group, go read that message on another client, the
> cron job will change the message-file's flags behind Gnus's back,
> causing Gnus to raise an error when I try to leave the group,
> complaining that a message file it thought existed no longer exists
> (because it's been renamed).
>
> Can Gnus cope better with other programs modifying messages concurrently?

Just to clarify -- these problems are with nnmaildir and not nnimap?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#25126: 26.0.50; Gnus reacts poorly to email flags being changed behind its back

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Daniel Colascione <dancol@dancol.org> writes:
>
>> I'm trying an email setup that involves pointing Gnus as a maildir tree
>> and having mbsync run from cron and, well, sync with an IMAP server.
>> Sometimes, if I open a mail group in Gnus, read a message, and before
>> leaving that mail group, go read that message on another client, the
>> cron job will change the message-file's flags behind Gnus's back,
>> causing Gnus to raise an error when I try to leave the group,
>> complaining that a message file it thought existed no longer exists
>> (because it's been renamed).
>>
>> Can Gnus cope better with other programs modifying messages concurrently?
>
> Just to clarify -- these problems are with nnmaildir and not nnimap?

And three days after that I renamed this bug to

retitle 25126 nndir reacts poorly to email flags being changed behind its back

which must be wrong.  I mean, it's either nnmaildir or nnimap.

So I have to ask again -- what Gnus backend is this about, and what are
the error messages?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#25126: 26.0.50; Gnus reacts poorly to email flags being changed behind its back

[Glenn Morris]

Lars Ingebrigtsen wrote:

> So I have to ask again -- what Gnus backend is this about, and what are
> the error messages?

The example in the OP is about nnmaildir.

I don't think the details are relevant, but it happens to be being
synced from imap by an external process. For example, a maildir file
might change name from "stuff:2,S" to "stuff:2,RS" if you reply to the
message via an imap client. Or it could have been moved to a different
folder, or deleted altogether. It's not obvious to me what Gnus could do
in this situation.

I imagine you could reproduce it by visiting a Maildir with Gnus, then
renaming one of the files externally.

I imagine none of the Gnus backends (except imap) are prepared for files
to be renamed by other processes while Gnus is running?

bug#25126: 26.0.50; Gnus reacts poorly to email flags being changed behind its back

[Lars Ingebrigtsen]

Glenn Morris <rgm@gnu.org> writes:

> The example in the OP is about nnmaildir.
>
> I don't think the details are relevant, but it happens to be being
> synced from imap by an external process. For example, a maildir file
> might change name from "stuff:2,S" to "stuff:2,RS" if you reply to the
> message via an imap client. Or it could have been moved to a different
> folder, or deleted altogether. It's not obvious to me what Gnus could do
> in this situation.

Ah, thanks for the explanation.

> I imagine you could reproduce it by visiting a Maildir with Gnus, then
> renaming one of the files externally.

I don't have any maildir dirs, which probably explains why all the
nnmaildir bug reports are languishing.  Could you pack up a simple one
and mail it to me as a tar file?

> I imagine none of the Gnus backends (except imap) are prepared for files
> to be renamed by other processes while Gnus is running?

Hm...  it depends.  I mean, ideally a `g' should feed Gnus any changed
information, but it might require that the backend in question implement
the "extended" interface that nnimap uses.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#25126: 26.0.50; Gnus reacts poorly to email flags being changed behind its back

[Glenn Morris]

Lars Ingebrigtsen wrote:

> I don't have any maildir dirs, which probably explains why all the
> nnmaildir bug reports are languishing.  Could you pack up a simple one
> and mail it to me as a tar file?

I will send you my spam folder off-list, you lucky person.

BTW, if you have imap, you can have maildir fairly easily with eg

http://www.offlineimap.org
http://isync.sourceforge.net/

bug#25126: 26.0.50; Gnus reacts poorly to email flags being changed behind its back

[Eric Abrahamsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Glenn Morris <rgm@gnu.org> writes:
>
>> The example in the OP is about nnmaildir.
>>
>> I don't think the details are relevant, but it happens to be being
>> synced from imap by an external process. For example, a maildir file
>> might change name from "stuff:2,S" to "stuff:2,RS" if you reply to the
>> message via an imap client. Or it could have been moved to a different
>> folder, or deleted altogether. It's not obvious to me what Gnus could do
>> in this situation.
>
> Ah, thanks for the explanation.
>
>> I imagine you could reproduce it by visiting a Maildir with Gnus, then
>> renaming one of the files externally.
>
> I don't have any maildir dirs, which probably explains why all the
> nnmaildir bug reports are languishing.  Could you pack up a simple one
> and mail it to me as a tar file?

I won't keep on with this, but the gnus-mock package in ELPA will give
you a clean, reproducible Gnus installation with a maildir server that
you can use for testing. I promise I'll shut up about it after this :)

bug#25126: 26.0.50; Gnus reacts poorly to email flags being changed behind its back

[Lars Ingebrigtsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> I won't keep on with this, but the gnus-mock package in ELPA will give
> you a clean, reproducible Gnus installation with a maildir server that
> you can use for testing. I promise I'll shut up about it after this :)

Thanks; haven't tried that yet...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no