bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Jean Louis]

Hello,

I am using Maildirs on my system. And I have 47682 various maildirs,
each belong to one email in the ordered way like:

~/Maildir/email1@example.com
~/Maildir/email2@example.com
~/Maildir/email3@example.com
~/Maildir/email4@example.com

and so on.

Gnus offers nice interface and functions which I would like to use
while reading email. Even though I like MH-E and Rmail more, they both
do not offer Maildir support.

I have made settings as following.

'(gnus-secondary-select-methods
   '((nnmaildir ""
	(directory "/home/data1/protected/Maildir/"))))
	
 '(gnus-select-method '(nnimap "my.imap"))

Now, I do not need to susbcribe to 47682 Maildirs at once, as under
~/Maildir I have cur, new, tmp and that is the Maildir I would like to
read as only one.

However, after setting the above, Gnus started doing something since
yesterday, and I still do not know what it is, it is maybe indexing or
setting up something, I do not know, process is still running for many
hours.

I think that this is bug.

What I think is that Gnus is now recursively visiting all Maildirs
instead of using just the main one.

Maybe there shall be option for the user to customize Gnus so that it
stops doing this recursively.

I have filed this as bug.

And my other question is, is there a way to quickly access
Maildir/email2@example.com by using Gnus? Some function maybe to just
write the email address or fetch it from database, and to open the
Maildir with nnmaildir?

Jean



In GNU Emacs 27.0.50 (build 10, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2019-07-16 built on protected.rcdrun.com
Repository revision: 288e83ae00fcf6da8064ecb5a8ffaec202c5e9ec
Repository branch: master
System Description: Hyperbola GNU/Linux-libre

Recent messages:
Auto-saving...done
Mark saved where search started [2 times]
Mark set [2 times]
Auto-saving...done
Auto-saving...done
Auto-saving...done
next-line: End of buffer
Auto-saving...done
Auto-saving...done
Making completion list...

Configured using:
 'configure --prefix=/package/text/emacs-2019-07-16 --with-modules
 --without-pop --with-mailutils --with-x-toolkit=lucid'

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG SOUND GPM DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL GNUTLS LIBXML2 FREETYPE HARFBUZZ M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES THREADS JSON PDUMPER
LCMS2 GMP

Important settings:
  value of $LC_ALL: de_DE.UTF-8
  value of $LANG: de_DE.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/home/data1/protected/.emacs.d/elpa/lispy-20190703.1529/elpa hides /home/data1/protected/.emacs.d/elpa/ivy-20190709.740/elpa
/home/data1/protected/Programming/emacs-lisp/whois hides /home/data1/protected/.emacs.d/elpa/whois-20190529.1554/whois
/home/data1/protected/.emacs.d/elpa/flim-20190526.1034/md4 hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/md4
/home/data1/protected/.emacs.d/elpa/flim-20190526.1034/hex-util hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/hex-util
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-css hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-css
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-dot hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-dot
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-sed hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-sed
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-stan hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-stan
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-sqlite hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-sqlite
/home/data1/protected/.emacs.d/elpa/org-20190708/org-src hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-src
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-lob hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-lob
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-calc hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-calc
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-mscgen hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-mscgen
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-core hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-core
/home/data1/protected/.emacs.d/elpa/org-20190708/ox-beamer hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ox-beamer
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-sass hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-sass
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-plantuml hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-plantuml
/home/data1/protected/.emacs.d/elpa/org-20190708/org-bibtex hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-bibtex
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-coq hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-coq
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-js hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-js
/home/data1/protected/.emacs.d/elpa/org-20190708/org-plot hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-plot
/home/data1/protected/.emacs.d/elpa/org-20190708/org-macro hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-macro
/home/data1/protected/.emacs.d/elpa/org-20190708/org-inlinetask hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-inlinetask
/home/data1/protected/.emacs.d/elpa/org-20190708/org-timer hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-timer
/home/data1/protected/.emacs.d/elpa/org-20190708/ox hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ox
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-forth hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-forth
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-groovy hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-groovy
/home/data1/protected/.emacs.d/elpa/org-20190708/org-bbdb hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-bbdb
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-perl hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-perl
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-gnuplot hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-gnuplot
/home/data1/protected/.emacs.d/elpa/org-20190708/ox-latex hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ox-latex
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-sql hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-sql
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-screen hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-screen
/home/data1/protected/.emacs.d/elpa/org-20190708/org-mhe hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-mhe
/home/data1/protected/.emacs.d/elpa/org-20190708/org-archive hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-archive
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-haskell hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-haskell
/home/data1/protected/.emacs.d/elpa/org-20190708/org-footnote hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-footnote
/home/data1/protected/.emacs.d/elpa/org-20190708/org-eww hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-eww
/home/data1/protected/.emacs.d/elpa/org-20190708/ox-man hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ox-man
/home/data1/protected/.emacs.d/elpa/org-20190708/org-protocol hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-protocol
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-ref hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-ref
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-processing hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-processing
/home/data1/protected/.emacs.d/elpa/org-20190708/org-habit hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-habit
/home/data1/protected/.emacs.d/elpa/org-20190708/org-indent hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-indent
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-maxima hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-maxima
/home/data1/protected/.emacs.d/elpa/org-20190708/org-list hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-list
/home/data1/protected/.emacs.d/elpa/org-20190708/org-entities hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-entities
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-fortran hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-fortran
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-comint hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-comint
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-ruby hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-ruby
/home/data1/protected/.emacs.d/elpa/org-20190708/org hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org
/home/data1/protected/.emacs.d/elpa/org-20190708/org-irc hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-irc
/home/data1/protected/.emacs.d/elpa/org-20190708/org-macs hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-macs
/home/data1/protected/.emacs.d/elpa/org-20190708/org-agenda hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-agenda
/home/data1/protected/.emacs.d/elpa/org-20190708/ox-org hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ox-org
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-C hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-C
/home/data1/protected/.emacs.d/elpa/org-20190708/org-install hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-install
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-makefile hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-makefile
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-java hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-java
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-org hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-org
/home/data1/protected/.emacs.d/elpa/org-20190708/org-table hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-table
/home/data1/protected/.emacs.d/elpa/org-20190708/ob hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob
/home/data1/protected/.emacs.d/elpa/org-20190708/org-info hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-info
/home/data1/protected/.emacs.d/elpa/org-20190708/org-id hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-id
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-eval hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-eval
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-clojure hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-clojure
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-ledger hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-ledger
/home/data1/protected/.emacs.d/elpa/org-20190708/org-w3m hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-w3m
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-shen hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-shen
/home/data1/protected/.emacs.d/elpa/org-20190708/org-docview hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-docview
/home/data1/protected/.emacs.d/elpa/org-20190708/ox-ascii hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ox-ascii
/home/data1/protected/.emacs.d/elpa/org-20190708/ox-publish hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ox-publish
/home/data1/protected/.emacs.d/elpa/org-20190708/ox-texinfo hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ox-texinfo
/home/data1/protected/.emacs.d/elpa/org-20190708/org-duration hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-duration
/home/data1/protected/.emacs.d/elpa/org-20190708/org-colview hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-colview
/home/data1/protected/.emacs.d/elpa/org-20190708/org-datetree hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-datetree
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-vala hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-vala
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-table hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-table
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-tangle hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-tangle
/home/data1/protected/.emacs.d/elpa/org-20190708/org-pcomplete hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-pcomplete
/home/data1/protected/.emacs.d/elpa/org-20190708/org-version hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-version
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-R hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-R
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-picolisp hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-picolisp
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-lua hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-lua
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-keys hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-keys
/home/data1/protected/.emacs.d/elpa/org-20190708/ox-odt hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ox-odt
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-awk hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-awk
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-exp hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-exp
/home/data1/protected/.emacs.d/elpa/org-20190708/ox-md hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ox-md
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-abc hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-abc
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-ocaml hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-ocaml
/home/data1/protected/.emacs.d/elpa/org-20190708/org-crypt hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-crypt
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-python hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-python
/home/data1/protected/.emacs.d/elpa/org-20190708/ox-html hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ox-html
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-matlab hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-matlab
/home/data1/protected/.emacs.d/elpa/org-20190708/org-attach hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-attach
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-hledger hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-hledger
/home/data1/protected/.emacs.d/elpa/org-20190708/org-loaddefs hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-loaddefs
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-octave hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-octave
/home/data1/protected/.emacs.d/elpa/org-20190708/org-ctags hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-ctags
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-asymptote hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-asymptote
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-ditaa hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-ditaa
/home/data1/protected/.emacs.d/elpa/org-20190708/org-gnus hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-gnus
/home/data1/protected/.emacs.d/elpa/org-20190708/org-compat hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-compat
/home/data1/protected/.emacs.d/elpa/org-20190708/org-feed hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-feed
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-J hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-J
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-shell hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-shell
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-lilypond hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-lilypond
/home/data1/protected/.emacs.d/elpa/org-20190708/org-rmail hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-rmail
/home/data1/protected/.emacs.d/elpa/org-20190708/org-element hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-element
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-io hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-io
/home/data1/protected/.emacs.d/elpa/org-20190708/org-faces hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-faces
/home/data1/protected/.emacs.d/elpa/org-20190708/org-capture hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-capture
/home/data1/protected/.emacs.d/elpa/org-20190708/org-eshell hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-eshell
/home/data1/protected/.emacs.d/elpa/org-20190708/org-lint hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-lint
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-lisp hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-lisp
/home/data1/protected/.emacs.d/elpa/org-20190708/org-clock hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-clock
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-ebnf hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-ebnf
/home/data1/protected/.emacs.d/elpa/org-20190708/org-mobile hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-mobile
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-scheme hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-scheme
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-latex hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-latex
/home/data1/protected/.emacs.d/elpa/org-20190708/ob-emacs-lisp hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ob-emacs-lisp
/home/data1/protected/.emacs.d/elpa/org-20190708/org-mouse hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/org-mouse
/home/data1/protected/.emacs.d/elpa/org-20190708/ox-icalendar hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/org/ox-icalendar
/home/data1/protected/.emacs.d/elpa/flim-20190526.1034/ntlm hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/net/ntlm
/home/data1/protected/.emacs.d/elpa/flim-20190526.1034/sasl-digest hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/net/sasl-digest
/home/data1/protected/.emacs.d/elpa/flim-20190526.1034/hmac-md5 hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/net/hmac-md5
/home/data1/protected/.emacs.d/elpa/flim-20190526.1034/sasl-cram hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/net/sasl-cram
/home/data1/protected/.emacs.d/elpa/flim-20190526.1034/sasl hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/net/sasl
/home/data1/protected/.emacs.d/elpa/flim-20190526.1034/hmac-def hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/net/hmac-def
/home/data1/protected/.emacs.d/elpa/flim-20190526.1034/sasl-ntlm hides /package/text/emacs-2019-07-16/share/emacs/27.0.50/lisp/net/sasl-ntlm

Features:
(shadow sort mail-extr emacsbug message rmc puny rfc822 mml mml-sec epa
derived epg gnus-util rmail rmail-loaddefs text-property-search
mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils
mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr
mail-utils misearch multi-isearch term disp-table ehelp shell vc-git
diff-mode rcd/business org-id org-element avl-tree generator
orgtbl-ascii-plot org-table org ob ob-tangle ob-ref ob-lob ob-table
ob-exp org-macro org-footnote org-src ob-comint ob-keys org-pcomplete
pcomplete comint ansi-color ring org-list org-faces org-entities
time-date noutline outline org-version ob-emacs-lisp ob-core ob-eval
org-compat org-macs org-loaddefs format-spec find-func cal-menu calendar
cal-loaddefs dired-x dired dired-loaddefs rcd/utilities rcd/percentages
rcd-db pq time-stamp helm easy-mmode edmacro kmacro helm-source
helm-multi-match helm-lib async finder-inf gh-common marshal
eieio-compat advice slime-autoloads cl tex-site info package easymenu
epg-config url-handlers url-parse auth-source cl-seq eieio eieio-core
cl-macs eieio-loaddefs password-cache json subr-x map url-vars seq
byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib term/xterm
xterm elec-pair tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 221847 12510)
 (symbols 48 19657 1)
 (strings 32 114835 33097)
 (string-bytes 1 3597790)
 (vectors 16 26220)
 (vector-slots 8 303693 7912)
 (floats 8 140 374)
 (intervals 56 609 0)
 (buffers 992 15))

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Eric Abrahamsen]

Jean Louis <bugs@gnu.support> writes:

> Hello,
>
> I am using Maildirs on my system. And I have 47682 various maildirs,
> each belong to one email in the ordered way like:
>
> ~/Maildir/email1@example.com
> ~/Maildir/email2@example.com
> ~/Maildir/email3@example.com
> ~/Maildir/email4@example.com
>
> and so on.
>
> Gnus offers nice interface and functions which I would like to use
> while reading email. Even though I like MH-E and Rmail more, they both
> do not offer Maildir support.
>
> I have made settings as following.
>
> '(gnus-secondary-select-methods
>    '((nnmaildir ""
> 	(directory "/home/data1/protected/Maildir/"))))
> 	
>  '(gnus-select-method '(nnimap "my.imap"))
>
> Now, I do not need to susbcribe to 47682 Maildirs at once, as under
> ~/Maildir I have cur, new, tmp and that is the Maildir I would like to
> read as only one.
>
> However, after setting the above, Gnus started doing something since
> yesterday, and I still do not know what it is, it is maybe indexing or
> setting up something, I do not know, process is still running for many
> hours.
>
> I think that this is bug.
>
> What I think is that Gnus is now recursively visiting all Maildirs
> instead of using just the main one.

I think you're right, and in a sense it is definitely a bug, but a lot
of people have run into this and I've seen some saying "there's not much
to be done". So that didn't sound very encouraging.

Could you do M-x toggle-debug-on-quit, start up Gnus, let it hang for a
bit, then do "C-g" and post the resulting backtrace here? It's fairly
obvious what's going on, but it would be good to see the specifics.

The nnmaildir servers keep track of their directory modtime, which is
both set and read only once, in `nnmaildir-request-scan'. So at least it
should be fairly easy to see what's happening there. Gnus doesn't save
the modtimes, though -- perhaps a potential solution could involve
saving the maildir modtimes in newsrc.eld.

> And my other question is, is there a way to quickly access
> Maildir/email2@example.com by using Gnus? Some function maybe to just
> write the email address or fetch it from database, and to open the
> Maildir with nnmaildir?

I'm not entirely sure what you mean here, but one suggestion I have is
to create four different nnmaildir select methods, one for each of your
email addresses. I think that's how Gnus is expecting this sort of thing
to be set up, and it might make it easier for you to access, as well.

Eric

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Jean Louis]

* Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-18 21:01]:
> I think you're right, and in a sense it is definitely a bug, but a lot
> of people have run into this and I've seen some saying there's not much
> to be done. So that didn't sound very encouraging.
> 
> Could you do M-x toggle-debug-on-quit, start up Gnus, let it hang for a
> bit, then do C-g and post the resulting backtrace here? It's fairly
> obvious what's going on, but it would be good to
> see the specifics.

Debugger entered--Lisp error: (quit)
  file-name-as-directory("/home/data1/protected/Maildir/info@codepink.org/.n...")
  nnmaildir--scan("info@codepink.org" nil #<hash-table equal 1913/47714 0xd41cb5> nil "/home/data1/protected/Maildir/" directory-files)
  nnmaildir-request-scan(find-new-groups "")
  nnmaildir-request-list("")
  nnmaildir-request-newgroups("Thu, 18 Jul 2019 18:11:09 +0200" "")
  gnus-request-newgroups("Thu, 18 Jul 2019 18:11:09 +0200" (nnmaildir "" (directory "~/Maildir")))
  gnus-ask-server-for-new-groups()
  gnus-find-new-newsgroups()
  gnus-setup-news(nil nil nil)
  #f(compiled-function () #<bytecode 0x995efd>)()
  gnus-1(nil nil nil)
  gnus(nil)
  funcall-interactively(gnus nil)
  call-interactively(gnus record nil)
  command-execute(gnus record)
  execute-extended-command(nil "gnus" "gnus")
  funcall-interactively(execute-extended-command nil "gnus" "gnus")
  call-interactively(execute-extended-command nil nil)
  command-execute(execute-extended-command)

> The nnmaildir servers keep track of their directory modtime, which is
> both set and read only once, in `nnmaildir-request-scan'. So at least it
> should be fairly easy to see what's happening there. Gnus doesn't save
> the modtimes, though -- perhaps a potential solution could involve
> saving the maildir modtimes in newsrc.eld.

Can that scan be turned off as option?

I do not even need to list the folders. I just
want to read the main ~/Maildir folder but not
sub-Maildirs within Gnus.

And then when at some point of time, when I need
to access the Maildir folder I should be able to
have function to give it folder directory and that
folder is being read, and that I can view messages
and reply to them.

Any clues? I would like to have that feature from
plain Emacs without extra packages, just to list,
read Maildirs, refile Maildirs, and be able to
reply.

Does that functionality exists within Gnus or
nnmaildir, even if not plainly visible?

> > And my other question is, is there a way to quickly access
> > Maildir/email2@example.com by using Gnus? Some function maybe to just
> > write the email address or fetch it from database, and to open the
> > Maildir with nnmaildir?
> 
> I'm not entirely sure what you mean here, but one suggestion I have is
> to create four different nnmaildir select methods, one for each of your
> email addresses. I think that's how Gnus is expecting this sort of thing
> to be set up, and it might make it easier for you to access, as well.

Would that be possible on the fly?

For example I have email address on screen, find
it at point and quickly switch to Maildir by email
address?

Jean

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Eric Abrahamsen]

On 07/18/19 21:43 PM, Jean Louis wrote:
> * Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-18 21:01]:
>> I think you're right, and in a sense it is definitely a bug, but a lot
>> of people have run into this and I've seen some saying there's not much
>> to be done. So that didn't sound very encouraging.
>> 
>> Could you do M-x toggle-debug-on-quit, start up Gnus, let it hang for a
>> bit, then do C-g and post the resulting backtrace here? It's fairly
>> obvious what's going on, but it would be good to
>> see the specifics.
>
> Debugger entered--Lisp error: (quit)
>   file-name-as-directory("/home/data1/protected/Maildir/info@codepink.org/.n...")
>   nnmaildir--scan("info@codepink.org" nil #<hash-table equal
> 1913/47714 0xd41cb5> nil "/home/data1/protected/Maildir/"
> directory-files)
>   nnmaildir-request-scan(find-new-groups "")
>   nnmaildir-request-list("")
>   nnmaildir-request-newgroups("Thu, 18 Jul 2019 18:11:09 +0200" "")
>   gnus-request-newgroups("Thu, 18 Jul 2019 18:11:09 +0200" (nnmaildir "" (directory "~/Maildir")))
>   gnus-ask-server-for-new-groups()
>   gnus-find-new-newsgroups()
>   gnus-setup-news(nil nil nil)
>   #f(compiled-function () #<bytecode 0x995efd>)()
>   gnus-1(nil nil nil)
>   gnus(nil)
>   funcall-interactively(gnus nil)
>   call-interactively(gnus record nil)
>   command-execute(gnus record)
>   execute-extended-command(nil "gnus" "gnus")
>   funcall-interactively(execute-extended-command nil "gnus" "gnus")
>   call-interactively(execute-extended-command nil nil)
>   command-execute(execute-extended-command)

I'm assuming your value of `gnus-check-new-newsgroups' is at its default
of 'ask-server. Try setting it to nil. That will (should) at least
prevent Gnus from scanning all the folders at startup. It doesn't solve
the underlying problem, but in your case it might avoid it.

>> The nnmaildir servers keep track of their directory modtime, which is
>> both set and read only once, in `nnmaildir-request-scan'. So at least it
>> should be fairly easy to see what's happening there. Gnus doesn't save
>> the modtimes, though -- perhaps a potential solution could involve
>> saving the maildir modtimes in newsrc.eld.
>
> Can that scan be turned off as option?
>
> I do not even need to list the folders. I just
> want to read the main ~/Maildir folder but not
> sub-Maildirs within Gnus.

The final setup you want is to be subscribed to the main folder, but
unsubscribed from the sub maildirs. The question is, can you get to the
point where you can do that without first having Gnus spend hours
scanning a million directories. Try the above fix and see if Gnus will
show you all the other (unwanted) directories.

[...]

>> > And my other question is, is there a way to quickly access
>> > Maildir/email2@example.com by using Gnus? Some function maybe to just
>> > write the email address or fetch it from database, and to open the
>> > Maildir with nnmaildir?
>> 
>> I'm not entirely sure what you mean here, but one suggestion I have is
>> to create four different nnmaildir select methods, one for each of your
>> email addresses. I think that's how Gnus is expecting this sort of thing
>> to be set up, and it might make it easier for you to access, as well.
>
> Would that be possible on the fly?
>
> For example I have email address on screen, find
> it at point and quickly switch to Maildir by email
> address?

We're talking about two different things here. One is defining each of
the maildirs as a separate server. So:

 '(gnus-secondary-select-methods
    '(
    (nnmaildir "email1"
 	(directory "/home/data1/protected/Maildir/email1@example.com/"))
    (nnmaildir "email2"
 	(directory "/home/data1/protected/Maildir/email2@example.com/"))
    (nnmaildir "email3"
 	(directory "/home/data1/protected/Maildir/email3@example.com/"))))

Etc.

The other thing you want -- quick switching to a particular group -- can
certainly be done, but will require a little bit of elisp. Something
like (totally untested):

(let* ((email (thing-at-point 'email))
       (account (car-safe (split-string email "@"))))
  (when account
    (gnus-group-read-group
     nil t (concat "nnmaildir+" account ":INBOX"))))

But see if you can get the servers functioning first!

Eric

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Jean Louis]

* Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-18 22:04]:
> I'm assuming your value of `gnus-check-new-newsgroups' is at its default
> of 'ask-server. Try setting it to nil. That will (should) at least
> prevent Gnus from scanning all the folders at startup. It doesn't solve
> the underlying problem, but in your case it
> might avoid it.

Thank you.

I did try to set it to nil, it is set. However,
"Checking new news" is seen and Gnus is working
with hard disk.

So it does not prevent it to scan Maildirs.

gnus-select-method is a variable defined in ‘gnus.el’.
Its value is (nnmaildir "" (directory "~/Maildir"))

> The final setup you want is to be subscribed to the main folder, but
> unsubscribed from the sub maildirs. The question is, can you get to the
> point where you can do that without first having Gnus spend hours
> scanning a million directories. Try the above fix and see if Gnus will
> show you all the other (unwanted) directories.

It does not work.

> We're talking about two different things here. One is defining each of
> the maildirs as a separate server. So:
> 
>  '(gnus-secondary-select-methods
>     '(
>     (nnmaildir email1
>  	(directory /home/data1/protected/Maildir/email1@example.com/))
>     (nnmaildir email2
>  	(directory /home/data1/protected/Maildir/email2@example.com/))
>     (nnmaildir email3
>  	(directory /home/data1/protected/Maildir/email3@example.com/))))
> 
> Etc.
> 
> The other thing you want -- quick switching to a particular group -- can
> certainly be done, but will require a little bit of elisp. Something
> like (totally untested):
> 
> (let* (( email "ss@rcdrrun.com")
>        (account (car-safe (split-string email @))))
>   (when account
>     (gnus-group-read-group
>      nil t (concat nnmaildir+ account :INBOX))))

I will try that, but I cannot get it without
starting to run or index whatever.

I was thinking that gnus-secondary-select-methods
has to be set before I start reading the folder,
and that above looks that you think that way.

I have tried this

(let* ((email "person@example.com")
	(gnus-secondary-select-methods '((nnmaildir email (directory (concat "~/Maildir/" email))))))
   (gnus-group-read-group nil t (concat "nnmaildir+" email :INBOX)))


but I am getting this below and I used real email
address. Do I need to assign the group somehow?

Debugger entered--Lisp error: (error "Couldn’t activate group nnmaildir+person@example.com: No such group: nnmaildir+person@example.com")
  signal(error ("Couldn’t activate group nnmaildir+person@example.com: No such group: nnmaildir+person@example.com"))
  error("Couldn't activate group %s: %s" "nnmaildir+person@example.com" "No such group: nnmaildir+person@example.com")
  gnus-select-newsgroup("nnmaildir+person@example.com" nil :INBOX)
  gnus-summary-read-group-1("nnmaildir+person@example.com" nil t nil nil :INBOX)
  gnus-summary-read-group("nnmaildir+person@example.com" nil t nil nil nil :INBOX)
  gnus-group-read-group(nil t "nnmaildir+person@example.com" :INBOX)

Jean

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Eric Abrahamsen]

On 07/18/19 22:48 PM, Jean Louis wrote:
> * Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-18 22:04]:
>> I'm assuming your value of `gnus-check-new-newsgroups' is at its default
>> of 'ask-server. Try setting it to nil. That will (should) at least
>> prevent Gnus from scanning all the folders at startup. It doesn't solve
>> the underlying problem, but in your case it
>> might avoid it.
>
> Thank you.
>
> I did try to set it to nil, it is set. However,
> "Checking new news" is seen and Gnus is working
> with hard disk.
>
> So it does not prevent it to scan Maildirs.

I guess that's not too surprising. If Gnus has never actually been able
to get off the ground, it's probably doing an initial scan of some sort.

> gnus-select-method is a variable defined in ‘gnus.el’.
> Its value is (nnmaildir "" (directory "~/Maildir"))
>
>> The final setup you want is to be subscribed to the main folder, but
>> unsubscribed from the sub maildirs. The question is, can you get to the
>> point where you can do that without first having Gnus spend hours
>> scanning a million directories. Try the above fix and see if Gnus will
>> show you all the other (unwanted) directories.
>
> It does not work.
>
>> We're talking about two different things here. One is defining each of
>> the maildirs as a separate server. So:
>>
>>  '(gnus-secondary-select-methods
>>     '(
>>     (nnmaildir email1
>>  	(directory /home/data1/protected/Maildir/email1@example.com/))
>>     (nnmaildir email2
>>  	(directory /home/data1/protected/Maildir/email2@example.com/))
>>     (nnmaildir email3
>>  	(directory /home/data1/protected/Maildir/email3@example.com/))))
>>
>> Etc.
>>
>> The other thing you want -- quick switching to a particular group -- can
>> certainly be done, but will require a little bit of elisp. Something
>> like (totally untested):
>>
>> (let* (( email "ss@rcdrrun.com")
>>        (account (car-safe (split-string email @))))
>>   (when account
>>     (gnus-group-read-group
>>      nil t (concat nnmaildir+ account :INBOX))))
>
> I will try that, but I cannot get it without
> starting to run or index whatever.
>
> I was thinking that gnus-secondary-select-methods
> has to be set before I start reading the folder,
> and that above looks that you think that way.

Yes, you should try setting your secondary select methods to
one-per-account before you try any of the rest of this.

> I have tried this
>
> (let* ((email "person@example.com")
> 	(gnus-secondary-select-methods '((nnmaildir email (directory (concat "~/Maildir/" email))))))
>    (gnus-group-read-group nil t (concat "nnmaildir+" email :INBOX)))
>
>
> but I am getting this below and I used real email
> address. Do I need to assign the group somehow?

First of all, this isn't going to work until you've got Gnus into a
basic functioning state -- ie, it's already done a successful scan of
your nnmaildir backends. I would leave this part aside until you've got
Gnus working normally. (Also you can't reference a variable inside a
quoted form without using backquoting and unquoting, and let-binding
`gnus-secondary-select-methods' is asking for trouble, and :INBOX needs
to be a string, but really don't bother messing with this at all until
Gnus is functioning.)

Second of all... unfortunately I don't know this code well enough to
figure it out without sitting down with a chunk of time. So far as I can
tell, nnmaildir scans the maildir directories using `directory-files',
which by itself is very quick. Presumably what's slow is registering all
the messages inside the directories, and I don't immediately see a way
of preventing that.

A dirty hack you might try is: temporarily remove the other directories
from one of the maildir accounts, let Gnus index that, and then (while
making sure that `gnus-check-new-newsgroups' is nil) move the other
directories back in. It's possible that Gnus will ignore them, but I'm
really not sure -- I would check with just one of your four accounts
first.

Until I or someone else has time to dig deeper, that's the best I can
suggest, sorry...

Eric

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Jean Louis]

* Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-19 02:27]:
> First of all, this isn't going to work until you've got Gnus into a
> basic functioning state -- ie, it's already done a successful scan of
> your nnmaildir backends.

Thank you for helping. Yet it is not usable for
me. I have too many maildirs and I was thinking
Gnus would read it as maildirs, instead it started
creating .nnmaildir directories inside with copies
of those emails for its own way of processing.

So Gnus does not handle maildirs by standard, it
does something on its own, this is not what I
need. 

Sadly manual did not say that nnmaildir would do
that kind of things, it has spent gigabytes of
space without notice or warning to the user or
choice being offered.

There is one package `maildir` I will see if that
can be improved for my needs.

Thanks
Jean

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Eric Abrahamsen]

On 07/19/19 08:15 AM, Jean Louis wrote:
> * Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-19 02:27]:
>> First of all, this isn't going to work until you've got Gnus into a
>> basic functioning state -- ie, it's already done a successful scan of
>> your nnmaildir backends.
>
> Thank you for helping. Yet it is not usable for
> me. I have too many maildirs and I was thinking
> Gnus would read it as maildirs, instead it started
> creating .nnmaildir directories inside with copies
> of those emails for its own way of processing.

Yeah, I don't think there's any way around the creation of the
.nnmaildir directories, at least not given the way Gnus currently
functions. They aren't actually copies of the emails -- just vectors of
headers for each mail -- but obviously if you've got a lot of mail they
still take up a lot of space.

Sorry Gnus won't work out!

Eric

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Jean Louis]

* Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-19 19:24]:
> 
> On 07/19/19 08:15 AM, Jean Louis wrote:
> > * Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-19 02:27]:
> >> First of all, this isn't going to work until you've got Gnus into a
> >> basic functioning state -- ie, it's already done a successful scan of
> >> your nnmaildir backends.
> >
> > Thank you for helping. Yet it is not usable for
> > me. I have too many maildirs and I was thinking
> > Gnus would read it as maildirs, instead it started
> > creating .nnmaildir directories inside with copies
> > of those emails for its own way of processing.
> 
> Yeah, I don't think there's any way around the creation of the
> .nnmaildir directories, at least not given the way Gnus currently
> functions. They aren't actually copies of the emails -- just vectors of
> headers for each mail -- but obviously if you've got a lot of mail they
> still take up a lot of space.

The solution I found is to bind my Read Mail menu
to eshell-visual-command mutt.

Thank you for being helpful.

The bug shall be closed.

But it would be good that Emacs has Maildir
reading capabiliies. As it is just a file system
with messages.

Jean

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Eric Abrahamsen]

On 07/19/19 19:31 PM, Jean Louis wrote:
> * Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-19 19:24]:
>> 
>> On 07/19/19 08:15 AM, Jean Louis wrote:
>> > * Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-19 02:27]:
>> >> First of all, this isn't going to work until you've got Gnus into a
>> >> basic functioning state -- ie, it's already done a successful scan of
>> >> your nnmaildir backends.
>> >
>> > Thank you for helping. Yet it is not usable for
>> > me. I have too many maildirs and I was thinking
>> > Gnus would read it as maildirs, instead it started
>> > creating .nnmaildir directories inside with copies
>> > of those emails for its own way of processing.
>> 
>> Yeah, I don't think there's any way around the creation of the
>> .nnmaildir directories, at least not given the way Gnus currently
>> functions. They aren't actually copies of the emails -- just vectors of
>> headers for each mail -- but obviously if you've got a lot of mail they
>> still take up a lot of space.
>
> The solution I found is to bind my Read Mail menu
> to eshell-visual-command mutt.
>
> Thank you for being helpful.
>
> The bug shall be closed.

Closing now.

> But it would be good that Emacs has Maildir
> reading capabiliies. As it is just a file system
> with messages.

I agree! Perhaps at some point we'll figure out a way for Gnus to handle
enormous maildirs -- I hope so.

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Basil L. Contovounesios]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Second of all... unfortunately I don't know this code well enough to
> figure it out without sitting down with a chunk of time. So far as I can
> tell, nnmaildir scans the maildir directories using `directory-files',
> which by itself is very quick. Presumably what's slow is registering all
> the messages inside the directories, and I don't immediately see a way
> of preventing that.

Would it be possible to do custom directory filtering in the
'directory-files' virtual server setting of the nnmaildir backend?

-- 
Basil

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Jean Louis]

* Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-19 19:24]:
> > Thank you for helping. Yet it is not usable for
> > me. I have too many maildirs and I was thinking
> > Gnus would read it as maildirs, instead it started
> > creating .nnmaildir directories inside with copies
> > of those emails for its own way of processing.
> 
> Yeah, I don't think there's any way around the creation of the
> .nnmaildir directories, at least not given the way Gnus currently
> functions. They aren't actually copies of the emails -- just vectors of
> headers for each mail -- but obviously if you've got a lot of mail they
> still take up a lot of space.

If they are not copies, they took as much space as
original emails.

That is opposite and contradictory to what Maildir
is supposed to be.

nnmaildir is thus not described well enough in the
Gnus manual, it is bug in itself.

When somebody mentions "Maildir" that means
managing emails in Maildir folders, and not making
indexes or vectors inside of those Maildir
folders, taking up gigabytes of spaces and
basically endangering integrity of files of the
user. 

Thus that behavior of nnmaildir is bug.

One shall explain it very well in Gnus manual that
nnmaildir is not managing Maildir folders but
rather using Maildir folders to make indexes,
vectors, having some kind of news overlay on top
of Maildir folders.

And one shall mention the limit, as it simple does
not work over certain number of Maildirs. I do not
know how many, as I do have many maildirs. It is
unusable for me.

The package in MELPA "maildir" by Nic Ferrier
http://github.com/nicferrier/emacs-maildir shows
that it is trivial to make Maildir reading and I
am using it sometimes.

True Maildir access is simple, not complex.

Jean

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Eric Abrahamsen]

On 07/22/19 09:34 AM, Basil L. Contovounesios wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Second of all... unfortunately I don't know this code well enough to
>> figure it out without sitting down with a chunk of time. So far as I can
>> tell, nnmaildir scans the maildir directories using `directory-files',
>> which by itself is very quick. Presumably what's slow is registering all
>> the messages inside the directories, and I don't immediately see a way
>> of preventing that.
>
> Would it be possible to do custom directory filtering in the
> 'directory-files' virtual server setting of the nnmaildir backend?

I guess it would be possible to set the directory-files setting to a
lambda that accepted the same arguments as `directory-files', and only
returned the desired folder. Again, I just don't know enough about the
inner workings of nnmaildir to say for sure.

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Eric Abrahamsen]

On 07/22/19 10:44 AM, Jean Louis wrote:
> * Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-19 19:24]:
>> > Thank you for helping. Yet it is not usable for
>> > me. I have too many maildirs and I was thinking
>> > Gnus would read it as maildirs, instead it started
>> > creating .nnmaildir directories inside with copies
>> > of those emails for its own way of processing.
>> 
>> Yeah, I don't think there's any way around the creation of the
>> .nnmaildir directories, at least not given the way Gnus currently
>> functions. They aren't actually copies of the emails -- just vectors of
>> headers for each mail -- but obviously if you've got a lot of mail they
>> still take up a lot of space.
>
> If they are not copies, they took as much space as
> original emails.
>
> That is opposite and contradictory to what Maildir
> is supposed to be.
>
> nnmaildir is thus not described well enough in the
> Gnus manual, it is bug in itself.
>
> When somebody mentions "Maildir" that means
> managing emails in Maildir folders, and not making
> indexes or vectors inside of those Maildir
> folders, taking up gigabytes of spaces and
> basically endangering integrity of files of the
> user. 
>
> Thus that behavior of nnmaildir is bug.

Gnus operates as a general newsreader/email client, not a dedicated
maildir reader. The maildir functionality needs to fit in with its
existing paradigms, and right now that means maintaining nov header
files alongside the messages themselves. Otherwise actually listing
groups and reading messages would be unbearably slow. I agree it's
essentially a bug, but it's one that can't be fixed without some
fundamental alteration of how Gnus works.

I also don't think this threatens the integrity of the user's files: the
extra data is maintained in parallel, and doesn't interfere with the
messages themselves.

> One shall explain it very well in Gnus manual that
> nnmaildir is not managing Maildir folders but
> rather using Maildir folders to make indexes,
> vectors, having some kind of news overlay on top
> of Maildir folders.
>
> And one shall mention the limit, as it simple does
> not work over certain number of Maildirs. I do not
> know how many, as I do have many maildirs. It is
> unusable for me.

The limit (defined as "the point at which the user gets annoyed") is
going to be different for different machines, and different users. But I
agree the manual should provide a more prominent warning.

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Eric Abrahamsen]

On 07/22/19 09:34 AM, Basil L. Contovounesios wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Second of all... unfortunately I don't know this code well enough to
>> figure it out without sitting down with a chunk of time. So far as I can
>> tell, nnmaildir scans the maildir directories using `directory-files',
>> which by itself is very quick. Presumably what's slow is registering all
>> the messages inside the directories, and I don't immediately see a way
>> of preventing that.
>
> Would it be possible to do custom directory filtering in the
> 'directory-files' virtual server setting of the nnmaildir backend?

It might also be possible to make the building of nov files incremental:
ie only create them as messages are actually viewed. It would slow down
first-time viewing, and potentially fill up the same amount of disk
space, if you view all messages, but at least you'd have an operational
maildir installation faster.

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Jean Louis]

* Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-22 19:40]:
> Gnus operates as a general newsreader/email client, not a dedicated
> maildir reader. The maildir functionality needs to fit in with its
> existing paradigms, and right now that means maintaining nov header
> files alongside the messages themselves. Otherwise actually listing
> groups and reading messages would be unbearably slow. I agree it's
> essentially a bug, but it's one that can't be fixed without some
> fundamental alteration of how Gnus works.

Sure, it is general newsreader.

But when acting as email client that reads Maildirs, then just nothing
need to be indexed extra witin maildir folders.

Just because something had to be complex for news, need not be complex
for maildirs.

Reference:
https://en.wikipedia.org/wiki/Maildir

If maildir folders are called groups in Gnus terminology, then listing
them is as simple as detecting if it is a maildir folder.

Without detecting them, wouldb be simple as `ls ~/Maildir`

Maildir has its file structure, it need no program to add something to
it. There are cur, new, tmp sub directories, there is no need for any
additional directories.

> I also don't think this threatens the integrity of the user's files: the
> extra data is maintained in parallel, and doesn't interfere with the
> messages themselves.

I did read nnmaildir info page, and I have not found anything that
would indicate what would Gnus do to my file system. In my case
maildir folders are many, and it makes no difference in speed when
reading it with the `mutt` mail client, but also no difference when
reading it with `maildir` package, but that one is not nicely
developed yet.

It started working on my files for more than 24 hours, and it brought
my system to being not responsive. I did try to leave it running, but
it never ended. So this is real world example.

It created gigabytes of files in those subdirectories without warning
me, without asking me.

That is a programm choice that does not respect the user.

Maildirs could be indexed, but that is left to some external tools

I don't think it should be Gnus responsibility to index them.

In reality, to access maildir is as fast as accessing directory and
files inside. To make index of those files, reading Subject in first
part of file or greping it somehow is enough.

I think that Emacs does have almost everything to read Maildir files
and to use some of mail sending tools. It just needs so little to have
proper Maildir based reading, sorting and sending files.

mu4e package is not good for that, it uses only indexed database, but
not the Maildirs straight.

`maildir` package shows how easy it can be, it is just not yet in best
shape. I can improve it formy own needs, but it would be so nice to
have GNU Emacs become able to replace `mutt`

Jean

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Jean Louis]

* Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-22 19:42]:
> > Would it be possible to do custom directory filtering in the
> > 'directory-files' virtual server setting of the nnmaildir backend?
> 
> It might also be possible to make the building of nov files incremental:
> ie only create them as messages are actually viewed. It would slow down
> first-time viewing, and potentially fill up the same amount of disk
> space, if you view all messages, but at least you'd have an operational
> maildir installation faster.

But why making it complex when it is not?

That is contradictory to Maildir "standards". There is nothing to do
to the Maildir directory but to read the To/From/Date/Subject to make the index of
it.

Then by clicking on the particular mail, it would open the file.

Anything else is not "maildir standard". It should warn
user that it makes much more to Maildir files than it is expected by
standard. 

With my 47708 maildirs, it created this many directories inside, and
duplicated the disk space that I was using.

In regards to info page "articles are deleted after one week"... I am
really worried about that.

Email messages are named articles, this creates confusion, and that
would further mean that some email messages which are supposed to
stay, would be deleted unless user is skilled to configure the
parameters.

In general Gnus is not usable and is dangerous to Maildirs. It creates
complexities where no such are required and necessary.

Jean

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Eric Abrahamsen]

Jean Louis <bugs@gnu.support> writes:

> * Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-22 19:40]:

[...]

>> I also don't think this threatens the integrity of the user's files: the
>> extra data is maintained in parallel, and doesn't interfere with the
>> messages themselves.
>
> I did read nnmaildir info page, and I have not found anything that
> would indicate what would Gnus do to my file system.

The first paragraph of the maildir section of the Gnus manual says:

‘nnmaildir’ also stores extra information in the ‘.nnmaildir/’ directory
within a maildir.

On the next page of the manual, there's a section on this group parameter:

‘nov-cache-size’
     An integer specifying the size of the NOV memory cache.  To speed
     things up, ‘nnmaildir’ keeps NOV data in memory for a limited
     number of articles in each group.  (This is probably not
     worthwhile, and will probably be removed in the future.)  This
     parameter’s value is noticed only the first time a group is seen
     after the server is opened—i.e., when you first start Gnus,
     typically.  The NOV cache is never resized until the server is
     closed and reopened.  The default is an estimate of the number of
     articles that would be displayed in the summary buffer: a count of
     articles that are either marked with ‘tick’ or not marked with
     ‘read’, plus a little extra.

Noting both that the nov cache will probably go away at some point, and
how to control its size in the meantime.

(add-to-list 'gnus-parameters
	     '("^nnmaildir:"
	       (nov-cache-size . 2)
	       (expire-age . never)))

bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Eric Abrahamsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Jean Louis <bugs@gnu.support> writes:
>
>> * Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-22 19:40]:
>
> [...]
>
>>> I also don't think this threatens the integrity of the user's files: the
>>> extra data is maintained in parallel, and doesn't interfere with the
>>> messages themselves.
>>
>> I did read nnmaildir info page, and I have not found anything that
>> would indicate what would Gnus do to my file system.
>
> The first paragraph of the maildir section of the Gnus manual says:
>
> ‘nnmaildir’ also stores extra information in the ‘.nnmaildir/’ directory
> within a maildir.
>
> On the next page of the manual, there's a section on this group parameter:
>
> ‘nov-cache-size’
>      An integer specifying the size of the NOV memory cache.  To speed
>      things up, ‘nnmaildir’ keeps NOV data in memory for a limited
>      number of articles in each group.  (This is probably not
>      worthwhile, and will probably be removed in the future.)  This
>      parameter’s value is noticed only the first time a group is seen
>      after the server is opened—i.e., when you first start Gnus,
>      typically.  The NOV cache is never resized until the server is
>      closed and reopened.  The default is an estimate of the number of
>      articles that would be displayed in the summary buffer: a count of
>      articles that are either marked with ‘tick’ or not marked with
>      ‘read’, plus a little extra.
>
> Noting both that the nov cache will probably go away at some point, and
> how to control its size in the meantime.
>
> (add-to-list 'gnus-parameters
> 	     '("^nnmaildir:"
> 	       (nov-cache-size . 2)
> 	       (expire-age . never)))

Actually, `expire-age' isn't really necessary: so long as you don't mark
any articles with the "E" key, nothing will get expired.