bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Eric Abrahamsen <eric@ericabrahamsen.net>]

On 07/18/19 21:43 PM, Jean Louis wrote:
> * Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-18 21:01]:
>> I think you're right, and in a sense it is definitely a bug, but a lot
>> of people have run into this and I've seen some saying there's not much
>> to be done. So that didn't sound very encouraging.
>> 
>> Could you do M-x toggle-debug-on-quit, start up Gnus, let it hang for a
>> bit, then do C-g and post the resulting backtrace here? It's fairly
>> obvious what's going on, but it would be good to
>> see the specifics.
>
> Debugger entered--Lisp error: (quit)
>   file-name-as-directory("/home/data1/protected/Maildir/info@codepink.org/.n...")
>   nnmaildir--scan("info@codepink.org" nil #<hash-table equal
> 1913/47714 0xd41cb5> nil "/home/data1/protected/Maildir/"
> directory-files)
>   nnmaildir-request-scan(find-new-groups "")
>   nnmaildir-request-list("")
>   nnmaildir-request-newgroups("Thu, 18 Jul 2019 18:11:09 +0200" "")
>   gnus-request-newgroups("Thu, 18 Jul 2019 18:11:09 +0200" (nnmaildir "" (directory "~/Maildir")))
>   gnus-ask-server-for-new-groups()
>   gnus-find-new-newsgroups()
>   gnus-setup-news(nil nil nil)
>   #f(compiled-function () #<bytecode 0x995efd>)()
>   gnus-1(nil nil nil)
>   gnus(nil)
>   funcall-interactively(gnus nil)
>   call-interactively(gnus record nil)
>   command-execute(gnus record)
>   execute-extended-command(nil "gnus" "gnus")
>   funcall-interactively(execute-extended-command nil "gnus" "gnus")
>   call-interactively(execute-extended-command nil nil)
>   command-execute(execute-extended-command)

I'm assuming your value of `gnus-check-new-newsgroups' is at its default
of 'ask-server. Try setting it to nil. That will (should) at least
prevent Gnus from scanning all the folders at startup. It doesn't solve
the underlying problem, but in your case it might avoid it.

>> The nnmaildir servers keep track of their directory modtime, which is
>> both set and read only once, in `nnmaildir-request-scan'. So at least it
>> should be fairly easy to see what's happening there. Gnus doesn't save
>> the modtimes, though -- perhaps a potential solution could involve
>> saving the maildir modtimes in newsrc.eld.
>
> Can that scan be turned off as option?
>
> I do not even need to list the folders. I just
> want to read the main ~/Maildir folder but not
> sub-Maildirs within Gnus.

The final setup you want is to be subscribed to the main folder, but
unsubscribed from the sub maildirs. The question is, can you get to the
point where you can do that without first having Gnus spend hours
scanning a million directories. Try the above fix and see if Gnus will
show you all the other (unwanted) directories.

[...]

>> > And my other question is, is there a way to quickly access
>> > Maildir/email2@example.com by using Gnus? Some function maybe to just
>> > write the email address or fetch it from database, and to open the
>> > Maildir with nnmaildir?
>> 
>> I'm not entirely sure what you mean here, but one suggestion I have is
>> to create four different nnmaildir select methods, one for each of your
>> email addresses. I think that's how Gnus is expecting this sort of thing
>> to be set up, and it might make it easier for you to access, as well.
>
> Would that be possible on the fly?
>
> For example I have email address on screen, find
> it at point and quickly switch to Maildir by email
> address?

We're talking about two different things here. One is defining each of
the maildirs as a separate server. So:

 '(gnus-secondary-select-methods
    '(
    (nnmaildir "email1"
 	(directory "/home/data1/protected/Maildir/email1@example.com/"))
    (nnmaildir "email2"
 	(directory "/home/data1/protected/Maildir/email2@example.com/"))
    (nnmaildir "email3"
 	(directory "/home/data1/protected/Maildir/email3@example.com/"))))

Etc.

The other thing you want -- quick switching to a particular group -- can
certainly be done, but will require a little bit of elisp. Something
like (totally untested):

(let* ((email (thing-at-point 'email))
       (account (car-safe (split-string email "@"))))
  (when account
    (gnus-group-read-group
     nil t (concat "nnmaildir+" account ":INBOX"))))

But see if you can get the servers functioning first!

Eric