dealing with a lot of emails in single dir

dealing with a lot of emails in single dir

[Aurélien Aptel]

Hi,

How do you deal with a maildir with a billion file in it? I'm on ext4
which should handle it but I can already feel some programs struggle
with a dir with that many entries (noticeable slow down).

I was thinking of archiving emails in yearly folders. How does notmuch
deal with moving emails around? Will I still be able to open them or
should I recreate the db?

If anyone is using lieer with gmail (https://github.com/gauteh/lieer)
will this trigger pulls of all the moved emails again?

Thanks

Re: dealing with a lot of emails in single dir

[Tomi Ollila]

On Tue, Aug 17 2021, Aurélien Aptel wrote:

> Hi,
>
> How do you deal with a maildir with a billion file in it? I'm on ext4
> which should handle it but I can already feel some programs struggle
> with a dir with that many entries (noticeable slow down).

That reminded me:

http://be-n.com/spw/you-can-list-a-million-files-in-a-directory-but-not-with-ls.html

that talks about 8 million files in a dir and already that requires special
attention (i.e. readdir(3) slow, have to use getdents(2) syscall directly...

>
> I was thinking of archiving emails in yearly folders. How does notmuch
> deal with moving emails around? Will I still be able to open them or
> should I recreate the db?

cannot remember how how notmuch detect renames -- so before doing so you
should backup tags with notmuch-dump...

Tomi
 

>
> If anyone is using lieer with gmail (https://github.com/gauteh/lieer)
> will this trigger pulls of all the moved emails again?
>
> Thanks
> _______________________________________________
> notmuch mailing list -- notmuch@notmuchmail.org
> To unsubscribe send an email to notmuch-leave@notmuchmail.org

Re: dealing with a lot of emails in single dir

[David Bremner]

Aurélien Aptel <aurelien.aptel@gmail.com> writes:

>
> I was thinking of archiving emails in yearly folders. How does notmuch
> deal with moving emails around? Will I still be able to open them or
> should I recreate the db?

"notmuch new" should detect the moves, but it may take some time. For a
sufficiently large number of moves it is faster to recreate the database.\r