* Avoiding the "huge INBOX of death"
@ 2016-07-19 9:42 Raphaël Fournier-S'niehotta
2016-07-19 10:26 ` David Bremner
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: Raphaël Fournier-S'niehotta @ 2016-07-19 9:42 UTC (permalink / raw
To: notmuch
Hello fellow notmuchers,
I am a longtime "classic" mutt user, willing to switch to a notmuch-based
solution (probably neomutt). However, as far as I understand, I will have to
deal with a "huge INBOX of death", as put by Anarcat in a recent blog post [1],
that when all emails remain in a single maildir, which may the become very
large.
I would like to avoid this, for quota reasons on my work imap server first, and
also because I sometimes have to use webmail access, with poor searching
capabilities.
Thus, I think I need to have my emails sorted into maildir folders according to
the notmuch tags. Namely, emails with the "inbox" tag should stay in the INBOX
maildir, those with "archive" should be moved to Archives (all synchronized with
the server). As a bonus, it would be great to have a few more features, like
"mails in Archives older than 6 months are moved to Archives/<year>" (kept
locally and not stored on the server).
As far as I understand, "afew" and its MailMover feature could be useful for my
case, but it does not seem currently maintained (plus, very few mentions of it
on the notmuch archives) and Michael Turquette said it's buggy [2], so I would
like more feedback to be sure before letting it mess with my maildir.
Another tool that could be useful is the "maildir-notmuch-sync" script written
by Ethan Schoonover and adapted by Michael Turquette [3]. However, it seems
dedicated to handling the synchronization of Gmail labels, not a generic imap
server hierarchy.
Perhaps there is another solution to deal with this in a notmuch-based setup, I
would very much like any of your feedback.
[1]: https://anarc.at/blog/2016-05-12-email-setup/
[2]: http://deferred.io/2016/01/17/state-of-the-dotfiles.html
[3]: http://deferred.io/2016/01/18/notmuch-tags-gmail-labels-bidirectional-sync.html
--
Raphaël Fournier-S'niehotta
http://raphael.fournier-sniehotta.fr/apropos
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Avoiding the "huge INBOX of death"
2016-07-19 9:42 Avoiding the "huge INBOX of death" Raphaël Fournier-S'niehotta
@ 2016-07-19 10:26 ` David Bremner
2016-07-19 10:36 ` Vladimir Marek
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: David Bremner @ 2016-07-19 10:26 UTC (permalink / raw
To: Raphaël Fournier-S'niehotta, notmuch
[-- Attachment #1: Type: text/plain, Size: 1737 bytes --]
Raphaël Fournier-S'niehotta <raphael@raphaelfournier.net> writes:
> Hello fellow notmuchers,
>
> I am a longtime "classic" mutt user, willing to switch to a notmuch-based
> solution (probably neomutt). However, as far as I understand, I will have to
> deal with a "huge INBOX of death", as put by Anarcat in a recent blog post [1],
> that when all emails remain in a single maildir, which may the become very
> large.
>
TBH, I don't understand what specific problem anarcat is having, it
sounds like a workflow issue. I currently have 1074 messages tagged
inbox, out of about 450K messages.
Notmuch doesn't care how you organize your mail files on disk, as long
as they are one message per file, and one tree of messages. So feel free
to store them e.g. by date using procmail if you are worried about
performance, or in some other set of folders with semantic
For performance I'd also say measure before you optimize. I currently
have about 77K messages in one of my maildirs, with no noticable
performance hit. Of course you want e.g. directory indexing on ext3,
but that is default on anything recent.
> Thus, I think I need to have my emails sorted into maildir folders according to
> the notmuch tags. Namely, emails with the "inbox" tag should stay in the INBOX
> maildir, those with "archive" should be moved to Archives (all synchronized with
> the server). As a bonus, it would be great to have a few more features, like
> "mails in Archives older than 6 months are moved to Archives/<year>" (kept
> locally and not stored on the server).
Generally, you should search directly by folder, rather than trying to
sync tags and folders.
I'm not familiar with afew as a user.
d
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 647 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Avoiding the "huge INBOX of death"
2016-07-19 9:42 Avoiding the "huge INBOX of death" Raphaël Fournier-S'niehotta
2016-07-19 10:26 ` David Bremner
@ 2016-07-19 10:36 ` Vladimir Marek
2016-07-19 11:42 ` Olivier Schwander
2016-07-19 14:51 ` fauno
3 siblings, 0 replies; 5+ messages in thread
From: Vladimir Marek @ 2016-07-19 10:36 UTC (permalink / raw
To: Raphaël Fournier-S'niehotta; +Cc: notmuch
I can't speak for everyone, but here's what I am using.
I do have 'messages' directory which contains 'imap' and 'archive'.
messages
\-imap
\-archive
The imap directory is being synchronized via offlineimap to the server.
In the offlineimap postsync hook I call a script which does some
bookkeeping on the newly received messages and it also marks old and
already read and not flagged emails by the 'old-tread' tag.
N=notmuch
while : ; do
AGE=100d # 100 days
# Find all threads containing old messages
OLD_MESSAGES=$( $N search --output=threads not tag:archive and date:..$AGE and not tag:unread and not tag:flagged )
[ -n "$OLD_MESSAGES" ] || break
# Find threads containing newer messages
NEW_MESSAGES=$( $N search --output=threads not tag:archive and not '(' date:..$AGE or tag:unread or tag:flagged ')' )
# remove the threads containing newer messages
OLD_THREADS=$( /usr/xpg4/bin/fgrep -v -x -f <(echo "$NEW_MESSAGES") < <(echo "$OLD_MESSAGES") )
[ -n "$OLD_THREADS" ] || break
$N tag +old-thread -- $OLD_THREADS
break
done
# If old thread received newer message, unmark it too
$N tag -old-thread -- tag:old-thread and '(' not date:..$AGE or tag:unread or tag:flagged or tag:archive ')'
Once in a while I review the old-thread messages and assign them by hand
'archive' flag. The postsync hook is looking for this flag and moves all
messages to be archived into archive folder. You can list files to be
archived by
$N search --output=files 'path:imap/**' tag:archive
I won't list the commands to move files from messages/imap to
messages/archive since I have a bit more complex scenario. But it should
be straightforward.
On next run offlineimap sees that the messages disappeared from 'imap'
folder and removes them from imap server.
$ notmuch count '*'
734346
$ notmuch count 'path:imap/**'
18419
In my case I am server side filtering the emails into several folders so
that I have at least some order in emails when reading email via phone,
so 'path:imap/**' contains all the messages in whole imap tree.
$ notmuch count 'path:imap/** and tag:inbox and tag:unread'
11
I'm trying to have my virtual INBOX empty most of the time.
I'm interested what tricks others use!
Cheers
--
Vlad
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Avoiding the "huge INBOX of death"
2016-07-19 9:42 Avoiding the "huge INBOX of death" Raphaël Fournier-S'niehotta
2016-07-19 10:26 ` David Bremner
2016-07-19 10:36 ` Vladimir Marek
@ 2016-07-19 11:42 ` Olivier Schwander
2016-07-19 14:51 ` fauno
3 siblings, 0 replies; 5+ messages in thread
From: Olivier Schwander @ 2016-07-19 11:42 UTC (permalink / raw
To: notmuch
Le 19 Jul 2016 11:42, Raphaël Fournier-S'niehotta a écrit:
> Hello fellow notmuchers,
>
> I am a longtime "classic" mutt user, willing to switch to a notmuch-based
> solution (probably neomutt). However, as far as I understand, I will have to
> deal with a "huge INBOX of death", as put by Anarcat in a recent blog post [1],
> that when all emails remain in a single maildir, which may the become very
> large.
>
> I would like to avoid this, for quota reasons on my work imap server first, and
> also because I sometimes have to use webmail access, with poor searching
> capabilities.
>
> Thus, I think I need to have my emails sorted into maildir folders according to
> the notmuch tags. Namely, emails with the "inbox" tag should stay in the INBOX
> maildir, those with "archive" should be moved to Archives (all synchronized with
> the server). As a bonus, it would be great to have a few more features, like
> "mails in Archives older than 6 months are moved to Archives/<year>" (kept
> locally and not stored on the server).
>
> As far as I understand, "afew" and its MailMover feature could be useful for my
> case, but it does not seem currently maintained (plus, very few mentions of it
> on the notmuch archives) and Michael Turquette said it's buggy [2], so I would
> like more feedback to be sure before letting it mess with my maildir.
>
> Another tool that could be useful is the "maildir-notmuch-sync" script written
> by Ethan Schoonover and adapted by Michael Turquette [3]. However, it seems
> dedicated to handling the synchronization of Gmail labels, not a generic imap
> server hierarchy.
>
> Perhaps there is another solution to deal with this in a notmuch-based setup, I
> would very much like any of your feedback.
>
> [1]: https://anarc.at/blog/2016-05-12-email-setup/
> [2]: http://deferred.io/2016/01/17/state-of-the-dotfiles.html
> [3]: http://deferred.io/2016/01/18/notmuch-tags-gmail-labels-bidirectional-sync.html
You could use fdm [1] to store messages into a folder hierarchy based on
the date in the messages. In the past, I used something like that to
periodically move messages in my inbox to sub-folder with the year:
account "archives" disabled maildirs {
"${mailpath}INBOX"
}
action "archive" maildir "${mailpath}%[maildir]/%[mail_year]"
match account "archives" {
match age > 6 months action "archive"
match all action keep
}
Nowadays, I use a custom script [2] to mirror some notmuch queries to
maildirs (ran periodically in my crontab), such like:
notmuch-imap.sh ~/Maildir/ NOT tag:archive AND NOT tag:killed AND NOT tag:spam AND tag:inbox
notmuch-imap.sh ~/Maildir/.todo NOT tag:archive AND NOT tag:killed AND NOT tag:spam AND tag:todo
Both solutions works server-side, since I control my own mail server but
it may probably used in an offlineimap hook.
Best,
Olivier
[1]: https://github.com/nicm/fdm
[2]: https://chadok.info/~oschwand/poubelle/e53f1d9ac30909fd17ca98f4160375bbc9a6aca5/notmuch-imap.sh.html
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Avoiding the "huge INBOX of death"
2016-07-19 9:42 Avoiding the "huge INBOX of death" Raphaël Fournier-S'niehotta
` (2 preceding siblings ...)
2016-07-19 11:42 ` Olivier Schwander
@ 2016-07-19 14:51 ` fauno
3 siblings, 0 replies; 5+ messages in thread
From: fauno @ 2016-07-19 14:51 UTC (permalink / raw
To: notmuch
[-- Attachment #1: Type: text/plain, Size: 402 bytes --]
Raphaël Fournier-S'niehotta <raphael@raphaelfournier.net> writes:
> Hello fellow notmuchers,
> Perhaps there is another solution to deal with this in a notmuch-based setup, I
> would very much like any of your feedback.
i made toomuch[^0] to archive email by year-month and it works really
well for me :P
[^0]: https://github.com/fauno/toomuch
--
http://utopia.partidopirata.com.ar/
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 584 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2016-07-19 14:59 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-07-19 9:42 Avoiding the "huge INBOX of death" Raphaël Fournier-S'niehotta
2016-07-19 10:26 ` David Bremner
2016-07-19 10:36 ` Vladimir Marek
2016-07-19 11:42 ` Olivier Schwander
2016-07-19 14:51 ` fauno
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).