How to index /var/spool/mail with notmuch

How to index /var/spool/mail with notmuch

[Tassilo Horn]

Hi all,

I'd like to try out notmuch.  My mail setup is as follows:

  - I run a local IMAP server (dovecot) and access it using Gnus
  - Dovecot stores its mails in /var/spool/mail/ in some one file per
    message format

But I get some permission problems when trying to index /var/spool/mail.
I was able to create a .notmuch/ directory in there with permissions set
to 700 for my user.  All mail files are readable for my user.

Unfortunately, there are some dovecot internal files, which should
neither be indexed by notmuch, and which have 600 permissions for the
mail user.  And that's where notmuch errors and stops indexing. :-(

All "real" mail files are named "u.<number>", so it would be cool if I
could provide a pattern to notmuch matching all files I'd like to index.
And maybe the other way round (a blacklist pattern) would be useful,
too.

Any thoughts?  Maybe there's a better approach for what I'm trying to
do?

Bye,
Tassilo

Re: How to index /var/spool/mail with notmuch

[Jed Brown]

On Sun, 22 Nov 2009 21:24:47 +0100, Tassilo Horn <tassilo@member.fsf.org> wrote:
> Hi all,
> 
> I'd like to try out notmuch.  My mail setup is as follows:
> 
>   - I run a local IMAP server (dovecot) and access it using Gnus
>   - Dovecot stores its mails in /var/spool/mail/ in some one file per
>     message format

How about

$ mkdir -p ~/mail/spool
$ ln -s /var/spool/mail/$USER/{cur,new,tmp} ~/mail/spool

and point notmuch at ~/mail (or specifically ~/mail/spool).


Jed

Re: How to index /var/spool/mail with notmuch

[Tassilo Horn]

Jed Brown <jed@59A2.org> writes:

Hi Jed,

>>   - I run a local IMAP server (dovecot) and access it using Gnus
>>   - Dovecot stores its mails in /var/spool/mail/ in some one file per
>>     message format
>
> How about
>
> $ mkdir -p ~/mail/spool
> $ ln -s /var/spool/mail/$USER/{cur,new,tmp} ~/mail/spool
>
> and point notmuch at ~/mail (or specifically ~/mail/spool).

Brilliant idea.  :-)

Ok, now it's indexing my 63000 mails.  10 minutes to go.

Bye,
Tassilo

Re: How to index /var/spool/mail with notmuch

[Carl Worth]

On Sun, 22 Nov 2009 21:24:47 +0100, Tassilo Horn <tassilo@member.fsf.org> wrote:
> But I get some permission problems when trying to index /var/spool/mail.
> I was able to create a .notmuch/ directory in there with permissions set
> to 700 for my user.  All mail files are readable for my user.
> 
> Unfortunately, there are some dovecot internal files, which should
> neither be indexed by notmuch, and which have 600 permissions for the
> mail user.  And that's where notmuch errors and stops indexing. :-(

Hi Tassilo, welcome to notmuch!

I'm glad you found a workaround for this problem, (thanks Jed!).

But perhaps these errors should be made into warnings instead? Any
thoughts on that anyone?

> All "real" mail files are named "u.<number>", so it would be cool if I
> could provide a pattern to notmuch matching all files I'd like to index.
> And maybe the other way round (a blacklist pattern) would be useful,
> too.

I've been planning on having a blacklist pattern for a
while. Originally, the only difficulty in implementing it was that we
had nowhere to store configuration information. But we have a
configuration file now, so this would be a pretty easy thing to
implement.

It's not as obvious that a whitelist pattern would be as widely useful,
but it would be possible too.

-Carl

Re: How to index /var/spool/mail with notmuch

[Tassilo Horn]

Carl Worth <cworth@cworth.org> writes:

Hi Carl,

>> Unfortunately, there are some dovecot internal files, which should
>> neither be indexed by notmuch, and which have 600 permissions for the
>> mail user.  And that's where notmuch errors and stops indexing. :-(
>
> Hi Tassilo, welcome to notmuch!
>
> I'm glad you found a workaround for this problem, (thanks Jed!).

Trying out notmuch already cashed out. ;-)

> But perhaps these errors should be made into warnings instead? Any
> thoughts on that anyone?

I think so.  For other dovecot internal index files it already says
something like "Ignoring non-mail file foobar.idx" and simply skips that
file.  I think that's the right thing to do for files where it has no
read permissions for, too.

>> All "real" mail files are named "u.<number>", so it would be cool if
>> I could provide a pattern to notmuch matching all files I'd like to
>> index.  And maybe the other way round (a blacklist pattern) would be
>> useful, too.
>
> I've been planning on having a blacklist pattern for a while.
> Originally, the only difficulty in implementing it was that we had
> nowhere to store configuration information.  But we have a
> configuration file now, so this would be a pretty easy thing to
> implement.
>
> It's not as obvious that a whitelist pattern would be as widely
> useful, but it would be possible too.

Well, for dovecot users using the dbox format, a whitelist pattern would
be much simpler because of the coherent mail file naming.  But I could
live with a blacklist pattern, too.

Bye,
Tassilo

Re: How to index /var/spool/mail with notmuch

[Carl Worth]

On Mon, 23 Nov 2009 08:42:09 +0100, Tassilo Horn <tassilo@member.fsf.org> wrote:
> Carl Worth <cworth@cworth.org> writes:
>
> > But perhaps these errors should be made into warnings instead? Any
> > thoughts on that anyone?
> 
> I think so.  For other dovecot internal index files it already says
> something like "Ignoring non-mail file foobar.idx" and simply skips that
> file.  I think that's the right thing to do for files where it has no
> read permissions for, too.

OK. I've added that to our TODO list.

-Carl