How to index arbitrary headers?

How to index arbitrary headers?

[Petri Savolainen]

[-- Attachment #1: Type: text/plain, Size: 379 bytes --]

Hello,

I could not find information anywhere in notmuch docs about what is
actually indexed - specifically, what email headers are indexed and
searchable? If a header is not indexed, does searching for its value still
result in a search hit?

It would be nice if one could just provide the list of headers to be
indexed in some configuration file or something.

Thanks,

 Petri

[-- Attachment #2: Type: text/html, Size: 478 bytes --]

Re: How to index arbitrary headers?

[Austin Clements]

Quoth Petri Savolainen on Oct 01 at  3:39 pm:
>    Hello,
>    I could not find information anywhere in notmuch docs about what is
>    actually indexed - specifically, what email headers are indexed and
>    searchable? If a header is not indexed, does searching for its value still
>    result in a search hit?
>    It would be nice if one could just provide the list of headers to be
>    indexed in some configuration file or something.
>    Thanks,
>     Petri

notmuch doesn't currently implement this, though it is an
oft-requested feature.  One (not insurmountable) difficulty is that
the database would have to be rebuilt if a user-configured list of
headers changed and there are technical limitations that prevent us
from simply indexing all headers.  Out of curiosity, what headers are
you interested in indexing?

The currently indexed headers are described in man
notmuch-search-terms.

Re: How to index arbitrary headers?

[Petri Savolainen]

[-- Attachment #1: Type: text/plain, Size: 1327 bytes --]

Hi,

thanks for your response. I am evaluating notmuch / xapian for building an
application for analyzing in various ways a fairly large number of emails
accumulated over several years. I am afraid the number of headers that
would ultimately need to be indexed is therefore quite a lot larger than
what notmuch currently indexes.

 Petri

2012/10/1 Austin Clements <amdragon@mit.edu>

> Quoth Petri Savolainen on Oct 01 at  3:39 pm:
> >    Hello,
> >    I could not find information anywhere in notmuch docs about what is
> >    actually indexed - specifically, what email headers are indexed and
> >    searchable? If a header is not indexed, does searching for its value
> still
> >    result in a search hit?
> >    It would be nice if one could just provide the list of headers to be
> >    indexed in some configuration file or something.
> >    Thanks,
> >     Petri
>
> notmuch doesn't currently implement this, though it is an
> oft-requested feature.  One (not insurmountable) difficulty is that
> the database would have to be rebuilt if a user-configured list of
> headers changed and there are technical limitations that prevent us
> from simply indexing all headers.  Out of curiosity, what headers are
> you interested in indexing?
>
> The currently indexed headers are described in man
> notmuch-search-terms.
>

[-- Attachment #2: Type: text/html, Size: 1751 bytes --]

Re: How to index arbitrary headers?

[Dmitrijs Ledkovs]

On 3 October 2012 19:32, Petri Savolainen <petri@koodaamo.fi> wrote:
> Hi,
>
> thanks for your response. I am evaluating notmuch / xapian for building an
> application for analyzing in various ways a fairly large number of emails
> accumulated over several years. I am afraid the number of headers that would
> ultimately need to be indexed is therefore quite a lot larger than what
> notmuch currently indexes.
>
>  Petri
>
> 2012/10/1 Austin Clements <amdragon@mit.edu>
>>
>> Quoth Petri Savolainen on Oct 01 at  3:39 pm:
>> >    Hello,
>> >    I could not find information anywhere in notmuch docs about what is
>> >    actually indexed - specifically, what email headers are indexed and
>> >    searchable? If a header is not indexed, does searching for its value
>> > still
>> >    result in a search hit?
>> >    It would be nice if one could just provide the list of headers to be
>> >    indexed in some configuration file or something.
>> >    Thanks,
>> >     Petri
>>
>> notmuch doesn't currently implement this, though it is an
>> oft-requested feature.  One (not insurmountable) difficulty is that
>> the database would have to be rebuilt if a user-configured list of
>> headers changed and there are technical limitations that prevent us
>> from simply indexing all headers.  Out of curiosity, what headers are
>> you interested in indexing?
>>
>> The currently indexed headers are described in man
>> notmuch-search-terms.
>

Use mapreduce instead: hadoop or discoproject or haddop with dumbo
should be faster.

Regards,

Dmitrijs.

Re: How to index arbitrary headers?

[Nicolás Reynolds]

[-- Attachment #1: Type: text/plain, Size: 1079 bytes --]

Austin Clements <amdragon@MIT.EDU> writes:

> Quoth Petri Savolainen on Oct 01 at  3:39 pm:
>>    Hello,
>>    I could not find information anywhere in notmuch docs about what is
>>    actually indexed - specifically, what email headers are indexed and
>>    searchable? If a header is not indexed, does searching for its value still
>>    result in a search hit?
>>    It would be nice if one could just provide the list of headers to be
>>    indexed in some configuration file or something.
>>    Thanks,
>>     Petri
>
> notmuch doesn't currently implement this, though it is an
> oft-requested feature.  One (not insurmountable) difficulty is that
> the database would have to be rebuilt if a user-configured list of
> headers changed and there are technical limitations that prevent us
> from simply indexing all headers.  Out of curiosity, what headers are
> you interested in indexing?
>
> The currently indexed headers are described in man
> notmuch-search-terms.

maybe related: is it possible to index only the headers and not the mail
body?

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: How to index arbitrary headers?

[Dmitrijs Ledkovs]

On 4 October 2012 13:51, Nicolás Reynolds <fauno@kiwwwi.com.ar> wrote:
> Austin Clements <amdragon@MIT.EDU> writes:
>
>> Quoth Petri Savolainen on Oct 01 at  3:39 pm:
>>>    Hello,
>>>    I could not find information anywhere in notmuch docs about what is
>>>    actually indexed - specifically, what email headers are indexed and
>>>    searchable? If a header is not indexed, does searching for its value still
>>>    result in a search hit?
>>>    It would be nice if one could just provide the list of headers to be
>>>    indexed in some configuration file or something.
>>>    Thanks,
>>>     Petri
>>
>> notmuch doesn't currently implement this, though it is an
>> oft-requested feature.  One (not insurmountable) difficulty is that
>> the database would have to be rebuilt if a user-configured list of
>> headers changed and there are technical limitations that prevent us
>> from simply indexing all headers.  Out of curiosity, what headers are
>> you interested in indexing?
>>
>> The currently indexed headers are described in man
>> notmuch-search-terms.
>
> maybe related: is it possible to index only the headers and not the mail
> body?
>

Checkout project `afew` which pipes complete emails and assigns tags
based on the parse result.
It may be sufficient for your purpose.
Or you can modify notmuch to index the stuff you want.

Regards,

Dmitrijs.

Re: How to index arbitrary headers?

[Nicolás Reynolds]

[-- Attachment #1: Type: text/plain, Size: 272 bytes --]

Dmitrijs Ledkovs <xnox@debian.org> writes:
> Checkout project `afew` which pipes complete emails and assigns tags
> based on the parse result.
> It may be sufficient for your purpose.

i'm a happy afew user :)

> Or you can modify notmuch to index the stuff you want.

ok

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]