Recommend an email index tool for Emacs?

Recommend an email index tool for Emacs?

[Leo]

Dear all,

There seems to be a handful of email indexing tools out there. I have
not used any of them because I used to keep all emails on the server
and use its search engine.

Now I got a few thousands of emails on my harddisk. I tried grep once
and it is not very fast. Thus do you think an indexing tool is handy
and which one to go?

Thank you.

-- 
Leo <sdl.web AT gmail.com>                         (GPG Key: 9283AA3F)

Re: Recommend an email index tool for Emacs?

[Xavier Maillard]

Hi,

It really depends on what tool you are using to read your emails.


-- 
Xavier

Re: Recommend an email index tool for Emacs?

[Leo]

Hi Xavier,

* Xavier Maillard (2006-12-31 13:29 +0100) said:
  ^^^^^^^^^^^^^^^
> It really depends on what tool you are using to read your emails.

Those emails are in Gnus nnml format.

But I really don't understand why an index tool should have knowledge
about the MUA. Email is a standard format, right?

regards,
-- 
Leo <sdl.web AT gmail.com>                         (GPG Key: 9283AA3F)

Re: Recommend an email index tool for Emacs?

[Charles philip Chan]

On 31 Dec 2006, sdl.web@gmail.com wrote:

> But I really don't understand why an index tool should have knowledge
> about the MUA. Email is a standard format, right?

We need to know what your emailer is so that we can tell you whether
there is any elisp code already written to interface your emailer with
the index and search program.

Charles

-- 
printk("; crashing the system because you wanted it\n");
        linux-2.6.6/fs/hpfs/super.c

Re: Recommend an email index tool for Emacs?

[Tim X]

Leo <sdl.web@gmail.com> writes:

> Hi Xavier,
>
> * Xavier Maillard (2006-12-31 13:29 +0100) said:
>   ^^^^^^^^^^^^^^^
>> It really depends on what tool you are using to read your emails.
>
> Those emails are in Gnus nnml format.
>
> But I really don't understand why an index tool should have knowledge
> about the MUA. Email is a standard format, right?
>

SMTP/ESTMP are standardised, but therre are a number of variations in
how your MUA will store restored files - mbox and mdirs are probably
the most common, but different mail readers often "enhance" or
slightly modify the format. For example, emacs VM has a slightly
modified mbox format, mh-e (mh) is a bit different to maildirs, I
can't remember what rmail's format is.  Consequently, if your mail
 indexing tool is going to be smarter than just a basic grep, it needs
to understand the format and be able to distinguish different
messages, body and header and hopefully know how to display the
message appropriately when you want to look at the results from a
search (i.e. know how to tell your MUA to retrieve a specific message
rather than just open a large mbox etc). 

Tim

-- 
tcross (at) rapttech dot com dot au

Re: Recommend an email index tool for Emacs?

[Leo]

* Tim X. (2007-01-02 10:22 +1100) said:
  ^^^^^^
> Leo <sdl.web@gmail.com> writes:
>
>> Hi Xavier,
>>
>> * Xavier Maillard (2006-12-31 13:29 +0100) said:
>>   ^^^^^^^^^^^^^^^
>>> It really depends on what tool you are using to read your emails.
>>
>> Those emails are in Gnus nnml format.
>>
>> But I really don't understand why an index tool should have knowledge
>> about the MUA. Email is a standard format, right?
>>
>
> SMTP/ESTMP are standardised, but therre are a number of variations in
> how your MUA will store restored files - mbox and mdirs are probably
> the most common, but different mail readers often "enhance" or
> slightly modify the format. For example, emacs VM has a slightly
> modified mbox format, mh-e (mh) is a bit different to maildirs, I
> can't remember what rmail's format is.  Consequently, if your mail
>  indexing tool is going to be smarter than just a basic grep, it needs
> to understand the format and be able to distinguish different
> messages, body and header and hopefully know how to display the
> message appropriately when you want to look at the results from a
> search (i.e. know how to tell your MUA to retrieve a specific message
> rather than just open a large mbox etc). 
>
> Tim

Thanks, Tim.

-- 
Leo <sdl.web AT gmail.com>                         (GPG Key: 9283AA3F)

Re: Recommend an email index tool for Emacs?

[Bill Wohler]

Leo <sdl.web@gmail.com> writes:

> Now I got a few thousands of emails on my harddisk. I tried grep once
> and it is not very fast. Thus do you think an indexing tool is handy
> and which one to go?

I use swish++, but mairix and namazu are also good choices.

MH-E handles these search engines out of the box. See
http://mh-e.sourceforge.net/manual/html/Searching.html#Searching

On my laptop, swish++ indexes my "latest" 45,000 messages for 570 MB
in a couple of minutes so I do it a couple of times a day.

-- 
Bill Wohler <wohler@newt.com>  http://www.newt.com/wohler/  GnuPG ID:610BD9AD

Re: Recommend an email index tool for Emacs?

[Leo]

* Bill Wohler (2007-01-07 13:58 -0800) said:
>> Now I got a few thousands of emails on my harddisk. I tried grep once
>> and it is not very fast. Thus do you think an indexing tool is handy
>> and which one to go?
>
> I use swish++, but mairix and namazu are also good choices.

Thanks.

> MH-E handles these search engines out of the box. See
> http://mh-e.sourceforge.net/manual/html/Searching.html#Searching
>
> On my laptop, swish++ indexes my "latest" 45,000 messages for 570 MB
> in a couple of minutes so I do it a couple of times a day.

Impressive.

-- 
Leo <sdl.web AT gmail.com>                         (GPG Key: 9283AA3F)

Re: Recommend an email index tool for Emacs?

[Charles philip Chan]

On 31 Dec 2006, sdl.web@gmail.com wrote:

> Now I got a few thousands of emails on my harddisk. I tried grep once
> and it is not very fast. Thus do you think an indexing tool is handy
> and which one to go?

  http://www.emacswiki.org/cgi-bin/emacs-en/IndexMail

I personally use Namazu. With some work Gnus can be setup to use mairix
too.

Charles

-- 
/* Nobody will ever see this message :-) */
panic("Cannot initialize video hardware\n");
        linux-2.0.38/arch/m68k/atari/atafb.c

Re: Recommend an email index tool for Emacs?

[Leo]

* Charles Philip Chan (2006-12-31 08:49 -0500) said:
  ^^^^^^^^^^^^^^^^^^^
> On 31 Dec 2006, sdl.web@gmail.com wrote:
>
>> Now I got a few thousands of emails on my harddisk. I tried grep once
>> and it is not very fast. Thus do you think an indexing tool is handy
>> and which one to go?
>
>   http://www.emacswiki.org/cgi-bin/emacs-en/IndexMail
>
> I personally use Namazu. With some work Gnus can be setup to use mairix
> too.
>
> Charles

I have read that Wiki page. The thing is it failed to compare each
tool.

-- 
Leo <sdl.web AT gmail.com>                         (GPG Key: 9283AA3F)

Re: Recommend an email index tool for Emacs?

[Charles philip Chan]

On 31 Dec 2006, sdl.web@gmail.com wrote:

> I have read that Wiki page. The thing is it failed to compare each
> tool.

All the variants of Swish and Namazu are general full-text search
engines. From what I know Namazu:

     http://namazu.org/index.html.en

is lighter than Swish on resources and faster to index. It also comes
with an Emacs frontend to search any Namazu db.

Mairix:

     http://www.rpcurnow.force9.co.uk/mairix/   

however is designed for just emails. It's indexing is very fast and
light on resources.

Charles

-- 
printk("CPU[%d]: Sending penguins to jail...",smp_processor_id());
        linux-2.4.8/arch/sparc64/kernel/smp.c

Re: Recommend an email index tool for Emacs?

[Leo]

* Charles Philip Chan (2006-12-31 10:49 -0500) said:
  ^^^^^^^^^^^^^^^^^^^
> On 31 Dec 2006, sdl.web@gmail.com wrote:
>
>> I have read that Wiki page. The thing is it failed to compare each
>> tool.
>
> All the variants of Swish and Namazu are general full-text search
> engines. From what I know Namazu:
>
>      http://namazu.org/index.html.en
>
> is lighter than Swish on resources and faster to index. It also comes
> with an Emacs frontend to search any Namazu db.
>
> Mairix:
>
>      http://www.rpcurnow.force9.co.uk/mairix/   
>
> however is designed for just emails. It's indexing is very fast and
> light on resources.
>
> Charles

Thank you Charles. I will look into those tools.

It seems Gnome is adopting another tool -- tracker (doesn't depend on
Gnome at all). It is new, so there isn't an Emacs interface to it.

http://freedesktop.org/wiki/Software/Tracker

,----[ tracker ]
| Tracker is a powerful desktop-neutral first class object database,
| tag/metadata database, search tool and indexer.
| 
| It consists of a common object database that allows entities to have
| an almost infinte number of properties, metadata (both
| embedded/harvested as well as user definable), a comprehensive
| database of keywords/tags and links to other entities.
| 
| It provides additional features for file based objects including
| context linking and audit trails for a file object.
| 
| It has the ability to index, store, harvest metadata. retrieve and
| search all types of files and other first class objects.
`----

-- 
Leo <sdl.web AT gmail.com>                         (GPG Key: 9283AA3F)

Re: Recommend an email index tool for Emacs?

[Charles philip Chan]

On 31 Dec 2006, sdl.web@gmail.com wrote:

> It seems Gnome is adopting another tool -- tracker (doesn't depend on
> Gnome at all). It is new, so there isn't an Emacs interface to it.

I have heard about Tracker, but I haven't read up on it until now- it
looks interesting. I am glad that Gnome is going to adopt
freedsektop.org's Tracker instead of sticking with Beagle (which I have
no kind words for).

Charles

-- 
panic("Aarggh: attempting to free lock with active wait queue - shoot Andy");
        linux-2.0.38/fs/locks.c

Re: Recommend an email index tool for Emacs?

[Tim X]

"Charles philip Chan" <cpchan@sympatico.ca> writes:

> On 31 Dec 2006, sdl.web@gmail.com wrote:
>
>> It seems Gnome is adopting another tool -- tracker (doesn't depend on
>> Gnome at all). It is new, so there isn't an Emacs interface to it.
>
> I have heard about Tracker, but I haven't read up on it until now- it
> looks interesting. I am glad that Gnome is going to adopt
> freedsektop.org's Tracker instead of sticking with Beagle (which I have
> no kind words for).
>

I must admit feeling a bit the same. At first, I thought beagle was
going to be good, but I'm not that impressed wiht it and it wouldn't
support my VM mail folders. I'm also not impressed with Novell's
"deal" with MS over patents and wonder about the future of mono.

Now I'm off to check out Tracker!

Tim

-- 
tcross (at) rapttech dot com dot au

Re: Recommend an email index tool for Emacs?

[Leo]

* Tim X. (2007-01-01 13:56 +1100) said:
  ^^^^^^
> "Charles philip Chan" <cpchan@sympatico.ca> writes:
>
>> On 31 Dec 2006, sdl.web@gmail.com wrote:
>>
>>> It seems Gnome is adopting another tool -- tracker (doesn't depend on
>>> Gnome at all). It is new, so there isn't an Emacs interface to it.
>>
>> I have heard about Tracker, but I haven't read up on it until now- it
>> looks interesting. I am glad that Gnome is going to adopt
>> freedsektop.org's Tracker instead of sticking with Beagle (which I have
>> no kind words for).
>>
>
> I must admit feeling a bit the same. At first, I thought beagle was
> going to be good, but I'm not that impressed wiht it and it wouldn't
> support my VM mail folders. I'm also not impressed with Novell's
> "deal" with MS over patents and wonder about the future of mono.
>
> Now I'm off to check out Tracker!
>
> Tim

Glad you like it.

It includes a cli tool `tracker-search'.

-- 
Leo <sdl.web AT gmail.com>                         (GPG Key: 9283AA3F)

Re: Recommend an email index tool for Emacs?

[Leo]

* Charles Philip Chan (2006-12-31 14:36 -0500) said:
  ^^^^^^^^^^^^^^^^^^^
> On 31 Dec 2006, sdl.web@gmail.com wrote:
>
>> It seems Gnome is adopting another tool -- tracker (doesn't depend
>> on Gnome at all). It is new, so there isn't an Emacs interface to
>> it.
>
> I have heard about Tracker, but I haven't read up on it until now-
> it looks interesting. I am glad that Gnome is going to adopt
> freedsektop.org's Tracker instead of sticking with Beagle (which I
> have no kind words for).

I don't like beagle either. Too obtrusive.

-- 
Leo <sdl.web AT gmail.com>                         (GPG Key: 9283AA3F)