On Thu, 08 Apr 2010 07:58:47 -0700, Dirk Hohndel <hohndel@infradead.org> wrote:
> On Thu, 08 Apr 2010 10:03:15 -0400, Jameson Rollins <jrollins@finestructure.net> wrote:
> > Also, fwiw, the folder: indexing is probably the new feature that I'm
> > most eagerly awaiting.  I've got all these ideas for ways to handle sent
> > mail and drafts if we can get this working.
> 
> Maybe I am missing some context here - what exactly does 'indexing' mean
> here?

Indexing here means shoving information into the database to make it
searchable.

I was a bit vague in my description of the features I want to enable
here. So let me start making it a bit more clear. I'd like to add
support for the following:

  1. A "folder:" prefix in the search syntax that will match components
     of the directory in which mail files are stored. So, if you've got
     mail filed into directories, (from using a folder-based email
     program, or from doing an imap sync from gmail that turns tags into
     directories), then you could search for things like "folder:spam",
     "folder:important", "folder:arbitrary/subset/of/some/path", etc.

  2. A "list:" prefix to match content from headers that identify
     mailing list. My perception is that there are likely a handful of
     different headers that have been used, and they should all be
     indexed so that "list:" will search any of them.

  3. Some prefix that can be used to match typical headers added by
     spam-filtering software, (maybe this would be a general
     prefix---see below).

I think the above are probably the three I can think of that pretty much
everybody has asked for, so that should be indexed by default.

Beyond that, I'd like to be able to provide support for arbitrary
headers in the email. I had envisioned allowing the user to configure
specific headers to index. Alternately, I had imagined having a
blacklist of "uninteresting" headers and indexing everything
else. (Though, now Dirk is very interested in Received and its perhaps
provides the most content of any of the headers I originally thought
would be "uninteresting".)

One experiment I should do is to measure how much my database would grow
if I were to index all of the header content. I don't really know what
kind of a percentage we're talking about.

Finally, we'd need a syntax for searching all of these headers. Rather
than trying to map each header name to a custom prefix, I think what
might be best would be a general thing that could look like:

	header:"X-Spam-Flag: YES"

We don't currently have the ability to tie a search to the beginning of
a line, but it occurs to me that we could do something fancy like index
each header with a specific term to indicate line beginnings. Then, with
a custom query parser we could use the common symbol of '^' to map to
that. That would enable something like:

	header:"^X-Spam-Flag: YES"

That's a lot of work though, (and perhaps not as important as it's
probably uncommon for a header name to appear within the value of some
other header).

> Hmm - haven't even thought about drafts, yet. How would the UI deal with
> those? 

I would imagine another "folder" alongside the others that would list
all drafts, and selecting any such message would bring up the message in
a mode to edit it.

Sup also displayed draft messages in their proper location in the
threads containing the message being replied to, (and highlighted such
threads in the search view).

All of that sounds quite easy to do by simply saving the draft within
the mailstore and giving it a "draft" tag.

Finally, many mail interfaces prompt from the "compose new message"
command whether an existing draft should be continued. That's useful to
help the user avoid forgetting to complete and send a draft that was
forgotten.

-Carl