Re: IMAP server [was: Q: V2 format]

[ebiederm@xmission.com (Eric W. Biederman)]

Johannes Berg <johannes@sipsolutions.net> writes:

> On Fri, 2018-09-28 at 23:01 +0200, Eric W. Biederman wrote:
>> 
>> I have looked at gnus and there is support in there for performing
>> searches via the old gmane web interface.  Public inbox already provides
>> an attribute that tells you what the web server is.  So all it will
>> really take is someone with a little time to wire up the search
>> interface.
>
> That's ... interesting, but of course completely out-of-band. I'm not
> sure it should or could be advocated that every email client actually
> implement that :-)
>
> But if you think broader than that, you don't even necessarily need a
> web server to run p-i.

>> Beyond that if you have the archives local (and that is easy) it is
>> quite possible to just git grep through them and find things of
>> interest.
>
> That also doesn't use the index, not sure how that's any better?

So for linux-kernel.  I have 7G for the git email archive and 65G more
for the indexes.  Which makes the indexes quite expensive.  So for
personal use I am not certain an archive is a benefit.  Especially when
the email archive fits in ram and the index does not.

I have to wonder if there is a way to make the indexes an order of
magnitude smaller.

>> I should verify this but I don't think IMAP has a good version of the
>> NNTP overview database.  Which seems to make IMAP quite a bit slower for
>> reading news.  Certainly gnus+public-inbox locally is running quite a
>> bit faster than my old gnus+cyrus-imap configuration.
>
> IMAP servers typically should do header/MIME parsing, so you should be
> able to query such a thing - but not as easily as XOVER, I suppose.
>
> However, I think FETCH could be made to return the data similar to
> XOVER, though it may not be backed by a pre-created database file, and
> it depends on what the client does to show the overview in the first
> place.

>> I tried to read through the IMAP search specification to see how it
>> compares with what public-inbox makes available and I did not get
>> particularly far.   It was not easy to match up the various search
>> capabilities.  The biggest issue is that IMAP tends to not talk
>> about message-ids.  Where that is fundamentally one of the most
>> important fields to index if you are dealing with threaded mail.
>
> You can search for arbitrary headers in search by using
>
> HEADER <field-name> <string>
>
> where the string is "contains", so you can use it for both Message-Id
> and References headers.

>> So long story short while I am not opposed to a read-only IMAP
>> configuration I think NNTP has much to recommend it.  I do think we need
>> little things like SSL support for NNTP.  Just to prevent inappropriate
>> access to traffic in flight.
>
> Sure. I'm not saying NNTP is bad, just saying that the choice of clients
> is rather limited. Also, posting isn't supported over NNTP, so if I had
> it all in my email client I could read in the public-inbox archive, and
> respond via normal email.

The thing I can  confirm and I have gotten as far as  is that nntp has a
sequential  message  id,  and  IMAP  has a  sequential  message  id  and
public-inbox has  a sequential message  id (now reliably based  upon the
order of the  messages in the git  archive).  So it is  very possible to
have a read-only IMAP view.

The really noticable downside of IMAP is that it does want to keep the
status of messages you have read on the server.  That makes a read-only
archive a bit of a pain.

So I am not certain the choice of clients when you restrict IMAP to what
is an advantage.  Nor am I certain the general IMAP search functionality
maps well to what public-inbox indexes or people want to search for.

Which is me again saying while things can make I am not certain IMAP is
the best protocol for the job.

>> It won't be for a while yet but I have some scripts I need to push at
>> least to the public-inbox scripts directory that simplify the process
>> taking a single email address subscribing to email and sorting it out
>> into different public-inbox git archives.  Currently I have every
>> mailling list I am subscribed to pushed into public-inbox.
>
> :-)

I do love that public-inbox makes it very easy to archive all my content
and still be able to take it all with me when I travel.

Eric