From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <e@80x24.org>
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net
X-Spam-Level: 
X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00
	shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2
Received: from localhost (dcvr.yhbt.net [127.0.0.1])
	by dcvr.yhbt.net (Postfix) with ESMTP id E88CA1F9FD;
	Fri, 26 Feb 2021 09:26:48 +0000 (UTC)
Date: Fri, 26 Feb 2021 09:26:48 +0000
From: Eric Wong <e@80x24.org>
To: meta@public-inbox.org
Subject: Re: lei: per-message keywords and externals
Message-ID: <20210226092648.GA30618@dcvr>
References: <20210224204950.GA2076@dcvr>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20210224204950.GA2076@dcvr>
List-Id: <meta.public-inbox.org>

Eric Wong <e@80x24.org> wrote:
> Something I've been pondering for a bit is how to handle
> keywords (Seen, Important, Replied, ...) for messages stored in
> externals.
> 
> I want "kw:" prefix to be a usable search term, like:
> 
> 	lei q something interesting kw:seen
> 	lei q something interesting NOT kw:seen
> 
> This is no problem for imported messages in ~/.local/share/lei/store.
> All the keyword info is stored in line with the rest of the
> Xapian index data.
> 
> But, I also don't want to be wasting users' space by duplicating
> index data if they're already hosting inboxes for public
> consumption.  So, it's looking like parsing out kw: ourselves
> and do extra filtering on our end when externals are in play is
> going to be a requirement...

Something I considered a few weeks ago, but decided against, but
am again coming around to is indexing just the overview header
info in lei/store.  In other words:

	$sto->set_eml($eml->header_obj, @kw)

instead of:

	$sto->set_eml($eml, @kw)

> Or, just don't support searching using "kw:" with externals, for
> now; but still stash keywords somewhere when writing to
> traditional mail stores.

Maybe it'll be another instance of LeiStore in a separate dir
for external keywords: ~/.local/share/lei/xkw-store

> And there's also HTTP/HTTPS externals, but those will have
> transparent caching/memoization into lei/store by default, soon.

Done: https://public-inbox.org/meta/20210224233718.19007-3-e@80x24.org/