Thoughts on search-based imap mailboxes

Thoughts on search-based imap mailboxes

[Konstantin Ryabitsev]

Hello:

While discussing something else on the kernel.org users list, the 
question of "virtual inbox folders" came up when talking about imap and 
public-inbox. Here's how I imagine it could work in a way that doesn't 
require any kind of real user management.

- any site visitor can create a saved search against any index, which 
  would be generated with a random name (e.g. guid4) and stored on the 
  server via a simple guid->(index, search terms) mapping
- if imap is enabled, any user can also create an imap "account" that 
  would also be a guid random string, perhaps with a random password as 
  well (or no password, since guid4 strings should suffice)
- users can freely associate saved searches with these accounts, perhaps 
  optionally allowing them to save them with aliases, such that the 
  record is:
  user_guid->(saved_search_guid, alias)
- when logging into imap with an actual random account name, the 
  associated inboxes would be displayed instead of the general listing
- accounts and saved searches that were not accessed within $DAYS would 
  be automatically purged

What are your thoughts about this scheme? I like it because it respects 
users' privacy, since we cannot map accounts to developers, and because 
it doesn't require any kind of central identity gatekeeping. Anyone 
setting up a public-inbox mirror can immediately start using this scheme 
without needing to set up any kind of identity management backends.

-K

Re: Thoughts on search-based imap mailboxes

[Eric Wong]

Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> Hello:
> 
> While discussing something else on the kernel.org users list, the 

Btw, is this list public?

> question of "virtual inbox folders" came up when talking about imap and 
> public-inbox. Here's how I imagine it could work in a way that doesn't 
> require any kind of real user management.
> 
> - any site visitor can create a saved search against any index, which 
>   would be generated with a random name (e.g. guid4) and stored on the 
>   server via a simple guid->(index, search terms) mapping
> - if imap is enabled, any user can also create an imap "account" that 
>   would also be a guid random string, perhaps with a random password as 
>   well (or no password, since guid4 strings should suffice)
> - users can freely associate saved searches with these accounts, perhaps 
>   optionally allowing them to save them with aliases, such that the 
>   record is:
>   user_guid->(saved_search_guid, alias)
> - when logging into imap with an actual random account name, the 
>   associated inboxes would be displayed instead of the general listing
> - accounts and saved searches that were not accessed within $DAYS would 
>   be automatically purged
> 
> What are your thoughts about this scheme? I like it because it respects 
> users' privacy, since we cannot map accounts to developers, and because 
> it doesn't require any kind of central identity gatekeeping. Anyone 
> setting up a public-inbox mirror can immediately start using this scheme 
> without needing to set up any kind of identity management backends.

I actually considered something nearly identical several years
ago with NNTP (and GUIDs or base-64-queries as NNTP group
names).

Maybe it can be done; but temporary storage + expiration
policies would require more admin overhead and possible
permissions problems if JMAP and IMAP are run as different users.
The GUID/base64 mapping might also require a dedicated
client-side tool and might not be very usable from normal MUAs.

Right now, one of the most annoying thing for me with using
public-inbox-imapd is the inability to mark messages as Seen
using normal MUAs.

A client-side tool is likely required anyways, I'm thinking
having saved search functionality in a local tool writing to
Maildir/mbox might be the best way forward as w.r.t. scalability
and offline access (and reading + saving flags from the
Maildir/mbox).

External/detached index I'm working on for global search will
also be appropriate for client-side tooling, I think, as it's
being designed with non-public-inbox data sources in mind, too.

Re: Thoughts on search-based imap mailboxes

[Konstantin Ryabitsev]

On Fri, Oct 02, 2020 at 08:08:30PM +0000, Eric Wong wrote:
> Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> > Hello:
> > 
> > While discussing something else on the kernel.org users list, the 
> 
> Btw, is this list public?

It's not, because it's supposed to be just for people with accounts on 
kernel.org and for things like maintenance and service outage 
notifications. However, discussions routinely veer into various 
directions, as one would expect.

> A client-side tool is likely required anyways, I'm thinking
> having saved search functionality in a local tool writing to
> Maildir/mbox might be the best way forward as w.r.t. scalability
> and offline access (and reading + saving flags from the
> Maildir/mbox).
> 
> External/detached index I'm working on for global search will
> also be appropriate for client-side tooling, I think, as it's
> being designed with non-public-inbox data sources in mind, too.

Instead of a "client-side" can it just be an intermediary tool that can 
run either on the client, or on any other piece of infrastructure? The 
main reason I'm interested in that is because several people I know use 
very thin clients for their work (chromebooks) and I would like to offer 
them this service as a perk of kernel.org instead of forcing them to set 
it up and maintain on their own.

-K

Re: Thoughts on search-based imap mailboxes

[Eric Wong]

Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> On Fri, Oct 02, 2020 at 08:08:30PM +0000, Eric Wong wrote:
> > A client-side tool is likely required anyways, I'm thinking
> > having saved search functionality in a local tool writing to
> > Maildir/mbox might be the best way forward as w.r.t. scalability
> > and offline access (and reading + saving flags from the
> > Maildir/mbox).
> > 
> > External/detached index I'm working on for global search will
> > also be appropriate for client-side tooling, I think, as it's
> > being designed with non-public-inbox data sources in mind, too.
> 
> Instead of a "client-side" can it just be an intermediary tool that can 
> run either on the client, or on any other piece of infrastructure? The 
> main reason I'm interested in that is because several people I know use 
> very thin clients for their work (chromebooks) and I would like to offer 
> them this service as a perk of kernel.org instead of forcing them to set 
> it up and maintain on their own.

What are the limitations of chromebooks?  Small eMMC storage?

Can they run curl, perl and git?  I could probably work with
that; maybe just curl + POSIX sh.  I get Xapian might be
stretching it, but they probably have SQLite already (though not
the Perl bindings)

CPU and RAM on Chromebooks seem to be as good or more powerful
than the ancient Thinkpads I use.