From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <e@80x24.org>
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net
X-Spam-Level: 
X-Spam-ASN:  
X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,
	T_SCC_BODY_TEXT_LINE shortcircuit=no autolearn=ham autolearn_force=no
	version=3.4.6
Received: from localhost (dcvr.yhbt.net [127.0.0.1])
	by dcvr.yhbt.net (Postfix) with ESMTP id 4581E1F406;
	Sun, 17 Dec 2023 07:59:05 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org;
	s=selector1; t=1702799945;
	bh=FvikJUTixmzeRH27imritKc6N/Lmgv9giqzYfwA7jXw=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=xdnU3NpzsJd8ZmSLOdx9p3BWwnXaMXcMukrcFfb1D9g1oVa/dXd50yFxEczomzXgW
	 MryMHCzyepChyCz3WRB9/MvWVn84rcJEnlHFQiiqGTqPT9BHttfpliFP6ESoq1l2k5
	 jXU1rgj/e/XqQipMTlfldfNXmJ7YHKS/V2v8jExY=
Date: Sun, 17 Dec 2023 07:59:05 +0000
From: Eric Wong <e@80x24.org>
To: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Cc: meta@public-inbox.org
Subject: Re: [PATCH] lei: support reading MH for convert+import+index
Message-ID: <20231217075905.M151589@dcvr>
References: <20231216130932.479628-1-e@80x24.org>
 <20231216-strange-hamster-of-research-e7a7ea@lemur>
 <20231216181718.M211068@dcvr>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <20231216181718.M211068@dcvr>
List-Id: <meta.public-inbox.org>

Eric Wong <e@80x24.org> wrote:
> Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> > Nice, so eventually we should be able to specify the following instead of
> > faking out a maildir?
> > 
> > watch=mh:/var/spool/mlmmj/list.name/archive
> 
> Yes, that's the plan.

Well, reading /usr/lib/python*/mailbox.py on my system makes me cry:

    def pack(self):
        """Re-name messages to eliminate numbering gaps. Invalidates keys."""

That's for the Python stdlib MH class where I was looking for a
non-racy write implementation.

And checking the nmh source[1] reveals packing happens there, too...

Packing makes sense for a memory-efficient representation of
.mh_sequences without resorting to a tree or hash table; but it
invalidates `lei index' and forces -watch to do a full rescan if
anybody uses pack.  Ugh...

Fortunately, this doesn't seem to be the default behavior of nmh
(`nopack' appears to be the default).

[1] https://git.savannah.gnu.org/git/nmh.git sbr/folder_pack.c

> > > inotify|EVFILT_VNODE watches aren't supported, yet, either.

At least lei should have a reasonably fast way to handle this
using mail_sync.sqlite3 to compare SHA-(1|256) without having
to decode MIME/QP/Base-64 to get comparisons... I suppose
-watch needs to start using that, too...

> > In the case of mlmmj it's sufficient to watch the
> > /var/spool/mlmmj/list.name/index file for updates, but I don't know how well
> > this lends itself to other implementations (I am not at all familiar with MH).
> 
> Just watching the directory itself is sufficient (like Maildir)
> and will report new files.  We just have to check /\A[0-9]+\z/

At least mlmmj won't pack because it's an archive (or at
least it shouldn't....)