From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <e@80x24.org>
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net
X-Spam-Level: 
X-Spam-ASN:  
X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,BAYES_00,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no
	autolearn=ham autolearn_force=no version=3.4.6
Received: from localhost (dcvr.yhbt.net [127.0.0.1])
	by dcvr.yhbt.net (Postfix) with ESMTP id 12F0A1F47D;
	Sun, 26 Feb 2023 17:09:32 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org;
	s=selector1; t=1677431372;
	bh=0aNzAHp961FgANDk0Qj61qGiO2kNuJD0VGswiVwG5B0=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=CSmSitm7siA0J6/wbWzLW3MOmrlIOCxgQkUb+uefDgH5DjyuICcuGAW4iVB5nYG2Z
	 DEa9iJQ3mqsnjTrZmWevlk1MNo5A4UOIkOi0qI7KULmpWnv0b30YxDEItvZA4zctkw
	 J7oD2A6MeS0Hyyy1T4AsM/0Tjr8SkvGCnWBeEWNo=
Date: Sun, 26 Feb 2023 17:09:31 +0000
From: Eric Wong <e@80x24.org>
To: Maxim Mikityanskiy <maxtram95@gmail.com>
Cc: meta@public-inbox.org, Kyle Meyer <kyle@kyleam.com>
Subject: Re: [PATCH] lei q: do not collapse threads with `-tt'
Message-ID: <20230226170931.M947721@dcvr>
References: <Y+pgBmj0jxR+cVkD@mail.gmail.com>
 <20230214024232.M64373@dcvr>
 <Y/tN2+pdG5z5WcYf@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <Y/tN2+pdG5z5WcYf@mail.gmail.com>
List-Id: <meta.public-inbox.org>

Maxim Mikityanskiy <maxtram95@gmail.com> wrote:
> On Tue, Feb 14, 2023 at 02:42:32AM +0000, Eric Wong wrote:
> > Maxim Mikityanskiy <maxtram95@gmail.com> wrote:
> > > lei q --no-save -a -o /tmp/lei-test -I 'https://lore.kernel.org/all' \
> > >     -tt 'a:syzbot AND rt:2023-01-01..2023-01-07'

<snip>

> > Yes, now it seems it's the collapsing optimization.

<snip>

> Sorry for taking too long, I finally found a minute to test it, and
> unfortunately I didn't see a difference. I queried for:
> 
> a:syzbot AND rt:2023-02-01..2023-02-07
> 
> and I still saw I lot of threads without a single flag.
> 
> I double-checked that the patch was actually applied, killed lei-daemon,
> and removed the mailbox directory, but it didn't help.

Ah, oops.  My original fix only works for locally-cloned inboxes;
but not remote (http/https) inboxes...

I think some inconsistency on the client side is also introduced
by using -I/--include vs --only; since -I/--include will use
previously-indexed messages in ~/.local/share/lei/store

Getting -tt to work on remote inboxes will take more effort.
I'm not sure which option is better:

1) Support t=2 natively in the WWW interface.  This requires
   both the server and client to be updated.  It may require
   extra dedupe step on the server, making it more expensive.
   Thinking out loud, I think the dedupe step can be avoided
   by sorting on THREADID...

2) use t=1 in the client as-is, but index the streamed mbox
   locally, first.  This requires a temporary Xapian DB to
   ensure there's no overlap if using --only.
   This only requires a client update, but likely adds more
   complexity.  It also delays updates to the Maildir,
   meaning all messages need to be downloaded before the MUA
   sees it...

I'm leaning towards 1...