From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <e@80x24.org>
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net
X-Spam-Level: 
X-Spam-ASN:  
X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,BAYES_00,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no
	autolearn=ham autolearn_force=no version=3.4.6
Received: from localhost (dcvr.yhbt.net [127.0.0.1])
	by dcvr.yhbt.net (Postfix) with ESMTP id CE7D41F41C;
	Mon, 27 Mar 2023 21:38:49 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org;
	s=selector1; t=1679953129;
	bh=5Ch1infpO/N6GScpLH5UPuUH/mJwB+Ap18dL4huZkAE=;
	h=Date:From:To:Cc:Subject:References:In-Reply-To:From;
	b=IdPg+YSOSymFpsmys8YVXdjd01xm1mZCGs19eiaB6NAERUw59BWLhxsuaq1/AgUGK
	 6GyF2kfOo1uu35XTgXI/QzVoS+4BA//HpR+4uRNrE8wGrfUU0RgbQ4UgtJspye6mUm
	 d4xUrfEfdCCFKLO8LWTVorX3BqU5IGdFQV2sIuKI=
Date: Mon, 27 Mar 2023 21:38:49 +0000
From: Eric Wong <e@80x24.org>
To: Konstantin Ryabitsev <konstantin@linuxfoundation.org>
Cc: meta@public-inbox.org
Subject: Re: Cheap way to check for new messages in a thread
Message-ID: <20230327213849.M743623@dcvr>
References: <aaniyhk7wfm4e6m5mbukcrhevzoc6ftctyrfwvmz4fkykwwtlj@mverfng6ytas>
 <20230327191049.M277377@dcvr>
 <tdgola6w5slak6y232ka4pryjvavi7jwgeqgwsbyvjhyuy3xir@a3777kv7imvz>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <tdgola6w5slak6y232ka4pryjvavi7jwgeqgwsbyvjhyuy3xir@a3777kv7imvz>
List-Id: <meta.public-inbox.org>

Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> On Mon, Mar 27, 2023 at 07:10:49PM +0000, Eric Wong wrote:
> > > For the bugzilla integration work I'm doing, I need a way to check if there
> > > were any updates to a thread since the last check. Right now, I'm just
> > > grabbing the full thread, parsing it and seeing if there are any new
> > > message-IDs that we don't know about, but it's very wasteful. Any way to just
> > > issue something like "how many messages are in a thread with this message-id"
> > > or "are there any updates to a thread with this message-id since
> > > YYYYMMDDHHMMSS?
> > 
> >   lei q -t --only /path/to/(inbox|extindex) mid:$MSGID rt:APPROXIDATE..
> > 
> > Returns JSON and won't retrieve message bodies from git.
> 
> Ah, I was hoping to have a fully remote way of doing this.
> 
> > I wouldn't query down to the second due to propagation delays,
> > clock skew, etc, though.
> > 
> > There might be a JMAP endpoint I can implement for WWW which
> > only retrieves that info, but getting backreferences (required
> > by the JMAP spec) to work properly seemed painful.
> 
> What about a "bodiless" atom feed? It's already available per thread, so
> perhaps there could be a mode that skips the bodies or trims them after the
> first paragraph?

I thought about that, too; but I'm worried about having one-off
stuff that ends up needing to be supported indefinitely.

JMAP for this would take more time, but I'd be more comfortable
carrying it long-term.

I don't expect trimming after the first paragraph to be a huge
improvement.  Retrieving any part of the message from git and
dealing with MIME is expensive, anyways.  I wouldn't expect it
to be a big (if any) improvement compared to POST-ing for the
mbox.gz (&x=m&t=1) endpoint with rt:$SINCE..

The mbox.gz endpoints should be a bit more efficient for the
server than Atom feeds; decoding MIME and HTML escaping takes up
considerable CPU time.