RFE: unified message-id lookup across all inboxes

RFE: unified message-id lookup across all inboxes

[Konstantin Ryabitsev]

Hello, all:

One of the features lore.kernel.org users have been asking for is
ability to retrieve a message based on its message-id regardless of
which inbox it's in. Perhaps look them up in the order they are listed
in the config file and display the first found?

The way it works currently is *almost* already what we want. E.g. if you
access:
https://lore.kernel.org/lkml/1545703503-20939-1-git-send-email-zhangkehong@hisilicon.com

It will tell you that the message is in another inbox (it was sent to
linux-arm-kernel and not cc'd to linux-kernel). The users don't like
that extra click, so they really just want a "show me the message if
it's anywhere on lore, and give me a link to marc.info, etc if not in
any of them).

It's been way too long since I've done Perl, which is why I'm not
offering patches, sorry! :)

Best,
-K

Re: RFE: unified message-id lookup across all inboxes

[Eric Wong]

Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> Hello, all:
> 
> One of the features lore.kernel.org users have been asking for is
> ability to retrieve a message based on its message-id regardless of
> which inbox it's in. Perhaps look them up in the order they are listed
> in the config file and display the first found?

Right, the NNTP interface already does that.  Ordering in NNTP
is random (because Perl hash order is random), but I guess there
could be an option to control sorting in the config.

There'll be another sorting-related config option for a feature
on the horizon...

> The way it works currently is *almost* already what we want. E.g. if you
> access:
> https://lore.kernel.org/lkml/1545703503-20939-1-git-send-email-zhangkehong@hisilicon.com

sidenote: I've been fighting memory use and that RFC is frightening :x

> It will tell you that the message is in another inbox (it was sent to
> linux-arm-kernel and not cc'd to linux-kernel). The users don't like
> that extra click, so they really just want a "show me the message if
> it's anywhere on lore, and give me a link to marc.info, etc if not in
> any of them).

Right.  I wanted to make it obvious to the user the message was
in a different inbox.  So I think the new endpoint would be
(without /$INBOX/ name):

	http://$HTTP_HOST/_/$MESSAGE_ID

And 302 the user to the first message.

> It's been way too long since I've done Perl, which is why I'm not
> offering patches, sorry! :)

No worries; I've got a bunch of other RFEs to take care of, too.

Also, I've been wondering if Ruby would have been a better
language for this project?

Python doesn't fit my brain, Xapian bindings for Lua aren't yet
in Debian, yet (RFP filed), but they're packaged for Ruby.  But
I've been hacking on the Ruby VM for nearly a decade...  I just
never get to use Ruby for development because I'm constantly
side-tracked into fixing and attempting to improve the VM and
language.  But VM hacking isn't really rewarding anymore, so I'm
taking a break from that.

Re: RFE: unified message-id lookup across all inboxes

[Konstantin Ryabitsev]

On Tue, Jan 08, 2019 at 01:54:20AM +0000, Eric Wong wrote:
> > It will tell you that the message is in another inbox (it was sent to
> > linux-arm-kernel and not cc'd to linux-kernel). The users don't like
> > that extra click, so they really just want a "show me the message if
> > it's anywhere on lore, and give me a link to marc.info, etc if not in
> > any of them).
> 
> Right.  I wanted to make it obvious to the user the message was
> in a different inbox.  So I think the new endpoint would be
> (without /$INBOX/ name):
> 
> 	http://$HTTP_HOST/_/$MESSAGE_ID
> 
> And 302 the user to the first message.

That sounds good to me.

> > It's been way too long since I've done Perl, which is why I'm not
> > offering patches, sorry! :)
> 
> No worries; I've got a bunch of other RFEs to take care of, too.
> 
> Also, I've been wondering if Ruby would have been a better
> language for this project?

I may be jaded from the times when Ruby's VM was really terrible, but
I'm still harbouring deep aversion to Ruby. I'll take Perl over Ruby any
day. :)

> Python doesn't fit my brain, Xapian bindings for Lua aren't yet
> in Debian, yet (RFP filed), but they're packaged for Ruby.  But
> I've been hacking on the Ruby VM for nearly a decade...  I just
> never get to use Ruby for development because I'm constantly
> side-tracked into fixing and attempting to improve the VM and
> language.  But VM hacking isn't really rewarding anymore, so I'm
> taking a break from that.

I don't think this is something that needs fixing. I expect it's easier
to just let people write custom web frontends to public-inbox
repositories (I suspect that's what LWN has done). In fact, I pondered
writing or commissioning a Django frontend if people needed more
shiny whizbang stuff out of the web interface, but kernel devs are the
kind of crowd that's served perfectly well by the current web frontend,
so I don't have a strong enough need to do anything there.

-K

Re: RFE: unified message-id lookup across all inboxes

[Eric Wong]

Konstantin Ryabitsev <konstantin@linuxfoundation.org> wrote:
> On Tue, Jan 08, 2019 at 01:54:20AM +0000, Eric Wong wrote:
> > 	http://$HTTP_HOST/_/$MESSAGE_ID
> > 
> > And 302 the user to the first message.
> 
> That sounds good to me.

Alright.  I think we can reserve '_' and some other punctuation
and make them invalid inbox names.

> I may be jaded from the times when Ruby's VM was really terrible, but
> I'm still harbouring deep aversion to Ruby. I'll take Perl over Ruby any
> day. :)

Heh, it was a large part of my full-time job for me to fix
RubyVM bugs for a few years.  So it ought to be better, now; and
all the TLS termination for https://public-inbox.org/git/ since
2016 has been done by a Ruby server without nginx (or similar).

> I don't think this is something that needs fixing. I expect it's easier
> to just let people write custom web frontends to public-inbox
> repositories (I suspect that's what LWN has done). In fact, I pondered
> writing or commissioning a Django frontend if people needed more
> shiny whizbang stuff out of the web interface, but kernel devs are the
> kind of crowd that's served perfectly well by the current web frontend,
> so I don't have a strong enough need to do anything there.

Yeah, I was wondering when somebody would write a heavier one
with more whizbang features.  But I also think
Thunderbird/sylpheed/... + NNTP (or POP3) could still do better.

I will certainly add user-customizable CSS for quote/diff
highlighting (and maybe code syntax highlighting, too).
At least I promised it last year to somebody privately.

But FWIW, I need to use giant fonts and can't deal with most
sites or UIs without "xrandr-invert-colors" (or "xcalib -i -a").

So UI is highly subjective and the only part of UI/UX I can
really understand are speed and stability.  To that end, I think
the WWW code be made even faster (and eliminate Varnish/nginx
even for busy sites on cheap VPS).  And the search queries +
mboxrd format won't change, so people can build on top of that
(but Xapian result sorting/accuracy may change).


V nyfb unir xvyyre srngher hc zl fyrrir juvpu V ubcr gb svavfu
vzcyrzragvat fbzrqnl...