From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 9444E1F597; Fri, 3 Aug 2018 19:20:56 +0000 (UTC) Date: Fri, 3 Aug 2018 19:20:56 +0000 From: Eric Wong To: Konstantin Ryabitsev Cc: meta@public-inbox.org Subject: Re: Threading/searching problem Message-ID: <20180803192056.5swqcf67bsdxbpg6@dcvr> References: <20180803182647.GA28438@chatter> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20180803182647.GA28438@chatter> List-Id: Konstantin Ryabitsev wrote: > Hi, all: > > Something I came over today that seems to be wonky. I was trying to find > this message: > > https://lore.kernel.org/lkml/CA+55aFz5EWE9OTbzDoMfsY2ez04Qv9eg0KQhwKfyJY0vFvoD3g@mail.gmail.com/ > > It's from Linus about WireGuard, so I searched for it: > > https://lore.kernel.org/lkml/?q=torvalds+wireguard OK, WireGuard is in the body, there... > And it doesn't show up, which is really odd. I see some of the other search > strings also not show up: > > https://lore.kernel.org/lkml/?q=DL_FLAG_AUTOREMOVE_SUPPLIER > > it's in today's message: > https://lore.kernel.org/lkml/53f9f939dbce11e1c96986ff41f29dd6e41b9220.camel@nxp.com/ Likewise, DL_FLAG_AUTOREMOVE_SUPPLIER only appears in the body. > I'm wondering if it's related to incremental index fixes, but the cover > letter mentions deleted messages, which we haven't been doing. Since you're on mda, I think it's related to the mda + message body indexing fixes I posted: https://public-inbox.org/meta/20180729093441.5250-1-e@80x24.org/ I think it takes about 3-4 hours to reindex my local LKML archives on my fastest SSD (way longer on a slower SSD), so it might only take around 2 hours onthe machine used for lore. > There's also an issue of threading which has been irking some people on > LKML. If you look at this message again: > > https://lore.kernel.org/lkml/CA+55aFz5EWE9OTbzDoMfsY2ez04Qv9eg0KQhwKfyJY0vFvoD3g@mail.gmail.com/ > > You will notice that PI tries to build a subject-based thread all the way > back to 2008 and gives up somewhere in 2015. This makes it impossible to > navigate that message thread. Oops. We'll have to tweak the ordering for giant threads to favor newer messages. Gotta run for a bit but should be done today. > Any help would be greatly appreciated. If anyone wants a copy of Xapian/git > I can provide it on request (they are large!). I've got my own; but it might be worth looking into supporting remote indices a feature of Xapian.