From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Eric Abrahamsen Newsgroups: gmane.emacs.bugs Subject: bug#44509: 28.0.50; Error querying with new gnus-search and notmuch Date: Wed, 11 Nov 2020 09:51:09 -0800 Message-ID: <877dqrhjea.fsf@ericabrahamsen.net> References: <87imag7kkh.fsf@gnus.jao.io> <87361kd1sg.fsf@ericabrahamsen.net> <87v9eg7ec3.fsf@gnus.jao.io> <874km0bgat.fsf@ericabrahamsen.net> <87blg8729u.fsf@gnus.jao.io> <875z6cxyue.fsf@gnus.jao.io> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="757"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux) Cc: 44509@debbugs.gnu.org To: "Jose A. Ortega Ruiz" Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Wed Nov 11 18:52:37 2020 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kcuIO-000AdT-Sp for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 11 Nov 2020 18:52:37 +0100 Original-Received: from localhost ([::1]:35558 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kcuIN-0003JC-Vr for geb-bug-gnu-emacs@m.gmane-mx.org; Wed, 11 Nov 2020 12:52:36 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:33604) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kcuHw-0002vR-Rb for bug-gnu-emacs@gnu.org; Wed, 11 Nov 2020 12:52:12 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:59440) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kcuHq-0007qO-WC for bug-gnu-emacs@gnu.org; Wed, 11 Nov 2020 12:52:06 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kcuHq-0004Z4-VT for bug-gnu-emacs@gnu.org; Wed, 11 Nov 2020 12:52:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Eric Abrahamsen Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Wed, 11 Nov 2020 17:52:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 44509 X-GNU-PR-Package: emacs Original-Received: via spool by 44509-submit@debbugs.gnu.org id=B44509.160511707917481 (code B ref 44509); Wed, 11 Nov 2020 17:52:02 +0000 Original-Received: (at 44509) by debbugs.gnu.org; 11 Nov 2020 17:51:19 +0000 Original-Received: from localhost ([127.0.0.1]:42750 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kcuH8-0004Xs-TU for submit@debbugs.gnu.org; Wed, 11 Nov 2020 12:51:19 -0500 Original-Received: from ericabrahamsen.net ([52.70.2.18]:48500 helo=mail.ericabrahamsen.net) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kcuH6-0004Xa-RU for 44509@debbugs.gnu.org; Wed, 11 Nov 2020 12:51:18 -0500 Original-Received: from localhost (24-113-150-48.wavecable.com [24.113.150.48]) (Authenticated sender: eric@ericabrahamsen.net) by mail.ericabrahamsen.net (Postfix) with ESMTPSA id 1A197FA16E; Wed, 11 Nov 2020 17:51:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericabrahamsen.net; s=mail; t=1605117071; bh=mqVQrQ2QdaWnziJ6B6aWwID3I8Akiuf5ypy4FRer3ms=; h=From:To:Cc:Subject:References:Date:In-Reply-To:From; b=ouI+2iYWAzwgpAc+OaLYwKPFv3l1SDrmvgJbKEQNCLndpkyQUARk4MM0Z/Boe2Gpx XWCSLpqpprKSGJ9hvjQzEXVeiHtIHqMxwZPNmwAqdz1M5pY5pjsRDHuuSxF0U1dL6C cjggXbqHdj6W5GsdGCYjArwjaMcn4SyYiHmuSxGk= In-Reply-To: <875z6cxyue.fsf@gnus.jao.io> (Jose A. Ortega Ruiz's message of "Wed, 11 Nov 2020 05:10:49 +0000") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:193120 Archived-At: "Jose A. Ortega Ruiz" writes: > Hi again, Eric. > > I was looking at the notmuch engine code and wondering why it wasn't > working for me in a leafnode directory, which puts its messages in a > nnmail-compatible format (as far as i can tell). And i discovered what > looks like a possible bug in gnus-search-indexed (or a misunderstanding > on my side). Concretely, on line 1363 of gnus-search, when implementing > gnus-search-indexed-parsed output, we're constructing a groups regexp > that looks like: > > (group-regexp (when groups > (regexp-opt > (mapcar > (lambda (x) (gnus-group-real-name x)) > groups)))) > > and then matching the returned list of files on that regexp (line 1377): > > (string-match-p group-regexp f-name))) > > But gnus-group-real-name is giving me back a dot-separated path for > nested folders (e.g. gmane.bugs.gnus), while the file paths in the > results are using slashes (gmane/bugs/gnus/234 etc.), so the match > test always fails and no results are returned. > > If i simply redefine group-regexp using: > > (replace-regexp-in-string "\\." "/" (gnus-group-real-name x))) This code that comes straight from the old nnir.el, and has always looked a bit fragile to me. The problem is that it really just expects each message to be in a regular file, with groups as folders. So you're using notmuch as a search engine for a local leafnode nntp server, and indexing its message store directly? Is there any leafnode setting that could influence how it stores its messages? Can it be convinced to store them in hierarchical folders? I suppose I could change the group-regexp to munge periods, but that could cause breakage in other cases, and I would be hesitant to do that. Otherwise, all the indexed search engines have a `gnus-search-indexed-extract' method that's used to actually return the file name from the results buffer. Each one's got something slightly different. It would be very easy to create a new notmuch search engine subclass that only overrides this method. To wit: (defclass gnus-search-leafnode-notmuch (gnus-search-notmuch)) (cl-defmethod gnus-search-indexed-extract ((_engine gnus-search-leafnode-notmuch)) (let ((results-string (buffer-substring-no-properties (line-beginning-position) (line-end-position)))) (prog1 (list (replace-regexp-in-string "periods" "forward slashes" results-string) 100) (forward-line)))) The `replace-regexp-in-string' changes the periods into forward slashes. Then in your Gnus config: '(nntp "your-local-leafnode" (gnus-search-engine gnus-search-leafnode-notmuch)) This is what I would do (I haven't tested the above, it might require some tweaking). It takes advantage of all the benefits of the generic-method approach, and lets you change behavior without messing with the rest of the gnus-search code. You might also find it helpful to tweak a few other slots or methods for this engine in particular. > On other news, i was trying to find a way in Gnus to go from Message-ID > to article no. for IMAP groups or nnmaildirs (which would make using > notmuch with dovecot really trivial), but without luck: anyone knows of > an easy way? Come to think of it, nnimap can already accept article numbers as message-ids. So if notmuch returns its results as message-ids, it should work transparently. The problem is that while the mechanism is there, it works by searching each message-id and getting the proper article number that way. My guess is that you'd be negating most of the speed advantage of using notmuch like this, and you'd still be better off using dovecot's own full-text indexing. You don't have to use xapian! https://doc.dovecot.org/configuration_manual/fts/ BUT, if you really wanted to do this, it would be relatively easy to check for "--output=messages" in 'switches, and use that instead of "--output=files". Eric