bug#44509: 28.0.50; Error querying with new gnus-search and notmuch

[Eric Abrahamsen <eric@ericabrahamsen.net>]

"Jose A. Ortega Ruiz" <jao@gnu.org> writes:

> Hi again, Eric.
>
> I was looking at the notmuch engine code and wondering why it wasn't
> working for me in a leafnode directory, which puts its messages in a
> nnmail-compatible format (as far as i can tell).  And i discovered what
> looks like a possible bug in gnus-search-indexed (or a misunderstanding
> on my side).  Concretely, on line 1363 of gnus-search, when implementing
> gnus-search-indexed-parsed output, we're constructing a groups regexp
> that looks like:
>
> 	(group-regexp (when groups
> 			(regexp-opt
> 			 (mapcar
> 			  (lambda (x) (gnus-group-real-name x))
> 			  groups))))
>
> and then matching the returned list of files on that regexp (line 1377):
>
>        (string-match-p group-regexp f-name)))
>
> But gnus-group-real-name is giving me back a dot-separated path for
> nested folders (e.g. gmane.bugs.gnus), while the file paths in the
> results are using slashes (gmane/bugs/gnus/234 etc.), so the match
> test always fails and no results are returned.
>
> If i simply redefine group-regexp using:
>
>     (replace-regexp-in-string "\\." "/" (gnus-group-real-name x)))

This code that comes straight from the old nnir.el, and has always
looked a bit fragile to me. The problem is that it really just expects
each message to be in a regular file, with groups as folders.

So you're using notmuch as a search engine for a local leafnode nntp
server, and indexing its message store directly?

Is there any leafnode setting that could influence how it stores its
messages? Can it be convinced to store them in hierarchical folders?

I suppose I could change the group-regexp to munge periods, but that
could cause breakage in other cases, and I would be hesitant to do that.

Otherwise, all the indexed search engines have a
`gnus-search-indexed-extract' method that's used to actually return the
file name from the results buffer. Each one's got something slightly
different. It would be very easy to create a new notmuch search engine
subclass that only overrides this method. To wit:

(defclass gnus-search-leafnode-notmuch (gnus-search-notmuch))

(cl-defmethod gnus-search-indexed-extract ((_engine gnus-search-leafnode-notmuch))
  (let ((results-string (buffer-substring-no-properties
			 (line-beginning-position)
			 (line-end-position))))
    (prog1
	(list (replace-regexp-in-string
	       "periods" "forward slashes" results-string)
	      100)
      (forward-line))))

The `replace-regexp-in-string' changes the periods into forward
slashes. Then in your Gnus config:

'(nntp "your-local-leafnode"
   (gnus-search-engine gnus-search-leafnode-notmuch))

This is what I would do (I haven't tested the above, it might require
some tweaking). It takes advantage of all the benefits of the
generic-method approach, and lets you change behavior without messing
with the rest of the gnus-search code. You might also find it helpful to
tweak a few other slots or methods for this engine in particular.

> On other news, i was trying to find a way in Gnus to go from Message-ID
> to article no. for IMAP groups or nnmaildirs (which would make using
> notmuch with dovecot really trivial), but without luck: anyone knows of
> an easy way?

Come to think of it, nnimap can already accept article numbers
as message-ids. So if notmuch returns its results as message-ids, it
should work transparently.

The problem is that while the mechanism is there, it works by searching
each message-id and getting the proper article number that way. My guess
is that you'd be negating most of the speed advantage of using notmuch
like this, and you'd still be better off using dovecot's own full-text
indexing. You don't have to use xapian!

https://doc.dovecot.org/configuration_manual/fts/

BUT, if you really wanted to do this, it would be relatively easy to
check for "--output=messages" in 'switches, and use that instead of
"--output=files".

Eric