I've been working on a generalized searching for Gnus, where a single query language gets translated into different search-engine-appropriate strings. This allows searching multiple backends at once. It's more or less working, and I'm attaching the new version of the nnir.el file here which can be used as a drop-in replacement for the existing file. Ideally, if accepted, this would get rebased on top of Andy Cohen's reworking of nnir/nnselect. How it works: The query entered by the user is parsed into a sexp structure, and then each engine is responsible for interpreting that. For instance, you mark one IMAP group, and one maildir group (indexed with notmuch). Then you enter a query: "from:john after:1w or -mark:!" Internally, this becomes: ((from . "john") (or (since 14 4 2017) (not (mark . "flag")))) The imap engine turns that into: "FROM john OR SINCE 14-Apr-2017 UNFLAGGED" And the notmuch engine turns it into: "from:john date:4/14/2017.. or not tag:flag" Results from both servers are put in the same summary buffer. That's pretty much it, I hope people will be interested in this. I've started writing tests, and will do documentation if this is accepted. I've pasted the complete docstring of nnir-search-parse-query below. --------------------- Notes for the curious: The search engines are now implemented as classes. This allowed for factoring out a bunch of common code. I nearly set this up for running multiple searches each in their own thread, allowing for limited search concurrency. I backed off at the last minute because of weird IMAP behavior, but the code is pretty much set up for threads, if IMAP can get sorted out. I re-implemented a limited version of the IMAP LITERAL+ code I wrote years ago. If the server advertises support, searches for non-ASCII strings will make use the LITERAL+ mechanism. ¡¡Turning this on enforces CHARSET UTF-8!! Ie, the assumption is that if a server can handle LITERAL+, it can handle CHARSET UTF-8. This is probably totally wrong, but it would be easy to shut off, or fix if I can figure out how to DTRT. So far as I can tell, Hyrex and Swish-e are defunct. They're still in there, but their search transformation is lacking because there are no good docs. Namazu docs are also lacking: they give the examples of searching on "message-id", "from", and "subject" headers, but are there more? I don't know. I can't test because mknmz errors on my machine. Things I'd like to add: 1. Support for IMAP MULTISEARCH and FUZZY 2. A command to automatically update all engine indexes. 3. Regular expression searches for engines that support them. 4. Engines for lucene, solr, raw xapian, sphinx... What else are people using? There's a base class for locally-indexed search engines, so these should be easy to add. 5. Create an offline index of gmane messages, to be updated monthly. The gmane search engine would search locally but request remotely (only partly joking). ------------------------------ nnir-search-parse-query is a Lisp closure. (nnir-search-parse-query STRING) Turn STRING into an s-expression based query. The resulting query structure is passed to the various search backends, each of which adapts it as needed. The search "language" is essentially a series of key:value expressions. Key is most often a mail header, but there are other keys. Value is a string, quoted if it contains spaces. Key and value are separated by a colon, no space. Expressions are implictly ANDed; the "or" keyword can be used to OR. "not" will negate the following expression, or keys can be prefixed with a "-". The "near" operator will work for engines that understand it; other engines will convert it to "or". Parenthetical groups work as expected. A key that matches the name of a mail header will search that header. Search keys can be abbreviated so long as they remain unambiguous, ie "f" will search the "from" header. "s" will raise an error. Other keys: "address" will search all sender and recipient headers. "recipient" will search "To", "Cc", and "Bcc". "before" will search messages sent before the specified date (date specifications to come later). Date is exclusive. "after" (or its synonym "since") will search messages sent after the specified date. Date is inclusive. "mark" will search messages that have some sort of mark. Likely values include "flag", "seen", "read", "replied". It’s also possible to use Gnus’ internal marks, ie "mark:R" will be interpreted as mark:read. "tag" will search tags -- right now that’s translated to "keyword" in IMAP, and left as "tag" for notmuch. At some point this should also be used to search marks in the Gnus registry. "contact" will search messages to/from a contact. Contact management packages must push a function onto ‘nnir-search-contact-sources’, the docstring of which see, for this to work. "contact-from" does what you’d expect. "contact-to" searches the same headers as "recipient". Other keys can be specified, provided that the search backends know how to interpret them. Date values (any key in ‘nnir-search-date-keys’) can be provided in any format that ‘parse-time-string’ can parse (note that this can produce weird results). Dates with missing bits will be interpreted as the most recent occurance thereof (ie "march 03" is the most recent March 3rd). Lastly, relative specifications such as 1d (one day ago) are understood. This also accepts w, m, and y. m is assumed to be 30 days. This function will accept pretty much anything as input. Its only job is to parse the query into a sexp, and pass that on -- it is the job of the search backends to make sense of the structured query. Malformed, unusable or invalid queries will typically be silently ignored.