bug#33122: 27.0.50; Allow use of Gnus search groups as notmuch :path search terms

bug#33122: 27.0.50; Allow use of Gnus search groups as notmuch :path search terms

[Eric Abrahamsen]

[-- Attachment #1: Type: text/plain, Size: 488 bytes --]


Searches in Gnus are always conducted on one or more groups. Right now,
using the notmuch search engine, it's not possible to turn those groups
into path: search terms in the notmuch query, though that's clearly the
sensible thing to do (it looks like rudimentary support for path
filtering was halfway added, but not quite).

This patch creates a new `nnir-notmuch-filter-group-names-function'
custom option which transforms group names into something notmuch is
likely to understand.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Allow-use-of-Gnus-search-groups-as-notmuch-path-sear.patch --]
[-- Type: text/x-patch, Size: 4140 bytes --]

From 4274d95d3eece4a58b1f712fbacb3a3baaab2986 Mon Sep 17 00:00:00 2001
From: Eric Abrahamsen <eric@ericabrahamsen.net>
Date: Tue, 23 Oct 2018 10:51:37 +0800
Subject: [PATCH] Allow use of Gnus search groups as notmuch path: search term

* lisp/gnus/nnir.el (nnir-notmuch-filter-group-names-function): New
  option governing whether and how to use Gnus' search groups as path:
  search terms to notmuch.
  (nnir-run-notmuch): Check and possibly use above variable.
---
 lisp/gnus/nnir.el | 56 ++++++++++++++++++++++++++++++++++-------------
 1 file changed, 41 insertions(+), 15 deletions(-)

diff --git a/lisp/gnus/nnir.el b/lisp/gnus/nnir.el
index 7e5f56e4dd..251e54e015 100644
--- a/lisp/gnus/nnir.el
+++ b/lisp/gnus/nnir.el
@@ -518,6 +518,26 @@ nnir-notmuch-remove-prefix
   :type '(regexp)
   :group 'nnir)
 
+(defcustom nnir-notmuch-filter-group-names-function
+  (lambda (g) (gnus-group-short-name g))
+  "Whether and how to use Gnus group names as \"path:\" search terms.
+When nil, the groups being searched in are not used as notmuch
+:path search terms.  It's still possible to use \"path:\" terms
+manually within the search query, however.
+
+When a function, map this function over all the group names.  By
+default this runs them through `gnus-group-short-name', and it is
+recommended to use this transform, at least.  Further
+transforms (for instance, converting \".\" to \"/\") can be
+added like so:
+
+\(add-function :filter-return
+   nnir-notmuch-filter-group-names-function
+   (lambda (g) (replace-regexp-in-string \"\\\\.\" \"/\" g)))"
+  :version "27.1"
+  :type '(choice function
+		 nil))
+
 ;;; Developer Extension Variable:
 
 (defvar nnir-engines
@@ -1505,23 +1525,28 @@ nnir-run-namazu
                                (> (nnir-artitem-rsv x)
                                   (nnir-artitem-rsv y)))))))))
 
-(defun nnir-run-notmuch (query server &optional _group)
+(defun nnir-run-notmuch (query server &optional groups)
   "Run QUERY against notmuch.
 Returns a vector of (group name, file name) pairs (also vectors,
-actually)."
-
-  ;; (when group
-  ;;   (error "The notmuch backend cannot search specific groups"))
+actually).  If GROUPS is a list of group names, use them to
+construct path: search terms (see the variable
+`nnir-notmuch-filter-group-names-function')."
 
   (save-excursion
-    (let ( (qstring (cdr (assq 'query query)))
-	   (groupspec (cdr (assq 'notmuch-group query)))
+    (let* ((qstring (cdr (assq 'query query)))
 	   (prefix (nnir-read-server-parm 'nnir-notmuch-remove-prefix server))
            artlist
 	   (article-pattern (if (string-match "\\`nnmaildir:"
 					      (gnus-group-server server))
-			       ":[0-9]+"
-			     "^[0-9]+$"))
+				":[0-9]+"
+			      "^[0-9]+$"))
+	   (groups (when nnir-notmuch-filter-group-names-function
+		     (mapcar nnir-notmuch-filter-group-names-function
+			     groups)))
+	   (pathquery (when groups
+			(mapconcat (lambda (g)
+				     (format " path:%s" g))
+				   groups " or")))
            artno dirnam filenam)
 
       (when (equal "" qstring)
@@ -1530,10 +1555,14 @@ nnir-run-notmuch
       (set-buffer (get-buffer-create nnir-tmp-buffer))
       (erase-buffer)
 
-      (if groupspec
-          (message "Doing notmuch query %s on %s..." qstring groupspec)
+      (if groups
+          (message "Doing notmuch query %s on %s..."
+		   qstring (mapconcat #'identity groups " "))
         (message "Doing notmuch query %s..." qstring))
 
+      (when groups
+	(setq qstring (concat qstring pathquery)))
+
       (let* ((cp-list `( ,nnir-notmuch-program
                          nil            ; input from /dev/null
                          t              ; output
@@ -1571,10 +1600,7 @@ nnir-run-notmuch
         (when (string-match article-pattern artno)
           (when (not (null dirnam))
 
-	    ;; maybe limit results to matching groups.
-	    (when (or (not groupspec)
-		      (string-match groupspec dirnam))
-	      (nnir-add-result dirnam artno "" prefix server artlist)))))
+	    (nnir-add-result dirnam artno "" prefix server artlist))))
 
       (message "Massaging notmuch output...done")
 
-- 
2.19.1

bug#33122: Acknowledgement (27.0.50; Allow use of Gnus search groups as notmuch :path search terms)

[Eric Abrahamsen]

[-- Attachment #1: Type: text/plain, Size: 94 bytes --]

Here's an updated version of the patch, thanks to Andreas Goesele for
reporting and testing.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Allow-use-of-Gnus-search-groups-as-notmuch-path-sear.patch --]
[-- Type: text/x-patch, Size: 4154 bytes --]

From 517ca66601e97babeda7a20150bee24df9c9975b Mon Sep 17 00:00:00 2001
From: Eric Abrahamsen <eric@ericabrahamsen.net>
Date: Tue, 23 Oct 2018 10:51:37 +0800
Subject: [PATCH] Allow use of Gnus search groups as notmuch path: search term

* lisp/gnus/nnir.el (nnir-notmuch-filter-group-names-function): New
  option governing whether and how to use Gnus' search groups as path:
  search terms to notmuch.
  (nnir-run-notmuch): Check and possibly use above variable.
---
 lisp/gnus/nnir.el | 58 +++++++++++++++++++++++++++++++++++------------
 1 file changed, 43 insertions(+), 15 deletions(-)

diff --git a/lisp/gnus/nnir.el b/lisp/gnus/nnir.el
index 7e5f56e4dd..ea7257d0c9 100644
--- a/lisp/gnus/nnir.el
+++ b/lisp/gnus/nnir.el
@@ -518,6 +518,26 @@ nnir-notmuch-remove-prefix
   :type '(regexp)
   :group 'nnir)
 
+(defcustom nnir-notmuch-filter-group-names-function
+  #'gnus-group-short-name
+  "Whether and how to use Gnus group names as \"path:\" search terms.
+When nil, the groups being searched in are not used as notmuch
+:path search terms.  It's still possible to use \"path:\" terms
+manually within the search query, however.
+
+When a function, map this function over all the group names.  By
+default this runs them through `gnus-group-short-name', and it is
+recommended to use this transform, at least.  Further
+transforms (for instance, converting \".\" to \"/\") can be
+added like so:
+
+\(add-function :filter-return
+   nnir-notmuch-filter-group-names-function
+   (lambda (g) (replace-regexp-in-string \"\\\\.\" \"/\" g)))"
+  :version "27.1"
+  :type '(choice function
+		 nil))
+
 ;;; Developer Extension Variable:
 
 (defvar nnir-engines
@@ -1505,23 +1525,30 @@ nnir-run-namazu
                                (> (nnir-artitem-rsv x)
                                   (nnir-artitem-rsv y)))))))))
 
-(defun nnir-run-notmuch (query server &optional _group)
+(defun nnir-run-notmuch (query server &optional groups)
   "Run QUERY against notmuch.
 Returns a vector of (group name, file name) pairs (also vectors,
-actually)."
-
-  ;; (when group
-  ;;   (error "The notmuch backend cannot search specific groups"))
+actually).  If GROUPS is a list of group names, use them to
+construct path: search terms (see the variable
+`nnir-notmuch-filter-group-names-function')."
 
   (save-excursion
-    (let ( (qstring (cdr (assq 'query query)))
-	   (groupspec (cdr (assq 'notmuch-group query)))
+    (let* ((qstring (cdr (assq 'query query)))
 	   (prefix (nnir-read-server-parm 'nnir-notmuch-remove-prefix server))
            artlist
 	   (article-pattern (if (string-match "\\`nnmaildir:"
 					      (gnus-group-server server))
-			       ":[0-9]+"
-			     "^[0-9]+$"))
+				":[0-9]+"
+			      "^[0-9]+$"))
+	   (groups (when nnir-notmuch-filter-group-names-function
+		     (mapcar nnir-notmuch-filter-group-names-function
+			     groups)))
+	   (pathquery (when groups
+			(concat "("
+			 (mapconcat (lambda (g)
+				      (format " path:%s" g))
+				    groups " or")
+			 ")")))
            artno dirnam filenam)
 
       (when (equal "" qstring)
@@ -1530,10 +1557,14 @@ nnir-run-notmuch
       (set-buffer (get-buffer-create nnir-tmp-buffer))
       (erase-buffer)
 
-      (if groupspec
-          (message "Doing notmuch query %s on %s..." qstring groupspec)
+      (if groups
+          (message "Doing notmuch query %s on %s..."
+		   qstring (mapconcat #'identity groups " "))
         (message "Doing notmuch query %s..." qstring))
 
+      (when groups
+	(setq qstring (concat qstring pathquery)))
+
       (let* ((cp-list `( ,nnir-notmuch-program
                          nil            ; input from /dev/null
                          t              ; output
@@ -1571,10 +1602,7 @@ nnir-run-notmuch
         (when (string-match article-pattern artno)
           (when (not (null dirnam))
 
-	    ;; maybe limit results to matching groups.
-	    (when (or (not groupspec)
-		      (string-match groupspec dirnam))
-	      (nnir-add-result dirnam artno "" prefix server artlist)))))
+	    (nnir-add-result dirnam artno "" prefix server artlist))))
 
       (message "Massaging notmuch output...done")
 
-- 
2.19.1

bug#33122: Acknowledgement (27.0.50; Allow use of Gnus search groups as notmuch :path search terms)

[Eli Zaretskii]

> From: Eric Abrahamsen <eric@ericabrahamsen.net>
> Date: Wed, 24 Oct 2018 08:46:29 +0800
> 
> Here's an updated version of the patch, thanks to Andreas Goesele for
> reporting and testing.

Thanks, pushed to the master branch.

bug#33122: Acknowledgement (27.0.50; Allow use of Gnus search groups as notmuch :path search terms)

[Eric Abrahamsen]

[-- Attachment #1: Type: text/plain, Size: 495 bytes --]


On 11/03/18 10:43 AM, Eli Zaretskii wrote:
>> From: Eric Abrahamsen <eric@ericabrahamsen.net>
>> Date: Wed, 24 Oct 2018 08:46:29 +0800
>> 
>> Here's an updated version of the patch, thanks to Andreas Goesele for
>> reporting and testing.
>
> Thanks, pushed to the master branch.

In hindsight, this behavior should be off by default: it's going to
require user intervention too often to be practical as on-by-default.
This patch changes that, fixes a small bug, and also documents the
option.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-Allow-use-of-Gnus-search-groups-as-notmuch-path-.patch --]
[-- Type: text/x-patch, Size: 3557 bytes --]

From 7c26e8db58f64446f60c6f255a1a614deba198a6 Mon Sep 17 00:00:00 2001
From: Eric Abrahamsen <eric@ericabrahamsen.net>
Date: Mon, 19 Nov 2018 10:03:16 -0800
Subject: [PATCH] Fix "Allow use of Gnus search groups as notmuch path: search
 term"

* lisp/gnus/nnir.el (nnir-notmuch-filter-group-names-function):
  Default to nil -- getting correct behavior requires user
  intervention too often to have this enabled by default.
* lisp/gnus/nnir.el (nnir-run-notmuch): If the user has turned this
  on, then also hardcode `gnus-group-short-name' as a filter -- things
  will never work without it. Also move leading space to before the
  opening parenthesis.
* doc/misc/gnus.texi: Document option.

(Bug#33122)
---
 doc/misc/gnus.texi | 12 ++++++++++++
 lisp/gnus/nnir.el  | 27 +++++++++++++--------------
 2 files changed, 25 insertions(+), 14 deletions(-)

diff --git a/doc/misc/gnus.texi b/doc/misc/gnus.texi
index fb9113f460..d1c746c2e5 100644
--- a/doc/misc/gnus.texi
+++ b/doc/misc/gnus.texi
@@ -21468,6 +21468,18 @@ The notmuch Engine
 to get a group name (albeit with @samp{/} instead of @samp{.}).  This
 is a regular expression.
 
+@item nnir-notmuch-filter-group-names-function
+A function used to transform the names of groups being searched in,
+for use as a ``path:'' search keyword for notmuch.  If nil, the
+default, ``path:'' keywords are not used.  Otherwise, this should be a
+callable which accepts a single group name and returns a transformed
+name as notmuch expects to see it.  In many mail backends, for
+instance, dots in group names must be converted to forward slashes: to
+achieve this, set this option to
+@example
+(lambda (g) (replace-regexp-in-string "\\." "/" g))
+@end example
+
 @end table
 
 
diff --git a/lisp/gnus/nnir.el b/lisp/gnus/nnir.el
index ea7257d0c9..084b154e8a 100644
--- a/lisp/gnus/nnir.el
+++ b/lisp/gnus/nnir.el
@@ -518,18 +518,16 @@ nnir-notmuch-remove-prefix
   :type '(regexp)
   :group 'nnir)
 
-(defcustom nnir-notmuch-filter-group-names-function
-  #'gnus-group-short-name
+(defcustom nnir-notmuch-filter-group-names-function nil
   "Whether and how to use Gnus group names as \"path:\" search terms.
 When nil, the groups being searched in are not used as notmuch
 :path search terms.  It's still possible to use \"path:\" terms
 manually within the search query, however.
 
-When a function, map this function over all the group names.  By
-default this runs them through `gnus-group-short-name', and it is
-recommended to use this transform, at least.  Further
-transforms (for instance, converting \".\" to \"/\") can be
-added like so:
+When a function, map this function over all the group names.  To
+use the group names unchanged, set to (lambda (g) g).  Multiple
+transforms (for instance, converting \".\" to \"/\") can be added
+like so:
 
 \(add-function :filter-return
    nnir-notmuch-filter-group-names-function
@@ -1541,14 +1539,15 @@ nnir-run-notmuch
 				":[0-9]+"
 			      "^[0-9]+$"))
 	   (groups (when nnir-notmuch-filter-group-names-function
-		     (mapcar nnir-notmuch-filter-group-names-function
-			     groups)))
+		     (delq nil
+			   (mapcar nnir-notmuch-filter-group-names-function
+				   (mapcar #'gnus-group-short-name groups)))))
 	   (pathquery (when groups
-			(concat "("
-			 (mapconcat (lambda (g)
-				      (format " path:%s" g))
-				    groups " or")
-			 ")")))
+			(concat " ("
+				(mapconcat (lambda (g)
+					     (format "path:%s" g))
+					   groups " or")
+				")")))
            artno dirnam filenam)
 
       (when (equal "" qstring)
-- 
2.19.1