bug#74905: [PATCH] Implement search for nnvirtual Gnus groups

bug#74905: [PATCH] Implement search for nnvirtual Gnus groups

[Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors]

[-- Attachment #1: Type: text/plain, Size: 3464 bytes --]

Tags: patch


This patch implements search for nnvirtual. I'm using publi-inbox's
with nnvirtual to group each group into one.
However searching wasn't possible in these nnvirtual groups.
I implemented gnus-search-run-search based on the existing
nnselect gnus-search-run-search function.

I'm looking for feedback on the patch. I don't exactly know how
the search function is called when multiple groups of the same type are
involved. For nnvirtual each group is its on server, does that mean the
function will be always called only for each group? In that case
everything should be good.



In GNU Emacs 31.0.50 (build 1, x86_64-suse-linux-gnu, GTK+ Version
3.24.43, cairo version 1.18.2)
Repository revision: b25fba0ef8f38f95a9e65548f9ebb6c92739857e
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101014
System Description: openSUSE Tumbleweed

Configured using:
 'configure --disable-build-details --without-pop --with-mailutils
 --without-hesiod --with-gameuser=:games --with-kerberos
 --with-kerberos5 --with-file-notification=inotify --with-modules
 --enable-autodepend --enable-link-time-optimization --prefix=/usr
 --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share
 --localstatedir=/var --sharedstatedir=/var/lib
 --libexecdir=/usr/libexec --with-file-notification=yes
 --libdir=/usr/lib64 --with-native-compilation=aot
 --enable-locallisppath=/usr/share/emacs/31.0.50/site-lisp:/usr/share/emacs/site-lisp
 --with-x --with-xim --with-sound --with-xpm --with-jpeg --with-tiff
 --with-gif --with-png --with-rsvg --with-dbus --with-xft --without-gpm
 --with-tree-sitter --with-x-toolkit=gtk --without-pgtk
 --with-toolkit-scroll-bars --x-includes=/usr/include
 --x-libraries=/usr/lib64 --with-libotf --with-m17n-flt --with-cairo
 --build=x86_64-suse-linux --with-dumping=pdumper
 build_alias=x86_64-suse-linux 'CC=sccache cc' 'CFLAGS=-O2 -Wall
 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong
 -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection
 -Werror=return-type -flto=auto -march=znver3 -mmmx -mpopcnt -msse
 -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 -mavx -mavx2 -msse4a -mno-fma4
 -mno-xop -mfma -mbmi -mbmi2 -maes -mpclmul -mno-gfni -mvpclmulqdq
 -mno-3dnow -madx -mabm -mno-cldemote -mclflushopt -mclwb -mclzero
 -mcx16 -mno-enqcmd -mf16c -mfsgsbase -mfxsr -mno-hle -msahf -mno-lwp
 -mlzcnt -mmovbe -mno-movdir64b -mno-movdiri -mmwaitx -mno-pconfig -mpku
 -mprfchw -mno-ptwrite -mrdpid -mrdrnd -mrdseed -mno-rtm -mno-serialize
 -mno-sgx -msha -mshstk -mno-tbm -mno-tsxldtrk -mvaes -mno-waitpkg
 -mwbnoinvd -mxsave -mxsavec -mxsaveopt -mxsaves -mno-amx-tile
 -mno-amx-int8 -mno-amx-bf16 -mno-uintr -mno-hreset -mno-kl -mno-widekl
 -mno-avxvnni -mno-avxifma -mno-avxvnniint8 -mno-avxneconvert
 -mno-cmpccxadd -mno-amx-fp16 -mno-prefetchi -mno-raoint
 -mno-amx-complex --param l1-cache-size=32 --param l1-cache-line-size=64
 --param l2-cache-size=512 -mtune=znver3 -fno-optimize-sibling-calls -O2
 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong
 -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection
 -Werror=return-type -flto=auto -g -D_GNU_SOURCE
 -DGDK_DISABLE_DEPRECATION_WARNINGS -DGLIB_DISABLE_DEPRECATION_WARNINGS
 -pipe -Wno-pointer-sign -Wno-unused-variable -Wno-unused-label
 -DPDMP_BASE='\''"emacs-gtk"'\''' LDFLAGS=-Wl,-O2 'CXX=sccache c++'
 PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Implement-search-for-nnvirtual-Gnus-groups.patch --]
[-- Type: text/patch, Size: 4159 bytes --]

From 4b24c759e19aa2998a150f424e8187b052d7d5e3 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Bj=C3=B6rn=20Bidar?= <bjorn.bidar@thaodan.de>
Date: Mon, 16 Dec 2024 01:28:16 +0200
Subject: [PATCH] Implement search for nnvirtual Gnus groups

* lisp/gnus/nnvirtual.el (nnvirtual-request-list): Implement
request list backend function for nnvirtual.
* lisp/gnus/gnus-search.el (gnus-search-nnvirtual): Implement
function to build search queries for nnvirtual.
(gnus-search-default-search-engines, gnus-search-server-to-enngine):
Include nnvirtual.
---
 lisp/gnus/gnus-search.el | 29 ++++++++++++++++++++++++++---
 lisp/gnus/nnvirtual.el   |  7 +++++++
 2 files changed, 33 insertions(+), 3 deletions(-)

diff --git a/lisp/gnus/gnus-search.el b/lisp/gnus/gnus-search.el
index ca82546ef82..4a7cf902b32 100644
--- a/lisp/gnus/gnus-search.el
+++ b/lisp/gnus/gnus-search.el
@@ -795,6 +795,9 @@ shared-initialize
 (defclass gnus-search-nnselect (gnus-search-engine)
   nil)
 
+(defclass gnus-search-nnvirtual (gnus-search-engine)
+  nil)
+
 (defclass gnus-search-imap (gnus-search-engine)
   ((literal-plus
     :initarg :literal-plus
@@ -953,14 +956,16 @@ 'nnir-method-default-engines
   'gnus-search-default-engines "28.1")
 
 (defcustom gnus-search-default-engines '((nnimap . gnus-search-imap)
-                                         (nnselect . gnus-search-nnselect))
+                                         (nnselect . gnus-search-nnselect)
+                                         (nnvirtual . gnus-search-nnvirtual))
   "Alist of default search engines keyed by server method."
-  :version "26.1"
+  :version "31.1"
   :type `(repeat (cons (choice (const nnimap) (const nntp) (const nnspool)
 			       (const nneething) (const nndir) (const nnmbox)
 			       (const nnml) (const nnmh) (const nndraft)
 			       (const nnfolder) (const nnmaildir)
-                               (const nnselect))
+                               (const nnselect)
+                               (const nnvirtual))
 		       (choice
 			,@(mapcar
 			   (lambda (el) (list 'const (intern (car el))))
@@ -1086,6 +1091,23 @@ gnus-search-run-search
                                (cons 'search-group-spec group-spec))))))))
     artlist))
 
+;; nnvirtual interface
+(cl-defmethod gnus-search-run-search ((_engine gnus-search-nnvirtual)
+                               srv query-spec _groups)
+  ;; fixme groups vs srv which of?
+  ;; in theory the function could get multiple groups of one server
+  ;; but for nnvirtual server = group
+  (save-excursion
+    (let* (;; Not really a server but the matching groups for the nnvirtual group
+          (grouplist (mapcar #'gnus-group-short-name (gnus-search-get-active srv)))
+          ;; group each group as list containing each server and their groups
+          (group-spec (nnselect-categorize grouplist (lambda (x) (gnus-group-server x))))
+          (artlist []))
+      (setq artlist (vconcat artlist
+                             (gnus-search-run-query
+                              (list (cons 'search-query-spec query-spec)
+                                    (cons 'search-group-spec group-spec)))))
+      artlist)))
 
 ;; imap interface
 (cl-defmethod gnus-search-run-search ((engine gnus-search-imap)
@@ -2153,6 +2175,7 @@ gnus-search-server-to-engine
 	    ('namazu 'gnus-search-namazu)
 	    ('find-grep 'gnus-search-find-grep)
 	    ('imap 'gnus-search-imap)
+            ('nnvirtual 'gnus-search-nnvirtual)
 	    (_ server))
 	  inst
 	  (cond
diff --git a/lisp/gnus/nnvirtual.el b/lisp/gnus/nnvirtual.el
index 79747dca31f..edc97a8ca63 100644
--- a/lisp/gnus/nnvirtual.el
+++ b/lisp/gnus/nnvirtual.el
@@ -293,6 +293,13 @@ nnvirtual-close-group
     (nnvirtual-update-read-and-marked t t))
   t)
 
+(deffoo nnvirtual-request-list (&optional server)
+  (when (nnvirtual-possibly-change-server server)
+    (with-current-buffer nntp-server-buffer
+      (erase-buffer)
+      (dolist (group nnvirtual-component-groups)
+        (insert (format "%S 0  1 y\n" group))))
+    t))
 
 (deffoo nnvirtual-request-newgroups (_date &optional _server)
   (nnheader-report 'nnvirtual "NEWGROUPS is not supported."))
-- 
2.45.2

bug#74905: [PATCH] Implement search for nnvirtual Gnus groups

[James Thomas via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Björn Bidar wrote:

> This patch implements search for nnvirtual. I'm using publi-inbox's
> with nnvirtual to group each group into one.
> However searching wasn't possible in these nnvirtual groups.
> I implemented gnus-search-run-search based on the existing
> nnselect gnus-search-run-search function.

Thanks! I haven't looked into it, but here are some quick comments:

> I'm looking for feedback on the patch. I don't exactly know how
> the search function is called when multiple groups of the same type
> are
> involved. For nnvirtual each group is its on server, does that mean
> the
> function will be always called only for each group? In that case
> everything should be good.

That seems to be the case: see gnus-group-read-ephemeral-search-group
and gnus-group-make-search-group.

> +(deffoo nnvirtual-request-list (&optional server)
> +  (when (nnvirtual-possibly-change-server server)
> +    (with-current-buffer nntp-server-buffer
> +      (erase-buffer)
> +      (dolist (group nnvirtual-component-groups)
> +        (insert (format "%S 0  1 y\n" group))))
> +    t))

Did you check if gnus-start.el#L1801 withstands this? It seems to me to
assume that nnvirtual doesn't have -request-list.

--

bug#74905: [PATCH] Implement search for nnvirtual Gnus groups

[Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors]

James Thomas <jimjoe@gmx.net> writes:

> Björn Bidar wrote:
>
>> This patch implements search for nnvirtual. I'm using publi-inbox's
>> with nnvirtual to group each group into one.
>> However searching wasn't possible in these nnvirtual groups.
>> I implemented gnus-search-run-search based on the existing
>> nnselect gnus-search-run-search function.
>
> Thanks! I haven't looked into it, but here are some quick comments:
>
>> I'm looking for feedback on the patch. I don't exactly know how
>> the search function is called when multiple groups of the same type
>> are
>> involved. For nnvirtual each group is its on server, does that mean
>> the
>> function will be always called only for each group? In that case
>> everything should be good.
>
> That seems to be the case: see gnus-group-read-ephemeral-search-group
> and gnus-group-make-search-group.

OK good than my understanding from my tests matched with the rest of the
code.
Thanks for these examples I haven't looked at the create group functions
as the searched methods don't have to create groups even when they start
a new search by another backend just like e.g. if the user would call a
search on another imap group.

>> +(deffoo nnvirtual-request-list (&optional server)
>> +  (when (nnvirtual-possibly-change-server server)
>> +    (with-current-buffer nntp-server-buffer
>> +      (erase-buffer)
>> +      (dolist (group nnvirtual-component-groups)
>> +        (insert (format "%S 0  1 y\n" group))))
>> +    t))
>
> Did you check if gnus-start.el#L1801 withstands this? It seems to me to
> assume that nnvirtual doesn't have -request-list.

It does. If the user has falsely add nnvirtual to one of the select
methods than it will call it try to call the function which doesn't fail
or do anything. The only thing that happens from that is it will show
the false results as groups contained in the nnvirtual method without a
parameter.
Should verify that nnvirtual has arguments? So far it is possible
to add nnvirtual to select methods with "" but this is invalid.

If nnvirtual isn't added to select methods nothing happens besides
the regular activation.

bug#74905: [PATCH] Implement search for nnvirtual Gnus groups

[James Thomas via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Björn Bidar wrote:

> James Thomas <jimjoe@gmx.net> writes:
>
>> Björn Bidar wrote:
>>
>>> This patch implements search for nnvirtual. I'm using publi-inbox's
>>> with nnvirtual to group each group into one.
>>> However searching wasn't possible in these nnvirtual groups.
>>> I implemented gnus-search-run-search based on the existing
>>> nnselect gnus-search-run-search function.
>>
>> Thanks! I haven't looked into it, but here are some quick comments:
>>
>>> I'm looking for feedback on the patch. I don't exactly know how
>>> the search function is called when multiple groups of the same type
>>> are
>>> involved. For nnvirtual each group is its on server, does that mean
>>> the
>>> function will be always called only for each group? In that case
>>> everything should be good.
>>
>> That seems to be the case: see
>> gnus-group-read-ephemeral-search-group
>> and gnus-group-make-search-group.
>
> OK good than my understanding from my tests matched with the rest of
> the
> code.
> Thanks for these examples I haven't looked at the create group
> functions
> as the searched methods don't have to create groups even when they
> start
> a new search by another backend just like e.g. if the user would call
> a
> search on another imap group.
>
>>> +(deffoo nnvirtual-request-list (&optional server)
>>> +  (when (nnvirtual-possibly-change-server server)
>>> +    (with-current-buffer nntp-server-buffer
>>> +      (erase-buffer)
>>> +      (dolist (group nnvirtual-component-groups)
>>> +        (insert (format "%S 0  1 y\n" group))))
>>> +    t))
>>
>> Did you check if gnus-start.el#L1801 withstands this? It seems to me
>> to
>> assume that nnvirtual doesn't have -request-list.
>
> It does. If the user has falsely add nnvirtual to one of the select
> methods than it will call it try to call the function which doesn't
> fail
> or do anything. The only thing that happens from that is it will show
> the false results as groups contained in the nnvirtual method without
> a
> parameter.

No, I mean, the point of that code seems to be that nnvirtual is
activated _last_, i.e. after any component groups of other backends. And
it's not just select methods: even Foreign Groups are included, no?

> Should verify that nnvirtual has arguments? So far it is possible
> to add nnvirtual to select methods with "" but this is invalid.

I'm not sure what you mean, but it does have arguments: the components
regexp, for one.

> If nnvirtual isn't added to select methods nothing happens besides
> the regular activation.

Couldn't someone have added one? Say, with the above argument?

(Btw I'm only helping out, y'know. Getting this in is up to others with
commit rights)

--

bug#74905: [PATCH] Implement search for nnvirtual Gnus groups

[James Thomas via Bug reports for GNU Emacs, the Swiss army knife of text editors]

James Thomas wrote:

> Björn Bidar wrote:
>
>> James Thomas <jimjoe@gmx.net> writes:
>>
>>> Björn Bidar wrote:
>>>
>>>> This patch implements search for nnvirtual. I'm using
>>>> publi-inbox's
>>>> with nnvirtual to group each group into one.
>>>> However searching wasn't possible in these nnvirtual groups.
>>>> I implemented gnus-search-run-search based on the existing
>>>> nnselect gnus-search-run-search function.
>>>
>>> Thanks! I haven't looked into it, but here are some quick comments:
>>>
>>>> I'm looking for feedback on the patch. I don't exactly know how
>>>> the search function is called when multiple groups of the same
>>>> type
>>>> are
>>>> involved. For nnvirtual each group is its on server, does that
>>>> mean
>>>> the
>>>> function will be always called only for each group? In that case
>>>> everything should be good.
>>>
>>> That seems to be the case: see
>>> gnus-group-read-ephemeral-search-group
>>> and gnus-group-make-search-group.
>>
>> OK good than my understanding from my tests matched with the rest of
>> the
>> code.
>> Thanks for these examples I haven't looked at the create group
>> functions
>> as the searched methods don't have to create groups even when they
>> start
>> a new search by another backend just like e.g. if the user would
>> call
>> a
>> search on another imap group.
>>
>>>> +(deffoo nnvirtual-request-list (&optional server)
>>>> +  (when (nnvirtual-possibly-change-server server)
>>>> +    (with-current-buffer nntp-server-buffer
>>>> +      (erase-buffer)
>>>> +      (dolist (group nnvirtual-component-groups)
>>>> +        (insert (format "%S 0  1 y\n" group))))
>>>> +    t))
>>>
>>> Did you check if gnus-start.el#L1801 withstands this? It seems to
>>> me
>>> to
>>> assume that nnvirtual doesn't have -request-list.
>>
>> It does. If the user has falsely add nnvirtual to one of the select
>> methods than it will call it try to call the function which doesn't
>> fail
>> or do anything. The only thing that happens from that is it will
>> show
>> the false results as groups contained in the nnvirtual method
>> without
>> a
>> parameter.
>
> No, I mean, the point of that code seems to be that nnvirtual is
> activated _last_, i.e. after any component groups of other backends.

No, sorry, I was a little confused about this. But it seems to me that
-request-group removes each group from the component list (unlike your
-request-list). Is that relevant? IDK. The latter seems to do info
updation as well, based on -always-rescan.

--

bug#74905: [PATCH] Implement search for nnvirtual Gnus groups

[Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors]

James Thomas <jimjoe@gmx.net> writes:

> James Thomas wrote:
>
>> Björn Bidar wrote:
>>
>>> James Thomas <jimjoe@gmx.net> writes:
>>>
>>>> Björn Bidar wrote:
>>>>
>>>>> This patch implements search for nnvirtual. I'm using
>>>>> publi-inbox's
>>>>> with nnvirtual to group each group into one.
>>>>> However searching wasn't possible in these nnvirtual groups.
>>>>> I implemented gnus-search-run-search based on the existing
>>>>> nnselect gnus-search-run-search function.
>>>>
>>>> Thanks! I haven't looked into it, but here are some quick comments:
>>>>
>>>>> I'm looking for feedback on the patch. I don't exactly know how
>>>>> the search function is called when multiple groups of the same
>>>>> type
>>>>> are
>>>>> involved. For nnvirtual each group is its on server, does that
>>>>> mean
>>>>> the
>>>>> function will be always called only for each group? In that case
>>>>> everything should be good.
>>>>
>>>> That seems to be the case: see
>>>> gnus-group-read-ephemeral-search-group
>>>> and gnus-group-make-search-group.
>>>
>>> OK good than my understanding from my tests matched with the rest of
>>> the
>>> code.
>>> Thanks for these examples I haven't looked at the create group
>>> functions
>>> as the searched methods don't have to create groups even when they
>>> start
>>> a new search by another backend just like e.g. if the user would
>>> call
>>> a
>>> search on another imap group.
>>>
>>>>> +(deffoo nnvirtual-request-list (&optional server)
>>>>> +  (when (nnvirtual-possibly-change-server server)
>>>>> +    (with-current-buffer nntp-server-buffer
>>>>> +      (erase-buffer)
>>>>> +      (dolist (group nnvirtual-component-groups)
>>>>> +        (insert (format "%S 0  1 y\n" group))))
>>>>> +    t))
>>>>
>>>> Did you check if gnus-start.el#L1801 withstands this? It seems to
>>>> me
>>>> to
>>>> assume that nnvirtual doesn't have -request-list.
>>>
>>> It does. If the user has falsely add nnvirtual to one of the select
>>> methods than it will call it try to call the function which doesn't
>>> fail
>>> or do anything. The only thing that happens from that is it will
>>> show
>>> the false results as groups contained in the nnvirtual method
>>> without
>>> a
>>> parameter.
>>
>> No, I mean, the point of that code seems to be that nnvirtual is
>> activated _last_, i.e. after any component groups of other backends.
>
> No, sorry, I was a little confused about this. But it seems to me that
> -request-group removes each group from the component list (unlike your
> -request-list). Is that relevant? IDK. The latter seems to do info
> updation as well, based on -always-rescan.
>

Request list supposed to list the available groups on the server it is
also for example used in gnus-search-imap class gnus-search-run-search
function. Since for the case of nnvirtual all groups on the server
are all groups contained in the group it is the right function.

I took all the explenation for each functions purpose from the manual
section linked below:
(info "(gnus) Required Back End Functions")

To me the code just reads like that that for nnvirtual
the groups are activated this way only because
it doesn't have request list.

>
> No, I mean, the point of that code seems to be that nnvirtual is
> activated _last_, i.e. after any component groups of other backends. And
> it's not just select methods: even Foreign Groups are included, no?
>
>> Should verify that nnvirtual has arguments? So far it is possible
>> to add nnvirtual to select methods with "" but this is invalid.
>
> I'm not sure what you mean, but it does have arguments: the components
> regexp, for one.
>

It does have arguments exactly but you can add nnvirtual without
arguments or technically an empty argument to select methods.

>> If nnvirtual isn't added to select methods nothing happens besides
>> the regular activation.
>
> Couldn't someone have added one? Say, with the above argument?

I guess so but that isn't the intended way of using nnvirtual,
also it wouldn't take care of the request of the setup such
as creating the associated group.
Regular activation meant in this context that the request-list function
is called but without a argument the nnvirtual-server doesn't contain
any groups.