bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[dick.r.chiang]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: patch --]
[-- Type: text/x-diff, Size: 27867 bytes --]

From 834327d458c54bb0e1f25c6259ee640df0ba8b0e Mon Sep 17 00:00:00 2001
From: dickmao <none>
Date: Fri, 8 Nov 2019 09:51:59 -0500
Subject: [PATCH] Make `gnus-group-get-new-news` a non-blocking thread

* lisp/gnus/gnus-demon.el (gnus-demon-scan-news):
Add threaded optional argument.
* lisp/gnus/gnus-group.el (gnus-group-get-new-news):
Add threaded optional argument.
(gnus-threaded-get-unread-articles): This defcustom activates threading.
It defaults to nil.
(gnus-1): Add threaded optional argument.
(gnus-instantiate-server-buffer):
Make a new nntp-server-buffer for each thread.
(gnus-get-unread-articles-pass-preceding):
Tack preceding return value to ARGS before applying F.
(gnus-thread-body):
Let-close gnus global variables, create private nntp-server-buffer,
run the threaded function, and kill the nntp-server-buffer.
(gnus-run-thread): Make the thread.  Populate with serially dependent
sequence of functions.
(gnus-mutex-get-unread-articles):
Getting unread articles is a criticial section.
(gnus-get-unread-articles):
Reorder for threading.
(gnus-read-active-for-groups): Reprosecute tabs versus spaces.
(gnus-read-active-file-1): Elide a logical redundancy.
* lisp/gnus/gnus-sum.el (gnus-summary-display-article):
Replace if-null with when.
* lisp/gnus/gnus-util.el (gnus-push-end):
Define a convenience macro.
* lisp/gnus/nnheader.el
(nnheader-init-server-buffer, nnheader-prep-server-buffer):
Refactor "setting the table" in `nnheader-init-server-buffer`.
* lisp/gnus/nnimap.el (nnimap-make-process-buffer):
Apply due diligence if user kills nnimap process buffer.
* lisp/gnus/nntp.el (nntp-open-connection):
Apply due diligence if user kills nntp process buffer.
* lisp/mh-e/mh-compat.el (defun):
Reword an ancient and very confusing sentence.
* src/fns.c (Frequire):
Reword an ancient and very confusing sentence.
---
 etc/gnus/news-server.ast |   2 +-
 lisp/gnus/gnus-demon.el  |   3 +-
 lisp/gnus/gnus-group.el  |  14 +-
 lisp/gnus/gnus-start.el  | 289 ++++++++++++++++++++++++++++-----------
 lisp/gnus/gnus-sum.el    |   3 +-
 lisp/gnus/gnus-util.el   |   3 +
 lisp/gnus/nnheader.el    |  15 +-
 lisp/gnus/nnimap.el      |  13 ++
 lisp/gnus/nntp.el        |  13 ++
 lisp/mh-e/mh-compat.el   |   3 +-
 src/fns.c                |   3 +-
 11 files changed, 257 insertions(+), 104 deletions(-)

diff --git a/etc/gnus/news-server.ast b/etc/gnus/news-server.ast
index df0bab4519..555ac47cd9 100644
--- a/etc/gnus/news-server.ast
+++ b/etc/gnus/news-server.ast
@@ -20,7 +20,7 @@ Port number: @variable{port}
 
 @node User name and password
 @type interstitial
-@next 
+@next
 (if (assistant-password-required-p)
     "Enter user name and password"
   "Want user name and password?")
diff --git a/lisp/gnus/gnus-demon.el b/lisp/gnus/gnus-demon.el
index 7ec471afc7..b4b9b62a4f 100644
--- a/lisp/gnus/gnus-demon.el
+++ b/lisp/gnus/gnus-demon.el
@@ -252,7 +252,8 @@ gnus-demon-scan-news
 	(save-window-excursion
 	  (when (gnus-alive-p)
 	    (with-current-buffer gnus-group-buffer
-	      (gnus-group-get-new-news))))
+	      (gnus-group-get-new-news nil nil
+                                       gnus-threaded-get-unread-articles))))
       (set-window-configuration win))))
 
 (defun gnus-demon-add-scan-timestamps ()
diff --git a/lisp/gnus/gnus-group.el b/lisp/gnus/gnus-group.el
index 742f8f4be5..19090c68ff 100644
--- a/lisp/gnus/gnus-group.el
+++ b/lisp/gnus/gnus-group.el
@@ -4014,13 +4014,15 @@ gnus-activate-all-groups
 	(gnus-activate-foreign-newsgroups level))
     (gnus-group-get-new-news)))
 
-(defun gnus-group-get-new-news (&optional arg one-level)
+(defun gnus-group-get-new-news (&optional arg one-level background)
   "Get newly arrived articles.
 If ARG is a number, it specifies which levels you are interested in
 re-scanning.  If ARG is non-nil and not a number, this will force
 \"hard\" re-reading of the active files from all servers.
 If ONE-LEVEL is not nil, then re-scan only the specified level,
-otherwise all levels below ARG will be scanned too."
+otherwise all levels below ARG will be scanned too.
+If BACKGROUND then run `gnus-get-unread-articles' in a separate thread.
+"
   (interactive "P")
   (require 'nnmail)
   (let ((gnus-inhibit-demon t)
@@ -4034,17 +4036,13 @@ gnus-group-get-new-news
     (unless gnus-slave
       (gnus-master-read-slave-newsrc))
 
-    (gnus-get-unread-articles (gnus-group-default-level arg t)
-			      nil one-level)
+    (gnus-get-unread-articles arg nil one-level background)
 
     ;; If the user wants it, we scan for new groups.
     (when (eq gnus-check-new-newsgroups 'always)
       (gnus-find-new-newsgroups))
 
-    (gnus-check-reasonable-setup)
-    (gnus-run-hooks 'gnus-after-getting-new-news-hook)
-    (gnus-group-list-groups (and (numberp arg)
-				 (max (car gnus-group-list-mode) arg)))))
+    (gnus-check-reasonable-setup)))
 
 (defun gnus-group-get-new-news-this-group (&optional n dont-scan)
   "Check for newly arrived news in the current group (and the N-1 next groups).
diff --git a/lisp/gnus/gnus-start.el b/lisp/gnus/gnus-start.el
index e142c438ee..4553fa2d78 100644
--- a/lisp/gnus/gnus-start.el
+++ b/lisp/gnus/gnus-start.el
@@ -36,6 +36,7 @@
 (autoload 'gnus-agent-save-local "gnus-agent")
 (autoload 'gnus-agent-possibly-alter-active "gnus-agent")
 (declare-function gnus-group-decoded-name "gnus-group" (string))
+(declare-function gnus-group-default-level "gnus-group")
 
 (eval-when-compile (require 'cl-lib))
 
@@ -377,6 +378,17 @@ gnus-options-not-subscribe
   :type '(choice regexp
 		 (const :tag "none" nil)))
 
+(defcustom gnus-threaded-get-unread-articles nil
+  "Instantiate parallel threads for `gnus-get-unread-articles' which encapsulates
+most of the network retrieval when `gnus-group-get-new-news' is run."
+  :group 'gnus-start
+  :type 'boolean
+  :set (lambda (symbol value)
+         (set-default symbol value)
+         (when value (unless (featurep 'threads)
+                       (set-default symbol nil)
+                       (gnus-message 5 "Threads unsupported")))))
+
 (defcustom gnus-modtime-botch nil
   "Non-nil means .newsrc should be deleted prior to save.
 Its use is due to the bogus appearance that .newsrc was modified on
@@ -755,7 +767,8 @@ gnus-1
 	(gnus-group-get-new-news
 	 (and (numberp arg)
 	      (> arg 0)
-	      (max (car gnus-group-list-mode) arg))))
+	      (max (car gnus-group-list-mode) arg))
+         nil gnus-threaded-get-unread-articles))
 
     (gnus-clear-system)
     (gnus-splash)
@@ -1580,9 +1593,82 @@ gnus-get-unread-articles-in-group
 	(setcar (gnus-group-entry (gnus-info-group info)) num))
       num)))
 
-;; Go though `gnus-newsrc-alist' and compare with `gnus-active-hashtb'
-;; and compute how many unread articles there are in each group.
-(defun gnus-get-unread-articles (&optional level dont-connect one-level)
+(defun gnus-instantiate-server-buffer (name)
+  (let ((buffer (generate-new-buffer (format " *gnus-thread %s*" name))))
+    (nnheader-prep-server-buffer buffer)
+    buffer))
+
+(defmacro gnus-get-unread-articles-pass-preceding (f args)
+  "Tack preceding return value to ARGS before applying F."
+  `(apply ,f (nconc ,args (list (and (boundp 'gnus-run-thread--subresult)
+                                     gnus-run-thread--subresult)))))
+
+(defvar gnus-newsgroup-marked)
+(defvar gnus-newsgroup-spam-marked)
+(defvar gnus-article-current)
+(defvar gnus-current-score-file)
+(defvar gnus-newsgroup-charset)
+(defun gnus-thread-body (thread-name mtx working fns)
+  (with-mutex mtx
+    (nnheader-message 9 "gnus-thread-body: start %s" thread-name)
+    (let (gnus-run-thread--subresult
+          current-fn
+          (nntp-server-buffer working)
+          (gnus-newsgroup-name gnus-newsgroup-name)
+          (gnus-newsgroup-marked gnus-newsgroup-marked)
+          (gnus-newsgroup-spam-marked gnus-newsgroup-spam-marked)
+          (gnus-newsgroup-unreads gnus-newsgroup-unreads)
+          (gnus-current-headers gnus-current-headers)
+          (gnus-newsgroup-data gnus-newsgroup-data)
+          (gnus-summary-buffer gnus-summary-buffer)
+          (gnus-article-buffer gnus-article-buffer)
+          (gnus-original-article-buffer gnus-original-article-buffer)
+          (gnus-article-current gnus-article-current)
+          (gnus-reffed-article-number gnus-reffed-article-number)
+          (gnus-current-score-file gnus-current-score-file)
+          (gnus-newsgroup-charset gnus-newsgroup-charset))
+      (condition-case err
+          (dolist (fn fns)
+            (setq current-fn fn)
+            (setq gnus-run-thread--subresult (funcall fn)))
+        (error (nnheader-message
+                4 "gnus-thread-body: '%s' in %S"
+                (error-message-string err) current-fn))))
+    (kill-buffer working)
+    (nnheader-message 9 "gnus-thread-body: finish %s" thread-name)))
+
+(defun gnus-run-thread (mtx thread-group &rest fns)
+  "MTX, if non-nil, is the mutex for the new thread.
+THREAD-GROUP is string useful for naming working buffer and threads.
+All FNS must finish before MTX is released."
+  (when fns
+    (let ((thread-name
+           (concat thread-group "-"
+                   (let* ((max-len 160)
+                          (full-name (pp-to-string (car fns)))
+                          (short-name (cl-subseq
+                                       full-name 0
+                                       (min max-len
+                                            (length full-name)))))
+                     (if (> (length full-name) (length short-name))
+                         (concat short-name "...")
+                       short-name)))))
+      (make-thread (apply-partially
+                    #'gnus-thread-body
+                    thread-name mtx
+                    (gnus-instantiate-server-buffer thread-group)
+                    fns)
+                   thread-name))))
+
+(defvar gnus-mutex-get-unread-articles (make-mutex "gnus-mutex-get-unread-articles")
+  "Updating or displaying state of unread articles are critical sections.")
+
+(cl-defun gnus-get-unread-articles (&optional requested-level dont-connect
+                                              one-level background
+                                    &aux (level (gnus-group-default-level
+                                                 requested-level t)))
+  "Go through `gnus-newsrc-alist' and compare with `gnus-active-hashtb'
+  and compute how many unread articles there are in each group."
   (setq gnus-server-method-cache nil)
   (require 'gnus-agent)
   (let* ((newsrc (cdr gnus-newsrc-alist))
@@ -1636,14 +1722,14 @@ gnus-get-unread-articles
 		'primary)
 	       (t
 		'foreign)))
-	(push (setq method-group-list (list method method-type nil nil))
+	(push (setq method-group-list (list method method-type nil))
 	      type-cache))
       ;; Only add groups that need updating.
       (if (or (and foreign-level (null (numberp foreign-level)))
-	   (funcall (if one-level #'= #'<=) (gnus-info-level info)
-		    (if (eq (cadr method-group-list) 'foreign)
-			foreign-level
-		      alevel)))
+	      (funcall (if one-level #'= #'<=) (gnus-info-level info)
+		       (if (eq (cadr method-group-list) 'foreign)
+			   foreign-level
+		         alevel)))
 	  (setcar (nthcdr 2 method-group-list)
 		  (cons info (nth 2 method-group-list)))
 	;; The group is inactive, so we nix out the number of unread articles.
@@ -1664,9 +1750,9 @@ gnus-get-unread-articles
 		     (gnus-method-rank (cadr c2) (car c2))))))
     ;; Go through the list of servers and possibly extend methods that
     ;; aren't equal (and that need extension; i.e., they are async).
-    (let ((methods nil))
+    (let (methods)
       (dolist (elem type-cache)
-	(cl-destructuring-bind (method method-type infos dummy) elem
+	(cl-destructuring-bind (method method-type infos) elem
 	  (let ((gnus-opened-servers methods))
 	    (when (and (gnus-similar-server-opened method)
 		       (gnus-check-backend-function
@@ -1687,68 +1773,107 @@ gnus-get-unread-articles
 	  (with-current-buffer nntp-server-buffer
 	    (gnus-read-active-file-1 method nil)))))
 
-    ;; Clear out all the early methods.
-    (dolist (elem type-cache)
-      (cl-destructuring-bind (method method-type infos dummy) elem
-	(when (and method
-		   infos
-		   (gnus-check-backend-function
-		    'retrieve-group-data-early (car method))
-		   (not (gnus-method-denied-p method)))
-	  (when (ignore-errors (gnus-get-function method 'open-server))
-	    (unless (gnus-server-opened method)
-	      (gnus-open-server method))
-	    (when (gnus-server-opened method)
-	      ;; Just mark this server as "cleared".
-	      (gnus-retrieve-group-data-early method nil))))))
-
-    ;; Start early async retrieval of data.
-    (let ((done-methods nil)
-	  sanity-spec)
-      (dolist (elem type-cache)
-	(cl-destructuring-bind (method method-type infos dummy) elem
-	  (setq sanity-spec (list (car method) (cadr method)))
-	  (when (and method infos
-		     (not (gnus-method-denied-p method)))
-	    ;; If the open-server method doesn't exist, then the method
-	    ;; itself doesn't exist, so we ignore it.
-	    (if (not (ignore-errors (gnus-get-function method 'open-server)))
-		(setq type-cache (delq elem type-cache))
-	      (unless (gnus-server-opened method)
-		(gnus-open-server method))
-	      (when (and
-		     ;; This is a sanity check, so that we never
-		     ;; attempt to start two async requests to the
-		     ;; same server, because that will fail.  This
-		     ;; should never happen, since the methods should
-		     ;; be unique at this point, but apparently it
-		     ;; does happen in the wild with some setups.
-		     (not (member sanity-spec done-methods))
-		     (gnus-server-opened method)
-		     (gnus-check-backend-function
-		      'retrieve-group-data-early (car method)))
-		(push sanity-spec done-methods)
-		(when (gnus-check-backend-function 'request-scan (car method))
-		  (gnus-request-scan nil method))
-		;; Store the token we get back from -early so that we
-		;; can pass it to -finish later.
-		(setcar (nthcdr 3 elem)
-			(gnus-retrieve-group-data-early method infos))))))))
-
-    ;; Do the rest of the retrieval.
-    (dolist (elem type-cache)
-      (cl-destructuring-bind (method method-type infos early-data) elem
-	(when (and method infos
-		   (not (gnus-method-denied-p method)))
-	  (let ((updatep (gnus-check-backend-function
-			  'request-update-info (car method))))
-	    ;; See if any of the groups from this method require updating.
-	    (gnus-read-active-for-groups method infos early-data)
-	    (dolist (info infos)
-	      (inline (gnus-get-unread-articles-in-group
-		       info (gnus-active (gnus-info-group info))
-		       updatep)))))))
-    (gnus-message 6 "Checking new news...done")))
+    ;; Must be able to `gnus-open-server'
+    (setq type-cache (seq-filter
+                      (lambda (elem)
+                        (cl-destructuring-bind (method _type _infos) elem
+                          (ignore-errors (gnus-get-function method 'open-server))))
+                      type-cache))
+
+    (let (methods
+          (coda (apply-partially
+                 (lambda (level*)
+                   (nnheader-message 9 "gnus-get-unread-articles: all done")
+                   (gnus-group-list-groups level*)
+                   (gnus-run-hooks 'gnus-after-getting-new-news-hook)
+                   (gnus-group-list-groups))
+                 (and (numberp level)
+                      (max (car gnus-group-list-mode) level)))))
+      (mapc (lambda (elem)
+              (cl-destructuring-bind
+                (method _type infos
+                 &aux
+                 (backend (car method))
+                 (already-p
+                  (cl-some (apply-partially
+                            #'gnus-methods-equal-p method)
+                           methods))
+                 (denied-p (gnus-method-denied-p method))
+                 (scan-p (gnus-check-backend-function 'request-scan backend))
+                 (early-p (gnus-check-backend-function
+                           'retrieve-group-data-early backend))
+                 (update-p (gnus-check-backend-function
+                            'request-update-info backend))
+                 commands early-data)
+                  elem
+                (when (and method infos (not denied-p) (not already-p))
+                  (push method methods)
+                  (gnus-push-end (apply-partially
+                                  #'gnus-open-server method)
+                                 commands)
+                  (when early-p
+                    ;; Just mark this server as "cleared".
+                    (gnus-push-end (apply-partially
+                                    #'gnus-retrieve-group-data-early method nil)
+                                   commands)
+
+                    ;; This is a sanity check, so that we never
+                    ;; attempt to start two async requests to the
+                    ;; same server, because that will fail.  This
+                    ;; should never happen, since the methods should
+                    ;; be unique at this point, but apparently it
+                    ;; does happen in the wild with some setups.
+                    (when scan-p
+                      (gnus-push-end (apply-partially #'gnus-request-scan nil method)
+                                     commands))
+
+                    ;; Store the token we get back from -early so that we
+                    ;; can pass it to -finish later.
+                    (gnus-push-end (apply-partially
+                                    #'gnus-retrieve-group-data-early
+                                    method infos)
+                                   commands))
+                  (gnus-push-end (apply-partially
+                                  (lambda (f &rest args)
+                                    (gnus-get-unread-articles-pass-preceding f args))
+                                  #'gnus-read-active-for-groups method infos)
+                                 commands)
+                  (gnus-push-end (apply-partially
+                                  (lambda (infos* update-p*)
+                                    (mapc (lambda (info)
+                                            (gnus-get-unread-articles-in-group
+                                             info
+                                             (gnus-active (gnus-info-group info))
+                                             update-p*))
+                                          infos*)
+                                    (gnus-message 6 "Checking new news...done"))
+                                  infos update-p)
+                                 commands)
+                  (if background
+                      (let ((thread-group "gnus-unread-articles"))
+                        (add-function
+                         :before-while coda
+                         (apply-partially
+                          (lambda (thread-group* &rest _args)
+                            "Proceed with before-while if I'm the last one."
+                            (<= (cl-count thread-group*
+                                          (all-threads)
+                                          :test (lambda (s thr)
+                                                  (cl-search s (thread-name thr))))
+                                1))
+                          thread-group))
+                        (gnus-push-end coda commands)
+                        (apply #'gnus-run-thread
+                               gnus-mutex-get-unread-articles
+                               thread-group
+                               commands))
+                    (let (gnus-run-thread--subresult)
+                      (mapc (lambda (fn)
+                              (setq gnus-run-thread--subresult (funcall fn)))
+                            commands))))))
+            type-cache)
+      (unless background
+        (funcall coda)))))
 
 (defun gnus-method-rank (type method)
   (cond
@@ -1780,7 +1905,7 @@ gnus-read-active-for-groups
        early-data
        (gnus-check-backend-function 'finish-retrieve-group-infos (car method))
        (or (not (gnus-agent-method-p method))
-	   (gnus-online method)))
+           (gnus-online method)))
       (gnus-finish-retrieve-group-infos method infos early-data)
       ;; We may have altered the data now, so mark the dribble buffer
       ;; as dirty so that it gets saved.
@@ -1789,12 +1914,12 @@ gnus-read-active-for-groups
      ;; Most backends have -retrieve-groups.
      ((gnus-check-backend-function 'retrieve-groups (car method))
       (when (gnus-check-backend-function 'request-scan (car method))
-	(gnus-request-scan nil method))
+        (gnus-request-scan nil method))
       (let (groups)
-	(gnus-read-active-file-2
-	 (dolist (info infos (nreverse groups))
-	   (push (gnus-group-real-name (gnus-info-group info)) groups))
-	 method)))
+        (gnus-read-active-file-2
+         (dolist (info infos (nreverse groups))
+           (push (gnus-group-real-name (gnus-info-group info)) groups))
+         method)))
      ;; Virtually all backends have -request-list.
      ((gnus-check-backend-function 'request-list (car method))
       (gnus-read-active-file-1 method nil))
@@ -1802,7 +1927,7 @@ gnus-read-active-for-groups
      ;; by one.
      (t
       (dolist (info infos)
-	(gnus-activate-group (gnus-info-group info) nil nil method t))))))
+        (gnus-activate-group (gnus-info-group info) nil nil method t))))))
 
 (defun gnus-make-hashtable-from-newsrc-alist ()
   "Create a hash table from `gnus-newsrc-alist'.
@@ -2042,9 +2167,7 @@ gnus-read-active-file-1
     (gnus-message 5 "%s" mesg)
     (when (gnus-check-server method)
       ;; Request that the backend scan its incoming messages.
-      (when (and (or (and gnus-agent
-			  (gnus-online method))
-		     (not gnus-agent))
+      (when (and (or (not gnus-agent) (gnus-online method))
 		 (gnus-check-backend-function 'request-scan (car method)))
 	(gnus-request-scan nil method))
       (cond
diff --git a/lisp/gnus/gnus-sum.el b/lisp/gnus/gnus-sum.el
index f21bc7584e..6f12ae6c13 100644
--- a/lisp/gnus/gnus-sum.el
+++ b/lisp/gnus/gnus-sum.el
@@ -7764,8 +7764,7 @@ gnus-summary-display-article
     (setq gnus-article-charset gnus-newsgroup-charset)
     (setq gnus-article-ignored-charsets gnus-newsgroup-ignored-charsets)
     (mm-enable-multibyte))
-  (if (null article)
-      nil
+  (when article
     (prog1
 	(funcall (or gnus-summary-display-article-function
                      #'gnus-article-prepare)
diff --git a/lisp/gnus/gnus-util.el b/lisp/gnus/gnus-util.el
index 3cf364fff8..48b0739dd1 100644
--- a/lisp/gnus/gnus-util.el
+++ b/lisp/gnus/gnus-util.el
@@ -106,6 +106,9 @@ gnus-eval-in-buffer-window
 (put 'gnus-eval-in-buffer-window 'lisp-indent-function 1)
 (put 'gnus-eval-in-buffer-window 'edebug-form-spec '(form body))
 
+(defmacro gnus-push-end (elt place)
+  `(push ,elt (if (consp ,place) (cdr (last ,place)) ,place)))
+
 (defsubst gnus-goto-char (point)
   (and point (goto-char point)))
 
diff --git a/lisp/gnus/nnheader.el b/lisp/gnus/nnheader.el
index 28c4cebb2d..d5d76e80ea 100644
--- a/lisp/gnus/nnheader.el
+++ b/lisp/gnus/nnheader.el
@@ -502,11 +502,10 @@ nnheader-file-coding-system
   "Coding system used in file backends of Gnus.")
 (defvar nnheader-callback-function nil)
 
-(defun nnheader-init-server-buffer ()
-  "Initialize the Gnus-backend communication buffer."
-  (unless (gnus-buffer-live-p nntp-server-buffer)
-    (setq nntp-server-buffer (get-buffer-create " *nntpd*")))
-  (with-current-buffer nntp-server-buffer
+(defsubst nnheader-prep-server-buffer (buffer)
+  "Refactor \"setting the table\" of BUFFER for `nnheader-init-server-buffer' and
+`gnus-instantiate-server-buffer'."
+  (with-current-buffer buffer
     (erase-buffer)
     (mm-enable-multibyte)
     (kill-all-local-variables)
@@ -514,6 +513,12 @@ nnheader-init-server-buffer
     (set (make-local-variable 'nntp-process-response) nil)
     t))
 
+(defun nnheader-init-server-buffer ()
+  "Initialize the Gnus-backend communication buffer."
+  (unless (gnus-buffer-live-p nntp-server-buffer)
+    (setq nntp-server-buffer (get-buffer-create " *nntpd*")))
+  (nnheader-prep-server-buffer nntp-server-buffer))
+
 ;;; Various functions the backends use.
 
 (defun nnheader-file-error (file)
diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el
index 1ec5522831..64f7cb46d6 100644
--- a/lisp/gnus/nnimap.el
+++ b/lisp/gnus/nnimap.el
@@ -371,6 +371,19 @@ nnimap-make-process-buffer
 		      :initial-resync 0))
     (push (list buffer (current-buffer)) nnimap-connection-alist)
     (push (current-buffer) nnimap-process-buffers)
+    (with-current-buffer buffer
+      (add-hook 'kill-buffer-hook
+                (apply-partially
+                 (lambda (buffer)
+                   (when-let ((pbuffer
+                               (car (alist-get buffer nnimap-connection-alist))))
+                     (setq nnimap-process-buffers
+                           (delq pbuffer nnimap-process-buffers))
+                     (kill-buffer pbuffer) ;; should HUP its process
+                     (setq nnimap-connection-alist
+                           (assq-delete-all buffer nnimap-connection-alist))))
+                 buffer)
+                nil t))
     (current-buffer)))
 
 (defvar auth-source-creation-prompts)
diff --git a/lisp/gnus/nntp.el b/lisp/gnus/nntp.el
index 3ddd53e46c..044e032134 100644
--- a/lisp/gnus/nntp.el
+++ b/lisp/gnus/nntp.el
@@ -1301,6 +1301,19 @@ nntp-open-connection
 	  (prog1
 	      (caar (push (list process buffer nil) nntp-connection-alist))
 	    (push process nntp-connection-list)
+            (with-current-buffer buffer
+              (add-hook 'kill-buffer-hook
+                        (apply-partially
+                         (lambda (buffer)
+                           (when-let ((process
+                                       (car (nntp-find-connection-entry buffer))))
+                             (setq nntp-connection-list
+                                   (delq process nntp-connection-list))
+                             (setq nntp-connection-alist
+                                   (assq-delete-all process nntp-connection-alist))
+                             (ignore-errors (delete-process process))))
+                         buffer)
+                        nil t))
 	    (with-current-buffer pbuffer
 	      (nntp-read-server-type)
 	      (erase-buffer)
diff --git a/lisp/mh-e/mh-compat.el b/lisp/mh-e/mh-compat.el
index 7c5bd3a987..43669cc1af 100644
--- a/lisp/mh-e/mh-compat.el
+++ b/lisp/mh-e/mh-compat.el
@@ -47,8 +47,7 @@
 (mh-do-in-xemacs
   (defun mh-require (feature &optional filename noerror)
     "If feature FEATURE is not loaded, load it from FILENAME.
-If FEATURE is not a member of the list `features', then the feature
-is not loaded; so load the file FILENAME.
+Loaded features are recorded in the list variable `features'.
 If FILENAME is omitted, the printname of FEATURE is used as the file name.
 If the optional third argument NOERROR is non-nil,
 then return nil if the file is not found instead of signaling an error.
diff --git a/src/fns.c b/src/fns.c
index cbb6879223..7d4ed7cab6 100644
--- a/src/fns.c
+++ b/src/fns.c
@@ -2917,8 +2917,7 @@ require_unwind (Lisp_Object old_value)
 
 DEFUN ("require", Frequire, Srequire, 1, 3, 0,
        doc: /* If feature FEATURE is not loaded, load it from FILENAME.
-If FEATURE is not a member of the list `features', then the feature is
-not loaded; so load the file FILENAME.
+Loaded features are recorded in the list variable `features'.
 
 If FILENAME is omitted, the printname of FEATURE is used as the file
 name, and `load' will try to load this name appended with the suffix
-- 
2.23.0

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Eric Abrahamsen]

dick.r.chiang@gmail.com writes:

> From 834327d458c54bb0e1f25c6259ee640df0ba8b0e Mon Sep 17 00:00:00 2001
> From: dickmao <none>
> Date: Fri, 8 Nov 2019 09:51:59 -0500
> Subject: [PATCH] Make `gnus-group-get-new-news` a non-blocking thread

I'm looking forward to testing this out this weekend!

Just a quick NB, which you should feel free to ignore: you'll probably
get much better uptake on these tickets if you include with them a
head-to-toe prose report on what they accomplish and why (in detail).
Emacs has a lot of bug reports, and a small number of bug hunters, and a
multi-hundred-line patch without so much as a "howdy-do" on it is ripe
to be ignored. Just something to consider.

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Lars Ingebrigtsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> dick.r.chiang@gmail.com writes:
>
>> From 834327d458c54bb0e1f25c6259ee640df0ba8b0e Mon Sep 17 00:00:00 2001
>> From: dickmao <none>
>> Date: Fri, 8 Nov 2019 09:51:59 -0500
>> Subject: [PATCH] Make `gnus-group-get-new-news` a non-blocking thread
>
> I'm looking forward to testing this out this weekend!

I haven't tested it, but it sounds like a great feature.

> Just a quick NB, which you should feel free to ignore: you'll probably
> get much better uptake on these tickets if you include with them a
> head-to-toe prose report on what they accomplish and why (in detail).
> Emacs has a lot of bug reports, and a small number of bug hunters, and a
> multi-hundred-line patch without so much as a "howdy-do" on it is ripe
> to be ignored. Just something to consider.

No, please don't ignore.  :-)

It'd be helpful if you wrote something about how errors during `g' are
reported and stuff and how stable this is, because my impression of the
Emacs thread support is that it is (so far) got some stability
problems.  

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Eric Abrahamsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> dick.r.chiang@gmail.com writes:
>>
>>> From 834327d458c54bb0e1f25c6259ee640df0ba8b0e Mon Sep 17 00:00:00 2001
>>> From: dickmao <none>
>>> Date: Fri, 8 Nov 2019 09:51:59 -0500
>>> Subject: [PATCH] Make `gnus-group-get-new-news` a non-blocking thread
>>
>> I'm looking forward to testing this out this weekend!

I finally got to road-testing this, and so far haven't been able to
break it.

> It'd be helpful if you wrote something about how errors during `g' are
> reported and stuff and how stable this is, because my impression of the
> Emacs thread support is that it is (so far) got some stability
> problems.  

I haven't tried to make it raise an error, but I did set
`nntp-connection-timeout' very low, and try to trigger an error that
way. Nothing bad happened -- I will take a look at the code and see how
timeouts are handled.

So far so good!

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Lars Ingebrigtsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> I haven't tried to make it raise an error, but I did set
> `nntp-connection-timeout' very low, and try to trigger an error that
> way. Nothing bad happened -- I will take a look at the code and see how
> timeouts are handled.

Try putting an (error) in `nntp-open-server' or something.  :-)

> So far so good!

Sounds good.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Eric Abrahamsen]

On 11/18/19 09:54 AM, Lars Ingebrigtsen wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> I haven't tried to make it raise an error, but I did set
>> `nntp-connection-timeout' very low, and try to trigger an error that
>> way. Nothing bad happened -- I will take a look at the code and see how
>> timeouts are handled.
>
> Try putting an (error) in `nntp-open-server' or something.  :-)

Oh, sure. I stuck errors in `nntp-open-server' and
`nntp-finish-retrieve-group-infos', and worked that both with a
nntp-connection-timeout and without, and wasn't able to drive it
completely off the rails -- unfortunately I did get two odd errors which
I was later unable to reproduce: one about selecting a deleted buffer,
which I think came out of `nntp-open-server', and another that looked
like "gnus-thread-group: " and then a chunk of bytecode. I don't know
how I got those, though!

The only other error I'm seeing now is if you start Gnus at a specified
level (ie "M-4 M-x gnus"), it will error out in
`gnus-get-unread-articles', line 1791, because it's expecting "(car
gnus-group-list-mode)" to be a number, but it hasn't been set yet.

In general, it would be great to have more code comments, particularly
in `gnus-get-unread-articles'. I'm afraid that we're going to end up
with more code that only one person really understands, touching on an
area of elisp (threading) that we're all kind of expecting to be
unstable.

That's all I've got for now.

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[dick.r.chiang]

I am very grateful for your interest and testing!

> selecting a deleted buffer

I get this quite a bit.  It occurs when `gnus-summary-buffer` in the main
thread gets usurped by a background thread.  I am either allowing this
important variable to get reassigned before the dynamic-let in `gnus-thread-body`
or I am not understanding dynamic-let in the presence of threads.
Incidentally, it's very difficult to point to lines of code I'm talking
about without git{hub,lab}.

> line 1791, because it's expecting "(car gnus-group-list-mode)" to be a
> number, but it hasn't been set yet.

I'll look into this, and add a test for `nntp-open-server`.

I am happy to let this marinate to get people thinking about Gnus's future.  There are many historical artifacts (like blocking `gnus-group-get-new-news`
and left-field variables like `gnus-secondary-select-methods`) that prevent
Gnus from becoming a viable MUA for more people.

Some other fellow recently posted about an ephemeral group branch which he
somehow got others to test for him in-parallel.  I don't know how he did that
outside the debbugs system.

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Eric Abrahamsen]

On 11/18/19 17:22 PM, dick.r.chiang@gmail.com wrote:
> I am very grateful for your interest and testing!

My pleasure, of course -- I think this is an important direction to be
going in.

>> selecting a deleted buffer
>
> I get this quite a bit.  It occurs when `gnus-summary-buffer` in the main
> thread gets usurped by a background thread.  I am either allowing this
> important variable to get reassigned before the dynamic-let in `gnus-thread-body`
> or I am not understanding dynamic-let in the presence of threads.
> Incidentally, it's very difficult to point to lines of code I'm talking
> about without git{hub,lab}.

We could consider turning on `lexical-binding' in gnus-start.el, and
just see what happens :)

Regarding code line numbers, etc, I just do it manually -- I'm looking
at the code to begin with, and it's trivial to check what line I'm on.

>> line 1791, because it's expecting "(car gnus-group-list-mode)" to be a
>> number, but it hasn't been set yet.
>
> I'll look into this, and add a test for `nntp-open-server`.
>
> I am happy to let this marinate to get people thinking about Gnus's
> future. There are many historical artifacts (like blocking
> `gnus-group-get-new-news`
> and left-field variables like `gnus-secondary-select-methods`) that prevent
> Gnus from becoming a viable MUA for more people.

Well they definitely prevent bug-hunters and feature-implementers from
making much progress. Stuff like `gnus-group-list-mode' drives me nuts:
an undocumented variable that might totally change Gnus' behavior. But
unpicking this complexity is slow work.

> Some other fellow recently posted about an ephemeral group branch which he
> somehow got others to test for him in-parallel.  I don't know how he did that
> outside the debbugs system.

I make liberal use of git worktrees. I made a local branch to apply your
patch, then checked it out into a separate directory with "git worktree
add" and built Emacs there. Then I run that Emacs. For Gnus stuff I have
a package called gnus-mock that provides a working Gnus environment with
dummy data. I point gnus-mock at the worktree directory, and it starts
up a clean Emacs with the Gnus data in place. I do most of my testing
there.

On the Emacs side it's projectile and magit: projectile indexes the
worktree as a separate project. If you send me a line number from your
branch to look at, I do "C-c p p" -> helm-projectile-switch-project
which opens up magit for the project, then "C-c p f" ->
helm-projectile-find-file, then the usual "M-g g" -> goto-line. Not too
inconvenient.

If you have access to the Emacs repos and are collaborating with others
who do, too, then it can make sense to push a "feature/foo" branch to
the repos, and share it. I've done testing that way.

What I'd like is a Gnus summary minor mode I could enable in, say,
"emacs.bugs", which turns commit hashes into hyperlinks and provides
commands for "apply the attached patch to some worktree". I think other
people have done that, I've just never gotten around to it.

Eric

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Robert Pluim]

>>>>> On Mon, 18 Nov 2019 15:18:10 -0800, Eric Abrahamsen <eric@ericabrahamsen.net> said:

    Eric> What I'd like is a Gnus summary minor mode I could enable in, say,
    Eric> "emacs.bugs", which turns commit hashes into hyperlinks and provides
    Eric> commands for "apply the attached patch to some worktree". I think other
    Eric> people have done that, I've just never gotten around to it.

You mean like 'debbugs-gnu' and 'debbugs-gnu-bugs' from the debbugs-gnu
package? That has commands to do exactly that.

Robert

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Eric Abrahamsen]

On 11/19/19 11:18 AM, Robert Pluim wrote:
>>>>>> On Mon, 18 Nov 2019 15:18:10 -0800, Eric Abrahamsen <eric@ericabrahamsen.net> said:
>
>     Eric> What I'd like is a Gnus summary minor mode I could enable in, say,
>     Eric> "emacs.bugs", which turns commit hashes into hyperlinks and provides
>     Eric> commands for "apply the attached patch to some worktree". I think other
>     Eric> people have done that, I've just never gotten around to it.
>
> You mean like 'debbugs-gnu' and 'debbugs-gnu-bugs' from the debbugs-gnu
> package? That has commands to do exactly that.

Ha, I guess that's true, I've just never gotten accustomed to using
debbugs-gnu. I wish I could make a persistent debbugs view just showing
the few bugs I'm looking at at any given time (never very many), which
is easy to do in a Gnus group by ticking articles.

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Lars Ingebrigtsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Ha, I guess that's true, I've just never gotten accustomed to using
> debbugs-gnu. I wish I could make a persistent debbugs view just showing
> the few bugs I'm looking at at any given time (never very many), which
> is easy to do in a Gnus group by ticking articles.

That wouldn't be very difficult to add to debbugs-gnu, actually.  It
already has the concept of "marked" bugs, so there could be a command
like, say, `debbugs-gnu-show-marked' that just shows that list of bugs.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> That wouldn't be very difficult to add to debbugs-gnu, actually.  It
> already has the concept of "marked" bugs, so there could be a command
> like, say, `debbugs-gnu-show-marked' that just shows that list of bugs.

Patches welcome :-)

I'm not so fluid in writing gnus patches, so don't count on me.

Best regards, Michael.

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Robert Pluim]

>>>>> On Wed, 20 Nov 2019 12:55:45 +0100, Michael Albinus <michael.albinus@gmx.de> said:

    Michael> Lars Ingebrigtsen <larsi@gnus.org> writes:
    >> That wouldn't be very difficult to add to debbugs-gnu, actually.  It
    >> already has the concept of "marked" bugs, so there could be a command
    >> like, say, `debbugs-gnu-show-marked' that just shows that list of bugs.

    Michael> Patches welcome :-)

    Michael> I'm not so fluid in writing gnus patches, so don't count on me.

Doesnʼt this already exist?

    M-x debbugs-gnu
    Hit 't' on the bugs that youʼre interested in
    quit debbugs
    M-x debbugs-gnu
    Erase the default severities, say 'tagged', then hit RET at the next prompt
    Hey presto, debbugs-gnu with only the bugs you hit 't' on earlier.

If you want to add a bug to that list:

    M-x debbugs-gnu-bug
    type the bug number, RET
    Hit 't'

Now the next time you run debbugs-gnu as above that bug will appear as
well.

Robert

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Robert Pluim <rpluim@gmail.com> writes:

> Doesnʼt this already exist?
>
>     M-x debbugs-gnu
>     Hit 't' on the bugs that youʼre interested in
>     quit debbugs
>     M-x debbugs-gnu
>     Erase the default severities, say 'tagged', then hit RET at the next prompt
>     Hey presto, debbugs-gnu with only the bugs you hit 't' on earlier.
>
> If you want to add a bug to that list:
>
>     M-x debbugs-gnu-bug
>     type the bug number, RET
>     Hit 't'
>
> Now the next time you run debbugs-gnu as above that bug will appear as
> well.

Sure (and I use it every single day). But I understood the request as
"Pls give me one gnus summary buffer containing all messages of bugs I
have tagged locally". This doesn't exist yet.

> Robert

Best regards, Michael.

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Robert Pluim]

(I dropped the bug and added emacs-devel)

>>>>> On Wed, 20 Nov 2019 14:00:04 +0100, Michael Albinus <michael.albinus@gmx.de> said:

    >> Now the next time you run debbugs-gnu as above that bug will appear as
    >> well.

    Michael> Sure (and I use it every single day). But I understood the request as
    Michael> "Pls give me one gnus summary buffer containing all messages of bugs I
    Michael> have tagged locally". This doesn't exist yet.

Ah, I misunderstood. How's this?

(defun debbugs-gnus-ephemeral-from-tagged ()
  (interactive)
  (when (file-exists-p debbugs-gnu-persistency-file)
    (with-temp-buffer
      (insert-file-contents debbugs-gnu-persistency-file)
      (eval (read (current-buffer)) t))
    (when debbugs-gnu-local-tags
      (gnus-read-ephemeral-bug-group
       debbugs-gnu-local-tags
       (alist-get 'emacs gnus-bug-group-download-format-alist)))))

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Robert Pluim <rpluim@gmail.com> writes:

>     Michael> Sure (and I use it every single day). But I understood
>     Michael> the request as "Pls give me one gnus summary buffer
>     Michael> containing all messages of bugs I have tagged
>     Michael> locally". This doesn't exist yet.
>
> Ah, I misunderstood. How's this?
>
> (defun debbugs-gnus-ephemeral-from-tagged ()
>   (interactive)
>   (when (file-exists-p debbugs-gnu-persistency-file)
>     (with-temp-buffer
>       (insert-file-contents debbugs-gnu-persistency-file)
>       (eval (read (current-buffer)) t))
>     (when debbugs-gnu-local-tags
>       (gnus-read-ephemeral-bug-group
>        debbugs-gnu-local-tags
>        (alist-get 'emacs gnus-bug-group-download-format-alist)))))

Works for me. However, the "Status:" messages shall be kicked off. And
it shall be checked, whether `debbugs-gnu-mail-backend' is bound to `gnus'.

If Lars doesn't object, you might commit it, plus docstring, and plus
documentation in debbugs-ug.texi.

Best regards, Michael.

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Robert Pluim]

>>>>> On Wed, 20 Nov 2019 14:43:46 +0100, Michael Albinus <michael.albinus@gmx.de> said:

    Michael> Works for me. However, the "Status:" messages shall be kicked off. And
    Michael> it shall be checked, whether `debbugs-gnu-mail-backend' is bound to `gnus'.

The clue is in the name of the function, but OK.

No more feature requests :-)

    Michael> If Lars doesn't object, you might commit it, plus docstring, and plus
    Michael> documentation in debbugs-ug.texi.

I donʼt think Iʼm set up to push to elpa.

(defun debbugs-gnus-ephemeral-from-tagged ()
  "Create an ephemeral Gnus group of locally tagged emacs bugs."
  (interactive)
  (unless (eq debbugs-gnu-mail-backend 'gnus)
    (error "This function only works with Gnus."))
  (when (file-exists-p debbugs-gnu-persistency-file)
    (with-temp-buffer
      (insert-file-contents debbugs-gnu-persistency-file)
      (eval (read (current-buffer)) t))
    (when debbugs-gnu-local-tags
      (let ((mbox-url (replace-regexp-in-string
                       ";mboxstat=yes" ""
                       (alist-get 'emacs gnus-bug-group-download-format-alist)
                       nil t)))
      (gnus-read-ephemeral-bug-group
       debbugs-gnu-local-tags
       mbox-url)))))

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Robert Pluim <rpluim@gmail.com> writes:

> I donʼt think Iʼm set up to push to elpa.

I believe everybody with write access to emacs can also do so for
elpa, but I'm not sure.

Let's wait what Lars says, and then I could commit it in your name.

Best regards, Michael.

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Robert Pluim]

>>>>> On Wed, 20 Nov 2019 15:30:06 +0100, Michael Albinus <michael.albinus@gmx.de> said:

    Michael> Robert Pluim <rpluim@gmail.com> writes:
    >> I donʼt think Iʼm set up to push to elpa.

    Michael> I believe everybody with write access to emacs can also do so for
    Michael> elpa, but I'm not sure.

I meant "I donʼt have an elpa checkout lying around, and Iʼm too lazy
to read up on what it would take to set things up so I can commit" :-)

Robert

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Robert Pluim <rpluim@gmail.com> writes:

> I meant "I donʼt have an elpa checkout lying around, and Iʼm too lazy
> to read up on what it would take to set things up so I can commit" :-)

That's a pretty good reason. Will commit in your name, when it's time for.

> Robert

Best regards, Michael.

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Eric Abrahamsen]

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Wed, 20 Nov 2019 14:43:46 +0100, Michael Albinus <michael.albinus@gmx.de> said:
>
>     Michael> Works for me. However, the "Status:" messages shall be kicked off. And
>     Michael> it shall be checked, whether `debbugs-gnu-mail-backend' is bound to `gnus'.
>
> The clue is in the name of the function, but OK.
>
> No more feature requests :-)
>
>     Michael> If Lars doesn't object, you might commit it, plus docstring, and plus
>     Michael> documentation in debbugs-ug.texi.
>
> I donʼt think Iʼm set up to push to elpa.
>
> (defun debbugs-gnus-ephemeral-from-tagged ()
>   "Create an ephemeral Gnus group of locally tagged emacs bugs."
>   (interactive)
>   (unless (eq debbugs-gnu-mail-backend 'gnus)
>     (error "This function only works with Gnus."))
>   (when (file-exists-p debbugs-gnu-persistency-file)
>     (with-temp-buffer
>       (insert-file-contents debbugs-gnu-persistency-file)
>       (eval (read (current-buffer)) t))
>     (when debbugs-gnu-local-tags
>       (let ((mbox-url (replace-regexp-in-string
>                        ";mboxstat=yes" ""
>                        (alist-get 'emacs gnus-bug-group-download-format-alist)
>                        nil t)))
>       (gnus-read-ephemeral-bug-group
>        debbugs-gnu-local-tags
>        mbox-url)))))

How nice to wake up and find that someone's implemented my (fairly
off-handed) feature request! In fact I wasn't necessarily demanding that
all the bugs be displayed in one summary buffer -- that actually skips
the overview buffer where you can issue the control messages, which is
the main benefit. Though I'm sure I'd use this ephemeral group approach,
the tagging approach is perfectly good. I was a little confused when
searching for tags prompted me to enter a particular user tag, but
it turns out just hitting <RET> at the prompt showed me the tagged
reports.

Thanks for both of these approaches!

Eric

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Robert Pluim]

>>>>> On Wed, 20 Nov 2019 08:42:47 -0800, Eric Abrahamsen <eric@ericabrahamsen.net> said:
    Eric> How nice to wake up and find that someone's implemented my (fairly
    Eric> off-handed) feature request! In fact I wasn't necessarily demanding that
    Eric> all the bugs be displayed in one summary buffer -- that actually skips
    Eric> the overview buffer where you can issue the control messages, which is
    Eric> the main benefit. Though I'm sure I'd use this ephemeral group approach,
    Eric> the tagging approach is perfectly good. I was a little confused when
    Eric> searching for tags prompted me to enter a particular user tag, but
    Eric> it turns out just hitting <RET> at the prompt showed me the tagged
    Eric> reports.

If you stick

      (debbugs-gnu-summary-mode 1)

at the end of my function, you'll get 'E' 'C' and 'M-m' bindings,
which do the same things as in 'debbugs-gnu'.

Robert

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Robert Pluim <rpluim@gmail.com> writes:

>     Eric> I was a little confused when searching for tags prompted me
>     Eric> to enter a particular user tag, but it turns out just
>     Eric> hitting <RET> at the prompt showed me the tagged reports.
>
> If you stick
>
>       (debbugs-gnu-summary-mode 1)
>
> at the end of my function, you'll get 'E' 'C' and 'M-m' bindings,
> which do the same things as in 'debbugs-gnu'.

Do they find already the proper bug-id? Cool, this saves me work :-)

> Robert

Best regards, Michael.

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Robert Pluim]

>>>>> On Wed, 20 Nov 2019 18:01:42 +0100, Michael Albinus <michael.albinus@gmx.de> said:

    Michael> Robert Pluim <rpluim@gmail.com> writes:
    Eric> I was a little confused when searching for tags prompted me
    Eric> to enter a particular user tag, but it turns out just
    Eric> hitting <RET> at the prompt showed me the tagged reports.
    >> 
    >> If you stick
    >> 
    >> (debbugs-gnu-summary-mode 1)
    >> 
    >> at the end of my function, you'll get 'E' 'C' and 'M-m' bindings,
    >> which do the same things as in 'debbugs-gnu'.

    Michael> Do they find already the proper bug-id? Cool, this saves me work :-)

Kind of, they search for #\\([0-9]+\\) on the current line, and then
from point-min, which will do the wrong thing if youʼre not on the
first line of the bug thread. They also look for an appropriate text
property, but thatʼs not set by 'gnus-read-ephemeral-bug-group'. I can
look at adding that, but if you have a debbugs-gnu list of tagged
bugs, you can visit each of them as a gnus group individually, so Iʼm
not sure itʼs worth the effort.

Robert

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Robert Pluim <rpluim@gmail.com> writes:

>     >> If you stick
>     >>
>     >> (debbugs-gnu-summary-mode 1)
>     >>
>     >> at the end of my function, you'll get 'E' 'C' and 'M-m' bindings,
>     >> which do the same things as in 'debbugs-gnu'.
>
>     Michael> Do they find already the proper bug-id? Cool, this saves
>     Michael> me work :-)
>
> Kind of, they search for #\\([0-9]+\\) on the current line, and then
> from point-min, which will do the wrong thing if youʼre not on the
> first line of the bug thread. They also look for an appropriate text
> property, but thatʼs not set by 'gnus-read-ephemeral-bug-group'. I can
> look at adding that, but if you have a debbugs-gnu list of tagged
> bugs, you can visit each of them as a gnus group individually, so Iʼm
> not sure itʼs worth the effort.

As you want. It's just convenience to send a control message immediately
while you're reading a given message in the accumulated summary buffer.

> Robert

Best regards, Michael.

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Robert Pluim]

>>>>> On Wed, 20 Nov 2019 18:34:11 +0100, Michael Albinus <michael.albinus@gmx.de> said:

    Michael> Robert Pluim <rpluim@gmail.com> writes:
    >> >> If you stick
    >> >>
    >> >> (debbugs-gnu-summary-mode 1)
    >> >>
    >> >> at the end of my function, you'll get 'E' 'C' and 'M-m' bindings,
    >> >> which do the same things as in 'debbugs-gnu'.
    >> 
    Michael> Do they find already the proper bug-id? Cool, this saves
    Michael> me work :-)
    >> 
    >> Kind of, they search for #\\([0-9]+\\) on the current line, and then
    >> from point-min, which will do the wrong thing if youʼre not on the
    >> first line of the bug thread. They also look for an appropriate text
    >> property, but thatʼs not set by 'gnus-read-ephemeral-bug-group'. I can
    >> look at adding that, but if you have a debbugs-gnu list of tagged
    >> bugs, you can visit each of them as a gnus group individually, so Iʼm
    >> not sure itʼs worth the effort.

    Michael> As you want. It's just convenience to send a control message immediately
    Michael> while you're reading a given message in the accumulated summary buffer.

Well, with debbug-gnu-summary-mode enabled you'll be able to do that,
you'll just have to be slightly more careful with those
commands.

Hmm, how about if I changed the detection code to do

- search current line
- search backwards
- search from (point-min)

That would change nothing for single-bug ephemeral groups.

Robert

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Robert Pluim <rpluim@gmail.com> writes:

> Well, with debbug-gnu-summary-mode enabled you'll be able to do that,
> you'll just have to be slightly more careful with those
> commands.
>
> Hmm, how about if I changed the detection code to do
>
> - search current line
> - search backwards
> - search from (point-min)
>
> That would change nothing for single-bug ephemeral groups.

Pls do. As always, I'm happy if I can avoid work :-)

> Robert

Best regards, Michael.

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Robert Pluim]

>>>>> On Wed, 20 Nov 2019 18:51:29 +0100, Michael Albinus <michael.albinus@gmx.de> said:

    Michael> Robert Pluim <rpluim@gmail.com> writes:
    >> Well, with debbug-gnu-summary-mode enabled you'll be able to do that,
    >> you'll just have to be slightly more careful with those
    >> commands.
    >> 
    >> Hmm, how about if I changed the detection code to do
    >> 
    >> - search current line
    >> - search backwards
    >> - search from (point-min)
    >> 
    >> That would change nothing for single-bug ephemeral groups.

    Michael> Pls do. As always, I'm happy if I can avoid work :-)

Iʼm going to have to go and read "How to commit to Elpa for dummies"
now, aren't I? :-)

Robert

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Robert Pluim <rpluim@gmail.com> writes:

> Iʼm going to have to go and read "How to commit to Elpa for dummies"
> now, aren't I? :-)

It's simple. The repository is at git.sv.gnu.org:/srv/git/emacs/elpa.git

After pulling, you call "make -k", because some of the packages don't
compile successfully.

If you want also to see the external packages, you need "make externals"
between "git pull" and "make -k". That's all.

> Robert

Best regards, Michael.

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Eric Abrahamsen]

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Wed, 20 Nov 2019 18:51:29 +0100, Michael Albinus <michael.albinus@gmx.de> said:
>
>     Michael> Robert Pluim <rpluim@gmail.com> writes:
>     >> Well, with debbug-gnu-summary-mode enabled you'll be able to do that,
>     >> you'll just have to be slightly more careful with those
>     >> commands.
>     >> 
>     >> Hmm, how about if I changed the detection code to do
>     >> 
>     >> - search current line
>     >> - search backwards
>     >> - search from (point-min)
>     >> 
>     >> That would change nothing for single-bug ephemeral groups.
>
>     Michael> Pls do. As always, I'm happy if I can avoid work :-)
>
> Iʼm going to have to go and read "How to commit to Elpa for dummies"
> now, aren't I? :-)

I don't think it's that hard! Clone from
<user>@git.sv.gnu.org:/srv/git/emacs/elpa.git, and everything else
should happen automatically.

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Robert Pluim]

>>>>> On Wed, 20 Nov 2019 10:43:19 -0800, Eric Abrahamsen <eric@ericabrahamsen.net> said:

    Eric> I don't think it's that hard! Clone from
    Eric> <user>@git.sv.gnu.org:/srv/git/emacs/elpa.git, and everything else
    Eric> should happen automatically.

Yes, itʼs not difficult. Iʼm all set now, I can commit, or push to a
scratch branch, or post patches here, whatever people want.

Robert

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

Hi Eric,

>> (defun debbugs-gnus-ephemeral-from-tagged ()

Btw, this will be `debbugs-gnu-ephemeral-from-tagged'.

> How nice to wake up and find that someone's implemented my (fairly
> off-handed) feature request! In fact I wasn't necessarily demanding that
> all the bugs be displayed in one summary buffer -- that actually skips
> the overview buffer where you can issue the control messages, which is
> the main benefit. Though I'm sure I'd use this ephemeral group approach,
> the tagging approach is perfectly good.

You can already issue control messages from existing ephemeral groups in
debbugs, Lars did implement it. I will see whether I could bring it also
to this accumulated group; shall be possible, because the bug number is
always in the subject.

But first I'll wait for a comment from Lars.

> I was a little confused when searching for tags prompted me to enter a
> particular user tag, but it turns out just hitting <RET> at the prompt
> showed me the tagged reports.

Yes, it is a little bit unfortune that we have both local tags (used
here) and user tags (propagated to the server). Should have been made it
more clear in the UI.

By the way, I have customized debbugs-gnu-default-severities to
'("tagged") - always starting with the local tagged bugs.

> Thanks for both of these approaches!
>
> Eric

Best regards, Michael.

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Robert Pluim]

>>>>> On Wed, 20 Nov 2019 17:59:18 +0100, Michael Albinus <michael.albinus@gmx.de> said:

    Michael> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
    Michael> Hi Eric,

    >>> (defun debbugs-gnus-ephemeral-from-tagged ()

    Michael> Btw, this will be `debbugs-gnu-ephemeral-from-tagged'.

'debbugs-gnu-gnus-ephemeral-from-tagged'? In theory you could
implement something similar for rmail. I hereby delegate the heavy
responsibility of naming this correctly to Somebody Else :-)

    >> How nice to wake up and find that someone's implemented my (fairly
    >> off-handed) feature request! In fact I wasn't necessarily demanding that
    >> all the bugs be displayed in one summary buffer -- that actually skips
    >> the overview buffer where you can issue the control messages, which is
    >> the main benefit. Though I'm sure I'd use this ephemeral group approach,
    >> the tagging approach is perfectly good.

    Michael> You can already issue control messages from existing ephemeral groups in
    Michael> debbugs, Lars did implement it. I will see whether I could bring it also
    Michael> to this accumulated group; shall be possible, because the bug number is
    Michael> always in the subject.

(debbugs-gnu-summary-mode 1)

(but see my caveats in my other message to Eric).

Robert

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Robert Pluim <rpluim@gmail.com> writes:

>     Michael> Btw, this will be `debbugs-gnu-ephemeral-from-tagged'.
>
> 'debbugs-gnu-gnus-ephemeral-from-tagged'? In theory you could
> implement something similar for rmail. I hereby delegate the heavy
> responsibility of naming this correctly to Somebody Else :-)

We have no `debbugs-gnus-*' functions. The two accidents left I've just
fixed in the repository.

Maybe we shall even avoid `ephemeral' in the name (its gnus-ish), and we
call it `debbugs-read-tagged-bugs-with-gnus'. With the upper command
`debbugs-read-tagged-bugs'.

> Robert

Best regards, Michael.

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Robert Pluim]

>>>>> On Wed, 20 Nov 2019 18:43:42 +0100, Michael Albinus <michael.albinus@gmx.de> said:

    Michael> Robert Pluim <rpluim@gmail.com> writes:
    Michael> Btw, this will be `debbugs-gnu-ephemeral-from-tagged'.
    >> 
    >> 'debbugs-gnu-gnus-ephemeral-from-tagged'? In theory you could
    >> implement something similar for rmail. I hereby delegate the heavy
    >> responsibility of naming this correctly to Somebody Else :-)

    Michael> We have no `debbugs-gnus-*' functions. The two accidents left I've just
    Michael> fixed in the repository.

    Michael> Maybe we shall even avoid `ephemeral' in the name (its gnus-ish), and we
    Michael> call it `debbugs-read-tagged-bugs-with-gnus'. With the upper command
    Michael> `debbugs-read-tagged-bugs'.

Hmm, is this not only for the GNU bugtracker, hence debbugs-gnu? Or
are you generalizing it to all types of debbugs instances?

Robert

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Robert Pluim <rpluim@gmail.com> writes:

>     Michael> We have no `debbugs-gnus-*' functions. The two accidents
>     Michael> left I've just fixed in the repository.
>
>     Michael> Maybe we shall even avoid `ephemeral' in the name (its
>     Michael> gnus-ish), and we call it
>     Michael> `debbugs-read-tagged-bugs-with-gnus'. With the upper
>     Michael> command `debbugs-read-tagged-bugs'.
>
> Hmm, is this not only for the GNU bugtracker, hence debbugs-gnu? Or
> are you generalizing it to all types of debbugs instances?

You are right, of course. But Lars(?) did break this rule already with
`debbugs-read-emacs-bug-with-gnus'.

Well, since this and ´debbugs-read-emacs-bug-with-rmail' are the only
exceptions (and no commands to preserve), we shall rename them to
debbugs-gnu-*. And your new functions shall start then also with
debbugs-gnu-.

I shall stop to write for today. Too much confusion.

> Robert

Best regards, Michael.

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> I donʼt think Iʼm set up to push to elpa.
>
> (defun debbugs-gnus-ephemeral-from-tagged ()
>   "Create an ephemeral Gnus group of locally tagged emacs bugs."
>   (interactive)
>   (unless (eq debbugs-gnu-mail-backend 'gnus)
>     (error "This function only works with Gnus."))

I think that's a good command, but it now what I thought was requested:
I thought the user just wanted the list from M-x debbugs-gnu, but only
with the tagged bug reports.  Because fetching and updating that list is
a million percentage faster than fetching the data for all the bug
reports.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Robert Pluim <rpluim@gmail.com> writes:
>
>> (defun debbugs-gnus-ephemeral-from-tagged ()
>>   "Create an ephemeral Gnus group of locally tagged emacs bugs."
>>   (interactive)
>>   (unless (eq debbugs-gnu-mail-backend 'gnus)
>>     (error "This function only works with Gnus."))
>
> I think that's a good command, but it now what I thought was requested:
> I thought the user just wanted the list from M-x debbugs-gnu, but only
> with the tagged bug reports.  Because fetching and updating that list is
> a million percentage faster than fetching the data for all the bug
> reports.

That's (debbugs-gnu '("tagged"))

Don't know whether it is worth to provide it as extra command. Maybe
just documenting in the debbugs user guide?

Best regards, Michael.

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Lars Ingebrigtsen]

Michael Albinus <michael.albinus@gmx.de> writes:

> That's (debbugs-gnu '("tagged"))

Ah, right.

> Don't know whether it is worth to provide it as extra command. Maybe
> just documenting in the debbugs user guide?

I think it's make a useful command.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Lars Ingebrigtsen <larsi@gnus.org> writes:

>> That's (debbugs-gnu '("tagged"))
>
> Ah, right.
>
>> Don't know whether it is worth to provide it as extra command. Maybe
>> just documenting in the debbugs user guide?
>
> I think it's make a useful command.

I've pushed a change to elpa, adding commands debbugs-gnu-tagged and
debbugs-org-tagged.

Robert, your command in preparation might change slightly the
scope. Instead of showing all messages for the tagged bugs, it might
show all messages for the bugs in the current Debbugs buffer. This is
more general.

Since the current buffer might contain many bugs, there should be a
limit. Let's say, show messages of max 50 bugs (the most highest ones
per number).

Once we have settled this command, I'll make a debbugs release for elpa.

Best regards, Michael.

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Robert Pluim]

>>>>> On Thu, 21 Nov 2019 15:54:00 +0100, Michael Albinus <michael.albinus@gmx.de> said:

    Michael> I've pushed a change to elpa, adding commands debbugs-gnu-tagged and
    Michael> debbugs-org-tagged.

    Michael> Robert, your command in preparation might change slightly the
    Michael> scope. Instead of showing all messages for the tagged bugs, it might
    Michael> show all messages for the bugs in the current Debbugs buffer. This is
    Michael> more general.

Hey, when did I sign up for this? :-)

    Michael> Since the current buffer might contain many bugs, there should be a
    Michael> limit. Let's say, show messages of max 50 bugs (the most highest ones
    Michael> per number).

Iʼm not sure what you mean with that parenthetical statement. This
does it for the first 10 bugs in the current debbugs list. I could add
an &optional all, but since in my testing Iʼve already accidentally
requested 1000 bugs from the server, that might not be a good idea.

diff --git a/packages/debbugs/debbugs-gnu.el b/packages/debbugs/debbugs-gnu.el
index bf3ed2cd1..3e16a78a8 100644
--- a/packages/debbugs/debbugs-gnu.el
+++ b/packages/debbugs/debbugs-gnu.el
@@ -1399,6 +1399,29 @@ (defun debbugs-read-emacs-bug-with-rmail (id status merged)
     (define-key rmail-mode-map "E" 'debbugs-gnu-make-control-message)
     (rmail-show-message 1)))
 
+(defvar debbugs-gnu-read-bugs-limit 10)
+
+(defun debbugs-gnu-read-current-bugs-with-gnus ()
+  "Create a Gnus group of the messages from the currently shown bugs."
+  (interactive)
+  (unless (eq debbugs-gnu-mail-backend 'gnus)
+    (error "This function only works with Gnus."))
+  (save-excursion
+    (let ((mbox-url (replace-regexp-in-string
+                     ";mboxstat=yes" ""
+                     (alist-get 'emacs gnus-bug-group-download-format-alist)
+                     nil t))
+          ids)
+      (goto-char (point-min))
+      (dotimes (i debbugs-gnu-read-bugs-limit)
+        (push (cdr (assq 'id (tabulated-list-get-id))) ids)
+        (forward-line 1))
+      (setq ids (delete nil (nreverse ids)))
+      (gnus-read-ephemeral-bug-group
+       ids
+       mbox-url)
+      (debbugs-gnu-summary-mode 1))))
+
 (defun debbugs-read-tagged-bugs-with-gnus ()
   "Create an ephemeral Gnus group of locally tagged Emacs bugs."
   (interactive)

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Robert Pluim <rpluim@gmail.com> writes:

>     Michael> Robert, your command in preparation might change slightly
>     Michael> the scope. Instead of showing all messages for the tagged
>     Michael> bugs, it might show all messages for the bugs in the
>     Michael> current Debbugs buffer. This is more general.
>
> Hey, when did I sign up for this? :-)

I could read your mind, you know? :-)

> +(defvar debbugs-gnu-read-bugs-limit 10)

I would make it a defcustom. Maybe we need also another defconst, which
is the upper limit this user option can be. Something like 50, or 100.

> +(defun debbugs-gnu-read-current-bugs-with-gnus ()
> +  "Create a Gnus group of the messages from the currently shown bugs."
> +  (interactive)
> +  (unless (eq debbugs-gnu-mail-backend 'gnus)
> +    (error "This function only works with Gnus."))
> +  (save-excursion
> +    (let ((mbox-url (replace-regexp-in-string
> +                     ";mboxstat=yes" ""
> +                     (alist-get 'emacs gnus-bug-group-download-format-alist)
> +                     nil t))
> +          ids)
> +      (goto-char (point-min))
> +      (dotimes (i debbugs-gnu-read-bugs-limit)
> +        (push (cdr (assq 'id (tabulated-list-get-id))) ids)
> +        (forward-line 1))
> +      (setq ids (delete nil (nreverse ids)))
> +      (gnus-read-ephemeral-bug-group
> +       ids
> +       mbox-url)
> +      (debbugs-gnu-summary-mode 1))))

That works so far for me, thanks. What I'm missing is handling of merged
bugs. That is, if a bug in a line shows also merged bug numbers.

Create a bug list via (debbugs-gnu-bugs 34322) . And then compare the
Gnus buffers, created by either (debbugs-gnu-select-report) or by
(debbugs-gnu-read-current-bugs-with-gnus) .

We would need also a command debbugs-gnu-read-current-bugs and key
bindings for this. Just to be prepared for a possible
debbugs-gnu-read-current-bugs-with-rmail.

If this is too much for your limited time, commit what you have, and I
will adapt the final bits then.

>  (defun debbugs-read-tagged-bugs-with-gnus ()
>    "Create an ephemeral Gnus group of locally tagged Emacs bugs."

I believe, this one is not needed anymore. A list with locally tagged
bugs is created now easily via debbugs-gnu-tagged. And there you call
debbugs-gnu-read-current-bugs.

Thanks, and best regards, Michael.

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Robert Pluim]

[-- Attachment #1: Type: text/plain, Size: 1428 bytes --]

>>>>> On Fri, 22 Nov 2019 09:30:49 +0100, Michael Albinus <michael.albinus@gmx.de> said:

>> +(defvar debbugs-gnu-read-bugs-limit 10)

    Michael> I would make it a defcustom. Maybe we need also another defconst, which
    Michael> is the upper limit this user option can be. Something like 50, or 100.

If we make it a defcustom we can encode checking the max in the
defcustom form.

    Michael> That works so far for me, thanks. What I'm missing is handling of merged
    Michael> bugs. That is, if a bug in a line shows also merged bug numbers.

Yes, done.

    Michael> We would need also a command debbugs-gnu-read-current-bugs and key
    Michael> bindings for this. Just to be prepared for a possible
    Michael> debbugs-gnu-read-current-bugs-with-rmail.

I chose 'A' for 'A'll, and did some renaming. Suggestions welcome.

    Michael> If this is too much for your limited time, commit what you have, and I
    Michael> will adapt the final bits then.

"You get what anybody gets: you get a lifetime" :-)

    >> (defun debbugs-read-tagged-bugs-with-gnus ()
    >> "Create an ephemeral Gnus group of locally tagged Emacs bugs."

    Michael> I believe, this one is not needed anymore. A list with locally tagged
    Michael> bugs is created now easily via debbugs-gnu-tagged. And there you call
    Michael> debbugs-gnu-read-current-bugs.

Yes, it was left over, Iʼve removed it.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-command-to-show-mailboxes-for-all-currently-show.patch --]
[-- Type: text/x-patch, Size: 4950 bytes --]

From 2378e7a1529fa372c1bc9aee0373b380df6602a2 Mon Sep 17 00:00:00 2001
From: Robert Pluim <rpluim@gmail.com>
Date: Fri, 22 Nov 2019 11:49:27 +0100
Subject: [PATCH] Add command to show mailboxes for all currently shown reports

* debbugs-gnu.el (debbugs-gnu-mode-map): Add binding for debbugs-gnu-select-current-bugs.
(debbugs-gnu-select-bugs-limit-max): New defconst.  Max for debbugs-gnu-select-bugs-limit.
(debbugs-gnu-select-bugs-limit): New defcustom.  Limits how many bugs
debbugs-gnu-select-current-bugs will retrieve.
(debbugs-gnu-select-current-bugs): New function. Shows mailboxes
for all currently shown bugs.
(debbugs-gnu-select-current-bugs-with-gnus): Implementation of mailbox
retrieval using Gnus.

* debbugs-ug.texi (Tabulated Lists): Add description of debbugs-gnu-select-current-bugs.
---
 packages/debbugs/debbugs-gnu.el  | 48 +++++++++++++++++++++++++++++++-
 packages/debbugs/debbugs-ug.texi |  8 ++++++
 2 files changed, 55 insertions(+), 1 deletion(-)

diff --git a/packages/debbugs/debbugs-gnu.el b/packages/debbugs/debbugs-gnu.el
index 303b62557..a4755694d 100644
--- a/packages/debbugs/debbugs-gnu.el
+++ b/packages/debbugs/debbugs-gnu.el
@@ -971,6 +971,7 @@ (defvar debbugs-gnu-mode-map
     (set-keymap-parent map tabulated-list-mode-map)
     (define-key map "\r" 'debbugs-gnu-select-report)
     (define-key map [mouse-2] 'debbugs-gnu-select-report)
+    (define-key map "A" 'debbugs-gnu-select-current-bugs)
     (define-key map "g" 'debbugs-gnu-rescan)
     (define-key map "R" 'debbugs-gnu-show-all-blocking-reports)
     (define-key map "C" 'debbugs-gnu-send-control-message)
@@ -990,10 +991,14 @@ (defvar debbugs-gnu-mode-map
     (define-key menu-map [debbugs-gnu-select-report]
       '(menu-item "Show Reports" debbugs-gnu-select-report
 		  :help "Show all reports belonging to this bug"))
+    (define-key-after menu-map [debbugs-gnu-select-current]
+      '(menu-item "Show Reports For All" debbugs-gnu-select-current-bugs
+		  :help "Show reports for all currently shown bugs")
+      'debbugs-gnu-select-report)
     (define-key-after menu-map [debbugs-gnu-rescan]
       '(menu-item "Refresh Bugs" debbugs-gnu-rescan
 		  :help "Refresh bug list")
-      'debbugs-gnu-select-report)
+      'debbugs-gnu-select-current)
     (define-key-after menu-map [debbugs-gnu-show-all-blocking-reports]
       '(menu-item "Show Release Blocking Bugs"
 		  debbugs-gnu-show-all-blocking-reports
@@ -1399,6 +1404,47 @@ (defun debbugs-read-emacs-bug-with-rmail (id status merged)
     (define-key rmail-mode-map "E" 'debbugs-gnu-make-control-message)
     (rmail-show-message 1)))
 
+(defconst debbugs-gnu-select-bugs-limit-max 50)
+(defcustom debbugs-gnu-select-bugs-limit 10
+  "Maximum number of bugs to retrieve for Gnus group.
+This applies for `debbugs-gnu-select-current-bugs.
+Maximum allowed value is 50 to avoid overloading the server."
+  :type '(integer
+          :validate
+          (lambda (widget)
+            (unless (<= 1
+                        (widget-value widget)
+                        debbugs-gnu-select-bugs-limit-max)
+              (widget-put widget :error
+                          "Invalid value: range is 1..50"))))
+  :version "27.1")
+
+(defun debbugs-gnu-select-current-bugs ()
+  "Retrieve the mailboxes for all currently shown bugs.
+Limited by `debbugs-gnu-select-bugs-limit'."
+  (interactive)
+  (unless (eq debbugs-gnu-mail-backend 'gnus)
+    (error "This function only works with Gnus."))
+  (debbugs-gnu-select-current-bugs-with-gnus))
+
+(defun debbugs-gnu-select-current-bugs-with-gnus ()
+  "Create a Gnus group of the messages from the currently shown bugs."
+  (save-excursion
+    (let ((mbox-url
+           (replace-regexp-in-string
+            ";mboxstat=yes" ""
+            (alist-get 'emacs gnus-bug-group-download-format-alist)
+            nil t))
+          ids)
+      (goto-char (point-min))
+      (dotimes (i debbugs-gnu-select-bugs-limit)
+        (push (debbugs-gnu-current-id t) ids)
+        (push (cdr (assq 'mergedwith (debbugs-gnu-current-status))) ids)
+        (forward-line 1))
+      (setq ids (delete nil (nreverse ids)))
+      (gnus-read-ephemeral-bug-group ids mbox-url)
+      (debbugs-gnu-summary-mode 1))))
+
 (defcustom debbugs-gnu-lars-workflow nil
   "If non-nil, set some Gnus vars as preferred by Lars."
   :type 'boolean
diff --git a/packages/debbugs/debbugs-ug.texi b/packages/debbugs/debbugs-ug.texi
index 8121f7bf8..6e86194e5 100644
--- a/packages/debbugs/debbugs-ug.texi
+++ b/packages/debbugs/debbugs-ug.texi
@@ -373,6 +373,14 @@
 Show the email messages that discuss the bug.
 
 @* @item
+@kindex @kbd{A}
+@kbd{A} @tab
+@code{debbugs-gnu-select-current-bugs} @*
+Show the email messages for all currently displayed messages
+(including merged reports).  Limited by user option
+@code{debbugs-gnu-select-bugs-limit}.
+
+@item
 @kindex @kbd{d}
 @kbd{d} @tab
 @code{debbugs-gnu-display-status} @*
-- 
2.24.0.155.gd9f6f3b619

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Robert Pluim <rpluim@gmail.com> writes:

> Subject: [PATCH] Add command to show mailboxes for all currently shown reports

Everything looks pretty good. Just some very minor nits:

> +(defconst debbugs-gnu-select-bugs-limit-max 50)

Pls add a docstring.

> +              (widget-put widget :error
> +                          "Invalid value: range is 1..50"))))

I would do (format "Invalid value: range is 1..%d" debbugs-gnu-select-bugs-limit-max)

Please commit to elpa, and thanks again!

Best regards, Michael.

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Robert Pluim]

>>>>> On Fri, 22 Nov 2019 12:41:56 +0100, Michael Albinus <michael.albinus@gmx.de> said:

    Michael> Please commit to elpa, and thanks again!

Done as 3dc62c20c

Iʼm unvolunteering myself for the rmail version: I haven't used rmail
in decades.

Robert

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Robert Pluim <rpluim@gmail.com> writes:

>     Michael> Please commit to elpa, and thanks again!
>
> Done as 3dc62c20c

Thanks! I've committed another code cleanup in debbugs, which was sleeping
for a while locally (I didn't want to disturb you while preparing your patch).

> Iʼm unvolunteering myself for the rmail version: I haven't used rmail
> in decades.

Maybe I give it a try.

> Robert

Best regards, Michael.

Re: bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

>> Iʼm unvolunteering myself for the rmail version: I haven't used rmail
>> in decades.
>
> Maybe I give it a try.

Implemented, and pushed. I'm not an Rmail user, so I hope this is useful.

>> Robert

Best regards, Michael.

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[dick.r.chiang]

EA> If you have access to the Emacs repos and are collaborating with others
EA> who do, too, then it can make sense to push a "feature/foo" branch to the
EA> repos, and share it.

Someone was somehow able to upload a gnus feature branch in the
official repo and get core devs to test it.

It's described here
https://lists.gnu.org/archive/html/emacs-devel/2019-10/msg00689.html.

Is this what you're referring to when you say "push a feature/foo"
branch?  Currently when I "git clone http://git.sv.gnu.org/git/emacs.git", git just
hangs.

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Eric Abrahamsen]

On 12/14/19 11:22 AM, dick.r.chiang@gmail.com wrote:
> EA> If you have access to the Emacs repos and are collaborating with others
> EA> who do, too, then it can make sense to push a "feature/foo" branch to the
> EA> repos, and share it.
>
> Someone was somehow able to upload a gnus feature branch in the
> official repo and get core devs to test it.
>
> It's described here
> https://lists.gnu.org/archive/html/emacs-devel/2019-10/msg00689.html.
>
> Is this what you're referring to when you say "push a feature/foo"
> branch?

Yup, that's what I meant. The "feature/" prefix is one convention, as is
"scratch/" for temporary stuff, or using your own name/handle as a prefix.

> Currently when I "git clone http://git.sv.gnu.org/git/emacs.git", git
> just hangs.

According to https://savannah.gnu.org/git/?group=emacs, it should be:

git clone https://git.savannah.gnu.org/git/emacs.git

Which works for me. Though, oddly, I've got:

girzel@git.sv.gnu.org:/srv/git/emacs.git

For my "member" remote, and that works fine. The two domain names seem
to point to the same server. Maybe it was the non-TLS url?

You'll need to be a member to push a branch, anyway.

Eric

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Lars Ingebrigtsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> According to https://savannah.gnu.org/git/?group=emacs, it should be:
>
> git clone https://git.savannah.gnu.org/git/emacs.git

This was over a year ago -- has there been any progress here?  That
patch itself here doesn't, as far as I understand it, actually work yet,
but I haven't tried it myself.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[dick.r.chiang]

My enthusiasm has waned.  It's sort of clear from the static on gnus.general
that gnusers are a dying, none-too-bright breed whose mild interest in
non-blocking fetch couldn't possibly justify the testing that a 3,300 line
change would require.

Closing this out, although I can never tell with debbugs whether my "control
messages" take.

bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread, bug#38136: [PATCH] Make gnus-group-get-new-news a non blocking thread

[Robert Pluim]

>>>>> On Fri, 08 Nov 2019 19:31:17 -0800, Eric Abrahamsen <eric@ericabrahamsen.net> said:

    Eric> dick.r.chiang@gmail.com writes:

    >> From 834327d458c54bb0e1f25c6259ee640df0ba8b0e Mon Sep 17 00:00:00 2001
    >> From: dickmao <none>
    >> Date: Fri, 8 Nov 2019 09:51:59 -0500
    >> Subject: [PATCH] Make `gnus-group-get-new-news` a non-blocking thread

    Eric> I'm looking forward to testing this out this weekend!

    Eric> Just a quick NB, which you should feel free to ignore: you'll probably
    Eric> get much better uptake on these tickets if you include with them a
    Eric> head-to-toe prose report on what they accomplish and why (in detail).
    Eric> Emacs has a lot of bug reports, and a small number of bug hunters, and a
    Eric> multi-hundred-line patch without so much as a "howdy-do" on it is ripe
    Eric> to be ignored. Just something to consider.

That, plus the patch has spurious whitspace changes, and things like
the below, which makes it all harder to read.

>>>>> On Fri, 08 Nov 2019 09:56:41 -0500, dick.r.chiang@gmail.com said:
    dick> diff --git a/src/fns.c b/src/fns.c
    dick> index cbb6879223..7d4ed7cab6 100644
    dick> --- a/src/fns.c
    dick> +++ b/src/fns.c
    dick> @@ -2917,8 +2917,7 @@ require_unwind (Lisp_Object old_value)
 
    dick>  DEFUN ("require", Frequire, Srequire, 1, 3, 0,
    dick>         doc: /* If feature FEATURE is not loaded, load it from FILENAME.
    dick> -If FEATURE is not a member of the list `features', then the feature is
    dick> -not loaded; so load the file FILENAME.
    dick> +Loaded features are recorded in the list variable `features'.
 
    dick>  If FILENAME is omitted, the printname of FEATURE is used as the file
    dick>  name, and `load' will try to load this name appended with the suffix
    dick> -- 
    dick> 2.23.0

bug#38136: https://github.com/dickmao/gnus

[dick.r.chiang]

RP> That, plus the patch has spurious whitspace changes, and things like the
RP> below, which makes it all harder to read.

I've a mind, of course, to bring non-blocking goodness to the main branch, but
the volume of changes (more than 200 commits and counting at
github.com/dickmao/gnus) requires a social effort that may never reach critical mass.

The original patch wishfully believed dynamic-letting globals like
`gnus-summary-buffer` would do an end-run around thread-safety.  But "leakage"
occurs far too often as manifested by "Selected deleted buffer" errors.

bug#38136: https://github.com/dickmao/gnus

[Eric Abrahamsen]

dick.r.chiang@gmail.com writes:

> RP> That, plus the patch has spurious whitspace changes, and things like the
> RP> below, which makes it all harder to read.
>
> I've a mind, of course, to bring non-blocking goodness to the main branch, but
> the volume of changes (more than 200 commits and counting at
> github.com/dickmao/gnus) requires a social effort that may never reach critical mass.

Don't give up hope! But I definitely recommend starting the social
effort part of things before you invest enormous amounts of time in code
(too late, I know). This is a collaborative effort, and people have
*very* strong opinions about how their Gnus behaves, and you're better
off floating ideas sooner rather than later -- for example, any one of
us could have told you that requiring users to set `gnus-select-methods'
via customization (and not `setq') was probably a non-starter. Which
could have saved you some time!

> The original patch wishfully believed dynamic-letting globals like
> `gnus-summary-buffer` would do an end-run around thread-safety.  But "leakage"
> occurs far too often as manifested by "Selected deleted buffer" errors.

Every time I've embarked on grand refactorings of Gnus I've run up
against problems like this -- every part of Gnus is tied so tightly to
every other part, it's very hard to unpick. So I've started working
backwards from the big goals. Do we want threaded server updates?
`nntp-server-buffer' is the problem. So each server should probably have
its own "work buffer". The easiest way to make that happen is if servers
are structs. And on down from there, trying to work down to a root
problem that, when solved, will reduce complexity instead of increasing
it, and boost modularity rather than causing tighter code integration.

Then I still screw it up in the end, but hey, I think it's the right
approach.

I think a lot of our problems will be made more tractable by getting rid
of code that assumes a "current" thing. The "current" server. The
"current" group. The "current" summary buffer. Pass arguments, don't
check dynamic variables. Maybe that would be a better place to start.

That patch that could end up enabling the big refactors later might not
look like much at first glance.