Summary by thread in rmail

Summary by thread in rmail

[Andrea Monaco]

Can rmail display a summary by thread?  That would be useful to me, and
I'm willing to work on it.



Andrea Monaco

Re: Summary by thread in rmail

[Eli Zaretskii]

> From: Andrea Monaco <andrea.monaco@autistici.org>
> Date: Wed, 05 Oct 2022 20:31:50 +0200
> 
> Can rmail display a summary by thread?

Not really.  There's "C-M-t TOPIC RET", but it matches TOPIC against
the Subject of the messages, so it only works as long as people didn't
change Subject mid-thread.

> That would be useful to me, and I'm willing to work on it.

Thanks, adding that would be useful, I think.

Re: Summary by thread in rmail

[Emanuel Berg]

Andrea Monaco wrote:

> Can rmail display a summary by thread? That would be useful
> to me, and I'm willing to work on it.

Great, but you are aware of Emacs Gnus?

-- 
underground experts united
https://dataswamp.org/~incal

Re: Summary by thread in rmail

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Can rmail display a summary by thread?  That would be useful to me, and
  > I'm willing to work on it.

I would appreciate that feature very much.

Another related feature that would be nice is one to generated a
summary by filtering the messages currently listed in the summary.
For instance, suppose that C-u C-M-t reduced the current summary by
excluding the messages whose subjects don't match the specified
regexp.

This would make it possible to check for a conjunction of conditions.
And membership in a thread, if you're added that, could be one of them.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Summary by thread in rmail

[Andrea Monaco]

As I've said, my current patch shows a summary of _all_ mail by thread,
but it deduces threads based on subject alone.  I can add detection of
other fields such as In-reply-to and References, as many people seem to
prefer it.


  > This would make it possible to check for a conjunction of
  > conditions.  And membership in a thread, if you're added that, could
  > be one of them.

Applying consecutive filters is a different thing, but it seems a
natural improvement, so I'll take a look.



Andrea Monaco

Re: Summary by thread in rmail

[Emanuel Berg]

Andrea Monaco wrote:

> As I've said, my current patch shows a summary of _all_ mail
> by thread, but it deduces threads based on subject alone.
> I can add detection of other fields such as In-reply-to and
> References, as many people seem to prefer it.

Yes, because it is the actual thread :)

-- 
underground experts united
https://dataswamp.org/~incal

[PATCH] Allow applying filters to summary consecutively (was: Summary by thread in rmail)

[Andrea Monaco]

This patch adds optional argument keep-filtering to rmail-summary-by-*
commands.  When t, the filter works on the currently displayed summary.
I also wrote a command rmail-summary-negate to invert the current
selection of displayed messages.


Andrea Monaco



diff --git a/lisp/mail/rmailsum.el b/lisp/mail/rmailsum.el
index b959f45250..24cad60bda 100644
--- a/lisp/mail/rmailsum.el
+++ b/lisp/mail/rmailsum.el
@@ -50,6 +50,13 @@ rmail-summary-line-count-flag
   :type 'boolean
   :group 'rmail-summary)
 
+(defvar rmail-summary-currently-displayed-msgs nil
+  "String made of 'y' and 'n'.  At index i it tells wether
+message i is shown on the summary or not.  First character is
+ignored.  Used when applying rmail-summary-by-* commands
+consecutively.")
+(put 'rmail-summary-currently-displayed-msgs 'permanent-local t)
+
 (defvar rmail-summary-font-lock-keywords
   '(("^ *[0-9]+D.*" . font-lock-string-face)			; Deleted.
     ("^ *[0-9]+-.*" . font-lock-type-face)			; Unread.
@@ -267,6 +274,33 @@ rmail-summary-mode-map
 (defun rmail-update-summary (&rest _)
   (apply (car rmail-summary-redo) (cdr rmail-summary-redo)))
 
+(defun rmail-summary-fill-displayed-messages ()
+  "Fill the rmail-summary-currently-displayed-msgs string."
+  (with-current-buffer rmail-buffer
+    (with-current-buffer rmail-summary-buffer
+      (setq rmail-summary-currently-displayed-msgs
+	    (make-string (1+ rmail-total-messages) ?n))
+      (goto-char (point-min))
+      (while (not (eobp))
+	(aset rmail-summary-currently-displayed-msgs
+	      (string-to-number (thing-at-point 'line))
+	      ?y)
+	(forward-line 1)))))
+
+(defun rmail-summary-negate ()
+  "Invert the current set of displayed messages in the summary."
+  (interactive)
+  (rmail-summary-fill-displayed-messages)
+  (rmail-new-summary "Negate"
+		     '(rmail-summary-by-regexp ".*" t)
+		     (lambda (msg)
+		       (if
+			   (char-equal (aref rmail-summary-currently-displayed-msgs msg)
+				       ?n)
+			   (progn
+			     (aset rmail-summary-currently-displayed-msgs msg ?y) t)
+			 (progn
+			   (aset rmail-summary-currently-displayed-msgs msg ?n) nil)))))
 ;;;###autoload
 (defun rmail-summary ()
   "Display a summary of all messages, one line per message."
@@ -274,7 +308,7 @@ rmail-summary
   (rmail-new-summary "All" '(rmail-summary) nil))
 
 ;;;###autoload
-(defun rmail-summary-by-labels (labels)
+(defun rmail-summary-by-labels (labels &optional keep-filtering)
   "Display a summary of all messages with one or more LABELS.
 LABELS should be a string containing the desired labels, separated by commas."
   (interactive "sLabels to summarize by: ")
@@ -282,25 +316,40 @@ rmail-summary-by-labels
       (setq labels (or rmail-last-multi-labels
 		       (error "No label specified"))))
   (setq rmail-last-multi-labels labels)
+  (if keep-filtering
+      (rmail-summary-fill-displayed-messages))
   (rmail-new-summary (concat "labels " labels)
-		     (list 'rmail-summary-by-labels labels)
-		     'rmail-message-labels-p
+		     (list 'rmail-summary-by-labels labels keep-filtering)
+		     (if keep-filtering
+			 (lambda (msg l)
+			   (and (char-equal (aref rmail-summary-currently-displayed-msgs msg)
+					    ?y)
+				(rmail-message-labels-p msg l)))
+		       'rmail-message-labels-p)
 		     (concat " \\("
 			     (mail-comma-list-regexp labels)
 			     "\\)\\(,\\|\\'\\)")))
 
 ;;;###autoload
-(defun rmail-summary-by-recipients (recipients &optional primary-only)
+(defun rmail-summary-by-recipients (recipients &optional primary-only keep-filtering)
   "Display a summary of all messages with the given RECIPIENTS.
 Normally checks the To, From and Cc fields of headers;
 but if PRIMARY-ONLY is non-nil (prefix arg given),
  only look in the To and From fields.
 RECIPIENTS is a regular expression."
   (interactive "sRecipients to summarize by: \nP")
+  (if keep-filtering
+      (rmail-summary-fill-displayed-messages))
   (rmail-new-summary
    (concat "recipients " recipients)
-   (list 'rmail-summary-by-recipients recipients primary-only)
-   'rmail-message-recipients-p recipients primary-only))
+   (list 'rmail-summary-by-recipients recipients primary-only keep-filtering)
+   (if keep-filtering
+       (lambda (msg r)
+	 (and (char-equal (aref rmail-summary-currently-displayed-msgs msg)
+			  ?y)
+	      (rmail-message-recipients-p msg r)))
+     'rmail-message-recipients-p)
+   recipients primary-only))
 
 (defun rmail-message-recipients-p (msg recipients &optional primary-only)
   (rmail-apply-in-message msg 'rmail-message-recipients-p-1
@@ -318,7 +367,7 @@ rmail-message-recipients-p-1
 ;; Also, the optional WHOLE-MESSAGE argument of r-s-by-topic would
 ;; seem more natural here.
 ;;;###autoload
-(defun rmail-summary-by-regexp (regexp)
+(defun rmail-summary-by-regexp (regexp &optional keep-filtering)
   "Display a summary of all messages according to regexp REGEXP.
 If the regular expression is found in the header of the message
 \(including in the date and other lines, as well as the subject line),
@@ -328,9 +377,16 @@ rmail-summary-by-regexp
       (setq regexp (or rmail-last-regexp
 			 (error "No regexp specified"))))
   (setq rmail-last-regexp regexp)
+  (if keep-filtering
+      (rmail-summary-fill-displayed-messages))
   (rmail-new-summary (concat "regexp " regexp)
-		     (list 'rmail-summary-by-regexp regexp)
-		     'rmail-message-regexp-p
+		     (list 'rmail-summary-by-regexp regexp keep-filtering)
+		     (if keep-filtering
+			 (lambda (msg r)
+			   (and (char-equal (aref rmail-summary-currently-displayed-msgs msg)
+					    ?y)
+				(rmail-message-regexp-p msg r)))
+		       'rmail-message-regexp-p)
                      regexp))
 
 (defun rmail-message-regexp-p (msg regexp)
@@ -363,7 +419,7 @@ rmail-message-regexp-p-1
     (rmail--decode-and-apply 're-search-forward regexp nil t)))
 
 ;;;###autoload
-(defun rmail-summary-by-topic (subject &optional whole-message)
+(defun rmail-summary-by-topic (subject &optional whole-message keep-filtering)
   "Display a summary of all messages with the given SUBJECT.
 Normally checks just the Subject field of headers; but with prefix
 argument WHOLE-MESSAGE is non-nil, looks in the whole message.
@@ -376,10 +432,18 @@ rmail-summary-by-topic
 			  (if subject ", default current subject" "")
 			  "): ")))
      (list (read-string prompt nil nil subject) current-prefix-arg)))
+  (if keep-filtering
+      (rmail-summary-fill-displayed-messages))
   (rmail-new-summary
    (concat "about " subject)
-   (list 'rmail-summary-by-topic subject whole-message)
-   'rmail-message-subject-p subject whole-message))
+   (list 'rmail-summary-by-topic subject whole-message keep-filtering)
+   (if keep-filtering
+       (lambda (msg s wo)
+	 (and (char-equal (aref rmail-summary-currently-displayed-msgs msg)
+			  ?y)
+	      (rmail-message-subject-p msg s wo)))
+     'rmail-message-subject-p)
+   subject whole-message))
 
 (defun rmail-message-subject-p (msg subject &optional whole-message)
   (if whole-message
@@ -389,7 +453,7 @@ rmail-message-subject-p
     (string-match subject (rmail-simplified-subject msg))))
 
 ;;;###autoload
-(defun rmail-summary-by-senders (senders)
+(defun rmail-summary-by-senders (senders &optional keep-filtering)
   "Display a summary of all messages whose \"From\" field matches SENDERS.
 SENDERS is a regular expression.  The default for SENDERS matches the
 sender of the current message."
@@ -402,9 +466,18 @@ rmail-summary-by-senders
 			  (if sender ", default this message's sender" "")
 			  "): ")))
      (list (read-string prompt nil nil sender))))
+  (if keep-filtering
+      (rmail-summary-fill-displayed-messages))
   (rmail-new-summary
    (concat "senders " senders)
-   (list 'rmail-summary-by-senders senders) 'rmail-message-senders-p senders))
+   (list 'rmail-summary-by-senders senders keep-filtering)
+   (if keep-filtering
+       (lambda (msg s)
+	 (and (char-equal (aref rmail-summary-currently-displayed-msgs msg)
+			  ?y)
+	      (rmail-message-senders-p msg s)))
+     'rmail-message-senders-p)
+   senders))
 
 (defun rmail-message-senders-p (msg senders)
   (string-match senders (or (rmail-get-header "From" msg) "")))

Re: [PATCH] Allow applying filters to summary consecutively (was: Summary by thread in rmail)

[Eli Zaretskii]

> From: Andrea Monaco <andrea.monaco@autistici.org>
> Cc: rms@gnu.org
> Date: Sun, 09 Oct 2022 22:33:01 +0200
> 
> This patch adds optional argument keep-filtering to rmail-summary-by-*
> commands.  When t, the filter works on the currently displayed summary.

Thanks.  But the rmail-summary-* are commands, not just functions.
How can an Rmail user invoke these commands to work only on the
messages in the currently-displayed summary?  Without some user
ability to take advantage of these new options, the feature is less
useful, IMO.

> +(defun rmail-summary-negate ()
> +  "Invert the current set of displayed messages in the summary."

The doc string should explain what is meant by "negation" here.

> +  (interactive)
> +  (rmail-summary-fill-displayed-messages)
> +  (rmail-new-summary "Negate"

I think "Negate" is not the best description of the summary being
produced.  How about "NOT" instead?  It should produce mode-line
display such as

   RMAIL Summary: NOT regexp ....

> +		     (lambda (msg)
> +		       (if
> +			   (char-equal (aref rmail-summary-currently-displayed-msgs msg)
> +				       ?n)

Shouldn't a simple '=' be enough here, instead of char-equal (here and
elsewhere in the patch)?  These y/n markers are completely internal,
AFAIU, so all the additional features of char-equal, such as
case-non-sensitivity, should not be relevant.

> -(defun rmail-summary-by-labels (labels)
> +(defun rmail-summary-by-labels (labels &optional keep-filtering)
>    "Display a summary of all messages with one or more LABELS.
>  LABELS should be a string containing the desired labels, separated by commas."

Please add the description of the new optional argument to the doc
strings of all the functions you are modifying.

Re: [PATCH] Allow applying filters to summary consecutively (was: Summary by thread in rmail)

[Andrea Monaco]

  > Without some user ability to take advantage of these new options,
  > the feature is less useful, IMO.

I agree.  One possible solution is that for every rmail-summary-by-*
function, let's say rmail-summary-by-topic, we add another simple
command called rmail-summary-further-by-topic that looks something like
this:

(defun rmail-summary-further-by-topic (subject &optional whole-message)
  (interactive)
  (rmail-summary-by-topic subject whole-message t))


I agree on the other objections too.



Andrea Monaco

Re: [PATCH] Allow applying filters to summary consecutively (was: Summary by thread in rmail)

[Eli Zaretskii]

> From: Andrea Monaco <andrea.monaco@autistici.org>
> Cc: emacs-devel@gnu.org, rms@gnu.org
> Date: Mon, 10 Oct 2022 10:38:19 +0200
> 
> 
>   > Without some user ability to take advantage of these new options,
>   > the feature is less useful, IMO.
> 
> I agree.  One possible solution is that for every rmail-summary-by-*
> function, let's say rmail-summary-by-topic, we add another simple
> command called rmail-summary-further-by-topic that looks something like
> this:
> 
> (defun rmail-summary-further-by-topic (subject &optional whole-message)
>   (interactive)
>   (rmail-summary-by-topic subject whole-message t))

I think it could be better to use the prefix argument.  AFAICT, only
one of the Rmail's summary commands already uses the prefix argument,
the rest don't.

Re: [PATCH] Allow applying filters to summary consecutively (was: Summary by thread in rmail)

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > This patch adds optional argument keep-filtering to rmail-summary-by-*
  > commands.  When t, the filter works on the currently displayed summary.
  > I also wrote a command rmail-summary-negate to invert the current
  > selection of displayed messages.

Thank you!

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v2] Allow applying filters to summary consecutively (was: Summary by thread in rmail)

[Andrea Monaco]

rmail-summary-by-recipients and rmail-summary-by-topic already use the
prefix argument for every non-nil value, so I don't know how to proceed.
I applied all other changes.


Andrea Monaco



diff --git a/lisp/mail/rmailsum.el b/lisp/mail/rmailsum.el
index b959f45250..5e51a8cd20 100644
--- a/lisp/mail/rmailsum.el
+++ b/lisp/mail/rmailsum.el
@@ -50,6 +50,13 @@ rmail-summary-line-count-flag
   :type 'boolean
   :group 'rmail-summary)
 
+(defvar rmail-summary-currently-displayed-msgs nil
+  "String made of 'y' and 'n'.  At index i it tells wether
+message i is shown on the summary or not.  First character is
+ignored.  Used when applying rmail-summary-by-* commands
+consecutively.")
+(put 'rmail-summary-currently-displayed-msgs 'permanent-local t)
+
 (defvar rmail-summary-font-lock-keywords
   '(("^ *[0-9]+D.*" . font-lock-string-face)			; Deleted.
     ("^ *[0-9]+-.*" . font-lock-type-face)			; Unread.
@@ -267,6 +274,35 @@ rmail-summary-mode-map
 (defun rmail-update-summary (&rest _)
   (apply (car rmail-summary-redo) (cdr rmail-summary-redo)))
 
+(defun rmail-summary-fill-displayed-messages ()
+  "Fill the rmail-summary-currently-displayed-msgs string."
+  (with-current-buffer rmail-buffer
+    (with-current-buffer rmail-summary-buffer
+      (setq rmail-summary-currently-displayed-msgs
+	    (make-string (1+ rmail-total-messages) ?n))
+      (goto-char (point-min))
+      (while (not (eobp))
+	(aset rmail-summary-currently-displayed-msgs
+	      (string-to-number (thing-at-point 'line))
+	      ?y)
+	(forward-line 1)))))
+
+(defun rmail-summary-negate ()
+  "Negate the current summary.  That is, show the messages that
+are not displayed, and vice versa."
+  (interactive)
+  (rmail-summary-fill-displayed-messages)
+  (rmail-new-summary "Negate"
+		     '(rmail-summary-by-regexp ".*" t)
+		     (lambda (msg)
+		       (if
+			   (= (aref rmail-summary-currently-displayed-msgs msg)
+			      ?n)
+			   (progn
+			     (aset rmail-summary-currently-displayed-msgs msg ?y) t)
+			 (progn
+			   (aset rmail-summary-currently-displayed-msgs msg ?n) nil)))))
+
 ;;;###autoload
 (defun rmail-summary ()
   "Display a summary of all messages, one line per message."
@@ -274,33 +310,52 @@ rmail-summary
   (rmail-new-summary "All" '(rmail-summary) nil))
 
 ;;;###autoload
-(defun rmail-summary-by-labels (labels)
+(defun rmail-summary-by-labels (labels &optional keep-filtering)
   "Display a summary of all messages with one or more LABELS.
-LABELS should be a string containing the desired labels, separated by commas."
-  (interactive "sLabels to summarize by: ")
+LABELS should be a string containing the desired labels, separated by commas.
+  If KEEP-FILTERING is non-nil or prefix argument is given, operate on the
+current summary instead of all mail."
+  (interactive "sLabels to summarize by: \nP")
   (if (string= labels "")
       (setq labels (or rmail-last-multi-labels
 		       (error "No label specified"))))
   (setq rmail-last-multi-labels labels)
+  (if keep-filtering
+      (rmail-summary-fill-displayed-messages))
   (rmail-new-summary (concat "labels " labels)
-		     (list 'rmail-summary-by-labels labels)
-		     'rmail-message-labels-p
+		     (list 'rmail-summary-by-labels labels keep-filtering)
+		     (if keep-filtering
+			 (lambda (msg l)
+			   (and (= (aref rmail-summary-currently-displayed-msgs msg)
+				   ?y)
+				(rmail-message-labels-p msg l)))
+		       'rmail-message-labels-p)
 		     (concat " \\("
 			     (mail-comma-list-regexp labels)
 			     "\\)\\(,\\|\\'\\)")))
 
 ;;;###autoload
-(defun rmail-summary-by-recipients (recipients &optional primary-only)
+(defun rmail-summary-by-recipients (recipients &optional primary-only keep-filtering)
   "Display a summary of all messages with the given RECIPIENTS.
 Normally checks the To, From and Cc fields of headers;
 but if PRIMARY-ONLY is non-nil (prefix arg given),
  only look in the To and From fields.
-RECIPIENTS is a regular expression."
+RECIPIENTS is a regular expression.
+  If KEEP-FILTERING is non-nil, operate on the current summary
+instead of all mail."
   (interactive "sRecipients to summarize by: \nP")
+  (if keep-filtering
+      (rmail-summary-fill-displayed-messages))
   (rmail-new-summary
    (concat "recipients " recipients)
-   (list 'rmail-summary-by-recipients recipients primary-only)
-   'rmail-message-recipients-p recipients primary-only))
+   (list 'rmail-summary-by-recipients recipients primary-only keep-filtering)
+   (if keep-filtering
+       (lambda (msg r)
+	 (and (= (aref rmail-summary-currently-displayed-msgs msg)
+		 ?y)
+	      (rmail-message-recipients-p msg r)))
+     'rmail-message-recipients-p)
+   recipients primary-only))
 
 (defun rmail-message-recipients-p (msg recipients &optional primary-only)
   (rmail-apply-in-message msg 'rmail-message-recipients-p-1
@@ -318,19 +373,28 @@ rmail-message-recipients-p-1
 ;; Also, the optional WHOLE-MESSAGE argument of r-s-by-topic would
 ;; seem more natural here.
 ;;;###autoload
-(defun rmail-summary-by-regexp (regexp)
+(defun rmail-summary-by-regexp (regexp &optional keep-filtering)
   "Display a summary of all messages according to regexp REGEXP.
 If the regular expression is found in the header of the message
 \(including in the date and other lines, as well as the subject line),
-Emacs will list the message in the summary."
-  (interactive "sRegexp to summarize by: ")
+Emacs will list the message in the summary.
+  If KEEP-FILTERING is non-nil or prefix argument is given, operate
+on the current summary instead of all mail."
+  (interactive "sRegexp to summarize by: \nP")
   (if (string= regexp "")
       (setq regexp (or rmail-last-regexp
 			 (error "No regexp specified"))))
   (setq rmail-last-regexp regexp)
+  (if keep-filtering
+      (rmail-summary-fill-displayed-messages))
   (rmail-new-summary (concat "regexp " regexp)
-		     (list 'rmail-summary-by-regexp regexp)
-		     'rmail-message-regexp-p
+		     (list 'rmail-summary-by-regexp regexp keep-filtering)
+		     (if keep-filtering
+			 (lambda (msg r)
+			   (and (= (aref rmail-summary-currently-displayed-msgs msg)
+				   ?y)
+				(rmail-message-regexp-p msg r)))
+		       'rmail-message-regexp-p)
                      regexp))
 
 (defun rmail-message-regexp-p (msg regexp)
@@ -363,11 +427,13 @@ rmail-message-regexp-p-1
     (rmail--decode-and-apply 're-search-forward regexp nil t)))
 
 ;;;###autoload
-(defun rmail-summary-by-topic (subject &optional whole-message)
+(defun rmail-summary-by-topic (subject &optional whole-message keep-filtering)
   "Display a summary of all messages with the given SUBJECT.
-Normally checks just the Subject field of headers; but with prefix
+Normally checks just the Subject field of headers; but when prefix
 argument WHOLE-MESSAGE is non-nil, looks in the whole message.
-SUBJECT is a regular expression."
+SUBJECT is a regular expression.
+  If KEEP-FILTERING is non-nil, operate on the current summary
+instead of all mail."
   (interactive
    ;; We quote the default subject, because if it contains regexp
    ;; special characters (eg "?"), it can fail to match itself.  (Bug#2333)
@@ -376,10 +442,18 @@ rmail-summary-by-topic
 			  (if subject ", default current subject" "")
 			  "): ")))
      (list (read-string prompt nil nil subject) current-prefix-arg)))
+  (if keep-filtering
+      (rmail-summary-fill-displayed-messages))
   (rmail-new-summary
    (concat "about " subject)
-   (list 'rmail-summary-by-topic subject whole-message)
-   'rmail-message-subject-p subject whole-message))
+   (list 'rmail-summary-by-topic subject whole-message keep-filtering)
+   (if keep-filtering
+       (lambda (msg s wo)
+	 (and (= (aref rmail-summary-currently-displayed-msgs msg)
+		 ?y)
+	      (rmail-message-subject-p msg s wo)))
+     'rmail-message-subject-p)
+   subject whole-message))
 
 (defun rmail-message-subject-p (msg subject &optional whole-message)
   (if whole-message
@@ -389,10 +463,12 @@ rmail-message-subject-p
     (string-match subject (rmail-simplified-subject msg))))
 
 ;;;###autoload
-(defun rmail-summary-by-senders (senders)
+(defun rmail-summary-by-senders (senders &optional keep-filtering)
   "Display a summary of all messages whose \"From\" field matches SENDERS.
 SENDERS is a regular expression.  The default for SENDERS matches the
-sender of the current message."
+sender of the current message.
+  If KEEP-FILTERING is non-nil or prefix argument is given, operate on
+the current summary instead of all mail."
   (interactive
    (let* ((def (rmail-get-header "From"))
           ;; We quote the default argument, because if it contains regexp
@@ -401,10 +477,20 @@ rmail-summary-by-senders
 	  (prompt (concat "Senders to summarize by (regexp"
 			  (if sender ", default this message's sender" "")
 			  "): ")))
-     (list (read-string prompt nil nil sender))))
+     (list (read-string prompt nil nil sender)
+	   current-prefix-arg)))
+  (if keep-filtering
+      (rmail-summary-fill-displayed-messages))
   (rmail-new-summary
    (concat "senders " senders)
-   (list 'rmail-summary-by-senders senders) 'rmail-message-senders-p senders))
+   (list 'rmail-summary-by-senders senders keep-filtering)
+   (if keep-filtering
+       (lambda (msg s)
+	 (and (= (aref rmail-summary-currently-displayed-msgs msg)
+		 ?y)
+	      (rmail-message-senders-p msg s)))
+     'rmail-message-senders-p)
+   senders))
 
 (defun rmail-message-senders-p (msg senders)
   (string-match senders (or (rmail-get-header "From" msg) "")))

Re: [PATCH v2] Allow applying filters to summary consecutively

[Robert Pluim]

>>>>> On Tue, 11 Oct 2022 09:16:52 +0200, Andrea Monaco <andrea.monaco@autistici.org> said:

    Andrea> rmail-summary-by-recipients and rmail-summary-by-topic already use the
    Andrea> prefix argument for every non-nil value, so I don't know how to proceed.
    Andrea> I applied all other changes.

The way this normally works in eg `list-packages' or gnus summary
buffers is that filtering automatically stacks: filter by x, filter by
y applies to previous results, etc, and then thereʼs a command to pop
the filtering stack.

Regards

Robert
-- 

Re: [PATCH v2] Allow applying filters to summary consecutively

[Andrea Monaco]

  > The way this normally works in eg `list-packages' or gnus summary
  > buffers is that filtering automatically stacks: filter by x, filter
  > by y applies to previous results, etc, and then thereʼs a command to
  > pop the filtering stack.


That seems very reasonable to me, but it would change current default
behavior, which is that every filter starts from scratch.  I don't know
if that could be a problem for some people.



Andrea Monaco

Re: [PATCH v2] Allow applying filters to summary consecutively

[Robert Pluim]

>>>>> On Wed, 12 Oct 2022 11:35:24 +0200, Andrea Monaco <andrea.monaco@autistici.org> said:

    >> The way this normally works in eg `list-packages' or gnus summary
    >> buffers is that filtering automatically stacks: filter by x, filter
    >> by y applies to previous results, etc, and then thereʼs a command to
    >> pop the filtering stack.


    Andrea> That seems very reasonable to me, but it would change current default
    Andrea> behavior, which is that every filter starts from scratch.  I don't know
    Andrea> if that could be a problem for some people.

This is what user options were invented for :-)

Robert
-- 

Re: [PATCH v2] Allow applying filters to summary consecutively

[Eli Zaretskii]

> From: Andrea Monaco <andrea.monaco@autistici.org>
> Cc: eliz@gnu.org,  emacs-devel@gnu.org,  rms@gnu.org
> Date: Wed, 12 Oct 2022 11:35:24 +0200
> 
> 
>   > The way this normally works in eg `list-packages' or gnus summary
>   > buffers is that filtering automatically stacks: filter by x, filter
>   > by y applies to previous results, etc, and then thereʼs a command to
>   > pop the filtering stack.
> 
> 
> That seems very reasonable to me, but it would change current default
> behavior, which is that every filter starts from scratch.  I don't know
> if that could be a problem for some people.

I'd rather not make such drastic changes in the behavior of Rmail.
Most people who use it are likely to be on the conservative side.

Re: [PATCH v2] Allow applying filters to summary consecutively

[Andrea Monaco]

  > I'd rather not make such drastic changes in the behavior of Rmail.
  > Most people who use it are likely to be on the conservative side.

We could make a new global variable called something like
rmail-summary-apply-filters-consecutively.  When it is non-nil, the new
behavior applies.  That shouldn't break anything.



Andrea Monaco

Re: [PATCH v2] Allow applying filters to summary consecutively

[Eli Zaretskii]

> From: Andrea Monaco <andrea.monaco@autistici.org>
> Cc: rpluim@gmail.com, emacs-devel@gnu.org, rms@gnu.org
> Date: Wed, 12 Oct 2022 20:13:22 +0200
> 
> 
>   > I'd rather not make such drastic changes in the behavior of Rmail.
>   > Most people who use it are likely to be on the conservative side.
> 
> We could make a new global variable called something like
> rmail-summary-apply-filters-consecutively.  When it is non-nil, the new
> behavior applies.  That shouldn't break anything.

Yes, but since no one has yet requested such a behavior, why introduce
yet another knob?

Re: [PATCH v2] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > That seems very reasonable to me, but it would change current default
  > behavior, which is that every filter starts from scratch.

Please don't change that default behavior!

It is ok to have user options to specify some other behavior.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Andrea Monaco]

I introduced a configuration item called
rmail-summary-apply-filters-consecutively.  When set to non-nil, the
filters stack.  I also removed the KEEP-FILTERING argument that I
previously added, because it is no longer needed now that there's a
global switch.


Andrea Monaco



diff --git a/lisp/mail/rmailsum.el b/lisp/mail/rmailsum.el
index b959f45250..7f000ea014 100644
--- a/lisp/mail/rmailsum.el
+++ b/lisp/mail/rmailsum.el
@@ -50,6 +50,20 @@ rmail-summary-line-count-flag
   :type 'boolean
   :group 'rmail-summary)
 
+(defcustom rmail-summary-apply-filters-consecutively nil
+  "Non-nil means that commands rmail-summary-by-* works on the
+current summary and so can be stacked one after the other."
+  :type 'boolean
+  :group 'rmail-summary)
+
+(defvar rmail-summary-currently-displayed-msgs nil
+  "String made of 'y' and 'n'.  At index i it tells wether
+message i is shown on the summary or not.  First character is
+ignored.  Used when applying rmail-summary-by-* commands
+consecutively.  Filled by
+rmail-summary-fill-displayed-messages.")
+(put 'rmail-summary-currently-displayed-msgs 'permanent-local t)
+
 (defvar rmail-summary-font-lock-keywords
   '(("^ *[0-9]+D.*" . font-lock-string-face)			; Deleted.
     ("^ *[0-9]+-.*" . font-lock-type-face)			; Unread.
@@ -267,6 +281,35 @@ rmail-summary-mode-map
 (defun rmail-update-summary (&rest _)
   (apply (car rmail-summary-redo) (cdr rmail-summary-redo)))
 
+(defun rmail-summary-fill-displayed-messages ()
+  "Fill the rmail-summary-currently-displayed-msgs string."
+  (with-current-buffer rmail-buffer
+    (with-current-buffer rmail-summary-buffer
+      (setq rmail-summary-currently-displayed-msgs
+	    (make-string (1+ rmail-total-messages) ?n))
+      (goto-char (point-min))
+      (while (not (eobp))
+	(aset rmail-summary-currently-displayed-msgs
+	      (string-to-number (thing-at-point 'line))
+	      ?y)
+	(forward-line 1)))))
+
+(defun rmail-summary-negate ()
+  "Negate the current summary.  That is, show the messages that
+are not displayed, and vice versa."
+  (interactive)
+  (rmail-summary-fill-displayed-messages)
+  (rmail-new-summary "Negate"
+		     '(rmail-summary-by-regexp ".*")
+		     (lambda (msg)
+		       (if
+			   (= (aref rmail-summary-currently-displayed-msgs msg)
+			      ?n)
+			   (progn
+			     (aset rmail-summary-currently-displayed-msgs msg ?y) t)
+			 (progn
+			   (aset rmail-summary-currently-displayed-msgs msg ?n) nil)))))
+
 ;;;###autoload
 (defun rmail-summary ()
   "Display a summary of all messages, one line per message."
@@ -282,9 +325,16 @@ rmail-summary-by-labels
       (setq labels (or rmail-last-multi-labels
 		       (error "No label specified"))))
   (setq rmail-last-multi-labels labels)
+  (if rmail-summary-apply-filters-consecutively
+      (rmail-summary-fill-displayed-messages))
   (rmail-new-summary (concat "labels " labels)
 		     (list 'rmail-summary-by-labels labels)
-		     'rmail-message-labels-p
+		     (if rmail-summary-apply-filters-consecutively
+			 (lambda (msg l)
+			   (and (= (aref rmail-summary-currently-displayed-msgs msg)
+				   ?y)
+				(rmail-message-labels-p msg l)))
+		       'rmail-message-labels-p)
 		     (concat " \\("
 			     (mail-comma-list-regexp labels)
 			     "\\)\\(,\\|\\'\\)")))
@@ -297,10 +347,18 @@ rmail-summary-by-recipients
  only look in the To and From fields.
 RECIPIENTS is a regular expression."
   (interactive "sRecipients to summarize by: \nP")
+  (if rmail-summary-apply-filters-consecutively
+      (rmail-summary-fill-displayed-messages))
   (rmail-new-summary
    (concat "recipients " recipients)
    (list 'rmail-summary-by-recipients recipients primary-only)
-   'rmail-message-recipients-p recipients primary-only))
+   (if rmail-summary-apply-filters-consecutively
+       (lambda (msg r &optional po)
+	 (and (= (aref rmail-summary-currently-displayed-msgs msg)
+		 ?y)
+	      (rmail-message-recipients-p msg r po)))
+     'rmail-message-recipients-p)
+   recipients primary-only))
 
 (defun rmail-message-recipients-p (msg recipients &optional primary-only)
   (rmail-apply-in-message msg 'rmail-message-recipients-p-1
@@ -328,9 +386,16 @@ rmail-summary-by-regexp
       (setq regexp (or rmail-last-regexp
 			 (error "No regexp specified"))))
   (setq rmail-last-regexp regexp)
+  (if rmail-summary-apply-filters-consecutively
+      (rmail-summary-fill-displayed-messages))
   (rmail-new-summary (concat "regexp " regexp)
 		     (list 'rmail-summary-by-regexp regexp)
-		     'rmail-message-regexp-p
+		     (if rmail-summary-apply-filters-consecutively
+			 (lambda (msg r)
+			   (and (= (aref rmail-summary-currently-displayed-msgs msg)
+				   ?y)
+				(rmail-message-regexp-p msg r)))
+		       'rmail-message-regexp-p)
                      regexp))
 
 (defun rmail-message-regexp-p (msg regexp)
@@ -365,7 +430,7 @@ rmail-message-regexp-p-1
 ;;;###autoload
 (defun rmail-summary-by-topic (subject &optional whole-message)
   "Display a summary of all messages with the given SUBJECT.
-Normally checks just the Subject field of headers; but with prefix
+Normally checks just the Subject field of headers; but when prefix
 argument WHOLE-MESSAGE is non-nil, looks in the whole message.
 SUBJECT is a regular expression."
   (interactive
@@ -376,10 +441,18 @@ rmail-summary-by-topic
 			  (if subject ", default current subject" "")
 			  "): ")))
      (list (read-string prompt nil nil subject) current-prefix-arg)))
+  (if rmail-summary-apply-filters-consecutively
+      (rmail-summary-fill-displayed-messages))
   (rmail-new-summary
    (concat "about " subject)
    (list 'rmail-summary-by-topic subject whole-message)
-   'rmail-message-subject-p subject whole-message))
+   (if rmail-summary-apply-filters-consecutively
+       (lambda (msg s &optional wm)
+	 (and (= (aref rmail-summary-currently-displayed-msgs msg)
+		 ?y)
+	      (rmail-message-subject-p msg s wm)))
+     'rmail-message-subject-p)
+   subject whole-message))
 
 (defun rmail-message-subject-p (msg subject &optional whole-message)
   (if whole-message
@@ -402,9 +475,18 @@ rmail-summary-by-senders
 			  (if sender ", default this message's sender" "")
 			  "): ")))
      (list (read-string prompt nil nil sender))))
+  (if rmail-summary-apply-filters-consecutively
+      (rmail-summary-fill-displayed-messages))
   (rmail-new-summary
    (concat "senders " senders)
-   (list 'rmail-summary-by-senders senders) 'rmail-message-senders-p senders))
+   (list 'rmail-summary-by-senders senders)
+   (if rmail-summary-apply-filters-consecutively
+       (lambda (msg s)
+	 (and (= (aref rmail-summary-currently-displayed-msgs msg)
+		 ?y)
+	      (rmail-message-senders-p msg s)))
+     'rmail-message-senders-p)
+   senders))
 
 (defun rmail-message-senders-p (msg senders)
   (string-match senders (or (rmail-get-header "From" msg) "")))

Re: [PATCH v3] Allow applying filters to summary consecutively

[Robert Pluim]

Hi Andrea, some comments below

>>>>> On Wed, 19 Oct 2022 16:23:13 +0200, Andrea Monaco <andrea.monaco@autistici.org> said:

    Andrea> +(defcustom rmail-summary-apply-filters-consecutively nil
    Andrea> +  "Non-nil means that commands rmail-summary-by-* works
    Andrea> on the

Iʼd call it `rmail-summary-stack-filters' and say

"Whether rmail-summary-by-* commands stack on each other."

    Andrea> +current summary and so can be stacked one after the other."
    Andrea> +  :type 'boolean
    Andrea> +  :group 'rmail-summary)

This needs a :version "29.1"

    Andrea> +(defvar rmail-summary-currently-displayed-msgs nil
    Andrea> +  "String made of 'y' and 'n'.  At index i it tells
    Andrea> wether

"whether"

    Andrea> +message i is shown on the summary or not.  First character is
    Andrea> +ignored.  Used when applying rmail-summary-by-* commands
    Andrea> +consecutively.  Filled by
    Andrea> +rmail-summary-fill-displayed-messages.")
    Andrea> +(put 'rmail-summary-currently-displayed-msgs 'permanent-local t)
    Andrea> +

Itʼs a string that you treat as a vector, so why not just make it a
vector? Also 'fill' in Emacs normally means wrapping paragraphs of
text, so perhaps `rmail-summary-populate-displayed-messages' or
similar is better.

    Andrea>  (defvar rmail-summary-font-lock-keywords
    Andrea>    '(("^ *[0-9]+D.*" . font-lock-string-face)			; Deleted.
    Andrea>      ("^ *[0-9]+-.*" . font-lock-type-face)			; Unread.
    Andrea> @@ -267,6 +281,35 @@ rmail-summary-mode-map
    Andrea>  (defun rmail-update-summary (&rest _)
    Andrea>    (apply (car rmail-summary-redo) (cdr rmail-summary-redo)))
 
    Andrea> +(defun rmail-summary-fill-displayed-messages ()
    Andrea> +  "Fill the rmail-summary-currently-displayed-msgs string."
    Andrea> +  (with-current-buffer rmail-buffer
    Andrea> +    (with-current-buffer rmail-summary-buffer
    Andrea> +      (setq rmail-summary-currently-displayed-msgs
    Andrea> +	    (make-string (1+ rmail-total-messages) ?n))
    Andrea> +      (goto-char (point-min))
    Andrea> +      (while (not (eobp))
    Andrea> +	(aset rmail-summary-currently-displayed-msgs
    Andrea> +	      (string-to-number (thing-at-point 'line))
    Andrea> +	      ?y)
    Andrea> +	(forward-line 1)))))

I donʼt remember the details of rmailʼs summary line format, but I
think youʼd be better served with

(thing-at-point 'number)

(that also avoids the `string-to-number')

    Andrea> +
    Andrea> +(defun rmail-summary-negate ()
    Andrea> +  "Negate the current summary.  That is, show the messages that
    Andrea> +are not displayed, and vice versa."

The first line of the docstring should be a single complete sentence
on a single line.

    Andrea> +  (interactive)
    Andrea> +  (rmail-summary-fill-displayed-messages)
    Andrea> +  (rmail-new-summary "Negate"
    Andrea> +		     '(rmail-summary-by-regexp ".*")
    Andrea> +		     (lambda (msg)
    Andrea> +		       (if
    Andrea> +			   (= (aref rmail-summary-currently-displayed-msgs msg)
    Andrea> +			      ?n)
    Andrea> +			   (progn
    Andrea> +			     (aset rmail-summary-currently-displayed-msgs msg ?y) t)
    Andrea> +			 (progn
    Andrea> +			   (aset rmail-summary-currently-displayed-msgs msg ?n) nil)))))
    Andrea> +

If you switch `rmail-summary-currently-displayed-msgs' to a vector,
you can store `nil' and `t' there directly and avoid the prognʼs (aset
returns the new value)

    Andrea>  ;;;###autoload
    Andrea>  (defun rmail-summary ()
    Andrea>    "Display a summary of all messages, one line per message."
    Andrea> @@ -282,9 +325,16 @@ rmail-summary-by-labels
    Andrea>        (setq labels (or rmail-last-multi-labels
    Andrea>  		       (error "No label specified"))))
    Andrea>    (setq rmail-last-multi-labels labels)
    Andrea> +  (if rmail-summary-apply-filters-consecutively
    Andrea> +      (rmail-summary-fill-displayed-messages))
    Andrea>    (rmail-new-summary (concat "labels " labels)
    Andrea>  		     (list 'rmail-summary-by-labels labels)
    Andrea> -		     'rmail-message-labels-p
    Andrea> +		     (if rmail-summary-apply-filters-consecutively
    Andrea> +			 (lambda (msg l)
    Andrea> +			   (and (= (aref rmail-summary-currently-displayed-msgs msg)
    Andrea> +				   ?y)
    Andrea> +				(rmail-message-labels-p msg l)))
    Andrea> +		       'rmail-message-labels-p)
    Andrea>  		     (concat " \\("
    Andrea>  			     (mail-comma-list-regexp labels)
    Andrea>  			     "\\)\\(,\\|\\'\\)")))

I think you have this same lambda about 4 times in the code, perhaps
`defun' it (using "rmail-summary--" as a prefix to indicate that itʼs
intended for internal usage)

Robert
-- 

Re: [PATCH v3] Allow applying filters to summary consecutively

[Andrea Monaco]

  > Iʼd call it `rmail-summary-stack-filters' and say

The metaphor of a stack is not perfect, as we don't have any "pop", that
is there's no command to cancel the last filter.


  > I donʼt remember the details of rmailʼs summary line format, but I
  > think youʼd be better served with (thing-at-point 'number)

There is some padding with whitespace at the beginning of each line, so
you'd still need a jump command.


  > I think you have this same lambda about 4 times in the code, perhaps
  > `defun' it (using "rmail-summary--" as a prefix to indicate that
  > itʼs intended for internal usage)

The five lambdas I added (one for each rmail-summary-by-* command) are
similar but not identical.  Maybe I could write a general function, but
I find it too much of a hassle for a little line saving.


I agree on the other objections.



Andrea Monaco

Re: [PATCH v3] Allow applying filters to summary consecutively

[Robert Pluim]

>>>>> On Thu, 20 Oct 2022 17:45:56 +0200, Andrea Monaco <andrea.monaco@autistici.org> said:

    >> Iʼd call it `rmail-summary-stack-filters' and say

    Andrea> The metaphor of a stack is not perfect, as we don't have any "pop", that
    Andrea> is there's no command to cancel the last filter.

Ah. How are people expected to get back to showing all messages?

    >> I donʼt remember the details of rmailʼs summary line format, but I
    >> think youʼd be better served with (thing-at-point 'number)

    Andrea> There is some padding with whitespace at the beginning of each line, so
    Andrea> you'd still need a jump command.

Well, you could search forward from bol for a digit easily enough.

    >> I think you have this same lambda about 4 times in the code, perhaps
    >> `defun' it (using "rmail-summary--" as a prefix to indicate that
    >> itʼs intended for internal usage)

    Andrea> The five lambdas I added (one for each rmail-summary-by-* command) are
    Andrea> similar but not identical.  Maybe I could write a general function, but
    Andrea> I find it too much of a hassle for a little line saving.

OK

    Andrea> I agree on the other objections.

Suggestions, not objections. Nothing Iʼve said stops the changes from
going in.

Robert
-- 

Re: [PATCH v3] Allow applying filters to summary consecutively

[Andrea Monaco]

  > Ah. How are people expected to get back to showing all messages?

With rmail-summary, bound to h by default.  Adding the ability to undo
the last filter only is probably easy, but it's not in my patch.



Andrea Monaco

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I introduced a configuration item called
  > rmail-summary-apply-filters-consecutively.  When set to non-nil, the
  > filters stack.

Saying that they "stack" seems to imply you can push an additional
filter, and pop it off later.  Is that the case?  It could be a useful
feature.  But "stack" does NOT imply that the second filter refines
the first.

I think the word "intersect" is needed to explain this meaning.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >   > Ah. How are people expected to get back to showing all messages?

  > With rmail-summary, bound to h by default.

That wouldk be very inconvenient for me.  I never use the full summary
because it is slow.  My RMAIL file has over 3000 messages.  I only
make filtered summaries.

What happens if you make a filtered summary and then kill it with q,
then make a new filtered summary?  Does that apply the one filter
specified for the new summary?  Or does it remember the filters of
the summary that you killed?

The former would be convenient enough.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v2] Allow applying filters to summary consecutively (was: Summary by thread in rmail)

[Eli Zaretskii]

> From: Andrea Monaco <andrea.monaco@autistici.org>
> Cc: emacs-devel@gnu.org, rms@gnu.org
> Date: Tue, 11 Oct 2022 09:16:52 +0200
> 
> 
> rmail-summary-by-recipients and rmail-summary-by-topic already use the
> prefix argument for every non-nil value, so I don't know how to proceed.
> I applied all other changes.

Thanks, I installed this.

Please in the future accompany the patches with a ChangeLog-style
commit log message.  You can find the details in the CONTRIBUTE file
in the Emacs Git repository.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Eli Zaretskii]

> From: Andrea Monaco <andrea.monaco@autistici.org>
> Cc: rpluim@gmail.com, eliz@gnu.org, emacs-devel@gnu.org
> Date: Wed, 19 Oct 2022 16:23:13 +0200
> 
> 
> I introduced a configuration item called
> rmail-summary-apply-filters-consecutively.  When set to non-nil, the
> filters stack.  I also removed the KEEP-FILTERING argument that I
> previously added, because it is no longer needed now that there's a
> global switch.

Thanks, installed.

In the future, please don't change the Subject of the messages, not
even to indicate the version of the patch: it makes looking for the
related messages harder.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Andrea Monaco]

  > Thanks, installed.

Thanks, but that patch has a bug: it breaks if the user doesn't have a
summary already opened.  I'm working on a little fix.



Andrea Monaco

Re: [PATCH v3] Allow applying filters to summary consecutively

[Andrea Monaco]

This version should be final.  I change the name of the new
configuration item.  It will also work as expected when generating a
filtered summary with no existing summary.  Moreover, pressing q
(rmail-summary-quit) will kill the summary so the next filter starts
from scratch.

I add a function called rmail-summary--exists-1.  The existing
rmail-summary-exists is not enough and I didn't know if changing it
could break something.


Andrea Monaco



Fix application of consecutive filters in rmail.

* lisp/mail/rmailsum.el (rmail-summary-intersect-consecutive-filters):
    Change name.
    (rmail-summary-currently-displayed-msgs):
    Change to a bool-vector that is more natural.
    (rmail-summary-populate-displayed-messages): Change name.
    (rmail-summary--exists-1): New function.
    (rmail-summary-by-labels, rmail-summary-by-recipients)
    (rmail-summary-by-regexp, rmail-summary-by-topic)
    (rmail-summary-by-senders): Fix so they work correctly when they are
    invoked and no summary exists.



diff --git a/lisp/mail/rmailsum.el b/lisp/mail/rmailsum.el
index 0144a34e5e..b522449157 100644
--- a/lisp/mail/rmailsum.el
+++ b/lisp/mail/rmailsum.el
@@ -50,21 +50,18 @@ rmail-summary-line-count-flag
   :type 'boolean
   :group 'rmail-summary)
 
-(defcustom rmail-summary-apply-filters-consecutively nil
-  "If non-nil, Rmail summary commands apply filtering on top existing filtering.
-When this variable is non-nil, `rmail-summary-by-*' commands work on the
-current summary, and so their filtering can be stacked one on top of another.
-This allows gradual narrowing of the selection of the messages."
+(defcustom rmail-summary-intersect-consecutive-filters nil
+  "Non-nil means that commands rmail-summary-by-* works on the
+current summary and so can be intersected one after the other."
   :type 'boolean
   :version "29.1"
   :group 'rmail-summary)
 
 (defvar rmail-summary-currently-displayed-msgs nil
-  "String made of `y' and `n'.
-The character at position i tells wether message i is shown in the
-summary or not.  First character is ignored.
-Used when applying `rmail-summary-by-*' commands consecutively.  Filled
-by `rmail-summary-fill-displayed-messages'.")
+  "Boolean vector that for index i tells whether message i is
+shown on the summary or not.  First element is ignored.  Used
+when applying rmail-summary-by-* commands consecutively.  Filled
+by rmail-summary-populate-displayed-messages.")
 (put 'rmail-summary-currently-displayed-msgs 'permanent-local t)
 
 (defvar rmail-summary-font-lock-keywords
@@ -284,33 +281,39 @@ rmail-summary-mode-map
 (defun rmail-update-summary (&rest _)
   (apply (car rmail-summary-redo) (cdr rmail-summary-redo)))
 
-(defun rmail-summary-fill-displayed-messages ()
-  "Fill the rmail-summary-currently-displayed-msgs string."
+(defun rmail-summary-populate-displayed-messages ()
+  "Populate the rmail-summary-currently-displayed-msgs vector."
   (with-current-buffer rmail-buffer
-    (with-current-buffer rmail-summary-buffer
-      (setq rmail-summary-currently-displayed-msgs
-	    (make-string (1+ rmail-total-messages) ?n))
-      (goto-char (point-min))
-      (while (not (eobp))
-	(aset rmail-summary-currently-displayed-msgs
-	      (string-to-number (thing-at-point 'line))
-	      ?y)
-	(forward-line 1)))))
+    (let ((totmsgs rmail-total-messages))
+      (with-current-buffer rmail-summary-buffer
+	(setq rmail-summary-currently-displayed-msgs
+	      (make-bool-vector (1+ totmsgs) nil))
+	(goto-char (point-min))
+	(while (not (eobp))
+	  (aset rmail-summary-currently-displayed-msgs
+		(string-to-number (thing-at-point 'line))
+		t)
+	  (forward-line 1))))))
 
 (defun rmail-summary-negate ()
-  "Toggle display of messages that match the summary and those which do not."
+  "Negate the current summary.  That is, show the messages that
+are not displayed, and vice versa."
   (interactive)
-  (rmail-summary-fill-displayed-messages)
+  (rmail-summary-populate-displayed-messages)
   (rmail-new-summary "Negate"
 		     '(rmail-summary-by-regexp ".*")
 		     (lambda (msg)
 		       (if
-			   (= (aref rmail-summary-currently-displayed-msgs msg)
-			      ?n)
-			   (progn
-			     (aset rmail-summary-currently-displayed-msgs msg ?y) t)
-			 (progn
-			   (aset rmail-summary-currently-displayed-msgs msg ?n) nil)))))
+			   (eq (aref rmail-summary-currently-displayed-msgs msg)
+			       nil)
+			   (aset rmail-summary-currently-displayed-msgs msg t)
+			 (aset rmail-summary-currently-displayed-msgs msg nil)))))
+
+(defun rmail-summary--exists-1 ()
+  "Like rmail-summary-exists, but works in both main and summary buffers."
+  (with-current-buffer rmail-buffer
+    (and rmail-summary-buffer (buffer-name rmail-summary-buffer)
+	 rmail-summary-buffer)))
 
 ;;;###autoload
 (defun rmail-summary ()
@@ -327,14 +330,16 @@ rmail-summary-by-labels
       (setq labels (or rmail-last-multi-labels
 		       (error "No label specified"))))
   (setq rmail-last-multi-labels labels)
-  (if rmail-summary-apply-filters-consecutively
-      (rmail-summary-fill-displayed-messages))
+  (if (and rmail-summary-intersect-consecutive-filters
+	   (rmail-summary--exists-1))
+      (rmail-summary-populate-displayed-messages))
   (rmail-new-summary (concat "labels " labels)
 		     (list 'rmail-summary-by-labels labels)
-		     (if rmail-summary-apply-filters-consecutively
+		     (if (and rmail-summary-intersect-consecutive-filters
+			      (rmail-summary--exists-1))
 			 (lambda (msg l)
-			   (and (= (aref rmail-summary-currently-displayed-msgs msg)
-				   ?y)
+			   (and (eq (aref rmail-summary-currently-displayed-msgs msg)
+				    t)
 				(rmail-message-labels-p msg l)))
 		       'rmail-message-labels-p)
 		     (concat " \\("
@@ -349,15 +354,17 @@ rmail-summary-by-recipients
  only look in the To and From fields.
 RECIPIENTS is a regular expression."
   (interactive "sRecipients to summarize by: \nP")
-  (if rmail-summary-apply-filters-consecutively
-      (rmail-summary-fill-displayed-messages))
+  (if (and rmail-summary-intersect-consecutive-filters
+	   (rmail-summary--exists-1))
+      (rmail-summary-populate-displayed-messages))
   (rmail-new-summary
    (concat "recipients " recipients)
    (list 'rmail-summary-by-recipients recipients primary-only)
-   (if rmail-summary-apply-filters-consecutively
+   (if (and rmail-summary-intersect-consecutive-filters
+	    (rmail-summary--exists-1))
        (lambda (msg r &optional po)
-	 (and (= (aref rmail-summary-currently-displayed-msgs msg)
-		 ?y)
+	 (and (eq (aref rmail-summary-currently-displayed-msgs msg)
+		  t)
 	      (rmail-message-recipients-p msg r po)))
      'rmail-message-recipients-p)
    recipients primary-only))
@@ -388,14 +395,16 @@ rmail-summary-by-regexp
       (setq regexp (or rmail-last-regexp
 			 (error "No regexp specified"))))
   (setq rmail-last-regexp regexp)
-  (if rmail-summary-apply-filters-consecutively
-      (rmail-summary-fill-displayed-messages))
+  (if (and rmail-summary-intersect-consecutive-filters
+	   (rmail-summary--exists-1))
+      (rmail-summary-populate-displayed-messages))
   (rmail-new-summary (concat "regexp " regexp)
 		     (list 'rmail-summary-by-regexp regexp)
-		     (if rmail-summary-apply-filters-consecutively
+		     (if (and rmail-summary-intersect-consecutive-filters
+			      (rmail-summary--exists-1))
 			 (lambda (msg r)
-			   (and (= (aref rmail-summary-currently-displayed-msgs msg)
-				   ?y)
+			   (and (eq (aref rmail-summary-currently-displayed-msgs msg)
+				    t)
 				(rmail-message-regexp-p msg r)))
 		       'rmail-message-regexp-p)
                      regexp))
@@ -443,15 +452,17 @@ rmail-summary-by-topic
 			  (if subject ", default current subject" "")
 			  "): ")))
      (list (read-string prompt nil nil subject) current-prefix-arg)))
-  (if rmail-summary-apply-filters-consecutively
-      (rmail-summary-fill-displayed-messages))
+  (if (and rmail-summary-intersect-consecutive-filters
+	   (rmail-summary--exists-1))
+      (rmail-summary-populate-displayed-messages))
   (rmail-new-summary
    (concat "about " subject)
    (list 'rmail-summary-by-topic subject whole-message)
-   (if rmail-summary-apply-filters-consecutively
+   (if (and rmail-summary-intersect-consecutive-filters
+	    (rmail-summary--exists-1))
        (lambda (msg s &optional wm)
-	 (and (= (aref rmail-summary-currently-displayed-msgs msg)
-		 ?y)
+	 (and (eq (aref rmail-summary-currently-displayed-msgs msg)
+		  t)
 	      (rmail-message-subject-p msg s wm)))
      'rmail-message-subject-p)
    subject whole-message))
@@ -477,15 +488,17 @@ rmail-summary-by-senders
 			  (if sender ", default this message's sender" "")
 			  "): ")))
      (list (read-string prompt nil nil sender))))
-  (if rmail-summary-apply-filters-consecutively
-      (rmail-summary-fill-displayed-messages))
+  (if (and rmail-summary-intersect-consecutive-filters
+	   (rmail-summary--exists-1))
+      (rmail-summary-populate-displayed-messages))
   (rmail-new-summary
    (concat "senders " senders)
    (list 'rmail-summary-by-senders senders)
-   (if rmail-summary-apply-filters-consecutively
+   (if (and rmail-summary-intersect-consecutive-filters
+	    (rmail-summary--exists-1))
        (lambda (msg s)
-	 (and (= (aref rmail-summary-currently-displayed-msgs msg)
-		 ?y)
+	 (and (eq (aref rmail-summary-currently-displayed-msgs msg)
+		  t)
 	      (rmail-message-senders-p msg s)))
      'rmail-message-senders-p)
    senders))

Re: [PATCH v3] Allow applying filters to summary consecutively

[Eli Zaretskii]

> From: Andrea Monaco <andrea.monaco@autistici.org>
> Cc: rms@gnu.org, rpluim@gmail.com, emacs-devel@gnu.org
> Date: Fri, 28 Oct 2022 15:26:39 +0200
> 
> This version should be final.

Famous last words ;-)

Seriously, though: having to apply 3 patches one on top of the other
is something I'd like to avoid, so in the future if you don't feel the
patch is final, please say so when you post it, to prevent me from
doing unnecessary jobs.

I have several comments to the patch, mainly on the doc strings:

> -(defcustom rmail-summary-apply-filters-consecutively nil
> -  "If non-nil, Rmail summary commands apply filtering on top existing filtering.
> -When this variable is non-nil, `rmail-summary-by-*' commands work on the
> -current summary, and so their filtering can be stacked one on top of another.
> -This allows gradual narrowing of the selection of the messages."
> +(defcustom rmail-summary-intersect-consecutive-filters nil
> +  "Non-nil means that commands rmail-summary-by-* works on the
> +current summary and so can be intersected one after the other."

The first line of a doc string must be a complete sentence.  In the
original code, I made sure that was so, but your changes destroy that.

In addition, I don't think I understand what you mean by
"intersected", and neither will users who will read this doc string.
Please think of some better, clearer description, if you think the
original one was inaccurate or incorrect for some reason.

>  (defvar rmail-summary-currently-displayed-msgs nil
> -  "String made of `y' and `n'.
> -The character at position i tells wether message i is shown in the
> -summary or not.  First character is ignored.
> -Used when applying `rmail-summary-by-*' commands consecutively.  Filled
> -by `rmail-summary-fill-displayed-messages'.")
> +  "Boolean vector that for index i tells whether message i is
> +shown on the summary or not.  First element is ignored.  Used
> +when applying rmail-summary-by-* commands consecutively.  Filled
> +by rmail-summary-populate-displayed-messages.")

Same here: the first line must be a complete sentence.  Also,
references to variables and functions inside doc string should be
quoted `like this', to allow Help commands to create hyperlinks from
them.

> -(defun rmail-summary-fill-displayed-messages ()
> -  "Fill the rmail-summary-currently-displayed-msgs string."
> +(defun rmail-summary-populate-displayed-messages ()
> +  "Populate the rmail-summary-currently-displayed-msgs vector."

Quoting of variable names again.

>  (defun rmail-summary-negate ()
> -  "Toggle display of messages that match the summary and those which do not."
> +  "Negate the current summary.  That is, show the messages that
> +are not displayed, and vice versa."

The first line of a doc string must be a single complete sentence.

And I don't really understand why you changed the original doc
string.  Was it incorrect?  I think using "negate" is not a good idea,
since it's too technical and doesn't explain clearly enough what this
command does.  The original doc string attempted to clarify that in
simple terms.

> +(defun rmail-summary--exists-1 ()
> +  "Like rmail-summary-exists, but works in both main and summary buffers."

Please quote `like this' the function name mentioned in the doc
string.

Thanks.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Andrea Monaco]

The words "negate" and "intersect" come from logic and mathematics.
They might not be especially common, but they are very precise.  So I
kept them and expanded on their meaning in the rest of the docstrings.


Andrea Monaco



diff --git a/lisp/mail/rmailsum.el b/lisp/mail/rmailsum.el
index 0144a34e5e..011a0e81ac 100644
--- a/lisp/mail/rmailsum.el
+++ b/lisp/mail/rmailsum.el
@@ -50,21 +50,19 @@ rmail-summary-line-count-flag
   :type 'boolean
   :group 'rmail-summary)
 
-(defcustom rmail-summary-apply-filters-consecutively nil
-  "If non-nil, Rmail summary commands apply filtering on top existing filtering.
-When this variable is non-nil, `rmail-summary-by-*' commands work on the
-current summary, and so their filtering can be stacked one on top of another.
-This allows gradual narrowing of the selection of the messages."
+(defcustom rmail-summary-intersect-consecutive-filters nil
+  "Non-nil means that consecutive rmail-summary-by-* commands intersect.
+This way, the user can apply one filter after the other and keep
+narrowing the result."
   :type 'boolean
   :version "29.1"
   :group 'rmail-summary)
 
 (defvar rmail-summary-currently-displayed-msgs nil
-  "String made of `y' and `n'.
-The character at position i tells wether message i is shown in the
-summary or not.  First character is ignored.
-Used when applying `rmail-summary-by-*' commands consecutively.  Filled
-by `rmail-summary-fill-displayed-messages'.")
+  "Boolean vector that tells which messages are displayed in the summary.
+First element is ignored.  Used when applying rmail-summary-by-*
+commands consecutively.  Filled by
+`rmail-summary-populate-displayed-messages'.")
 (put 'rmail-summary-currently-displayed-msgs 'permanent-local t)
 
 (defvar rmail-summary-font-lock-keywords
@@ -284,33 +282,40 @@ rmail-summary-mode-map
 (defun rmail-update-summary (&rest _)
   (apply (car rmail-summary-redo) (cdr rmail-summary-redo)))
 
-(defun rmail-summary-fill-displayed-messages ()
-  "Fill the rmail-summary-currently-displayed-msgs string."
+(defun rmail-summary-populate-displayed-messages ()
+  "Populate the `rmail-summary-currently-displayed-msgs' vector."
   (with-current-buffer rmail-buffer
-    (with-current-buffer rmail-summary-buffer
-      (setq rmail-summary-currently-displayed-msgs
-	    (make-string (1+ rmail-total-messages) ?n))
-      (goto-char (point-min))
-      (while (not (eobp))
-	(aset rmail-summary-currently-displayed-msgs
-	      (string-to-number (thing-at-point 'line))
-	      ?y)
-	(forward-line 1)))))
+    (let ((totmsgs rmail-total-messages))
+      (with-current-buffer rmail-summary-buffer
+	(setq rmail-summary-currently-displayed-msgs
+	      (make-bool-vector (1+ totmsgs) nil))
+	(goto-char (point-min))
+	(while (not (eobp))
+	  (aset rmail-summary-currently-displayed-msgs
+		(string-to-number (thing-at-point 'line))
+		t)
+	  (forward-line 1))))))
 
 (defun rmail-summary-negate ()
-  "Toggle display of messages that match the summary and those which do not."
+  "Negate the current summary.
+That is, show the messages that are not displayed, and vice
+versa."
   (interactive)
-  (rmail-summary-fill-displayed-messages)
+  (rmail-summary-populate-displayed-messages)
   (rmail-new-summary "Negate"
 		     '(rmail-summary-by-regexp ".*")
 		     (lambda (msg)
 		       (if
-			   (= (aref rmail-summary-currently-displayed-msgs msg)
-			      ?n)
-			   (progn
-			     (aset rmail-summary-currently-displayed-msgs msg ?y) t)
-			 (progn
-			   (aset rmail-summary-currently-displayed-msgs msg ?n) nil)))))
+			   (eq (aref rmail-summary-currently-displayed-msgs msg)
+			       nil)
+			   (aset rmail-summary-currently-displayed-msgs msg t)
+			 (aset rmail-summary-currently-displayed-msgs msg nil)))))
+
+(defun rmail-summary--exists-1 ()
+  "Like `rmail-summary-exists', but works in both main and summary buffers."
+  (with-current-buffer rmail-buffer
+    (and rmail-summary-buffer (buffer-name rmail-summary-buffer)
+	 rmail-summary-buffer)))
 
 ;;;###autoload
 (defun rmail-summary ()
@@ -327,14 +332,16 @@ rmail-summary-by-labels
       (setq labels (or rmail-last-multi-labels
 		       (error "No label specified"))))
   (setq rmail-last-multi-labels labels)
-  (if rmail-summary-apply-filters-consecutively
-      (rmail-summary-fill-displayed-messages))
+  (if (and rmail-summary-intersect-consecutive-filters
+	   (rmail-summary--exists-1))
+      (rmail-summary-populate-displayed-messages))
   (rmail-new-summary (concat "labels " labels)
 		     (list 'rmail-summary-by-labels labels)
-		     (if rmail-summary-apply-filters-consecutively
+		     (if (and rmail-summary-intersect-consecutive-filters
+			      (rmail-summary--exists-1))
 			 (lambda (msg l)
-			   (and (= (aref rmail-summary-currently-displayed-msgs msg)
-				   ?y)
+			   (and (eq (aref rmail-summary-currently-displayed-msgs msg)
+				    t)
 				(rmail-message-labels-p msg l)))
 		       'rmail-message-labels-p)
 		     (concat " \\("
@@ -349,15 +356,17 @@ rmail-summary-by-recipients
  only look in the To and From fields.
 RECIPIENTS is a regular expression."
   (interactive "sRecipients to summarize by: \nP")
-  (if rmail-summary-apply-filters-consecutively
-      (rmail-summary-fill-displayed-messages))
+  (if (and rmail-summary-intersect-consecutive-filters
+	   (rmail-summary--exists-1))
+      (rmail-summary-populate-displayed-messages))
   (rmail-new-summary
    (concat "recipients " recipients)
    (list 'rmail-summary-by-recipients recipients primary-only)
-   (if rmail-summary-apply-filters-consecutively
+   (if (and rmail-summary-intersect-consecutive-filters
+	    (rmail-summary--exists-1))
        (lambda (msg r &optional po)
-	 (and (= (aref rmail-summary-currently-displayed-msgs msg)
-		 ?y)
+	 (and (eq (aref rmail-summary-currently-displayed-msgs msg)
+		  t)
 	      (rmail-message-recipients-p msg r po)))
      'rmail-message-recipients-p)
    recipients primary-only))
@@ -388,14 +397,16 @@ rmail-summary-by-regexp
       (setq regexp (or rmail-last-regexp
 			 (error "No regexp specified"))))
   (setq rmail-last-regexp regexp)
-  (if rmail-summary-apply-filters-consecutively
-      (rmail-summary-fill-displayed-messages))
+  (if (and rmail-summary-intersect-consecutive-filters
+	   (rmail-summary--exists-1))
+      (rmail-summary-populate-displayed-messages))
   (rmail-new-summary (concat "regexp " regexp)
 		     (list 'rmail-summary-by-regexp regexp)
-		     (if rmail-summary-apply-filters-consecutively
+		     (if (and rmail-summary-intersect-consecutive-filters
+			      (rmail-summary--exists-1))
 			 (lambda (msg r)
-			   (and (= (aref rmail-summary-currently-displayed-msgs msg)
-				   ?y)
+			   (and (eq (aref rmail-summary-currently-displayed-msgs msg)
+				    t)
 				(rmail-message-regexp-p msg r)))
 		       'rmail-message-regexp-p)
                      regexp))
@@ -443,15 +454,17 @@ rmail-summary-by-topic
 			  (if subject ", default current subject" "")
 			  "): ")))
      (list (read-string prompt nil nil subject) current-prefix-arg)))
-  (if rmail-summary-apply-filters-consecutively
-      (rmail-summary-fill-displayed-messages))
+  (if (and rmail-summary-intersect-consecutive-filters
+	   (rmail-summary--exists-1))
+      (rmail-summary-populate-displayed-messages))
   (rmail-new-summary
    (concat "about " subject)
    (list 'rmail-summary-by-topic subject whole-message)
-   (if rmail-summary-apply-filters-consecutively
+   (if (and rmail-summary-intersect-consecutive-filters
+	    (rmail-summary--exists-1))
        (lambda (msg s &optional wm)
-	 (and (= (aref rmail-summary-currently-displayed-msgs msg)
-		 ?y)
+	 (and (eq (aref rmail-summary-currently-displayed-msgs msg)
+		  t)
 	      (rmail-message-subject-p msg s wm)))
      'rmail-message-subject-p)
    subject whole-message))
@@ -477,15 +490,17 @@ rmail-summary-by-senders
 			  (if sender ", default this message's sender" "")
 			  "): ")))
      (list (read-string prompt nil nil sender))))
-  (if rmail-summary-apply-filters-consecutively
-      (rmail-summary-fill-displayed-messages))
+  (if (and rmail-summary-intersect-consecutive-filters
+	   (rmail-summary--exists-1))
+      (rmail-summary-populate-displayed-messages))
   (rmail-new-summary
    (concat "senders " senders)
    (list 'rmail-summary-by-senders senders)
-   (if rmail-summary-apply-filters-consecutively
+   (if (and rmail-summary-intersect-consecutive-filters
+	    (rmail-summary--exists-1))
        (lambda (msg s)
-	 (and (= (aref rmail-summary-currently-displayed-msgs msg)
-		 ?y)
+	 (and (eq (aref rmail-summary-currently-displayed-msgs msg)
+		  t)
 	      (rmail-message-senders-p msg s)))
      'rmail-message-senders-p)
    senders))

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

We were discussing what the behavior should be for intersecting
filters -- when to do that, when not to.  What is the behavior
that you have implemented in this version?

(That will be needed for documenting it.)

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Andrea Monaco]

  > What is the behavior that you have implemented in this version?

When rmail-summary-intersect-consecutive-filters is nil, the old
behavior applies.  When it is non-nil, filters intersect and pressing q
(rmail-summary-quit) cancels all filters.



Andrea Monaco

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > +  "Non-nil means that consecutive rmail-summary-by-* commands intersect.
  > +This way, the user can apply one filter after the other and keep
  > +narrowing the result."

The words "consecutive commands" mean that one immediately follows the
other, with no other commands in between.  Is that the way you've
implemented it?  I am not sure, but my guess is not.

Does it mean "using one summary filtering cpmmand when a filtered
summary already exists"?  That is the meaning I would expect in this
case.  But if so, "consecutive commands" is not the way to say it.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: eliz@gnu.org, rpluim@gmail.com, emacs-devel@gnu.org
> Date: Wed, 09 Nov 2022 23:04:02 -0500
> 
> Does it mean "using one summary filtering cpmmand when a filtered
> summary already exists"?  That is the meaning I would expect in this
> case.  But if so, "consecutive commands" is not the way to say it.

Your guess is correct.  I was struggling with finding a good
description for this.  What would you suggest as a good wording?

Re: [PATCH v3] Allow applying filters to summary consecutively

[Robert Pluim]

>>>>> On Thu, 10 Nov 2022 10:06:00 +0200, Eli Zaretskii <eliz@gnu.org> said:

    >> From: Richard Stallman <rms@gnu.org>
    >> Cc: eliz@gnu.org, rpluim@gmail.com, emacs-devel@gnu.org
    >> Date: Wed, 09 Nov 2022 23:04:02 -0500
    >> 
    >> Does it mean "using one summary filtering cpmmand when a filtered
    >> summary already exists"?  That is the meaning I would expect in this
    >> case.  But if so, "consecutive commands" is not the way to say it.

    Eli> Your guess is correct.  I was struggling with finding a good
    Eli> description for this.  What would you suggest as a good wording?

I think we can keep it down to one sentence if we follow the 'describe
what it affects' guidance for docstrings, which allows dropping the
'if non-nil' bit. So something like:

"Whether summary filtering commands apply to an existing summary.
If nil, a new filtered summary is created, if non-nil, subsequent
filtering commands apply to an existing filtered summary.  This allows
the user to apply one filter after the other and keep narrowing the
results."

(the name of the option may need to change)

Robert
-- 

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Your guess is correct.  I was struggling with finding a good
  > description for this.  What would you suggest as a good wording?

Well, how about "Using a summary filtering command when a filtered
summary already exists"?

It may not be the best possible wordin, but at least it's correct and
unambiguous.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: andrea.monaco@autistici.org, rpluim@gmail.com, emacs-devel@gnu.org
> Date: Thu, 10 Nov 2022 23:36:15 -0500
> 
>   > Your guess is correct.  I was struggling with finding a good
>   > description for this.  What would you suggest as a good wording?
> 
> Well, how about "Using a summary filtering command when a filtered
> summary already exists"?
> 
> It may not be the best possible wordin, but at least it's correct and
> unambiguous.

That's okay, thanks.  But it's only the beginning of my problem.  How
do you describe the effect of applying a filter on top of one or more
other filters?  Andrea wants to use "intersection", but I tend to
think this is too "mathematical" and too vague to explain clearly what
happens.  What would be a good terminology for that?

Re: [PATCH v3] Allow applying filters to summary consecutively

[Juanma Barranquero]

[-- Attachment #1: Type: text/plain, Size: 202 bytes --]

> How do you describe the effect of applying a filter on top of one or more
other filters?

Perhaps they "compose", using the verb in the meaning "to form (a whole) by
ordering or arranging the parts".

[-- Attachment #2: Type: text/html, Size: 471 bytes --]

Re: [PATCH v3] Allow applying filters to summary consecutively

[Eli Zaretskii]

> From: Juanma Barranquero <lekktu@gmail.com>
> Date: Fri, 11 Nov 2022 15:59:10 +0100
> Cc: rms@gnu.org, andrea.monaco@autistici.org, rpluim@gmail.com, 
>  emacs-devel@gnu.org
> 
> > How do you describe the effect of applying a filter on top of one or more other filters?
> 
> Perhaps they "compose", using the verb in the meaning "to form (a whole) by ordering or arranging the
> parts".

Thanks, but that's not what this feature does.

RE: [External] : Re: [PATCH v3] Allow applying filters to summary consecutively

[Drew Adams]

Not really following this thread, but ... HTH.

I think what's being described is what I call 
"progressive" filtering/narrowing.

Whether or not other operations are carried
out between two successive filtering
operations is, IMO, immaterial. "Consecutive"
application of filters doesn't imply absence
of other operations between filtering
operations (but someone might think that).
"Successive" is better than "consecutive" in
this regard: "consecutive" is "successive
without a break".

How filtering is done (pattern matching,
predicate application, something else) is
also immaterial.  You're just narrowing a
set of <whatevers> in some way, IIUC.

For Icicles "progressive completion", which
is the same kind of thing, I also describe
it as similar to `grep' chaining/cascading,
which involves piping the output of one
`grep' to the next one (possibly after
other operations that also emit a stream of
chars).  Not all Emacs users are programmers,
but many are familiar with piping `grep'.

And I do describe it as set intersection, as
well as ANDing (Boolean multiplication).
Key sequences for it include the use of key
`*' (or `&' when a predicate is used instead
of a pattern).  (For set union / ORing, keys 
include the use of `+', for Boolean addition.)

I don't think "intersect(ion)" is hard for
Emacs users (including nonprogrammers) to 
understand.  It's not only "mathematical",
and it's not esoteric.

Just one opinion.  HTH.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Gregory Heytings]

>> Well, how about "Using a summary filtering command when a filtered 
>> summary already exists"?
>>
>> It may not be the best possible wordin, but at least it's correct and 
>> unambiguous.
>
> That's okay, thanks.  But it's only the beginning of my problem.  How do 
> you describe the effect of applying a filter on top of one or more other 
> filters?  Andrea wants to use "intersection", but I tend to think this 
> is too "mathematical" and too vague to explain clearly what happens. 
> What would be a good terminology for that?
>

I think there are two possibilities when adding another filter: either 
broadening the already existing filtering ("union" in mathematical terms) 
or narrowing the already existing filtering ("intersection" in 
mathematical terms).

Re: [PATCH v3] Allow applying filters to summary consecutively

[Eli Zaretskii]

> Date: Fri, 11 Nov 2022 18:00:06 +0000
> From: Gregory Heytings <gregory@heytings.org>
> cc: rms@gnu.org, andrea.monaco@autistici.org, rpluim@gmail.com, 
>     emacs-devel@gnu.org
> 
> > That's okay, thanks.  But it's only the beginning of my problem.  How do 
> > you describe the effect of applying a filter on top of one or more other 
> > filters?  Andrea wants to use "intersection", but I tend to think this 
> > is too "mathematical" and too vague to explain clearly what happens. 
> > What would be a good terminology for that?
> 
> I think there are two possibilities when adding another filter: either 
> broadening the already existing filtering ("union" in mathematical terms) 
> or narrowing the already existing filtering ("intersection" in 
> mathematical terms).

It's what you call "intersection", obviously.

RE: [External] : Re: [PATCH v3] Allow applying filters to summary consecutively

[Drew Adams]

> I think there are two possibilities when adding another filter: either
> broadening the already existing filtering ("union" in mathematical terms)
> or narrowing the already existing filtering ("intersection" in
> mathematical terms).

I didn't realize that's what this is about.

If so, it's not "filtering".  Filtering
_subtracts_ from the input set - it filters
out zero or more members.  Filtering can't
_add_ to the input set.

A filter is a sieve: "a device that removes
something from whatever passes through it".

https://www.wordwebonline.com/search.pl?w=filter

If you have an operation that starts with a
(possibly empty) set and lets you add to it,
that can't reasonably be called "filtering".

For what I think you're describing, I use
the verbs "narrow" (for filtering - set
intersection) and "widen" (for set union).

In terms of predicates / Boolean, it's
AND (multiplication) and OR (addition),
respectively.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Gregory Heytings]

>>> That's okay, thanks.  But it's only the beginning of my problem.  How 
>>> do you describe the effect of applying a filter on top of one or more 
>>> other filters?  Andrea wants to use "intersection", but I tend to 
>>> think this is too "mathematical" and too vague to explain clearly what 
>>> happens. What would be a good terminology for that?
>>
>> I think there are two possibilities when adding another filter: either 
>> broadening the already existing filtering ("union" in mathematical 
>> terms) or narrowing the already existing filtering ("intersection" in 
>> mathematical terms).
>
> It's what you call "intersection", obviously.
>

Yes, my point was to mention the two less mathematical words: broadening 
and narrowing.

That being said, I don't user Rmail but I don't think it obvious that 
filtering is always "intersection".  Someone may filter their inbox to see 
mails from Alice, and add mails from Alice to the filtering.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Gregory Heytings]

>>>> That's okay, thanks.  But it's only the beginning of my problem. 
>>>> How do you describe the effect of applying a filter on top of one or 
>>>> more other filters?  Andrea wants to use "intersection", but I tend 
>>>> to think this is too "mathematical" and too vague to explain clearly 
>>>> what happens. What would be a good terminology for that?
>>> 
>>> I think there are two possibilities when adding another filter: either 
>>> broadening the already existing filtering ("union" in mathematical 
>>> terms) or narrowing the already existing filtering ("intersection" in 
>>> mathematical terms).
>> 
>> It's what you call "intersection", obviously.
>
> Yes, my point was to mention the two less mathematical words: broadening 
> and narrowing.
>
> That being said, I don't user Rmail but I don't think it obvious that 
> filtering is always "intersection".  Someone may filter their inbox to 
> see mails from Alice, and add mails from Alice to the filtering.
>

Too many typos, sorry: That being said, I don't use Rmail but I don't 
think it is obvious that filtering is always "intersection".  Someone may 
filter their inbox to see mails from Alice, and add mails from Bob to the 
filtering.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Eli Zaretskii]

> Date: Fri, 11 Nov 2022 18:52:28 +0000
> From: Gregory Heytings <gregory@heytings.org>
> cc: rms@gnu.org, andrea.monaco@autistici.org, rpluim@gmail.com, 
>     emacs-devel@gnu.org
> 
> Yes, my point was to mention the two less mathematical words: broadening 
> and narrowing.

Narrowing is already being used, but it needs explanation, being a
general notion.  I was looking for a single word or max 2 words,

> That being said, I don't user Rmail but I don't think it obvious that 
> filtering is always "intersection".  Someone may filter their inbox to see 
> mails from Alice, and add mails from Alice to the filtering.

You could read about it in the manual.  It doesn't make a lot of sense
to suggest terminology for features you are not familiar with.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Eli Zaretskii]

> Date: Fri, 11 Nov 2022 18:54:36 +0000
> From: Gregory Heytings <gregory@heytings.org>
> cc: rms@gnu.org, andrea.monaco@autistici.org, rpluim@gmail.com, 
>     emacs-devel@gnu.org
> 
> Too many typos, sorry: That being said, I don't use Rmail but I don't 
> think it is obvious that filtering is always "intersection".  Someone may 
> filter their inbox to see mails from Alice, and add mails from Bob to the 
> filtering.

It isn't called "filtering", and it is always "intersection".

Re: [PATCH v3] Allow applying filters to summary consecutively

[Gregory Heytings]

>> Yes, my point was to mention the two less mathematical words: 
>> broadening and narrowing.
>
> Narrowing is already being used, but it needs explanation, being a 
> general notion.  I was looking for a single word or max 2 words,
>

This subtread is about rmail-summary-apply-filters-consecutively, right? 
Here's an attempt to make it clearer:

If non-nil, apply Rmail summary commands on the already existing 
filtering.

When this variable is non-nil, `rmail-summary-by-*' commands work on the 
already existing summary, if any.  This allows gradual narrowing of the 
selection of messages, that is, each `rmail-summary-by-*' command 
restricts the selection of messages further.

When this variable is nil, each `rmail-summary-by-*' command is applied 
anew on the entire mailbox.

>> That being said, I don't user Rmail but I don't think it obvious that 
>> filtering is always "intersection".  Someone may filter their inbox to 
>> see mails from Alice, and add mails from Alice to the filtering.
>
> You could read about it in the manual.  It doesn't make a lot of sense 
> to suggest terminology for features you are not familiar with.
>

Well, the fact that I don't use Rmail doesn't mean that I'm not familiar 
with that feature, which exists in other MUAs.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > That's okay, thanks.  But it's only the beginning of my problem.  How
  > do you describe the effect of applying a filter on top of one or more
  > other filters?  Andrea wants to use "intersection", but I tend to
  > think this is too "mathematical" and too vague to explain clearly what
  > happens.  What would be a good terminology for that?

How about "filter further"?  "Filter down">  "Filter more strictly"?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Perhaps "serial filtering" is a good way to describe the new feature.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Eli Zaretskii]

> Date: Fri, 11 Nov 2022 20:50:10 +0000
> From: Gregory Heytings <gregory@heytings.org>
> cc: rms@gnu.org, andrea.monaco@autistici.org, rpluim@gmail.com, 
>     emacs-devel@gnu.org
> 
> 
> >> Yes, my point was to mention the two less mathematical words: 
> >> broadening and narrowing.
> >
> > Narrowing is already being used, but it needs explanation, being a 
> > general notion.  I was looking for a single word or max 2 words,
> >
> 
> This subtread is about rmail-summary-apply-filters-consecutively, right? 

No, it's about the name of the variable and the first sentence of its
doc string.  The rest is easy and not an issue.

> If non-nil, apply Rmail summary commands on the already existing 
> filtering.

"Filtering" is the wrong word here, since we are talking about Rmail
summaries.  Also, this is not about applying summary commands, this is
about generating the summary itself.

> >> That being said, I don't user Rmail but I don't think it obvious that 
> >> filtering is always "intersection".  Someone may filter their inbox to 
> >> see mails from Alice, and add mails from Alice to the filtering.
> >
> > You could read about it in the manual.  It doesn't make a lot of sense 
> > to suggest terminology for features you are not familiar with.
> >
> 
> Well, the fact that I don't use Rmail doesn't mean that I'm not familiar 
> with that feature, which exists in other MUAs.

If you think about this as "filtering", then you have different
features in mind, not the ones discussed here.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: andrea.monaco@autistici.org, rpluim@gmail.com, emacs-devel@gnu.org
> Date: Fri, 11 Nov 2022 22:35:54 -0500
> 
>   > That's okay, thanks.  But it's only the beginning of my problem.  How
>   > do you describe the effect of applying a filter on top of one or more
>   > other filters?  Andrea wants to use "intersection", but I tend to
>   > think this is too "mathematical" and too vague to explain clearly what
>   > happens.  What would be a good terminology for that?
> 
> How about "filter further"?  "Filter down">  "Filter more strictly"?

I have a fundamental problems with using "filtering" in this context.
Existing Rmail commands that create summaries don't mention filtering
at all.  Here's a typical example:

  rmail-summary-by-topic is an interactive native-compiled Lisp function
  in ‘rmailsum.el’.

  (rmail-summary-by-topic SUBJECT &optional WHOLE-MESSAGE)

  Display a summary of all messages with the given SUBJECT.
  Normally checks just the Subject field of headers; but with prefix
  argument WHOLE-MESSAGE is non-nil, looks in the whole message.
  SUBJECT is a regular expression.

So talking about "filtering" in this context will be "out of the
blue", unless we also change the doc strings of all rmail-summary-by-*
commands and the manual to talk about "filtering".

Moreover, "filtering" is somewhat wrong: the messages themselves
aren't "filtered": they aren't removed from the inbox.  They are just
omitted from the produced summary, and commands that move by summary
lines skip messages that are not in the summary.  So "filtering" here
is not real, it's imaginary, and the documentation will need to
explain that if we want to use that term.

Before we embark on such a massive documentation change, I'd like to
try to find a less invasive change of terminology, if that exists.  Do
you see a way to do that?

Basically, what this feature offers is a way to produce a summary from
messages that are already "summarized" by some criteria.  So I'd
prefer that our terminology alluded to this aspect rather than to
"filtering", because then it would be natural and won't require
significant changes in the documentation of what Rmail summary
commands do.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: eliz@gnu.org, andrea.monaco@autistici.org, rpluim@gmail.com,
> 	emacs-devel@gnu.org
> Date: Fri, 11 Nov 2022 22:37:42 -0500
> 
> Perhaps "serial filtering" is a good way to describe the new feature.

No, "serial" is too general, and doesn't catch the main trait of the
feature, which is that each additional "filter" narrows the selection.

How about "progressive summary"? or maybe "progressively narrowing
summary"?

Re: [PATCH v3] Allow applying filters to summary consecutively

[Gregory Heytings]

>> Well, the fact that I don't use Rmail doesn't mean that I'm not 
>> familiar with that feature, which exists in other MUAs.
>
> If you think about this as "filtering", then you have different features 
> in mind, not the ones discussed here.
>

I just tried Rmail, again I had a very positive impression, and again I 
regret that it works only with mbox files and not with Maildirs.

Anyway, unless I misunderstand something, the "summary" in Rmail is called 
the message "index" or "list" by other MUAs.  And the rmail-summary-by-* 
commands are called "filter", "select" (a subset of messages) or "limit" 
(the index to a subset of messages) by other MUAs.  Is "limit" perhaps a 
better word?

For example, rmail-summary-by-recipients would become "Display a summary 
limited to messages with the given RECIPIENTS."  This seems close to what 
the manual already says: "C-M-r makes a partial summary mentioning only 
the messages that have one or more recipients matching the regular 
expression rcpts".

And rmail-summary-apply-filters-consecutively would be 
rmail-summary-gradual-limiting or rmail-summary-incremental-limiting.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Andrea Monaco]

  > I just tried Rmail, again I had a very positive impression, and
  > again I regret that it works only with mbox files and not with
  > Maildirs.

I don't think that's too hard to implement, maybe I'll take a look.


About the customization variable name: maybe something like
rmail-summary-keep-narrowing or even
rmail-summary-new-filters-keep-narrowing?

Another option I proposed some time ago is to discard the global
variable entirely and make separate commands like
rmail-summary-further-by-*, for example
rmail-summary-further-by-recipients.



Andrea Monaco

Re: [PATCH v3] Allow applying filters to summary consecutively

[Gregory Heytings]

>> I just tried Rmail, again I had a very positive impression, and again I 
>> regret that it works only with mbox files and not with Maildirs.
>
> I don't think that's too hard to implement, maybe I'll take a look.
>

That would be great!

>
> Another option I proposed some time ago is to discard the global 
> variable entirely and make separate commands like 
> rmail-summary-further-by-*, for example 
> rmail-summary-further-by-recipients.
>

If I may, I would suggest something different: an option to query the user 
how they want the rmail-summary-by-* command to be applied: on the entire 
folder, to limit the existing summary, to extend the existing summary.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > When rmail-summary-intersect-consecutive-filters is nil, the old
  > behavior applies.  When it is non-nil, filters intersect and pressing q
  > (rmail-summary-quit) cancels all filters.

Thanks.  That's exactly what I would expect.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >   > I just tried Rmail, again I had a very positive impression, and
  >   > again I regret that it works only with mbox files and not with
  >   > Maildirs.

I can imagine various possible meanings for those words.
What exactly is the feature that you propose to implement in Rmail?

  > I don't think that's too hard to implement, maybe I'll take a look.

Depending on the meaning, it might require a total rewrite.



-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Before we embark on such a massive documentation change, I'd like to
  > try to find a less invasive change of terminology, if that exists.  Do
  > you see a way to do that?

I would not object to changing the existing Rmail partial-summary commands
to use the term "filtered" if that makes the whole of Rmail summaries
clearer.

  > Anyway, unless I misunderstand something, the "summary" in Rmail is called 
  > the message "index" or "list" by other MUAs.

"List" in the context of Emacs will tend to cause confusion.
"Index" might be ok to switch to.  Does anyone see a possible
confusion in use of "index" here?

  > How about "progressive summary"? or maybe "progressively narrowing
  > summary"?

If it is ok to use the term "narrowing" for this.
"progressively narrowing" would make sense.

It could also be "progressive filtering".

  >   Is "limit" perhaps a 
  > better word?

I think "limit" is not helpful here.
That's because "progressively limiting" is not a clear way to
decribe this.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > If I may, I would suggest something different: an option to query the user 
  > how they want the rmail-summary-by-* command to be applied: on the entire 
  > folder, to limit the existing summary, to extend the existing summary.

The first of these three options is not useful -- to get that result,
just type q before the command.  So there would be only two that are needed:
union and intersection.

C-u would be a natural way to specify this.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Gregory Heytings]

>> If I may, I would suggest something different: an option to query the 
>> user how they want the rmail-summary-by-* command to be applied: on the 
>> entire folder, to limit the existing summary, to extend the existing 
>> summary.
>
> The first of these three options is not useful -- to get that result, 
> just type q before the command.  So there would be only two that are 
> needed: union and intersection.
>

A third option would be more natural, I think.  After all, the first 
option is AFAIU the current effect of the rmail-summary-by-* command, they 
are applied on the entire folder and create a new summary.

>
> C-u would be a natural way to specify this.
>

C-u is already used by the rmail-summary-by-topic and 
rmail-summary-by-recipients commands for another purpose.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > >> If I may, I would suggest something different: an option to query the 
  > >> user how they want the rmail-summary-by-* command to be applied: on the 
  > >> entire folder, to limit the existing summary, to extend the existing 
  > >> summary.
  > >
  > > The first of these three options is not useful -- to get that result, 
  > > just type q before the command.  So there would be only two that are 
  > > needed: union and intersection.
  > >

  > A third option would be more natural, I think.

I don't follow the reasoning.

                                                    After all, the first 
  > option is AFAIU the current effect of the rmail-summary-by-* command, they 
  > are applied on the entire folder and create a new summary.

That is true -- but what does it have to do with what user interface
to use for choosing between union and intersection here?
I don't see how it relates.

My point is that we already have q as a way to specify to disregard
the previous filtering and start from "all the messages in the file."

  > > C-u would be a natural way to specify this.
  > >

  > C-u is already used by the rmail-summary-by-topic and 
  > rmail-summary-by-recipients commands for another purpose.

Indeed it is.  Hmm.

Nonetheless, prompting for the choice of set operation
is not very convenient.  It would be nicer to have something
the user can type before the filtering command to specify how
to combine it.

Here's an idea.  Type & to do intersection with the following filter,
and type | to do union with the following filter.  Both & and | are
undefined in Rmail mode, so they are available for this.  They
will be easy to remember, too.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Gregory Heytings]

>> I just tried Rmail, again I had a very positive impression, and again I 
>> regret that it works only with mbox files and not with Maildirs.
>
> I can imagine various possible meanings for those words. What exactly is 
> the feature that you propose to implement in Rmail?
>

Currently Rmail only works with mbox files: each mail folder is a single 
file, in which all mails are placed one after the other.  As you probably 
know, Maildirs were proposed about 25 years ago to address the limitations 
of mbox files, and in particular to remove the need of a locking mechanism 
when the mail folder is accessed concurrently by two programs (e.g. the 
mail delivery agent and the mail user agent).  With Maildirs, which are 
now the standard way to store mails, each mail folder is a directory, in 
which each individual mail is stored in its own file.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Gregory Heytings]

>> After all, the first option is AFAIU the current effect of the 
>> rmail-summary-by-* command, they are applied on the entire folder and 
>> create a new summary.
>
> That is true -- but what does it have to do with what user interface to 
> use for choosing between union and intersection here? I don't see how it 
> relates.
>

It seems more natural, at least to me, to ask the user how they want the 
command to be applied.  For example, currently C-M-r asks a single 
question:

Recipients to summarize by:

and each C-M-r invocation starts again from all messages in the file. 
Instead C-M-r could ask two questions, the first one being a multiple 
choice similar to y-or-n-p:

New summary/Limit current summary/Extend current summary (n or l or e):

>
> My point is that we already have q as a way to specify to disregard the 
> previous filtering and start from "all the messages in the file."
>

Did you bind 'q' to some other command in your local customizations, 
perhaps?  'q' is by default bound to rmail-quit/rmail-summary-quit, which 
quits Rmail.

RE: [External] : Re: [PATCH v3] Allow applying filters to summary consecutively

[Drew Adams]

> Here's an idea.  Type & to do intersection with the following filter,
> and type | to do union with the following filter.  Both & and | are
> undefined in Rmail mode, so they are available for this.  They
> will be easy to remember, too.

FWIW, this is what you can do using Icicles - HTH.

The idea is that (1) you can narrow to a subset of
the current set of candidates, or (2) if you've
narrowed, you can widen to the current set plus
new matches against the last set from which you
narrowed.

IOW, you can widen if you've previously narrowed
- otherwise, widening is a no-op, as there's no
superset to match against.

It works like this:

1. When you complete, the original domain of
completion candidates is that used by (passed to)
`completing-read' (or similar) - an alist or whatever.

2. During completion you can use a key to narrow or
widen the set of candidates.

3. Each time you narrow, a recursive minibuffer is
used, completing with the result of narrowing, i.e.,
completing against the set of matches in the parent
minibuffer level.

4. `C-g' pops up one level of minibuffer, i.e., one
level of recursion (out of the minibuffer altogether
if already at the top-level minibuffer).

5. #4 means you can narrow, do something with some
candidates, then `C-g' to remove the last narrowing
filter, narrow again with a different filter, etc.

6. When you widen, the current set of candidates is
matched against a regexp that's an alternation (\|)
of the current match pattern and whatever widening
pattern you provide.  Instead of entering a recursive
minibuffer, you stay at the same level, but with the
match-pattern changed to the alternation.

Narrowing matches patterns independently - IOW it's
orderless.  This is possible because it uses a fresh
minibuffer with a different set of candidates (those
that matched the previous pattern of the new AND).

Widening just changes the current match-pattern to
an alternation regexp (OR).  It doesn't invoke a
recursive minibuffer.  It's not orderless.
____

`M-*' narrows without completing
`S-SPC' narrows and completes
`M-+' widens without completing
`S-<backspace>' widens and completes
`S-TAB' completes (`TAB' does also, but differently)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It seems more natural, at least to me, to ask the user how they want the 
  > command to be applied.

To me that seems the least convenient interface.
I'd prefer an interface which doesn't ask questions.

Here's a proposal.

A summary filter command _in the Rmail buffer_ should always
make a new summary, discarding the previous summary first.

A summary filter command _in a summary buffer_ should always modify
the existing summary in that buffer.  If you type | first, you get
union; otherwise, intersection.

I think this is very natural.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Did you bind 'q' to some other command in your local customizations, 
  > perhaps?  'q' is by default bound to rmail-quit/rmail-summary-quit, which 
  > quits Rmail.

Indeed, I did that.  I didn't remenber that I had chaged that binding in
summaries.

(I think the binding of q in summaries would be a good thing to change
by default.)

  > It seems more natural, at least to me, to ask the user how they want the 
  > command to be applied.

To me that seems the least convenient kind of interface.
I'd prefer an interface which doesn't need to ask questions.

Here's a proposal which doesn't involve use of q.

A summary filter command _in the Rmail buffer_ should always
make a new summary, discarding the previous summary first.

A summary filter command _in a summary buffer_ should always modify
the existing summary in that buffer.  If you type | first, you get
union; otherwise, intersection.

I think this will be both natural and convenient.

Additional idea: ~ in a summary buffer could invert that summary.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >   As you probably 
  > know, Maildirs were proposed about 25 years ago to address the limitations 
  > of mbox files,

I don't consider mbox files limited.  Why do you think so?

  > in particular to remove the need of a locking mechanism 
  > when the mail folder is accessed concurrently by two programs (e.g. the 
  > mail delivery agent and the mail user agent).

This is a complication in the design; but given that we implemented
this interlocking already (in the 1980s). in practice it is not a
limitation any more.

  >   With Maildirs, which are 
  > now the standard way to store mails, each mail folder is a directory, in 
  > which each individual mail is stored in its own file.

I'm concerned that this will make Rmail files bigger and make Rmail slower.

My RMAIL file contains 5000 messages and is 286987559 bytes long.
Rmauk can read it and parse it in 3 seconds or so.  This is on a Thinkpad x200,
which is slow by current standards,

How long would it take to read and digest a Maildir on this machine?

Would it have to read each message when it starts up?  Or else, would
making a summary or searching thru messages have to read each message?

If you tell me about a mail user agent that uses Maildirs, and how I
could read my RMAIL file into one without changing anything, then if
that mail user agent is packaged in Trisquel, I could use it to
estimate what the speed of Rmail would be.

How would Rmail manage memory if these 5000 messages were in a
Maildir?  Would it keep only the current message in memory?  Would it
generate 5000 buffers at startup, once for each message?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Eli Zaretskii]

> From: Andrea Monaco <andrea.monaco@autistici.org>
> Cc: rms@gnu.org, rpluim@gmail.com, emacs-devel@gnu.org
> Date: Mon, 07 Nov 2022 15:13:13 +0100
> 
> 
> The words "negate" and "intersect" come from logic and mathematics.
> They might not be especially common, but they are very precise.  So I
> kept them and expanded on their meaning in the rest of the docstrings.

Thanks.  I eventually went with "invert" instead of "negate", and with
rmail-summary-progressively-narrow instead of
rmail-summary-apply-filters-consecutively.

Please in the future provide ChangeLog-style commit log message for
the changes.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Gregory Heytings]

>> As you probably know, Maildirs were proposed about 25 years ago to 
>> address the limitations of mbox files,
>
> I don't consider mbox files limited.  Why do you think so?
>

My opinions are not important, facts are.  The fact is that:

1. Mbox files do not scale.  It is not uncommon to have mail folders that 
are several Gb large, and opening a 50 Kb email in say a 2 Gb folder means 
reading the whole 2 Gb file.

2. Mbox files are fragile.  If for whatever reason a program crashes while 
updating an mbox file, the whole mbox file can be corrupted, or even lost.

3. Mbox files are slow.  For example, when an email in the middle of a 
mail folder is deleted, or moved to another mail folder, the whole mbox 
file must be written again.

4. Mbox files cannot be accessed concurrently by two programs, e.g. the 
mail delivery agent and the mail user agent, or two mail user agents.  In 
practice most people access their mail with several mail user agents, e.g. 
on laptop computer, a desktop computer, a tablet and/or a mobile phone. 
Another common pattern is two or more people using a shared mail folder, 
e.g. for the "contact" email address of a company.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > 1. Mbox files do not scale.  It is not uncommon to have mail folders that 
  > are several Gb large, and opening a 50 Kb email in say a 2 Gb folder means 
  > reading the whole 2 Gb file.

Even my RMAIL file is not a whole GB in size.

How does a mail agent find a specific message in a maildir?
Does it have to read thousands of files one by one?

  > 2. Mbox files are fragile.  If for whatever reason a program crashes while 
  > updating an mbox file, the whole mbox file can be corrupted, or even lost.

Rmail does not have that problem.  Emacs writes the new file under
another name, then renames it into place.  That is atomic.

  > 3. Mbox files are slow.  For example, when an email in the middle of a 
  > mail folder is deleted, or moved to another mail folder, the whole mbox 
  > file must be written again.

Rmail does not try to save the mbox every time you change it.  It
saves when you say to save.  For my 300 meg Rmail file, that is
not painful.

  > 4. Mbox files cannot be accessed concurrently by two programs, e.g. the 
  > mail delivery agent and the mail user agent, or two mail user agents.

That is the case for the incoming mbox.  We have interlocking for
that.  Also, you don't ever leave the mail in the incoming mbox.  You
copy it into the RMAIL file.

What it comes down to is that your mail user agant is designed to be
convenient and easy with maildirs, and Rmail is designed to be
convenient and easy with mboxes.  If you suppose that Rmail works
like a user agent that was designed for maildirs, you will expect
some operations to be problematical, which in fact are not.



-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Gregory Heytings]

>
> What it comes down to is that your mail user agant is designed to be 
> convenient and easy with maildirs, and Rmail is designed to be 
> convenient and easy with mboxes.  If you suppose that Rmail works like a 
> user agent that was designed for maildirs, you will expect some 
> operations to be problematical, which in fact are not.
>

I think we are miscommunicating.

I'm not supposing anything about mail user agents.

I'm just trying to explain why mbox files are incompatible with the way 
most people deal with their mails nowadays.  It's not without reason that 
Maildirs have replaced mboxes (and IMAP has replaced POP).  I understand 
that the limitations of mbox files are apparently not a problem for you 
personally.

More generally, I'm trying to explain why adding a support for Maildirs to 
Rmail might increase its adoption.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I'm just trying to explain why mbox files are incompatible with the way 
  > most people deal with their mails nowadays.

The situation looks symmetrical to me.
I fear that maildirs are incompatible with the way I use email.

I really am intrerested in the answers to the questions I asked
about slownesses that might -- or might not -- be a problem for
use of maildirs.  If they are not in fact a problem, I'd like to know that.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Gregory Heytings]

>> I'm just trying to explain why mbox files are incompatible with the way 
>> most people deal with their mails nowadays.
>
> The situation looks symmetrical to me.
>
> I fear that maildirs are incompatible with the way I use email.
>
> I really am intrerested in the answers to the questions I asked about 
> slownesses that might -- or might not -- be a problem for use of 
> maildirs.  If they are not in fact a problem, I'd like to know that.
>

They are not a problem.  Mail user agents which support Maildirs index the 
individual mails in a database, either an (small) index of the headers or 
a (larger) "full text" index.  With such an index even opening a folder 
with tens of thousands of messages only takes a fraction of a second.

Re: [PATCH v3] Allow applying filters to summary consecutively

[chad]

[-- Attachment #1: Type: text/plain, Size: 814 bytes --]

FWIW, many years ago, I had to switch from RMAIL to mh due to some of the
factors Gregory mentioned, especially infrequent but troublesome corrupted
rmail files and problems with multiple concurrent clients. These problems
are rare when users only use a single device to read from local storage,
but even 25 years ago I was using multiple geographically separated
computers and shared networked storage. RMAIL also suffered,
performance-wise, as email volume increased. Certainly, some of these
problems were more due to movemail than BABYL.

That said, I think the existence of very old email clients such as nmh (and
mh-e) and alpine/mutt and newer clients such as notmuch and mu/m4e provide
enough alternatives to rmail that (I suppose) the interest in an updated
rmail is very small.

I hope that helps,
~Chad

[-- Attachment #2: Type: text/html, Size: 896 bytes --]

Re: [PATCH v3] Allow applying filters to summary consecutively

[Eli Zaretskii]

> From: chad <yandros@gmail.com>
> Date: Wed, 23 Nov 2022 15:57:39 -0500
> Cc: Gregory Heytings <gregory@heytings.org>, andrea.monaco@autistici.org,
>  emacs-devel@gnu.org
> 
> FWIW, many years ago, I had to switch from RMAIL to mh due to some of the factors Gregory mentioned,
> especially infrequent but troublesome corrupted rmail files and problems with multiple concurrent clients.
> These problems are rare when users only use a single device to read from local storage, but even 25 years
> ago I was using multiple geographically separated computers and shared networked storage. RMAIL also
> suffered, performance-wise, as email volume increased. Certainly, some of these problems were more due
> to movemail than BABYL.

Rmail's treatment of mbox files and the way it stores processed email have
been completely redesigned and reimplemented since the time you used it.  In
particular, Rmail no longer uses BABYL format, and doesn't rewrite the
incoming mbox stuff at all, but just processes it in-place using
buffer-swap-text.  The move to Unicode in Emacs and years of cleaning up the
code which encodes/decodes non-ASCII also contributed to solving many
problems.  So now Rmail has none of the problems that probably caused you to
switch from Rmail.  I use it every day, including to read and reply to this
email.

> That said, I think the existence of very old email clients such as nmh (and mh-e) and alpine/mutt and newer
> clients such as notmuch and mu/m4e provide enough alternatives to rmail that (I suppose) the interest in an
> updated rmail is very small.

I don't have objections to adding maildir support to Rmail, I just don't
quite see how that could be done without coding a complete new backend, or
even redesigning (again) how Rmail processes and presents email.  The
backbone of Rmail is a single mbox-formatted buffer which Rmail narrows to a
message it displays, and whose text it swaps with the display buffer using
buffer-swap-text, or reshuffles when some sorting is required, or searches
through when the user issues an Rmail search command.  To use maildir
without changing this basic design, Someone would have to find a way of
presenting multiple files in multiple directories as a single large buffer.
That should be possible, but the naïve ways of doing so will probably be
kludgey and unclean, let alone inefficient.

Re: [PATCH v3] Allow applying filters to summary consecutively

[Andrea Monaco]

The implementation of union and intersection of filters is still not
complete.  Here's my proposal: pressing '&' and '|' in either buffer
could dictate how the next filter operates, by intersection and union
respectively.  Those keys could toggle two mutually exclusive minor
modes that affect the next filter (or all subsequent filters until the
user switches again).



Andrea Monaco

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > FWIW, many years ago, I had to switch from RMAIL to mh due to some of the
  > factors Gregory mentioned, especially infrequent but troublesome corrupted
  > rmail files and problems with multiple concurrent clients. These problems
  > are rare when users only use a single device to read from local storage,

It seems there are real users whose use of Rmail would not be with a
single device, reading from local storage.  I have to believe it, but
I am surprised becaause I don't see how that would happen.  Would you
like to tell me the scenarios of usage that you were using?

Emacs always reads or writes the entire file,
and does it in a way designed to avoid any chance
of half-written files.  Also, it has interlocking for
writing a file from multiple Emacses.  That applies
to all files, not just Rmail flles.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > The implementation of union and intersection of filters is still not
  > complete.  Here's my proposal: pressing '&' and '|' in either buffer
  > could dictate how the next filter operates, by intersection and union
  > respectively.  Those keys could toggle two mutually exclusive minor
  > modes that affect the next filter (or all subsequent filters until the
  > user switches again).

It's not _bad_ but it is unnecessarily heavyweight and complex, and extra
work to use.

I suggest making & the default, so you don't need to specify that.  If
you type |, then if the next command is a filter command, it or's.

Simple to use, and simple to document.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I don't have objections to adding maildir support to Rmail, I just don't
  > quite see how that could be done without coding a complete new backend, or
  > even redesigning (again) how Rmail processes and presents email.

That is my opinion too.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [PATCH v3] Allow applying filters to summary consecutively

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > They are not a problem.  Mail user agents which support Maildirs index the 
  > individual mails in a database, either an (small) index of the headers or 
  > a (larger) "full text" index.  With such an index even opening a folder 
  > with tens of thousands of messages only takes a fraction of a second.

Thanks.  It seems to be a good solution as long as the mail agent
knows how to recalculate the database when the directory has changed.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)