bug#72442: 31.0.50; Gnus or message.el doesn't handle international message reply subject prefixes and insults users

[Robert Pluim <rpluim@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 379 bytes --]

>>>>> On Thu, 22 Aug 2024 15:15:03 +0300, Eli Zaretskii <eliz@gnu.org> said:

    Eli> For backward compatibility, let's leave rmail-re-abbrevs in place, and
    Eli> just initialize it from that new defcustom.

Patch attached. Works for gnus for me, and manual testing of the rmail
regexp works. Did you want me to obsolote the rmail and message
variables as well?

Robert
-- 


[-- Attachment #2: 0001-Teach-gnus-message-about-international-Re-variants.patch --]
[-- Type: text/x-diff, Size: 9823 bytes --]

From ae84c547234ab28cc50ae2bb288bd401978ec293 Mon Sep 17 00:00:00 2001
From: Robert Pluim <rpluim@gmail.com>
Date: Thu, 22 Aug 2024 18:49:12 +0200
Subject: [PATCH] Teach gnus/message about international Re: variants
To: emacs-devel@gnu.org

Bug#72442

* lisp/mail/mail-utils.el (mail-re-regexps): New defcustom, contains the
components used to construct 'rmail-re-abbrevs' and
'message-subject-re-regexp'.
* lisp/gnus/message.el (message-subject-re-regexp): Derive from
'mail-re-regexps'.
(message-strip-subject-re): Make the match case-insensitive.
* lisp/mail/rmail.el (rmail-re-abbrevs): Derive from 'mail-re-regexps'.
Update 'rmail-reply-regexp' when it changes.
(rmail-reply-regexp): Set to nil, 'rmail-re-abbrevs' will set it.

* doc/emacs/rmail.texi (Rmail Reply): Describe 'mail-re-regexps'.
* doc/misc/message.texi (Message Headers): Describe 'mail-re-regexps'.
---
 doc/emacs/rmail.texi    |  8 ++++++++
 doc/misc/message.texi   | 13 ++++++++++++-
 etc/NEWS                | 26 ++++++++++++++++++++++++++
 lisp/gnus/message.el    | 29 ++++++++++++++++++++++-------
 lisp/mail/mail-utils.el | 19 +++++++++++++++++++
 lisp/mail/rmail.el      | 31 ++++++++++++++++++++-----------
 6 files changed, 107 insertions(+), 19 deletions(-)

diff --git a/doc/emacs/rmail.texi b/doc/emacs/rmail.texi
index 9c20d30c442..e090ec6fe76 100644
--- a/doc/emacs/rmail.texi
+++ b/doc/emacs/rmail.texi
@@ -776,6 +776,14 @@ Rmail Reply
 sent the message you received, and the @samp{CC} field starts out with
 all the other recipients of that message.
 
+@vindex rmail-re-abbrevs
+@vindex rmail-reply-prefix
+@vindex mail-re-regexps
+The @samp{Subject} header field may contain one or more instances of
+@samp{Re:} or localized variants thereof.  These are removed if they
+match @code{rmail-re-abbrevs} (which is initialized from
+@code{mail-re-regexps}), and @code{rmail-reply-prefix} is prepended.
+
 @vindex mail-dont-reply-to-names
   You can exclude certain recipients from being included automatically
 in replies, using the variable @code{mail-dont-reply-to-names}.  Its
diff --git a/doc/misc/message.texi b/doc/misc/message.texi
index 468bf81599d..f5399aecee1 100644
--- a/doc/misc/message.texi
+++ b/doc/misc/message.texi
@@ -1693,13 +1693,14 @@ Message Headers
 
 @item message-subject-re-regexp
 @vindex message-subject-re-regexp
+@vindex mail-re-regexps
 @cindex Aw
 @cindex Sv
 @cindex Re
 Responses to messages have subjects that start with @samp{Re: }.  This
 is @emph{not} an abbreviation of the English word ``response'', but it
 comes from the Latin ``res'', and means ``in the matter of''.  Some
-illiterate nincompoops have failed to grasp this fact, and have
+standards-challenged companies have failed to grasp this fact, and have
 ``internationalized'' their software to use abominations like
 @samp{Aw: } (``antwort'') or @samp{Sv: } (``svar'') instead, which is
 meaningless and evil.  However, you may have to deal with users that
@@ -1731,6 +1732,16 @@ Message Headers
        ))
 @end lisp
 
+You shouldn't need to do this, since the default value of
+@code{message-subject-re-regexp} is initialized based on
+@code{mail-re-regexps}, which covers most known cases of such
+internationalization, and is a lot easier to customize.  Customizing
+@code{mail-re-regexps} updates @code{message-subject-re-regexp} to
+match.
+
+Note that the regexp is matched case-insensitively against the
+@samp{Subject} header contents.
+
 @item message-subject-trailing-was-query
 @vindex message-subject-trailing-was-query
 @vindex message-subject-trailing-was-ask-regexp
diff --git a/etc/NEWS b/etc/NEWS
index 07d1cce0966..82c0eac4e5c 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -162,6 +162,32 @@ By passing '-t' or '--timeout', you can specify a maximum time to wait
 for the processes to exit.  Additionally, you can now wait for external
 processes by passing their PIDs.
 
++++
+** Mail-util
+
+*** New user option 'mail-re-regexps'.
+This contains the list of regular expressions used to match "Re:" and
+international variants of it when modifying the Subject field in
+replies.
+
++++
+** Rmail
+
+*** 'rmail-re-abbrevs' default value is now derived from 'mail-re-regexps'.
+'mail-re-regexps' is a new user option that is easier to customize than
+'rmail-re-abbrevs'.  'rmail-re-abbrevs' is still honored if it was
+already set.
+
++++
+** Message
+
+*** 'message-subject-re-regexp' default value is now derived from 'mail-re-regexps'.
+'mail-re-regexps' is a new user option that is easier to customize than
+'message-subject-re-regexp'.  'message-subject-re-regexp' is still
+honored if it was already set.
+
+*** 'message-strip-subject-re' now matches case-insensitively.
+
 ** SHR
 
 +++
diff --git a/lisp/gnus/message.el b/lisp/gnus/message.el
index 98f63571389..081c6986a66 100644
--- a/lisp/gnus/message.el
+++ b/lisp/gnus/message.el
@@ -305,11 +305,24 @@ message-ignored-supersedes-headers
 		 regexp))
 
 (defcustom message-subject-re-regexp
-  "^[ \t]*\\([Rr][Ee]\\(\\[[0-9]*\\]\\)* ?:[ \t]*\\)*[ \t]*"
-  "Regexp matching \"Re: \" in the subject line."
+  (concat "^[ \t]*"
+          "\\("
+          "\\("
+          (string-join mail-re-regexps "\\|")
+          "\\)"
+          "\\(\\[[0-9]*\\]\\)*"
+          " ?\u00a0*[:：][ \t]*"
+          "\\)"
+          "*[ \t]*")
+  "Regexp matching \"Re: \" in the subject line.
+Matching is done case-insensitively.
+Initialized from the value of `mail-re-regexps', which is easier to
+customize."
   :group 'message-various
   :link '(custom-manual "(message)Message Headers")
-  :type 'regexp)
+  :type 'regexp
+  :set-after '(mail-re-regexps)
+  :version "31.1")
 
 (defcustom message-screenshot-command '("import" "png:-")
   "Command to take a screenshot.
@@ -2257,10 +2270,12 @@ message-strip-list-identifiers
       subject)))
 
 (defun message-strip-subject-re (subject)
-  "Remove \"Re:\" from subject lines in string SUBJECT."
-  (if (string-match message-subject-re-regexp subject)
-      (substring subject (match-end 0))
-    subject))
+  "Remove \"Re:\" from subject lines in string SUBJECT.
+This uses `mail-re-regexps, matching is done case-insensitively."
+  (let ((case-fold-search t))
+    (if (string-match message-subject-re-regexp subject)
+        (substring subject (match-end 0))
+      subject)))
 
 (defcustom message-replacement-char "."
   "Replacement character used instead of unprintable or not decodable chars."
diff --git a/lisp/mail/mail-utils.el b/lisp/mail/mail-utils.el
index c9e4d0b3812..a04092b5727 100644
--- a/lisp/mail/mail-utils.el
+++ b/lisp/mail/mail-utils.el
@@ -46,6 +46,25 @@ mail-dont-reply-to-names
   :type '(choice regexp (const :tag "Your Name" nil))
   :group 'mail)
 
+;;;###autoload
+(defcustom mail-re-regexps
+  '("RE" "R\u00c9\\.?" "FWD?" "رد" "回复" "回覆" "SV" "Antw\\.?"
+  "VS" "REF" "AW" "ΑΠ" "ΣΧΕΤ" "השב" "Vá" "R" "RIF" "BLS" "RES"
+  "Odp" "YNT" "ATB")
+  "List of localized \"Re:\" abbreviations in various languages.
+Each component can be a regular expression or a simple string.  Matching
+is done case-insensitively.  Used to initialize the legacy
+`rmail-re-abbrevs' and `message-subject-re-regexp' user options."
+  :type '(repeat regexp)
+  :set (lambda (sym val)
+         (custom-set-default sym val)
+         (dolist (sym '(rmail-re-abbrevs
+                        message-subject-re-regexp))
+           (when (get sym 'standard-value)
+             (custom-reevaluate-setting sym))))
+  :group 'mail
+  :version "31.1")
+
 (defvar epa-inhibit)
 ;; Returns t if file FILE is an Rmail file.
 ;;;###autoload
diff --git a/lisp/mail/rmail.el b/lisp/mail/rmail.el
index e38ab12fae6..4965907916c 100644
--- a/lisp/mail/rmail.el
+++ b/lisp/mail/rmail.el
@@ -527,23 +527,32 @@ rmail-automatic-folder-directives
 (defvar rmail-reply-prefix "Re: "
   "String to prepend to Subject line when replying to a message.")
 
-;; Note: this is matched with case-fold-search bound to t.
-(defcustom rmail-re-abbrevs
-  "\\(RE\\|رد\\|回复\\|回覆\\|SV\\|Antw\\|VS\\|REF\\|AW\\|ΑΠ\\|ΣΧΕΤ\\|השב\\|Vá\\|R\\|RIF\\|BLS\\|RES\\|Odp\\|YNT\\|ATB\\)"
-  "Regexp with localized \"Re:\" abbreviations in various languages."
-  :version "28.1"
-  :type 'regexp)
-
 ;; Some mailers use "Re(2):" or "Re^2:" or "Re: Re:" or "Re[2]:".
 ;; This pattern should catch all the common variants.
 ;; rms: I deleted the change to delete tags in square brackets
 ;; because they mess up RT tags.
-(defvar rmail-reply-regexp
-  (concat "\\`\\("
-          rmail-re-abbrevs
-          "\\(([0-9]+)\\|\\[[0-9]+\\]\\|\\^[0-9]+\\)?\u00a0*[:：] *\\)*")
+(defvar rmail-reply-regexp nil ;; set by `rmail-re-abbrevs
   "Regexp to delete from Subject line before inserting `rmail-reply-prefix'.")
 
+;; Note: this is matched with case-fold-search bound to t.
+(defcustom rmail-re-abbrevs
+  (concat "\\("
+          (string-join mail-re-regexps "\\|")
+          "\\)")
+  "Regexp with localized \"Re:\" abbreviations in various languages.
+Matching is done case-insensitively.
+Initialized from `mail-re-regexps', which is easier to customize."
+  :set-after '(mail-re-regexps)
+  :set (lambda (sym val)
+         (custom-set-default sym val)
+         (setq rmail-reply-regexp
+               (concat "\\`[ \t]*\\("
+                       val
+                       "\\(([0-9]+)\\|\\[[0-9]+\\]\\|\\^[0-9]+\\)?"
+                       " ?\u00a0*[:：][ \t]*\\)*[ \t]*")))
+  :type 'regexp
+  :version "31.1")
+
 (defcustom rmail-display-summary nil
   "If non-nil, Rmail always displays the summary buffer."
   :group 'rmail-summary
-- 
2.39.2