bug#66330: 28.2; Gnus wide reply (S W) changes Subject header when used from Debbugs thread

bug#66330: 28.2; Gnus wide reply (S W) changes Subject header when used from Debbugs thread

[Maxim Cournoyer]

Hi,

When attempting to reply from an Emacs Debbugs thread specific email,
the Subject line gets set to the subject of the first email in the
thread rather than to that of the specific email I'm replying to, which
looses context.

Example/reproducer:

$ guix shell --container emacs-no-x emacs-debbugs -- emacs

From the launched Emacs:

1. M-x debbugs-gnu-bugs 65479

2. Press 'a' to load the bug despite the lack of SSL certificates.

3. Navigate to the message titled:
[bug#65479] [PATCH core-updates 02/61] gnu: Add docbook-xml-5.

4. Press S W to send a wide reply.

5. Observe that the Subject header in the email response being edited
has been changed to "Re: bug#65479: [PATCH core-updates 00/61] The
Draining of the XML & DocBook Swamp.", that is, the original subject of
the "bug" (its first message).

I haven't been able to explain what causes that, but I've found where it
occurs.  Following the original 'gnus-article-wide-reply-with-original'
call, it eventually calls to 'message-setup-1', and the Subject
modification happens by unknown code (hooks?) just before the line 6870
("(run-hooks 'message-setup-hook)") runs:

--8<---------------cut here---------------start------------->8---
(defun message-setup-1 (headers &optional yank-action actions return-action)
  (dolist (action actions)
    (condition-case nil
        ;; FIXME: Use functions rather than expressions!
	(add-to-list 'message-send-actions
		     `(apply #',(car action) ',(cdr action)))))
  (setq message-return-action return-action)
  
[...]

  (run-hooks 'message-signature-setup-hook)
  (message-insert-signature)
  (save-restriction
    (message-narrow-to-headers)
    (run-hooks 'message-header-setup-hook))
  (setq buffer-undo-list nil)
  (when message-generate-hashcash
    ;; Generate hashcash headers for recipients already known
    (mail-add-payment-async))

===> Subject header changed HERE

  ;; Gnus posting styles are applied via buffer-local `message-setup-hook'
  ;; values.
  (run-hooks 'message-setup-hook)
  ;; Do this last to give it precedence over posting styles, etc.
  (when (message-mail-p)
    (save-restriction
      (message-narrow-to-headers)
      (if message-alternative-emails
	  (message-use-alternative-email-as-from))))
  (message-position-point)
  ;; Allow correct handling of `message-checksum' in `message-yank-original':
  (set-buffer-modified-p nil)
  (undo-boundary)
  ;; rmail-start-mail expects message-mail to return t (Bug#9392)
  t)
--8<---------------cut here---------------end--------------->8---

Would someone be able to explain what is responsible for this?  It seems
to me it'd be preferable if the subject of the message I'm replying to
was kept rather than replaced this way, to preserve context.

-- 
Thanks,
Maxim

bug#66330: 28.2; Gnus wide reply (S W) changes Subject header when used from Debbugs thread

[Eli Zaretskii]

> Cc: Bruno Victal <mirai@makinata.eu>
> From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
> Date: Tue, 03 Oct 2023 20:26:26 -0400
> 
> When attempting to reply from an Emacs Debbugs thread specific email,
> the Subject line gets set to the subject of the first email in the
> thread rather than to that of the specific email I'm replying to, which
> looses context.

The Subject lines of emails pertaining to a discussion of a bug should
never be changed.  So I see no problem here, unless I'm missing
something.

Michael, am I wrong?

> 1. M-x debbugs-gnu-bugs 65479
> 
> 2. Press 'a' to load the bug despite the lack of SSL certificates.
> 
> 3. Navigate to the message titled:
> [bug#65479] [PATCH core-updates 02/61] gnu: Add docbook-xml-5.
> 
> 4. Press S W to send a wide reply.
> 
> 5. Observe that the Subject header in the email response being edited
> has been changed to "Re: bug#65479: [PATCH core-updates 00/61] The
> Draining of the XML & DocBook Swamp.", that is, the original subject of
> the "bug" (its first message).

This is a bad Subject line to begin with, from where I stand: it
should not include the patch series numbers, and it should not assume
there will be additional messages with different series numbers.

The Subject lines in a bug/issue discussion should describe the
bug/issue, not the patches sent to fix the issue.

bug#66330: 28.2; Gnus wide reply (S W) changes Subject header when used from Debbugs thread

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

>> When attempting to reply from an Emacs Debbugs thread specific email,
>> the Subject line gets set to the subject of the first email in the
>> thread rather than to that of the specific email I'm replying to, which
>> looses context.
>
> The Subject lines of emails pertaining to a discussion of a bug should
> never be changed.  So I see no problem here, unless I'm missing
> something.
>
> Michael, am I wrong?

Well, we haven't documented such a rule. But based on my experience I
would agree. There's often a message in the bug-gnu-emacs@gnu.org ML
with the subject "bug#xxxxx: Patch" or alike, which doesn't help. I'm
too old to remember all the bug numbers and their related topic, and I
read only messages in that ML with topics I'm interested in.

>> 1. M-x debbugs-gnu-bugs 65479
>>
>> 2. Press 'a' to load the bug despite the lack of SSL certificates.
>>
>> 3. Navigate to the message titled:
>> [bug#65479] [PATCH core-updates 02/61] gnu: Add docbook-xml-5.
>>
>> 4. Press S W to send a wide reply.
>>
>> 5. Observe that the Subject header in the email response being edited
>> has been changed to "Re: bug#65479: [PATCH core-updates 00/61] The
>> Draining of the XML & DocBook Swamp.", that is, the original subject of
>> the "bug" (its first message).
>
> This is a bad Subject line to begin with, from where I stand: it
> should not include the patch series numbers, and it should not assume
> there will be additional messages with different series numbers.
>
> The Subject lines in a bug/issue discussion should describe the
> bug/issue, not the patches sent to fix the issue.

D'accord. But I guess the subject line quoted here is created by a tool
of the guix-patches toolchain. So first we shall agree about the rules
of that toolchain (I'm from Emacs, I won't dictate anything!).

If this is an agreed behavior, we could discuss about a debbugs-gnu user
option how to handle the subject line of replies, with a different
default value for guix-patches messages. The global default shouldn't be
changed.

Well, we even have already guix specific code, see debbugs-guix.el. I
would appreciate if somebody from the guix project supports me in
writing such a patch.

Best regards, Michael.

bug#66330: 28.2; Gnus wide reply (S W) changes Subject header when used from Debbugs thread

[Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Michael Albinus <michael.albinus@gmx.de> writes:

>>> 5. Observe that the Subject header in the email response being edited
>>> has been changed to "Re: bug#65479: [PATCH core-updates 00/61] The
>>> Draining of the XML & DocBook Swamp.", that is, the original subject of
>>> the "bug" (its first message).
>>
>> This is a bad Subject line to begin with, from where I stand: it
>> should not include the patch series numbers, and it should not assume
>> there will be additional messages with different series numbers.
>>
>> The Subject lines in a bug/issue discussion should describe the
>> bug/issue, not the patches sent to fix the issue.
>
> D'accord. But I guess the subject line quoted here is created by a tool
> of the guix-patches toolchain. So first we shall agree about the rules
> of that toolchain (I'm from Emacs, I won't dictate anything!).

The `[PATCH patch-number/patch-count] Subject` subject format is also the
generate format for Git patch-series. The Guix subject format is a
variant of this.

The subject changing could be an issue outside of Gnus. E.g when
responding to a patch in git patch series outside of Guix.

However I think it is a general issue that mail clients append Re: or
some localized variation of it. The issue should also be reported to
Guix.
Other ticket/bugtracking systems might not like changing the subject
either.

bug#66330: 28.2; Gnus wide reply (S W) changes Subject header when used from Debbugs thread

[Eli Zaretskii]

> From: Björn Bidar <bjorn.bidar@thaodan.de>
> Cc: Eli Zaretskii <eliz@gnu.org>,  mirai@makinata.eu,
>   66330@debbugs.gnu.org,  Maxim Cournoyer <maxim.cournoyer@gmail.com>
> Date: Wed, 04 Oct 2023 11:31:55 +0300
> 
> Michael Albinus <michael.albinus@gmx.de> writes:
> 
> >> The Subject lines in a bug/issue discussion should describe the
> >> bug/issue, not the patches sent to fix the issue.
> >
> > D'accord. But I guess the subject line quoted here is created by a tool
> > of the guix-patches toolchain. So first we shall agree about the rules
> > of that toolchain (I'm from Emacs, I won't dictate anything!).
> 
> The `[PATCH patch-number/patch-count] Subject` subject format is also the
> generate format for Git patch-series. The Guix subject format is a
> variant of this.
> 
> The subject changing could be an issue outside of Gnus. E.g when
> responding to a patch in git patch series outside of Guix.
> 
> However I think it is a general issue that mail clients append Re: or
> some localized variation of it.

Re: and its localized variants are not a problem: all Emacs MUAs
support that and know how to ignore them for threading purposes.

bug#66330: 28.2; Gnus wide reply (S W) changes Subject header when used from Debbugs thread

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

>> The subject changing could be an issue outside of Gnus. E.g when
>> responding to a patch in git patch series outside of Guix.
>>
>> However I think it is a general issue that mail clients append Re: or
>> some localized variation of it.
>
> Re: and its localized variants are not a problem: all Emacs MUAs
> support that and know how to ignore them for threading purposes.

This is right in general. They have problems with cascaded localized
prefixes, like "Re: AW: ". But that's another problem, and an edge case
rather. Nothing I worry about.

Best regards, Michael.

bug#66330: 28.2; Gnus wide reply (S W) changes Subject header when used from Debbugs thread

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: Björn Bidar <bjorn.bidar@thaodan.de>,
>   mirai@makinata.eu,
>   66330@debbugs.gnu.org,  maxim.cournoyer@gmail.com
> Date: Wed, 04 Oct 2023 11:11:23 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Re: and its localized variants are not a problem: all Emacs MUAs
> > support that and know how to ignore them for threading purposes.
> 
> This is right in general. They have problems with cascaded localized
> prefixes, like "Re: AW: ".

Maybe some of them do, but not all.

> But that's another problem, and an edge case rather. Nothing I worry
> about.

Right.

bug#66330: 28.2; Gnus wide reply (S W) changes Subject header when used from Debbugs thread

[Björn Bidar via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Michael Albinus <michael.albinus@gmx.de> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
> Hi Eli,
>
>>> The subject changing could be an issue outside of Gnus. E.g when
>>> responding to a patch in git patch series outside of Guix.
>>>
>>> However I think it is a general issue that mail clients append Re: or
>>> some localized variation of it.
>>
>> Re: and its localized variants are not a problem: all Emacs MUAs
>> support that and know how to ignore them for threading purposes.
>
> This is right in general. They have problems with cascaded localized
> prefixes, like "Re: AW: ". But that's another problem, and an edge case
> rather. Nothing I worry about.

It depends on how far much you interact with people that don't speak
English as their native language.

But then it's a minor issue even it's not an edge case.

bug#66330: 28.2; Gnus wide reply (S W) changes Subject header when used from Debbugs thread

[Michael Albinus]

Björn Bidar <bjorn.bidar@thaodan.de> writes:

Hi Björn,

>> D'accord. But I guess the subject line quoted here is created by a tool
>> of the guix-patches toolchain. So first we shall agree about the rules
>> of that toolchain (I'm from Emacs, I won't dictate anything!).
>
> The `[PATCH patch-number/patch-count] Subject` subject format is also the
> generate format for Git patch-series. The Guix subject format is a
> variant of this.

So perhaps we check the subject of the message we reply to for

--8<---------------cut here---------------start------------->8---
(rx "[PATCH" (? blank (1+ digit) "/" (1+ digit)) "]")
--8<---------------cut here---------------end--------------->8---

If it matches, we use the subject of the message for the reply, rather
than the title of the bug report.

The regexp should be offered as user option, of course.

Best regards, Michael.

bug#66330: 28.2; Gnus wide reply (S W) changes Subject header when used from Debbugs thread

[Maxim Cournoyer]

Hello,

Michael Albinus <michael.albinus@gmx.de> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
> Hi Eli,
>
>>> When attempting to reply from an Emacs Debbugs thread specific email,
>>> the Subject line gets set to the subject of the first email in the
>>> thread rather than to that of the specific email I'm replying to, which
>>> looses context.
>>
>> The Subject lines of emails pertaining to a discussion of a bug should
>> never be changed.  So I see no problem here, unless I'm missing
>> something.
>>
>> Michael, am I wrong?
>
> Well, we haven't documented such a rule. But based on my experience I
> would agree. There's often a message in the bug-gnu-emacs@gnu.org ML
> with the subject "bug#xxxxx: Patch" or alike, which doesn't help. I'm
> too old to remember all the bug numbers and their related topic, and I
> read only messages in that ML with topics I'm interested in.
>
>>> 1. M-x debbugs-gnu-bugs 65479
>>>
>>> 2. Press 'a' to load the bug despite the lack of SSL certificates.
>>>
>>> 3. Navigate to the message titled:
>>> [bug#65479] [PATCH core-updates 02/61] gnu: Add docbook-xml-5.
>>>
>>> 4. Press S W to send a wide reply.
>>>
>>> 5. Observe that the Subject header in the email response being edited
>>> has been changed to "Re: bug#65479: [PATCH core-updates 00/61] The
>>> Draining of the XML & DocBook Swamp.", that is, the original subject of
>>> the "bug" (its first message).
>>
>> This is a bad Subject line to begin with, from where I stand: it
>> should not include the patch series numbers, and it should not assume
>> there will be additional messages with different series numbers.
>>
>> The Subject lines in a bug/issue discussion should describe the
>> bug/issue, not the patches sent to fix the issue.

When working on bugs, I agree this behavior makes sense, where the
original bug title should be kept around.  I'm fine with that behavior
for the 'bug-guix' tracker, which tracks bugs.  Where I'd like to be
able to customize/change that behavior is when working with *patches*
posted to guix-patches, which is also hosted on Debbugs.

These patches typically get sent via 'git send-email' and include in
their subject the patch number, e.g. [PATCH v2 12/62].  I want this
preserved when reviewing the patch, so that participant knows what was
reviewed.

> D'accord. But I guess the subject line quoted here is created by a tool
> of the guix-patches toolchain. So first we shall agree about the rules
> of that toolchain (I'm from Emacs, I won't dictate anything!).
>
> If this is an agreed behavior, we could discuss about a debbugs-gnu user
> option how to handle the subject line of replies, with a different
> default value for guix-patches messages. The global default shouldn't be
> changed.

That sounds good to me.

> Well, we even have already guix specific code, see debbugs-guix.el. I
> would appreciate if somebody from the guix project supports me in
> writing such a patch.

I'm willing to work on it.  It seems you draft a plan in another reply,
which I'll look at now.

-- 
Thanks,
Maxim

bug#66330: 28.2; Gnus wide reply (S W) changes Subject header when used from Debbugs thread

[Michael Albinus]

[-- Attachment #1: Type: text/plain, Size: 473 bytes --]

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> Hi,

Hi Maxim,

> Would someone be able to explain what is responsible for this?  It seems
> to me it'd be preferable if the subject of the message I'm replying to
> was kept rather than replaced this way, to preserve context.

FTR, this behavior is due to setting a default subject via
gnus-posting-styles in debbugs-gnu-summary-mode. Could you please check
whether the appended patch helps?

Best regards, Michael.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1058 bytes --]

diff --git a/debbugs-gnu.el b/debbugs-gnu.el
index f366fba11d..cd549e364d 100644
--- a/debbugs-gnu.el
+++ b/debbugs-gnu.el
@@ -1747,6 +1747,12 @@ MERGED is the list of bugs merged with this one."
 	 (format "Re: bug#%d: %s" id (alist-get 'subject status)))
     (debbugs-gnu-summary-mode 1)))

+(defcustom debbugs-gnu-summary-keep-subject
+  (rx "[PATCH" (? (0+ (not (any digit "/]"))) (1+ digit) "/" (1+ digit)) "]")
+  "Regular expression which keeps the original message subject in replies."
+  :version "29.1"
+  :type 'regexp)
+
 (defvar debbugs-gnu-summary-mode-map
   (let ((map (make-sparse-keymap)))
     (define-key map "C" #'debbugs-gnu-send-control-message)
@@ -1778,6 +1784,8 @@ MERGED is the list of bugs merged with this one."
 			    (cons new new))
 			address))))))
 	  ,@(and debbugs-gnu-subject
+                 (not (string-match-p debbugs-gnu-summary-keep-subject
+                                      debbugs-gnu-subject))
 		 `((subject ,debbugs-gnu-subject)))))))

 (defun debbugs-gnu-guess-current-id ()

bug#66330: 28.2; Gnus wide reply (S W) changes Subject header when used from Debbugs thread

[Maxim Cournoyer]

Hi Michael,

Michael Albinus <michael.albinus@gmx.de> writes:

> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
>
>> Hi,
>
> Hi Maxim,
>
>> Would someone be able to explain what is responsible for this?  It seems
>> to me it'd be preferable if the subject of the message I'm replying to
>> was kept rather than replaced this way, to preserve context.
>
> FTR, this behavior is due to setting a default subject via
> gnus-posting-styles in debbugs-gnu-summary-mode. Could you please check
> whether the appended patch helps?

I've applied the patch and retried my original reproducer; it passes the
test!  So it seems useful to me already in the current form, thank you!

Another thing that could be nice for the future would be some way for
the user to customize the gnus-posting-styles used by Debbugs.  That way
I believe you could modify its behavior only for some particular
"group", e.g. when sending to 'guix-patches'; but that seems trickier
to implement.

So I think the solution you propose is satisfactory!

-- 
Thanks,
Maxim

bug#66330: 28.2; Gnus wide reply (S W) changes Subject header when used from Debbugs thread

[Michael Albinus]

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> Hi Michael,

Hi Maxim,

>> FTR, this behavior is due to setting a default subject via
>> gnus-posting-styles in debbugs-gnu-summary-mode. Could you please check
>> whether the appended patch helps?
>
> I've applied the patch and retried my original reproducer; it passes the
> test!  So it seems useful to me already in the current form, thank you!
>
> Another thing that could be nice for the future would be some way for
> the user to customize the gnus-posting-styles used by Debbugs.  That way
> I believe you could modify its behavior only for some particular
> "group", e.g. when sending to 'guix-patches'; but that seems trickier
> to implement.
>
> So I think the solution you propose is satisfactory!

Thanks for the feedback. I've applied the patch to the GNU ELPA
repository, and released debbugs 0.37. Shall be visible in a couple of
hours.

Further changes might be possible. But as you've said it is more
complex, because some of the settings in gnus-posting-styles must be
preserved. So we'll better wait for further requests if any.

Closing the bug.

Best regards, Michael.

bug#66330: 28.2; Gnus wide reply (S W) changes Subject header when used from Debbugs thread

[Maxim Cournoyer]

Hi Michael,

Michael Albinus <michael.albinus@gmx.de> writes:

[...]

> Thanks for the feedback. I've applied the patch to the GNU ELPA
> repository, and released debbugs 0.37. Shall be visible in a couple of
> hours.
>
> Further changes might be possible. But as you've said it is more
> complex, because some of the settings in gnus-posting-styles must be
> preserved. So we'll better wait for further requests if any.
>
> Closing the bug.

Excellent, thanks for the prompt resolution :-).

-- 
Thanks,
Maxim