bug#67638: Description of the variable message-mail-user-agent in Emacs 29.1

bug#67638: Description of the variable message-mail-user-agent in Emacs 29.1

[rameiko87]

I would appreciate clarifying this variable.

I have difficulties understanding what the variable 
message-mail-user-agent means and what the options mean. The description 
for this variable doesn't help:

> Like ‘mail-user-agent’.
I don't know what this sentence means.

> Except if it is nil, use Gnus native MUA; if it is t, use 
> ‘mail-user-agent’.
Please remove Except from the beginning of the sentence, because 
non-native English speakers like myself think that "Except if it is nil" 
means "Whenever not nil". It may be correct, but it's confusing to some 
people. Perhaps the confusion arises because there's a full stop rather 
than a comma.

If the except links the sentence to the one above, then why write "if it 
is t, use ‘mail-user-agent’ ", which is a repetition at this point? Now 
it looks from the description like there should be two options, but 
instead there's five.

I'm totally at loss!

bug#67638: Description of the variable message-mail-user-agent in Emacs 29.1

[Eli Zaretskii]

> Date: Mon, 04 Dec 2023 23:26:39 +0000
> From: rameiko87@posteo.net
> 
> I would appreciate clarifying this variable.

Thanks, I did that on the emacs-29 branch.

> > Like ‘mail-user-agent’.
> I don't know what this sentence means.

It means you should click the link and read on mail-user-agent, before
you continue reading this doc string.  This is the style we use to
avoid repeating the same information in more than one doc string.

> > Except if it is nil, use Gnus native MUA; if it is t, use 
> > ‘mail-user-agent’.
> Please remove Except from the beginning of the sentence, because 
> non-native English speakers like myself think that "Except if it is nil" 
> means "Whenever not nil". It may be correct, but it's confusing to some 
> people. Perhaps the confusion arises because there's a full stop rather 
> than a comma.
> 
> If the except links the sentence to the one above, then why write "if it 
> is t, use ‘mail-user-agent’ ", which is a repetition at this point? Now 
> it looks from the description like there should be two options, but 
> instead there's five.
> 
> I'm totally at loss!

Here's the new doc string:

    "Your preferred mail composition package when reading email with message.el.
  Like `mail-user-agent' (which see), this specifies the mail-sending
  package you prefer.
  The value can be any value accepted by `mail-user-agent', and in
  addition it can be nil or t.  If the value is nil, use the Gnus native
  Mail User Agent (MUA); if it is t, use the value of `mail-user-agent'."

bug#67638: Description of the variable message-mail-user-agent in Emacs 29.1

[rameiko87]

Dear Eli,
Thanks very much for your swift reply. Please understand that I mean no 
disturbance through my questions, I'm only trying to understand. What is 
confusing me still is that mail-user-agent sets the composition package 
for outgoing emails (in my personal configuration this is the Message 
package, in my mind this is the MUA being used when writing emails). The 
message-mail-user-agent variable instead it's not clear what is setting.

> "Your preferred mail composition package when reading email with 
> message.el.

But I read my emails with Rmail. Perhaps you meant "when composing 
emails with message.el"? If yes, I thought that the message package was 
the email composition package and that I assigned this with the variable 
mail-user-agent, so I'm not sure what I'm setting here.

>   Like `mail-user-agent' (which see), this specifies the mail-sending 
> package you prefer.

Now the composition package from the first quote turned into sending 
package! What is a mail sending package? How is this variable the 
"analogous" of mail-user-agent (I really don't like that "Like")?

>   The value can be any value accepted by `mail-user-agent', and in 
> addition it can be nil or t.  If the value is nil, use the Gnus native 
> Mail User Agent (MUA); if it is t, use the value of `mail-user-agent'."

Why is the default the Gnus package? The problem is that I still don't 
understand the difference with mail-user-agent.

bug#67638: Description of the variable message-mail-user-agent in Emacs 29.1

[Eli Zaretskii]

> Date: Tue, 05 Dec 2023 16:22:17 +0000
> From: rameiko87@posteo.net
> Cc: 67638@debbugs.gnu.org
> 
> Dear Eli,
> Thanks very much for your swift reply. Please understand that I mean no 
> disturbance through my questions, I'm only trying to understand. What is 
> confusing me still is that mail-user-agent sets the composition package 
> for outgoing emails (in my personal configuration this is the Message 
> package, in my mind this is the MUA being used when writing emails). The 
> message-mail-user-agent variable instead it's not clear what is setting.

message.el basically implements the message-mode, which is a
specialized text-editing mode.  It does not itself deal with
email-specific stuff, like mail headers, the structure of the message
according to the various RFCs, and neither does it handle the sending
process itself.

> > "Your preferred mail composition package when reading email with 
> > message.el.
> 
> But I read my emails with Rmail. Perhaps you meant "when composing 
> emails with message.el"?

No.

> If yes, I thought that the message package was 
> the email composition package

No, see above.  "Mail User Agent" is separate from message.el.

> and that I assigned this with the variable 
> mail-user-agent, so I'm not sure what I'm setting here.

If you want mail-user-agent to take precedence, you need to customize
message-mail-user-agent to the value t.  That's what the doc string
says.

> >   Like `mail-user-agent' (which see), this specifies the mail-sending 
> > package you prefer.
> 
> Now the composition package from the first quote turned into sending 
> package! What is a mail sending package?

Mail User Agent is both.  See below.

> How is this variable the 
> "analogous" of mail-user-agent (I really don't like that "Like")?

Once again, "like", with a hyperlink after it, invites you to read the
documentation of mail-user-agent.  This is our usual style in doc
strings to avoid repetition.

> >   The value can be any value accepted by `mail-user-agent', and in 
> > addition it can be nil or t.  If the value is nil, use the Gnus native 
> > Mail User Agent (MUA); if it is t, use the value of `mail-user-agent'."
> 
> Why is the default the Gnus package?

Because message.el was originally part of Gnus, and Gnus users expect
that default.

> The problem is that I still don't understand the difference with
> mail-user-agent.

They are two separate options, and users have the freedom which one to
customize and how.

Next iteration of the doc string below.

   "Your preferred package for composing and sending email when using message.el.
 Like `mail-user-agent' (which see), this specifies the package you prefer
 to use for composing and sending email messages.
 The value can be anything accepted by `mail-user-agent', and in addition
 it can be nil or t.  If the value is nil, use the Gnus native Mail User
 Agent (MUA); if it is t, use the value of `mail-user-agent'.
 For more about mail user agents, see Info node `(emacs)Mail Methods'"

bug#67638: Description of the variable message-mail-user-agent in Emacs 29.1

[rameiko87]

Dear Eli,
Please understand that I'm really trying my best to get this variable, 
which remains obscure and mysterious despite my technical proficiency, 
which I consider to be above-average. This is the only variable which I 
currently have a problem with (as far as the Mail group is concerned).

>   "Your preferred package for composing and sending email when using 
> message.el.

I read very attentively your previous message. mail-user-agent is the 
package which is started when, say, I click on a mailto link. In my 
case, this is the Message package, which as far as I understand is 
exactly message.el

>  Like `mail-user-agent' (which see), this specifies the package you 
> prefer to use for composing and sending email messages.

Then, I understand that while we're using message.el, and we're done 
writing the email, we need to send. Is it here that 
message-mail-user-agent intervenes?

>  The value can be anything accepted by `mail-user-agent', and in 
> addition it can be nil or t. If the value is nil, use the Gnus native 
> Mail User Agent (MUA); if it is t, use the value of `mail-user-agent'.

This doesn't check out, even just because mail-user-agent accepts five 
options which are Message, Mail package, MH, Message_withGnusfeatures, 
but instead here we miss the options for "Message package" and 
"Message_withGnusfeatures", so message-mail-user-agent doesn't offer all 
the options that mail-user-agent offers. One explanation for this would 
be that "Message" doesn't help with the task which 
message-mail-user-agent wants to accomplish, therefore "Message" is 
absent from the options and "Message_withGnusfeatures" becomes "Gnus 
native" in the options. But it can't be the case because then a 
contradiction would follow: say I set the value of 
message-mail-user-agent to t, and the value of mail-user-agent is 
"Message package": this is an absurdity because then "Message" would be 
tasked with what message-mail-user-agent wants to accomplish, and we 
excluded this by hypothesis.

And also setting the value of message-mail-user-agent to nil: is it the 
same as setting the value of message-mail-user-agent to t and the value 
of mail-user-agent to Message_withGnusfeatures?

bug#67638: Description of the variable message-mail-user-agent in Emacs 29.1

[Eli Zaretskii]

> Date: Wed, 06 Dec 2023 23:53:19 +0000
> From: rameiko87@posteo.net
> Cc: 67638@debbugs.gnu.org
> 
> Dear Eli,
> Please understand that I'm really trying my best to get this variable, 
> which remains obscure and mysterious despite my technical proficiency, 
> which I consider to be above-average. This is the only variable which I 
> currently have a problem with (as far as the Mail group is concerned).
> 
> >   "Your preferred package for composing and sending email when using 
> > message.el.
> 
> I read very attentively your previous message. mail-user-agent is the 
> package which is started when, say, I click on a mailto link. In my 
> case, this is the Message package, which as far as I understand is 
> exactly message.el
> 
> >  Like `mail-user-agent' (which see), this specifies the package you 
> > prefer to use for composing and sending email messages.
> 
> Then, I understand that while we're using message.el, and we're done 
> writing the email, we need to send. Is it here that 
> message-mail-user-agent intervenes?

No, it intervenes also _before_ you write the message.  For example,
if you reply to an incoming message (type 'r' in Rmail), the
mail-user-agent's compose function is called to prepare the headers of
the response for you, before you type the body of your response.  Then
the send function of mail-user-agent is invoked when you type the key
to send the response.

I added the link to the Info manual so you could read up on Mail User
Agents and understand what they are for.

> >  The value can be anything accepted by `mail-user-agent', and in 
> > addition it can be nil or t. If the value is nil, use the Gnus native 
> > Mail User Agent (MUA); if it is t, use the value of `mail-user-agent'.
> 
> This doesn't check out, even just because mail-user-agent accepts five 
> options which are Message, Mail package, MH, Message_withGnusfeatures, 
> but instead here we miss the options for "Message package" and 
> "Message_withGnusfeatures", so message-mail-user-agent doesn't offer all 
> the options that mail-user-agent offers.

"Like" doesn't mean "100% identical", it means "similar".  The intent
is for you to read the doc string of that other variable, and then
continue reading the doc string of this variable, with the information
about mail-user-agent in mind.

> And also setting the value of message-mail-user-agent to nil: is it the 
> same as setting the value of message-mail-user-agent to t and the value 
> of mail-user-agent to Message_withGnusfeatures?

I think so, yes.  But I don't use message.el, so I wouldn't know for
sure.  Why won't you try this and report back?

bug#67638: Description of the variable message-mail-user-agent in Emacs 29.1

[rameiko87]

Dear Eli,

Can you please kindly explain in one sentence where 
message-mail-user-agent and mail-user-agent intervene during the process 
of replying to an email message which I'm reading using Rmail?

>> And also setting the value of message-mail-user-agent to nil: is it 
>> the same as setting the value of message-mail-user-agent to t and the 
>> value of mail-user-agent to Message_withGnusfeatures?
> 
> I think so, yes.  But I don't use message.el, so I wouldn't know for 
> sure.  Why won't you try this and report back?

I'm happy to try this, but before I do I must understand what these two 
variables are for.

>> >  The value can be anything accepted by `mail-user-agent', and in addition it can be nil or t.

I still think this sentence doesn't check out, because the value of 
message-mail-user-agent cannot be "Message package" and 
"Message_withGnusfeatures", which instead are accepted values for the 
variable mail-user-agent.

bug#67638: Description of the variable message-mail-user-agent in Emacs 29.1

[Eli Zaretskii]

> Date: Thu, 07 Dec 2023 14:47:20 +0000
> From: rameiko87@posteo.net
> Cc: 67638@debbugs.gnu.org
> 
> Can you please kindly explain in one sentence where 
> message-mail-user-agent and mail-user-agent intervene during the process 
> of replying to an email message which I'm reading using Rmail?

When you type 'r', Emacs invokes rmail-reply, which calls
rmail-start-mail, which calls compose-mail.  compose-mail invokes the
composition function of mail-user-agent, and when you invoke the
command to send the response, Emacs invokes the send-mail function of
mail-user-agent.

rmail-start-mail (and thus compose-mail) is also called by commands
rmail-mail, rmail-continue, rmail-forward, and a couple of others.

bug#67638: Description of the variable message-mail-user-agent in Emacs 29.1

[rameiko87]

>> Can you please kindly explain in one sentence where 
>> message-mail-user-agent and mail-user-agent intervene during the 
>> process of replying to an email message which I'm reading using Rmail?
> 
> When you type 'r', Emacs invokes rmail-reply, which calls 
> rmail-start-mail, which calls compose-mail. compose-mail invokes the 
> composition function of mail-user-agent, and when you invoke the 
> command to send the response, Emacs invokes the send-mail function of 
> mail-user-agent.
> 
> rmail-start-mail (and thus compose-mail) is also called by commands 
> rmail-mail, rmail-continue, rmail-forward, and a couple of others.

Thanks. Can you please explain where the variable 
message-mail-user-agent intervenes in the process above? Because only 
mail-user-agent appears in your description.

bug#67638: Description of the variable message-mail-user-agent in Emacs 29.1

[Eli Zaretskii]

> Date: Thu, 07 Dec 2023 17:27:21 +0000
> From: rameiko87@posteo.net
> Cc: 67638@debbugs.gnu.org
> 
> >> Can you please kindly explain in one sentence where 
> >> message-mail-user-agent and mail-user-agent intervene during the 
> >> process of replying to an email message which I'm reading using Rmail?
> > 
> > When you type 'r', Emacs invokes rmail-reply, which calls 
> > rmail-start-mail, which calls compose-mail. compose-mail invokes the 
> > composition function of mail-user-agent, and when you invoke the 
> > command to send the response, Emacs invokes the send-mail function of 
> > mail-user-agent.
> > 
> > rmail-start-mail (and thus compose-mail) is also called by commands 
> > rmail-mail, rmail-continue, rmail-forward, and a couple of others.
> 
> Thanks. Can you please explain where the variable 
> message-mail-user-agent intervenes in the process above? Because only 
> mail-user-agent appears in your description.

Please take a look at the code of compose-mail.  It is defined in
simple.el.  You will see the answer there.

It might also be educational to look at define-mail-user-agent (in
subr.el), which is the function used to define all the
mail-user-agent's we have in Emacs.  Its doc string describes, among
other things, what constitutes a "mail user agent" in Emacs.

bug#67638: Description of the variable message-mail-user-agent in Emacs 29.1

[Eli Zaretskii]

> Cc: 67638@debbugs.gnu.org
> Date: Thu, 07 Dec 2023 19:40:31 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > > rmail-start-mail (and thus compose-mail) is also called by commands 
> > > rmail-mail, rmail-continue, rmail-forward, and a couple of others.
> > 
> > Thanks. Can you please explain where the variable 
> > message-mail-user-agent intervenes in the process above? Because only 
> > mail-user-agent appears in your description.

Oh, and message-setup binds mail-user-agent to a value derived from
message-mail-user-agent.  Maybe this is the missing piece you were
looking for.

bug#67638: Description of the variable message-mail-user-agent in Emacs 29.1

[rameiko87]

Dear Eli,

> Oh, and message-setup binds mail-user-agent to a value derived from
> message-mail-user-agent.  Maybe this is the missing piece you were
> looking for.

This may be the key. Let's see if I understand correctly. I would be 
immensely grateful if you could check the "decision-making pseudocode" 
below.

CASE A. We're outside of message.el

In this case check the variable mail-user-agent to decide how to create 
the email. mail-user-agent may say to use something different from the 
Message package, in which case message-mail-user-agent will never 
intervene unless we use Gnus, and it doesn't matter what it's set up to.

CASE B. We're inside of message.el

This may happen either because mail-user-agent is set to the Message 
package, so brings us to use message.el, or because we're inside Gnus 
which always calls the Message package no matter the value of 
mail-user-agent.

Now that we're in message.el, first we look at message-mail-user-agent 
and temporarily ignore mail-user-agent:
CASE B.1 It's the Message package, so send the email this way. Note that 
mail-user-agent does not intervene at all here in B.1, except perhaps to 
choose case B instead of case A.
CASE B.2 It's something else from the Message package and is explicitly 
set: use that whatever it is. Note that also in this case then 
mail-user-agent doesn't matter, except perhaps to choose case B instead 
of case A.
Case B.3 It says to look at the value of mail-user-agent. Then if it's 
the message package, this is the same as case B.1 (except that inside 
B.3 we have used the value of mail-user-agent), otherwise use whatever 
other package mail-user-agent says to use.

Do you think this is correct?

bug#67638: Description of the variable message-mail-user-agent in Emacs 29.1

[Eli Zaretskii]

> Date: Fri, 08 Dec 2023 11:41:34 +0000
> From: rameiko87@posteo.net
> Cc: 67638@debbugs.gnu.org
> 
> > Oh, and message-setup binds mail-user-agent to a value derived from
> > message-mail-user-agent.  Maybe this is the missing piece you were
> > looking for.
> 
> This may be the key. Let's see if I understand correctly. I would be 
> immensely grateful if you could check the "decision-making pseudocode" 
> below.
> 
> CASE A. We're outside of message.el
> 
> In this case check the variable mail-user-agent to decide how to create 
> the email. mail-user-agent may say to use something different from the 
> Message package, in which case message-mail-user-agent will never 
> intervene unless we use Gnus, and it doesn't matter what it's set up to.
> 
> CASE B. We're inside of message.el
> 
> This may happen either because mail-user-agent is set to the Message 
> package, so brings us to use message.el, or because we're inside Gnus 
> which always calls the Message package no matter the value of 
> mail-user-agent.
> 
> Now that we're in message.el, first we look at message-mail-user-agent 
> and temporarily ignore mail-user-agent:
> CASE B.1 It's the Message package, so send the email this way. Note that 
> mail-user-agent does not intervene at all here in B.1, except perhaps to 
> choose case B instead of case A.
> CASE B.2 It's something else from the Message package and is explicitly 
> set: use that whatever it is. Note that also in this case then 
> mail-user-agent doesn't matter, except perhaps to choose case B instead 
> of case A.
> Case B.3 It says to look at the value of mail-user-agent. Then if it's 
> the message package, this is the same as case B.1 (except that inside 
> B.3 we have used the value of mail-user-agent), otherwise use whatever 
> other package mail-user-agent says to use.
> 
> Do you think this is correct?

Almost.  In case B.1 we do involve mail-user-agent: message.el binds
its value to what message-mail-user-agent tells it.  IOW, the
lower-level functionality of composing and sending email always looks
at mail-user-agent, so any package that wants to override the value of
mail-user-agent must bind it to some other value.

bug#67638: Description of the variable message-mail-user-agent in Emacs 29.1

[CHENG Gao via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Grepping shows:

,----
| 
| (define-mail-user-agent 'gnus-user-agent 'gnus-msg-mail 'message-send-and-exit 'message-kill-buffer 'message-send-hook)
| (define-mail-user-agent 'message-user-agent 'message-mail 'message-send-and-exit 'message-kill-buffer 'message-send-hook)
| (define-mail-user-agent 'mh-e-user-agent 'mh-user-agent-compose 'mh-send-letter 'mh-fully-kill-draft 'mh-before-send-letter-hook)
| (define-mail-user-agent 'sendmail-user-agent #'sendmail-user-agent-compose #'mail-send-and-exit)
`----

Seems they are all mail-user-agents defined in current Emacs, that
define how to compose mail (using Mail mode in sendmail.el or Message mode
in message.el) and how to send mail (using sendmail in sendmail.el or
smtp in smtpmail.el).

My understanding is

For composing mail, gnus-user-agent is like message-user-agent plus, with some extra
headers and rendering

Other than sending mail, sendmail package is also for composing mail
(Mail mode), as used by RMAIL IIUC.

Not sure about mh-e-user-agent. Seems it uses its own composing mode
other than Mail and Message modes. And uses sendmail package for sending
mail.

Among these four mail-user-agents defined, only gnus-user-agent is
used also as variable (to define what to put in User-Agent header of
mail or news message). All others are only used for property list of
named symbols. And only COMPOSEFUNC & SENDFUNC of sendmail-user-agent
are used in current Emacs codes (for bug reporting).

I'd say it is in fact fairly complicated and to some extend misleading
(esp. sendmail-user-agent serving composition besides delivery).

bug#67638: Description of the variable message-mail-user-agent in Emacs 29.1

[rameiko87]

>> This may be the key. Let's see if I understand correctly. I would be
>> immensely grateful if you could check the "decision-making pseudocode"
>> below.
>> 
>> CASE A. We're outside of message.el
>> 
>> In this case check the variable mail-user-agent to decide how to 
>> create
>> the email. mail-user-agent may say to use something different from the
>> Message package, in which case message-mail-user-agent will never
>> intervene unless we use Gnus, and it doesn't matter what it's set up 
>> to.
>> 
>> CASE B. We're inside of message.el
>> 
>> This may happen either because mail-user-agent is set to the Message
>> package, so brings us to use message.el, or because we're inside Gnus
>> which always calls the Message package no matter the value of
>> mail-user-agent.
>> 
>> Now that we're in message.el, first we look at message-mail-user-agent
>> and temporarily ignore mail-user-agent:
>> CASE B.1 It's the Message package, so send the email this way. Note 
>> that
>> mail-user-agent does not intervene at all here in B.1, except perhaps 
>> to
>> choose case B instead of case A.
>> CASE B.2 It's something else from the Message package and is 
>> explicitly
>> set: use that whatever it is. Note that also in this case then
>> mail-user-agent doesn't matter, except perhaps to choose case B 
>> instead
>> of case A.
>> Case B.3 It says to look at the value of mail-user-agent. Then if it's
>> the message package, this is the same as case B.1 (except that inside
>> B.3 we have used the value of mail-user-agent), otherwise use whatever
>> other package mail-user-agent says to use.
>> 
>> Do you think this is correct?
> 
> Almost.  In case B.1 we do involve mail-user-agent: message.el binds 
> its value to what message-mail-user-agent tells it.  IOW, the 
> lower-level functionality of composing and sending email always looks 
> at mail-user-agent, so any package that wants to override the value of 
> mail-user-agent must bind it to some other value.

Wait up I disagree: case B.1 means that the value of 
message-mail-user-agent is nil, therefore in message-setup the 
expression:

https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/gnus/message.el?id=841614c72272146fff1b9afa05d52150da6d6e66#n6814

is true, therefore we call:

https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/gnus/message.el?id=841614c72272146fff1b9afa05d52150da6d6e66#n6815

and there's no mail-user-agent involved and message.el doesn't touch the 
value of mail-user-agent in case B.1. The value of mail-user-agent is 
changed only in

https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/gnus/message.el?id=841614c72272146fff1b9afa05d52150da6d6e66#n6825

which lies in the part of the "if" that is not run.

bug#67638: Description of the variable message-mail-user-agent in Emacs 29.1

[Eli Zaretskii]

> Date: Fri, 08 Dec 2023 14:37:30 +0000
> From: rameiko87@posteo.net
> Cc: 67638@debbugs.gnu.org
> 
> >> Do you think this is correct?
> > 
> > Almost.  In case B.1 we do involve mail-user-agent: message.el binds 
> > its value to what message-mail-user-agent tells it.  IOW, the 
> > lower-level functionality of composing and sending email always looks 
> > at mail-user-agent, so any package that wants to override the value of 
> > mail-user-agent must bind it to some other value.
> 
> Wait up I disagree

Noted.