bug#55926: 29.0.50; message.el does not normalize In-Reply-To field from web links

bug#55926: 29.0.50; message.el does not normalize In-Reply-To field from web links

[Ignacio Casso]

Hello,

I've recently replied to an email in an org mail list thread using the
"Reply To" button in lists.gnu.org/archive/..., and a reader (Max, in
CC) brought to my attention that the thread was broken in Thunderbird
and that it was because the In-Reply-To field was not normalized and had
not angle brackets around the message id.

He suggested me to report it as a mu4e bug, but mu4e is built in top of
message.el, and after disabling mu4e and trying the same with message.el
I got the same result, so I guess it's actually an Emacs bug.

I saw that Lars was the author of message.el so I added him in CC too.

Sorry if my report is not clear enough, or if the bug is actually in the
website reply button, I don't really know much about email technical
details.

To reproduce the bug, you can follow these steps:

1) configure Emacs to open mail links (I don't
remember the exact steps to do so now, but I can check it out),

2) visit
https://lists.gnu.org/archive/html/emacs-orgmode/2022-06/msg00226.html
with your browser

3) Click the button that says "reply via email to Ignacio Casso" at the
end of the message.

4) In the email compose buffer, the In-Reply-To field will look like
this:

In-Reply-To: 
DB6PR0601MB208724FE4A1EB6D98A176F03C6A99@DB6PR0601MB2087.eurprd06.prod.outlook.com

but it should look like this:

In-Reply-To: 
<DB6PR0601MB208724FE4A1EB6D98A176F03C6A99@DB6PR0601MB2087.eurprd06.prod.outlook.com>


Best regards,

Ignacio

bug#55926: 29.0.50; message.el does not normalize In-Reply-To field from web links

[Lars Ingebrigtsen]

Ignacio Casso <ignaciocasso@hotmail.com> writes:

> 1) configure Emacs to open mail links (I don't
> remember the exact steps to do so now, but I can check it out),

Yes, that would be helpful to allow reproducing the problem.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#55926: 29.0.50; message.el does not normalize In-Reply-To field from web links

[Ignacio Casso]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Ignacio Casso <ignaciocasso@hotmail.com> writes:
>
>> 1) configure Emacs to open mail links (I don't
>> remember the exact steps to do so now, but I can check it out),
>
> Yes, that would be helpful to allow reproducing the problem.

Sorry, I assumed that there was one standard way to do it and that many
of you would already use Emacs for mail. But now that I see it I may not
be doing it the usual way. My default mail client is still Thunderbird,
but I have configured Firefox to use Emacs to open mailto links. To do
so, I have set the default application for mailto, in Settings -> General
-> Applications, to the following script:

  #!/bin/bash

  # Choose this script as default application for opening mailto links
  # (e.g., in firefox)

  emacsclient -c -e "(progn
      (select-frame-set-input-focus (selected-frame))
      (let ((mu4e-compose-context-policy 'pick-first)) (browse-url \"$@\")))"


I have checked the url that is passed to that script for the example in
my bug report, and it's the following:

"mailto:ignaciocasso@hotmail.com?In-Reply-To=DB6PR0601MB208724FE4A1EB6D98A176F03C6A99%40DB6PR0601MB2087.eurprd06.prod.outlook.com&Subject=Re%3A%20%5BBUG%5D%20org-capture%20autoload%20bug%3F%20%5B9.5.2%20%289.5.2-gfbff08%20%40%20/home/ignacio/.emacs.d/elpa/org-9.5.2/%29%5D"

I have also checked the docstring of `browse-url', and it uses the
function specified by the variable `browse-url-mailto-function' to open
mailto links, whose default value, at leas in my Emacs, is
`browse-url-mail'.

So the bug, if it's indeed a bug, would be that `browse-url-mail' does
not normalize the In-Reply-To field by adding angle brackets around.

Regards,

Ignacio

bug#55926: 29.0.50; message.el does not normalize In-Reply-To field from web links

[Lars Ingebrigtsen]

Ignacio Casso <ignaciocasso@hotmail.com> writes:

>       (let ((mu4e-compose-context-policy 'pick-first)) (browse-url \"$@\")))"
>
> I have checked the url that is passed to that script for the example in
> my bug report, and it's the following:
>
> "mailto:ignaciocasso@hotmail.com?In-Reply-To=DB6PR0601MB208724FE4A1EB6D98A176F03C6A99%40DB6PR0601MB2087.eurprd06.prod.outlook.com&Subject=Re%3A%20%5BBUG%5D%20org-capture%20autoload%20bug%3F%20%5B9.5.2%20%289.5.2-gfbff08%20%40%20/home/ignacio/.emacs.d/elpa/org-9.5.2/%29%5D"

Thanks.  I think the right thing to do here is make message-mail fix up
this, so I've now done so in Emacs 29.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#55926: 29.0.50; message.el does not normalize In-Reply-To field from web links

[Robert Pluim]

>>>>> On Sun, 12 Jun 2022 13:44:31 +0200, Ignacio Casso <ignaciocasso@hotmail.com> said:


    Ignacio> Sorry if my report is not clear enough, or if the bug is actually in the
    Ignacio> website reply button, I don't really know much about email technical
    Ignacio> details.

I took a look at `message-mailto', and it pretty much just inserts
what's been passwed to it, so I suspect itʼs an issue with the website
reply button. One way to check is to put a call to `message' in
`message-mailto' just after the call to `interactive' to log exactly
what's being sent to emacs.

Something like

(message "message-mailto received '%s'" url)

and then take a look in "*Messages*"

Thanks

Robert
-- 

bug#55926: 29.0.50; message.el does not normalize In-Reply-To field from web links

[Ignacio Casso]

> One way to check is to put a call to `message' in `message-mailto'
> just after the call to `interactive' to log exactly what's being sent
> to emacs.
>
> Something like
>
> (message "message-mailto received '%s'" url)
>
> and then take a look in "*Messages*"

I've advised `message-mailto' with a message as the one you suggested,
but it seems that function is not being called. What is being called is
`browse-url' -> `browse-url-mail' -> `compose-mail' -> `message-mail',
already defined in message.el. By that time some fields, like "from" and "to",
have already been extracted from the URL, but the In-Reply-To field is
still in the OTHER-HEADERS argument, pending to be parsed.

> I took a look at `message-mailto', and it pretty much just inserts
> what's been passwed to it, so I suspect itʼs an issue with the website
> reply button.

So yes, it probably just inserts what's been passed to it, and if it's
the reply button the one that should ensure that the In-Reply-To field
is normalized, then the bug is in that side. Still, maybe message.el
could ensure that it's normalized anyway just in case?

Regards,

Ignacio

bug#55926: 29.0.50; message.el does not normalize In-Reply-To field from web links

[Max Nikulin]

On 13/06/2022 19:41, Robert Pluim wrote:
> 
> I took a look at `message-mailto', and it pretty much just inserts
> what's been passwed to it, so I suspect itʼs an issue with the website
> reply button.

Certainly lists.gnu.org should be fixed, but its maintainers are likely 
busy with other activities.

On the other hand mail user agents should be more tolerant to input 
data, so it is better to ensure proper format despite not fully correct 
input. Even an example in (already obsoleted) rfc2368 for mailto: URIs 
has no closing %3e: https://datatracker.ietf.org/doc/html/rfc2368#section-6

I do not use Emacs as a mail client, so I have never tried to setup it 
as a mailto: scheme handler, but I expect that 
etc/emacsclient-mail.desktop was created for such purpose. I do not 
think that Exec values are really safe, but it is another issue.

bug#55926: 29.0.50; message.el does not normalize In-Reply-To field from web links

[Eli Zaretskii]

> Cc: larsi@gnus.org, 55926@debbugs.gnu.org
> Date: Mon, 13 Jun 2022 23:14:39 +0700
> From: Max Nikulin <manikulin@gmail.com>
> 
> On 13/06/2022 19:41, Robert Pluim wrote:
> > 
> > I took a look at `message-mailto', and it pretty much just inserts
> > what's been passwed to it, so I suspect itʼs an issue with the website
> > reply button.
> 
> Certainly lists.gnu.org should be fixed, but its maintainers are likely 
> busy with other activities.

I suggest to write to mailman@gnu.org, that's where you can find the
maintainers of lists.gnu.org.

bug#55926: 29.0.50; message.el does not normalize In-Reply-To field from web links

[Max Nikulin]

On 13/06/2022 23:33, Eli Zaretskii wrote:
>> Date: Mon, 13 Jun 2022 23:14:39 +0700
>> From: Max Nikulin
>>
>> Certainly lists.gnu.org should be fixed, but its maintainers are likely
>> busy with other activities.
> 
> I suggest to write to mailman@gnu.org, that's where you can find the
> maintainers of lists.gnu.org.

They are aware of the problem.

I am not sure to which degree it is expensive to regenerate pages for 
all messages from all mail lists hosted on lists.gnu.org.

I do not mind web sites should be strict concerning links they generate.

On the other hand it is Emacs that sends mails with invalid header. That 
is why I asked to add a workaround for a mistake that can be easily made 
by soft on external sites.

Unsure if it is possible to do something really weird through a 
specially crafted mailto: link (by adding some special headers), but it 
looks like it is possible to add something that sender may not like to 
see in its message. So it is better to sanitize input link parameters 
that are used to generate headers.

P.S. From my opinion lists.debian.org and bugs.debian.org are more 
friendly to mail users than lists.gnu.org and debbugs.gnu.org.

bug#55926: 29.0.50; message.el does not normalize In-Reply-To field from web links

[Robert Pluim]

>>>>> On Tue, 14 Jun 2022 23:11:45 +0700, Max Nikulin <manikulin@gmail.com> said:

    Max> Unsure if it is possible to do something really weird through a
    Max> specially crafted mailto: link (by adding some special headers), but
    Max> it looks like it is possible to add something that sender may not like
    Max> to see in its message. So it is better to sanitize input link
    Max> parameters that are used to generate headers.

Iʼm not aware of any code in Emacs that calls `eval' or similar on
parameters passed to `browse-url' or `message-mailto', but you never
know. Donʼt use Emacs to connect to your bank's website :-)

I think Lars' changes here are enough.

Robert
-- 

bug#55926: 29.0.50; message.el does not normalize In-Reply-To field from web links

[Max Nikulin]

On 14/06/2022 23:27, Robert Pluim wrote:
>>>>>> On Tue, 14 Jun 2022 23:11:45 +0700, Max Nikulin said:
> 
>      Max> Unsure if it is possible to do something really weird through a
>      Max> specially crafted mailto: link (by adding some special headers), but
>      Max> it looks like it is possible to add something that sender may not like
>      Max> to see in its message. So it is better to sanitize input link
>      Max> parameters that are used to generate headers.
> 
> Iʼm not aware of any code in Emacs that calls `eval' or similar on
> parameters passed to `browse-url' or `message-mailto', but you never
> know. Donʼt use Emacs to connect to your bank's website :-)

Actually I did not thought about eval as elisp. I do not like shell 
command in emacsclient-mail.desktop, but this time I wrote about adding 
something suspicious to email messages. However there no way to protect 
against honeypots as Cc aimed to put sender into spammer blocking lists.

> I think Lars' changes here are enough.

I thank Lars for the fix.

There is e.g. References header for the same purpose of proper 
threading, but it may contain list of Message-IDs and there is no 
example of improper format at some site.

I expected something more general e.g. similar to file local variables 
that may be safe or not and sanitizer map for particular headers. It may 
be postponed till next bug report.