Re: encoding of message-ids

[Daniel Kahn Gillmor <dkg@fifthhorseman.net>]

On Tue 2016-02-16 07:38:09 -0500, David Bremner wrote:
> I spent a little time this morning staring at the code, and it seems
> that all of the message-ids are parsed via g_mime_decode_text, which
> deals with RFC2047 encodings and makes guesses at decoding 8bit
> characters. In practice this means that in the notmuch database all
> headers are UTF-8. Since message-id's are supposed to be printable ascii
> [at least in rfc5322], this seems like not such a terrible decision, but
> I wonder if we should document this potential conversion somewhere?

i think you mean g_mime_utils_header_decode_text, not gmime_decode_text,
right?

What do you think are the potential risks here?

 * if all incoming message-ids are standards-compliant (lower-case
   ascii, with an @ sign in the middle and surrounded by angle-brackets
   [0], then it cannot be interpreted as RFC 2047 text because it does
   not have the leading =? or the trailing ?=, so gmime shouldn't
   translate it.

 * if some incoming message-ids are not standards-compliant, then it's
   possible that they will be transformed into other,
   non-standards-compliant message IDs.  Some of them might even be
   transformed into standards-compliant message-IDs.  for example,
   '=?UTF-8?q?<abc@example.net>?=' will be transformed into
   '<abc@example.net>'.

the main risk, i suppose, is that someone could craft a message with a
different literal Message-ID than an existing message, and could trigger
an otherwise undetectable message ID collision.  This seems not much
worse than the existing (detectable) mesage ID collision problems
notmuch already has.

That said, RFC 2047 suggest that its encodings are only relevant in
places where a "text" token would be used.  Message-ID (and References
and In-Reply-To) are intended to only contain dot-atom-text tokens.  So
probably it would be more correct to avoid applying to these specific
fields.

i dunno that it's a big deal though, given the analysis above.

        --dkg

[0] https://tools.ietf.org/html/rfc5322#section-3.6.4
[1] https://tools.ietf.org/html/rfc2047#section-5