Handling mislabeled emails encoded with Windows-1252

[Sebastian Poeplau <sebastian.poeplau@eurecom.fr>]

[-- Attachment #1: Type: text/plain, Size: 1621 bytes --]

Hi,

This email is to suggest a minor change in how notmuch handles text
encoding when displaying emails. The motivation is the following: I keep
receiving emails that are encoded with Windows-1252 but claim to be
ISO 8859-1. The two character sets only differ in the range between 0x80
and 0x9F where Windows-1252 contains special characters (e.g. “quotation
marks”) while ISO 8859-1 only has non-printable ones. The mislabeling
thus causes some special characters in such emails to be displayed with
a replacement symbol for non-printable characters.

Of course, it would be best to fix the problem on the sender's side,
making their mail client declare the encoding correctly. However,
sometimes this is just not possible and we need to make do with what we
receive. The change I would thus like to suggest is to always treat
ISO 8859-1 as Windows-1252; since the former only contains non-printable
characters in the range where the two differ, we would not lose any
printable information. According to Wikipedia, this substitution is
common in email clients and browsers because of the frequent
mislabeling [1].

Attached you find a simple patch that illustrates my suggestion. While
it works well for my limited use cases, it's obviously not entirely
reliable. Does anyone have a good idea how to better handle the issue? I
searched GMime for related functionality but didn't quite find what I
was looking for. Do you feel that the issue should be raised with the
GMime people instead?

Best regards,
Sebastian

[1] https://en.wikipedia.org/wiki/ISO/IEC_8859-1#Windows-1252


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: charset.patch --]
[-- Type: text/x-patch, Size: 609 bytes --]

diff -ura notmuch-0.27/notmuch-show.c notmuch-0.27-patched/notmuch-show.c
--- notmuch-0.27/notmuch-show.c	2018-06-13 03:42:34.000000000 +0200
+++ notmuch-0.27-patched/notmuch-show.c	2018-07-11 10:32:56.000456518 +0200
@@ -291,6 +291,8 @@
     charset = g_mime_object_get_content_type_parameter (part, "charset");
     if (charset) {
 	GMimeFilter *charset_filter;
+	if (!strcmp(charset, "iso-8859-1"))
+	    charset = "CP1252";
 	charset_filter = g_mime_filter_charset_new (charset, "UTF-8");
 	/* This result can be NULL for things like "unknown-8bit".
 	 * Don't set a NULL filter as that makes GMime print