bug#44307: 27.1; UTF-8 parts transferred as 8bit in multipart messages fail to decode

[Alexandre Duret-Lutz <adl@lrde.epita.fr>]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> What about the following patch?
>
> @@ -1351,7 +1351,8 @@ nnmaildir-request-article
> -	(nnheader-insert-file-contents nnmaildir-article-file-name))
> +	(let ((nnheader-file-coding-system nnmail-file-coding-system))
> +	  (nnheader-insert-file-contents nnmaildir-article-file-name)))

I was playing with something similar this morning:

@@ -1351,7 +1351,9 @@ nnmaildir-request-article
	(throw 'return nil))
       (with-current-buffer (or to-buffer nntp-server-buffer)
	(erase-buffer)
-	(nnheader-insert-file-contents nnmaildir-article-file-name))
+       (mm-disable-multibyte)
+	(let ((coding-system-for-read mm-text-coding-system))
+	  (nnheader-insert-file-contents nnmaildir-article-file-name)))
       (cons gname num-msgid))))

mm-text-coding-system and nnmail-file-coding-system both default
to 'raw-text.

Without (mm-disable-multibyte), the patch makes no difference to me.

The documentation for 'raw-text on
https://www.gnu.org/software/emacs/manual/html_node/emacs/Coding-Systems.html
states that 'raw-text causes enable-multibyte-characters to be set to
nil, but it's not clear when this should occur, and printing
enable-multibyte-characters after the call to
nnheader-insert-file-contents still shows t.

Adding (mm-disable-multibyte) to the patch seems help a lot, although
the first impression is much worse:

1. When a mail is first displayed (using RET or g), the article buffer
   is unibyte with all non-ascii characters displayed as backslash
   sequences.  This occurs for all mails, even QP-encoded ones.

2. When a mail is displayed for the second time (using g on the same
   article or RET to change article and come back), the display is
   *perfect*.  I.e., plain/text and plain/html parts that are encoded
   with either utf-8 or windows-1252 are correctly displayed for me.

3. Running M-x gnus-backlog-shutdown gets me back to 1. where
   all non-ascii characters are displayed as backslashes.


PS: all of this is with an updated emacs 28, including the reverted
mm-with-part change.

--
Alexandre Duret-Lutz