From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Alexandre Duret-Lutz Newsgroups: gmane.emacs.bugs Subject: bug#44307: 27.1; UTF-8 parts transferred as 8bit in multipart messages fail to decode Date: Tue, 05 Jan 2021 11:00:58 +0100 Message-ID: <87o8i3g0th.fsf@goulash.lrde.epita.fr> References: <8735zj6q6h.fsf@goulash.lrde.epita.fr> <87h7nwbc6t.fsf@goulash.lrde.epita.fr> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="36729"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) Cc: larsi@gnus.org To: 44307@debbugs.gnu.org Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Tue Jan 05 11:02:17 2021 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1kwjAP-0009Dq-Sk for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 05 Jan 2021 11:02:17 +0100 Original-Received: from localhost ([::1]:41418 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kwjAO-0001FX-H6 for geb-bug-gnu-emacs@m.gmane-mx.org; Tue, 05 Jan 2021 05:02:16 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:41548) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kwjAA-0001E0-RH for bug-gnu-emacs@gnu.org; Tue, 05 Jan 2021 05:02:02 -0500 Original-Received: from debbugs.gnu.org ([209.51.188.43]:57538) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kwjAA-0006du-Hg for bug-gnu-emacs@gnu.org; Tue, 05 Jan 2021 05:02:02 -0500 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kwjAA-0000zP-Ct; Tue, 05 Jan 2021 05:02:02 -0500 X-Loop: help-debbugs@gnu.org Resent-From: Alexandre Duret-Lutz Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org, bugs@gnus.org Resent-Date: Tue, 05 Jan 2021 10:02:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 44307 X-GNU-PR-Package: emacs,gnus Original-Received: via spool by 44307-submit@debbugs.gnu.org id=B44307.16098408703743 (code B ref 44307); Tue, 05 Jan 2021 10:02:02 +0000 Original-Received: (at 44307) by debbugs.gnu.org; 5 Jan 2021 10:01:10 +0000 Original-Received: from localhost ([127.0.0.1]:40851 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kwj9J-0000yJ-NU for submit@debbugs.gnu.org; Tue, 05 Jan 2021 05:01:09 -0500 Original-Received: from mail-wm1-f50.google.com ([209.85.128.50]:52371) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kwj9G-0000xd-68 for 44307@debbugs.gnu.org; Tue, 05 Jan 2021 05:01:09 -0500 Original-Received: by mail-wm1-f50.google.com with SMTP id a6so2460184wmc.2 for <44307@debbugs.gnu.org>; Tue, 05 Jan 2021 02:01:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version; bh=prnbIVH5sEUyAu2Fa/Rr93plI6gU0pjkway9LRV4gVY=; b=gQepVHFdcoAWEi8JuxE/1c4uhd8lepGKPmLVJJJ6G1tooqMCzv0y7ebugzwCMpmPgO lz02x7BJqGaYyVgTVxOKzSGcOjPR7KIxGeiVidcns/ZW/DEsVFQeELtG0iXq7QDnC/Pi aI9lli54KfVAJBNI7MI7vzGOPhiPD6qOb6fMSIzFMEWWinL9KF0fWycZ5mFjPoreV90t KVyOYwmxw5cvFtyNjETyBHF0C5VM87cLxBP36tXvXXpdY/ojlOfhbPskqmwwCM5ayZhc cUFPVgTevn7TfDVYN9LoulhBKsl2Phdql2yxG0T2g3T9/NOB1fLcIW55OgpqUu1jBOt8 3ntg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:references:date :in-reply-to:message-id:user-agent:mime-version; bh=prnbIVH5sEUyAu2Fa/Rr93plI6gU0pjkway9LRV4gVY=; b=gEYGhuXEIc5/CdvW7xwIN6JOFbZCTVfXxDNo/G/HAVaBHTU25oWF2AFTc3LbsC6KZQ sxZzs9ij88/0aLWk/AHfElJXn8c5Mh0AdOIF5X2xhH8Ln+alcsEGMxNa76ziz5xtDn+0 bbh06XRQ4oOFDxAIWBXEVpfNdfpGswCz37T2pDs/xC+S102DrfItajDBVcDuceE/dSvL ZFhVnJO1sbrZIATaD2M3hx9XwGyqgvPfnqkS6sZmBFTCRPfamJtsnH7HKulG9lZ8YCBn XhzLpCex5EmPimlQFg49yR4mcIk7zVvORpoITfYF/FBpk9dfdXz9zAAWGd3j/ZbPbycD Q/8w== X-Gm-Message-State: AOAM533dJnX7+HyMSiCndESQTrprG4mzsauAXEw/uw+/V1vcYkkhcezO 8bJ1prU8VREAsVEQG3uPGR4= X-Google-Smtp-Source: ABdhPJyyx8VMBNp0tBaIxMdFMjHg160BZ0tXFQvq6XssxK9tWvZSrm0RSkYQHQI/RVtZBxx1kUMeeA== X-Received: by 2002:a1c:64c4:: with SMTP id y187mr2761137wmb.165.1609840860128; Tue, 05 Jan 2021 02:01:00 -0800 (PST) Original-Received: from goulash (89-109-190-109.dsl.ovh.fr. [109.190.109.89]) by smtp.gmail.com with ESMTPSA id j59sm101314757wrj.13.2021.01.05.02.00.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Jan 2021 02:00:59 -0800 (PST) In-Reply-To: <87h7nwbc6t.fsf@goulash.lrde.epita.fr> (Alexandre Duret-Lutz's message of "Mon, 04 Jan 2021 22:54:18 +0100") X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:197365 Archived-At: Alexandre Duret-Lutz writes: > In my case the (mm-handle-buffer handle) is multibyte. This > multibyteness was preserved by mm-copy-to-buffer while creating the > handle buffer, but a did not check the original source of it, since the > comment above the macro suggests that having multibyte parts is OK. I think I understand why the bug cannot be reproduced using nndoc in emacs 28. The following patch, which fixes the unibyteness of the nndoc buffer is not part of emacs 27: commit 9d0385d7c7adc810dfd06321b783593b7afb3d58 Author: Lars Ingebrigtsen Date: Fri Aug 21 15:36:45 2020 +0200 Fix problem with 8bit content-transfer-encoding in nndoc mbox files * lisp/gnus/nndoc.el (nndoc-possibly-change-buffer): If we're reading an mbox file, it may contain messages that use content-transfer-encoding 8bit, which means that we have to treat the file as a sequence of byte (bug#42951). This avoids double-decoding -- once by Emacs when inserting the mbox into the buffer, and once by Gnus when displaying the articles. So when mm-with-part process the MIME parts of the problematic message read from the nndoc group, it receives a multibyte buffer in Emacs 27 but a unibyte buffer in Emacs 28. However the issue is not restricted to nndoc. With both emacs 27 & 28 I'm still having the original issue with mail read using nnmaildir. So maybe nnmaildir deserves a similar fix? I'm not quite sure where that would go. nnmaildir-request-article? In any way I'm currently running Gnus with the following change: --- a/lisp/gnus/mm-decode.el +++ b/lisp/gnus/mm-decode.el @@ -1264,8 +1264,8 @@ mm-with-part (when (and (mm-handle-buffer handle) (buffer-name (mm-handle-buffer handle))) (with-temp-buffer - (mm-disable-multibyte) (insert-buffer-substring (mm-handle-buffer handle)) + (mm-disable-multibyte) (mm-decode-content-transfer-encoding (mm-handle-encoding handle) (mm-handle-media-type handle)) and that seems to solve my problem in both version of Emacs, and in both nndoc or nnmaildir. -- Alexandre Duret-Lutz