From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Mark Lillibridge Newsgroups: gmane.emacs.bugs Subject: bug#13328: 24.2; Rmail does not properly decode MIME messages containing "From " lines or save such attachments correctly Date: Mon, 10 Oct 2022 11:00:44 -0700 Message-ID: <8735bvlfsj.fsf@alum.mit.edu> References: <8738ykvb8s.fsf@foil.strangled.net> Reply-To: mdl@alum.mit.edu Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="31964"; mail-complaints-to="usenet@ciao.gmane.io" Cc: eliz@gnu.org, 13328@debbugs.gnu.org To: Lars Ingebrigtsen Original-X-From: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Mon Oct 10 21:14:39 2022 Return-path: Envelope-to: geb-bug-gnu-emacs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1ohyEZ-00088X-3v for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 10 Oct 2022 21:14:39 +0200 Original-Received: from localhost ([::1]:60392 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ohyEY-0000SE-00 for geb-bug-gnu-emacs@m.gmane-mx.org; Mon, 10 Oct 2022 15:14:38 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:60822) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ohx6J-0007im-0f for bug-gnu-emacs@gnu.org; Mon, 10 Oct 2022 14:02:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:50775) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1ohx6I-0005Nk-7Y for bug-gnu-emacs@gnu.org; Mon, 10 Oct 2022 14:02:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1ohx6H-0001x1-W5 for bug-gnu-emacs@gnu.org; Mon, 10 Oct 2022 14:02:02 -0400 X-Loop: help-debbugs@gnu.org Resent-From: Mark Lillibridge Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 10 Oct 2022 18:02:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 13328 X-GNU-PR-Package: emacs Original-Received: via spool by 13328-submit@debbugs.gnu.org id=B13328.16654248637410 (code B ref 13328); Mon, 10 Oct 2022 18:02:01 +0000 Original-Received: (at 13328) by debbugs.gnu.org; 10 Oct 2022 18:01:03 +0000 Original-Received: from localhost ([127.0.0.1]:49847 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ohx5K-0001vR-U0 for submit@debbugs.gnu.org; Mon, 10 Oct 2022 14:01:03 -0400 Original-Received: from mail-mw2nam04on2070.outbound.protection.outlook.com ([40.107.101.70]:37359 helo=NAM04-MW2-obe.outbound.protection.outlook.com) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ohx5J-0001uq-Eg for 13328@debbugs.gnu.org; Mon, 10 Oct 2022 14:01:02 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=isoj5k9Sawr2iuR3ezeSIQOt4udvBP9yoPuGW4zMMw0ZtJpcUiXZ2PP8VwVO3QDjbvXCe7oRofxLnZiYiPwlF2kz4Lo2a0ARRSz+vYCjbzbVXqbmke+iP3vx3+hSa12h7jsZzVC6BG77esrGSWXUAscOXSlnQOqFPn+8m4WLL8RmGXZSx33FGoOUUXMy6msYH8RlzVoHoxbKhyFf+7qIg1eB/ZF+LBKo7lQTn44ZMwmbOyw0Jdb+DmOg+9oNTqJ5KIDxP/OKSt26joz+U+/MmH3ef9DvN+AtsmlqmmvMr06ldt6QWgxcxrF+rFv7sNGT7gaX58MXWkvl9d706SjMFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8ChXp0rRISCtjknuFMVQSyJzhg+4wAcEoq4E3Ca+ddc=; b=c7ncqQzmIL5ZO+WUkdLiD4REAf94jpfzpLjUnuycJfi03K8aBrvjDd+jgflYOrcCmSal7JYr2AtCAJWhzaHAaPHRc3NWhMBkMtrYimt6DhzYfNAIRqJgLEElDdRgcf1/bpzCfYFXQV9XDbTOSGqh0vK07TZS/bXpKFk7gJWu5/jVI3Pi0XvcrEy+kiffvQYahBedGGyExbVW0tX1wMsIOXKuNVMOWc4RWOa1LYTnQIESNUWNidUFe8A0kBYZn185/7Lz2dkDgGFiTd9cuUcHXXrVVbiufJbDCSRbdSGwZGaIoc/yRxEkX4m6oyBrCvgXilDTYeLnMUImp+TeJVEqSQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 18.7.68.33) smtp.rcpttodomain=gnus.org smtp.mailfrom=alum.mit.edu; dmarc=pass (p=none sp=none pct=100) action=none header.from=alum.mit.edu; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alum.mit.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8ChXp0rRISCtjknuFMVQSyJzhg+4wAcEoq4E3Ca+ddc=; b=KUVUI2zAbZHEV1agABx8yWQ8GJI3qx6e0Ofef2MzkK6b02u0v768OOis5X0XR+g5qPPnmz+1n6I21U/+psTD197JNUpzrMMXTDy6RPwwSVvcFOObAcBeVI35LVT6RpCHiGq6YD5zJAyT0HgfdUmWUclaaRC/5omB1EKM9UUATEQ= Original-Received: from BN0PR04CA0189.namprd04.prod.outlook.com (2603:10b6:408:e9::14) by BY5PR12MB5013.namprd12.prod.outlook.com (2603:10b6:a03:1dc::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5709.15; Mon, 10 Oct 2022 18:00:50 +0000 Original-Received: from BN1NAM02FT048.eop-nam02.prod.protection.outlook.com (2603:10b6:408:e9:cafe::8) by BN0PR04CA0189.outlook.office365.com (2603:10b6:408:e9::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5709.15 via Frontend Transport; Mon, 10 Oct 2022 18:00:48 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 18.7.68.33) smtp.mailfrom=alum.mit.edu; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=alum.mit.edu; Received-SPF: Pass (protection.outlook.com: domain of alum.mit.edu designates 18.7.68.33 as permitted sender) receiver=protection.outlook.com; client-ip=18.7.68.33; helo=outgoing-alum.mit.edu; pr=C Original-Received: from outgoing-alum.mit.edu (18.7.68.33) by BN1NAM02FT048.mail.protection.outlook.com (10.13.2.157) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5709.10 via Frontend Transport; Mon, 10 Oct 2022 18:00:48 +0000 Original-Received: from foil (c-71-198-24-3.hsd1.ca.comcast.net [71.198.24.3]) (authenticated bits=0) (User authenticated as mdl@ALUM.MIT.EDU) by outgoing-alum.mit.edu (8.14.7/8.12.4) with ESMTP id 29AI0iOU029317 (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 10 Oct 2022 14:00:46 -0400 In-Reply-To: <87r1o0fezn.fsf@gnus.org> (message from Lars Ingebrigtsen on Tue, 08 Dec 2020 17:18:36 +0100) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1NAM02FT048:EE_|BY5PR12MB5013:EE_ X-MS-Office365-Filtering-Correlation-Id: 0c6e1c37-bfe9-4038-599b-08daaae95c76 X-LD-Processed: 3326b102-c043-408b-a990-b89e477d582f,ExtAddr,ExtFwd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7mc/ul5lb8WJq5ZvJ0QAGvQigDvIWI7cuz20AUPlLqD5JkZOILDmYt30KMNP76zedRoBt5Sw2LUr/Y5vdlQBDzp6LvLERNVuVjTBcYXXppdpSTwdQreidE8rgTefbNEE/MHqvQhJs0I4sSVmYmDRK/u3aJ3cR1Biee0aiOYkMxafOIYEg2OqcSior0XT51QhZReol+UxXfm0jL9wjnSivAKOXWSPZjW+NRjCXUz9ZqoNpQ2Spa+Y1RJ7KrZ+PZXUNis88XyOYU1vhdpTmSTNaPqOmDyzNrA5fdLahAXuWyNCjrM89oH2JgCZ51anf8fWsvjHNgBoWYDRLhfh6fhujhlvNC/oUGW2zLT0XW5Z78ygf6Y8+5pzvi3q+YPY4aLbNxespk952EqdQE2jHQB8X9szm2nl8p0XmGRfiBJpPPuCXzzURXugVewFjQ6w7lZMkb37DtdOKeXP/p8ToGKys8gHaseS6pIQYmNtSW0QEabqz2WXClIx6foAklajy4mPkm6bX176taM6vSGE0k0StAX418njKBLilgQu93VKteclL7J6D70UXyBQs9DZXQTMZPd0d/WAh53yEZvNzJrAj2X9c4/LTOKLzoZyGqQW0NH56evlzFoND7bh20PCkBQogeje8ErsatGJLCQYya1zBqz+aD8jtEiOG+addvk4P4WnklYR0KhMusVTNtzGXdSytwTa68chw0ze5NBu6yKNVw== X-Forefront-Antispam-Report: CIP:18.7.68.33; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:outgoing-alum.mit.edu; PTR:outgoing-alum.mit.edu; CAT:NONE; SFS:(13230022)(376002)(39860400002)(346002)(396003)(136003)(451199015)(36840700001)(46966006)(786003)(316002)(478600001)(966005)(66899015)(70586007)(70206006)(6916009)(4326008)(8676002)(82310400005)(5660300002)(41300700001)(235185007)(26005)(8936002)(47076005)(3450700001)(2906002)(956004)(336012)(186003)(2616005)(15650500001)(356005)(83380400001)(36860700001)(75432002)(82740400003)(7596003)(41320700001)(86362001)(40480700001); DIR:OUT; SFP:1101; X-OriginatorOrg: alum.mit.edu X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Oct 2022 18:00:48.0927 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0c6e1c37-bfe9-4038-599b-08daaae95c76 X-MS-Exchange-CrossTenant-Id: 3326b102-c043-408b-a990-b89e477d582f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3326b102-c043-408b-a990-b89e477d582f; Ip=[18.7.68.33]; Helo=[outgoing-alum.mit.edu] X-MS-Exchange-CrossTenant-AuthSource: BN1NAM02FT048.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB5013 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-gnu-emacs@gnu.org List-Id: "Bug reports for GNU Emacs, the Swiss army knife of text editors" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-gnu-emacs-bounces+geb-bug-gnu-emacs=m.gmane-mx.org@gnu.org Original-Sender: "bug-gnu-emacs" Xref: news.gmane.io gmane.emacs.bugs:245039 Archived-At: --=-=-= Content-Type: text/plain > From: Lars Ingebrigtsen > To: Eli Zaretskii > Cc: mdl@alum.mit.edu, 13328@debbugs.gnu.org > Subject: Re: bug#13328: 24.2; Rmail does not properly decode MIME messages > containing "From " lines or save such attachments correctly > Date: Tue, 08 Dec 2020 17:18:36 +0100 > In-Reply-To: <83lfe8714d.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 08 Dec > 2020 17:45:22 +0200") > > Eli Zaretskii writes: > > > AFAIK, mbox format requires that every new message begins with a line > > that starts with "From " (sorry, I forgot the space in my previous > > message). See, for example, this page: > > > > https://www.loc.gov/preservation/digital/formats/fdd/fdd000383.shtml > > The mbox format is more restrictive than just that. > `message-unix-mail-delimiter' is a regexp that matches these lines. First, I wanted to report that this bug is still present in Emacs 28.1. Start emacs 28.1 -q, find file the gunziped version of the attachment in this message (should be same mbox as quoted one in original report), then do M-x rmail-mode. You will see the last 2 messages w/ extra >'s on the Froms, which continues when you save the attachments. Back to the discussion. I think the confusion here is that there are multiple levels of quoting. At the outermost level there is quoting of messages to store them in the mbox file. This is what adds the >'s in front of from lines. Without this quoting, a single message containing From lines would be decoded as multiple messages. Because messages in mbox files are separated by a blank line followed by a line started with "From ", quoting traditionally on Unix for mbox (e.g., fetchmail, getmail) is adding a > in front of any line matching regexp "^>*From ". When extracting the messages, un-quoting must be done by removing one > from each line in the message matching that regular expression. Why can there be multiple >'s? Because we have to safely quote message lines like ">From ". There are some other formats of mbox that do not do this quoting in a data preserving manner. They should not be relevant except when importing mbox files (as opposed to receiving messages) from other mailing systems. MIME may additionally quote parts of its messages internally as well. Note that the mbox quoting is not part of the messages and is never supposed to be seen by the user or exposed when sending messages -- ideally, it should be removed whenever a message is read from the mbox reader component. Rmail, unfortunately due to bugs, does not properly do this. What should happen is that the current message is unquoted and copied to a seperate view buffer, which is then displayed, possibly after decoding various MIME or other quoting (e.g., base 64). What I'm guessing is happening for MIME is that the region of the mbox containing the message body is being passed directly to the MIME decoder. This likely explains bug #10080 as well as the blank line of the message separator is not being correctly excluded. You can look at https://en.wikipedia.org/wiki/Mbox for more on mbox formats; we are talking about mboxrd format in that article's terms. - Mark --=-=-= Content-Type: application/gzip Content-Disposition: attachment; filename=mbox_13328.gz Content-Transfer-Encoding: base64 H4sICF5YRGMAA21ib3hfMTMzMjgA3VdtT+NIEv6Mf0Vpvyyc6Kb97oQNUoZ4Fm4TiOLMzo1GK9Sx O0nv+CXY7Qy5X3/VTmAIJHCMZnW6NRa2uzpd9TxVrn78viwyyJIU4KNI4DpWYHlgsrYZtN0ALGaa xr/IaNC97JPueDy6fPdhHEZtICR8T/B4MP4Wfvp4Peqh6R84Fi14RiLFVV214ao4hiouStEhDvWh FLe1LEXScSkDJSpVdd51P4XRDWPHo/PfezeXVze9q+hj/2YQ9o6Ng9HwZtAdn1/gDG0GXqsiFbzM O3XOl1ymfJIKWIqykkXesalNTeM9omrDgJdfoC/TVE5KmcwE/JLhCE0fj3B1BvMFjYvszBgXbfhl Us/ILK+JyHhcre34SItydmZE9eRPEas2WOjmFMoMvRORSEXiuixFrkgmqorjujxJKhB3quTw89nP FagCdEwVpdTocSXaEHF1DJa1phxpBtNqM7ft+kCYz5gxEot0RZqQXo7aMJDoUsFXqebAYaSDgvs4 DtVcVlDkAs1pCklxBHmhIC5yxWUu8xkMLgchqDlXxmawglTmQv//ItAgYFqkafEV57YN40yjgFQs RQqmcfb40cLHx8+20aRhXVzjWsA/eQ7Q4LT8tulr1LbRE6nE3ImkgfoM5Eyj0TiRj1jgzKQNU71q YyCLYkFTOiuKWSr0LPjsO9S0XIqnyYI/jIPJCuPHJSpMRY6zEpqLDVfD66ENh1Oh4nmzmKdLJzgy DqZFiaRj2E0IaRHzdF5UeHtYSb0GScpicXSqQR0DMxtgGsw3aIQFjMHhMBofPQocYzEZdVr6bK1j iAbjIcgEbltxtfACFjj2rchPDdgcjYtnHliw5QHfwSc+XIt6LYq/2PYya2Wl23Ic22t5y6SmrYCa tusz32JWYNneE7/PoT1xPBKqLnMy5Gr+apk+yR9P64xo2isRk1KkfEVMi2ZSUZHUcNjtfxgQ3Vii 8JyMwn73kzYPLsc07H2g8NkMqE+9gNrWH0cPQSP27O5xNTTgQ40+0vCLoJKOZzq26bFlsqS+rymy KTP1ietbPmXBw3KH9z1l3I+W5kkU9Zc2xHIxF2XnenwRjo7eztd+DqqY57koib+HhOi8e3UVjtC+ YeEbCWYLSdCV/gqpOts0OAk01ODoETuanJKZv438L0tm+cx2n74EeuX7hc5On5el26DVhe82vWsL Z8YTXnI6X6R0XQ1w+Hzos9nCokUsGJ2zC88OgqJVxnMlYhg07U63s1+xuX7lq6Ptwu+61MU8W5bt 0nctz7FD5m5Q7IcBh+GOpC2wE/B0C8zzoc/mugU1lw2YHSxgJhzqnGwuF8M+GXZhpPO2MztuVzDL tFuOcfBSbfYuQjKKuqQbRpbrkeiiCxOJuyw+6G1STledq+vxsz73WorXBe2tC/opK5NapglJxETy nJi03oL5klEzFVDLYdTe8LSD4IYg+2Rz2fB0UU92suT0pphoFrinxsEeEM4GxPfLhJ2sfdMHWtY8 7MGPNlvtENWAzAEj4kpx3HpQNmw2e6UXfsH1Rjns6zX3qNYvgyCXPWzKt9lcLvxZtrr7N51W0/Vy LyTkzNCCgPy+Lq82YGs0zhGAFjfj1UJobHfqZJEiogdDT1aYNamaX3yDdQpTmYqcZ6IzbcSPulN7 1KQ+PpAfoy9spKJts7+jvriH9r36wvRazsv6ovGAN6/qC18zwLa9TBytL1BdWKa1TL7qaWt94Viu zRzzpf1yl+P/M31Rm5W0nJYZ+A7Kqzli3xYYeKOr5wcJjN2EvS4w7FcEhr1DYOzckN8uMPqOP6yZ zWzPf+vu4zVorY3A6Nb4YZIrGXPdc7A8qzpV+KlZoSBAE61EuZSxoDLRs6ZSlH+tJrG/Q5OEFpKs NYll0XfO+72aZAv5bk2S8UqJMq+ewHk+qHdb7BABdX6QKOk73foWMfi+9z8RJeuXgP3lomQnw2+Q JVj5f9rMDFzT2SVLGhj3Hwt/G1myRuXvkSVznrPsLgh+kCzJsAPIBS/VSSbvRHIKk6LOsbhXnZ86 BP9+ekF5EFQehDTT9osdw5gUCcpTdGH8F9O/Rxu9Uf5sYtDR/wediOc5xxMAAA== --=-=-=--