bug#33612: Gnus doesn’t display EBCDIC encoded mails correctly

bug#33612: Gnus doesn’t display EBCDIC encoded mails correctly

[Thomas Schneider]

If you read this subject and ask yourself “What in the name of—?!”, yes,
exactly.  This is cursed and should really not be a real-world issue,
but I still consider it a bug.

When viewing a mail whose headers and/or body is EBCDIC encoded, Gnus
does not decode it correctly.

Given this mail …

```
From: =?EBCDIC-INT?B?44iWlIGiQOKDiJWFiYSFmQ==?= <qsx@qsx.re>
To: =?EBCDIC-INT?B?44iWlIGiQOKDiJWFiYSFmQ==?= <qsx@qsx.re>
Date: Tue, 04 Dec 2018 16:55:15 +0100
Subject: =?EBCDIC-INT?B?44iJokCipIKRhYOjQImiQImVQMXCw8TJww==?=
Content-Type: text/plain; charset="EBCDIC-INT"
Content-Transfer-Encoding: base64
Message-ID: <F7A3S3POYB.3P9ZHF8WLUSKQ@coruscant.chaotikum.eu>

44iFQJikiYOSQIKZlqaVQIaWp0CRpJSXokCWpYWZQKOIhUCTgamoQISWh0sl
```

the Summary buffer shows (with gnus-summary-line-format "%U%R
%&user-date; %(%[%5k: %-23,23f%]%)%B%s\n") …

```
 .           T16:55:15 [ 1.0k: =?EBCDIC-INT?B?44iWlIGi]  =?EBCDIC-INT?B?44iJokCipIKRhYOjQImiQImVQMXCw8TJww==?=
```

and the Article buffer shows …

```
From: =?EBCDIC-INT?B?44iWlIGiQOKDiJWFiYSFmQ==?= <qsx@qsx.re>
Sender: qsx@chaotikum.eu
To: =?EBCDIC-INT?B?44iWlIGiQOKDiJWFiYSFmQ==?= <qsx@qsx.re>
Subject: =?EBCDIC-INT?B?44iJokCipIKRhYOjQImiQImVQMXCw8TJww==?=
Date: Tue, 04 Dec 2018 16:55:15 +0100 (32 minutes, 23 seconds ago)
Message-ID: <F7A3S3POYB.3P9ZHF8WLUSKQ@coruscant.chaotikum.eu>

\343\210\205@\230\244\211\203\222@\202\231\226\246\225@\206\226\247@\221\244\224\227\242@\226\245\205\231@\243\210\205@\223\201\251\250@\204\226\207K%
```

This was copy-pasted from a terminal Emacs, it shows the raw bytes
escaped as octal numbers.

So while the decoding routines work fine on more common charsets like
UTF-8 and ISO 8859-1, they fail on EBCDIC.  Not something likely to be
encountered in real life, but still.


Gnus v5.13
GNU Emacs 26.1 (build 2, x86_64-pc-linux-gnu, GTK+ Version 3.24.1)
 of 2018-10-25

bug#33612: Gnus doesn’t display EBCDIC encoded mails correctly

[Eli Zaretskii]

> From: Thomas Schneider <qsx@chaotikum.eu>
> Date: Tue, 04 Dec 2018 17:34:16 +0100
> 
> If you read this subject and ask yourself “What in the name of—?!”, yes,
> exactly.  This is cursed and should really not be a real-world issue,
> but I still consider it a bug.
> 
> When viewing a mail whose headers and/or body is EBCDIC encoded, Gnus
> does not decode it correctly.
> 
> Given this mail …
> 
> ```
> From: =?EBCDIC-INT?B?44iWlIGiQOKDiJWFiYSFmQ==?= <qsx@qsx.re>
> To: =?EBCDIC-INT?B?44iWlIGiQOKDiJWFiYSFmQ==?= <qsx@qsx.re>
> Date: Tue, 04 Dec 2018 16:55:15 +0100
> Subject: =?EBCDIC-INT?B?44iJokCipIKRhYOjQImiQImVQMXCw8TJww==?=
> Content-Type: text/plain; charset="EBCDIC-INT"
> Content-Transfer-Encoding: base64
> Message-ID: <F7A3S3POYB.3P9ZHF8WLUSKQ@coruscant.chaotikum.eu>
> 
> 44iFQJikiYOSQIKZlqaVQIaWp0CRpJSXokCWpYWZQKOIhUCTgamoQISWh0sl
> ```
> 
> the Summary buffer shows (with gnus-summary-line-format "%U%R
> %&user-date; %(%[%5k: %-23,23f%]%)%B%s\n") …
> 
> ```
>  .           T16:55:15 [ 1.0k: =?EBCDIC-INT?B?44iWlIGi]  =?EBCDIC-INT?B?44iJokCipIKRhYOjQImiQImVQMXCw8TJww==?=
> ```

Does it help to evaluate the below?

  (define-coding-system-alias 'ebcdic-int 'ebcdic-us)

bug#33612: Gnus doesn’t display EBCDIC encoded mails correctly

[Andreas Schwab]

On Dez 04 2018, Thomas Schneider <qsx@chaotikum.eu> wrote:

> So while the decoding routines work fine on more common charsets like
> UTF-8 and ISO 8859-1, they fail on EBCDIC.  Not something likely to be
> encountered in real life, but still.

Looks like Emacs doesn't know about IBM038 aka EBCDIC-INT.  Other EBCDIC
encodings should work, though.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#33612: Gnus doesn’t display EBCDIC encoded mails correctly

[Eli Zaretskii]

> From: Andreas Schwab <schwab@linux-m68k.org>
> Date: Tue, 04 Dec 2018 19:03:45 +0100
> Cc: 33612@debbugs.gnu.org
> 
> On Dez 04 2018, Thomas Schneider <qsx@chaotikum.eu> wrote:
> 
> > So while the decoding routines work fine on more common charsets like
> > UTF-8 and ISO 8859-1, they fail on EBCDIC.  Not something likely to be
> > encountered in real life, but still.
> 
> Looks like Emacs doesn't know about IBM038 aka EBCDIC-INT.  Other EBCDIC
> encodings should work, though.

Right, I just added IBM038 on master.

bug#33612: Gnus doesn’t display EBCDIC encoded mails correctly

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Andreas Schwab <schwab@linux-m68k.org>
>> Date: Tue, 04 Dec 2018 19:03:45 +0100
>> Cc: 33612@debbugs.gnu.org
>> 
>> On Dez 04 2018, Thomas Schneider <qsx@chaotikum.eu> wrote:
>> 
>> > So while the decoding routines work fine on more common charsets like
>> > UTF-8 and ISO 8859-1, they fail on EBCDIC.  Not something likely to be
>> > encountered in real life, but still.
>> 
>> Looks like Emacs doesn't know about IBM038 aka EBCDIC-INT.  Other EBCDIC
>> encodings should work, though.
>
> Right, I just added IBM038 on master.

And I used the test case posted here, and Gnus indeed now displays this
message correctly.  :-)

So I'm closing the bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no