bug#66760: 29.1; [BUG] GB18030 Incorrect Encoding

[Eli Zaretskii <eliz@gnu.org>]

> Cc: 66760@debbugs.gnu.org
> From: Andreas Schwab <schwab@suse.de>
> Date: Thu, 26 Oct 2023 16:20:59 +0200
> 
> On Okt 26 2023, Ruijie Yu wrote:
> 
> > I have noticed that in GB18030 encoding, certain ranges of characters
> > have incorrect encodings.
> >
> > One example is U+217A (SMALL ROMAN NUMERAL ELEVEN).  The expected
> > encoding is 81 36 C5 30 (as can be seen from the GB18030 standard [1]
> > and verified from other programs such as iconv and MySQL), whereas the
> > observed encoding within Emacs is 81 36 C4 39, with a 1-codepoint
> > offset.
> 
> This is a bug in the generation of GB180304.map.  The gb180303.awk
> script assumes that the 4-byte encodings of GB18030 are filling the
> holes in sequence of characters with a 2-byte encoding by Unicode
> codepoint order, but there are some places where codepoints from the PUA
> area are inserted into the sequence.  For example, U+1E3E maps to 81 35
> F4 36, the next codepoint not mapped to a 2-byte code is U+1E40, but
> that maps to 81 35 F4 38, whereas 81 35 F4 37 is the encoding of U+E7C7.
> So the output gets out of sync.

Thanks.  I don't think I understand the issue well enough, so patches
are welcome to fix this problem in the Awk script.