* UTF16 encoding adds BOM everywhere?
@ 2022-07-14 19:16 Jean Abou Samra
2022-07-20 20:42 ` Mark H Weaver
0 siblings, 1 reply; 3+ messages in thread
From: Jean Abou Samra @ 2022-07-14 19:16 UTC (permalink / raw)
To: Guile User
Hi,
With this code:
(let ((p (open-output-file "x.txt")))
(set-port-encoding! p "UTF16")
(display "ABC" p)
(close-port p))
the sequence of bytes in the output file x.txt is
['FF', 'FE', '41', '0', 'FF', 'FE', '42', '0', 'FF', 'FE', '43', '0']
FFE is a little-endian Byte Order Mark (BOM), fine.
But why is Guile adding it before every character
instead of just at the beginning of the string?
Is that expected?
This is a curiosity question; since what I want is
big endian, I just used "UTF16BE", which outputs
big endian, and doesn't add the BOM -- I can then just
add a BOM manually. Still, I'm puzzled by this
behavior.
Thanks,
Jean
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: UTF16 encoding adds BOM everywhere?
2022-07-14 19:16 UTF16 encoding adds BOM everywhere? Jean Abou Samra
@ 2022-07-20 20:42 ` Mark H Weaver
2022-07-20 21:45 ` Jean Abou Samra
0 siblings, 1 reply; 3+ messages in thread
From: Mark H Weaver @ 2022-07-20 20:42 UTC (permalink / raw)
To: Jean Abou Samra; +Cc: guile-user
Hi,
Jean Abou Samra <jean@abou-samra.fr> wrote:
> With this code:
>
> (let ((p (open-output-file "x.txt")))
> (set-port-encoding! p "UTF16")
> (display "ABC" p)
> (close-port p))
>
> the sequence of bytes in the output file x.txt is
>
> ['FF', 'FE', '41', '0', 'FF', 'FE', '42', '0', 'FF', 'FE', '43', '0']
>
> FFE is a little-endian Byte Order Mark (BOM), fine.
> But why is Guile adding it before every character
> instead of just at the beginning of the string?
> Is that expected?
No, this is certainly a bug. It sounds like the
'at_stream_start_for_bom_write' port flag is not being cleared, as it
should be, after the first character is written. I suspect that it
worked correctly when I first implemented proper BOM handling in 2013
(commit cdd3d6c9f423d5b95f05193fe3c27d50b56957e9), but the ports code
has seen some major reworking since then. I guess that BOM handling was
broken somewhere along the way.
I would suggest filing a bug report. I don't have time to look into it,
sorry. I don't work on Guile anymore. I only happened to see your
message by chance.
Regards,
Mark
--
Disinformation flourishes because many people care deeply about injustice
but very few check the facts. Ask me about <https://stallmansupport.org>.
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: UTF16 encoding adds BOM everywhere?
2022-07-20 20:42 ` Mark H Weaver
@ 2022-07-20 21:45 ` Jean Abou Samra
0 siblings, 0 replies; 3+ messages in thread
From: Jean Abou Samra @ 2022-07-20 21:45 UTC (permalink / raw)
To: Mark H Weaver; +Cc: guile-user
Hi Mark,
Thanks for your reply!
Le 20/07/2022 à 22:42, Mark H Weaver a écrit :
> Hi,
>
> No, this is certainly a bug. It sounds like the
> 'at_stream_start_for_bom_write' port flag is not being cleared, as it
> should be, after the first character is written. I suspect that it
> worked correctly when I first implemented proper BOM handling in 2013
> (commit cdd3d6c9f423d5b95f05193fe3c27d50b56957e9), but the ports code
> has seen some major reworking since then. I guess that BOM handling was
> broken somewhere along the way.
>
> I would suggest filing a bug report.
OK, opened
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=56675
> I don't have time to look into it, sorry. I don't work on Guile anymore.
A pity :-)
Cheers,
Jean
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-07-20 21:45 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-07-14 19:16 UTF16 encoding adds BOM everywhere? Jean Abou Samra
2022-07-20 20:42 ` Mark H Weaver
2022-07-20 21:45 ` Jean Abou Samra
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).