Re: 23.0.60; [nxml] BOM and utf-8

[Patrick Drechsler <patrick@pdrechsler.de>]

mah@everybody.org (Mark A. Hershberger) writes:

> Patrick Drechsler <patrick@pdrechsler.de> writes:
>
>> is the attached xml file (simple.xml) really invalid (as indicated by
>> nxhtml) or is this a bug in nxhtml?

s/nxhtml/nxml/

> The file simple.xml is really invalid.
>
> http://www.w3.org/TR/2006/REC-xml-20060816/#sec-prolog-dtd
>
> The XML spec gives the following syntax description for the prolog of an
> XML file (I've only copied the relevant parts):
>
>     [3]    S	    ::=  (#x20 | #x9 | #xD | #xA)+
>     [22]   prolog   ::=  XMLDecl? Misc* (doctypedecl  Misc*)?
>     [23]   XMLDecl  ::= '<?xml' VersionInfo EncodingDecl? SDDecl? S? '?>'
>
> Note that there is no S before the literal “<?xml" and that "<?xml" is
> optional.
>
> So, yes, an file that contains whitespace before "<?xml" is invalid XML.

Finally having some spare time I read the specs from above and I have a
followup question concerning your last sentence:

The BOM in my example file is not whitespace, it is xEF xBB xBF (it is
only displayed as whitespace by Emacs). According to the W3C site this
is valid:

,----[ http://www.w3.org/TR/2006/REC-xml-20060816/#charencoding ]
| Entities encoded in UTF-16 MUST and entities encoded in UTF-8 MAY
| begin with the Byte Order Mark described by Annex H of [ISO/IEC
| 10646:2000], section 2.4 of [Unicode], and section 2.7 of [Unicode3]
| (the ZERO WIDTH NO-BREAK SPACE character, #xFEFF). This is an encoding
| signature, not part of either the markup or the character data of the
| XML document. XML processors MUST be able to use this character to
| differentiate between UTF-8 and UTF-16 encoded documents.
`----

Also http://www.w3.org/TR/2006/REC-xml-20060816/#sec-guessing-no-ext-info

and

,----[ http://www.w3.org/TR/2006/REC-xml-20060816/#sec-guessing-with-ext-info ]
| If an XML entity is in a file, the Byte-Order Mark and encoding
| declaration are used (if present) to determine the character encoding.
`----

sound like a BOM is a legal (although optional) part of a xml file coded
in utf-8.

But I am not an expert, so please correct my potentially incorrect
interpretation.

In case my interpretation is correct, this is a bug in emacs' nxml mode.

Cheers,

Patrick 
-- 
Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it.  (Kernighan)