From mboxrd@z Thu Jan  1 00:00:00 1970
Path: news.gmane.org!not-for-mail
From: Patrick Drechsler <patrick@pdrechsler.de>
Newsgroups: gmane.emacs.devel,gmane.emacs.pretest.bugs
Subject: Re: 23.0.60; [nxml] BOM and utf-8
Date: Thu, 22 May 2008 00:20:00 +0200
Organization: none
Message-ID: <87r6bvs3jj.fsf@pdrechsler.de>
References: <87od75kt78.fsf@pdrechsler.de> <87d4nk8y3q.fsf@everybody.org>
NNTP-Posting-Host: lo.gmane.org
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Trace: ger.gmane.org 1211408445 24721 80.91.229.12 (21 May 2008 22:20:45 GMT)
X-Complaints-To: usenet@ger.gmane.org
NNTP-Posting-Date: Wed, 21 May 2008 22:20:45 +0000 (UTC)
Cc: emacs-pretest-bug@gnu.org
To: emacs-devel@gnu.org
Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu May 22 00:21:21 2008
Return-path: <emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org>
Envelope-to: ged-emacs-devel@m.gmane.org
Original-Received: from lists.gnu.org ([199.232.76.165])
	by lo.gmane.org with esmtp (Exim 4.50)
	id 1JywgA-0002D1-5h
	for ged-emacs-devel@m.gmane.org; Thu, 22 May 2008 00:21:02 +0200
Original-Received: from localhost ([127.0.0.1]:51002 helo=lists.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43)
	id 1JywfN-0004Yu-I4
	for ged-emacs-devel@m.gmane.org; Wed, 21 May 2008 18:20:13 -0400
Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43)
	id 1JywfG-0004VO-MY
	for emacs-devel@gnu.org; Wed, 21 May 2008 18:20:06 -0400
Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43)
	id 1JywfE-0004TV-MV
	for emacs-devel@gnu.org; Wed, 21 May 2008 18:20:05 -0400
Original-Received: from [199.232.76.173] (port=55010 helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1JywfE-0004TG-Gg
	for emacs-devel@gnu.org; Wed, 21 May 2008 18:20:04 -0400
Original-Received: from main.gmane.org ([80.91.229.2]:51740 helo=ciao.gmane.org)
	by monty-python.gnu.org with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32)
	(Exim 4.60) (envelope-from <ged-emacs-devel@m.gmane.org>)
	id 1JywfD-0003Sl-Vz
	for emacs-devel@gnu.org; Wed, 21 May 2008 18:20:04 -0400
Original-Received: from list by ciao.gmane.org with local (Exim 4.43)
	id 1JywfA-0007tG-8o
	for emacs-devel@gnu.org; Wed, 21 May 2008 22:20:00 +0000
Original-Received: from dslb-088-065-089-160.pools.arcor-ip.net ([88.65.89.160])
	by main.gmane.org with esmtp (Gmexim 0.1 (Debian))
	id 1AlnuQ-0007hv-00
	for <emacs-devel@gnu.org>; Wed, 21 May 2008 22:20:00 +0000
Original-Received: from patrick by dslb-088-065-089-160.pools.arcor-ip.net with local
	(Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00
	for <emacs-devel@gnu.org>; Wed, 21 May 2008 22:20:00 +0000
X-Injected-Via-Gmane: http://gmane.org/
Original-Lines: 67
Original-X-Complaints-To: usenet@ger.gmane.org
X-Gmane-NNTP-Posting-Host: dslb-088-065-089-160.pools.arcor-ip.net
Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAFVBMVEUAAACtra19fX3y8vKA
	gID///9AQEBawxxrAAABeUlEQVR4nHXSwXKDIBAA0G3a8Vw05JxY9E4MObeK3h1Cz2qt//8J3UWj
	ppIdBxmfu4sC2CcB070+PQHDuAcyhQPzwLVFuSlfqQK7eOFirfQ2ly7JC8a3Kip1sx6QLOaRB0wo
	IUg8gFUa2D2BIFVb+OYGhC23cB4aUDbaQt4AgifjCACt9HwgQfrjWZWDwNPjEwHbtBu44kPC5D9Y
	FknMgFf8aZFawDDGIuoTkAYndYczzd5hjqAYwXzQ/QCraB2401HvHyQhKMb2tbw0M7whuBNoIs4M
	W/Xh056fQzwLPF/g14FkrcLN4nKp9WLByIMImSwyycLjDDsOF93pVFdC6yofHkEjxDhWumtWsMdH
	edc7SOf2/QiYQiByUU0gSldKD4LyBLUbK+kCaoI+diNlxgOGmIFWhVDpe5RgqEiPRR6Buwx8PxXE
	G3CGzeMVmAnctUB5h3Hsthljp34Ne/wjbu4+cIovBVbqfH4vFdOkoI2SzEWUZdMMQ9k/5csXnnRd
	prMAAAAASUVORK5CYII=
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.60 (gnu/linux)
Cancel-Lock: sha1:ExKSZks2BVAQnSmvKWjNa0ebSb0=
X-detected-kernel: by monty-python.gnu.org: Linux 2.6, seldom 2.4 (older, 4)
X-BeenThere: emacs-devel@gnu.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: "Emacs development discussions." <emacs-devel.gnu.org>
List-Unsubscribe: <http://lists.gnu.org/mailman/listinfo/emacs-devel>,
	<mailto:emacs-devel-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/pipermail/emacs-devel>
List-Post: <mailto:emacs-devel@gnu.org>
List-Help: <mailto:emacs-devel-request@gnu.org?subject=help>
List-Subscribe: <http://lists.gnu.org/mailman/listinfo/emacs-devel>,
	<mailto:emacs-devel-request@gnu.org?subject=subscribe>
Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org
Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org
Xref: news.gmane.org gmane.emacs.devel:97488 gmane.emacs.pretest.bugs:22403
Archived-At: <http://permalink.gmane.org/gmane.emacs.devel/97488>

mah@everybody.org (Mark A. Hershberger) writes:

> Patrick Drechsler <patrick@pdrechsler.de> writes:
>
>> is the attached xml file (simple.xml) really invalid (as indicated by
>> nxhtml) or is this a bug in nxhtml?

s/nxhtml/nxml/

> The file simple.xml is really invalid.
>
> http://www.w3.org/TR/2006/REC-xml-20060816/#sec-prolog-dtd
>
> The XML spec gives the following syntax description for the prolog of an
> XML file (I've only copied the relevant parts):
>
>     [3]    S	    ::=  (#x20 | #x9 | #xD | #xA)+
>     [22]   prolog   ::=  XMLDecl? Misc* (doctypedecl  Misc*)?
>     [23]   XMLDecl  ::= '<?xml' VersionInfo EncodingDecl? SDDecl? S? '?>'
>
> Note that there is no S before the literal “<?xml" and that "<?xml" is
> optional.
>
> So, yes, an file that contains whitespace before "<?xml" is invalid XML.

Finally having some spare time I read the specs from above and I have a
followup question concerning your last sentence:

The BOM in my example file is not whitespace, it is xEF xBB xBF (it is
only displayed as whitespace by Emacs). According to the W3C site this
is valid:

,----[ http://www.w3.org/TR/2006/REC-xml-20060816/#charencoding ]
| Entities encoded in UTF-16 MUST and entities encoded in UTF-8 MAY
| begin with the Byte Order Mark described by Annex H of [ISO/IEC
| 10646:2000], section 2.4 of [Unicode], and section 2.7 of [Unicode3]
| (the ZERO WIDTH NO-BREAK SPACE character, #xFEFF). This is an encoding
| signature, not part of either the markup or the character data of the
| XML document. XML processors MUST be able to use this character to
| differentiate between UTF-8 and UTF-16 encoded documents.
`----

Also http://www.w3.org/TR/2006/REC-xml-20060816/#sec-guessing-no-ext-info

and

,----[ http://www.w3.org/TR/2006/REC-xml-20060816/#sec-guessing-with-ext-info ]
| If an XML entity is in a file, the Byte-Order Mark and encoding
| declaration are used (if present) to determine the character encoding.
`----

sound like a BOM is a legal (although optional) part of a xml file coded
in utf-8.

But I am not an expert, so please correct my potentially incorrect
interpretation.

In case my interpretation is correct, this is a bug in emacs' nxml mode.

Cheers,

Patrick 
-- 
Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it.  (Kernighan)