From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: David Kastrup Newsgroups: gmane.emacs.devel Subject: Re: Fwd: Re: Inadequate documentation of silly characters on screen. Date: Thu, 19 Nov 2009 17:55:10 +0100 Organization: Organization?!? Message-ID: <87aayiihe9.fsf@lola.goethe.zz> References: <20091118191258.GA2676@muc.de> <20091119082040.GA1720@muc.de> <874ooq8xay.fsf@wanchan.jasonrumney.net> <20091119141852.GC1720@muc.de> <20091119155848.GB1314@muc.de> NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Trace: ger.gmane.org 1258650702 8117 80.91.229.12 (19 Nov 2009 17:11:42 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Thu, 19 Nov 2009 17:11:42 +0000 (UTC) To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Nov 19 18:11:36 2009 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1NBAXe-0004sJ-JG for ged-emacs-devel@m.gmane.org; Thu, 19 Nov 2009 18:11:34 +0100 Original-Received: from localhost ([127.0.0.1]:38575 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1NBAXe-0000up-3X for ged-emacs-devel@m.gmane.org; Thu, 19 Nov 2009 12:11:34 -0500 Original-Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1NBAIO-0003pC-5W for emacs-devel@gnu.org; Thu, 19 Nov 2009 11:55:48 -0500 Original-Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43) id 1NBAIJ-0003m0-C6 for emacs-devel@gnu.org; Thu, 19 Nov 2009 11:55:47 -0500 Original-Received: from [199.232.76.173] (port=51107 helo=monty-python.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1NBAIJ-0003lt-5l for emacs-devel@gnu.org; Thu, 19 Nov 2009 11:55:43 -0500 Original-Received: from lo.gmane.org ([80.91.229.12]:42564) by monty-python.gnu.org with esmtps (TLS-1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1NBAII-0004ZP-KF for emacs-devel@gnu.org; Thu, 19 Nov 2009 11:55:42 -0500 Original-Received: from list by lo.gmane.org with local (Exim 4.50) id 1NBAIF-0005I6-GA for emacs-devel@gnu.org; Thu, 19 Nov 2009 17:55:39 +0100 Original-Received: from p5b2c20b9.dip.t-dialin.net ([91.44.32.185]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 19 Nov 2009 17:55:39 +0100 Original-Received: from dak by p5b2c20b9.dip.t-dialin.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 19 Nov 2009 17:55:39 +0100 X-Injected-Via-Gmane: http://gmane.org/ Original-Lines: 57 Original-X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: p5b2c20b9.dip.t-dialin.net X-Face: 2FEFf>]>q>2iw=B6, xrUubRI>pR&Ml9=ao@P@i)L:\urd*t9M~y1^:+Y]'C0~{mAl`oQuAl \!3KEIp?*w`|bL5qr,H)LFO6Q=qx~iH4DN; i"; /yuIsqbLLCh/!U#X[S~(5eZ41to5f%E@'ELIi$t^ Vc\LWP@J5p^rst0+('>Er0=^1{]M9!p?&:\z]|;&=NP3AhB!B_bi^]Pfkw User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.1.50 (gnu/linux) Cancel-Lock: sha1:gI5aTz5uqC3UrlVTQ8CMoqfDUyw= X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6 (newer, 3) X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.devel:117264 Archived-At: Alan Mackenzie writes: > On Thu, Nov 19, 2009 at 10:30:18AM -0500, Stefan Monnier wrote: >> > The actual character in the string is ñ (#x3f). > >> No: the string does not contain any characters, only bytes, because >> it's a unibyte string. > > I'm thinking from the lisp viewpoint. The string is a data structure > which contains characters. I really don't want to have to think about > the difference between "chars" and "bytes" when I'm hacking lisp. If > I do, then the abstraction "string" is broken. > >> So it contains the byte 241, not the character ñ. > > That is then a bug. I wrote "(aset nl 0 ?ñ)", not "(aset nl 0 241)". Huh? ?ñ is the Emacs code point of ñ. Which is pretty much identical to the Unicode code point in Emacs 23. >> The byte 241 can be inserted in multibyte strings and buffers because >> it is also a char of code 4194289 (which gets displayed as \361). > > Hang on a mo'! How can the byte 241 "be" a char of code 4194289? > This is some strange usage of the word "be" that I wasn't previously > aware of. ;-) Emacs encodes most of its things in utf-8. A Unicode code point is an integer. You can encode it in different encodings, resulting in different byte streams. Inside of a byte stream encoded in utf-8, the isolated byte 241 does not correspond to a Unicode character. It is not valid utf-8. When Emacs reads a file supposedly in utf-8, it wants to represent _all_ possible byte streams in order to be able to save unchanged data unmolested. So it encodes the entity "illegal isolated byte 241 in an utf-8 document" with the character code 4194289 which has a representation in Emacs' internal variant of utf-8, but is outside of the range of Unicode. > At this point, would you please just agree with me that when I do > > (setq nl "\n") > (aset nl 0 ?ñ) > (insert nl) > > , what should appear on the screen should be "ñ", NOT "\361"? Thanks! You assume that ?ñ is a character. But in Emacs, it is an integer, a Unicode code point in Emacs 23. As long as there is something like a unibyte string, there is no way to distinguish the character 241 and the byte 241 except when Emacs is told explicitly. Because Emacs has no separate "character" data type. -- David Kastrup