From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Oliver Scholz Newsgroups: gmane.emacs.help Subject: Re: replacing characters and whacky trans-buffer conversion Date: Tue, 06 Mar 2007 17:41:37 +0100 Organization: Arcor Message-ID: <45ed99c3$0$20293$9b4e6d93@newsspool3.arcor-online.net> References: NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Trace: sea.gmane.org 1173202907 27955 80.91.229.12 (6 Mar 2007 17:41:47 GMT) X-Complaints-To: usenet@sea.gmane.org NNTP-Posting-Date: Tue, 6 Mar 2007 17:41:47 +0000 (UTC) To: help-gnu-emacs@gnu.org Original-X-From: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Tue Mar 06 18:41:36 2007 Return-path: Envelope-to: geh-help-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([199.232.76.165]) by lo.gmane.org with esmtp (Exim 4.50) id 1HOdfE-0001ph-7B for geh-help-gnu-emacs@m.gmane.org; Tue, 06 Mar 2007 18:41:29 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1HOdf3-00050x-6I for geh-help-gnu-emacs@m.gmane.org; Tue, 06 Mar 2007 12:41:17 -0500 Original-Path: shelby.stanford.edu!newshub.stanford.edu!news.tele.dk!news.tele.dk!small.news.tele.dk!newshunter!cosy.sbg.ac.at!newsfeed.utanet.at!newsfeed01.chello.at!newsfeed.arcor.de!newsspool1.arcor-online.net!news.arcor.de.POSTED!not-for-mail Original-Newsgroups: gnu.emacs.help User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.0.95 (gnu/linux) Original-Lines: 81 Original-NNTP-Posting-Date: 06 Mar 2007 17:41:39 CET Original-NNTP-Posting-Host: 39eba09b.newsspool3.arcor-online.net Original-X-Trace: DXC=OiDHALk4Fbm:i=48; n?Z:`McF=Q^Z^V3h4Fo<]lROoRaFl8W>\BH3Yb`5>PTYg0k1k`o_`O>47QEm`m=1aUoR62fJjVmD3_`gc` Original-X-Complaints-To: usenet-abuse@arcor.de Original-Xref: shelby.stanford.edu gnu.emacs.help:146115 X-BeenThere: help-gnu-emacs@gnu.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Users list for the GNU Emacs text editor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Original-Sender: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Errors-To: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.help:41719 Archived-At: ken writes: > An email comes in with this (emdash) character in it: – > > It looks like an em-dash until the text containing it is pasted into an > emacs buffer; then it appears as a series of "garbage characters". > (Copy and paste the emdash into an emacs buffer yourself, and perhaps > you'll see what I mean.) I am afraid, your're making assumptions here that don't hold true. I am using Emacs to read mail and news. Your message was encoded in UTF-8 and what I see is an en-dash U+2013 (are you sure this was supposed to be an e_m_-dash?). This does not change when I kill&yank it into another buffer. I assume you are using another (non-Emacs) E-Mail client. And you are copying&pasting it via X-selection, Gnome-Clipboard, MS Windows, the MacOS GUI or something like this? You might be interested in fixing this instead of the retroactive fix you're trying to undertake. (I wouldn't know how though, X-selection is a mystery to me.) (Have you tried saving the message from your Email application instead of c&p?) > To me and, possibly to you, this emdash appears in emacs as nine (9) > "garbage" characters. > > Because I want to programmatically replace these 9 garbage characters > into something latin1-friendly, I copy-and-paste these nine characters > into an *.el file containing a line like this: > > (replace-string "–" "--" nil (point-min) (point-max)) ^^^ I assume this consisted of nine garbage characters when you wrote your message? Because here it showed up as a valid U+2013 (EN DASH) encoded in UTF-8. (I'd have no explanation for this though, because EN DASH in unrecognized UTF-8 would show up as 3 garbage characters, not 9.) > The sought string (i.e., the first argument above) isn't found, however > because, for some whacky reason, the emdash pasted into the *.el file is > different-- by one character-- from exactly the same emdash pasted into > the other emacs buffer (the one I'm saving the email in). The reason is: characters aren't bytes. Because of some bug or missing feature you are exposed to the internal byte/octet representation of character in one way or the other (I'd guess the internal representation used by the clipboard.) Emacs tries to fit this into its own internal representation, unfortunately it has two of them: unibyte and multibyte. > In the emacs buffer containing the email, the fourth garbage character > (as shown by C-u C-x=) is: [...] > buffer code: 0x86 0xE2 > file code: not encodable by coding system undecided-unix This is a multibyte buffer. > In the *.el buffer, the fourth garbage character (which should be > exactly the same character) is: [...] > buffer code: 0xE2 > file code: 0xE2 (encoded by coding system raw-text-unix) This is a unibyte buffer. Normally el-files shouldn't be encoded in raw-text-unix. Were you visiting this file with find-file-literally? If not what setting has caused this? It is not a proper solution, but if you are really keen on searching and replacing the octets rather than fixing the character encoding, then the proper way would be to make sure that *both* buffers are unibyte, because that's for dealing with binaries. Oliver -- 16 Ventôse an 215 de la Révolution Liberté, Egalité, Fraternité!