From mboxrd@z Thu Jan 1 00:00:00 1970 From: dmg Subject: Re: Re: org-protocol: non-ASCII characters Date: Fri, 12 Feb 2010 14:23:54 -0800 Message-ID: <16e9be801002121423y5bdff5bdy2e9e7503f8deb154@mail.gmail.com> References: <86tyu2d5xw.fsf@mn.cs.uvic.ca> <4B693FBD.9010608@jboecker.de> <86d40k9702.fsf_-_@mn.cs.uvic.ca> <4B6D73A2.8000403@jboecker.de> <4B6D7E1A.3030807@jboecker.de> <878wb4rmm6.fsf@gmx.de> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Return-path: Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43) id 1Ng3va-0004nZ-Fp for emacs-orgmode@gnu.org; Fri, 12 Feb 2010 17:23:58 -0500 Received: from [140.186.70.92] (port=55548 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Ng3vZ-0004nR-FI for emacs-orgmode@gnu.org; Fri, 12 Feb 2010 17:23:57 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1Ng3vY-0008My-6u for emacs-orgmode@gnu.org; Fri, 12 Feb 2010 17:23:57 -0500 Received: from mail-yx0-f186.google.com ([209.85.210.186]:52838) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Ng3vY-0008Ms-4G for emacs-orgmode@gnu.org; Fri, 12 Feb 2010 17:23:56 -0500 Received: by yxe16 with SMTP id 16so2578031yxe.1 for ; Fri, 12 Feb 2010 14:23:54 -0800 (PST) In-Reply-To: <878wb4rmm6.fsf@gmx.de> List-Id: "General discussions about Org-mode." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org Errors-To: emacs-orgmode-bounces+geo-emacs-orgmode=m.gmane.org@gnu.org To: Sebastian Rose Cc: emacs-orgmode > Basically, it is OK to url-encode each character who's binary > representation start with 1 (i.e., the value of the character is higher > than 127). The text to be url-encoded should be UTF-8 ideally. > > If you use glib::ustring, it's easy to transform any iso-8859 string to > utf-8. Each character, whos binary representation start with a 1, has to > be url-encoded as well as the `%' character [1], but you could as > url-encode the entire utf-8 string. > > Ok, I think I understand the problem now. I have updated xournal to encode = the filename from its encoding to uft8. that seems to work. See http://github.com/dmgerman/xournal For evince, I think I have found a problem in the parsing of the link. Evince already encodes the URL, but it does not encode the '/', hence you will get a link like thi= s: emacsclient 'org-protocol://remember://docview/tmp/00%C3%A1%C3%A9%C3%AD%C3%= B3%C3%BA.pdf::1' the filename is /tmp/00=E1=E9=ED=F3=FA.pdf But emacs incorrectly stops parsing the link after tmp/ By the way, xournal now supports store-link --dmg > > > > > The function that does the decoding is `org-protocol-unhex-string' which > in turn uses `org-protocol-unhex-compound'. > > > `man utf-8` shows, how org-protocol tries to decode characters. > > > The JavaScript-Funktion `encodeURIComponent()' returns exactly what we > need. It recodes a string to utf-8 and then encodes all characters, > except digits, ASCII letters and these punctuation characters: -_.!~*'() > > See ECMA-262 Standard, Section 15.1.3 > (http://bclary.com/2004/11/07/ecma-262.html#a-15.1.3 [2]): > > =A0 "The character is first transformed into a sequence of octets using > =A0 =A0the UTF-8 transformation..." > > > Again, note, that the decoding mechanism relies on the fact, that the > sequence to decode is url-encoded UTF-8. > > > > > > Example: > > =A0The url-encoded unicode representation of the German umlaut `=F6' is > =A0`%C3%B6'. Thus > > =A0 =A0 (org-protocol-unhex-string "%C3%B6") > > =A0gives you "=F6". > > =A0In iso-8859-1, the url-encoded representation of the same character `= =F6' was > =A0`%F6'. But > > =A0 =A0 (org-protocol-unhex-string "%F6") > > =A0gives you "" - the empty string. There is no utf-8 character with this= binary > =A0representation, since every byte starting with a 1 (i.e. is bigger tha= n 127) > =A0starts a multibyte sequence (2 or more bytes). > > =A0But: > > =A0 =A0 (org-protocol-unhex-string "%2F%3C") > > =A0gives you, as expected, =A0"/<" which shows, that you could savely > =A0url-encode each and every character of a utf-8 encoded string. > > > =3D=3D =A0Footnotes: > > [1] The percent character `%' has to be encoded, if followed by > =A0 =A0[0-9A-Fa-f]{2}, because org-protocol will assume, that a sequence > =A0 =A0matching "\\(%[0-9a-f][0-9a-f]\\)+" is an encoded character. That > =A0 =A0said, a `%' has to be url-encoded, since one will hardly ever > =A0 =A0know for sure, that a `%' is never followed by "[0-9a-f][0-9a-f]". > > [2] Get a PDF version of ECMA-262 third edition here: > =A0 =A0http://www.ecma-international.org/publications/standards/Ecma-262.= htm > > --=20 --dmg --- Daniel M. German http://turingmachine.org