From mboxrd@z Thu Jan  1 00:00:00 1970
Path: news.gmane.org!not-for-mail
From: Yuri Khan <yuri.v.khan@gmail.com>
Newsgroups: gmane.emacs.help
Subject: Re: Getting Emacs to play nice with Hunspell and apostrophes
Date: Sat, 14 Jun 2014 14:11:51 +0700
Message-ID: <CAP_d_8XmN+_K78YPK=iYVFBjGL5q7oG4nC8zr-TBe7JgOvvArg@mail.gmail.com>
References: <mailman.3187.1402155569.1147.help-gnu-emacs@gnu.org>
	<87ha3s71mt.fsf@debian.uxu>
	<mailman.3375.1402464243.1147.help-gnu-emacs@gnu.org>
	<87tx7rsevi.fsf@debian.uxu>
	<mailman.3418.1402499010.1147.help-gnu-emacs@gnu.org>
	<8738fbscao.fsf@debian.uxu>
	<mailman.3473.1402551809.1147.help-gnu-emacs@gnu.org>
	<87sin8use8.fsf@debian.uxu>
NNTP-Posting-Host: plane.gmane.org
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Trace: ger.gmane.org 1402729930 19507 80.91.229.3 (14 Jun 2014 07:12:10 GMT)
X-Complaints-To: usenet@ger.gmane.org
NNTP-Posting-Date: Sat, 14 Jun 2014 07:12:10 +0000 (UTC)
Cc: "help-gnu-emacs@gnu.org" <help-gnu-emacs@gnu.org>
To: Emanuel Berg <embe8573@student.uu.se>
Original-X-From: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Sat Jun 14 09:12:06 2014
Return-path: <help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org>
Envelope-to: geh-help-gnu-emacs@m.gmane.org
Original-Received: from lists.gnu.org ([208.118.235.17])
	by plane.gmane.org with esmtp (Exim 4.69)
	(envelope-from <help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org>)
	id 1Wvi8D-0006qE-R3
	for geh-help-gnu-emacs@m.gmane.org; Sat, 14 Jun 2014 09:12:06 +0200
Original-Received: from localhost ([::1]:34416 helo=lists.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org>)
	id 1Wvi8D-0003Ez-Dj
	for geh-help-gnu-emacs@m.gmane.org; Sat, 14 Jun 2014 03:12:05 -0400
Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:40058)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <yurivkhan@gmail.com>) id 1Wvi82-0003Eu-L0
	for help-gnu-emacs@gnu.org; Sat, 14 Jun 2014 03:11:56 -0400
Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <yurivkhan@gmail.com>) id 1Wvi80-000671-Gi
	for help-gnu-emacs@gnu.org; Sat, 14 Jun 2014 03:11:54 -0400
Original-Received: from mail-qc0-x229.google.com ([2607:f8b0:400d:c01::229]:61588)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <yurivkhan@gmail.com>) id 1Wvi80-00066C-BC
	for help-gnu-emacs@gnu.org; Sat, 14 Jun 2014 03:11:52 -0400
Original-Received: by mail-qc0-f169.google.com with SMTP id c9so5443545qcz.0
	for <help-gnu-emacs@gnu.org>; Sat, 14 Jun 2014 00:11:52 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=mime-version:sender:in-reply-to:references:date:message-id:subject
	:from:to:cc:content-type:content-transfer-encoding;
	bh=iXow06B2hMGvCZOYARBwgiXeDG2do3YiYaCoiFS1zXQ=;
	b=kD0Hwl+Xa0RQTubaOKuOmR09wHtETCvlPCC0EYN/a7iLstpQ65ZsC9zmGXEAROvkZ4
	1axTbTdleQWzNgB1psFmjy49MU7H7DU4oWZNm1i2A5iJsam/MA5tnnLz1chd7Nu4ySmh
	uSmaJTc5fN7S+qHP0jk0mj5w2Lm6vg4EEeB5TqSh5pLtFBj7jI7CLzaAXJkI6HfXFoZE
	bBXZez1YkVnShgFd7ZsjR8QZqRIJE1erffSpVRn89Q7T1cKCrUyR/rI6khV+DVrww6JQ
	vlKna3XrL3m6NfW/rNaAVa4zHPMl+3xfPMT18zwnrvH+BqW4Ih6Dn3NyjDYqBGCvXWx8
	r1FQ==
X-Received: by 10.224.55.130 with SMTP id u2mr9758790qag.67.1402729912022;
	Sat, 14 Jun 2014 00:11:52 -0700 (PDT)
Original-Received: by 10.96.154.73 with HTTP; Sat, 14 Jun 2014 00:11:51 -0700 (PDT)
In-Reply-To: <87sin8use8.fsf@debian.uxu>
X-Google-Sender-Auth: 2ANeuFL12IO1s1TLmg17nIw7mr0
X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address
	(bad octet value).
X-Received-From: 2607:f8b0:400d:c01::229
X-BeenThere: help-gnu-emacs@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Users list for the GNU Emacs text editor <help-gnu-emacs.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/help-gnu-emacs>,
	<mailto:help-gnu-emacs-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/help-gnu-emacs>
List-Post: <mailto:help-gnu-emacs@gnu.org>
List-Help: <mailto:help-gnu-emacs-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/help-gnu-emacs>,
	<mailto:help-gnu-emacs-request@gnu.org?subject=subscribe>
Errors-To: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org
Original-Sender: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org
Xref: news.gmane.org gmane.emacs.help:98239
Archived-At: <http://permalink.gmane.org/gmane.emacs.help/98239>

On Sat, Jun 14, 2014 at 9:38 AM, Emanuel Berg <embe8573@student.uu.se> wrot=
e:
> Yuri Khan <yuri.v.khan@gmail.com> writes:
>
>> You could order a book in an Internet shop, have them
>> completely b0rk up the encoding of the shipping
>> address:
>>
>> http://cdn.imagepush.to/in/625x2090/i/3/30/301/24.jpg
>>
>> Then somebody at the postal system might decode the
>> characters and the package would still be delivered
>> at the intended address.
>
> Ha-ha, unbelievable! How did that happen? First you
> wrote in Russian at the Internet shop's web page - then
> it got like that because of them translating Unicode
> (?) to ISO-8859-1 (which is 8-bit, with the ASCII as
> its lower half) - ? Why didn't the Internet shop do it?

First I must say it=E2=80=99s not mine and likely not a common occurrence f=
or
the Russian Post which is nowadays notorious for its lack of customer
orientedness.

In technical terms, I can think of the following sequence of events:

* The user comes to a website containing an order form. (The form
contains a free input <textarea> for the street address and possibly
an <input> for the recipient name, and a <select> for the country. The
latter ensures that the word RUSSIE is printed in its legible form.)
* The user enters her address and name into the web form, in Russian;
also selects Russian Federation from the country dropdown.
* The browser encodes the address in KOI8-R, one of the three code
pages used in Russia. In this encoding, the string =D0=9C=D0=BE=D1=81=D0=BA=
=D0=B2=D0=B0 (Moscow) has
the following byte representation: ED CF D3 CB D7 C1. (The KOI8-R
encoding was designed in such a way that it remains readable if the
high bit is stripped: mOSKWA. Too bad the links were already
8-bit-clean at the time Harry Potter was published.)
* The browser sends the form data to the web server, labeled as
Content-Type: application/x-www-form-urlencoded; encoding=3DKOI8-r. (At
that time, Unicode was not as ubiquitous as it is now; browsers
operated in an encoding that best matched the user=E2=80=99s input.)
* The web server passes the form data to the backend script (Perl CGI
or possibly PHP running as a module).
* The backend script disregards the encoding=3D parameter, reinterprets
the string as if it were encoded in ISO-8859-1 (or possibly
windows-1252, which is an extension of ISO-8859-1). The byte
representation ED CF D3 CB D7 C1 decodes into =C3=AD=C3=8F=C3=93=C3=8B=C3=
=97=C3=81 (small i with
acute, capital I with diaeresis, capital O with acute, capital E with
diaeresis, multiplication sign, capital A with acute). This string
then gets stored in the database (which is likely configured to
operate in ISO-8859-1 or windows-1251) and lives happily ever after.

> Did they actually think that was a language or some
> transcription of Russian?

Most probably, at the time a human being at the merchant side got
involved, the address was already mangled. They did not have the
knowledge of Russian code pages, and decided to make a best reasonable
effort =E2=80=94 =E2=80=9Csend it as is and let those crazy Russians sort i=
t out=E2=80=9D.

> How was it translated to
> Russian at the postal office? I can only make out the
> first line: Russia, Moscow.

The package contains two pieces of information =E2=80=94 the country name i=
n
French (RUSSIE) and the postal code 119415 =E2=80=94 which get the package =
to
the postal office 119415 at 14 Udaltsov street in Moscow, near the
customer=E2=80=99s place of residence. (Postal codes are unique within Russ=
ia,
the first three digits unambiguously identifying the city.)

https://goo.gl/maps/4TK3D (pin at the post office building).

The worker at the post office might be familiar with both the KOI8-R
and Windows-1250 encoding tables, but that is highly unlikely.

Alternatively, the worker might regard the mysteriously labeled
package as a peculiar form of a substitution cypher puzzle. [Challenge
Accepted] He takes a red pen and starts scribbling right on the
package.

* First, he notices that the two middle letters in the first word are
identical, and guesses that this word must be Rossi[ya] (Russia).
* This allows him to decode two letters of the next word, which can
then be guessed as Moskva (Moscow) =E2=80=94 what else could be
* Substituting the known letters into the customer=E2=80=99s first name giv=
es
=E2=80=9C=D0=A1=D0=B2***=D0=B0**=E2=80=9D (Sv***a**), which the postal work=
er recognizes as Svetlana
(a fairly common Russian feminine name, and the most common of those
starting with Sv). (The last letter does not match because of
grammatical case declension.)
* This now gives enough information to decode and guess the street as
pr. Vernadskogo and deliver the package to the Moscow State University
dormitory at Vernadskogo, 37 (other marker at the map linked above),
room 1817-1. Probably also lecture Svetlana that, until all web sites
embrace Unicode, it=E2=80=99s safer to write your address in transliteratio=
n.


Now, while this all makes for great war stories, it Should Not be
necessary. Unicode should be used in all stages of Internet shop order
processing, and addresses written in any local language should be
deliverable without post office workers having to solve a challenge.