From mboxrd@z Thu Jan  1 00:00:00 1970
Path: news.gmane.org!not-for-mail
From: Bozhidar Batsov <bozhidar.batsov@gmail.com>
Newsgroups: gmane.emacs.devel
Subject: Re: Using utf-8-auto as a process coding system
Date: Mon, 28 Jul 2014 18:05:50 +0300
Message-ID: <etPan.53d666ce.1f16e9e8.6d75@Bozhidars-MacBook-Pro.local>
References: <etPan.53cd1254.4353d0cd.daeb@Bozhidars-MacBook-Pro.local>
	<mvmwqb6lu8g.fsf@hawking.suse.de>
	<etPan.53cd29b3.2ca88611.daeb@Bozhidars-MacBook-Pro.local>
	<83fvhu4tbg.fsf@gnu.org>
	<etPan.53cdfce9.643c9869.3e7d@bozhidar-mini.local>
	<83zjfxio9a.fsf@gnu.org>
NNTP-Posting-Host: plane.gmane.org
Mime-Version: 1.0
Content-Type: multipart/alternative; boundary="53d666ce_1190cde7_6d75"
X-Trace: ger.gmane.org 1406559982 4496 80.91.229.3 (28 Jul 2014 15:06:22 GMT)
X-Complaints-To: usenet@ger.gmane.org
NNTP-Posting-Date: Mon, 28 Jul 2014 15:06:22 +0000 (UTC)
Cc: schwab@suse.de, emacs-devel@gnu.org
To: Eli Zaretskii <eliz@gnu.org>
Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Mon Jul 28 17:06:17 2014
Return-path: <emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org>
Envelope-to: ged-emacs-devel@m.gmane.org
Original-Received: from lists.gnu.org ([208.118.235.17])
	by plane.gmane.org with esmtp (Exim 4.69)
	(envelope-from <emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org>)
	id 1XBmVD-0001DU-2v
	for ged-emacs-devel@m.gmane.org; Mon, 28 Jul 2014 17:06:15 +0200
Original-Received: from localhost ([::1]:40161 helo=lists.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org>)
	id 1XBmVC-0007Fz-J3
	for ged-emacs-devel@m.gmane.org; Mon, 28 Jul 2014 11:06:14 -0400
Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:52596)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bozhidar.batsov@gmail.com>) id 1XBmV4-0007Ah-Ty
	for emacs-devel@gnu.org; Mon, 28 Jul 2014 11:06:11 -0400
Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <bozhidar.batsov@gmail.com>) id 1XBmV0-0007UV-O0
	for emacs-devel@gnu.org; Mon, 28 Jul 2014 11:06:06 -0400
Original-Received: from mail-wg0-x234.google.com ([2a00:1450:400c:c00::234]:44728)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <bozhidar.batsov@gmail.com>)
	id 1XBmUv-0007TO-Ob; Mon, 28 Jul 2014 11:05:58 -0400
Original-Received: by mail-wg0-f52.google.com with SMTP id a1so7355884wgh.11
	for <multiple recipients>; Mon, 28 Jul 2014 08:05:55 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=date:from:to:cc:message-id:in-reply-to:references:subject
	:mime-version:content-type;
	bh=+09Hi5NGWV/iB7NoDGyrZwrLm738jLEPXPB5jfs4HPY=;
	b=r2qlm92zhZgIL3EGw94dyHtfqTHFgBjPNIReWkk4PTDwBDnKK6fgB1OF2g59imQFH4
	dAfWAIhIiPaop9QoDzkBbos8Ly7yOJxo+wRlkzpKSGSXHOv3dGhqSCRj41B1xzQnQO4y
	xRlVXVfeO03Hdnutplr8nUwgjw0osbxXsD7rHgqzkhlw6NRfcYoGp5Qifco/RF6iR6kJ
	7MHE2iMHypnoKENkjaQitzPOe+K52ASBJf0hATM9TaTBuddUI5gc5U94d4OXy5+SM1fK
	H+sF4Cm/vS6U0scIj/BirzTuXnqu/6k4eKP7uUyl+W+SMNQSVsFmtCTxtmMKGyitqY6h
	bKLg==
X-Received: by 10.194.134.70 with SMTP id pi6mr51278018wjb.1.1406559954337;
	Mon, 28 Jul 2014 08:05:54 -0700 (PDT)
Original-Received: from Bozhidars-MacBook-Pro.local ([95.87.231.111])
	by mx.google.com with ESMTPSA id q5sm30084871wiw.5.2014.07.28.08.05.53
	for <multiple recipients>
	(version=TLSv1.2 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
	Mon, 28 Jul 2014 08:05:53 -0700 (PDT)
In-Reply-To: <83zjfxio9a.fsf@gnu.org>
X-Mailer: Airmail (247)
X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address
	(bad octet value).
X-Received-From: 2a00:1450:400c:c00::234
X-BeenThere: emacs-devel@gnu.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: "Emacs development discussions." <emacs-devel.gnu.org>
List-Unsubscribe: <https://lists.gnu.org/mailman/options/emacs-devel>,
	<mailto:emacs-devel-request@gnu.org?subject=unsubscribe>
List-Archive: <http://lists.gnu.org/archive/html/emacs-devel>
List-Post: <mailto:emacs-devel@gnu.org>
List-Help: <mailto:emacs-devel-request@gnu.org?subject=help>
List-Subscribe: <https://lists.gnu.org/mailman/listinfo/emacs-devel>,
	<mailto:emacs-devel-request@gnu.org?subject=subscribe>
Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org
Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org
Xref: news.gmane.org gmane.emacs.devel:173214
Archived-At: <http://permalink.gmane.org/gmane.emacs.devel/173214>

--53d666ce_1190cde7_6d75
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

On July 25, 2014 at 4:03:58 PM, Eli Zaretskii (eliz=40gnu.org) wrote:
> Date: Tue, 22 Jul 2014 08:55:53 +0300=C2=A0
> =46rom: Bozhidar Batsov <bozhidar.batsov=40gmail.com>=C2=A0
> Cc: schwab=40suse.de, emacs-devel=40gnu.org=C2=A0
>=C2=A0
> Ops, I actually used an incorrect link. That=E2=80=99s the proper one -=
=C2=A0https://github.com/clojure-emacs/cider/issues/532=C2=A0

OK, but that one is very short ;-)=C2=A0

> The gist of my problem is that Windows users have encoding related prob=
lems running cider (discussion here=C2=A0https://github.com/clojure-emacs=
/cider/issues/474).=C2=A0

There's a lot of confusion in that discussion.=C2=A0

> I guess the problem stems from this bit of code:=C2=A0=C2=A0
>=C2=A0
> (set-process-coding-system process 'utf-8-unix 'utf-8-unix)=C2=A0

'-auto' is not about end-of-line (EOL) format, it is about the Byte=C2=A0=

Order Mark BOM (http://en.wikipedia.org/wiki/Byte=5Forder=5Fmark).=C2=A0

Does Cider on Windows indeed output UT=46-8 encoded text preceded by a=C2=
=A0
BOM=3F I'd be surprised, as the BOM is normally not needed with UT=46-8.=C2=
=A0
In which case all this -auto thing just comes from another confused=C2=A0=

user.=C2=A0
The problem is Java, not cider. cider is an Emacs Lisp client for a Cloju=
re REPL server (nREPL), which runs on top of the JVM. I seem to recall th=
at Java doesn=E2=80=99t use UT=46-8 at all, think it was using the older =
UT=46-16.

I guess it=E2=80=99s compatible with UT=46-8 to some extent.



If there's no BOM, the first thing I'd try on Windows is this:=C2=A0

(set-process-coding-system process 'utf-8-dos 'utf-8-unix)=C2=A0

This is because Windows programs will normally accept Unix EOL format=C2=A0=

on input, but will usually output Windows CR-L=46 EOL, which Emacs needs=C2=
=A0
to decode into a single newline character.=C2=A0

A more elegant solution, which should be platform-independent, is to=C2=A0=

use something like below (untested)=C2=A0

(set-process-coding-system process=C2=A0
(cons (coding-system-change-text-conversion=C2=A0
(car default-process-coding-system)=C2=A0
'utf-8)=C2=A0
(coding-system-change-text-conversion=C2=A0
(cdr default-process-coding-system)=C2=A0
'utf-8)))=C2=A0

This has the advantage that it uses the wisdom already invested in=C2=A0
setting the defaults for each platform.=C2=A0
Thanks for the suggestions. I=E2=80=99ll ask some of the Windows users to=
 try them both and see what works and what doesn=E2=80=99t.



--53d666ce_1190cde7_6d75
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

<html><head><style>body=7Bfont-family:Helvetica,Arial;font-size:13px=7D</=
style></head><body style=3D=22word-wrap: break-word; -webkit-nbsp-mode: s=
pace; -webkit-line-break: after-white-space;=22><div id=3D=22bloop=5Fcust=
omfont=22 style=3D=22font-family:Helvetica,Arial;font-size:13px; color: r=
gba(0,0,0,1.0); margin: 0px; line-height: auto;=22>On July 25, 2014 at 4:=
03:58 PM, Eli Zaretskii (<a href=3D=22mailto:eliz=40gnu.org=22>eliz=40gnu=
.org</a>) wrote:</div> <div><blockquote type=3D=22cite=22 class=3D=22clea=
n=5Fbq=22 style=3D=22color: rgb(0, 0, 0); font-family: Helvetica, Arial; =
font-size: 13px; font-style: normal; font-variant: normal; font-weight: n=
ormal; letter-spacing: normal; line-height: normal; orphans: auto; text-a=
lign: start; text-indent: 0px; text-transform: none; white-space: normal;=
 widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; backgro=
und-color: rgb(255, 255, 255);=22><span><div><div></div><div>&gt; Date: T=
ue, 22 Jul 2014 08:55:53 +0300<span class=3D=22Apple-converted-space=22>&=
nbsp;</span><br>&gt; =46rom: Bozhidar Batsov &lt;bozhidar.batsov=40gmail.=
com&gt;<span class=3D=22Apple-converted-space=22>&nbsp;</span><br>&gt; Cc=
: schwab=40suse.de, emacs-devel=40gnu.org<span class=3D=22Apple-converted=
-space=22>&nbsp;</span><br>&gt;<span class=3D=22Apple-converted-space=22>=
&nbsp;</span><br>&gt; Ops, I actually used an incorrect link. That=E2=80=99=
s the proper one -&nbsp;https://github.com/clojure-emacs/cider/issues/532=
<span class=3D=22Apple-converted-space=22>&nbsp;</span><br><br>OK, but th=
at one is very short ;-)<span class=3D=22Apple-converted-space=22>&nbsp;<=
/span><br><br>&gt; The gist of my problem is that Windows users have enco=
ding related problems running cider (discussion here&nbsp;https://github.=
com/clojure-emacs/cider/issues/474).<span class=3D=22Apple-converted-spac=
e=22>&nbsp;</span><br><br>There's a lot of confusion in that discussion.<=
span class=3D=22Apple-converted-space=22>&nbsp;</span><br><br>&gt; I gues=
s the problem stems from this bit of code:&nbsp;<span class=3D=22Apple-co=
nverted-space=22>&nbsp;</span><br>&gt;<span class=3D=22Apple-converted-sp=
ace=22>&nbsp;</span><br>&gt; (set-process-coding-system process 'utf-8-un=
ix 'utf-8-unix)<span class=3D=22Apple-converted-space=22>&nbsp;</span><br=
><br>'-auto' is not about end-of-line (EOL) format, it is about the Byte<=
span class=3D=22Apple-converted-space=22>&nbsp;</span><br>Order Mark BOM =
(http://en.wikipedia.org/wiki/Byte=5Forder=5Fmark).<span class=3D=22Apple=
-converted-space=22>&nbsp;</span><br><br>Does Cider on Windows indeed out=
put UT=46-8 encoded text preceded by a<span class=3D=22Apple-converted-sp=
ace=22>&nbsp;</span><br>BOM=3F I'd be surprised, as the BOM is normally n=
ot needed with UT=46-8.<span class=3D=22Apple-converted-space=22>&nbsp;</=
span><br>In which case all this -auto thing just comes from another confu=
sed<span class=3D=22Apple-converted-space=22>&nbsp;</span><br>user.<span =
class=3D=22Apple-converted-space=22>&nbsp;</span></div></div></span></blo=
ckquote></div><p>The problem is Java, not cider. cider is an Emacs Lisp c=
lient for a Clojure REPL server (nREPL), which runs on top of the JVM. I =
seem to recall that Java doesn=E2=80=99t use UT=46-8 at all, think it was=
 using the older UT=46-16.</p><p>I guess it=E2=80=99s compatible with UT=46=
-8 to some extent.</p><div><div><blockquote type=3D=22cite=22 class=3D=22=
clean=5Fbq=22 style=3D=22color: rgb(0, 0, 0); font-family: Helvetica, Ari=
al; font-size: 13px; font-style: normal; font-variant: normal; font-weigh=
t: normal; letter-spacing: normal; line-height: normal; orphans: auto; te=
xt-align: start; text-indent: 0px; text-transform: none; white-space: nor=
mal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; bac=
kground-color: rgb(255, 255, 255);=22><span><div><div><br><br>If there's =
no BOM, the first thing I'd try on Windows is this:<span class=3D=22Apple=
-converted-space=22>&nbsp;</span><br><br>(set-process-coding-system proce=
ss 'utf-8-dos 'utf-8-unix)<span class=3D=22Apple-converted-space=22>&nbsp=
;</span><br><br>This is because Windows programs will normally accept Uni=
x EOL format<span class=3D=22Apple-converted-space=22>&nbsp;</span><br>on=
 input, but will usually output Windows CR-L=46 EOL, which Emacs needs<sp=
an class=3D=22Apple-converted-space=22>&nbsp;</span><br>to decode into a =
single newline character.<span class=3D=22Apple-converted-space=22>&nbsp;=
</span><br><br>A more elegant solution, which should be platform-independ=
ent, is to<span class=3D=22Apple-converted-space=22>&nbsp;</span><br>use =
something like below (untested)<span class=3D=22Apple-converted-space=22>=
&nbsp;</span><br><br>(set-process-coding-system process<span class=3D=22A=
pple-converted-space=22>&nbsp;</span><br>(cons (coding-system-change-text=
-conversion<span class=3D=22Apple-converted-space=22>&nbsp;</span><br>(ca=
r default-process-coding-system)<span class=3D=22Apple-converted-space=22=
>&nbsp;</span><br>'utf-8)<span class=3D=22Apple-converted-space=22>&nbsp;=
</span><br>(coding-system-change-text-conversion<span class=3D=22Apple-co=
nverted-space=22>&nbsp;</span><br>(cdr default-process-coding-system)<spa=
n class=3D=22Apple-converted-space=22>&nbsp;</span><br>'utf-8)))<span cla=
ss=3D=22Apple-converted-space=22>&nbsp;</span><br><br>This has the advant=
age that it uses the wisdom already invested in<span class=3D=22Apple-con=
verted-space=22>&nbsp;</span><br>setting the defaults for each platform.<=
span class=3D=22Apple-converted-space=22>&nbsp;</span></div></div></span>=
</blockquote></div><p>Thanks for the suggestions. I=E2=80=99ll ask some o=
f the Windows users to try them both and see what works and what doesn=E2=
=80=99t.</p><div><blockquote type=3D=22cite=22 class=3D=22clean=5Fbq=22 s=
tyle=3D=22color: rgb(0, 0, 0); font-family: Helvetica, Arial; font-size: =
13px; font-style: normal; font-variant: normal; font-weight: normal; lett=
er-spacing: normal; line-height: normal; orphans: auto; text-align: start=
; text-indent: 0px; text-transform: none; white-space: normal; widows: au=
to; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: =
rgb(255, 255, 255);=22><span><div><div><br></div></div></span></blockquot=
e></div></div></body></html>
--53d666ce_1190cde7_6d75--