From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!not-for-mail From: Stefan Monnier Newsgroups: gmane.emacs.help Subject: Re: python-shell-send-region uses wrong encoding? Date: Tue, 29 Oct 2013 13:28:38 -0400 Message-ID: References: <20131029113044.GA28039@doriath.local> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Trace: ger.gmane.org 1383067762 27114 80.91.229.3 (29 Oct 2013 17:29:22 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 29 Oct 2013 17:29:22 +0000 (UTC) Cc: =?windows-1252?Q?Fabi=E1n?= Ezequiel Gallina To: help-gnu-emacs@gnu.org Original-X-From: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Tue Oct 29 18:29:26 2013 Return-path: Envelope-to: geh-help-gnu-emacs@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1VbD6a-00050j-Cv for geh-help-gnu-emacs@m.gmane.org; Tue, 29 Oct 2013 18:29:24 +0100 Original-Received: from localhost ([::1]:48560 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VbD6Z-0002V2-Ub for geh-help-gnu-emacs@m.gmane.org; Tue, 29 Oct 2013 13:29:23 -0400 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:39614) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VbD6I-0002T1-Ak for help-gnu-emacs@gnu.org; Tue, 29 Oct 2013 13:29:13 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VbD6B-0004q2-0S for help-gnu-emacs@gnu.org; Tue, 29 Oct 2013 13:29:06 -0400 Original-Received: from plane.gmane.org ([80.91.229.3]:59270) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VbD6A-0004pu-QK for help-gnu-emacs@gnu.org; Tue, 29 Oct 2013 13:28:58 -0400 Original-Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1VbD68-0004rZ-BE for help-gnu-emacs@gnu.org; Tue, 29 Oct 2013 18:28:56 +0100 Original-Received: from 108.161.119.233 ([108.161.119.233]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 29 Oct 2013 18:28:56 +0100 Original-Received: from monnier by 108.161.119.233 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 29 Oct 2013 18:28:56 +0100 X-Injected-Via-Gmane: http://gmane.org/ Original-Lines: 39 Original-X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 108.161.119.233 User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) Cancel-Lock: sha1:Qw8DEHNH60IX4Tx0NY2jwvEGSac= X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 80.91.229.3 X-BeenThere: help-gnu-emacs@gnu.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Users list for the GNU Emacs text editor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Original-Sender: help-gnu-emacs-bounces+geh-help-gnu-emacs=m.gmane.org@gnu.org Xref: news.gmane.org gmane.emacs.help:94255 Archived-At: > This is the same output that I get when I run the script in a terminal. > However, if I select the lines after 'from __future__ ...' until the end and > do python-shell-send-region (C-c C-r) I get this output instead: > u'W\xf6rterbuch' > u'W\xc3\xb6rterbuch' > The second line of output seems to indicate that the text was sent in a > different encoding compared to python-shell-send-buffer. No, it indicates that Python interpreted the bytes sent by Emacs in a different way: the first line (where you explicitly asked for a utf-8 decoding) indicates that the bytes indeed use the right utf-8 encoding, but the second indicates that Python does not decode the input as utf-8 but as something else (presumably latin-1). E.g. the patch below (which causes python-shell-send-string to tell Python that the file sent is using utf-8) should fix your problem (tho it's not a proper fix, since we shouldn't hardcode utf-8 here, but copy which ever -*- coding: -*- coding is in the file). Fabián, could you write a cleaner fix? Stefan === modified file 'lisp/progmodes/python.el' --- lisp/progmodes/python.el 2013-10-07 18:51:26 +0000 +++ lisp/progmodes/python.el 2013-10-29 17:25:11 +0000 @@ -2047,6 +2047,7 @@ (temp-file-name (make-temp-file "py")) (file-name (or (buffer-file-name) temp-file-name))) (with-temp-file temp-file-name + (insert "# -*- coding: utf-8 -*-") (insert string) (delete-trailing-whitespace)) (python-shell-send-file file-name process temp-file-name))