bug#15274: 24.3; space instead of codepoint 261 on windows console with cp852

bug#15274: 24.3; space instead of codepoint 261 on windows console with cp852

[Jarek Czekalski]

Hi!

I produced a file (under a different text editor) containing 16
Polish international characters. This file is encoded in ibm852
(cp852) and displays correctly in gui emacs on windows.
It also displays correctly in a console after typing

type win_cp.txt

I start emacs -Q -nw, set terminal encoding to cp852.
When the file is loaded into console emacs (-nw) and the correct
file encoding is assured (revert-buffer-with-coding, cp852)
then some characters are displayed correctly and some not. Those
displayed incorrectly are replaced with spaces. For example
the first one in the file ("a" with a tail, U+0105) is displayed
incorrectly and the second one ("c" with an accent, U+0107)
is displayed correctly.

I employed C-u C-x = command to see the details of both characters.
The info seems to be correct including the character codes in the
encoding: A5, 86 - respectively. The same code appears under
"terminal code" entry, what seems to be correct.

I regret there is no thing like "ascii with \U java escapes" encoding.
It would be perfect for sending this file to you.
If you suggest another encoding, I'll provide the file in that
encoding.


If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
     `bt full' and `xbacktrace'.
For information about debugging Emacs, please read the file
d:/Program_Files/emacs/etc/DEBUG.


In GNU Emacs 24.3.1 (i386-mingw-nt6.1.7601)
  of 2013-03-17 on MARVIN
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
  `configure --with-gcc (4.7) --cflags
  -ID:/devel/emacs/libs/libXpm-3.5.8/include
  -ID:/devel/emacs/libs/libXpm-3.5.8/src
  -ID:/devel/emacs/libs/libpng-dev_1.4.3-1/include
  -ID:/devel/emacs/libs/zlib-dev_1.2.5-2/include
  -ID:/devel/emacs/libs/giflib-4.1.4-1/include
  -ID:/devel/emacs/libs/jpeg-6b-4/include
  -ID:/devel/emacs/libs/tiff-3.8.2-1/include
  -ID:/devel/emacs/libs/gnutls-3.0.9/include
  -ID:/devel/emacs/libs/libiconv-1.13.1-1-dev/include
  -ID:/devel/emacs/libs/libxml2-2.7.8/include/libxml2'

Important settings:
   value of $LANG: PLK
   locale-coding-system: cp1250
   default enable-multibyte-characters: t

Major mode: Text

Minor modes in effect:
   tooltip-mode: t
   mouse-wheel-mode: t
   tool-bar-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   auto-composition-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   line-number-mode: t
   transient-mark-mode: t

Recent input:
C-x C-f w i n _ c p <tab> <return> M-x r e v e <tab>
r t - b u <tab> - <tab> <return> c p 8 5 2 <return>
y e s <return> M-x d e s c r i <tab> c o d <tab> <return>
<return> C-u C-x = M-x r e p o <tab> r <tab> <retu
rn>

Recent messages:
("d:\\Program_Files\\emacs\\bin\\emacs.exe")
For information about GNU Emacs and the GNU system, type C-h C-a.
win_cp.txt has auto save data; consider M-x recover-this-file
Making completion list...
You can run the command `revert-buffer-with-coding-system' with C-x RET r
You can run the command `describe-coding-system' with C-h C

Char: ± (261, #o405, #x105, file #xA5) point=1 of 20 (0%) column=0
Making completion list...

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils pp wid-edit descr-text mule-diag help-mode
easymenu time-date tooltip ediff-hook vc-hooks lisp-float-type mwheel
dos-w32 ls-lisp w32-common-fns disp-table w32-win w32-vars tool-bar dnd
fontset image regexp-opt fringe tabulated-list newcomment lisp-mode
register page menu-bar rfn-eshadow timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer loaddefs button faces cus-face macroexp files text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process w32 multi-tty
emacs)

bug#15274: 24.3; space instead of codepoint 261 on windows console with cp852

[Eli Zaretskii]

> Date: Thu, 05 Sep 2013 12:17:05 +0200
> From: Jarek Czekalski <jarekczek@poczta.onet.pl>
> 
> I start emacs -Q -nw, set terminal encoding to cp852.
> When the file is loaded into console emacs (-nw) and the correct
> file encoding is assured (revert-buffer-with-coding, cp852)
> then some characters are displayed correctly and some not. Those
> displayed incorrectly are replaced with spaces. For example
> the first one in the file ("a" with a tail, U+0105) is displayed
> incorrectly and the second one ("c" with an accent, U+0107)
> is displayed correctly.

Does the file display correctly if you visit it with the following
command, which instructs Emacs to visit the file as cp852 to begin
with:

  C-x RET c cp852 RET C-x C-f win_cp.txt RET

instead of first visiting it with the default encoding, and then
reverting it with the correct encoding?

bug#15274: 24.3; space instead of codepoint 261 on windows console with cp852

[Lars Ingebrigtsen]

Jarek Czekalski <jarekczek@poczta.onet.pl> writes:

> Another way of reproducing
>
> 1. enter scratch pad
> 2. set-terminal-coding-system cp852
> 2. set-input-method polish-slash
> 3. type /c/a - the first character displays correctly, the second is a space

(I'm going through old bug reports that unfortunately got little response at
the time.)

Do you still see this issue in recent Emacs/Windows versions?  (I tried
reproducing it on Debian, but things seemed to work as expected here.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#15274: 24.3; space instead of codepoint 261 on windows console with cp852

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Do you still see this issue in recent Emacs/Windows versions?  (I tried
> reproducing it on Debian, but things seemed to work as expected here.)

More information was requested, but no response was given within a
month, so I'm closing this bug report.  If the problem still exists,
please respond to this email and we'll reopen the bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no