bug#19347: 24.4.50; Display of bengali text

bug#19347: 24.4.50; Display of bengali text

[Richard Stallman]

This word of six characters of Bengali text, followed by a space,

|        |

fills 7 columns on this Linux terminal with nonspace output.
It causes cursor motion anomalies; when followed by more text,
it causes the line to get too much text in it.




In GNU Emacs 24.4.50.1 (i686-pc-linux-gnu, GTK+ Version 2.24.10)
 of 2014-07-18 on rms
Repository revision: 117549 dmantipov@yandex.ru-20140718110437-w0451dqdaa977r4w
System Description:	Trisquel GNU/Linux 6.0.1, Toutatis

Configured using:
 `configure 'CFLAGS=-g -O0''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND GPM DBUS GCONF GSETTINGS NOTIFY
LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  shell-dirtrack-mode: t
  gpm-mouse-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  abbrev-mode: t

Recent messages:
Mark set
File /home/rms/foo already exists; overwrite anyway? (y or n) y
Wrote /home/rms/foo

Char: SPC (32, #o40, #x20) point=32 of 80 (39%) column=30
Quit

Char:   (2480, #o4660, #x9b0, file ...) point=9 of 56 (14%) column=7
Saving file /home/rms/foo...
Wrote /home/rms/foo

Load-path shadows:
/home/rms/emacs-bzr/trunk/lisp/leim/quail/ethiopic hides /home/rms/emacs-bzr/trunk/lisp/language/ethiopic
/home/rms/emacs-bzr/trunk/lisp/leim/quail/slovak hides /home/rms/emacs-bzr/trunk/lisp/language/slovak
/home/rms/emacs-bzr/trunk/lisp/leim/quail/lao hides /home/rms/emacs-bzr/trunk/lisp/language/lao
/home/rms/emacs-bzr/trunk/lisp/leim/quail/indian hides /home/rms/emacs-bzr/trunk/lisp/language/indian
/home/rms/emacs-bzr/trunk/lisp/leim/quail/cyrillic hides /home/rms/emacs-bzr/trunk/lisp/language/cyrillic
/home/rms/emacs-bzr/trunk/lisp/leim/quail/tibetan hides /home/rms/emacs-bzr/trunk/lisp/language/tibetan
/home/rms/emacs-bzr/trunk/lisp/leim/quail/georgian hides /home/rms/emacs-bzr/trunk/lisp/language/georgian
/home/rms/emacs-bzr/trunk/lisp/leim/quail/greek hides /home/rms/emacs-bzr/trunk/lisp/language/greek
/home/rms/emacs-bzr/trunk/lisp/leim/quail/japanese hides /home/rms/emacs-bzr/trunk/lisp/language/japanese
/home/rms/emacs-bzr/trunk/lisp/leim/quail/thai hides /home/rms/emacs-bzr/trunk/lisp/language/thai
/home/rms/emacs-bzr/trunk/lisp/leim/quail/hebrew hides /home/rms/emacs-bzr/trunk/lisp/language/hebrew
/home/rms/emacs-bzr/trunk/lisp/leim/quail/czech hides /home/rms/emacs-bzr/trunk/lisp/language/czech
/home/rms/emacs-bzr/trunk/lisp/progmodes/cc-compat hides /home/rms/emacs-bzr/trunk/lisp/obsolete/cc-compat

Features:
(descr-text log-edit easy-mmode pcvs-util add-log vc vc-dispatcher
info time-stamp texinfo etags rect eieio-opt speedbar sb-image ezimage
dframe pp shadow emacsbug sh-script smie executable debug epa-file
wid-edit cl-loaddefs cl-lib novice two-column iso-transl compare-w
rmailkwd ffap thingatpt url-parse auth-source eieio byte-opt bytecomp
byte-compile cconv eieio-core gnus-util password-cache url-vars kmacro
parse-time vc-cvs sgml-mode battery tex-mode latexenc jka-compr vc-bzr
find-func shell pcomplete grep compile comint ansi-color ring
dired-aux rmailsum misearch multi-isearch mule-util cal-move cal-menu
calendar cal-loaddefs rmailout epa-mail epa derived epg epg-config
ispell quail help-mode dabbrev mailalias qp rmailmm message sendmail
format-spec rfc822 mml easymenu mml-sec mm-decode mm-bodies mm-encode
mailabbrev gmm-utils mailheader mail-parse rfc2231 rmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils dired t-mouse package
time-date paren cus-start cus-load advice help-fns tooltip electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment
lisp-mode prog-mode register page menu-bar rfn-eshadow timer select
scroll-bar mouse jit-lock font-lock syntax facemenu font-core frame
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet
lao korean japanese hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese case-table epa-hook jka-cmpr-hook
help simple abbrev minibuffer nadvice loaddefs button faces cus-face
macroexp files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote
make-network-process dbusbind gfilenotify dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty emacs)

Memory information:
((conses 8 421151 108317)
 (symbols 24 37401 12)
 (miscs 20 2567 6250)
 (strings 16 62570 5533)
 (string-bytes 1 2279950)
 (vectors 8 31487)
 (vector-slots 4 1518782 46642)
 (floats 8 151 607)
 (intervals 28 83552 4625)
 (buffers 512 151)
 (heap 1024 23423 6674))
[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

bug#19347: 24.4.50; Display of bengali text

[Andreas Schwab]

Richard Stallman <rms@gnu.org> writes:

> This word of six characters of Bengali text, followed by a space,
>
> |        |

There are only eight spaces.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#19347: 24.4.50; Display of bengali text

[Eli Zaretskii]

> From: Andreas Schwab <schwab@suse.de>
> Date: Thu, 11 Dec 2014 14:29:20 +0100
> Cc: 19347@debbugs.gnu.org
> 
> Richard Stallman <rms@gnu.org> writes:
> 
> > This word of six characters of Bengali text, followed by a space,
> >
> > |        |
> 
> There are only eight spaces.

I think there's some general problem in how Richard has his email set
up, either in Emacs or elsewhere.  This is not the first time I see
non-ASCII characters in his mails replaced by blanks.

bug#19347: 24.4.50; Display of bengali text

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

This time I hope they come through.

|সাজেদুর |


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

bug#19347: 24.4.50; Display of bengali text

[Eli Zaretskii]

> Date: Fri, 12 Dec 2014 11:41:22 -0500
> From: Richard Stallman <rms@gnu.org>
> CC: schwab@suse.de, 19347@debbugs.gnu.org
> 
> This time I hope they come through.
> 
> |সাজেদুর |

It did, thanks.  However, I see no anomalies with these characters,
neither on a TTY nor on a GUI frame.  (But I don't have access to a
Linux terminal.)

What does evaluating (terminal-coding-system) on that terminal yield?

bug#19347: 24.4.50; Display of bengali text

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > What does evaluating (terminal-coding-system) on that terminal yield?

utf-8.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

bug#19347: 24.4.50; Display of bengali text

[Andreas Schwab]

Richard Stallman <rms@gnu.org> writes:

> fills 7 columns on this Linux terminal with nonspace output.

Whatis "this Linux terminal"?

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#19347: 24.4.50; Display of bengali text

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Whatis "this Linux terminal"?

I run Emacs on a Linux terminal, on my GNU/Linux system.
It is more convenient than using a graphics terminal and having
to worry about the pointer.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

bug#19347: 24.4.50; Display of bengali text

[Andreas Schwab]

Richard Stallman <rms@gnu.org> writes:

> I run Emacs on a Linux terminal, on my GNU/Linux system.
> It is more convenient than using a graphics terminal and having
> to worry about the pointer.

Then there is nothing that can be done about the display of your text,
since that is fully controlled by the kernel.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#19347: 24.4.50; Display of bengali text

[Stefan Monnier]

>> I run Emacs on a Linux terminal, on my GNU/Linux system.
>> It is more convenient than using a graphics terminal and having
>> to worry about the pointer.
> Then there is nothing that can be done about the display of your text,
> since that is fully controlled by the kernel.

Maybe fbterm handles those characters better?


        Stefan

bug#19347: 24.4.50; Display of bengali text

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Maybe fbterm handles those characters better?

What is fbterm, and how does it relate?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

bug#19347: 24.4.50; Display of bengali text

[Stefan Monnier]

>> Maybe fbterm handles those characters better?
> What is fbterm, and how does it relate?

It's kind of like xterm but instead of running under X11 it uses
directly the framebuffer device.  So it is similar to Linux's builtin
terminal emulator (it uses up the whole screen, there's no GUI layer of
any kind), but it's a user program.

Try `aptitude install fbterm' and then run `fbterm' from your console
command line.


        Stefan

bug#19347: 24.4.50; Display of bengali text

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Wed, 17 Dec 2014 16:40:17 -0500
> Cc: schwab@suse.de, 19347@debbugs.gnu.org
> 
> >> Maybe fbterm handles those characters better?
> > What is fbterm, and how does it relate?
> 
> It's kind of like xterm but instead of running under X11 it uses
> directly the framebuffer device.  So it is similar to Linux's builtin
> terminal emulator (it uses up the whole screen, there's no GUI layer of
> any kind), but it's a user program.
> 
> Try `aptitude install fbterm' and then run `fbterm' from your console
> command line.

Are we sure fbterm won't try to compose those characters, and create a
similar problem?  At least PuTTY seems to do that.

bug#19347: 24.4.50; Display of bengali text

[Stefan Monnier]

>> >> Maybe fbterm handles those characters better?
[...]
> Are we sure fbterm won't try to compose those characters,

Definitely not, which is why I said "maybe".
IME, fbterm is nice but is not super-polished (e.g. when I move the
mouse, the screen gets garbled, revealing the "underlying Linux
terminal text").


        Stefan

bug#19347: 24.4.50; Display of bengali text

[Wolfgang Jenkner]

On Thu, Dec 18 2014, Stefan Monnier wrote:

>>> >> Maybe fbterm handles those characters better?
> [...]
>> Are we sure fbterm won't try to compose those characters,
>
> Definitely not, which is why I said "maybe".
> IME, fbterm is nice but is not super-polished (e.g. when I move the
> mouse, the screen gets garbled, revealing the "underlying Linux
> terminal text").

Which reminds me that a term/fbterm.el could be added rather easily,
I think, see

http://permalink.gmane.org/gmane.emacs.devel/143665

and the corresponding thread (but I don't actually know how initializing
emacs for a terminal really works).

bug#19347: 24.4.50; Display of bengali text

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It's kind of like xterm but instead of running under X11 it uses
  > directly the framebuffer device.  So it is similar to Linux's builtin
  > terminal emulator (it uses up the whole screen, there's no GUI layer of
  > any kind), but it's a user program.

It's good if fbterm avoids the problem, but
Emacs needs to work correctly with Linux terminals.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

bug#19347: 24.4.50; Display of bengali text

[Andreas Schwab]

Richard Stallman <rms@gnu.org> writes:

> It's good if fbterm avoids the problem, but
> Emacs needs to work correctly with Linux terminals.

Which font are you using for the Linux console?  With lat9w-16.psfu I
don't see any glitches.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#19347: 24.4.50; Display of bengali text

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Which font are you using for the Linux console?  With lat9w-16.psfu I
  > don't see any glitches.

Can you tell me what command to find that with?


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

bug#19347: 24.4.50; Display of bengali text

[Andreas Schwab]

Richard Stallman <rms@gnu.org> writes:

>   > Which font are you using for the Linux console?  With lat9w-16.psfu I
>   > don't see any glitches.
>
> Can you tell me what command to find that with?

On my system, it's set from CONSOLE_FONT in /etc/sysconfig/console.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#19347: 24.4.50; Display of bengali text

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > I run Emacs on a Linux terminal, on my GNU/Linux system.
  > > It is more convenient than using a graphics terminal and having
  > > to worry about the pointer.

  > Then there is nothing that can be done about the display of your text,
  > since that is fully controlled by the kernel.

Emacs decides what characters to send to the terminal.  Perhaps those
should be different.

Or maybe they are correct, and this is a bug in Linux.

How can we determine which?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

bug#19347: 24.4.50; Display of bengali text

[Andreas Schwab]

Richard Stallman <rms@gnu.org> writes:

> Emacs decides what characters to send to the terminal.  Perhaps those
> should be different.

Emacs cannot know how the terminal renders the output.  It only knows
the defined meaning of the (control) characters.  If the terminal
doesn't adhere to this meaning then it has a bug.

> How can we determine which?

Look at the typescript.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#19347: 24.4.50; Display of bengali text

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > How can we determine which?

  > Look at the typescript.

I do not understand.  What do you mean, "the typescript"?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

bug#19347: 24.4.50; Display of bengali text

[Andreas Schwab]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > > How can we determine which?
>
>   > Look at the typescript.
>
> I do not understand.  What do you mean, "the typescript"?

Sorry, I meant the termscript.

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#19347: 24.4.50; Display of bengali text

[Eli Zaretskii]

> Date: Tue, 16 Dec 2014 00:20:06 -0500
> From: Richard Stallman <rms@gnu.org>
> Cc: 19347@debbugs.gnu.org
> 
>   > Whatis "this Linux terminal"?
> 
> I run Emacs on a Linux terminal, on my GNU/Linux system.

Can you describe in more detail what exactly goes wrong when these
characters are displayed?

Also, do the problems go away if you say

  C-x RET t iso-safe RET

(The characters will be displayed as "?", but what I'd like to know is
whether the problems you saw disappear then.)

bug#19347: 24.4.50; Display of bengali text

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Can you describe in more detail what exactly goes wrong when these
  > characters are displayed?

The 6 characters fill 7 columns with the blob that indicates a character
that can't be displayed.  This causes anomalies in cursor motion
within the line.

If the line is almost 80 characters, it overflows
the line on the terminal, causing vertical positioning anomalies too.

These consequences typically occur when text sent to the terminal
occupies more columns than Emacs expects it to occupy.

  > Also, do the problems go away if you say

  >   C-x RET t iso-safe RET

Yes, they go away.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

bug#19347: 24.4.50; Display of bengali text

[Andreas Schwab]

Richard Stallman <rms@gnu.org> writes:

> The 6 characters fill 7 columns with the blob that indicates a character
> that can't be displayed.  This causes anomalies in cursor motion
> within the line.

What happens if you echo the characters to the terminal outside of
Emacs?

Andreas.

-- 
Andreas Schwab, SUSE Labs, schwab@suse.de
GPG Key fingerprint = 0196 BAD8 1CE9 1970 F4BE  1748 E4D4 88E3 0EEA B9D7
"And now for something completely different."

bug#19347: 24.4.50; Display of bengali text

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > What happens if you echo the characters to the terminal outside of
  > Emacs?

They occupy 7 colunms that way, just as they do in Emacs.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

bug#19347: 24.4.50; Display of bengali text

[Eli Zaretskii]

> Date: Wed, 17 Dec 2014 10:44:24 -0500
> From: Richard Stallman <rms@gnu.org>
> CC: schwab@suse.de, 19347@debbugs.gnu.org
> 
> The 6 characters fill 7 columns with the blob that indicates a character
> that can't be displayed.  This causes anomalies in cursor motion
> within the line.
> 
> If the line is almost 80 characters, it overflows
> the line on the terminal, causing vertical positioning anomalies too.
> 
> These consequences typically occur when text sent to the terminal
> occupies more columns than Emacs expects it to occupy.

Right.  Does it help to disable compositions?

  M-x auto-composition-mode RET

bug#19347: 24.4.50; Display of bengali text

[Andreas Schwab]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Wed, 17 Dec 2014 10:44:24 -0500
>> From: Richard Stallman <rms@gnu.org>
>> CC: schwab@suse.de, 19347@debbugs.gnu.org
>> 
>> The 6 characters fill 7 columns with the blob that indicates a character
>> that can't be displayed.  This causes anomalies in cursor motion
>> within the line.
>> 
>> If the line is almost 80 characters, it overflows
>> the line on the terminal, causing vertical positioning anomalies too.
>> 
>> These consequences typically occur when text sent to the terminal
>> occupies more columns than Emacs expects it to occupy.
>
> Right.  Does it help to disable compositions?
>
>   M-x auto-composition-mode RET

Composition mode has no effect in a termcap frame.  I can see a similar
effect in emacs -nw running in Kconsole, because Konsole is apparently
doing composition on its own, causing the (uncomposed) characters no
longer occupy exactly one column.  Moving over such a composed (by the
terminal) character temporarily breaks up the composition.

But on the Linux console I'm only seeing replacement characters without
any display glitches.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#19347: 24.4.50; Display of bengali text

[Eli Zaretskii]

> From: Andreas Schwab <schwab@suse.de>
> Cc: rms@gnu.org,  19347@debbugs.gnu.org
> Date: Wed, 17 Dec 2014 21:02:53 +0100
> 
> >   M-x auto-composition-mode RET
> 
> Composition mode has no effect in a termcap frame.  I can see a similar
> effect in emacs -nw running in Kconsole, because Konsole is apparently
> doing composition on its own, causing the (uncomposed) characters no
> longer occupy exactly one column.  Moving over such a composed (by the
> terminal) character temporarily breaks up the composition.

I see something similar on PuTTY, which duped me into thinking
composition mode is somehow a factor.

So I guess this means we should somehow disable the composition on
TTYs, or tell users to do that, right?

bug#19347: 24.4.50; Display of bengali text

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Right.  Does it help to disable compositions?

  >   M-x auto-composition-mode RET

When that mode is disabled, the problem is fixed.
I can't reproduce any problem with that text
with Auto-Composition mode disabled.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

bug#19347: 24.4.50; Display of bengali text

[Eli Zaretskii]

> Date: Thu, 18 Dec 2014 06:25:22 -0500
> From: Richard Stallman <rms@gnu.org>
> CC: schwab@suse.de, 19347@debbugs.gnu.org
> 
>   > Right.  Does it help to disable compositions?
> 
>   >   M-x auto-composition-mode RET
> 
> When that mode is disabled, the problem is fixed.
> I can't reproduce any problem with that text
> with Auto-Composition mode disabled.

That's a surprise.  Can you look with "od -c" at termscript created
with the Auto-Composition mode enabled and disabled, and see if Emacs
sends different bytes to the screen in these two cases?

bug#19347: 24.4.50; Display of bengali text

[Eli Zaretskii]

> Date: Thu, 18 Dec 2014 06:25:22 -0500
> From: Richard Stallman <rms@gnu.org>
> CC: schwab@suse.de, 19347@debbugs.gnu.org
> 
>   > Right.  Does it help to disable compositions?
> 
>   >   M-x auto-composition-mode RET
> 
> When that mode is disabled, the problem is fixed.
> I can't reproduce any problem with that text
> with Auto-Composition mode disabled.

Could you please send the data I requested, i.e. the stuff Emacs sends
to the terminal with and without Auto-Composition mode, as you find it
in the termscript file?  I'd like to figure out why Auto-Composition
had any effect on TTY output in this case.

Thanks.

bug#19347: 24.4.50; Display of bengali text

[Richard Stallman]

[-- Attachment #1: Type: text/plain, Size: 537 bytes --]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

enabled.out is output with Auto-Composition mode enabled.
disabled.out is output with Auto-Composition mode disabled.  They
don't correspond to the same commands, but they both display the same
buffer.  With Auto-Composition mode enabled, the screen was messed up.
With Auto-Composition mode disabled, display was correct.

[-- Attachment #2: enabled.out --]
[-- Type: application/octet-stream, Size: 1603 bytes --]

[-- Attachment #3: disabled.out --]
[-- Type: application/octet-stream, Size: 908 bytes --]

[-- Attachment #4: Type: text/plain, Size: 219 bytes --]


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

bug#19347: 24.4.50; Display of bengali text

[Eli Zaretskii]

Added Handa-san, as this seems to be some subtle issue with composed
characters.

The beginning of this thread is here:

  http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19347

The files Richard sent are here:

  http://debbugs.gnu.org/cgi/bugreport.cgi?bug=19347#101

> Date: Sat, 27 Dec 2014 17:55:42 -0500
> From: Richard Stallman <rms@gnu.org>
> CC: schwab@suse.de, 19347@debbugs.gnu.org
> 
> enabled.out is output with Auto-Composition mode enabled.
> disabled.out is output with Auto-Composition mode disabled.  They
> don't correspond to the same commands, but they both display the same
> buffer.  With Auto-Composition mode enabled, the screen was messed up.
> With Auto-Composition mode disabled, display was correct.

The difference is the character whose codepoint is 0x09C1.  This
character's general category is "non-spacing mark".  When
Auto-Composition mode is disabled, we compute the width of this
character as zero, and don't add it to the glyph row, so it doesn't
get written to the terminal.  When Auto-Composition mode is enabled,
we compose this character with the preceding one, and send them both
to the terminal.

More importantly, it looks like, when a TTY frame displays on a
terminal in a UTF-8 locale, we send composed characters to the
terminal, and I'm not sure text-mode terminals and/or our TTY display
code are ready for that.  For example, moving the cursor seems to send
only part of the composition sequence to the screen, which produces
garbled display.

Should we disable (or bypass) Auto-Composition mode on TTY frames?

bug#19347: 24.4.50; Display of bengali text

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Should we disable (or bypass) Auto-Composition mode on TTY frames?

(I'm going through old bug reports that unfortunately weren't resolved
at the time.)

It seems like this was fixed in term/linux.el by doing pretty much that
about a year ago, so I'm therefore closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no