combining chars issue

combining chars issue

[Emanuele Giaquinta]

[-- Attachment #1: Type: text/plain, Size: 476 bytes --]

Hi,

with latest cvs combining chars are not handled properly in emacs -Q -nw.
It seems emacs does not write the combining chars at all on stdout, so
they are not displayed.
Attached is an utf-8 encoded file that should exhibit the problem and a
typescript of emacs $file. As you can see from the typescript, emacs
writes U+39b but not U+300. However, what-cursor-position reports U+300
in the cell next to the U+39b one, so it seems only a display issue.

Emanuele Giaquinta

[-- Attachment #2: combining.txt --]
[-- Type: text/plain, Size: 5 bytes --]

Λ̀

[-- Attachment #3: combining-typescript.txt --]
[-- Type: text/plain, Size: 775 bytes --]

Script started on Thu Jun 19 15:40:15 2008
^[[?1049h^[[?1049h^[[?25h^[=^[[H^[[2J^[[35d^[[K^[>^[[?25h^[[r^[[?1049l^[[39;49m\r^[[?1049h^[[?1049h^[[?25h^[=^[[H^[[2J^[[35d^[[?25lWhen done with a buffer, type C-x #^[[K^[[H^[[?25h^[[?25h^[[35d^[[?25lWhen done with a buffer, type C-x #^[[K^[[HΛ^[[K
\b^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[K
^[[38;5;0m^[[48;5;85m-UUU:@----F15  ^[[39;49m^[[1m^[[38;5;0m^[[48;5;85mfoo         ^[[m\x0f^[[39;49m^[[38;5;0m^[[48;5;85m   All (1,0)      (Fundamental Server)------------------------------------^[[39;49m
^[[A^[[H^[[?25h^[[?25h^[[35d^[[K^[[H^[[35d^[[?25l(No files need saving)^[[H^[[?25h^[[?25h^[[35d^[[K^[>^[[?25h^[[r^[[?1049l^[[39;49m
Script done on Thu Jun 19 15:40:16 2008

Re: combining chars issue

[Trent W. Buck]

Emanuele Giaquinta <emanuele.giaquinta@gmail.com> writes:

> with latest cvs combining chars are not handled properly in emacs -Q -nw.

I can reproduce this behaviour.  Specifically, if I "cat x.txt", I see
an uppercase lambda with what looks like a grave diacritic, but if I
open the file with "emacs -nw -Q x.txt", I only see the lambda -- no
diacritic.

I'm running emacs-snapshot (http://emacs.orebokech.com) on Debian Lenny/Sid.

emacs-snapshot 1:20080613-1
screen         4.0.3-9
xterm          235-1

Re: combining chars issue

[Emanuele Giaquinta]

[-- Attachment #1: Type: text/plain, Size: 388 bytes --]

On Thu, Jun 19, 2008 at 04:30:57PM +0200, Emanuele Giaquinta wrote:

> with latest cvs combining chars are not handled properly in emacs -Q -nw.
> It seems emacs does not write the combining chars at all on stdout, so
> they are not displayed.

The problem seems to be in term.c:append_glyph, which does not append
any glyph when it->pixel_width == 0.
Does the attached patch make sense?

[-- Attachment #2: combining.diff --]
[-- Type: text/x-diff, Size: 590 bytes --]

diff --git a/src/term.c b/src/term.c
index 533104d..281834e 100644
--- a/src/term.c
+++ b/src/term.c
@@ -1512,7 +1512,7 @@ append_glyph (it)
   end = it->glyph_row->glyphs[1 + it->area];
 
   for (i = 0;
-       i < it->pixel_width && glyph < end;
+       i < it->nglyphs && glyph < end;
        ++i)
     {
       glyph->type = CHAR_GLYPH;
@@ -1638,7 +1638,7 @@ produce_glyphs (it)
   else
     {
       it->pixel_width = CHAR_WIDTH (it->c);
-      it->nglyphs = it->pixel_width;
+      it->nglyphs = it->pixel_width ? it->pixel_width : 1;
 
       if (it->glyph_row)
 	append_glyph (it);

Re: combining chars issue

[Kenichi Handa]

Sorry for the late response on this matter.
In article <20080619143057.GA85105@orion.lan>, Emanuele Giaquinta <emanuele.giaquinta@gmail.com> writes:

> with latest cvs combining chars are not handled properly in emacs -Q -nw.
> It seems emacs does not write the combining chars at all on stdout, so
> they are not displayed.
> Attached is an utf-8 encoded file that should exhibit the problem and a
> typescript of emacs $file. As you can see from the typescript, emacs
> writes U+39b but not U+300. However, what-cursor-position reports U+300
> in the cell next to the U+39b one, so it seems only a display issue.

I've just installed a fix.

> The problem seems to be in term.c:append_glyph, which does not append
> any glyph when it->pixel_width == 0.
> Does the attached patch make sense?

No.  The sequence of base character and the following
combining characters must be composed to one grapheme
cluster to provide a reasonable cursor movement.  The
problem was that auto-composition-mode didn't handle the
case of terminal display.

---
Kenichi Handa
handa@ni.aist.go.jp

Re: combining chars issue

[Emanuele Giaquinta]

On Thu, Jun 26, 2008 at 05:07:11PM +0900, Kenichi Handa wrote:

> > with latest cvs combining chars are not handled properly in emacs -Q -nw.
> > It seems emacs does not write the combining chars at all on stdout, so
> > they are not displayed.
> > Attached is an utf-8 encoded file that should exhibit the problem and a
> > typescript of emacs $file. As you can see from the typescript, emacs
> > writes U+39b but not U+300. However, what-cursor-position reports U+300
> > in the cell next to the U+39b one, so it seems only a display issue.
> 
> I've just installed a fix.
> 
> > The problem seems to be in term.c:append_glyph, which does not append
> > any glyph when it->pixel_width == 0.
> > Does the attached patch make sense?
> 
> No.  The sequence of base character and the following
> combining characters must be composed to one grapheme
> cluster to provide a reasonable cursor movement.  The
> problem was that auto-composition-mode didn't handle the
> case of terminal display.

Ok I see, thank you.