bug#6026: 23.1.96; Cursor drawing problems with the Khmer script

bug#6026: 23.1.96; Cursor drawing problems with the Khmer script

[Eli Zaretskii]

Type "C-h H" in "emacs -Q", then go to the line of the Khmer greeting
and move cursor.  Both forward and backward movement has problems:
with forward movement, the cursor sometimes disappears (or maybe
degenerates into very small bitmap), and backward movement leaves
strange artifacts on the screen.

This problem exists in Emacs 23.1 as well.


In GNU Emacs 23.1.96.1 (i386-mingw-nt5.1.2600)
 of 2010-04-20 on HOME-C4E4A596F7
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (3.4)'

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: ENU
  value of $XMODIFIERS: nil
  locale-coding-system: cp1255
  default enable-multibyte-characters: t

Major mode: Mail

Minor modes in effect:
  diff-auto-refine-mode: t
  shell-dirtrack-mode: t
  flyspell-mode: t
  desktop-save-mode: t
  show-paren-mode: t
  display-time-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  temp-buffer-resize-mode: t
  line-number-mode: t
  abbrev-mode: t

Recent input:
<M-backspace> t r <tab> s r <tab> C h a n g <tab> <return> 
<up> <up> <up> <up> <right> C-SPC <down> <down> <down> 
C-w <down> <down> <down> <down> <down> <down> <up> 
<up> <up> <up> <up> <up> C-y C-/ <down> <down> <down> 
<down> <down> <down> C-k C-k <down> <down> C-k C-k 
C-x C-s <help-echo> <switch-frame> <help-echo> <help-echo> 
<help-echo> <switch-frame> <M-end> C-x k <return> <help-echo> 
<M-home> <up> <up> <up> <up> <up> <up> <up> <up> C-SPC 
<down> <down> <down> <down> M-w <M-end> <help-echo> 
<help-echo> C-y <delete> C-x C-x <return> <up> F D 
<backspace> i x SPC c u r s o r SPC m o t i o n SPC 
a t SPC e n d SPC o f SPC l i n e SPC t h a t SPC e 
n d s SPC i n SPC a SPC c o m p o s i t e SPC c h a 
r a c t e r . <return> <C-left> <C-left> <C-right> 
<backspace> <backspace> <backspace> e d <down> <right> 
<delete> <delete> <down> <delete> SPC <down> <delete> 
SPC <down> <delete> SPC <down> C-x C-s C-x # <help-echo> 
<help-echo> <switch-frame> p p p p p p p d p p p r 
<C-home> <down> <right> <right> <right> <right> C-SPC 
<down> C-w <C-end> C-c C-y C-x C-x <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <C-end> C-w <return> F i x SPC i t , SPC I S-SPC 
t h i n k SPC ( r e v n o SPC 1 0 0 0 2 5 ) . <return> 
<right> C-c C-s <help-echo> <switch-frame> o <return> 
p p p p p p n d p p p p p p n p p <M-home> <C-end> 
<M-end> <switch-frame> M-x r e p o r t - e m <tab> 
<return>

Recent messages:
Mark set
Saving file d:/usr/tmp/bzr_log.uvwh-x...
Wrote d:/usr/tmp/bzr_log.uvwh-x
Mark set [5 times]
call-interactively: End of buffer
Sending...
Added to d:/usr/eli/rmail/SENT.MAIL
Sending...done
Added to d:/usr/eli/rmail/BIDI.rmail
Mark set

Load-path shadows:
None found.

Features:
(vc-dispatcher smerge-mode diff-mode thai-util thai-word mule-util
lao-util rmailmm gud shell rmailedit find-func shadow emacsbug
multi-isearch help-mode view texinfo auth-source message ecomplete
rfc822 mml mml-sec password-cache mm-decode mm-bodies mm-encode
mailcap mail-parse rfc2231 rfc2047 rfc2045 qp ietf-drums nnheader
gnus-util netrc mm-util mail-prsvr gmm-utils wid-edit mailheader
canlock hashcash smtpmail dabbrev mailalias mailabbrev sendmail
rmailout conf-mode newcomment ld-script sh-script executable dired-x
dired-aux dired tcl generic parse-time nxml-uchnm rng-xsd xsd-regexp
rng-cmpct rng-nxml rng-valid rng-loc rng-uri rng-parse nxml-parse
rng-match rng-dt rng-util rng-pttrn nxml-ns nxml-mode nxml-outln
nxml-rap nxml-util nxml-glyph nxml-enc xmltok sgml-mode arc-mode
archive-mode make-mode python-21 python comint ring jka-compr vc-cvs
info add-log flyspell ispell org-wl org-w3m org-vm org-rmail org-mhe
org-mew org-irc org-jsinfo org-infojs org-html org-exp org-exp-blocks
org-agenda org-info org-gnus org-bibtex org-bbdb org byte-opt bytecomp
byte-compile advice help-fns advice-preload org-footnote org-src
org-list org-faces org-compat org-macs time-date noutline outline
easy-mmode vc-bzr sha1 hex-util cc-mode cc-fonts easymenu cc-menus
cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs regexp-opt
rmailsum rmail mail-utils desktop server filecache saveplace generic-x
paren battery time tooltip ediff-hook vc-hooks lisp-float-type mwheel
dos-w32 disp-table ls-lisp w32-win w32-vars tool-bar dnd fontset image
fringe lisp-mode register page menu-bar rfn-eshadow timer select
scroll-bar mldrag mouse jit-lock font-lock syntax facemenu font-core
frame cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev loaddefs button minibuffer faces
cus-face files text-properties overlay md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote
make-network-process multi-tty emacs)

bug#6026: 23.1.96; Cursor drawing problems with the Khmer script

[Kenichi Handa]

In article <83iq7h13wb.fsf@gnu.org>, Eli Zaretskii <eliz@gnu.org> writes:

> Type "C-h H" in "emacs -Q", then go to the line of the Khmer greeting
> and move cursor.  Both forward and backward movement has problems:
> with forward movement, the cursor sometimes disappears (or maybe
> degenerates into very small bitmap), and backward movement leaves
> strange artifacts on the screen.

> This problem exists in Emacs 23.1 as well.

I don't see that problem on my GNU/Linux systems.  Perhaps,
you have a font that contains Khmer characters but doesn't
have proper OpenType tables.  Which font is used for Khmer?

---
Kenichi Handa
handa@m17n.org

bug#6026: 23.1.96; Cursor drawing problems with the Khmer script

[Eli Zaretskii]

> From: Kenichi Handa <handa@m17n.org>
> Cc: 6026@debbugs.gnu.org
> Date: Mon, 26 Apr 2010 20:35:54 +0900
> 
> I don't see that problem on my GNU/Linux systems.  Perhaps,
> you have a font that contains Khmer characters but doesn't
> have proper OpenType tables.

How to know that?

> Which font is used for Khmer?

This one:

  uniscribe:-outline-Code2000-normal-normal-normal-*-13-*-*-*-p-*-iso10646-1

bug#6026: 23.1.96; Cursor drawing problems with the Khmer script

[Jason Rumney]

On 27/04/2010 01:27, Eli Zaretskii wrote:
>> From: Kenichi Handa<handa@m17n.org>
>> Cc: 6026@debbugs.gnu.org
>> Date: Mon, 26 Apr 2010 20:35:54 +0900
>>
>> I don't see that problem on my GNU/Linux systems.  Perhaps,
>> you have a font that contains Khmer characters but doesn't
>> have proper OpenType tables.
>>      
> How to know that?
>    

(list-fonts (font-spec :registry "iso10646-1" :otf '(khmr nil (pres))))

>> Which font is used for Khmer?
>>      
> This one:
>
>    uniscribe:-outline-Code2000-normal-normal-normal-*-13-*-*-*-p-*-iso10646-1
>    

I doubt that has OTF tables for Khmer - try a Khmer specific font (I 
have one installed called "Khmer OS System")

bug#6026: 23.1.96; Cursor drawing problems with the Khmer script

[Eli Zaretskii]

> Date: Tue, 27 Apr 2010 08:17:52 +0800
> From: Jason Rumney <jasonr@f2s.com>
> Cc: 
> 
> On 27/04/2010 01:27, Eli Zaretskii wrote:
> >> From: Kenichi Handa<handa@m17n.org>
> >> Cc: 6026@debbugs.gnu.org
> >> Date: Mon, 26 Apr 2010 20:35:54 +0900
> >>
> >> I don't see that problem on my GNU/Linux systems.  Perhaps,
> >> you have a font that contains Khmer characters but doesn't
> >> have proper OpenType tables.
> >>      
> > How to know that?
> >    
> 
> (list-fonts (font-spec :registry "iso10646-1" :otf '(khmr nil (pres))))

This produces a long list, where I see Code2000 several times.  What
am I looking for?

> I doubt that has OTF tables for Khmer - try a Khmer specific font (I 
> have one installed called "Khmer OS System")

Will do.

bug#6026: 23.1.96; Cursor drawing problems with the Khmer script

[Jason Rumney]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Tue, 27 Apr 2010 08:17:52 +0800
>> From: Jason Rumney <jasonr@f2s.com>
>> Cc: 
>> 
>> On 27/04/2010 01:27, Eli Zaretskii wrote:
>> >> From: Kenichi Handa<handa@m17n.org>
>> >> Cc: 6026@debbugs.gnu.org
>> >> Date: Mon, 26 Apr 2010 20:35:54 +0900
>> >>
>> >> I don't see that problem on my GNU/Linux systems.  Perhaps,
>> >> you have a font that contains Khmer characters but doesn't
>> >> have proper OpenType tables.
>> >>      
>> > How to know that?
>> >    
>> 
>> (list-fonts (font-spec :registry "iso10646-1" :otf '(khmr nil (pres))))
>
> This produces a long list, where I see Code2000 several times.  What
> am I looking for?

I'm surprised you get a long list for such a minority language - my
system only has "Khmer OS System", which I installed specifically for
Khmer support.

>> I doubt that has OTF tables for Khmer - try a Khmer specific font (I 
>> have one installed called "Khmer OS System")

Actually, it must have some otf tables for khmer, as it is one of the
fonts recommended for viewing of Khmer here (though prefixed with "quick
and dirty"):

http://www.bauhahnm.clara.net/Khmer/Welcome.html

Now that I've tried your original report on both GNU/Linux and Windows,
I see the bug on Windows. It may be due to the version of usp10.dll - I
remember when I originally worked on the uniscribe code, I had to
upgrade usp10.dll manually to get support for some languages - Tibetan
and Sinhalese I specifically remember, but there may have been others.
But according to info I've found, the last update to usp10.dll was
shipped as part of XP SP2, bringing Khmer support amongst others, so
there may be another bug here, or just a difference in the way uniscribe
maps glyphs compared to libotf which affects Khmer (and I think
Burmese) but not the Indic scripts.


Also, according to this page: 

  http://www.microsoft.com/typography/otfntdev/khmerot/features.aspx

the following otf gsub features are required to properly display Khmer:

  pref blwf abvf pstf pres blws abvs psts clig

So the following might be a better font spec, at least for Windows:

(font-spec :registry "iso10646-1" :otf '(khmr nil
                                        (pref blwf abvf pstf pres blws
                                         abvs psts clig)))

bug#6026: 23.1.96; Cursor drawing problems with the Khmer script

[Eli Zaretskii]

> From: Jason Rumney <jasonr@gnu.org>
> Cc: bug-gnu-emacs@gnu.org
> Date: Wed, 28 Apr 2010 00:14:49 +0800
> 
> Now that I've tried your original report on both GNU/Linux and Windows,
> I see the bug on Windows. It may be due to the version of usp10.dll - I
> remember when I originally worked on the uniscribe code, I had to
> upgrade usp10.dll manually to get support for some languages - Tibetan
> and Sinhalese I specifically remember, but there may have been others.
> But according to info I've found, the last update to usp10.dll was
> shipped as part of XP SP2, bringing Khmer support amongst others, so
> there may be another bug here, or just a difference in the way uniscribe
> maps glyphs compared to libotf which affects Khmer (and I think
> Burmese) but not the Indic scripts.

FWIW, this is XP SP2.

> Also, according to this page: 
> 
>   http://www.microsoft.com/typography/otfntdev/khmerot/features.aspx
> 
> the following otf gsub features are required to properly display Khmer:
> 
>   pref blwf abvf pstf pres blws abvs psts clig
> 
> So the following might be a better font spec, at least for Windows:
> 
> (font-spec :registry "iso10646-1" :otf '(khmr nil
>                                         (pref blwf abvf pstf pres blws
>                                          abvs psts clig)))

If I invoke list-fonts on this, I get nil.  I will try to install a
better font when I can.

Are you suggesting to modify some Emacs source to put this spec?  If
so, where to put it?

bug#6026: 23.1.96; Cursor drawing problems with the Khmer script

[Juanma Barranquero]

On Tue, Apr 27, 2010 at 18:14, Jason Rumney <jasonr@gnu.org> wrote:

> I'm surprised you get a long list for such a minority language - my
> system only has "Khmer OS System", which I installed specifically for
> Khmer support.

FWIW, on Windows 7 Home I get four matches, and I haven't installed
any Khmer support (at least, not directly; perhaps some product?).

#<font-entity uniscribe outline DaunPenh nil iso10646-1 normal normal
normal 0 nil 0 nil ((:format . opentype) (:script khmer latin))>
#<font-entity uniscribe outline Khmer\ UI sans iso10646-1 bold normal
normal 0 nil 0 nil ((:format . opentype) (:script symbol khmer
latin))>
#<font-entity uniscribe outline Khmer\ UI sans iso10646-1 normal
normal normal 0 nil 0 nil ((:format . opentype) (:script symbol khmer
latin))>
#<font-entity uniscribe outline MoolBoran sans iso10646-1 normal
normal normal 0 nil 0 nil ((:format . opentype) (:script symbol khmer
latin))>

> It may be due to the version of usp10.dll - I
> remember when I originally worked on the uniscribe code, I had to
> upgrade usp10.dll manually to get support for some languages - Tibetan
> and Sinhalese I specifically remember, but there may have been others.
> But according to info I've found, the last update to usp10.dll was
> shipped as part of XP SP2

According to http://en.wikipedia.org/wiki/Uniscribe since the Sinhala
Enabling Patch there's been about a dozen new versions, the last one
in Windows 7 and the beta for Microsoft Office 10.

> So the following might be a better font spec, at least for Windows:
>
> (font-spec :registry "iso10646-1" :otf '(khmr nil
>                                        (pref blwf abvf pstf pres blws
>                                         abvs psts clig)))
>

That gets me the same list.

As for the bug, I don't see it either. Moving the cursor on the Khmer
list, the cursor appears as wide as the character it is upon (i.e., is
very small but perfectly visible over U+179A KHMER LETTER RO), and
neither forward nor backward movement shows anything unexpected.


    Juanma

bug#6026: 23.1.96; Cursor drawing problems with the Khmer script

[Jason Rumney]

On 28/04/2010 06:29, Juanma Barranquero wrote:
> On Tue, Apr 27, 2010 at 18:14, Jason Rumney<jasonr@gnu.org>  wrote:
> FWIW, on Windows 7 Home I get four matches, and I haven't installed
> any Khmer support (at least, not directly; perhaps some product?).
>
> #<font-entity uniscribe outline DaunPenh nil iso10646-1 normal normal
> normal 0 nil 0 nil ((:format . opentype) (:script khmer latin))>
> #<font-entity uniscribe outline Khmer\ UI sans iso10646-1 bold normal
> normal 0 nil 0 nil ((:format . opentype) (:script symbol khmer
> latin))>
> #<font-entity uniscribe outline Khmer\ UI sans iso10646-1 normal
> normal normal 0 nil 0 nil ((:format . opentype) (:script symbol khmer
> latin))>
> #<font-entity uniscribe outline MoolBoran sans iso10646-1 normal
> normal normal 0 nil 0 nil ((:format . opentype) (:script symbol khmer
> latin))>
>    

These all look like Khmer specific fonts to me. I get one match on 
Windows XP SP3, which is for Khmer OS System, a font I manually installed.

>> So the following might be a better font spec, at least for Windows:
>>
>> (font-spec :registry "iso10646-1" :otf '(khmr nil
>>                                         (pref blwf abvf pstf pres blws
>>                                          abvs psts clig)))
>>
>>      
> That gets me the same list.
>    

Me too, but it filters out the fonts that Eli has which appear not to 
fully support Khmer OTF tables, but contain enough support to match the 
current font-spec.  This may be part of the problem Eli is seeing, but I 
also see display corruption (cursor being left behind) when moving 
backwards through Khmer text, so there is probably also a problem with 
the version of usp10.dll on Windows XP.


> As for the bug, I don't see it either. Moving the cursor on the Khmer
> list, the cursor appears as wide as the character it is upon (i.e., is
> very small but perfectly visible over U+179A KHMER LETTER RO), and
> neither forward nor backward movement shows anything unexpected.
>    

This is different than GNU/Linux with libotf and m17n. There, the cursor 
moves over groups of characters, so even on versions of uniscribe that 
fully support Khmer fonts, the behaviour is still different than libotf.

bug#6026: 23.1.96; Cursor drawing problems with the Khmer script

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Type "C-h H" in "emacs -Q", then go to the line of the Khmer greeting
> and move cursor.  Both forward and backward movement has problems:
> with forward movement, the cursor sometimes disappears (or maybe
> degenerates into very small bitmap), and backward movement leaves
> strange artifacts on the screen.

I'm not able to reproduce this bug on Emacs 24 under Linux.  Do you
still see this problem?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#6026: 23.1.96; Cursor drawing problems with the Khmer script

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Cc: 6026@debbugs.gnu.org
> Date: Sun, 18 Sep 2011 23:02:35 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Type "C-h H" in "emacs -Q", then go to the line of the Khmer greeting
> > and move cursor.  Both forward and backward movement has problems:
> > with forward movement, the cursor sometimes disappears (or maybe
> > degenerates into very small bitmap), and backward movement leaves
> > strange artifacts on the screen.
> 
> I'm not able to reproduce this bug on Emacs 24 under Linux.  Do you
> still see this problem?

Yes.

It might be specific to the font, though.

bug#6026: 23.1.96; Cursor drawing problems with the Khmer script

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
>> Cc: 6026@debbugs.gnu.org
>> Date: Sun, 18 Sep 2011 23:02:35 +0200
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > Type "C-h H" in "emacs -Q", then go to the line of the Khmer greeting
>> > and move cursor.  Both forward and backward movement has problems:
>> > with forward movement, the cursor sometimes disappears (or maybe
>> > degenerates into very small bitmap), and backward movement leaves
>> > strange artifacts on the screen.
>> 
>> I'm not able to reproduce this bug on Emacs 24 under Linux.  Do you
>> still see this problem?
>
> Yes.
>
> It might be specific to the font, though.

That was 8 years ago.

I'm unable to reproduce this.  Are you still seeing this?

Best regards,
Stefan Kangas

bug#6026: 23.1.96; Cursor drawing problems with the Khmer script

[Eli Zaretskii]

> From: Stefan Kangas <stefan@marxist.se>
> Cc: Lars Magne Ingebrigtsen <larsi@gnus.org>,  6026@debbugs.gnu.org
> Date: Sat, 18 Jan 2020 02:21:43 +0100
> 
> >> > Type "C-h H" in "emacs -Q", then go to the line of the Khmer greeting
> >> > and move cursor.  Both forward and backward movement has problems:
> >> > with forward movement, the cursor sometimes disappears (or maybe
> >> > degenerates into very small bitmap), and backward movement leaves
> >> > strange artifacts on the screen.
> >> 
> >> I'm not able to reproduce this bug on Emacs 24 under Linux.  Do you
> >> still see this problem?
> >
> > Yes.
> >
> > It might be specific to the font, though.
> 
> That was 8 years ago.
> 
> I'm unable to reproduce this.  Are you still seeing this?

It is still present in Emacs 26.3 with the Uniscribe font backend, but
not in Emacs 27 with HarfBuzz.  So I'm closing the bug, as HarfBuzz is
our default font backend.

Thanks.

bug#6026: 23.1.96; Cursor drawing problems with the Khmer script

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

> It is still present in Emacs 26.3 with the Uniscribe font backend, but
> not in Emacs 27 with HarfBuzz.  So I'm closing the bug, as HarfBuzz is
> our default font backend.

I suspected that was the case, thanks for clearing it up.

Best regards,
Stefan Kangas