bug#5080: indic text is not displayed correctly in emacs shell

bug#5080: indic text is not displayed correctly in emacs shell

[Praveen A]

[-- Attachment #1: Type: text/plain, Size: 587 bytes --]

Hi,

Indic text is not rendered correctly in emacs shell (M-x shell).
Attached images show indic text displayed as escaped unicode values.
output of ls (when file names has indic characters) and cat (when
content of a file has indic characters) is displayed like that.

- Praveen

-- 
പ്രവീണ്‍ അരിമ്പ്രത്തൊടിയില്‍
<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)
Join The DRM Elimination Crew Now!
http://fci.wikia.com/wiki/Anti-DRM-Campaign

[-- Attachment #2: emacs-shell-indic-display-bug.png --]
[-- Type: image/png, Size: 99548 bytes --]

[-- Attachment #3: emacs-shell-rendering.png --]
[-- Type: image/png, Size: 69868 bytes --]

bug#5080: indic text is not displayed correctly in emacs shell

[Kenichi Handa]

In article <3f2beab60911300543l2436ff3ar8e5b5124db9db218@mail.gmail.com>, Praveen A <pravi.a@gmail.com> writes:

> Indic text is not rendered correctly in emacs shell (M-x shell).
> Attached images show indic text displayed as escaped unicode values.
> output of ls (when file names has indic characters) and cat (when
> content of a file has indic characters) is displayed like that.

It seems that the coding system for your shell buffer is
iso-8859-1.  In which locale did you invoke Emacs?  And
please try to change the coding system for the shell buffer
to utf-8 (C-x C-m p utf-8 RET utf-8 RET) and run ls and cat
again.

---
Kenichi Handa
handa@m17n.org

bug#5080: indic text is not displayed correctly in emacs shell

[Parag Nemade]

Hi Kenichi,
On Tuesday 01 December 2009 07:06 AM, Kenichi Handa wrote:
> In article<3f2beab60911300543l2436ff3ar8e5b5124db9db218@mail.gmail.com>, Praveen A<pravi.a@gmail.com>  writes:
>
>    
>> Indic text is not rendered correctly in emacs shell (M-x shell).
>> Attached images show indic text displayed as escaped unicode values.
>> output of ls (when file names has indic characters) and cat (when
>> content of a file has indic characters) is displayed like that.
>>      
> It seems that the coding system for your shell buffer is
> iso-8859-1.  In which locale did you invoke Emacs?  And
> please try to change the coding system for the shell buffer
> to utf-8 (C-x C-m p utf-8 RET utf-8 RET) and run ls and cat
> again.
>
>    
   I start emacs in English locale and then shell buffer and write some 
Indic text there and found rendering is broken but when same text is 
copied to gedit its rendering correctly.
   I did what you asked and found still rendering broken with utf-8. I 
even tried to run emacs in Marathi locale and see if it renders 
correctly but still when I did cat marathi.txt, the contents printed in 
shell have wrong rendering.
   I am using Fedora 12 emacs-23.1-10.fc12 rpm.

Please ask for any input needed.

Thanks & Regards,
Parag.

bug#5080: indic text is not displayed correctly in emacs shell

[Kenichi Handa]

In article <4B149E53.7060106@redhat.com>, Parag Nemade <pnemade@redhat.com> writes:

> > It seems that the coding system for your shell buffer is
> > iso-8859-1.  In which locale did you invoke Emacs?  And
> > please try to change the coding system for the shell buffer
> > to utf-8 (C-x C-m p utf-8 RET utf-8 RET) and run ls and cat
> > again.
> >
> >    
>    I start emacs in English locale and then shell buffer and write some 
> Indic text there and found rendering is broken but when same text is 
> copied to gedit its rendering correctly.
>    I did what you asked and found still rendering broken with utf-8. I 
> even tried to run emacs in Marathi locale and see if it renders 
> correctly but still when I did cat marathi.txt, the contents printed in 
> shell have wrong rendering.
>    I am using Fedora 12 emacs-23.1-10.fc12 rpm.

In this case, which do you mean by "wrong rendering"?

(1) As Praveen originally reported, Latin characters and
some binary characters (\2xx) are shown instead of Indic
(devanagari?, malayalam?) charactes.

(2) Indic characters are shown but with wrong rendering.

---
Kenichi Handa
handa@m17n.org

bug#5080: indic text is not displayed correctly in emacs shell

[Parag Nemade]

On Tuesday 01 December 2009 10:55 AM, Kenichi Handa wrote:
> In article<4B149E53.7060106@redhat.com>, Parag Nemade<pnemade@redhat.com>  writes:
>
>    
>>> It seems that the coding system for your shell buffer is
>>> iso-8859-1.  In which locale did you invoke Emacs?  And
>>> please try to change the coding system for the shell buffer
>>> to utf-8 (C-x C-m p utf-8 RET utf-8 RET) and run ls and cat
>>> again.
>>>
>>>
>>>        
>>     I start emacs in English locale and then shell buffer and write some
>> Indic text there and found rendering is broken but when same text is
>> copied to gedit its rendering correctly.
>>     I did what you asked and found still rendering broken with utf-8. I
>> even tried to run emacs in Marathi locale and see if it renders
>> correctly but still when I did cat marathi.txt, the contents printed in
>> shell have wrong rendering.
>>     I am using Fedora 12 emacs-23.1-10.fc12 rpm.
>>      
> In this case, which do you mean by "wrong rendering"?
>
> (1) As Praveen originally reported, Latin characters and
> some binary characters (\2xx) are shown instead of Indic
> (devanagari?, malayalam?) charactes.
>    
That is why I sent you my problem separately and I am not sure how its 
related to what Praveen has reported. Let Praveen answer your query first.

> (2) Indic characters are shown but with wrong rendering.
>    
      For me conjuncts are rendered as separately which is wrong which 
can be seen in screenshot I sent you already with comparison to gedit 
rendering.

Regards,
Parag.

bug#5080: indic text is not displayed correctly in emacs shell

[Praveen A]

2009/12/1 Parag Nemade <pnemade@redhat.com>:
>     For me conjuncts are rendered as separately which is wrong which can be
> seen in screenshot I sent you already with comparison to gedit rendering.

Parag,

That is a separate bug related to build options in Fedora.
https://bugzilla.redhat.com/show_bug.cgi?id=542657

- Praveen
-- 
പ്രവീണ്‍ അരിമ്പ്രത്തൊടിയില്‍
<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)
Join The DRM Elimination Crew Now!
http://fci.wikia.com/wiki/Anti-DRM-Campaign

bug#5080: indic text is not displayed correctly in emacs shell

[Parag Nemade]

On 12/1/2009 3:02 PM, Praveen A wrote:
> 2009/12/1 Parag Nemade<pnemade@redhat.com>:
>    
>>      For me conjuncts are rendered as separately which is wrong which can be
>> seen in screenshot I sent you already with comparison to gedit rendering.
>>      
> Parag,
>
> That is a separate bug related to build options in Fedora.
> https://bugzilla.redhat.com/show_bug.cgi?id=542657
>
> - Praveen
>    
Thanks. I am trying to say same thing to Kenichi but seems he 
misunderstood that we both have same problem.

Regards,
Parag.

bug#5080: indic text is not displayed correctly in emacs shell

[Kenichi Handa]

In article <4B14FB8B.1040801@redhat.com>, Parag Nemade <pnemade@redhat.com> writes:

> Thanks. I am trying to say same thing to Kenichi but seems he 
> misunderstood that we both have same problem.

Yes, sorry.  As I got the similar reports almost at the same
time, I was confused.  As both reporters are reading this
mail, I'll summarize the problems.

(1) ZWJ is not handled correctly for some Indic scripts.
(2) Indic text is not rendered correctly in *shell* buffer.
(3) Emacs included in Fedora 12 doesn't render Indic correctly.
(4) Emacs doesn't render Indic correctly with some specific fonts.

Do I miss something?

(1) was fixed by the trial code I sent.  So, I've just
applied that change in lisp/language/indian.el.  Please try
again with the latest CVS code.

(2) With the fix of (1) and the correct setting of process
coding system, the output of 'cat' is ok but still 'ls' is
bad.  I think this is because your ls is aliased to
something "ls --color".  Please try this:

% type ls
% unalias ls
% ls

It seems that when 'ls' is invoked with --color, it output
some escape sequences (e.g. ESC [ 0) that are not handled by
shell-mode.

(3) It seems that m17n-db is not fully installed on Fedora 12.
Please install it.  Parhaps the package name is:
"Multilingualization datafiles for m17n-lib"

(4) I updated my ttf-freefont package.  FreeMono and
FreeSans doesn't contain Malayalam glyphs.  FreeSerif does
have the glyphs and mlym OpenType table and thus can render
Malayalam correctly with Emacs.  Praveen wrote that gedit
can render Malayalam with FreeMono font, but that's
unbelievable.  I suspect gedit is using some other fallback
font.  I don't know how to check exactly which font gedit is
using for each Indic characters.

By the way, to tell Emacs to use a specific font for, for
instance, Malayalam, try this:

(set-fontset-font t 'malayalam '("FONT_FAMILY_NAME" . "unicode-bmp"))

And to see which font is used for an Indic character, put
cursor on it, and type C-u C-x = RET.

---
Kenichi Handa
handa@m17n.org

bug#5080: indic text is not displayed correctly in emacs shell

[Parag Nemade]

Hi,

On Thursday 03 December 2009 06:33 AM, Kenichi Handa wrote:
> In article<4B14FB8B.1040801@redhat.com>, Parag Nemade<pnemade@redhat.com>  writes:
>
>    
>> Thanks. I am trying to say same thing to Kenichi but seems he
>> misunderstood that we both have same problem.
>>      
> Yes, sorry.  As I got the similar reports almost at the same
> time, I was confused.  As both reporters are reading this
> mail, I'll summarize the problems.
>
> (1) ZWJ is not handled correctly for some Indic scripts.
> (2) Indic text is not rendered correctly in *shell* buffer.
> (3) Emacs included in Fedora 12 doesn't render Indic correctly.
> (4) Emacs doesn't render Indic correctly with some specific fonts.
>
> Do I miss something?
>
> (1) was fixed by the trial code I sent.  So, I've just
> applied that change in lisp/language/indian.el.  Please try
> again with the latest CVS code.
>
> (2) With the fix of (1) and the correct setting of process
> coding system, the output of 'cat' is ok but still 'ls' is
> bad.  I think this is because your ls is aliased to
> something "ls --color".  Please try this:
>
> % type ls
> % unalias ls
> % ls
>
> It seems that when 'ls' is invoked with --color, it output
> some escape sequences (e.g. ESC [ 0) that are not handled by
> shell-mode.
>
> (3) It seems that m17n-db is not fully installed on Fedora 12.
> Please install it.  Parhaps the package name is:
> "Multilingualization datafiles for m17n-lib"
>    
Thank you so much. I got this working after I installed 
m17n-db-datafiles rpm.
> (4) I updated my ttf-freefont package.  FreeMono and
> FreeSans doesn't contain Malayalam glyphs.  FreeSerif does
> have the glyphs and mlym OpenType table and thus can render
> Malayalam correctly with Emacs.  Praveen wrote that gedit
> can render Malayalam with FreeMono font, but that's
> unbelievable.  I suspect gedit is using some other fallback
> font.  I don't know how to check exactly which font gedit is
> using for each Indic characters.
>
> By the way, to tell Emacs to use a specific font for, for
> instance, Malayalam, try this:
>
> (set-fontset-font t 'malayalam '("FONT_FAMILY_NAME" . "unicode-bmp"))
>
> And to see which font is used for an Indic character, put
> cursor on it, and type C-u C-x = RET.
>
>    
Regards,
Parag.

bug#5080: indic text is not displayed correctly in emacs shell

[Kenichi Handa]

In article <3f2beab60912030046nc76ba08x39e438f4f23366cb@mail.gmail.com>, Praveen A <pravi.a@gmail.com> writes:
> > (1) was fixed by the trial code I sent.  So, I've just
> > applied that change in lisp/language/indian.el.  Please try
> > again with the latest CVS code.

> Thanks. I will request fedora and debian emacs maintainers to include this.

That's good, thank you.

> > (3) It seems that m17n-db is not fully installed on Fedora 12.
> > Please install it.  Parhaps the package name is:
> > "Multilingualization datafiles for m17n-lib"

> yes. Parag has requested a dependency be added to emacs package.

Good too, thank you.

> > (4) I updated my ttf-freefont package.  FreeMono and
> > FreeSans doesn't contain Malayalam glyphs.  FreeSerif does
> > have the glyphs and mlym OpenType table and thus can render
> > Malayalam correctly with Emacs.  Praveen wrote that gedit
> > can render Malayalam with FreeMono font, but that's
> > unbelievable.  I suspect gedit is using some other fallback
> > font.  I don't know how to check exactly which font gedit is
> > using for each Indic characters.

> yes, FreeMono does not have Malayalam glyphs, It would be a fallback
> font. (Dejavu Sans Mono/FreeMono fallback to Lohit)

> > By the way, to tell Emacs to use a specific font for, for
> > instance, Malayalam, try this:
> >
> > (set-fontset-font t 'malayalam '("FONT_FAMILY_NAME" . "unicode-bmp"))

> It is not working for me. I tried it as is and substituting
> unicode-bmp with Rachana. Both cases it went to Lohit. Is it taking
> this preference from fontconfig?

No, no, you must specify "Rachana" (if it is the font family
name) in place of "FONT_FAMILY_NAME" as this:

(set-fontset-font t 'malayalam '("rachana" . "unicode-bmp"))

I installed rachana font and confirmed that the above code
worked.

---
Kenichi Handa
handa@m17n.org

bug#5080: indic text is not displayed correctly in emacs shell

[Praveen A]

2009/12/3 Kenichi Handa <handa@m17n.org>:
> In article <3f2beab60912030046nc76ba08x39e438f4f23366cb@mail.gmail.com>, Praveen A <pravi.a@gmail.com> writes:
>> Thanks. I will request fedora and debian emacs maintainers to include this.
>
> That's good, thank you.

The request is filed.

> No, no, you must specify "Rachana" (if it is the font family
> name) in place of "FONT_FAMILY_NAME" as this:
>
> (set-fontset-font t 'malayalam '("rachana" . "unicode-bmp"))
>
> I installed rachana font and confirmed that the above code
> worked.

Thanks. It works for me now. But how to make it default? Where does
emacs take this information from ?

I have my fontconfig preferences to prefer Meera, but emacs does not
respect that and takes Lohit Malayalam instead.

So from emacs point of view issue (3) is resolved. I have filed
another bug for issue (2). Issue (4) still needs attention.

(4a) Why is it taking Lohit Malayalam by default
(4b) Why is the display cluttered/overlapped with some fonts like Meera

(1a) now with your patch, emacs handles ZWJ
(1b) There is a problem with Lohit Malayalam, I had a discussion with
Lohit developer (Pravin Satpute) and he says it is an issue with
m17n/redering engine.

Meera and Rachana uses substitution for complex glyphs, but Lohit uses
some fatures of OpenType specification and which needs to be
implemented in rendering engine used by emacs.

Thanks
Praveen
-- 
പ്രവീണ്‍ അരിമ്പ്രത്തൊടിയില്‍
<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)
Join The DRM Elimination Crew Now!
http://fci.wikia.com/wiki/Anti-DRM-Campaign

bug#5080: indic text is not displayed correctly in emacs shell

[Pravin Satpute]

On 12/03/2009 05:37 PM, Praveen A wrote:
> 2009/12/3 Kenichi Handa <handa@m17n.org>:
>   
>> In article <3f2beab60912030046nc76ba08x39e438f4f23366cb@mail.gmail.com>, Praveen A <pravi.a@gmail.com> writes:
>>     
>>> Thanks. I will request fedora and debian emacs maintainers to include this.
>>>       
>> That's good, thank you.
>>     
> The request is filed.
>
>   
>> No, no, you must specify "Rachana" (if it is the font family
>> name) in place of "FONT_FAMILY_NAME" as this:
>>
>> (set-fontset-font t 'malayalam '("rachana" . "unicode-bmp"))
>>
>> I installed rachana font and confirmed that the above code
>> worked.
>>     
> Thanks. It works for me now. But how to make it default? Where does
> emacs take this information from ?
>
> I have my fontconfig preferences to prefer Meera, but emacs does not
> respect that and takes Lohit Malayalam instead.
>
> So from emacs point of view issue (3) is resolved. I have filed
> another bug for issue (2). Issue (4) still needs attention.
>
> (4a) Why is it taking Lohit Malayalam by default
> (4b) Why is the display cluttered/overlapped with some fonts like Meera
>
> (1a) now with your patch, emacs handles ZWJ
> (1b) There is a problem with Lohit Malayalam, I had a discussion with
> Lohit developer (Pravin Satpute) and he says it is an issue with
> m17n/redering engine.
>   

Yeah, Speaking about lohit-malayalam
since it work well with all other rendering engine like, pango, qt and ICU.

I think there is some problem processing its open type feature by
present m17n rendering engine.
Is it latest one?
I think its work well with Meera font since, Meera font has lots of
ligature defined, and most of the shape get form it very early stage of
open type feature processing by rendering engine.

though i have not tested much i think this is the case. I will do some
more testing with other fonts may be on monday and will let you know
exact case.

Regards
-
Pravin S

bug#5080: indic text is not displayed correctly in emacs shell

[Kenichi Handa]

In article <3f2beab60912030407o5609514bidc65538848064e4@mail.gmail.com>, Praveen A <pravi.a@gmail.com> writes:

> > (set-fontset-font t 'malayalam '("rachana" . "unicode-bmp"))
> >
> > I installed rachana font and confirmed that the above code
> > worked.

> Thanks. It works for me now. But how to make it default?

Please put that in your .emacs.

> I have my fontconfig preferences to prefer Meera, but emacs does not
> respect that and takes Lohit Malayalam instead.

Currently Emacs just uses the first font found to support
Malayalam.  What is returned when you do this?
% fc-list :lang=ml

For fontconfig, we should modify the function ftfont_list in
(src/ftfont.c) to respect user configuration, but currently
I don't have a time to do that.

> (4a) Why is it taking Lohit Malayalam by default
> (4b) Why is the display cluttered/overlapped with some fonts like Meera

> (1a) now with your patch, emacs handles ZWJ
> (1b) There is a problem with Lohit Malayalam, I had a discussion with
> Lohit developer (Pravin Satpute) and he says it is an issue with
> m17n/redering engine.

> Meera and Rachana uses substitution for complex glyphs, but Lohit uses
> some fatures of OpenType specification and which needs to be
> implemented in rendering engine used by emacs.

For a rendering problem with a specific font, please send a
set of the following info separately for each problem.  You
may have already sent some of them in the other mails, but,
as I'm not a malayalam expert, I can't figure out easily
which part is actually bad and how to reproduce the problem.

(1) font file name (if possible, with the version number)
(2) problematic character sequence
(3) image of Emacs' rendering of that sequence
(4) image of the correct rendering with the same font

---
Kenichi Handa
handa@m17n.org

bug#5080: indic text is not displayed correctly in emacs shell

[Praveen A]

[-- Attachment #1: Type: text/plain, Size: 3040 bytes --]

2009/12/4 Kenichi Handa <handa@m17n.org>:
> In article <3f2beab60912030407o5609514bidc65538848064e4@mail.gmail.com>, Praveen A <pravi.a@gmail.com> writes:
>
>> > (set-fontset-font t 'malayalam '("rachana" . "unicode-bmp"))
>> >
>> > I installed rachana font and confirmed that the above code
>> > worked.
>
>> Thanks. It works for me now. But how to make it default?
>
> Please put that in your .emacs.

Yes, I did that. I was thinking more on the lines of taking default
from system/fontconfig configuration.

>
>> I have my fontconfig preferences to prefer Meera, but emacs does not
>> respect that and takes Lohit Malayalam instead.
>
> Currently Emacs just uses the first font found to support
> Malayalam.  What is returned when you do this?
> % fc-list :lang=ml
>
pravi@savannah:~$ fc-list :lang=ml
Meera:style=Regular
FreeSerif:style=Mittel,µεσαία,Medium,Normal,Gemiddeld,navadno
Rachana:style=Regular
suruma:style=Medium
AnjaliOldLipi:style=Regular
Lohit Malayalam:style=Regular
RaghuMalayalam:style=Sans

pravi@savannah:~$ fc-match :lang=ml
Meera_04.ttf: "Meera" "Regular"
pravi@savannah:~$


I think fc-match would be a better option, but even the the preference
in fc-list is not respected in this case.

> For fontconfig, we should modify the function ftfont_list in
> (src/ftfont.c) to respect user configuration, but currently
> I don't have a time to do that.
>
>> (4a) Why is it taking Lohit Malayalam by default
>> (4b) Why is the display cluttered/overlapped with some fonts like Meera
>
>> (1a) now with your patch, emacs handles ZWJ
>> (1b) There is a problem with Lohit Malayalam, I had a discussion with
>> Lohit developer (Pravin Satpute) and he says it is an issue with
>> m17n/redering engine.
>
>> Meera and Rachana uses substitution for complex glyphs, but Lohit uses
>> some fatures of OpenType specification and which needs to be
>> implemented in rendering engine used by emacs.
>
> For a rendering problem with a specific font, please send a
> set of the following info separately for each problem.  You
> may have already sent some of them in the other mails, but,
> as I'm not a malayalam expert, I can't figure out easily
> which part is actually bad and how to reproduce the problem.
>
> (1) font file name (if possible, with the version number)

lohit_ml.ttf (ttf-malayalam-fonts 1:0.5.6)
http://ftp.de.debian.org/debian/pool/main/t/ttf-indic-fonts/ttf-indic-fonts_0.5.6.tar.gz

> (2) problematic character sequence

0x0D23 0x0D4D 0x200D

(ണ ് ‍)

> (3) image of Emacs' rendering of that sequence

emacs-lohit-zwj.png

> (4) image of the correct rendering with the same font

gedit-lohit-zwj


Thanks
Praveen
-- 
പ്രവീണ്‍ അരിമ്പ്രത്തൊടിയില്‍
<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)
Join The DRM Elimination Crew Now!
http://fci.wikia.com/wiki/Anti-DRM-Campaign

[-- Attachment #2: emacs-lohit-zwj.png --]
[-- Type: image/png, Size: 21050 bytes --]

[-- Attachment #3: gedit-lohit-zwj.png --]
[-- Type: image/png, Size: 36596 bytes --]

bug#5080: indic text is not displayed correctly in emacs shell

[Kenichi Handa]

In article <3f2beab60912040049we5870bbk9991a3eb37111b21@mail.gmail.com>, Praveen A <pravi.a@gmail.com> writes:

> (1) font file name (if possible, with the version number)
> 
> lohit_ml.ttf (ttf-malayalam-fonts 1:0.5.6)
> http://ftp.de.debian.org/debian/pool/main/t/ttf-indic-fonts/ttf-indic-fonts_0.5.6.tar.gz
> 
> (2) problematic character sequence
> 
> 0x0D23 0x0D4D 0x200D
> 
> (ണ ് ‍)

Thank you for the data.  I downloaded lohit_ml.ttf and
confirmed the bug.  I'll check what is wrong.

By the way, you also wrote: 

> (4b) Why is the display cluttered/overlapped with some fonts like Meera

Please give me the data for that bug.

---
Kenichi Handa
handa@m17n.org

bug#5080: indic text is not displayed correctly in emacs shell

[Kenichi Handa]

In article <tl7zl5zaqdf.fsf@m17n.org>, Kenichi Handa <handa@m17n.org> writes:

> In article <3f2beab60912040049we5870bbk9991a3eb37111b21@mail.gmail.com>, Praveen A <pravi.a@gmail.com> writes:
> > (1) font file name (if possible, with the version number)
> > 
> > lohit_ml.ttf (ttf-malayalam-fonts 1:0.5.6)
> > http://ftp.de.debian.org/debian/pool/main/t/ttf-indic-fonts/ttf-indic-fonts_0.5.6.tar.gz
> > 
> > (2) problematic character sequence
> > 
> > 0x0D23 0x0D4D 0x200D
> > 
> > (ണ ് ‍)

> Thank you for the data.  I downloaded lohit_ml.ttf and
> confirmed the bug.  I'll check what is wrong.

Please put the attached file (MLYM-OTF.flt) under ~/.m17n.d/
and try again.  If it works, move that file to
/usr/share/m17n.

---
Kenichi Handa
handa@m17n.org

--- MLYM-OTF.flt ---
;; MLYM-OTF.flt -- Font Layout Table for Malayalam OpenType font
;; Copyright (C) 2003, 2004, 2005, 2007, 2008, 2009  AIST (H15PRO112)
;; See the end for copying conditions.

(font layouter mlym-otf nil
      (font (nil nil unicode-bmp :otf=mlym=akhn)))

;;; <li> MLYM-OTF.flt
;;;
;;; For Malayalam OpenType fonts to draw the reformed Malayalam script.

;; Tested with the following fonts.  All of these fonts require Halant
;; movement for syllable-final YA and VA, but not for RA and LA.

;; AnjaliOldLipi.ttf
;; Dyuthi3.ttf
;; Kalyani121.ttf
;; Meera_04.ttf
;; Rachana_04.ttf
;; RaghuMalayalamSans2.ttf
;; suruma2.ttf

(category
 ;; C: ordinary consonants (except P)
 ;; P: consonants that have a post form
 ;; H: HALANT
 ;; m: vowel sign (pre)
 ;; b: vowel sign (below)
 ;; p: vowel sign (post)
 ;; A: vowel modifier (post)
 ;; V: independent vowel
 ;; N: ZWNJ (ZERO WIDTH NON-JOINER)
 ;; J: ZWJ (ZERO WIDTH JOINER)
 ;; Z: internal use
 ;; X: generic
 (0x0D00 0x0D7F	?X)			; generic
 (0x0D00        ?Z)			; internal use
 (0x0D02 0x0D03	?A)			; SIGN ANUSVARA .. VISARGA
 (0x0D05 0x0D14	?V)			; LETTER A .. AU
 (0x0D15 0x0D39 ?C)			; LETTER KA .. HA
 (0x0D2F	?P)			; LETTER YA
 (0x0D35	?P)			; LETTER VA
 (0x0D3E 0x0D44	?p)			; VOWEL SIGN AA .. VOCALIC RR
 (0x0D46 0x0D48	?m)			; VOWEL SIGN E .. AI
 (0x0D4D	?H)			; SIGN VIRAMA
 (0x0D57	?p)			; AU LENGTH MARK
 (0x0D60 0x0D61	?V)			; LETTER VOCALIC RR .. VOCALIC LL
 (0x0D62 0x0D63 ?b)			; VOWEL SIGN VOCALIC L .. LL
 (0x0964 0x0965	?X)			; DANDA .. DOUBLE DANDA
 (0x200C	?N)			; ZWNJ
 (0x200D	?J)			; ZWJ
 )

;; Stage 0
;; Decompose two-part vowels.
(generator
 (0
  (cond
   ((0x0D4A)
    0x0D46 0x0D3E)
   ((0x0D4B)
    0x0D47 0x0D3E)
   ((0x0D4C)
    0x0D46 0x0D57)
   ("." =))
  *))

;; Stage 1
;; Syllable identification and reordering.  
;; Put the mark Z after the base consonant.
(generator
 (0
  (cond
   ;; A syllable with a left vowel sign.
   ;;1    2        3  4
   ("([CP](H[CP])*)(m)(p?A?)"
    < | (3 =) (1 move-h) (4 = *) | >)

   ;; A syllable with a right vowel sign, a below vowel sign
   ;; and/or a vowel modifier.
   ;;1    2        3
   ("([CP](H[CP])*)(bA?|pA?|A)"
    < | (1 move-h) (3 = *) | >)

   ;; Forced chillu form.
   ;;1    2        3
   ("([CP](H[CP])*)(HJ)"
    < | (1 move-h) (3 = =) | >)

   ;; A syllable with no vowel signs/modifiers, optionally followed by
   ;; an H or an HN.
   ;;1    2        3
   ("([CP](H[CP])*)(H|HN)?"
    < | (1 move-h) (3 0x0D4D) | >)

   ;; Starting with an independent vowel.
   ("VA?"
    < | (0 = *) | >)

   ("." =))
  *)


 ;; Move the halant that follows the base consonant to the end.
 (move-h
  (cond
   ;; All consonants are P's.
   ("(P)(H)([PH]*)$"
    (1 =) 0x0D00 (3 = *) (2 =))

   ;; One or more Non-P consonants.
   ("([CPH]*C)(H)([PH]+)$"
    (1 = *) 0x0D00 (3 = *) (2 =))

   ;; Otherwise, final consonant is not P.  No need to move H.
   (".+"
    = * 0x0D00)))
 )

;; Stage 2
;; Apply GSUB features.
(generator
 (0
  (cond
   ;; The tested fonts make LA's below form with 1. 'half' + 'blws',
   ;; 2. 'half' + 'pres' or 3. 'akhn'.
   ;; 1   2       3  4
   (" (m)?([CPH]*)(Z)([PHbpAJ]*) "
    | (1 =) (2 otf:mlym=akhn,half,pstf,pres,abvs,blws+) (3 =)
    (4 otf:mlym=pstf+) |)

   ("." =))
  *))

;; Stage 3
;; Second reordering.  If there are glyphs before the base, move left
;; vowel sign to the left of the base.
(generator
 (0
  (cond
   ;; 1   2       3      4
   (" (m)?([^Z ]*)([^Z])Z([^ ]*) "
    | (2 = *) (1 =) (3 =) (4 = *) |)

   ("." =))
  *))

;; Stage 4
;; Apply remaining GSUB and GPOS features to all syllables.  To get
;; correct Y.YA and V.VA conjunctions, we must apply 'psts' to the
;; entire syllable.
(generator
 (0
  (cond
   (" ([^ ]+) "
    (1 otf:mlym=akhn,psts,haln+abvm,blwm,dist))
   ("."
    [ = ]))
  *))

;; Copyright (C) 2003, 2004, 2005, 2007, 2008, 2009
;;   National Institute of Advanced Industrial Science and Technology (AIST)
;;   Registration Number H15PRO112

;; This file is part of the m17n database; a sub-part of the m17n
;; library.

;; The m17n library is free software; you can redistribute it and/or
;; modify it under the terms of the GNU Lesser General Public License
;; as published by the Free Software Foundation; either version 2.1 of
;; the License, or (at your option) any later version.

;; The m17n library is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
;; Lesser General Public License for more details.

;; You should have received a copy of the GNU Lesser General Public
;; License along with the m17n library; if not, write to the Free
;; Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
;; Boston, MA 02110-1301, USA.

;; Local Variables:
;; mode: emacs-lisp
;; End:

bug#5080: indic text is not displayed correctly in emacs shell

[Praveen A]

2009/12/4 Kenichi Handa <handa@m17n.org>:
> In article <tl7zl5zaqdf.fsf@m17n.org>, Kenichi Handa <handa@m17n.org> writes:
> Please put the attached file (MLYM-OTF.flt) under ~/.m17n.d/
> and try again.  If it works, move that file to
> /usr/share/m17n.
>

I can confirm it is working after using this file. Thanks for the quick fixes.

- Praveen

-- 
പ്രവീണ്‍ അരിമ്പ്രത്തൊടിയില്‍
<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)
Join The DRM Elimination Crew Now!
http://fci.wikia.com/wiki/Anti-DRM-Campaign

bug#5080: indic text is not displayed correctly in emacs shell

[Kenichi Handa]

In article <3f2beab60912040510n9bc3257tdf310f0c5a10c69a@mail.gmail.com>, Praveen A <pravi.a@gmail.com> writes:

>>> (4b) Why is the display cluttered/overlapped with some fonts like Meera
> >
> > Please give me the data for that bug.

> See attachments. emacs+Rachana and gedit+Meera displays the text
> correctly, but with emacs+Meera some letters are completely gone (as
> in case 1 marked) or overlapped (as in case 2).

Please give me the actual character sequence too.

---
Kenichi Handa
handa@m17n.org

bug#5080: indic text is not displayed correctly in emacs shell

[Praveen A]

2009/12/7 Kenichi Handa <handa@m17n.org>:
> Please give me the actual character sequence too.
>
This sample text is taken from first paragraph of
http://ml.wikipedia.org/wiki/താരാപഥം
>>> ml=u'നക്ഷത്രങ്ങളും നക്ഷത്രാവശിഷ്ടങ്ങളും'
>>> print repr(ml)
u'\u0d28\u0d15\u0d4d\u0d37\u0d24\u0d4d\u0d30\u0d19\u0d4d\u0d19{\u0d33\u0d41\u0d02}
\u0d28\u0d15\u0d4d\u0d37\u0d24\u0d4d\u0d30\u0d3e\u0d35\u0d36\u0d3f{\u0d37\u0d4d\u0d1f}{\u0d19\u0d4d\u0d19}{\u0d33\u0d41\u0d02}'

problem sequence is marked inside {}

(1) \u0d33\u0d41\u0d02 before the first space is completely lost.
Actually the next character ie, \u0d28 takes the position of the
missing sequence.

First glyph of all words eclipses/overlaps with the last glyph of the
previous word. I think this is because 'space' character seems to be
rendered as having zero width.

(2) \u0d19\u0d4d\u0d19 is overlapped with the previous
(\u0d37\u0d4d\u0d1f) and next (\u0d33\u0d41\u0d02) glyphs in the word.

I think this two examples are representative of the issue, if you need
more example sequences with display problems, I can provide those.

Thanks
- Praveen
-- 
പ്രവീണ്‍ അരിമ്പ്രത്തൊടിയില്‍
<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)
Join The DRM Elimination Crew Now!
http://fci.wikia.com/wiki/Anti-DRM-Campaign

bug#5080: indic text is not displayed correctly in emacs shell

[Kenichi Handa]

[-- Attachment #1: Type: text/plain, Size: 589 bytes --]

In article <3f2beab60912070057n8fb9b2ap4eb8241a95df77c5@mail.gmail.com>, Praveen A <pravi.a@gmail.com> writes:

> This sample text is taken from first paragraph of
> http://ml.wikipedia.org/wiki/താരാപഥം
>>>> ml=u'നക്ഷത്രങ്ങളും നക്ഷത്രാവശിഷ്ടങ്ങളും'

Hmmm, strange, I can't reproduce it.  It seems that my Emacs
correctly renders that text.  See the attached image.
The library versions are the same as what you wrote before:
  libotf 0.9.10
  m17n-flt 1.5.4

---
Kenichi Handa
handa@m17n.org


[-- Attachment #2: Malayalam rendering test --]
[-- Type: image/png, Size: 60522 bytes --]

bug#5080: indic text is not displayed correctly in emacs shell

[Praveen A]

2009/12/7 Kenichi Handa <handa@m17n.org>:
> In article <3f2beab60912070057n8fb9b2ap4eb8241a95df77c5@mail.gmail.com>, Praveen A <pravi.a@gmail.com> writes:
>
>> This sample text is taken from first paragraph of
>> http://ml.wikipedia.org/wiki/താരാപഥം
>>>>> ml=u'നക്ഷത്രങ്ങളും നക്ഷത്രാവശിഷ്ടങ്ങളും'
>
> Hmmm, strange, I can't reproduce it.  It seems that my Emacs
> correctly renders that text.  See the attached image.
> The library versions are the same as what you wrote before:
>  libotf 0.9.10
>  m17n-flt 1.5.4
>

Can you try both words in one line?

Also can you copy + paste the following paragraph?

"നക്ഷത്രങ്ങളും നക്ഷത്രാവശിഷ്ടങ്ങളും നക്ഷത്രാന്തരീയമാദ്ധ്യമവും
തമോദ്രവ്യവും ചേര്‍ന്നുള്ള പിണ്ഡമേറിയതും ഗുരുത്വാകര്‍ഷണബന്ധിതവുമായ
വ്യൂഹമാണ് താരാപഥം അഥവാ ഗാലക്സി. ക്ഷീരപഥത്തെ
സൂചിപ്പിക്കാനുപയോഗിച്ചിരുന്ന പാലുപോലുള്ള എന്നര്‍ഥം വരുന്ന ഗാലക്സിയാസ്
എന്ന പദത്തില്‍ നിന്നാണ് ഗാലക്സി എന്ന ഇംഗ്ലീഷ് വാക്ക് ഉരുത്തിരിഞ്ഞത്.
ഒരു കോടിയോളം നക്ഷത്രങ്ങള്‍ അടങ്ങുന്ന കുള്ളന്‍ ഗാലക്സികള്‍ മുതല്‍ ഒരു
ലക്ഷം കോടി നക്ഷത്രങ്ങള്‍ അടങ്ങുന്ന അതിഭീമ ഗാലക്സികള്‍ വരെ
പ്രപഞ്ചത്തില്‍ ഉണ്ട്. താരാപഥത്തിലെ നക്ഷത്രങ്ങളെല്ലാം അതിന്റെ
പിണ്ഡകേന്ദ്രത്തെ ചുറ്റിസഞ്ചരിക്കുന്നു. സൂര്യനും അതിനെ കേന്ദ്രമാക്കി
സഞ്ചരിക്കുന്ന ഭൂമിയുള്‍പ്പെടെയുള്ള മറ്റ് വസ്തുക്കളും അടങ്ങിയ സൗരയൂഥം
ക്ഷീരപഥം എന്ന താരാപഥത്തിലാണ് സ്ഥിതിചെയ്യുന്നത്."

Thanks
Praveen
-- 
പ്രവീണ്‍ അരിമ്പ്രത്തൊടിയില്‍
<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)
Join The DRM Elimination Crew Now!
http://fci.wikia.com/wiki/Anti-DRM-Campaign

bug#5080: indic text is not displayed correctly in emacs shell

[Kenichi Handa]

In article <3f2beab60912070415l7f284f25ibf30019d5341b5e8@mail.gmail.com>, Praveen A <pravi.a@gmail.com> writes:

> 2009/12/7 Kenichi Handa <handa@m17n.org>:
> > In article <3f2beab60912070057n8fb9b2ap4eb8241a95df77c5@mail.gmail.com>, Praveen A <pravi.a@gmail.com> writes:
> >
>>> This sample text is taken from first paragraph of
>>> http://ml.wikipedia.org/wiki/താരാപഥം
>>>>>> ml=u'നക്ഷത്രങ്ങളും നക്ഷത്രാവശിഷ്ടങ്ങളും'
> >
> > Hmmm, strange, I can't reproduce it.  It seems that my Emacs
> > correctly renders that text.  See the attached image.
> > The library versions are the same as what you wrote before:
> >  libotf 0.9.10
> >  m17n-flt 1.5.4

> Can you try both words in one line?

> Also can you copy + paste the following paragraph?

> "നക്ഷത്രങ്ങളും നക്ഷത്രാവശിഷ്ടങ്ങളും നക്ഷത്രാന്തരീയമാദ്ധ്യമവും
[...]

I tried but still Emacs rendered it correctly.  Could you
try this?

(1) make a file "temp.txt" that contains only this line:
നക്ഷത്രങ്ങളും നക്ഷത്രാവശിഷ്ടങ്ങളും
(2) start Emacs as this:
   % MDEBUG_FLT=3 emacs >& log
(3) Evalute these two lines in Emacs:
(set-fontset-font t 'malayalam '("meera" . "unicode-bmp"))
(insert-file "temp.txt")
(4) exit emacs

then send me the created file "log".

---
Kenichi Handa
handa@m17n.org

bug#5080: indic text is not displayed correctly in emacs shell

[Praveen A]

[-- Attachment #1: Type: text/plain, Size: 937 bytes --]

2009/12/7 Kenichi Handa <handa@m17n.org>:
>
> I tried but still Emacs rendered it correctly.  Could you
> try this?
>
> (1) make a file "temp.txt" that contains only this line:
> നക്ഷത്രങ്ങളും നക്ഷത്രാവശിഷ്ടങ്ങളും

done
> (2) start Emacs as this:
>   % MDEBUG_FLT=3 emacs >& log

> (3) Evalute these two lines in Emacs:
> (set-fontset-font t 'malayalam '("meera" . "unicode-bmp"))

I added the above line to ~/.emacs

started emacs as in (2)

> (insert-file "temp.txt")

C-x C-f ~/temp.txt
> (4) exit emacs

C-x C-c
> then send me the created file "log".

log file is attached.

-- 
പ്രവീണ്‍ അരിമ്പ്രത്തൊടിയില്‍
<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)
Join The DRM Elimination Crew Now!
http://fci.wikia.com/wiki/Anti-DRM-Campaign

[-- Attachment #2: log --]
[-- Type: application/octet-stream, Size: 19001 bytes --]

bug#5080: indic text is not displayed correctly in emacs shell

[Kenichi Handa]

In article <3f2beab60912070503o48b666cfkd876730b6f643c81@mail.gmail.com>, Praveen A <pravi.a@gmail.com> writes:

> log file is attached.

Thank you.  Your log file is the same as mine.  So the
backend libraries has no problem.  Then the problem is in
Emacs itself.  What I'm now suspecting is the cache of
glyph-string.

When you did this in the fresh Emacs, was the rendering
still corrupted?

> C-x C-f ~/temp.txt

---
Kenichi Handa
handa@m17n.org

bug#5080: indic text is not displayed correctly in emacs shell

[Praveen A]

2009/12/8 Kenichi Handa <handa@m17n.org>:
> In article <3f2beab60912070503o48b666cfkd876730b6f643c81@mail.gmail.com>, Praveen A <pravi.a@gmail.com> writes:
>
>> log file is attached.
>
> Thank you.  Your log file is the same as mine.  So the
> backend libraries has no problem.  Then the problem is in
> Emacs itself.  What I'm now suspecting is the cache of
> glyph-string.
>
> When you did this in the fresh Emacs, was the rendering
> still corrupted?
>
>> C-x C-f ~/temp.txt

yes, it is still corrupted.


-- 
പ്രവീണ്‍ അരിമ്പ്രത്തൊടിയില്‍
<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)
Join The DRM Elimination Crew Now!
http://fci.wikia.com/wiki/Anti-DRM-Campaign

bug#5080: indic text is not displayed correctly in emacs shell

[Praveen A]

2009/12/7 Kenichi Handa <handa@m17n.org>:
> Hmmm, strange, I can't reproduce it.  It seems that my Emacs
> correctly renders that text.  See the attached image.
> The library versions are the same as what you wrote before:
>  libotf 0.9.10
>  m17n-flt 1.5.4

I captured the session in a video.
See http://blip.tv/file/2941317
Direct link: http://blip.tv/file/get/Pravi-emacsRenderingMalayalamTextWithMeeraFont914.ogv

- Praveen
-- 
പ്രവീണ്‍ അരിമ്പ്രത്തൊടിയില്‍
<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)
Join The DRM Elimination Crew Now!
http://fci.wikia.com/wiki/Anti-DRM-Campaign

bug#5080: indic text is not displayed correctly in emacs shell

[Praveen A]

2009/12/3 Praveen A <pravi.a@gmail.com>:
> (4a) Why is it taking Lohit Malayalam by default
> (4b) Why is the display cluttered/overlapped with some fonts like Meera

4b is now resolved, details in
http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=5146

Now only remaining issue in the list is about selecting default font.
I think it should be taken from fc-match :lang=ml

Even as per the current policy of taking first one from fc-list
:lang=ml, it is not working

Though Meera is selected as the default font (fc-match gives this and
it comes as the first result in fc-list) in fontconfig configuration,
Lohit Malayalam is used as the default font to display Malyalam if no
preference is set in .emacs.

-- 
പ്രവീണ്‍ അരിമ്പ്രത്തൊടിയില്‍
<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)
Join The DRM Elimination Crew Now!
http://fci.wikia.com/wiki/Anti-DRM-Campaign

bug#5080: indic text is not displayed correctly in emacs shell

[Kenichi Handa]

In article <3f2beab60912120741n7651670as1c532d3865efbc23@mail.gmail.com>, Praveen A <pravi.a@gmail.com> writes:

> 2009/12/3 Praveen A <pravi.a@gmail.com>:
> > (4a) Why is it taking Lohit Malayalam by default
> > (4b) Why is the display cluttered/overlapped with some fonts like Meera

> 4b is now resolved, details in
> http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=5146

Thank you for confirming that. 

> Now only remaining issue in the list is about selecting default font.
> I think it should be taken from fc-match :lang=ml

We can't simply use fc-match (i.e. FcFontMatch ()) because
it returns a font that doesn't match with :lang=ml if
there's no such font.  Perhaps we can use FcFontSetSort()
function on the result of FcFontList().

> Even as per the current policy of taking first one from fc-list
> :lang=ml, it is not working

> Though Meera is selected as the default font (fc-match gives this and
> it comes as the first result in fc-list) in fontconfig configuration,
> Lohit Malayalam is used as the default font to display Malyalam if no
> preference is set in .emacs.

I found that this is because qsort doesn't preserve the
original order if scores of two elements are equal.

Please wait again for a while.  I'm now working on it.

---
Kenichi Handa
handa@m17n.org

bug#5080: indic text is not displayed correctly in emacs shell

[Lars Magne Ingebrigtsen]

Praveen A <pravi.a@gmail.com> writes:

>> (4a) Why is it taking Lohit Malayalam by default
>> (4b) Why is the display cluttered/overlapped with some fonts like Meera
>
> 4b is now resolved, details in
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=5146
>
> Now only remaining issue in the list is about selecting default font.
> I think it should be taken from fc-match :lang=ml

As far as I can tell from this thread, all the bugs were resolved,
except possibly this one.

Is this still a problem in Emacs 24?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#5080: indic text is not displayed correctly in emacs shell

[Kenichi Handa]

In article <m362kqry9u.fsf@stories.gnus.org>, Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Praveen A <pravi.a@gmail.com> writes:
>>> (4a) Why is it taking Lohit Malayalam by default
>>> (4b) Why is the display cluttered/overlapped with some fonts like Meera
> >
> > 4b is now resolved, details in
> > http://debbugs.gnu.org/cgi/bugreport.cgi?bug=5146
> >
> > Now only remaining issue in the list is about selecting default font.
> > I think it should be taken from fc-match :lang=ml

> As far as I can tell from this thread, all the bugs were resolved,
> except possibly this one.

> Is this still a problem in Emacs 24?

Yes.  I have not yet fixed the font selection mechanism to
reflect the configuration of fontconfig correctly.

---
Kenichi Handa
handa@m17n.org

bug#5080: indic text is not displayed correctly in emacs shell

[Stefan Kangas]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Praveen A <pravi.a@gmail.com> writes:
>
>>> (4a) Why is it taking Lohit Malayalam by default
>>> (4b) Why is the display cluttered/overlapped with some fonts like Meera
>>
>> 4b is now resolved, details in
>> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=5146
>>
>> Now only remaining issue in the list is about selecting default font.
>> I think it should be taken from fc-match :lang=ml
>
> As far as I can tell from this thread, all the bugs were resolved,
> except possibly this one.
>
> Is this still a problem in Emacs 24?

That was 9 years ago.  Now that Emacs 27.1 is out, with harfbuzz support
and many other changes, could you please try again and see if you are
still seeing any errors?

FWIW, I tried quickly inserting indic text into a shell-mode buffer and
couldn't see anything wrong with it at a glance.  (But my inspection may
be overly superficial I can't read the script.)

Thanks in advance.

Best regards,
Stefan Kangas

bug#5080: indic text is not displayed correctly in emacs shell

[Stefan Kangas]

Stefan Kangas <stefan@marxist.se> writes:

> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>
>> Praveen A <pravi.a@gmail.com> writes:
>>
>>>> (4a) Why is it taking Lohit Malayalam by default
>>>> (4b) Why is the display cluttered/overlapped with some fonts like Meera
>>>
>>> 4b is now resolved, details in
>>> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=5146
>>>
>>> Now only remaining issue in the list is about selecting default font.
>>> I think it should be taken from fc-match :lang=ml
>>
>> As far as I can tell from this thread, all the bugs were resolved,
>> except possibly this one.
>>
>> Is this still a problem in Emacs 24?
>
> That was 9 years ago.  Now that Emacs 27.1 is out, with harfbuzz support
> and many other changes, could you please try again and see if you are
> still seeing any errors?
>
> FWIW, I tried quickly inserting indic text into a shell-mode buffer and
> couldn't see anything wrong with it at a glance.  (But my inspection may
> be overly superficial I can't read the script.)
>
> Thanks in advance.

More information was requested, but none was given within 14 weeks, so
I'm closing this bug.  If this is still an issue, please reply to this
email (use "Reply to all" in your email client) and we can reopen the
bug report.

bug#5080: indic text is not displayed correctly in emacs shell

[Praveen A]

[-- Attachment #1: Type: text/plain, Size: 945 bytes --]

2009/12/1 Kenichi Handa <handa@m17n.org>:
> It seems that the coding system for your shell buffer is
> iso-8859-1.  In which locale did you invoke Emacs?  And
> please try to change the coding system for the shell buffer
> to utf-8 (C-x C-m p utf-8 RET utf-8 RET) and run ls and cat
> again.

I invoked emacs in en_US locale. I followed your instructions to
change encoding for shell buffer and output of cat is working as
expected, but ls output is better than previous, but still some junk
characters are coming (see attached screenshot).

Launching emacs in ml_IN locale had the same results as changing shell
buffer encoding.

-- 
പ്രവീണ്‍ അരിമ്പ്രത്തൊടിയില്‍
<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)
Join The DRM Elimination Crew Now!
http://fci.wikia.com/wiki/Anti-DRM-Campaign

[-- Attachment #2: emacs-en-locale.png --]
[-- Type: image/png, Size: 84134 bytes --]