bug#5081: zwj should not be ignored in indic text

bug#5081: zwj should not be ignored in indic text

[Praveen A]

[-- Attachment #1: Type: text/plain, Size: 703 bytes --]

Hi,

ZWJ is required to display some characters (like Malayalam chillu or
pure consonants) in many indic languages (Persian and Arabic also
needs these characters). These should not be ignored while displaying
indic text.

See attachment for an example.

പ്രവീ(ണ്‍) സറുമാ(ന്‍) ഹോഗ്‌വാ(ര്‍)ഡ്സ്

Reference display with gedit is also attached.

- Praveen
-- 
പ്രവീണ്‍ അരിമ്പ്രത്തൊടിയില്‍
<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)
Join The DRM Elimination Crew Now!
http://fci.wikia.com/wiki/Anti-DRM-Campaign

[-- Attachment #2: emacs-indic-zwj.png --]
[-- Type: image/png, Size: 94451 bytes --]

[-- Attachment #3: sample-indic-with-ZWJ-gedit.png --]
[-- Type: image/png, Size: 2932 bytes --]

bug#5081: zwj should not be ignored in indic text

[Kenichi Handa]

> ZWJ is required to display some characters (like Malayalam chillu or
> pure consonants) in many indic languages (Persian and Arabic also
> needs these characters). These should not be ignored while displaying
> indic text.

Please try to evaluate this:

(let ((script-regexp-alist
       `((devanagari . "[\x900-\x97F\x200C\x200D]+")
	 (bengali . "[\x980-\x9FF\x200C\x200D]+")
	 (gurmukhi . "[\xA00-\xA7F\x200C\x200D]+")
	 (gujarati . "[\xA80-\xAFF\x200C\x200D]+")
	 (oriya . "[\xB00-\xB7F\x200C\x200D]+")
	 (tamil . "[\xB80-\xBFF\x200C\x200D]+")
	 (telugu . "[\xC00-\xC7F\x200C\x200D]+")
	 (kannada . "[\xC80-\xCFF\x200C\x200D]+")
	 (malayalam . "[\xD00-\xD7F\x200C\x200D]+"))))
  (map-char-table
   #'(lambda (key val)
       (let ((slot (assq val script-regexp-alist)))
	 (if slot
	     (set-char-table-range
	      composition-function-table key
	      (list (vector (cdr slot) 0 'font-shape-gstring))))))
   char-script-table))

If it still doesn't work, please tell me the version numbers
of libotf and libm17n-flt, and also tell me which font is selected
for Malayalam.

% pkg-config --modversion libotf
% pkg-config --modversion m17n-flt

---
Kenichi Handa
handa@m17n.org

bug#5081: zwj should not be ignored in indic text

[Praveen A]

[-- Attachment #1: Type: text/plain, Size: 2294 bytes --]

2009/12/1 Kenichi Handa <handa@m17n.org>:
>> ZWJ is required to display some characters (like Malayalam chillu or
>> pure consonants) in many indic languages (Persian and Arabic also
>> needs these characters). These should not be ignored while displaying
>> indic text.
>
> Please try to evaluate this:
>
> (let ((script-regexp-alist
>       `((devanagari . "[\x900-\x97F\x200C\x200D]+")
>         (bengali . "[\x980-\x9FF\x200C\x200D]+")
>         (gurmukhi . "[\xA00-\xA7F\x200C\x200D]+")
>         (gujarati . "[\xA80-\xAFF\x200C\x200D]+")
>         (oriya . "[\xB00-\xB7F\x200C\x200D]+")
>         (tamil . "[\xB80-\xBFF\x200C\x200D]+")
>         (telugu . "[\xC00-\xC7F\x200C\x200D]+")
>         (kannada . "[\xC80-\xCFF\x200C\x200D]+")
>         (malayalam . "[\xD00-\xD7F\x200C\x200D]+"))))
>  (map-char-table
>   #'(lambda (key val)
>       (let ((slot (assq val script-regexp-alist)))
>         (if slot
>             (set-char-table-range
>              composition-function-table key
>              (list (vector (cdr slot) 0 'font-shape-gstring))))))
>   char-script-table))

Now it is displayed as a separate glyph, but there is no substitution
happening. ണ + ് + ZWJ should be displayed as one glyph (gpos
substitution).


> If it still doesn't work, please tell me the version numbers
> of libotf and libm17n-flt, and also tell me which font is selected
> for Malayalam.

It started working after I changed the default font to Rachana. It
also worked with Meera, but there is another problem when using Meera
https://savannah.nongnu.org/bugs/?28129  ttf-freefont is coming as a
default. I confirmed the need for above code by setting Rachana as
default font without the above code and I can confirm, the above code
is required to display zwj.

>
> % pkg-config --modversion libotf
> % pkg-config --modversion m17n-flt

libotf 0.9.10
m17n-flt 1.5.4

- Praveen

-- 
പ്രവീണ്‍ അരിമ്പ്രത്തൊടിയില്‍
<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)
Join The DRM Elimination Crew Now!
http://fci.wikia.com/wiki/Anti-DRM-Campaign

[-- Attachment #2: emacs-zwj-after.png --]
[-- Type: image/png, Size: 68816 bytes --]

[-- Attachment #3: zwj-with-rachana.png --]
[-- Type: image/png, Size: 70100 bytes --]

bug#5081: zwj should not be ignored in indic text

[Praveen A]

2009/12/1 Praveen A <pravi.a@gmail.com>:
> ttf-freefont is coming as a
> default.

Just to confirm it is not a bug with ttf-freefont, I tested a glyph
with zwj substitution in gedit with FreeMono and it was working
correctly there. So there is still some issue when ttf-freefont is
used in emacs to display complex glyphs containing ZWJ, which does not
happen with Rachana (included in ttf-malayalam-fonts in debian).

- Praveen
-- 
പ്രവീണ്‍ അരിമ്പ്രത്തൊടിയില്‍
<GPLv2> I know my rights; I want my phone call!
<DRM> What use is a phone call, if you are unable to speak?
(as seen on /.)
Join The DRM Elimination Crew Now!
http://fci.wikia.com/wiki/Anti-DRM-Campaign

bug#5081: zwj should not be ignored in indic text

[Lars Magne Ingebrigtsen]

Praveen A <pravi.a@gmail.com> writes:

> ZWJ is required to display some characters (like Malayalam chillu or
> pure consonants) in many indic languages (Persian and Arabic also
> needs these characters). These should not be ignored while displaying
> indic text.
>
> See attachment for an example.
>
> പ്രവീ(ണ്‍) സറുമാ(ന്‍) ഹോഗ്‌വാ(ര്‍)ഡ്സ്

I can't read Indic, but the text seems to be displayed (for me) in
pretty much the same way as in gedit, and not as they were displayed in
the Emacs screenshot.

So has this been fixed in Emacs 24?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

bug#5081: zwj should not be ignored in indic text

[Stefan Kangas]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Praveen A <pravi.a@gmail.com> writes:
>
>> ZWJ is required to display some characters (like Malayalam chillu or
>> pure consonants) in many indic languages (Persian and Arabic also
>> needs these characters). These should not be ignored while displaying
>> indic text.
>>
>> See attachment for an example.
>>
>> പ്രവീ(ണ്‍) സറുമാ(ന്‍) ഹോഗ്‌വാ(ര്‍)ഡ്സ്
>
> I can't read Indic, but the text seems to be displayed (for me) in
> pretty much the same way as in gedit, and not as they were displayed in
> the Emacs screenshot.
>
> So has this been fixed in Emacs 24?

That was 8 years ago, and I'm seeing the same thing as Lars here on
Emacs 26.1 and current master.  In other words, it looks like the
gedit screenshot and not the incorrect emacs one.

Can you still reproduce this on a modern version of Emacs?  If I don't
hear back from you in a couple of weeks, I'll just assume that this has
been fixed since this was reported and close this bug.

Best regards,
Stefan Kangas

bug#5081: zwj should not be ignored in indic text

[Stefan Kangas]

Stefan Kangas <stefan@marxist.se> writes:

> Can you still reproduce this on a modern version of Emacs?  If I don't
> hear back from you in a couple of weeks, I'll just assume that this has
> been fixed since this was reported and close this bug.

More information was requested, but none was given within 4 weeks.
I'm therefore closing this bug report now.  If anyone can reproduce
the original issue on a modern version of Emacs, please reopen.

Best regards,
Stefan Kangas