Persian RTL problem

Persian RTL problem

[mehran khajavi]

[-- Attachment #1.1: Type: text/plain, Size: 545 bytes --]

Hi
I have problem with RTL for Persian language in Emacs
the letters that I'm writing in emacs  is separated!! (see
wrongbidipersian.png)
The correct form of lines are in correctbidipersian.png (in attachments)


I'm using GNU/Linux with latest version of linux kernel
latest development snapshot of emacs24 from repository

variables:
 tab-width: 32
 bidi-display-reordering: t

C-h H showing that Arabic bidi not working (see emacs-languages.png in
attachments)
Font that I'm using : Sans




-- 
-----
Mehran Khajavi
http://mehran.khajavi.ir

[-- Attachment #1.2: Type: text/html, Size: 922 bytes --]

[-- Attachment #2: emacs-languages.png --]
[-- Type: image/png, Size: 35115 bytes --]

[-- Attachment #3: wrongbidipersian.png --]
[-- Type: image/png, Size: 22354 bytes --]

[-- Attachment #4: correctbidipersian.png --]
[-- Type: image/png, Size: 2257 bytes --]

Re: Persian RTL problem

[Eli Zaretskii]

> From: mehran khajavi <sinoohe.mkh@gmail.com>
> Date: Sat, 24 Sep 2011 18:02:41 +0330
> 
> I have problem with RTL for Persian language in Emacs
> the letters that I'm writing in emacs  is separated!! (see
> wrongbidipersian.png)
> The correct form of lines are in correctbidipersian.png (in attachments)

What you are missing is the so-called "Arabic shaping", right?
(Forgive me for my ignorance, I don't read Arabic.)

If so, can you please show the line from your src/Makefile that starts
with TOOLKIT_LIBW?  It's possible that you don't have the necessary
libraries installed that are needed for shaping support. 

Re: Persian RTL problem

[mehran khajavi]

[-- Attachment #1: Type: text/plain, Size: 1024 bytes --]

$ grep "TOOLKIT_LIBW" src/Makefile
TOOLKIT_LIBW=-pthread -lgtk-x11-2.0 -lgdk-x11-2.0 -latk-1.0 -lgio-2.0
-lpangoft2-1.0 -lpangocairo-1.0 -lgdk_pixbuf-2.0 -lm -lcairo -lpng14
-lpango-1.0 -lfreetype -lfontconfig -lgobject-2.0 -lgmodule-2.0
-lgthread-2.0 -lrt -lglib-2.0
LIBXT=$(TOOLKIT_LIBW) $(LIBXT_OTHER)

On Sat, Sep 24, 2011 at 6:19 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: mehran khajavi <sinoohe.mkh@gmail.com>
> > Date: Sat, 24 Sep 2011 18:02:41 +0330
> >
> > I have problem with RTL for Persian language in Emacs
> > the letters that I'm writing in emacs  is separated!! (see
> > wrongbidipersian.png)
> > The correct form of lines are in correctbidipersian.png (in attachments)
>
> What you are missing is the so-called "Arabic shaping", right?
> (Forgive me for my ignorance, I don't read Arabic.)
>
> If so, can you please show the line from your src/Makefile that starts
> with TOOLKIT_LIBW?  It's possible that you don't have the necessary
> libraries installed that are needed for shaping support.
>
>

[-- Attachment #2: Type: text/html, Size: 1493 bytes --]

Re: Persian RTL problem

[Eli Zaretskii]

> From: mehran khajavi <sinoohe.mkh@gmail.com>
> Date: Sat, 24 Sep 2011 18:36:06 +0330
> Cc: emacs-devel@gnu.org
> 
> $ grep "TOOLKIT_LIBW" src/Makefile
> TOOLKIT_LIBW=-pthread -lgtk-x11-2.0 -lgdk-x11-2.0 -latk-1.0 -lgio-2.0
> -lpangoft2-1.0 -lpangocairo-1.0 -lgdk_pixbuf-2.0 -lm -lcairo -lpng14
> -lpango-1.0 -lfreetype -lfontconfig -lgobject-2.0 -lgmodule-2.0
> -lgthread-2.0 -lrt -lglib-2.0

I think you are missing some or all of the following, mentioned in the
file INSTALL:

  * Complex Text Layout support libraries

  Emacs needs the optional libraries "m17n-db", "libm17n-flt", "libotf"
  to correctly display such complex scripts as Indic and Khmer.
  On some systems, particularly GNU/Linux, these libraries may be
  already present or available as additional packages.  Note that if
  there is a separate `dev' or `devel' package, for use at compilation
  time rather than run time, you will need that as well as the
  corresponding run time package; typically the dev package will contain
  header files and a library archive.  Otherwise, you can download and
  build libraries from sources.

  The sources of these libraries are available by anonymous CVS from
  cvs.m17n.org.

      % cvs -d :pserver:anonymous@cvs.m17n.org:/cvs/m17n login
      % cvs -d :pserver:anonymous@cvs.m17n.org:/cvs/m17n co m17n-db
      % cvs -d :pserver:anonymous@cvs.m17n.org:/cvs/m17n co m17n-lib
      % cvs -d :pserver:anonymous@cvs.m17n.org:/cvs/m17n co libotf

  For m17n-lib, if you have problems with making the whole package
  because you lack some other packages on which m17n-lib depends, try to
  configure it with the option "--without-gui".

Handa-san, can you please tell which ones of these, if any, are needed
for Arabic shaping support?  I think at least libotf is needed, but
I'm not sure.

Re: Persian RTL problem

[Kenichi Handa]

In article <83wrcyrmnj.fsf@gnu.org>, Eli Zaretskii <eliz@gnu.org> writes:

> I think you are missing some or all of the following, mentioned in the
> file INSTALL:

>   * Complex Text Layout support libraries

>   Emacs needs the optional libraries "m17n-db", "libm17n-flt", "libotf"
>   to correctly display such complex scripts as Indic and Khmer.
>   On some systems, particularly GNU/Linux, these libraries may be
>   already present or available as additional packages.  Note that if
>   there is a separate `dev' or `devel' package, for use at compilation
>   time rather than run time, you will need that as well as the
>   corresponding run time package; typically the dev package will contain
>   header files and a library archive.  Otherwise, you can download and
>   build libraries from sources.

>   The sources of these libraries are available by anonymous CVS from
>   cvs.m17n.org.

>       % cvs -d :pserver:anonymous@cvs.m17n.org:/cvs/m17n login
>       % cvs -d :pserver:anonymous@cvs.m17n.org:/cvs/m17n co m17n-db
>       % cvs -d :pserver:anonymous@cvs.m17n.org:/cvs/m17n co m17n-lib
>       % cvs -d :pserver:anonymous@cvs.m17n.org:/cvs/m17n co libotf

>   For m17n-lib, if you have problems with making the whole package
>   because you lack some other packages on which m17n-lib depends, try to
>   configure it with the option "--without-gui".

> Handa-san, can you please tell which ones of these, if any, are needed
> for Arabic shaping support?  I think at least libotf is needed, but
> I'm not sure.

For Arabic shaping, you need all of them; i.e. m17n-db,
m17n-lib, and libotf.

---
handa@m17n.org

Re: Persian RTL problem

[Eli Zaretskii]

> From: Kenichi Handa <handa@m17n.org>
> Cc: sinoohe.mkh@gmail.com, emacs-devel@gnu.org
> Date: Sun, 25 Sep 2011 12:58:01 +0900
> 
> For Arabic shaping, you need all of them; i.e. m17n-db,
> m17n-lib, and libotf.

Thanks.

But I was mistaken in my previous mail: these libraries, if present,
are mentioned in different Makefile variables than the one I asked
about.  Here's what I have in my src/Makefile:

  FREETYPE_CFLAGS = -I/usr/include/freetype2
  FREETYPE_LIBS = -lfreetype
  LIBOTF_CFLAGS = -I/usr/include/freetype2
  LIBOTF_LIBS = -lz -lotf -lfreetype
  M17N_FLT_CFLAGS =
  M17N_FLT_LIBS = -lm17n-flt -lm17n-core

Please make sure you have this in your src/Makefile.  If one or more
of these variables are empty or mention only some of the libraries,
config.log should tell you why the configure script didn't think they
were available.

You can see if GCC is able to find these libraries like this:

  gcc --print-file-name=libotf.a

If GCC displays an absolute name in response to this command, it means
it can find it during compilation.  If it just repeats the name
without leading directories, it cannot.

Re: Persian RTL problem

[mehran khajavi]

[-- Attachment #1: Type: text/plain, Size: 1411 bytes --]

wow great
it solved
but only  with gtk+ screen, not from emacs -nw option!
any solution to get it work in terminal?!

On Sun, Sep 25, 2011 at 8:27 AM, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Kenichi Handa <handa@m17n.org>
> > Cc: sinoohe.mkh@gmail.com, emacs-devel@gnu.org
> > Date: Sun, 25 Sep 2011 12:58:01 +0900
> >
> > For Arabic shaping, you need all of them; i.e. m17n-db,
> > m17n-lib, and libotf.
>
> Thanks.
>
> But I was mistaken in my previous mail: these libraries, if present,
> are mentioned in different Makefile variables than the one I asked
> about.  Here's what I have in my src/Makefile:
>
>  FREETYPE_CFLAGS = -I/usr/include/freetype2
>  FREETYPE_LIBS = -lfreetype
>  LIBOTF_CFLAGS = -I/usr/include/freetype2
>  LIBOTF_LIBS = -lz -lotf -lfreetype
>  M17N_FLT_CFLAGS =
>  M17N_FLT_LIBS = -lm17n-flt -lm17n-core
>
> Please make sure you have this in your src/Makefile.  If one or more
> of these variables are empty or mention only some of the libraries,
> config.log should tell you why the configure script didn't think they
> were available.
>
> You can see if GCC is able to find these libraries like this:
>
>  gcc --print-file-name=libotf.a
>
> If GCC displays an absolute name in response to this command, it means
> it can find it during compilation.  If it just repeats the name
> without leading directories, it cannot.
>



-- 
-----
Mehran Khajavi
http://mehran.khajavi.ir

[-- Attachment #2: Type: text/html, Size: 2116 bytes --]

Re: Persian RTL problem

[Eli Zaretskii]

> From: mehran khajavi <sinoohe.mkh@gmail.com>
> Date: Sun, 25 Sep 2011 21:07:56 +0330
> Cc: Kenichi Handa <handa@m17n.org>, emacs-devel@gnu.org
> 
> wow great
> it solved
> but only  with gtk+ screen, not from emacs -nw option!
> any solution to get it work in terminal?!

To have Arabic shaping in "emacs -nw", you need a terminal that
supports Arabic shaping.  Emacs cannot shape by itself when it works
on a TTY, because TTYs support only single-character writes.

I have no idea which terminals support shaping, sorry.  If you find
out, please post the information here, so we could include it in the
relevant documentation.

Re: Persian RTL problem

[mehran khajavi]

[-- Attachment #1: Type: text/plain, Size: 1594 bytes --]

I searched a lot but couldn't find any terminal with this attribute
I tested xterm but it doesn't work

another problems:
In some buffers for example ERC irc channels RTL doesn't work
It aligned as Left , Is there any function to set buffer as RTL?
or is there any option to set if most of the text is in Persian language set
this line as RTL?

2.there is some bugs that I see in Emacs 24 pretest with RTL
according to attachment , text that bordered with sign "1" should be in
right of the buffer
when I enter a blank line and enter the same line it will be aligned in the
right way(signed as 4)
after that I entered an English text without blank line (signed as 2),it
aligned at right but it should be in left
numbers of persian language will be aligned at left but it should aligned in
right(signed as 3)

On Sun, Sep 25, 2011 at 9:36 PM, Eli Zaretskii <eliz@gnu.org> wrote:

> > From: mehran khajavi <sinoohe.mkh@gmail.com>
> > Date: Sun, 25 Sep 2011 21:07:56 +0330
> > Cc: Kenichi Handa <handa@m17n.org>, emacs-devel@gnu.org
> >
> > wow great
> > it solved
> > but only  with gtk+ screen, not from emacs -nw option!
> > any solution to get it work in terminal?!
>
> To have Arabic shaping in "emacs -nw", you need a terminal that
> supports Arabic shaping.  Emacs cannot shape by itself when it works
> on a TTY, because TTYs support only single-character writes.
>
> I have no idea which terminals support shaping, sorry.  If you find
> out, please post the information here, so we could include it in the
> relevant documentation.
>
>


-- 
-----
Mehran Khajavi
http://mehran.khajavi.ir

[-- Attachment #2: Type: text/html, Size: 2456 bytes --]

Re: Persian RTL problem

[mehran khajavi]

[-- Attachment #1: Type: text/plain, Size: 1807 bytes --]

sorry
i forgot the screenshot


On 9/27/11, mehran khajavi <sinoohe.mkh@gmail.com> wrote:
> I searched a lot but couldn't find any terminal with this attribute
> I tested xterm but it doesn't work
>
> another problems:
> In some buffers for example ERC irc channels RTL doesn't work
> It aligned as Left , Is there any function to set buffer as RTL?
> or is there any option to set if most of the text is in Persian language
> set
> this line as RTL?
>
> 2.there is some bugs that I see in Emacs 24 pretest with RTL
> according to attachment , text that bordered with sign "1" should be in
> right of the buffer
> when I enter a blank line and enter the same line it will be aligned in the
> right way(signed as 4)
> after that I entered an English text without blank line (signed as 2),it
> aligned at right but it should be in left
> numbers of persian language will be aligned at left but it should aligned
> in
> right(signed as 3)
>
> On Sun, Sep 25, 2011 at 9:36 PM, Eli Zaretskii <eliz@gnu.org> wrote:
>
>> > From: mehran khajavi <sinoohe.mkh@gmail.com>
>> > Date: Sun, 25 Sep 2011 21:07:56 +0330
>> > Cc: Kenichi Handa <handa@m17n.org>, emacs-devel@gnu.org
>> >
>> > wow great
>> > it solved
>> > but only  with gtk+ screen, not from emacs -nw option!
>> > any solution to get it work in terminal?!
>>
>> To have Arabic shaping in "emacs -nw", you need a terminal that
>> supports Arabic shaping.  Emacs cannot shape by itself when it works
>> on a TTY, because TTYs support only single-character writes.
>>
>> I have no idea which terminals support shaping, sorry.  If you find
>> out, please post the information here, so we could include it in the
>> relevant documentation.
>>
>>
>
>
> --
> -----
> Mehran Khajavi
> http://mehran.khajavi.ir
>


-- 
-----
Mehran Khajavi
http://mehran.khajavi.ir

[-- Attachment #2: RTL.png --]
[-- Type: image/png, Size: 52008 bytes --]

Re: Persian RTL problem

[Eli Zaretskii]

> From: mehran khajavi <sinoohe.mkh@gmail.com>
> Date: Tue, 27 Sep 2011 15:49:07 +0330
> Cc: emacs-devel@gnu.org, handa@m17n.org
> 
> In some buffers for example ERC irc channels RTL doesn't work
> It aligned as Left , Is there any function to set buffer as RTL?
> or is there any option to set if most of the text is in Persian language set
> this line as RTL?

I think you want bidi-paragraph-direction, it's described in the user
manual.  When it is set to its default nil value, Emacs determines the
paragraph direction dynamically from the paragraph contents.  It is
possible that ERC sets this variable to `left-to-right', in which case
all the paragraphs will be aligned to the left.  This is a user
variable, specific to each buffer, so you can change it if the default
does not suit you.

> 2.there is some bugs that I see in Emacs 24 pretest with RTL
> according to attachment , text that bordered with sign "1" should be in
> right of the buffer
> when I enter a blank line and enter the same line it will be aligned in the
> right way(signed as 4)
> after that I entered an English text without blank line (signed as 2),it
> aligned at right but it should be in left

These are all by design.  Emacs does not consider each line a separate
paragraph, because that would look dreadfully in usual Emacs
plain-text buffers that have a newline at the end of each line.
Instead, Emacs implements what the Unicode Bidirectional Algorithm
(UBA) calls "higher protocols" to define paragraph beginning; an empty
line signals a new paragraph.  See the manual for details (node
"Bidirectional Editing").

> numbers of persian language will be aligned at left but it should aligned in
> right(signed as 3)

This is what the UBA mandates.  Arabic numbers are not strong
directional characters, they are weak characters.  Only paragraphs
that begin with a character of type R (strong right-to-left) or AL
(strong right-to-left Arabic) cause the paragraph direction to become
right-to-left.  If you follow the numbers with a Persian letter, the
paragraph will become right-to-left.

Here's the relevant portion from the UBA:

  3.3.1 The Paragraph Level
  P1. Split the text into separate paragraphs. A paragraph separator is kept with the previous paragraph. Within each paragraph, apply all the other rules of this algorithm.

  P2. In each paragraph, find the first character of type L, AL, or R.

  Because paragraph separators delimit text in this algorithm, this will generally be the first strong character after a paragraph separator or at the very beginning of the text. Note that the characters of type LRE, LRO, RLE, or RLO are ignored in this rule. This is because typically they are used to indicate that the embedded text is the opposite direction than the paragraph level.

  P3. If a character is found in P2 and it is of type AL or R, then set the paragraph embedding level to one; otherwise, set it to zero.

"Paragraph embedding level" being one means the paragraph is aligned
to the right and displayed right to left.

Thanks.

Re: Persian RTL problem

[Eli Zaretskii]

> From: Kenichi Handa <handa@m17n.org>
> Cc: sinoohe.mkh@gmail.com, emacs-devel@gnu.org
> Date: Sun, 25 Sep 2011 12:58:01 +0900
> 
> In article <83wrcyrmnj.fsf@gnu.org>, Eli Zaretskii <eliz@gnu.org> writes:
> 
> >       % cvs -d :pserver:anonymous@cvs.m17n.org:/cvs/m17n login
> >       % cvs -d :pserver:anonymous@cvs.m17n.org:/cvs/m17n co m17n-db
> >       % cvs -d :pserver:anonymous@cvs.m17n.org:/cvs/m17n co m17n-lib
> >       % cvs -d :pserver:anonymous@cvs.m17n.org:/cvs/m17n co libotf
> 
> >   For m17n-lib, if you have problems with making the whole package
> >   because you lack some other packages on which m17n-lib depends, try to
> >   configure it with the option "--without-gui".
> 
> > Handa-san, can you please tell which ones of these, if any, are needed
> > for Arabic shaping support?  I think at least libotf is needed, but
> > I'm not sure.
> 
> For Arabic shaping, you need all of them; i.e. m17n-db,
> m17n-lib, and libotf.

I added this to INSTALL.