Not able to display \u110BD and \u110CD in Emacs

Not able to display \u110BD and \u110CD in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1.1: Type: text/plain, Size: 2055 bytes --]

Hi! I was trying to implement the Kaithi script in Emacs. Everything seems
to be working fine except for two characters: Kaithi Number Sign (\u110BD)
and Kaithi Number Sign Above (\u110CD), they do not appear when inserted in
a buffer, instead it is just whitespace.
According to the Unicode standard, they behave like the Arabic Number Sign
(\u0600) which spans numbers. I have attached a render from hb-view

This is what I have written in lisp/language/indian.el
Please tell me what I am doing wrong.

;; Kaithi composition rules
(let ((consonant "[\x1108D-\x110AF]")
      (non-consonant "[^\x1108D-\x110AF\x110B9]")
      (vowel "[\x110B0-\x110B8\x110C2]")
      (anusvara-candrabindu "[\x11080\x11081]")
      (virama "\x110B9")
      (number-sign "\x110BD")
      (number-sign-above "\x110CD")
      (numerals "\x966-\x96F"))
  (set-char-table-range composition-function-table
                        '(#x110B0 . #x110B8)
                        (list (vector
                               (concat consonant vowel anusvara-candrabindu
"?")
                               1 'font-shape-gstring)))
  (set-char-table-range composition-function-table
                        '(#x110B9 . #x110B9)
                        (list (vector
                               (concat consonant "\\(?:" virama consonant
"\\)+"
                                       vowel "*\\|" consonant virama)
                               1 'font-shape-gstring)))
  (set-char-table-range composition-function-table
                        '(#x110BD . #x110BD)
                        (list (vector
                               (concat number-sign numerals)
                               1 'font-shape-gstring)))
  (set-char-table-range composition-function-table
                        '(#x110CD . #x110CD)
                        (list (vector
                               (concat number-sign-above numerals)
                               1 'font-shape-gstring))))

In lisp/internation/fontset.el
(kaithi ,(font-spec :registry "iso10646-1" :otf '(kthi nil (rphf))))

[-- Attachment #1.2: Type: text/html, Size: 2723 bytes --]

[-- Attachment #2: hb-view.png --]
[-- Type: image/png, Size: 12978 bytes --]

Re: Not able to display \u110BD and \u110CD in Emacs

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh
>  <lumarzeli30@gmail.com>
> Date: Fri, 29 Apr 2022 04:47:08 +0530
> 
> Hi! I was trying to implement the Kaithi script in Emacs. Everything seems
> to be working fine except for two characters: Kaithi Number Sign (\u110BD)
> and Kaithi Number Sign Above (\u110CD), they do not appear when inserted in
> a buffer, instead it is just whitespace.
> According to the Unicode standard, they behave like the Arabic Number Sign
> (\u0600) which spans numbers. I have attached a render from hb-view
> 
> This is what I have written in lisp/language/indian.el
> Please tell me what I am doing wrong.

You didn't show any Kaithi text that you think is not being displayed
correctly by Emacs.  Without that, it's hard to look into the problem.

>       (numerals "\x966-\x96F"))

I believe this should be

       (numerals "[\x966-\x96F]")

IOW, the "[..]" brackets are missing.  Maybe that is the cause of your
problem.

Re: Not able to display \u110BD and \u110CD in Emacs

[Eli Zaretskii]

> Date: Fri, 29 Apr 2022 08:52:18 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> >       (numerals "\x966-\x96F"))
> 
> I believe this should be
> 
>        (numerals "[\x966-\x96F]")
> 
> IOW, the "[..]" brackets are missing.  Maybe that is the cause of your
> problem.

And one more potential problem.  Since the rules for the number signs
are:

  (set-char-table-range composition-function-table
                        '(#x110BD . #x110BD)
                        (list (vector
                               (concat number-sign numerals)
                               1 'font-shape-gstring)))
  (set-char-table-range composition-function-table
                        '(#x110CD . #x110CD)
                        (list (vector
                               (concat number-sign-above numerals)
                               1 'font-shape-gstring))))

which means the number signs always come _before_ the character with
which it should be composed, you should use 0 in the rule, not 1.
That number means how many characters to look back for finding the
beginning of a composable sequence, and in these two rules the
sequence _begins_ with the character which triggers composition, so
the look-back is zero, not 1.

Re: Not able to display \u110BD and \u110CD in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1.1: Type: text/plain, Size: 3015 bytes --]

I did the changes mentioned.
1. Added the square brackets
2. Changed the 1 to a 0

;; Kaithi composition rules
(let ((consonant "[\x1108D-\x110AF]")
      (non-consonant "[^\x1108D-\x110AF\x110B9]")
      (vowel "[\x110B0-\x110B8\x110C2]")
      (anusvara-candrabindu "[\x11080\x11081]")
      (virama "\x110B9")
      (number-sign "\x110BD")
      (number-sign-above "\x110CD")
      (numerals "[\x966-\x96F]"))
  (set-char-table-range composition-function-table
                        '(#x110B0 . #x110B8)
                        (list (vector
                               (concat consonant vowel anusvara-candrabindu
"?")
                               1 'font-shape-gstring)))
  (set-char-table-range composition-function-table
                        '(#x110B9 . #x110B9)
                        (list (vector
                               (concat consonant "\\(?:" virama consonant
"\\)+"
                                       vowel "*\\|" consonant virama)
                               1 'font-shape-gstring)))
  (set-char-table-range composition-function-table
                        '(#x110BD . #x110BD)
                        (list (vector
                               (concat number-sign numerals)
                               0 'font-shape-gstring)))
  (set-char-table-range composition-function-table
                        '(#x110CD . #x110CD)
                        (list (vector
                               (concat number-sign-above numerals)
                               0 'font-shape-gstring))))

Now when typed alone they do not appear but when written with numbers, they
appear as NOTDEF glyphs

On Fri, Apr 29, 2022 at 12:23 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Fri, 29 Apr 2022 08:52:18 +0300
> > From: Eli Zaretskii <eliz@gnu.org>
> > Cc: emacs-devel@gnu.org
> >
> > >       (numerals "\x966-\x96F"))
> >
> > I believe this should be
> >
> >        (numerals "[\x966-\x96F]")
> >
> > IOW, the "[..]" brackets are missing.  Maybe that is the cause of your
> > problem.
>
> And one more potential problem.  Since the rules for the number signs
> are:
>
>   (set-char-table-range composition-function-table
>                         '(#x110BD . #x110BD)
>                         (list (vector
>                                (concat number-sign numerals)
>                                1 'font-shape-gstring)))
>   (set-char-table-range composition-function-table
>                         '(#x110CD . #x110CD)
>                         (list (vector
>                                (concat number-sign-above numerals)
>                                1 'font-shape-gstring))))
>
> which means the number signs always come _before_ the character with
> which it should be composed, you should use 0 in the rule, not 1.
> That number means how many characters to look back for finding the
> beginning of a composable sequence, and in these two rules the
> sequence _begins_ with the character which triggers composition, so
> the look-back is zero, not 1.
>

[-- Attachment #1.2: Type: text/html, Size: 4316 bytes --]

[-- Attachment #2: emacs-kaithi-number-signs.png --]
[-- Type: image/png, Size: 52850 bytes --]

Re: Not able to display \u110BD and \u110CD in Emacs

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> Date: Fri, 29 Apr 2022 15:15:41 +0530
> Cc: emacs-devel@gnu.org
> 
> I did the changes mentioned.
> 1. Added the square brackets
> 2. Changed the 1 to a 0
> 
> ;; Kaithi composition rules
> (let ((consonant "[\x1108D-\x110AF]")
>       (non-consonant "[^\x1108D-\x110AF\x110B9]")
>       (vowel "[\x110B0-\x110B8\x110C2]")
>       (anusvara-candrabindu "[\x11080\x11081]")
>       (virama "\x110B9")
>       (number-sign "\x110BD")
>       (number-sign-above "\x110CD")
>       (numerals "[\x966-\x96F]"))
>   (set-char-table-range composition-function-table
>                         '(#x110B0 . #x110B8)
>                         (list (vector
>                                (concat consonant vowel anusvara-candrabindu "?")
>                                1 'font-shape-gstring)))
>   (set-char-table-range composition-function-table
>                         '(#x110B9 . #x110B9)
>                         (list (vector
>                                (concat consonant "\\(?:" virama consonant "\\)+"
>                                        vowel "*\\|" consonant virama)
>                                1 'font-shape-gstring)))
>   (set-char-table-range composition-function-table
>                         '(#x110BD . #x110BD)
>                         (list (vector
>                                (concat number-sign numerals)
>                                0 'font-shape-gstring)))
>   (set-char-table-range composition-function-table
>                         '(#x110CD . #x110CD)
>                         (list (vector
>                                (concat number-sign-above numerals)
>                                0 'font-shape-gstring))))
> 
> Now when typed alone they do not appear but when written with numbers, they appear as NOTDEF glyphs

Are you sure the font supports all of those characters?  What font is
used for them in each case.

And again, you don't show the text that doesn't render correctly (as
characters), so it's impossible to investigate the cases you tried.
Images help to see how stuff is displayed, but they cannot be used to
reliably figure out what text was displayed.  I asked you to show the
actual text many times.

Re: Not able to display \u110BD and \u110CD in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 3090 bytes --]

They seem to be working after changing the devanagari font to Noto Sans
Kaithi but only for one digit though, also they do not appear when alone.
Since the kaithi unicode does not have its own numerals and uses the
devanagari ones, is there any way to only change the devanagari font when
it is near the kaithi number signs?

Here is the Text I entered, sorry for not including them earlier, I thought
that mentioning their codepoints should be enough.
KAITHI NUMBER SIGN: 𑂽
KAITHI NUMBER SIGN + DEVANAGARI 0: 𑂽१

KAITHI NUMBER SIGN ABOVE: 𑃍
KAITHI NUMBER SIGN ABOVE + DEVANAGARI 0: 𑃍०

On Fri, Apr 29, 2022 at 4:20 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> > Date: Fri, 29 Apr 2022 15:15:41 +0530
> > Cc: emacs-devel@gnu.org
> >
> > I did the changes mentioned.
> > 1. Added the square brackets
> > 2. Changed the 1 to a 0
> >
> > ;; Kaithi composition rules
> > (let ((consonant "[\x1108D-\x110AF]")
> >       (non-consonant "[^\x1108D-\x110AF\x110B9]")
> >       (vowel "[\x110B0-\x110B8\x110C2]")
> >       (anusvara-candrabindu "[\x11080\x11081]")
> >       (virama "\x110B9")
> >       (number-sign "\x110BD")
> >       (number-sign-above "\x110CD")
> >       (numerals "[\x966-\x96F]"))
> >   (set-char-table-range composition-function-table
> >                         '(#x110B0 . #x110B8)
> >                         (list (vector
> >                                (concat consonant vowel
> anusvara-candrabindu "?")
> >                                1 'font-shape-gstring)))
> >   (set-char-table-range composition-function-table
> >                         '(#x110B9 . #x110B9)
> >                         (list (vector
> >                                (concat consonant "\\(?:" virama
> consonant "\\)+"
> >                                        vowel "*\\|" consonant virama)
> >                                1 'font-shape-gstring)))
> >   (set-char-table-range composition-function-table
> >                         '(#x110BD . #x110BD)
> >                         (list (vector
> >                                (concat number-sign numerals)
> >                                0 'font-shape-gstring)))
> >   (set-char-table-range composition-function-table
> >                         '(#x110CD . #x110CD)
> >                         (list (vector
> >                                (concat number-sign-above numerals)
> >                                0 'font-shape-gstring))))
> >
> > Now when typed alone they do not appear but when written with numbers,
> they appear as NOTDEF glyphs
>
> Are you sure the font supports all of those characters?  What font is
> used for them in each case.
>
> And again, you don't show the text that doesn't render correctly (as
> characters), so it's impossible to investigate the cases you tried.
> Images help to see how stuff is displayed, but they cannot be used to
> reliably figure out what text was displayed.  I asked you to show the
> actual text many times.
>

[-- Attachment #2: Type: text/html, Size: 4204 bytes --]

Re: Not able to display \u110BD and \u110CD in Emacs

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> Date: Fri, 29 Apr 2022 16:47:22 +0530
> Cc: emacs-devel@gnu.org
> 
> They seem to be working after changing the devanagari font to Noto Sans Kaithi but only for one digit
> though

But that's what the rule you wrote says:

      (numerals "[\x966-\x96F]"))
  [...]
  (set-char-table-range composition-function-table
                        '(#x110BD . #x110BD)
                        (list (vector
                               (concat number-sign numerals)
                               0 'font-shape-gstring)))

The regexp in 'numerals' will match only a single character, so just
one character after number-sign will be composed.  if you want to
compose several ones, you need to say this instead:

      (numerals "[\x966-\x96F]+"))

> also they do not appear when alone.

Do they have any glyphs in the font?  Did you try to use
font-get-glyphs to see if the font can display those characters when
they are alone?

> Since the kaithi unicode does not have its own numerals and uses the devanagari ones, is there any way to
> only change the devanagari font when it is near the kaithi number signs?

That's what your composition rules already do: they are only triggered
when the character preceding the numerals is a number sign.  So I
don't think I understand the problem.

> 
> Here is the Text I entered, sorry for not including them earlier, I thought that mentioning their codepoints
> should be enough.
> KAITHI NUMBER SIGN: 𑂽
> KAITHI NUMBER SIGN + DEVANAGARI 0: 𑂽१
> 
> KAITHI NUMBER SIGN ABOVE: 𑃍
> KAITHI NUMBER SIGN ABOVE + DEVANAGARI 0: 𑃍०

When you put the cursor at the number sign character, don't you see a
thin 1-pixel space there?

Re: Not able to display \u110BD and \u110CD in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 3827 bytes --]

>
> The regexp in 'numerals' will match only a single character, so just
> one character after number-sign will be composed.  if you want to
> compose several ones, you need to say this instead:
>
>       (numerals "[\x966-\x96F]+"))


It is working fine now, Thank You!

 Do they have any glyphs in the font?  Did you try to use
> font-get-glyphs to see if the font can display those characters when
> they are alone?
>

I am an extreme novice, so please bear with me, but I cannot get the
function to work. I tried to enter the following as the argument FONT
OBJECT but none worked:
#<font-object "-GOOG-Noto Sans
Kaithi-regular-normal-normal-*-23-*-*-*-*-0-iso10646-1">
"-GOOG-Noto Sans Kaithi-regular-normal-normal-*-23-*-*-*-*-0-iso10646-1"
"#<font-object -GOOG-Noto Sans
Kaithi-regular-normal-normal-*-23-*-*-*-*-0-iso10646-1>"
"Noto Sans Kaithi"
"/usr/share/fonts/noto/NotoSansKaithi-Regular.ttf"

Though they do appear alone in Firefox and LibreOffice Writer.

That's what your composition rules already do: they are only triggered
> when the character preceding the numerals is a number sign.  So I
> don't think I understand the problem.
>

I want the font of devanagari and kaithi to be different, but since kaithi
uses devanagari numerals and a devanagari font other than Noto Sans Kaithi
does not render the number signs, I was asking that is it possible to only
change the devanagari font to Noto Sans Kaithi if it is around a number
sign.
Though this is a very specific problem of no significance, so it is fine if
you don't answer. I will not even use the number signs very much.

When you put the cursor at the number sign character, don't you see a
> thin 1-pixel space there?
>

Yes, when the character is not visible in Emacs and whenI put the cursor in
its place there is a thin 1-pixel space there.

On Fri, Apr 29, 2022 at 5:48 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> > Date: Fri, 29 Apr 2022 16:47:22 +0530
> > Cc: emacs-devel@gnu.org
> >
> > They seem to be working after changing the devanagari font to Noto Sans
> Kaithi but only for one digit
> > though
>
> But that's what the rule you wrote says:
>
>       (numerals "[\x966-\x96F]"))
>   [...]
>   (set-char-table-range composition-function-table
>                         '(#x110BD . #x110BD)
>                         (list (vector
>                                (concat number-sign numerals)
>                                0 'font-shape-gstring)))
>
> The regexp in 'numerals' will match only a single character, so just
> one character after number-sign will be composed.  if you want to
> compose several ones, you need to say this instead:
>
>       (numerals "[\x966-\x96F]+"))
>
> > also they do not appear when alone.
>
> Do they have any glyphs in the font?  Did you try to use
> font-get-glyphs to see if the font can display those characters when
> they are alone?
>
> > Since the kaithi unicode does not have its own numerals and uses the
> devanagari ones, is there any way to
> > only change the devanagari font when it is near the kaithi number signs?
>
> That's what your composition rules already do: they are only triggered
> when the character preceding the numerals is a number sign.  So I
> don't think I understand the problem.
>
> >
> > Here is the Text I entered, sorry for not including them earlier, I
> thought that mentioning their codepoints
> > should be enough.
> > KAITHI NUMBER SIGN: 𑂽
> > KAITHI NUMBER SIGN + DEVANAGARI 0: 𑂽१
> >
> > KAITHI NUMBER SIGN ABOVE: 𑃍
> > KAITHI NUMBER SIGN ABOVE + DEVANAGARI 0: 𑃍०
>
> When you put the cursor at the number sign character, don't you see a
> thin 1-pixel space there?
>

[-- Attachment #2: Type: text/html, Size: 5419 bytes --]

Re: Not able to display \u110BD and \u110CD in Emacs

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> Date: Fri, 29 Apr 2022 20:56:56 +0530
> Cc: emacs-devel@gnu.org
> 
>   Do they have any glyphs in the font?  Did you try to use
>  font-get-glyphs to see if the font can display those characters when
>  they are alone?
> 
>  
> I am an extreme novice, so please bear with me, but I cannot get the function to work. I tried to enter the
> following as the argument FONT OBJECT but none worked:
> #<font-object "-GOOG-Noto Sans Kaithi-regular-normal-normal-*-23-*-*-*-*-0-iso10646-1">
> "-GOOG-Noto Sans Kaithi-regular-normal-normal-*-23-*-*-*-*-0-iso10646-1" 
> "#<font-object -GOOG-Noto Sans Kaithi-regular-normal-normal-*-23-*-*-*-*-0-iso10646-1>"
> "Noto Sans Kaithi"
> "/usr/share/fonts/noto/NotoSansKaithi-Regular.ttf"

I suggest to use font-at to get the font-object you need for
font-get-glyphs.

>  That's what your composition rules already do: they are only triggered
>  when the character preceding the numerals is a number sign.  So I
>  don't think I understand the problem.
>  
> I want the font of devanagari and kaithi to be different, but since kaithi uses devanagari numerals and a
> devanagari font other than Noto Sans Kaithi does not render the number signs, I was asking that is it
> possible to only change the devanagari font to Noto Sans Kaithi if it is around a number sign.

No, that's not possible, sorry.

>  When you put the cursor at the number sign character, don't you see a
>  thin 1-pixel space there?
> 
> Yes, when the character is not visible in Emacs and whenI put the cursor in its place there is a thin 1-pixel
> space there.

So the character is actually visible, it is just displayed as a thin
space.  Which means that either its glyph in the font is like that, or
that the font lacks a glyph for it.  What does "C-u C-x =" say when
the cursor is on that thin 1-pixel space?

Re: Not able to display \u110BD and \u110CD in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1.1: Type: text/plain, Size: 3928 bytes --]

>
> I suggest to use font-at to get the font-object you need for
> font-get-glyphs.

I had already used that and got
#<font-object "-GOOG-Noto Sans
Kaithi-regular-normal-normal-*-23-*-*-*-*-0-iso10646-1">
but font-get-glyphs was not accepting it.


> So the character is actually visible, it is just displayed as a thin
> space.  Which means that either its glyph in the font is like that, or
> that the font lacks a glyph for it.  What does "C-u C-x =" say when
> the cursor is on that thin 1-pixel space?
>

position: 89 of 89 (99%), column: 0
            character: 𑂽 (displayed as 𑂽) (codepoint 69821, #o210275,
#x110bd)
              charset: unicode (Unicode (ISO10646))
code point in charset: 0x110BD
               script: kaithi
               syntax: w which means: word
             category: L:Strong L2R
             to input: type "C-x 8 RET 110bd" or "C-x 8 RET KAITHI NUMBER
SIGN"
          buffer code: #xF0 #x91 #x82 #xBD
            file code: #xF0 #x91 #x82 #xBD (encoded by coding system utf-8)
              display: by this font (glyph code):
    ftcrhb:-GOOG-Noto Sans
Kaithi-regular-normal-normal-*-23-*-*-*-*-0-iso10646-1 (#x48)

Character code properties: customize what to show
  name: KAITHI NUMBER SIGN
  general-category: Cf (Other, Format)
  decomposition: (69821) ('𑂽')

There is an overlay here:
 From 89 to 90
  face                 hl-line
  priority             -50
  window               #<window 3 on *scratch*>


There are text properties here:
  fontified            t
  rear-nonsticky       t

In the character section the character is not displayed in emacs but
displayed in firefox, but it is displayed in the decomposition section in
emacs. I have attached the images
Opening the font file in font forge also shows their glyphs

On Fri, Apr 29, 2022 at 9:43 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> > Date: Fri, 29 Apr 2022 20:56:56 +0530
> > Cc: emacs-devel@gnu.org
> >
> >   Do they have any glyphs in the font?  Did you try to use
> >  font-get-glyphs to see if the font can display those characters when
> >  they are alone?
> >
> >
> > I am an extreme novice, so please bear with me, but I cannot get the
> function to work. I tried to enter the
> > following as the argument FONT OBJECT but none worked:
> > #<font-object "-GOOG-Noto Sans
> Kaithi-regular-normal-normal-*-23-*-*-*-*-0-iso10646-1">
> > "-GOOG-Noto Sans Kaithi-regular-normal-normal-*-23-*-*-*-*-0-iso10646-1"
> > "#<font-object -GOOG-Noto Sans
> Kaithi-regular-normal-normal-*-23-*-*-*-*-0-iso10646-1>"
> > "Noto Sans Kaithi"
> > "/usr/share/fonts/noto/NotoSansKaithi-Regular.ttf"
>
> I suggest to use font-at to get the font-object you need for
> font-get-glyphs.
>
> >  That's what your composition rules already do: they are only triggered
> >  when the character preceding the numerals is a number sign.  So I
> >  don't think I understand the problem.
> >
> > I want the font of devanagari and kaithi to be different, but since
> kaithi uses devanagari numerals and a
> > devanagari font other than Noto Sans Kaithi does not render the number
> signs, I was asking that is it
> > possible to only change the devanagari font to Noto Sans Kaithi if it is
> around a number sign.
>
> No, that's not possible, sorry.
>
> >  When you put the cursor at the number sign character, don't you see a
> >  thin 1-pixel space there?
> >
> > Yes, when the character is not visible in Emacs and whenI put the cursor
> in its place there is a thin 1-pixel
> > space there.
>
> So the character is actually visible, it is just displayed as a thin
> space.  Which means that either its glyph in the font is like that, or
> that the font lacks a glyph for it.  What does "C-u C-x =" say when
> the cursor is on that thin 1-pixel space?
>

[-- Attachment #1.2: Type: text/html, Size: 5186 bytes --]

[-- Attachment #2: firefox-gmail.png --]
[-- Type: image/png, Size: 6935 bytes --]

[-- Attachment #3: emacs.png --]
[-- Type: image/png, Size: 9979 bytes --]

[-- Attachment #4: emacs-decomposition.png --]
[-- Type: image/png, Size: 6317 bytes --]

[-- Attachment #5: font-forge.png --]
[-- Type: image/png, Size: 2728 bytes --]

Re: Not able to display \u110BD and \u110CD in Emacs

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> Date: Sat, 30 Apr 2022 10:22:00 +0530
> Cc: emacs-devel@gnu.org
> 
>  I suggest to use font-at to get the font-object you need for
>  font-get-glyphs.
> 
> I had already used that and got
> #<font-object "-GOOG-Noto Sans Kaithi-regular-normal-normal-*-23-*-*-*-*-0-iso10646-1">
> but font-get-glyphs was not accepting it.

"Not accepting" how?  It works for me, so I don't understand what goes
wrong in your case.

>  So the character is actually visible, it is just displayed as a thin
>  space.  Which means that either its glyph in the font is like that, or
>  that the font lacks a glyph for it.  What does "C-u C-x =" say when
>  the cursor is on that thin 1-pixel space?
> 
> position: 89 of 89 (99%), column: 0
>             character: 𑂽 (displayed as 𑂽) (codepoint 69821, #o210275, #x110bd)
>               charset: unicode (Unicode (ISO10646))
> code point in charset: 0x110BD
>                script: kaithi
>                syntax: w which means: word
>              category: L:Strong L2R
>              to input: type "C-x 8 RET 110bd" or "C-x 8 RET KAITHI NUMBER SIGN"
>           buffer code: #xF0 #x91 #x82 #xBD
>             file code: #xF0 #x91 #x82 #xBD (encoded by coding system utf-8)
>               display: by this font (glyph code):
>     ftcrhb:-GOOG-Noto Sans Kaithi-regular-normal-normal-*-23-*-*-*-*-0-iso10646-1 (#x48)

Since this shows the "by this font" part, it means the character _is_
displayed according to the font's glyph for it, and according to
Emacs's rules for displaying such "format-control" characters.  So now
I don't think I understand why you say this character is not shown by
Emacs, when the above clearly says it is displayed.

> In the character section the character is not displayed in emacs but displayed in firefox, but it is displayed in
> the decomposition section in emacs. I have attached the images
> Opening the font file in font forge also shows their glyphs

You are confusing the display of a lone codepoint with what Font Forge
does and what Emacs does in the "decomposition" display.  Those do not
show the lone character, they show it with special characters before
or after, to show how the character will look when combined with
others.

IOW, I think your expectations from how this character should be
displayed as a lone character are incorrect.

Can you tell why you want this to be displayed differently when it is
a lone character?  AFAIU, the only meaningful display of this
character is when it precedes numerals.

Re: Not able to display \u110BD and \u110CD in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 3429 bytes --]

>
> Can you tell why you want this to be displayed differently when it is
> a lone character?  AFAIU, the only meaningful display of this
> character is when it precedes numerals.

Yes you are correct, I was just confused as to why it was not being
displayed, but this works too.

btw, is the way how I have written the composition table correct? I have
copied it from the Brahmi composition one, or is it more apporpriate to do
it in the way of the devanagari, bengali, malayalam ones?

Also I plan to include more writing scripts in emacs, so should I send the
patches one by one or in bulk?



On Sat, Apr 30, 2022 at 11:33 AM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> > Date: Sat, 30 Apr 2022 10:22:00 +0530
> > Cc: emacs-devel@gnu.org
> >
> >  I suggest to use font-at to get the font-object you need for
> >  font-get-glyphs.
> >
> > I had already used that and got
> > #<font-object "-GOOG-Noto Sans
> Kaithi-regular-normal-normal-*-23-*-*-*-*-0-iso10646-1">
> > but font-get-glyphs was not accepting it.
>
> "Not accepting" how?  It works for me, so I don't understand what goes
> wrong in your case.
>
> >  So the character is actually visible, it is just displayed as a thin
> >  space.  Which means that either its glyph in the font is like that, or
> >  that the font lacks a glyph for it.  What does "C-u C-x =" say when
> >  the cursor is on that thin 1-pixel space?
> >
> > position: 89 of 89 (99%), column: 0
> >             character: 𑂽 (displayed as 𑂽) (codepoint 69821, #o210275,
> #x110bd)
> >               charset: unicode (Unicode (ISO10646))
> > code point in charset: 0x110BD
> >                script: kaithi
> >                syntax: w which means: word
> >              category: L:Strong L2R
> >              to input: type "C-x 8 RET 110bd" or "C-x 8 RET KAITHI
> NUMBER SIGN"
> >           buffer code: #xF0 #x91 #x82 #xBD
> >             file code: #xF0 #x91 #x82 #xBD (encoded by coding system
> utf-8)
> >               display: by this font (glyph code):
> >     ftcrhb:-GOOG-Noto Sans
> Kaithi-regular-normal-normal-*-23-*-*-*-*-0-iso10646-1 (#x48)
>
> Since this shows the "by this font" part, it means the character _is_
> displayed according to the font's glyph for it, and according to
> Emacs's rules for displaying such "format-control" characters.  So now
> I don't think I understand why you say this character is not shown by
> Emacs, when the above clearly says it is displayed.
>
> > In the character section the character is not displayed in emacs but
> displayed in firefox, but it is displayed in
> > the decomposition section in emacs. I have attached the images
> > Opening the font file in font forge also shows their glyphs
>
> You are confusing the display of a lone codepoint with what Font Forge
> does and what Emacs does in the "decomposition" display.  Those do not
> show the lone character, they show it with special characters before
> or after, to show how the character will look when combined with
> others.
>
> IOW, I think your expectations from how this character should be
> displayed as a lone character are incorrect.
>
> Can you tell why you want this to be displayed differently when it is
> a lone character?  AFAIU, the only meaningful display of this
> character is when it precedes numerals.
>

[-- Attachment #2: Type: text/html, Size: 4425 bytes --]

Re: Not able to display \u110BD and \u110CD in Emacs

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> Date: Sat, 30 Apr 2022 11:40:03 +0530
> Cc: emacs-devel@gnu.org
> 
> btw, is the way how I have written the composition table correct? I have copied it from the Brahmi
> composition one, or is it more apporpriate to do it in the way of the devanagari, bengali, malayalam ones?

It doesn't matter.  Whatever is more convenient for reading and
understanding the rules is preferable.

> Also I plan to include more writing scripts in emacs, so should I send the patches one by one or in bulk?

Separate patches, please.

Please also include with the patches the relevant additions to the
etc/HELLO file, and also don't forget to make sure
script-representative-chars has representative characters for the
script -- this is important for selecting fonts for those characters.

However, if you intend to submit more patches, I suggest to start your
legal paperwork of assigning the copyright to the FSF, so that we
could accept those future contributions from you.  If you agree, I
will send you the form now.

Re: Not able to display \u110BD and \u110CD in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 1291 bytes --]

Yes, please send the form.

शनि, 30 अप्रैल 2022, 12:30 pm को Eli Zaretskii <eliz@gnu.org> ने लिखा:

> > From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> > Date: Sat, 30 Apr 2022 11:40:03 +0530
> > Cc: emacs-devel@gnu.org
> >
> > btw, is the way how I have written the composition table correct? I have
> copied it from the Brahmi
> > composition one, or is it more apporpriate to do it in the way of the
> devanagari, bengali, malayalam ones?
>
> It doesn't matter.  Whatever is more convenient for reading and
> understanding the rules is preferable.
>
> > Also I plan to include more writing scripts in emacs, so should I send
> the patches one by one or in bulk?
>
> Separate patches, please.
>
> Please also include with the patches the relevant additions to the
> etc/HELLO file, and also don't forget to make sure
> script-representative-chars has representative characters for the
> script -- this is important for selecting fonts for those characters.
>
> However, if you intend to submit more patches, I suggest to start your
> legal paperwork of assigning the copyright to the FSF, so that we
> could accept those future contributions from you.  If you agree, I
> will send you the form now.
>

[-- Attachment #2: Type: text/html, Size: 1768 bytes --]

Re: Not able to display \u110BD and \u110CD in Emacs

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> Date: Sat, 30 Apr 2022 12:33:11 +0530
> Cc: emacs-devel@gnu.org
> 
> Yes, please send the form.

Thanks, form sent off-list.

Re: Not able to display \u110BD and \u110CD in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 556 bytes --]

> From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> Date: Sat, 30 Apr 2022 12:33:11 +0530
> Cc: emacs-devel@gnu.org
>
> Yes, please send the form.

Thanks, form sent off-list.

Thanks a lot! I have sent the form to assign@gnu.org


On Sat, Apr 30, 2022 at 12:49 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> > Date: Sat, 30 Apr 2022 12:33:11 +0530
> > Cc: emacs-devel@gnu.org
> >
> > Yes, please send the form.
>
> Thanks, form sent off-list.
>

[-- Attachment #2: Type: text/html, Size: 1244 bytes --]

Re: Not able to display \u110BD and \u110CD in Emacs

[Madhu]

* Eli Zaretskii <837d779kgx.fsf @gnu.org> :
Wrote on Sat, 30 Apr 2022 09:03:10 +0300:

> Since this shows the "by this font" part, it means the character _is_
> displayed according to the font's glyph for it, and according to
> Emacs's rules for displaying such "format-control" characters.  So now
> I don't think I understand why you say this character is not shown by
> Emacs, when the above clearly says it is displayed.
>
>> In the character section the character is not displayed in emacs but
>> displayed in firefox, but it is displayed in
>> the decomposition section in emacs. I have attached the images
>> Opening the font file in font forge also shows their glyphs
>
> You are confusing the display of a lone codepoint with what Font Forge
> does and what Emacs does in the "decomposition" display.  Those do not
> show the lone character, they show it with special characters before
> or after, to show how the character will look when combined with
> others.
>
> IOW, I think your expectations from how this character should be
> displayed as a lone character are incorrect.
>
> Can you tell why you want this to be displayed differently when it is
> a lone character?  AFAIU, the only meaningful display of this
> character is when it precedes numerals.

I believe there is a similar problen with composition of devanagari
stress accents with devanagari numerals: The devanagari numerals 1
(#x967) & 3 (#x969) can be combined with the svarita (#x951, incorrectly
named in unicode as DEVANAGARI STRESS SIGN UDATTA) and the anudatta
(#x952) to form the 4 different accents.

numeral one + anudatta + svarita
preceding anudatta + numeral three + anudatta + svarita
numeral one +  anudatta
numeral three + anudatta + svarita

e.g.  (#x969 #x951 #x952), when composed, the devanagari numeral 3
should have an anudatta ("line below") and svarita ("vertical line
above"), but it ends up rendering the numeral three, and two then two
thin empty vertical boxes. If the boxes render the stress accents I
can't see them, they just appear as boxes.

  name: DEVANAGARI STRESS SIGN ANUDATTA
          buffer code: #xE0 #xA5 #x92
              display: by this font (glyph code):
    ftcrhb:-IITB-Shobhika-regular-normal-normal-*-17-*-*-*-*-0-iso10646-1 (#x312)

Re: Not able to display \u110BD and \u110CD in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 2710 bytes --]

>
> I believe there is a similar problen with composition of devanagari
> stress accents with devanagari numerals


Does it even make sense for the stress accents to appear with the numerals?
Shouldn't they only appear with the consonants and vowels?
Please correct me if I am wrong.

On Thu, May 5, 2022 at 9:26 PM Madhu <enometh@meer.net> wrote:

> * Eli Zaretskii <837d779kgx.fsf @gnu.org> :
> Wrote on Sat, 30 Apr 2022 09:03:10 +0300:
>
> > Since this shows the "by this font" part, it means the character _is_
> > displayed according to the font's glyph for it, and according to
> > Emacs's rules for displaying such "format-control" characters.  So now
> > I don't think I understand why you say this character is not shown by
> > Emacs, when the above clearly says it is displayed.
> >
> >> In the character section the character is not displayed in emacs but
> >> displayed in firefox, but it is displayed in
> >> the decomposition section in emacs. I have attached the images
> >> Opening the font file in font forge also shows their glyphs
> >
> > You are confusing the display of a lone codepoint with what Font Forge
> > does and what Emacs does in the "decomposition" display.  Those do not
> > show the lone character, they show it with special characters before
> > or after, to show how the character will look when combined with
> > others.
> >
> > IOW, I think your expectations from how this character should be
> > displayed as a lone character are incorrect.
> >
> > Can you tell why you want this to be displayed differently when it is
> > a lone character?  AFAIU, the only meaningful display of this
> > character is when it precedes numerals.
>
> I believe there is a similar problen with composition of devanagari
> stress accents with devanagari numerals: The devanagari numerals 1
> (#x967) & 3 (#x969) can be combined with the svarita (#x951, incorrectly
> named in unicode as DEVANAGARI STRESS SIGN UDATTA) and the anudatta
> (#x952) to form the 4 different accents.
>
> numeral one + anudatta + svarita
> preceding anudatta + numeral three + anudatta + svarita
> numeral one +  anudatta
> numeral three + anudatta + svarita
>
> e.g.  (#x969 #x951 #x952), when composed, the devanagari numeral 3
> should have an anudatta ("line below") and svarita ("vertical line
> above"), but it ends up rendering the numeral three, and two then two
> thin empty vertical boxes. If the boxes render the stress accents I
> can't see them, they just appear as boxes.
>
>   name: DEVANAGARI STRESS SIGN ANUDATTA
>           buffer code: #xE0 #xA5 #x92
>               display: by this font (glyph code):
>     ftcrhb:-IITB-Shobhika-regular-normal-normal-*-17-*-*-*-*-0-iso10646-1
> (#x312)
>
>
>

[-- Attachment #2: Type: text/html, Size: 3563 bytes --]

Re: Not able to display \u110BD and \u110CD in Emacs

[Madhu]

*  Sameer Singh <CAOR1sLw5J_vrJHo9H8N=SS=tSidAvNytfUoXYT6B8HG1HpEHFg @mail.gmail.com> :
Wrote on Thu, 5 May 2022 21:39:29 +0530:
>> I believe there is a similar problen with composition of devanagari
>> stress accents with devanagari numerals
>
> Does it even make sense for the stress accents to appear with the numerals?
> Shouldn't they only appear with the consonants and vowels?
> Please correct me if I am wrong.

Can't speak for the unicode consortium, but I believe it is the practice
in marking certain specific accents in rig veda and atharva veda (which
I enumerated). check the table on the last page of Wikner's document at
http://www.evertype.com/standards/iso10646/pdf/vedic/Vedic_accents_doc.pdf
which shows the examples I wrote about.

(I think I've come across unicode text in the wild which used these, and
so assumed these are rendered "visually correctly" by other software)

Re: Not able to display \u110BD and \u110CD in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 1259 bytes --]

>check the table on the last page of Wikner's >document at
>http://www.evertype.com/standards/iso10646/pdf/vedic/Vedic_accents_doc.pdf
>which shows the examples I wrote about.

Got it. Thanks 👍

गुरु, 5 मई 2022, 10:26 pm को Madhu <enometh@meer.net> ने लिखा:

> *  Sameer Singh <CAOR1sLw5J_vrJHo9H8N=SS=tSidAvNytfUoXYT6B8HG1HpEHFg @
> mail.gmail.com> :
> Wrote on Thu, 5 May 2022 21:39:29 +0530:
> >> I believe there is a similar problen with composition of devanagari
> >> stress accents with devanagari numerals
> >
> > Does it even make sense for the stress accents to appear with the
> numerals?
> > Shouldn't they only appear with the consonants and vowels?
> > Please correct me if I am wrong.
>
> Can't speak for the unicode consortium, but I believe it is the practice
> in marking certain specific accents in rig veda and atharva veda (which
> I enumerated). check the table on the last page of Wikner's document at
> http://www.evertype.com/standards/iso10646/pdf/vedic/Vedic_accents_doc.pdf
> which shows the examples I wrote about.
>
> (I think I've come across unicode text in the wild which used these, and
> so assumed these are rendered "visually correctly" by other software)
>
>
>

[-- Attachment #2: Type: text/html, Size: 1995 bytes --]

Re: Not able to display \u110BD and \u110CD in Emacs

[Eli Zaretskii]

> From: Madhu <enometh@meer.net>
> Date: Thu, 05 May 2022 21:23:50 +0530
> 
> > Can you tell why you want this to be displayed differently when it is
> > a lone character?  AFAIU, the only meaningful display of this
> > character is when it precedes numerals.
> 
> I believe there is a similar problen with composition of devanagari
> stress accents with devanagari numerals:

AFAIU, it's definitely NOT the same problem.

> The devanagari numerals 1
> (#x967) & 3 (#x969) can be combined with the svarita (#x951, incorrectly
> named in unicode as DEVANAGARI STRESS SIGN UDATTA) and the anudatta
> (#x952) to form the 4 different accents.
> 
> numeral one + anudatta + svarita
> preceding anudatta + numeral three + anudatta + svarita
> numeral one +  anudatta
> numeral three + anudatta + svarita

I see no sign of such composition rules for Devanagai in
lisp/language/indian.el.  So it isn't a surprise that Emacs doesn't
display those sequences like you expect.

I will take a look at this when I have time, but please file a bug
report about this issue.  If you can include in the report the results
of running hb-view (from the HarfBuzz distribution) on the text
sequences you describe, that would help.

Thanks.

Re: Not able to display \u110BD and \u110CD in Emacs

[Madhu]

* Eli Zaretskii <835ymk2afp.fsf@gnu.org> :
Wrote on Thu, 05 May 2022 19:47:38 +0300:
>> From: Madhu <enometh@meer.net>
>> Date: Thu, 05 May 2022 21:23:50 +0530
>> > Can you tell why you want this to be displayed differently when it is
>> > a lone character?  AFAIU, the only meaningful display of this
>> > character is when it precedes numerals.
>> I believe there is a similar problen with composition of devanagari
>> stress accents with devanagari numerals:
>
> AFAIU, it's definitely NOT the same problem.

I'm sure you're right.

[bear with me: I downloaded NotoSansKaithi-Regular.ttf and tried to
replicate what Sameer had done: First I was not able to get hb-view to
produce the output shown in the first post: the #x110bc number sign
doesn't span the following numbers. Then After mutating the
composition-function-table as indicated, i had to set

(set-fontset-font "fontset-default" '(#x11080 . #x110cf)
		  (font-spec :name "Noto Sans Kaithi"
			     ;; :otf '(kthi nil (rphf))
			     :registry "iso10646-1" ))

is this incorrect?  if i add the :otf key then "find-font doesn't find
the font", but then i still see no span-composition, and only see the
thin white space with #x110bd (even after using this font for devanagri
numerals (#x966 . #x96f)) - so I haven't understood what the resolution
was]

>> The devanagari numerals 1
>> (#x967) & 3 (#x969) can be combined with the svarita (#x951, incorrectly
>> named in unicode as DEVANAGARI STRESS SIGN UDATTA) and the anudatta
>> (#x952) to form the 4 different accents.
>
> I see no sign of such composition rules for Devanagai in
> lisp/language/indian.el.  So it isn't a surprise that Emacs doesn't
> display those sequences like you expect.
>
> I will take a look at this when I have time, but please file a bug
> report about this issue.  If you can include in the report the results
> of running hb-view (from the HarfBuzz distribution) on the text
> sequences you describe, that would help.

I'll try to this by the following week.

Re: Not able to display \u110BD and \u110CD in Emacs

[Eli Zaretskii]

> From: Madhu <enometh@meer.net>
> Date: Fri, 06 May 2022 09:29:45 +0530
> 
> [bear with me: I downloaded NotoSansKaithi-Regular.ttf and tried to
> replicate what Sameer had done: First I was not able to get hb-view to
> produce the output shown in the first post:

I don't think I see any hb-view output in the first post; what did I
miss?

> the #x110bc number sign
> doesn't span the following numbers. Then After mutating the
> composition-function-table as indicated, i had to set
> 
> (set-fontset-font "fontset-default" '(#x11080 . #x110cf)
> 		  (font-spec :name "Noto Sans Kaithi"
> 			     ;; :otf '(kthi nil (rphf))
> 			     :registry "iso10646-1" ))
> 
> is this incorrect?

You should probably use 'prepend as the ADD argument of
set-fontset-font, but otherwise this looks OK to me.  (But does Kaithi
really need the rphf feature?)

> if i add the :otf key then "find-font doesn't find
> the font"

Maybe this font doesn't have some of these OTF features?

> > I will take a look at this when I have time, but please file a bug
> > report about this issue.  If you can include in the report the results
> > of running hb-view (from the HarfBuzz distribution) on the text
> > sequences you describe, that would help.
> 
> I'll try to this by the following week.

TIA

Re: Not able to display \u110BD and \u110CD in Emacs

[Eli Zaretskii]

> From: Madhu <enometh@meer.net>
> Date: Thu, 05 May 2022 21:23:50 +0530
> 
> The devanagari numerals 1 (#x967) & 3 (#x969) can be combined with
> the svarita (#x951, incorrectly named in unicode as DEVANAGARI
> STRESS SIGN UDATTA) and the anudatta (#x952) to form the 4 different
> accents.
> 
> numeral one + anudatta + svarita
> preceding anudatta + numeral three + anudatta + svarita
> numeral one +  anudatta
> numeral three + anudatta + svarita
> 
> e.g.  (#x969 #x951 #x952), when composed, the devanagari numeral 3
> should have an anudatta ("line below") and svarita ("vertical line
> above")

Where are these rules documented?  I don't see them in the latest
Unicode Standard 14.0.

Re: Not able to display \u110BD and \u110CD in Emacs

[Madhu]

*  Eli Zaretskii <eliz@gnu.org> <83h761zv0w.fsf@gnu.org>
Wrote on Sat, 07 May 2022 10:01:03 +0300

>> From: Madhu <enometh@meer.net>
>> Date: Thu, 05 May 2022 21:23:50 +0530
>>
>> The devanagari numerals 1 (#x967) & 3 (#x969) can be combined with
>> the svarita (#x951, incorrectly named in unicode as DEVANAGARI
>> STRESS SIGN UDATTA) and the anudatta (#x952) to form the 4 different
>> accents.
>>
>> numeral one + anudatta + svarita
>> preceding anudatta + numeral three + anudatta + svarita
>> numeral one +  anudatta
>> numeral three + anudatta + svarita
>>
>> e.g.  (#x969 #x951 #x952), when composed, the devanagari numeral 3
>> should have an anudatta ("line below") and svarita ("vertical line
>> above")
>
> Where are these rules documented?  I don't see them in the latest
> Unicode Standard 14.0.

I have no idea.  I assumed the rulesfollow from (some defined) rules
for composition, but I stopped dealing with unicode for sanskrit (
vedic accents) back in 2002 when I saw the direction it was going in.

Re: Not able to display \u110BD and \u110CD in Emacs

[Eli Zaretskii]

> Date: Sat, 07 May 2022 19:49:20 +0530 (IST)
> Cc: emacs-devel@gnu.org
> From: Madhu <enometh@meer.net>
> 
> *  Eli Zaretskii <eliz@gnu.org> <83h761zv0w.fsf@gnu.org>
> Wrote on Sat, 07 May 2022 10:01:03 +0300
> 
> >> From: Madhu <enometh@meer.net>
> >> Date: Thu, 05 May 2022 21:23:50 +0530
> >>
> >> The devanagari numerals 1 (#x967) & 3 (#x969) can be combined with
> >> the svarita (#x951, incorrectly named in unicode as DEVANAGARI
> >> STRESS SIGN UDATTA) and the anudatta (#x952) to form the 4 different
> >> accents.
> >>
> >> numeral one + anudatta + svarita
> >> preceding anudatta + numeral three + anudatta + svarita
> >> numeral one +  anudatta
> >> numeral three + anudatta + svarita
> >>
> >> e.g.  (#x969 #x951 #x952), when composed, the devanagari numeral 3
> >> should have an anudatta ("line below") and svarita ("vertical line
> >> above")
> >
> > Where are these rules documented?  I don't see them in the latest
> > Unicode Standard 14.0.
> 
> I have no idea.  I assumed the rulesfollow from (some defined) rules
> for composition, but I stopped dealing with unicode for sanskrit (
> vedic accents) back in 2002 when I saw the direction it was going in.

It is strange that I seem to be unable to find such rules anywhere on
the Internet, not just in the Unicode Standard text.

Re: Not able to display \u110BD and \u110CD in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1.1: Type: text/plain, Size: 1920 bytes --]

>
> It is strange that I seem to be unable to find such rules anywhere on
> the Internet, not just in the Unicode Standard text.

I was not able to find the rules, but they definitely exist.

http://वेद.com/en/rigveda/1/38/3 <http://xn--j2b3a4c.com/en/rigveda/1/38/3>
http://वेद.com/en/rigveda/1/34/9 <http://xn--j2b3a4c.com/en/rigveda/1/34/9>
http://वेद.com/en/rigveda/1/48/6 <http://xn--j2b3a4c.com/en/rigveda/1/48/6>

On Sat, May 7, 2022 at 7:53 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > Date: Sat, 07 May 2022 19:49:20 +0530 (IST)
> > Cc: emacs-devel@gnu.org
> > From: Madhu <enometh@meer.net>
> >
> > *  Eli Zaretskii <eliz@gnu.org> <83h761zv0w.fsf@gnu.org>
> > Wrote on Sat, 07 May 2022 10:01:03 +0300
> >
> > >> From: Madhu <enometh@meer.net>
> > >> Date: Thu, 05 May 2022 21:23:50 +0530
> > >>
> > >> The devanagari numerals 1 (#x967) & 3 (#x969) can be combined with
> > >> the svarita (#x951, incorrectly named in unicode as DEVANAGARI
> > >> STRESS SIGN UDATTA) and the anudatta (#x952) to form the 4 different
> > >> accents.
> > >>
> > >> numeral one + anudatta + svarita
> > >> preceding anudatta + numeral three + anudatta + svarita
> > >> numeral one +  anudatta
> > >> numeral three + anudatta + svarita
> > >>
> > >> e.g.  (#x969 #x951 #x952), when composed, the devanagari numeral 3
> > >> should have an anudatta ("line below") and svarita ("vertical line
> > >> above")
> > >
> > > Where are these rules documented?  I don't see them in the latest
> > > Unicode Standard 14.0.
> >
> > I have no idea.  I assumed the rulesfollow from (some defined) rules
> > for composition, but I stopped dealing with unicode for sanskrit (
> > vedic accents) back in 2002 when I saw the direction it was going in.
>
> It is strange that I seem to be unable to find such rules anywhere on
> the Internet, not just in the Unicode Standard text.
>
>

[-- Attachment #1.2: Type: text/html, Size: 3000 bytes --]

[-- Attachment #2: rig-veda1.png --]
[-- Type: image/png, Size: 32198 bytes --]

[-- Attachment #3: rig-veda3.png --]
[-- Type: image/png, Size: 16821 bytes --]

Re: Not able to display \u110BD and \u110CD in Emacs

[Eli Zaretskii]

> From: समीर सिंह Sameer Singh <lumarzeli30@gmail.com>
> Date: Sat, 7 May 2022 19:59:10 +0530
> Cc: Madhu <enometh@meer.net>, emacs-devel@gnu.org
> 
>  It is strange that I seem to be unable to find such rules anywhere on
>  the Internet, not just in the Unicode Standard text.
> 
> I was not able to find the rules, but they definitely exist.
> 
> http://वेद.com/en/rigveda/1/38/3
> http://वेद.com/en/rigveda/1/34/9
> http://वेद.com/en/rigveda/1/48/6 

Very well, I've now added those rules.

Re: Not able to display \u110BD and \u110CD in Emacs

[Madhu]

*  Eli Zaretskii <eliz@gnu.org> <83mtftxugf.fsf@gnu.org>
Wrote on Sat, 07 May 2022 17:56:16 +0300
>> http://वेद.com/en/rigveda/1/48/6
>
> Very well, I've now added those rules.

Thanks, I'll skip on the promised bugreport then (unless I notice
something when I update :)

[BTW, on a subthread I mentioned I couldn't get hb-view to produce the
output shown on the first post: the post is
https://lists.gnu.org/archive/html/emacs-devel/2022-04/msg01297.html
and the displayed image is at
https://lists.gnu.org/archive/html/emacs-devel/2022-04/png4ClBZc1zV4.png
]

Re: Not able to display \u110BD and \u110CD in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 1161 bytes --]

> [BTW, on a subthread I mentioned I couldn't get hb-view to produce the
> output shown on the first post

Can you reproduce these steps?
1. hb-view /usr/share/fonts/noto/NotoSansKaithi-Regular.ttf
--output-file=hb-view.png (Press Enter)
2, Then type: 𑂽०१२३४५६७८९ (Press Enter)
3. Then type: 𑃍०१२३४५६७८९ (Press Enter)
4. Press Ctrl-D

This should produce the output as needed.

On Sat, May 7, 2022 at 9:17 PM Madhu <enometh@meer.net> wrote:

> *  Eli Zaretskii <eliz@gnu.org> <83mtftxugf.fsf@gnu.org>
> Wrote on Sat, 07 May 2022 17:56:16 +0300
> >> http://वेद.com/en/rigveda/1/48/6
> <http://xn--j2b3a4c.com/en/rigveda/1/48/6>
> >
> > Very well, I've now added those rules.
>
> Thanks, I'll skip on the promised bugreport then (unless I notice
> something when I update :)
>
> [BTW, on a subthread I mentioned I couldn't get hb-view to produce the
> output shown on the first post: the post is
> https://lists.gnu.org/archive/html/emacs-devel/2022-04/msg01297.html
> and the displayed image is at
> https://lists.gnu.org/archive/html/emacs-devel/2022-04/png4ClBZc1zV4.png
> ]
>

[-- Attachment #2: Type: text/html, Size: 1964 bytes --]

Re: Not able to display \u110BD and \u110CD in Emacs

[Madhu]

[-- Attachment #1: Type: Text/Plain, Size: 706 bytes --]

*  Sameer Singh <lumarzeli30@gmail.com> <CAOR1sLyB1BEkgzh=HwE5qinm5h79Z0zNx6rxQUrhh3aimfwmyQ@mail.gmail.com>
Wrote on Sat, 7 May 2022 21:31:51 +0530

>> [BTW, on a subthread I mentioned I couldn't get hb-view to produce the
>> output shown on the first post
>
> Can you reproduce these steps?
> 1. hb-view /usr/share/fonts/noto/NotoSansKaithi-Regular.ttf
> --output-file=hb-view.png (Press Enter)

Unfortunately not. My system has harfbuzz 3.0.0 and the font file I
acquired is probably from the 2017 release:
87260 Mar 24  2020 NotoSansKaithi-Regular.ttf
My output is linear.
hb-view --font-file=$HOME/.fonts/NotoSansKaithi-Regular.ttf --text-file=12.txt --output-file 12.png

I'll try updating hb later.

[-- Attachment #2: 12.txt --]
[-- Type: Text/Plain, Size: 28 bytes --]

𑂽१०८
𑂼१०८

[-- Attachment #3: 12.png --]
[-- Type: Image/Png, Size: 17196 bytes --]

Re: Not able to display \u110BD and \u110CD in Emacs

[समीर सिंह Sameer Singh]

[-- Attachment #1: Type: text/plain, Size: 926 bytes --]

I have harfbuzz 4.2.1 so yeah maybe the old version is the problem.

शनि, 7 मई 2022, 10:14 pm को Madhu <enometh@meer.net> ने लिखा:

> *  Sameer Singh <lumarzeli30@gmail.com> <CAOR1sLyB1BEkgzh=
> HwE5qinm5h79Z0zNx6rxQUrhh3aimfwmyQ@mail.gmail.com>
> Wrote on Sat, 7 May 2022 21:31:51 +0530
>
> >> [BTW, on a subthread I mentioned I couldn't get hb-view to produce the
> >> output shown on the first post
> >
> > Can you reproduce these steps?
> > 1. hb-view /usr/share/fonts/noto/NotoSansKaithi-Regular.ttf
> > --output-file=hb-view.png (Press Enter)
>
> Unfortunately not. My system has harfbuzz 3.0.0 and the font file I
> acquired is probably from the 2017 release:
> 87260 Mar 24  2020 NotoSansKaithi-Regular.ttf
> My output is linear.
> hb-view --font-file=$HOME/.fonts/NotoSansKaithi-Regular.ttf
> --text-file=12.txt --output-file 12.png
>
> I'll try updating hb later.
>

[-- Attachment #2: Type: text/html, Size: 1433 bytes --]

Re: Not able to display \u110BD and \u110CD in Emacs

[Madhu]

*  Sameer Singh <CAOR1sLwk4opLov7O95XnF56iiF7Kh88EuDsxfB6QdRqisLzTHA@mail.gmail.com>
Wrote on Sat, 7 May 2022 22:24:04 +0530

> I have harfbuzz 4.2.1 so yeah maybe the old version is the problem.

I updated harfbuzz to 4.2.1 (and upgraded freetype after that) and the
output is byte-identical to the earlier version 3.0.0.  I'm not sure
what I'm missing.

a few more notes to self: The Khaiti number signs also compose only
linearly in emacs and only when I use the same font for numerals, say:

(set-fontset-font "fontset-default" '(#x966 . #x96f)  "NotoSans Kaithi"

Otherwise I can't see glyph for #x110bd whic precedes a number, I only
see a thin vertical empty box rendered.

Likewise for the newly installed composition rules, I can only see
them with the Siddhanta font. All the other fonts that display
devanagari seem to show the thin empty vertical box to render the
stress accents.


>> *  Sameer Singh <lumarzeli30@gmail.com> <CAOR1sLyB1BEkgzh=
>> HwE5qinm5h79Z0zNx6rxQUrhh3aimfwmyQ@mail.gmail.com>
>> Wrote on Sat, 7 May 2022 21:31:51 +0530
>>
>> >> [BTW, on a subthread I mentioned I couldn't get hb-view to produce the
>> >> output shown on the first post
>> >
>> > Can you reproduce these steps?
>> > 1. hb-view /usr/share/fonts/noto/NotoSansKaithi-Regular.ttf
>> > --output-file=hb-view.png (Press Enter)
>>
>> Unfortunately not. My system has harfbuzz 3.0.0 and the font file I
>> acquired is probably from the 2017 release:
>> 87260 Mar 24  2020 NotoSansKaithi-Regular.ttf
>> My output is linear.
>> hb-view --font-file=$HOME/.fonts/NotoSansKaithi-Regular.ttf
>> --text-file=12.txt --output-file 12.png
>>
>> I'll try updating hb later.

Re: Not able to display \u110BD and \u110CD in Emacs

[Eli Zaretskii]

> Date: Sun, 08 May 2022 22:55:53 +0530 (IST)
> Cc: eliz@gnu.org, emacs-devel@gnu.org
> From: Madhu <enometh@meer.net>
> 
> I updated harfbuzz to 4.2.1 (and upgraded freetype after that) and the
> output is byte-identical to the earlier version 3.0.0.  I'm not sure
> what I'm missing.

And I don't understand what problems you see.  I don't think you
described them in enough detail.

> a few more notes to self: The Khaiti number signs also compose only
> linearly in emacs

What do you mean by "compose linearly"?

> and only when I use the same font for numerals, say:

Emacs can only compose characters if all of the characters of a
composable sequence are supported by the same font.  It is
practically impossible to compose character glyphs from different
fonts, so Emacs doesn't.  If that's what you see, then it's the
intended behavior.

> Likewise for the newly installed composition rules, I can only see
> them with the Siddhanta font. All the other fonts that display
> devanagari seem to show the thin empty vertical box to render the
> stress accents.

If those "other fonts" cause the characters in a sequence not to be
supported by a single font, then again, this is the intended behavior.

Re: Not able to display \u110BD and \u110CD in Emacs

[Madhu]

*  Eli Zaretskii <eliz@gnu.org> <83h760udua.fsf@gnu.org>
Wrote on Sun, 08 May 2022 20:35:41 +0300
>> Date: Sun, 08 May 2022 22:55:53 +0530 (IST)
>> From: Madhu <enometh@meer.net>
>>
>> I updated harfbuzz to 4.2.1 (and upgraded freetype after that) and the
>> output is byte-identical to the earlier version 3.0.0.  I'm not sure
>> what I'm missing.
>
> And I don't understand what problems you see.  I don't think you
> described them in enough detail.

I've explained it in the messages upthread with the
references. briefly Sameer exhibited this picture
https://lists.gnu.org/archive/html/emacs-devel/2022-04/png4ClBZc1zV4.png
in his first message on this thread
https://lists.gnu.org/archive/html/emacs-devel/2022-04/msg01297.html

The composition shows that the first character spans the numerals.

I tried to reproduce it. I showed my work in this message:
https://lists.gnu.org/archive/html/emacs-devel/2022-05/msg00341.html
where I showed the shell command to invoke hb-view, and attached the
text file 12.txt and the output 12.png. (the png attachment is not
visible on lists.gnu.org - what happened?), message-id was
<20220507.221454.1080380889007217525.enometh@meer.net>
>
>> a few more notes to self: The Khaiti number signs also compose only
>> linearly in emacs
>
> What do you mean by "compose linearly"?

the 12.png file would have show the composition is "linear" in that
#x110bd and #x110bc glyphs do not span the following digits like in
the png url which Sameer posted.

>> and only when I use the same font for numerals, say:
>
> Emacs can only compose characters if all of the characters of a
> composable sequence are supported by the same font.  It is
> practically impossible to compose character glyphs from different
> fonts, so Emacs doesn't.  If that's what you see, then it's the
> intended behavior.

Yes, that is what I believe I'm seeing.

>> Likewise for the newly installed composition rules, I can only see
>> them with the Siddhanta font. All the other fonts that display
>> devanagari seem to show the thin empty vertical box to render the
>> stress accents.
>
> If those "other fonts" cause the characters in a sequence not to be
> supported by a single font, then again, this is the intended behavior.

Thanks

Re: Not able to display \u110BD and \u110CD in Emacs

[Eli Zaretskii]

> Date: Sun, 08 May 2022 23:51:57 +0530 (IST)
> Cc: lumarzeli30@gmail.com, emacs-devel@gnu.org
> From: Madhu <enometh@meer.net>
> 
> I've explained it in the messages upthread with the
> references. briefly Sameer exhibited this picture
> https://lists.gnu.org/archive/html/emacs-devel/2022-04/png4ClBZc1zV4.png
> in his first message on this thread
> https://lists.gnu.org/archive/html/emacs-devel/2022-04/msg01297.html
> 
> The composition shows that the first character spans the numerals.
> 
> I tried to reproduce it. I showed my work in this message:
> https://lists.gnu.org/archive/html/emacs-devel/2022-05/msg00341.html
> where I showed the shell command to invoke hb-view, and attached the
> text file 12.txt and the output 12.png. (the png attachment is not
> visible on lists.gnu.org - what happened?), message-id was
> <20220507.221454.1080380889007217525.enometh@meer.net>
> >
> >> a few more notes to self: The Khaiti number signs also compose only
> >> linearly in emacs
> >
> > What do you mean by "compose linearly"?
> 
> the 12.png file would have show the composition is "linear" in that
> #x110bd and #x110bc glyphs do not span the following digits like in
> the png url which Sameer posted.
> 
> >> and only when I use the same font for numerals, say:
> >
> > Emacs can only compose characters if all of the characters of a
> > composable sequence are supported by the same font.  It is
> > practically impossible to compose character glyphs from different
> > fonts, so Emacs doesn't.  If that's what you see, then it's the
> > intended behavior.
> 
> Yes, that is what I believe I'm seeing.
> 
> >> Likewise for the newly installed composition rules, I can only see
> >> them with the Siddhanta font. All the other fonts that display
> >> devanagari seem to show the thin empty vertical box to render the
> >> stress accents.
> >
> > If those "other fonts" cause the characters in a sequence not to be
> > supported by a single font, then again, this is the intended behavior.
> 
> Thanks

So are we clear about what should and does happen, or are there still
issues to be looked into in this respect?

Re: Not able to display \u110BD and \u110CD in Emacs

[Madhu]

*  Eli Zaretskii <eliz@gnu.org> <83ee13voin.fsf@gnu.org>
Wrote on Sun, 08 May 2022 21:59:44 +0300

> So are we clear about what should and does happen, or are there
> still issues to be looked into in this respect?

What's not clear is the behaviour of 'arfbuzz, which on my system does
not match what is indicated in the discussion.  This possibly affects
how emacs renders the text but it wouldn't be an emacs problem.

Re: Not able to display \u110BD and \u110CD in Emacs

[Eli Zaretskii]

> Date: Mon, 09 May 2022 07:36:57 +0530 (IST)
> Cc: lumarzeli30@gmail.com, emacs-devel@gnu.org
> From: Madhu <enometh@meer.net>
> 
> *  Eli Zaretskii <eliz@gnu.org> <83ee13voin.fsf@gnu.org>
> Wrote on Sun, 08 May 2022 21:59:44 +0300
> 
> > So are we clear about what should and does happen, or are there
> > still issues to be looked into in this respect?
> 
> What's not clear is the behaviour of 'arfbuzz, which on my system does
> not match what is indicated in the discussion.  This possibly affects
> how emacs renders the text but it wouldn't be an emacs problem.

Then please describe the behavior you expect and what you actually
see.  Given the missing information and images in your previous posts,
I think I need a clear description of the issue to be of any help
here.

Thanks.

Re: Not able to display \u110BD and \u110CD in Emacs

[Madhu]

*  Eli Zaretskii <eliz@gnu.org> <83pmkpxw15.fsf@gnu.org>
Wrote on Sat, 07 May 2022 17:22:14 +0300
>> Date: Sat, 07 May 2022 19:49:20 +0530 (IST)
>> Cc: emacs-devel@gnu.org
>> From: Madhu <enometh@meer.net>
>> *  Eli Zaretskii <eliz@gnu.org> <83h761zv0w.fsf@gnu.org>
>> Wrote on Sat, 07 May 2022 10:01:03 +0300
>> >> From: Madhu <enometh@meer.net>
>> >> Date: Thu, 05 May 2022 21:23:50 +0530
>> >> e.g.  (#x969 #x951 #x952), when composed, the devanagari numeral 3
>> >> should have an anudatta ("line below") and svarita ("vertical line
>> >> above")
>> >
>> > Where are these rules documented?  I don't see them in the latest
>> > Unicode Standard 14.0.
>>
>> I have no idea.  I assumed the rulesfollow from (some defined) rules
>> for composition, but I stopped dealing with unicode for sanskrit (
>> vedic accents) back in 2002 when I saw the direction it was going in.
>
> It is strange that I seem to be unable to find such rules anywhere on
> the Internet, not just in the Unicode Standard text.

The Wikner I posted upthread mentions the tradition concisely in on
p.3/6 under (a)Rgveda:

http://www.evertype.com/standards/iso10646/pdf/vedic/Vedic_accents_doc.pdf

   (a) R.gveda has udattaa unmarked the svarita (jatya or suddha) is
   marked with a vertical line above the syllable (e.g. XX), but the
   kampa is indicated by a numeral 1 following the vowel if it is
   short, or the numeral 3 if it is long, and in both cases there is a
   vertical line above the numeral and a horizontal line below it in
   the case of the long kampa vowel the syllable usually also has a
   horizontal line below it (e.g. XX ).

The canonical work (this version not searchable unfortunately)
https://archive.org/detauls/vedicgrammar00macduoft/vedicgrammar00macduoft
- pdf p.468, p.450 talks about the accents 1 and 3 numerals used in
this way.

I've seen a number of pdfs introducing vedic accents in the wild that
document this. Maybe I'll try to make a bibliography