bug#21260: 23.2; Devanagari windows 10

bug#21260: 23.2; Devanagari windows 10

[Jim Funderburk]

*** E-Mail body has been placed on clipboard, please paste it h
DESCRIPTION OF THE DEVANAGARI WINDOWS 10 PROBLEM
I recently updated to Windows 10.  Now, I have problems viewing
text files which contain utf-8 encoded text representing Devanagari.
The text renders as boxes.
A small sample file is in this dropbox link.
https://www.dropbox.com/s/7uedn6dgwf1p6xj/devanagari_example.txt?dl=0

This file views fine (using Lucida Console, or Arial fonts) under
Windows Vista or Windows 7 , with this same version of Emacs.
I've also tried a newer version 24.x.y version of Emacs on Windows 10,
and have the same problem.
This file also looks fine in Windows 10 using Notepad++ or even lowly
Notepad,  and (with slight adjustment of html tags) in Chrome browser.

My email is funderburk1@verizon.net.  I hope you'll let me know if this
can be fixed, since I have used Emacs for 20+ years, and hope to use it
in the future
Thanks.
Jim Funderburk



In GNU Emacs 23.2.1 (i386-mingw-nt6.2.9200)
  of 2010-05-08 on G41R2F1
Windowing system distributor `Microsoft Corp.', version 6.2.9200
configured using `configure --with-gcc (3.4) --no-opt --cflags 
-Ic:/xpm/include'

Important settings:
   value of $LC_ALL: nil
   value of $LC_COLLATE: nil
   value of $LC_CTYPE: nil
   value of $LC_MESSAGES: nil
   value of $LC_MONETARY: nil
   value of $LC_NUMERIC: nil
   value of $LC_TIME: nil
   value of $LANG: ENU
   value of $XMODIFIERS: nil
   locale-coding-system: cp1252
   default enable-multibyte-characters: t

Major mode: Text

Minor modes in effect:
   tooltip-mode: t
   mouse-wheel-mode: t
   tool-bar-mode: t
   menu-bar-mode: t
   file-name-shadow-mode: t
   global-font-lock-mode: t
   font-lock-mode: t
   blink-cursor-mode: t
   auto-encryption-mode: t
   auto-compression-mode: t
   line-number-mode: t
   transient-mark-mode: t

Recent input:
<help-echo> <help-echo> <help-echo> <help-echo> <down-mouse-1>
<mouse-1> <down-mouse-1> <mouse-1> <down-mouse-1> <mouse-1>
M-x r e p o r t - <tab> <return>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
None found.

Features:
(shadow sort mail-extr message ecomplete rfc822 mml easymenu mml-sec
password-cache mm-decode mm-bodies mm-encode mailcap mail-parse rfc2231
rfc2047 rfc2045 qp ietf-drums mailabbrev nnheader gnus-util netrc
time-date mm-util mail-prsvr gmm-utils wid-edit mailheader canlock sha1
hex-util hashcash mail-utils emacsbug tooltip ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 disp-table ls-lisp w32-win w32-vars
tool-bar dnd fontset image fringe lisp-mode register page menu-bar
rfn-eshadow timer select scroll-bar mldrag mouse jit-lock font-lock
syntax facemenu font-core frame cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev loaddefs button
minibuffer faces cus-face files text-properties overlay md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote make-network-process multi-tty emacs)
ere! ***

bug#21260: 23.2; Devanagari windows 10

[Eli Zaretskii]

> From: Jim Funderburk <funderburk1@verizon.net>
> Date: Fri, 14 Aug 2015 16:14:58 -0400
> 
> I recently updated to Windows 10.  Now, I have problems viewing
> text files which contain utf-8 encoded text representing Devanagari.
> The text renders as boxes.
> A small sample file is in this dropbox link.
> https://www.dropbox.com/s/7uedn6dgwf1p6xj/devanagari_example.txt?dl=0

I don't see boxes there, I see what looks like Devanagari characters.
(I don't read any of the languages that use the Devanagari script.)

What do you see if you type "C-h H" in a session started with
"emacs -Q"?  Can you post an image showing what you see?

> This file views fine (using Lucida Console, or Arial fonts) under
> Windows Vista or Windows 7 , with this same version of Emacs.
> I've also tried a newer version 24.x.y version of Emacs on Windows 10,
> and have the same problem.
> This file also looks fine in Windows 10 using Notepad++ or even lowly
> Notepad,  and (with slight adjustment of html tags) in Chrome browser.
> 
> My email is funderburk1@verizon.net.  I hope you'll let me know if this
> can be fixed, since I have used Emacs for 20+ years, and hope to use it
> in the future

I don't have access to Windows 10, and cannot debug this (and don't
see the problem in the sample you posted anyway).  I hope someone else
will be able to tell what is wrong there.

Sorry.

bug#21260: 23.2; Devanagari windows 10

[Eli Zaretskii]

> From: Jim Funderburk <funderburk1@verizon.net>
> Date: Fri, 14 Aug 2015 16:14:58 -0400
> 
> I recently updated to Windows 10.  Now, I have problems viewing
> text files which contain utf-8 encoded text representing Devanagari.
> The text renders as boxes.
> A small sample file is in this dropbox link.
> https://www.dropbox.com/s/7uedn6dgwf1p6xj/devanagari_example.txt?dl=0

I don't see boxes there, I see what looks like Devanagari characters.
(I don't read any of the languages that use the Devanagari script.)

What do you see if you type "C-h H" in a session started with
"emacs -Q"?  Can you post an image showing what you see?

Also, what font does Emacs try using for this script on Windows 10?

> This file views fine (using Lucida Console, or Arial fonts) under
> Windows Vista or Windows 7 , with this same version of Emacs.
> I've also tried a newer version 24.x.y version of Emacs on Windows 10,
> and have the same problem.
> This file also looks fine in Windows 10 using Notepad++ or even lowly
> Notepad,  and (with slight adjustment of html tags) in Chrome browser.
> 
> My email is funderburk1@verizon.net.  I hope you'll let me know if this
> can be fixed, since I have used Emacs for 20+ years, and hope to use it
> in the future

I don't have access to Windows 10, and cannot debug this (and don't
see the problem in the sample you posted anyway).  I hope someone else
will be able to tell what is wrong there.

Sorry.

bug#21260: 23.2; Devanagari windows 10

[Jim Funderburk]

Hi, Eli -
   Here is another dropbox link with further info you requested:
https://www.dropbox.com/s/3er09sc7jc8m5kc/emacs-devanagari.zip?dl=0
   There are two images:
   1. emacs-devanagari.png    This is a screen shot of me opening up the 
file
      in Emacs via  command line,  and of what the file looks like on my 
system.
      It looks like all boxes.
   2. emacs-Ctl-h-H.png     This is screen shot of what emacs shows 
after Ctl-h-H command.
        As you can see, the South Asian language shows up in boxes, 
also  'Hindi' and 'Kannada'
        Also  (maybe outside of the image)  Tamil and Telugu - all South 
Asian languages.
   You asked what font -  when I do menu Options/Set Default Font, it 
shows as Courier New.
       I have tried changing to Arial, Lucida Sans Unicode, Consolas  
with no improvement.
    Note:  I thought maybe  this is due to a deficiency in the fonts in 
Windows 10.  However,
                When I open the file in Notepad,   the Devanagari 
displays just fine;  Notepad
                 shows that it is using Consolas Font.
    Note2:  Also, I have a Windows Vista system and this displays fine 
in the same Emacs 23 on that system.
    Note3:  ALso, I tried Emacs 24 on Windows 10, and the same problem 
occurs.

    So, As of now, it seems like there is a bug in displaying South 
Asian Unicode text in Emacs on Windows 10.

THanks for looking at it.   Let me know if I can do anything more to 
help the issue to get resolved.

Regards,
  Jim Funderburk

On 8/14/2015 4:43 PM, Eli Zaretskii wrote:
>> From: Jim Funderburk <funderburk1@verizon.net>
>> Date: Fri, 14 Aug 2015 16:14:58 -0400
>>
>> I recently updated to Windows 10.  Now, I have problems viewing
>> text files which contain utf-8 encoded text representing Devanagari.
>> The text renders as boxes.
>> A small sample file is in this dropbox link.
>> https://www.dropbox.com/s/7uedn6dgwf1p6xj/devanagari_example.txt?dl=0
> I don't see boxes there, I see what looks like Devanagari characters.
> (I don't read any of the languages that use the Devanagari script.)
>
> What do you see if you type "C-h H" in a session started with
> "emacs -Q"?  Can you post an image showing what you see?
>
> Also, what font does Emacs try using for this script on Windows 10?
>
>> This file views fine (using Lucida Console, or Arial fonts) under
>> Windows Vista or Windows 7 , with this same version of Emacs.
>> I've also tried a newer version 24.x.y version of Emacs on Windows 10,
>> and have the same problem.
>> This file also looks fine in Windows 10 using Notepad++ or even lowly
>> Notepad,  and (with slight adjustment of html tags) in Chrome browser.
>>
>> My email is funderburk1@verizon.net.  I hope you'll let me know if this
>> can be fixed, since I have used Emacs for 20+ years, and hope to use it
>> in the future
> I don't have access to Windows 10, and cannot debug this (and don't
> see the problem in the sample you posted anyway).  I hope someone else
> will be able to tell what is wrong there.
>
> Sorry.
>

bug#21260: 23.2; Devanagari windows 10

[Eli Zaretskii]

> Cc: 21260@debbugs.gnu.org
> From: Jim Funderburk <funderburk1@verizon.net>
> Date: Sat, 15 Aug 2015 15:55:23 -0400
> 
>    Here is another dropbox link with further info you requested:
> https://www.dropbox.com/s/3er09sc7jc8m5kc/emacs-devanagari.zip?dl=0
>    There are two images:
>    1. emacs-devanagari.png    This is a screen shot of me opening up the 
> file
>       in Emacs via  command line,  and of what the file looks like on my 
> system.
>       It looks like all boxes.
>    2. emacs-Ctl-h-H.png     This is screen shot of what emacs shows 
> after Ctl-h-H command.
>         As you can see, the South Asian language shows up in boxes, 
> also  'Hindi' and 'Kannada'
>         Also  (maybe outside of the image)  Tamil and Telugu - all South 
> Asian languages.
>    You asked what font -  when I do menu Options/Set Default Font, it 
> shows as Courier New.
>        I have tried changing to Arial, Lucida Sans Unicode, Consolas  
> with no improvement.
>     Note:  I thought maybe  this is due to a deficiency in the fonts in 
> Windows 10.  However,
>                 When I open the file in Notepad,   the Devanagari 
> displays just fine;  Notepad
>                  shows that it is using Consolas Font.
>     Note2:  Also, I have a Windows Vista system and this displays fine 
> in the same Emacs 23 on that system.
>     Note3:  ALso, I tried Emacs 24 on Windows 10, and the same problem 
> occurs.
> 
>     So, As of now, it seems like there is a bug in displaying South 
> Asian Unicode text in Emacs on Windows 10.

Does this work better?

  emacs -Q -xrm Emacs.fontBackend:gdi

bug#21260: 23.2; Devanagari windows 10

[Eli Zaretskii]

> Date: Sun, 16 Aug 2015 05:40:59 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 21260@debbugs.gnu.org
> 
> Does this work better?
> 
>   emacs -Q -xrm Emacs.fontBackend:gdi

Actually, it looks like Emacs already uses the GDI backend, for some
reason.  Could you please verify that?  Go to any character that is
displayed (not as an empty box), and type "C-u C-x =".  If the buffer
that pops up says "gdi:-SOME-FONT-SPECIFICATION", then Emacs is
falling back on the GDI backend, which is incapable of displaying
complex scripts such as Devanagari.

If this is what happens, please see if you have usp10.dll on your
machine somewhere where Emacs can find it.

bug#21260: 23.2; Devanagari windows 10

[Eli Zaretskii]

[Please keep the bug address on the CC list.]

> Date: Sun, 16 Aug 2015 16:21:53 -0400
> From: Jim Funderburk <funderburk1@verizon.net>
> 
>    1.  When I open emacs 'normally'  (c:\emacs-23.2\bin\emacs.exe), type 
> some text (in *scratch* buffer)
>          'blah blah', and position the insertion point at the first 'a', 
> and then type "C-u C-x =",
>          here's what shows in the *Help* buffer:
>          character: a (97, #o141, #x61)
> preferred charset: ascii (ASCII (ISO646 IRV))
>         code point: 0x61
>             syntax: w     which means: word
>           category: .:Base, a:ASCII, l:Latin, r:Roman
>        buffer code: #x61
>          file code: #x61 (encoded by coding system iso-latin-1-dos)
>            display: by this font (glyph code)
>      uniscribe:-outline-Courier New-normal-normal-normal-mono-13-*-*-*-c-*-iso8859-1 (#x44)

OK, so Emacs does try to use Uniscribe on that system.

> 2.  Regarding usp10.dll -  I searched for it in the c:\Windows 
> directory, and found it in 4 places.
>       One of those was in System32 folder.  I copied usp10.dll to the 
> c:\emacs-23.2\bin folder, where emacs.exe resides.

The one in System32 is the wrong one: it's a 64-bit DLL, whereas Emacs
is a 32-bit executable, it should use the one in C:/Windows/SysWOW64
instead.  Please remove the DLL you put near emacs.exe, as it could
get in the way as we continue digging into this problem.

Can you show the full list of all the different usp10.dll files you
have there, including their size and time stamp?  Also, could you
please use some program like Process Explorer (from SysInternals) to
find out which one of these DLLs Emacs actually loads?

Next, there's the question with the fonts you have there.  On Windows
XP and Windows 7, Emacs uses the Mangal font to display the Hindi
script; on Windows 8.1 it uses Kokila instead.  Do you have any of
these fonts on your system?  If so, could you please show the OpenType
properties of these fonts, in particular the scripts they support and
the features they support for each script?  One program that can show
this information is FontTesterPlus, which you should be able to
download and install (I have version 1.4).

I'm sorry to ask you to do all this, but I have no access to Windows
10, and I see no such problems on all other versions through 8.1.

> 2a. In the "C-U C-X =" output of (1) above,  the presence of 'uniscribe' 
> makes me think that usp10.dll is being used already,

Yes, it is.

>            display: no font available
> 
> Character code properties: customize what to show
>    name: DEVANAGARI DOUBLE DANDA
>    general-category: Po (Punctuation, Other)
> 
> NOTE: It seems to be analyzing the character properly ---- But  I wonder 
> why it shows 'display: no font available'.

That's the crux of your problem: for some reason, Emacs rejects all
the fonts you have that are capable of supporting Devanagari.  I'm
trying to figure out why.

bug#21260: 23.2; Devanagari windows 10

[Jim Funderburk]

Hello, Eli -
   Here's the latest installment.
  1. Removed the ups10.dll from C:\emacs-23.2\bin
  2.  Here are all the usp10.dll  files in the c:\windows directory  
(from a Windows Search).
       a. All files have same Date-modified: 7/10/2015 7:00AM
       b. First two have Size 75.5KB; last two have size 76.5KB
"C:\Windows\WinSxS\amd64_microsoft-windows-usp_31bf3856ad364e35_10.0.10240.16384_none_b4faeabcf329aad2\usp10.dll"
"C:\Windows\SysWOW64\usp10.dll"
"C:\Windows\WinSxS\x86_microsoft-windows-usp_31bf3856ad364e35_10.0.10240.16384_none_58dc4f393acc399c\usp10.dll"
"C:\Windows\System32\usp10.dll"

3.  Regarding fonts on windows 10 (from  Control Panel/Fonts):
      a. Neither Mangal nor Kokila shows up
      b. The only font that I see which mentions Devanagari and other 
Indian languages is 'Nirmala UI'
           Note: In light of your comment about Emacs and Indian 
Language Fonts, this difference between Windows 10
                      and prior versions of Windows may be the basis of 
the problem.
      c. When I set the Default Font to Nirmala UI,  Then the Devanagari 
in the sample files displays properly !!

4. Regarding which usp10.dll emacs loads:
     a. Downloaded Process Explorer from Microsoft
     b.  Using find dll feature, learned emacs.exe is ising 
c:\Windows\SysWOW64\usp10.dll

It seems that 3c provides a kludge solution, at least provisionally:  
Anytime I need to view Devanagari, I can change the font
   to Nirmala UI.

Regards,
  Jim

On 8/17/2015 12:34 PM, Eli Zaretskii wrote:
> [Please keep the bug address on the CC list.]
>
>> Date: Sun, 16 Aug 2015 16:21:53 -0400
>> From: Jim Funderburk <funderburk1@verizon.net>
>>
>>     1.  When I open emacs 'normally'  (c:\emacs-23.2\bin\emacs.exe), type
>> some text (in *scratch* buffer)
>>           'blah blah', and position the insertion point at the first 'a',
>> and then type "C-u C-x =",
>>           here's what shows in the *Help* buffer:
>>           character: a (97, #o141, #x61)
>> preferred charset: ascii (ASCII (ISO646 IRV))
>>          code point: 0x61
>>              syntax: w     which means: word
>>            category: .:Base, a:ASCII, l:Latin, r:Roman
>>         buffer code: #x61
>>           file code: #x61 (encoded by coding system iso-latin-1-dos)
>>             display: by this font (glyph code)
>>       uniscribe:-outline-Courier New-normal-normal-normal-mono-13-*-*-*-c-*-iso8859-1 (#x44)
> OK, so Emacs does try to use Uniscribe on that system.
>
>> 2.  Regarding usp10.dll -  I searched for it in the c:\Windows
>> directory, and found it in 4 places.
>>        One of those was in System32 folder.  I copied usp10.dll to the
>> c:\emacs-23.2\bin folder, where emacs.exe resides.
> The one in System32 is the wrong one: it's a 64-bit DLL, whereas Emacs
> is a 32-bit executable, it should use the one in C:/Windows/SysWOW64
> instead.  Please remove the DLL you put near emacs.exe, as it could
> get in the way as we continue digging into this problem.
>
> Can you show the full list of all the different usp10.dll files you
> have there, including their size and time stamp?  Also, could you
> please use some program like Process Explorer (from SysInternals) to
> find out which one of these DLLs Emacs actually loads?
>
> Next, there's the question with the fonts you have there.  On Windows
> XP and Windows 7, Emacs uses the Mangal font to display the Hindi
> script; on Windows 8.1 it uses Kokila instead.  Do you have any of
> these fonts on your system?  If so, could you please show the OpenType
> properties of these fonts, in particular the scripts they support and
> the features they support for each script?  One program that can show
> this information is FontTesterPlus, which you should be able to
> download and install (I have version 1.4).
>
> I'm sorry to ask you to do all this, but I have no access to Windows
> 10, and I see no such problems on all other versions through 8.1.
>
>> 2a. In the "C-U C-X =" output of (1) above,  the presence of 'uniscribe'
>> makes me think that usp10.dll is being used already,
> Yes, it is.
>
>>             display: no font available
>>
>> Character code properties: customize what to show
>>     name: DEVANAGARI DOUBLE DANDA
>>     general-category: Po (Punctuation, Other)
>>
>> NOTE: It seems to be analyzing the character properly ---- But  I wonder
>> why it shows 'display: no font available'.
> That's the crux of your problem: for some reason, Emacs rejects all
> the fonts you have that are capable of supporting Devanagari.  I'm
> trying to figure out why.
>

bug#21260: 23.2; Devanagari windows 10

[Eli Zaretskii]

> Cc: 21260@debbugs.gnu.org
> From: Jim Funderburk <funderburk1@verizon.net>
> Date: Mon, 17 Aug 2015 16:16:21 -0400
> 
> 3.  Regarding fonts on windows 10 (from  Control Panel/Fonts):
>       a. Neither Mangal nor Kokila shows up
>       b. The only font that I see which mentions Devanagari and other 
> Indian languages is 'Nirmala UI'
>            Note: In light of your comment about Emacs and Indian 
> Language Fonts, this difference between Windows 10
>                       and prior versions of Windows may be the basis of 
> the problem.
>       c. When I set the Default Font to Nirmala UI,  Then the Devanagari 
> in the sample files displays properly !!

I found the Nirmala UI font on Windows 8.1 and looked at its OTF
features.  I think I know what causes the problem.  Please try
evaluating the following expression in "emacs -Q" immediately after
start, and then see whether display of Devanagari works okay after
that:

  (set-fontset-font "fontset-default" 'devanagari
                    (font-spec :registry "iso10646-1" :otf '(dev2 nil (rphf)))
                    nil 'prepend)

If the above doesn't work, try this instead:

  (set-fontset-font "fontset-default" 'devanagari "Nirmala UI" nil 'prepend)

If any of these work, you can put it in your .emacs init file to make
it permanent.

bug#21260: 23.2; Devanagari windows 10

[Jim Funderburk]

Eli -
   I tried both 'set-fontset-font'  suggestions -- and both worked!
   I could not tell any difference between the two solutions.

   Also, I put the first option in c:\.emacs file, per your suggestion.
   Now, the sample devanagari file opens properly when I use Emacs in
   the usual way (i.e., without -Q).

   As far as I can see at the moment,  the problem is satisfactorily
   solved.
   Thank you for working through the options with me and finding
    a good solution.

Regards,
  Jim Funderburk

On 8/18/2015 10:19 AM, Eli Zaretskii wrote:
>> Cc: 21260@debbugs.gnu.org
>> From: Jim Funderburk <funderburk1@verizon.net>
>> Date: Mon, 17 Aug 2015 16:16:21 -0400
>>
>> 3.  Regarding fonts on windows 10 (from  Control Panel/Fonts):
>>        a. Neither Mangal nor Kokila shows up
>>        b. The only font that I see which mentions Devanagari and other
>> Indian languages is 'Nirmala UI'
>>             Note: In light of your comment about Emacs and Indian
>> Language Fonts, this difference between Windows 10
>>                        and prior versions of Windows may be the basis of
>> the problem.
>>        c. When I set the Default Font to Nirmala UI,  Then the Devanagari
>> in the sample files displays properly !!
> I found the Nirmala UI font on Windows 8.1 and looked at its OTF
> features.  I think I know what causes the problem.  Please try
> evaluating the following expression in "emacs -Q" immediately after
> start, and then see whether display of Devanagari works okay after
> that:
>
>    (set-fontset-font "fontset-default" 'devanagari
>                      (font-spec :registry "iso10646-1" :otf '(dev2 nil (rphf)))
>                      nil 'prepend)
>
> If the above doesn't work, try this instead:
>
>    (set-fontset-font "fontset-default" 'devanagari "Nirmala UI" nil 'prepend)
>
> If any of these work, you can put it in your .emacs init file to make
> it permanent.
>

bug#21260: 23.2; Devanagari windows 10

[Eli Zaretskii]

> Cc: 21260@debbugs.gnu.org
> From: Jim Funderburk <funderburk1@verizon.net>
> Date: Tue, 18 Aug 2015 15:23:05 -0400
> 
>    I tried both 'set-fontset-font'  suggestions -- and both worked!
>    I could not tell any difference between the two solutions.

The first one is preferable because it is not specific to the Nirmala
UI font.  It allows any font that declares support for Devanagari via
the "dev2" OTF tag to be used by Emacs.

>    Also, I put the first option in c:\.emacs file, per your suggestion.
>    Now, the sample devanagari file opens properly when I use Emacs in
>    the usual way (i.e., without -Q).
> 
>    As far as I can see at the moment,  the problem is satisfactorily
>    solved.

OK, thanks for testing.  The equivalent of the first call to
set-fontset-font is already in the Emacs development sources -- I have
discovered this issue (with other scripts) while looking at the
related code that's involved in your use case, and fixed it yesterday.

I also have a few more improvements and bugfixes for minor problems I
discovered in the related parts of Uniscribe support, which I will
commit soon, before I mark this bug done.

>    Thank you for working through the options with me and finding
>     a good solution.

Thanks for helping me debug this.

Btw, I suggest to upgrade to the latest Emacs 24.5, as the version you
were using is quite old, in particular with respect to display of
complex scripts.  For example, Emacs 24 support reordering of
bidirectional text per the Unicode Bidirectional Algorithm.

bug#21260: 23.2; Devanagari windows 10

[Jim Funderburk]

Eli - Just a note as to why I prefer the old Emacs 23 version.
    Somewhere in the evolution to version 24, Emacs dropped a small feature
    that I find important for productivity.  Namely, in Emacs23, when you
    select text with the mouse (by double clicking a word, triple 
clicking a line,
    or dragging ),  the selected text automatically goes into the system 
clipboard
    and you can immediately paste elsewhere in some Emacs buffer or into 
some
    other application.   By contrast, for Emacs 24, you have to further 
click the 'copy'
    button (or some keyboard equivalent).    In some of my work flows, I 
do a lot
    (several hundred) such operations,  and the time involved for the 
extra step
   in Emacs 24 is significant.

    However, I appreciate the tip regarding bidirectional text support 
in Emacs 24,
    as I have noticed that Emacs 23 does not handle Arabic properly.

  Jim


On 8/18/2015 3:40 PM, Eli Zaretskii wrote:
>> Cc: 21260@debbugs.gnu.org
>> From: Jim Funderburk <funderburk1@verizon.net>
>> Date: Tue, 18 Aug 2015 15:23:05 -0400
>>
>>     I tried both 'set-fontset-font'  suggestions -- and both worked!
>>     I could not tell any difference between the two solutions.
> The first one is preferable because it is not specific to the Nirmala
> UI font.  It allows any font that declares support for Devanagari via
> the "dev2" OTF tag to be used by Emacs.
>
>>     Also, I put the first option in c:\.emacs file, per your suggestion.
>>     Now, the sample devanagari file opens properly when I use Emacs in
>>     the usual way (i.e., without -Q).
>>
>>     As far as I can see at the moment,  the problem is satisfactorily
>>     solved.
> OK, thanks for testing.  The equivalent of the first call to
> set-fontset-font is already in the Emacs development sources -- I have
> discovered this issue (with other scripts) while looking at the
> related code that's involved in your use case, and fixed it yesterday.
>
> I also have a few more improvements and bugfixes for minor problems I
> discovered in the related parts of Uniscribe support, which I will
> commit soon, before I mark this bug done.
>
>>     Thank you for working through the options with me and finding
>>      a good solution.
> Thanks for helping me debug this.
>
> Btw, I suggest to upgrade to the latest Emacs 24.5, as the version you
> were using is quite old, in particular with respect to display of
> complex scripts.  For example, Emacs 24 support reordering of
> bidirectional text per the Unicode Bidirectional Algorithm.
>

bug#21260: 23.2; Devanagari windows 10

[Eli Zaretskii]

> Cc: 21260@debbugs.gnu.org
> From: Jim Funderburk <funderburk1@verizon.net>
> Date: Tue, 18 Aug 2015 15:52:23 -0400
> 
> Eli - Just a note as to why I prefer the old Emacs 23 version.
>     Somewhere in the evolution to version 24, Emacs dropped a small feature
>     that I find important for productivity.  Namely, in Emacs23, when you
>     select text with the mouse (by double clicking a word, triple 
> clicking a line,
>     or dragging ),  the selected text automatically goes into the system 
> clipboard
>     and you can immediately paste elsewhere in some Emacs buffer or into 
> some
>     other application.   By contrast, for Emacs 24, you have to further 
> click the 'copy'
>     button (or some keyboard equivalent).

There's an explanation in NEWS how to get the behavior you like back;
search for "selection".

>     However, I appreciate the tip regarding bidirectional text support 
> in Emacs 24,
>     as I have noticed that Emacs 23 does not handle Arabic properly.

Exactly.

bug#21260: 23.2; Devanagari windows 10

[Eli Zaretskii]

> Date: Tue, 18 Aug 2015 22:40:08 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 21260@debbugs.gnu.org
> 
> I also have a few more improvements and bugfixes for minor problems I
> discovered in the related parts of Uniscribe support, which I will
> commit soon, before I mark this bug done.

Done now, closing.

bug#21260: 23.2; Devanagari windows 10

[Andy Moreton]

On Wed 19 Aug 2015, Eli Zaretskii wrote:

>> Date: Tue, 18 Aug 2015 22:40:08 +0300
>> From: Eli Zaretskii <eliz@gnu.org>
>> Cc: 21260@debbugs.gnu.org
>> 
>> I also have a few more improvements and bugfixes for minor problems I
>> discovered in the related parts of Uniscribe support, which I will
>> commit soon, before I mark this bug done.
>
> Done now, closing.

The recent changes broke the mingw64 (64bit) and cygwin w32 builds of
emacs.

Cygwin and msys2 mingw64 headers from w32api have this in usp10.h:

  22  #if !defined (UNISCRIBE_OPENTYPE) && (_WIN32_WINNT >= 0x0600)
  23  #define UNISCRIBE_OPENTYPE 0x0100
  24  #endif

  67  #if UNISCRIBE_OPENTYPE >= 0x0100
  68  typedef ULONG OPENTYPE_TAG;
  69  #endif

This seems to fix the build:

diff --git a/src/w32uniscribe.c b/src/w32uniscribe.c
index b1056bc104e0..c311cade6c1f 100644
--- a/src/w32uniscribe.c
+++ b/src/w32uniscribe.c
@@ -25,6 +25,8 @@ along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
    of calling non-existent functions.  */
 #undef _WIN32_WINNT
 #define _WIN32_WINNT 0x500
+#undef UNISCRIBE_OPENTYPE
+#define UNISCRIBE_OPENTYPE 0x0100
 #include <windows.h>
 #include <usp10.h>

bug#21260: 23.2; Devanagari windows 10

[Eli Zaretskii]

> From: Andy Moreton <andrewjmoreton@gmail.com>
> Date: Wed, 19 Aug 2015 21:26:38 +0100
> 
> The recent changes broke the mingw64 (64bit) and cygwin w32 builds of
> emacs.

I hate those gratuitous incompatibilities in MinGW64 headers!

> Cygwin and msys2 mingw64 headers from w32api have this in usp10.h:
> 
>   22  #if !defined (UNISCRIBE_OPENTYPE) && (_WIN32_WINNT >= 0x0600)
>   23  #define UNISCRIBE_OPENTYPE 0x0100
>   24  #endif
> 
>   67  #if UNISCRIBE_OPENTYPE >= 0x0100
>   68  typedef ULONG OPENTYPE_TAG;
>   69  #endif
> 
> This seems to fix the build:
> 
> diff --git a/src/w32uniscribe.c b/src/w32uniscribe.c
> index b1056bc104e0..c311cade6c1f 100644
> --- a/src/w32uniscribe.c
> +++ b/src/w32uniscribe.c
> @@ -25,6 +25,8 @@ along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
>     of calling non-existent functions.  */
>  #undef _WIN32_WINNT
>  #define _WIN32_WINNT 0x500
> +#undef UNISCRIBE_OPENTYPE
> +#define UNISCRIBE_OPENTYPE 0x0100
>  #include <windows.h>
>  #include <usp10.h>

Thanks, I fixed this in 092e17b.

bug#21260: 23.2; Devanagari windows 10

[Jim Funderburk]

Dear Eli -
  Please refer to this dropbox link for images: 
https://www.dropbox.com/s/dqxe4edz5d4nmnc/emacs-gdi.zip?dl=0
   1. Opening with gdi is a move in the right direction. Here's the 
command used:
       c:\emacs-23.2\bin\emacs.exe -Q -xrm Emacs.fontBackend:gdi 
c:/Users/Jim/Dropbox/ejf/devanagari_example.txt
      The result is shown in 'open_with_gdi.png' -  the Devanagari renders
   2. Not all is exactly right, however.  Notice the highlighted part of 
other image (open-with_gdi_prob.png)
       When I moved the mouse and clicked on a character in the last 
line,  the rendering of the character changed
        Obviously, it should not do this.  [This happens elsewhere, on 
most, but not all characters.]
        However, things are definitely better with this option.
I'll respond to the 2nd email separately.
Thanks,
  Jim

On 8/15/2015 10:40 PM, Eli Zaretskii wrote:
>> Cc: 21260@debbugs.gnu.org
>> From: Jim Funderburk <funderburk1@verizon.net>
>> Date: Sat, 15 Aug 2015 15:55:23 -0400
>>
>>     Here is another dropbox link with further info you requested:
>> https://www.dropbox.com/s/3er09sc7jc8m5kc/emacs-devanagari.zip?dl=0
>>     There are two images:
>>     1. emacs-devanagari.png    This is a screen shot of me opening up the
>> file
>>        in Emacs via  command line,  and of what the file looks like on my
>> system.
>>        It looks like all boxes.
>>     2. emacs-Ctl-h-H.png     This is screen shot of what emacs shows
>> after Ctl-h-H command.
>>          As you can see, the South Asian language shows up in boxes,
>> also  'Hindi' and 'Kannada'
>>          Also  (maybe outside of the image)  Tamil and Telugu - all South
>> Asian languages.
>>     You asked what font -  when I do menu Options/Set Default Font, it
>> shows as Courier New.
>>         I have tried changing to Arial, Lucida Sans Unicode, Consolas
>> with no improvement.
>>      Note:  I thought maybe  this is due to a deficiency in the fonts in
>> Windows 10.  However,
>>                  When I open the file in Notepad,   the Devanagari
>> displays just fine;  Notepad
>>                   shows that it is using Consolas Font.
>>      Note2:  Also, I have a Windows Vista system and this displays fine
>> in the same Emacs 23 on that system.
>>      Note3:  ALso, I tried Emacs 24 on Windows 10, and the same problem
>> occurs.
>>
>>      So, As of now, it seems like there is a bug in displaying South
>> Asian Unicode text in Emacs on Windows 10.
> Does this work better?
>
>    emacs -Q -xrm Emacs.fontBackend:gdi
>

bug#21260: 23.2; Devanagari windows 10

[Eli Zaretskii]

> Cc: 21260@debbugs.gnu.org
> From: Jim Funderburk <funderburk1@verizon.net>
> Date: Sun, 16 Aug 2015 16:21:44 -0400
> 
>   Please refer to this dropbox link for images: 
> https://www.dropbox.com/s/dqxe4edz5d4nmnc/emacs-gdi.zip?dl=0
>    1. Opening with gdi is a move in the right direction.

Actually, no, not in the direction you need to enjoy the correct
display of Devanagari.  The GDI font back-end is unable to display
scripts that need complex shaping and ligation, and Devanagari
requires those features.

I asked you to try GDI to confirm my suspicion that something is wrong
with Uniscribe, the main font back-end we use on Windows.  Now the
suspicion is confirmed.

>       The result is shown in 'open_with_gdi.png' -  the Devanagari renders
>    2. Not all is exactly right, however.  Notice the highlighted part of 
> other image (open-with_gdi_prob.png)
>        When I moved the mouse and clicked on a character in the last 
> line,  the rendering of the character changed
>         Obviously, it should not do this.  [This happens elsewhere, on 
> most, but not all characters.]

This is expected with the GDI back-end.