bug#19910: 24.4; Japanese font names are decoded incorrectly in Cygwin's emacs-w32 in LANG=ja_JP.UTF-8

bug#19910: 24.4; Japanese font names are decoded incorrectly in Cygwin's emacs-w32 in LANG=ja_JP.UTF-8

[Fujii Hironori]

(font-family-list) returns incorrectly decoded Japanese font names.
My locale-coding-system is utf-8-unix.

If I do (setq locale-coding-system 'cp932), it returns correct font names.
But, locale-coding-system is used in other places (e.g. M-x term and M-x man).
locale-coding-system must be utf-8 in my Emacs.



In GNU Emacs 24.4.1 (x86_64-unknown-cygwin)
 of 2015-02-13 on desktop-new
Windowing system distributor `Microsoft Corp.', version 6.1.7601
Configured using:
 `configure
 --srcdir=/home/kbrown/src/cygemacs/emacs-24.4-3.x86_64/src/emacs-24.4
 --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin
 --libexecdir=/usr/libexec --datadir=/usr/share --localstatedir=/var
 --sysconfdir=/etc --libdir=/usr/lib --datarootdir=/usr/share
 --docdir=/usr/share/doc/emacs --htmldir=/usr/share/doc/emacs/html -C
 --with-w32 'CFLAGS=-ggdb -O2 -pipe -Wimplicit-function-declaration
 -fdebug-prefix-map=/home/kbrown/src/cygemacs/emacs-24.4-3.x86_64/build=/usr/src/debug/emacs-24.4-3
 -fdebug-prefix-map=/home/kbrown/src/cygemacs/emacs-24.4-3.x86_64/src/emacs-24.4=/usr/src/debug/emacs-24.4-3'
 CPPFLAGS= LDFLAGS='

Important settings:
  value of $LANG: ja_JP.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Fundamental

Minor modes in effect:
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<language-change> <help-echo> <help-echo> <help-echo>
<help-echo> <help-echo> <help-echo> <help-echo> <menu-bar>
<help-menu> <send-emacs-bug-report>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
No docstring slot for setup-japanese-environment-internal

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
easymenu mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util help-fns mail-prsvr mail-utils time-date japan-util tooltip
electric uniquify ediff-hook vc-hooks lisp-float-type mwheel
w32-common-fns disp-table w32-win w32-vars tool-bar dnd fontset image
regexp-opt fringe tabulated-list newcomment lisp-mode prog-mode register
page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core frame cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese hebrew
greek romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dbusbind
gfilenotify w32 multi-tty emacs)

Memory information:
((conses 16 76005 6531)
 (symbols 48 17442 0)
 (miscs 40 60 88)
 (strings 32 10617 5193)
 (string-bytes 1 268020)
 (vectors 16 9545)
 (vector-slots 8 454592 39464)
 (floats 8 57 94)
 (intervals 56 193 0)
 (buffers 960 12))

bug#19910: 24.4; Japanese font names are decoded incorrectly in Cygwin's emacs-w32 in LANG=ja_JP.UTF-8

[Eli Zaretskii]

> Date: Fri, 20 Feb 2015 19:39:55 +0900
> From: Fujii Hironori <fujii.hironori@gmail.com>
> 
> (font-family-list) returns incorrectly decoded Japanese font names.
> My locale-coding-system is utf-8-unix.
> 
> If I do (setq locale-coding-system 'cp932), it returns correct font names.
> But, locale-coding-system is used in other places (e.g. M-x term and M-x man).
> locale-coding-system must be utf-8 in my Emacs.

The problem is in w32font.c: it should call the "wide" (a.k.a.
"Unicode") APIs, and then decode strings using utf-16le, like we do in
w32fns.c with encoding strings we pass to w32 GUI APIs.

bug#19910: 24.4; Japanese font names are decoded incorrectly in Cygwin's emacs-w32 in LANG=ja_JP.UTF-8

[Fujii Hironori]

[-- Attachment #1: Type: text/plain, Size: 375 bytes --]

Tags: patch

On Fri, Feb 20, 2015 at 8:21 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> The problem is in w32font.c: it should call the "wide" (a.k.a.
> "Unicode") APIs, and then decode strings using utf-16le, like we do in
> w32fns.c with encoding strings we pass to w32 GUI APIs.

Unicode API patch is attached. Could you review it?
Should I use GetProcAddress for Windows 9x?

[-- Attachment #2: font.patch --]
[-- Type: text/x-patch, Size: 23949 bytes --]

diff --git a/src/w32font.c b/src/w32font.c
index 422f082..631cecf 100644
--- a/src/w32font.c
+++ b/src/w32font.c
@@ -19,6 +19,7 @@ along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
 #include <config.h>
 #include <windows.h>
 #include <stdio.h>
+#include <wchar.h>
 #include <math.h>
 #include <ctype.h>
 #include <commdlg.h>
@@ -57,37 +58,37 @@ along with GNU Emacs.  If not, see <http://www.gnu.org/licenses/>.  */
 #define JOHAB_CHARSET 130
 #endif
 
-static void fill_in_logfont (struct frame *, LOGFONT *, Lisp_Object);
+static void fill_in_logfont (struct frame *, LOGFONTW *, Lisp_Object);
 
 static BYTE w32_antialias_type (Lisp_Object);
 static Lisp_Object lispy_antialias_type (BYTE);
 
 static Lisp_Object font_supported_scripts (FONTSIGNATURE *);
-static int w32font_full_name (LOGFONT *, Lisp_Object, int, char *, int);
+static int w32font_full_name (LOGFONTW *, Lisp_Object, int, wchar_t *, int);
 static void compute_metrics (HDC, struct w32font_info *, unsigned int,
                              struct w32_metric_cache *);
 
 static Lisp_Object w32_registry (LONG, DWORD);
 
-/* EnumFontFamiliesEx callbacks.  */
-static int CALLBACK ALIGN_STACK add_font_entity_to_list (ENUMLOGFONTEX *,
-							 NEWTEXTMETRICEX *,
+/* EnumFontFamiliesExW callbacks.  */
+static int CALLBACK ALIGN_STACK add_font_entity_to_list (ENUMLOGFONTEXW *,
+							 NEWTEXTMETRICEXW *,
 							 DWORD, LPARAM);
-static int CALLBACK ALIGN_STACK add_one_font_entity_to_list (ENUMLOGFONTEX *,
-							     NEWTEXTMETRICEX *,
+static int CALLBACK ALIGN_STACK add_one_font_entity_to_list (ENUMLOGFONTEXW *,
+							     NEWTEXTMETRICEXW *,
 							     DWORD, LPARAM);
-static int CALLBACK ALIGN_STACK add_font_name_to_list (ENUMLOGFONTEX *,
-						       NEWTEXTMETRICEX *,
+static int CALLBACK ALIGN_STACK add_font_name_to_list (ENUMLOGFONTEXW *,
+						       NEWTEXTMETRICEXW *,
 						       DWORD, LPARAM);
 
-/* struct passed in as LPARAM arg to EnumFontFamiliesEx, for keeping track
+/* struct passed in as LPARAM arg to EnumFontFamiliesExW, for keeping track
    of what we really want.  */
 struct font_callback_data
 {
-  /* The logfont we are matching against. EnumFontFamiliesEx only matches
+  /* The logfont we are matching against. EnumFontFamiliesExW only matches
      face name and charset, so we need to manually match everything else
      in the callback function.  */
-  LOGFONT pattern;
+  LOGFONTW pattern;
   /* The original font spec or entity.  */
   Lisp_Object orig_font_spec;
   /* The frame the font is being loaded on.  */
@@ -98,7 +99,7 @@ struct font_callback_data
   bool opentype_only;
 };
 
-/* Handles the problem that EnumFontFamiliesEx will not return all
+/* Handles the problem that EnumFontFamiliesExW will not return all
    style variations if the font name is not specified.  */
 static void list_all_matching_fonts (struct font_callback_data *);
 
@@ -239,9 +240,9 @@ memq_no_quit (Lisp_Object elt, Lisp_Object list)
 }
 
 Lisp_Object
-intern_font_name (char * string)
+intern_font_name (wchar_t * string)
 {
-  Lisp_Object str = DECODE_SYSTEM (build_string (string));
+  Lisp_Object str = from_unicode_buffer (string);
   ptrdiff_t len = SCHARS (str);
   Lisp_Object obarray = check_obarray (Vobarray);
   Lisp_Object tem = oblookup (obarray, SDATA (str), len, len);
@@ -291,7 +292,7 @@ static Lisp_Object
 w32font_list_family (struct frame *f)
 {
   Lisp_Object list = Qnil;
-  LOGFONT font_match_pattern;
+  LOGFONTW font_match_pattern;
   HDC dc;
 
   memset (&font_match_pattern, 0, sizeof (font_match_pattern));
@@ -299,9 +300,9 @@ w32font_list_family (struct frame *f)
 
   dc = get_frame_dc (f);
 
-  EnumFontFamiliesEx (dc, &font_match_pattern,
-                      (FONTENUMPROC) add_font_name_to_list,
-                      (LPARAM) &list, 0);
+  EnumFontFamiliesExW (dc, &font_match_pattern,
+		       (FONTENUMPROCW) add_font_name_to_list,
+		       (LPARAM) &list, 0);
   release_frame_dc (f, dc);
 
   return list;
@@ -756,7 +757,7 @@ w32font_list_internal (struct frame *f, Lisp_Object font_spec,
   match_data.list = Qnil;
   XSETFRAME (match_data.frame, f);
 
-  memset (&match_data.pattern, 0, sizeof (LOGFONT));
+  memset (&match_data.pattern, 0, sizeof (LOGFONTW));
   fill_in_logfont (f, &match_data.pattern, font_spec);
 
   /* If the charset is unrecognized, then we won't find a font, so don't
@@ -776,9 +777,9 @@ w32font_list_internal (struct frame *f, Lisp_Object font_spec,
   if (opentype_only)
     match_data.pattern.lfOutPrecision = OUT_OUTLINE_PRECIS;
 
-  if (match_data.pattern.lfFaceName[0] == '\0')
+  if (match_data.pattern.lfFaceName[0] == L'\0')
     {
-      /* EnumFontFamiliesEx does not take other fields into account if
+      /* EnumFontFamiliesExW does not take other fields into account if
          font name is blank, so need to use two passes.  */
       list_all_matching_fonts (&match_data);
     }
@@ -786,9 +787,9 @@ w32font_list_internal (struct frame *f, Lisp_Object font_spec,
     {
       dc = get_frame_dc (f);
 
-      EnumFontFamiliesEx (dc, &match_data.pattern,
-                          (FONTENUMPROC) add_font_entity_to_list,
-                          (LPARAM) &match_data, 0);
+      EnumFontFamiliesExW (dc, &match_data.pattern,
+			   (FONTENUMPROCW) add_font_entity_to_list,
+			   (LPARAM) &match_data, 0);
       release_frame_dc (f, dc);
     }
 
@@ -809,7 +810,7 @@ w32font_match_internal (struct frame *f, Lisp_Object font_spec,
   XSETFRAME (match_data.frame, f);
   match_data.list = Qnil;
 
-  memset (&match_data.pattern, 0, sizeof (LOGFONT));
+  memset (&match_data.pattern, 0, sizeof (LOGFONTW));
   fill_in_logfont (f, &match_data.pattern, font_spec);
 
   match_data.opentype_only = opentype_only;
@@ -818,9 +819,9 @@ w32font_match_internal (struct frame *f, Lisp_Object font_spec,
 
   dc = get_frame_dc (f);
 
-  EnumFontFamiliesEx (dc, &match_data.pattern,
-                      (FONTENUMPROC) add_one_font_entity_to_list,
-                      (LPARAM) &match_data, 0);
+  EnumFontFamiliesExW (dc, &match_data.pattern,
+		       (FONTENUMPROCW) add_one_font_entity_to_list,
+		       (LPARAM) &match_data, 0);
   release_frame_dc (f, dc);
 
   return NILP (match_data.list) ? Qnil : XCAR (match_data.list);
@@ -831,7 +832,7 @@ w32font_open_internal (struct frame *f, Lisp_Object font_entity,
 		       int pixel_size, Lisp_Object font_object)
 {
   int len, size;
-  LOGFONT logfont;
+  LOGFONTW logfont;
   HDC dc;
   HFONT hfont, old_font;
   Lisp_Object val;
@@ -859,7 +860,7 @@ w32font_open_internal (struct frame *f, Lisp_Object font_entity,
     size = pixel_size;
 
   logfont.lfHeight = -size;
-  hfont = CreateFontIndirect (&logfont);
+  hfont = CreateFontIndirectW (&logfont);
 
   if (hfont == NULL)
     return 0;
@@ -892,24 +893,24 @@ w32font_open_internal (struct frame *f, Lisp_Object font_entity,
   w32_font->hfont = hfont;
 
   {
-    char *name;
+    wchar_t *name;
 
     /* We don't know how much space we need for the full name, so start with
-       96 bytes and go up in steps of 32.  */
+       96 characters and go up in steps of 32.  */
     len = 96;
-    name = alloca (len);
+    name = alloca (len * sizeof (wchar_t));
     while (name && w32font_full_name (&logfont, font_entity, pixel_size,
                                       name, len) < 0)
       {
         len += 32;
-        name = alloca (len);
+        name = alloca (len * sizeof (wchar_t));
       }
     if (name)
       font->props[FONT_FULLNAME_INDEX]
-        = DECODE_SYSTEM (build_string (name));
+        = from_unicode_buffer (name);
     else
       font->props[FONT_FULLNAME_INDEX]
-	= DECODE_SYSTEM (build_string (logfont.lfFaceName));
+	= from_unicode_buffer (logfont.lfFaceName);
   }
 
   font->max_width = w32_font->metrics.tmMaxCharWidth;
@@ -955,18 +956,18 @@ w32font_open_internal (struct frame *f, Lisp_Object font_entity,
   return 1;
 }
 
-/* Callback function for EnumFontFamiliesEx.
+/* Callback function for EnumFontFamiliesExW.
  * Adds the name of a font to a Lisp list (passed in as the lParam arg).  */
 static int CALLBACK ALIGN_STACK
-add_font_name_to_list (ENUMLOGFONTEX *logical_font,
-		       NEWTEXTMETRICEX *physical_font,
+add_font_name_to_list (ENUMLOGFONTEXW *logical_font,
+		       NEWTEXTMETRICEXW *physical_font,
 		       DWORD font_type, LPARAM list_object)
 {
   Lisp_Object* list = (Lisp_Object *) list_object;
   Lisp_Object family;
 
   /* Skip vertical fonts (intended only for printing)  */
-  if (logical_font->elfLogFont.lfFaceName[0] == '@')
+  if (logical_font->elfLogFont.lfFaceName[0] == L'@')
     return 1;
 
   family = intern_font_name (logical_font->elfLogFont.lfFaceName);
@@ -982,14 +983,14 @@ static int w32_encode_weight (int);
 /* Convert an enumerated Windows font to an Emacs font entity.  */
 static Lisp_Object
 w32_enumfont_pattern_entity (Lisp_Object frame,
-			     ENUMLOGFONTEX *logical_font,
-			     NEWTEXTMETRICEX *physical_font,
+			     ENUMLOGFONTEXW *logical_font,
+			     NEWTEXTMETRICEXW *physical_font,
 			     DWORD font_type,
-			     LOGFONT *requested_font,
+			     LOGFONTW *requested_font,
 			     Lisp_Object backend)
 {
   Lisp_Object entity, tem;
-  LOGFONT *lf = (LOGFONT*) logical_font;
+  LOGFONTW *lf = (LOGFONTW*) logical_font;
   BYTE generic_type;
   DWORD full_type = physical_font->ntmTm.ntmFlags;
 
@@ -1110,7 +1111,7 @@ w32_generic_family (Lisp_Object name)
 }
 
 static int
-logfonts_match (LOGFONT *font, LOGFONT *pattern)
+logfonts_match (LOGFONTW *font, LOGFONTW *pattern)
 {
   /* Only check height for raster fonts.  */
   if (pattern->lfHeight && font->lfOutPrecision == OUT_STRING_PRECIS
@@ -1134,9 +1135,9 @@ logfonts_match (LOGFONT *font, LOGFONT *pattern)
 #define CSB_CHINESE ((1 << 18) | (1 << 20))
 
 static int
-font_matches_spec (DWORD type, NEWTEXTMETRICEX *font,
+font_matches_spec (DWORD type, NEWTEXTMETRICEXW *font,
 		   Lisp_Object spec, Lisp_Object backend,
-		   LOGFONT *logfont)
+		   LOGFONTW *logfont)
 {
   Lisp_Object extra, val;
 
@@ -1354,14 +1355,16 @@ w32font_coverage_ok (FONTSIGNATURE * coverage, BYTE charset)
   return 1;
 }
 
-#ifndef WINDOWSNT
-#define _strlwr strlwr
+#ifdef WINDOWSNT
+#define xwcslwr _wcslwr
+#else
+#define xwcslwr CharLowerW
 #endif /* !WINDOWSNT */
 
 static int
-check_face_name (LOGFONT *font, char *full_name)
+check_face_name (LOGFONTW *font, wchar_t *full_name)
 {
-  char full_iname[LF_FULLFACESIZE+1];
+  wchar_t full_iname[LF_FULLFACESIZE+1];
 
   /* Just check for names known to cause problems, since the full name
      can contain expanded abbreviations, prefixed foundry, postfixed
@@ -1373,39 +1376,39 @@ check_face_name (LOGFONT *font, char *full_name)
      to avoid non-truetype fonts, and ends up mixing the Type-1 Helvetica
      with Arial's characteristics, since that attempt to use TrueType works
      some places, but not others.  */
-  if (!xstrcasecmp (font->lfFaceName, "helvetica"))
+  if (!lstrcmpiW (font->lfFaceName, L"helvetica"))
     {
-      strncpy (full_iname, full_name, LF_FULLFACESIZE);
+      lstrcpynW (full_iname, full_name, LF_FULLFACESIZE);
       full_iname[LF_FULLFACESIZE] = 0;
-      _strlwr (full_iname);
-      return strstr ("helvetica", full_iname) != NULL;
+      xwcslwr (full_iname);
+      return wcsstr (L"helvetica", full_iname) != NULL;
     }
   /* Same for Helv.  */
-  if (!xstrcasecmp (font->lfFaceName, "helv"))
+  if (!lstrcmpiW (font->lfFaceName, L"helv"))
     {
-      strncpy (full_iname, full_name, LF_FULLFACESIZE);
+      lstrcpynW (full_iname, full_name, LF_FULLFACESIZE);
       full_iname[LF_FULLFACESIZE] = 0;
-      _strlwr (full_iname);
-      return strstr ("helv", full_iname) != NULL;
+      xwcslwr (full_iname);
+      return wcsstr (L"helv", full_iname) != NULL;
     }
 
   /* Since Times is mapped to Times New Roman, a substring
      match is not sufficient to filter out the bogus match.  */
-  else if (!xstrcasecmp (font->lfFaceName, "times"))
-    return xstrcasecmp (full_name, "times") == 0;
+  else if (!lstrcmpiW (font->lfFaceName, L"times"))
+    return lstrcmpiW (full_name, L"times") == 0;
 
   return 1;
 }
 
 
-/* Callback function for EnumFontFamiliesEx.
+/* Callback function for EnumFontFamiliesExW.
  * Checks if a font matches everything we are trying to check against,
  * and if so, adds it to a list. Both the data we are checking against
  * and the list to which the fonts are added are passed in via the
  * lparam argument, in the form of a font_callback_data struct. */
 static int CALLBACK ALIGN_STACK
-add_font_entity_to_list (ENUMLOGFONTEX *logical_font,
-			 NEWTEXTMETRICEX *physical_font,
+add_font_entity_to_list (ENUMLOGFONTEXW *logical_font,
+			 NEWTEXTMETRICEXW *physical_font,
 			 DWORD font_type, LPARAM lParam)
 {
   struct font_callback_data *match_data
@@ -1445,7 +1448,7 @@ add_font_entity_to_list (ENUMLOGFONTEX *logical_font,
      by a foundry, we accept raster fonts if the font name is found
      anywhere within the full name.  */
   if ((logical_font->elfLogFont.lfOutPrecision == OUT_STRING_PRECIS
-       && !strstr (logical_font->elfFullName,
+       && !wcsstr (logical_font->elfFullName,
 		   logical_font->elfLogFont.lfFaceName))
       /* Check for well known substitutions that mess things up in the
 	 presence of Type-1 fonts of the same name.  */
@@ -1520,11 +1523,11 @@ add_font_entity_to_list (ENUMLOGFONTEX *logical_font,
   return 1;
 }
 
-/* Callback function for EnumFontFamiliesEx.
+/* Callback function for EnumFontFamiliesExW.
  * Terminates the search once we have a match. */
 static int CALLBACK ALIGN_STACK
-add_one_font_entity_to_list (ENUMLOGFONTEX *logical_font,
-			     NEWTEXTMETRICEX *physical_font,
+add_one_font_entity_to_list (ENUMLOGFONTEXW *logical_font,
+			     NEWTEXTMETRICEXW *physical_font,
 			     DWORD font_type, LPARAM lParam)
 {
   struct font_callback_data *match_data
@@ -1881,9 +1884,9 @@ w32_to_fc_weight (int n)
   return Qlight;
 }
 
-/* Fill in all the available details of LOGFONT from FONT_SPEC.  */
+/* Fill in all the available details of LOGFONTW from FONT_SPEC.  */
 static void
-fill_in_logfont (struct frame *f, LOGFONT *logfont, Lisp_Object font_spec)
+fill_in_logfont (struct frame *f, LOGFONTW *logfont, Lisp_Object font_spec)
 {
   Lisp_Object tmp, extra;
   int dpi = FRAME_RES_Y (f);
@@ -1953,9 +1956,10 @@ fill_in_logfont (struct frame *f, LOGFONT *logfont, Lisp_Object font_spec)
            user input.  */
       else if (SYMBOLP (tmp))
 	{
-	  strncpy (logfont->lfFaceName,
-		   SDATA (ENCODE_SYSTEM (SYMBOL_NAME (tmp))), LF_FACESIZE);
-	  logfont->lfFaceName[LF_FACESIZE-1] = '\0';
+	  Lisp_Object t;
+	  wcsncpy (logfont->lfFaceName,
+		   to_unicode (SYMBOL_NAME (tmp), &t), LF_FACESIZE);
+	  logfont->lfFaceName[LF_FACESIZE-1] = L'\0';
 	}
     }
 
@@ -2044,22 +2048,22 @@ list_all_matching_fonts (struct font_callback_data *match_data)
 	 on Windows, so we can keep backwards compatibility with
 	 Windows 9x/ME by using non-Unicode font enumeration without
 	 sacrificing internationalization here.  */
-      char *name;
-      Lisp_Object family = CAR (families);
+      wchar_t *name;
+      Lisp_Object family = CAR (families), t;
       families = CDR (families);
       if (NILP (family))
         continue;
       else if (SYMBOLP (family))
-        name = SDATA (ENCODE_SYSTEM (SYMBOL_NAME (family)));
+        name = to_unicode (SYMBOL_NAME (family), &t);
       else
 	continue;
 
-      strncpy (match_data->pattern.lfFaceName, name, LF_FACESIZE);
-      match_data->pattern.lfFaceName[LF_FACESIZE - 1] = '\0';
+      wcsncpy (match_data->pattern.lfFaceName, name, LF_FACESIZE);
+      match_data->pattern.lfFaceName[LF_FACESIZE - 1] = L'\0';
 
-      EnumFontFamiliesEx (dc, &match_data->pattern,
-                          (FONTENUMPROC) add_font_entity_to_list,
-                          (LPARAM) match_data, 0);
+      EnumFontFamiliesExW (dc, &match_data->pattern,
+			   (FONTENUMPROCW) add_font_entity_to_list,
+			   (LPARAM) match_data, 0);
     }
 
   release_frame_dc (f, dc);
@@ -2239,14 +2243,14 @@ font_supported_scripts (FONTSIGNATURE * sig)
    The full name is in fcname format, with weight, slant and antialiasing
    specified if they are not "normal".  */
 static int
-w32font_full_name (LOGFONT * font, Lisp_Object font_obj,
-		   int pixel_size, char *name, int nbytes)
+w32font_full_name (LOGFONTW * font, Lisp_Object font_obj,
+		   int pixel_size, wchar_t *name, int size)
 {
   int len, height, outline;
-  char *p;
+  wchar_t *p;
   Lisp_Object antialiasing, weight = Qnil;
 
-  len = strlen (font->lfFaceName);
+  len = wcslen (font->lfFaceName);
 
   outline = EQ (AREF (font_obj, FONT_FOUNDRY_INDEX), Qoutline);
 
@@ -2271,11 +2275,11 @@ w32font_full_name (LOGFONT * font, Lisp_Object font_obj,
     len += 11 + SBYTES (SYMBOL_NAME (antialiasing)); /* :antialias=NAME */
 
   /* Check that the buffer is big enough  */
-  if (len > nbytes)
+  if (len > size)
     return -1;
 
   p = name;
-  p += sprintf (p, "%s", font->lfFaceName);
+  p += swprintf (p, name + size - p, L"%ls", font->lfFaceName);
 
   height = font->lfHeight ? eabs (font->lfHeight) : pixel_size;
 
@@ -2287,20 +2291,20 @@ w32font_full_name (LOGFONT * font, Lisp_Object font_obj,
           /* Round to nearest half point.  floor is used, since round is not
 	     supported in MS library.  */
           pointsize = floor (pointsize * 2 + 0.5) / 2;
-          p += sprintf (p, "-%1.1f", pointsize);
+          p += swprintf (p, name + size - p, L"-%1.1f", pointsize);
         }
       else
-        p += sprintf (p, ":pixelsize=%d", height);
+        p += swprintf (p, name + size - p, L":pixelsize=%d", height);
     }
 
   if (SYMBOLP (weight) && ! NILP (weight))
-    p += sprintf (p, ":%s", SDATA (SYMBOL_NAME (weight)));
+    p += swprintf (p, name + size - p, L":%ls", SDATA (SYMBOL_NAME (weight)));
 
   if (font->lfItalic)
-    p += sprintf (p, ":italic");
+    p += swprintf (p, name + size - p, L":italic");
 
   if (SYMBOLP (antialiasing) && ! NILP (antialiasing))
-    p += sprintf (p, ":antialias=%s", SDATA (SYMBOL_NAME (antialiasing)));
+    p += swprintf (p, name + size - p, L":antialias=%ls", SDATA (SYMBOL_NAME (antialiasing)));
 
   return (p - name);
 }
@@ -2312,13 +2316,13 @@ w32font_full_name (LOGFONT * font, Lisp_Object font_obj,
    the function returns -1, otherwise it returns the number of bytes
    written to FCNAME.  */
 static int
-logfont_to_fcname (LOGFONT* font, int pointsize, char *fcname, int size)
+logfont_to_fcname (LOGFONTW* font, int pointsize, wchar_t *fcname, int size)
 {
   int len, height;
-  char *p = fcname;
+  wchar_t *p = fcname;
   Lisp_Object weight = Qnil;
 
-  len = strlen (font->lfFaceName) + 2;
+  len = wcslen (font->lfFaceName) + 2;
   height = pointsize / 10;
   while (height /= 10)
     len++;
@@ -2337,15 +2341,15 @@ logfont_to_fcname (LOGFONT* font, int pointsize, char *fcname, int size)
   if (len > size)
     return -1;
 
-  p += sprintf (p, "%s-%d", font->lfFaceName, pointsize / 10);
+  p += swprintf (p, fcname + size - p, L"%ls-%d", font->lfFaceName, pointsize / 10);
   if (pointsize % 10)
-    p += sprintf (p, ".%d", pointsize % 10);
+    p += swprintf (p, fcname + size - p, L".%d", pointsize % 10);
 
   if (SYMBOLP (weight) && !NILP (weight))
-    p += sprintf (p, ":%s", SDATA (SYMBOL_NAME (weight)));
+    p += swprintf (p, fcname + size - p, L":%ls", SDATA (SYMBOL_NAME (weight)));
 
   if (font->lfItalic)
-    p += sprintf (p, ":italic");
+    p += swprintf (p, fcname + size - p, L":italic");
 
   return (p - fcname);
 }
@@ -2395,12 +2399,12 @@ in the font selection dialog. */)
   (Lisp_Object frame, Lisp_Object exclude_proportional)
 {
   struct frame *f = decode_window_system_frame (frame);
-  CHOOSEFONT cf;
-  LOGFONT lf;
+  CHOOSEFONTW cf;
+  LOGFONTW lf;
   TEXTMETRIC tm;
   HDC hdc;
   HANDLE oldobj;
-  char buf[100];
+  wchar_t buf[100];
 
   memset (&cf, 0, sizeof (cf));
   memset (&lf, 0, sizeof (lf));
@@ -2420,7 +2424,7 @@ in the font selection dialog. */)
      default font.  */
   hdc = GetDC (FRAME_W32_WINDOW (f));
   oldobj = SelectObject (hdc, FONT_HANDLE (FRAME_FONT (f)));
-  GetTextFace (hdc, LF_FACESIZE, lf.lfFaceName);
+  GetTextFaceW (hdc, LF_FACESIZE, lf.lfFaceName);
   if (GetTextMetrics (hdc, &tm))
     {
       lf.lfHeight = tm.tmInternalLeading - tm.tmHeight;
@@ -2434,11 +2438,11 @@ in the font selection dialog. */)
   SelectObject (hdc, oldobj);
   ReleaseDC (FRAME_W32_WINDOW (f), hdc);
 
-  if (!ChooseFont (&cf)
+  if (!ChooseFontW (&cf)
       || logfont_to_fcname (&lf, cf.iPointSize, buf, 100) < 0)
     return Qnil;
 
-  return DECODE_SYSTEM (build_string (buf));
+  return from_unicode_buffer (buf);
 }
 
 static const char *const w32font_booleans [] = {
diff --git a/src/w32font.h b/src/w32font.h
index 0ad0125..06e5f31 100644
--- a/src/w32font.h
+++ b/src/w32font.h
@@ -80,9 +80,9 @@ int w32font_draw (struct glyph_string *s, int from, int to,
                   int x, int y, bool with_background);
 
 
-int uniscribe_check_otf (LOGFONT *font, Lisp_Object otf_spec);
+int uniscribe_check_otf (LOGFONTW *font, Lisp_Object otf_spec);
 
-Lisp_Object intern_font_name (char *);
+Lisp_Object intern_font_name (wchar_t *);
 
 extern void syms_of_w32font (void);
 extern void globals_of_w32font (void);
diff --git a/src/w32uniscribe.c b/src/w32uniscribe.c
index 73c0410..d58f600 100644
--- a/src/w32uniscribe.c
+++ b/src/w32uniscribe.c
@@ -47,9 +47,9 @@ struct uniscribe_font_info
 
 int uniscribe_available = 0;
 
-/* EnumFontFamiliesEx callback.  */
-static int CALLBACK ALIGN_STACK add_opentype_font_name_to_list (ENUMLOGFONTEX *,
-								NEWTEXTMETRICEX *,
+/* EnumFontFamiliesExW callback.  */
+static int CALLBACK ALIGN_STACK add_opentype_font_name_to_list (ENUMLOGFONTEXW *,
+								NEWTEXTMETRICEXW *,
 								DWORD, LPARAM);
 /* Used by uniscribe_otf_capability.  */
 static Lisp_Object otf_features (HDC context, char *table);
@@ -84,7 +84,7 @@ static Lisp_Object
 uniscribe_list_family (struct frame *f)
 {
   Lisp_Object list = Qnil;
-  LOGFONT font_match_pattern;
+  LOGFONTW font_match_pattern;
   HDC dc;
 
   memset (&font_match_pattern, 0, sizeof (font_match_pattern));
@@ -93,8 +93,8 @@ uniscribe_list_family (struct frame *f)
 
   dc = get_frame_dc (f);
 
-  EnumFontFamiliesEx (dc, &font_match_pattern,
-                      (FONTENUMPROC) add_opentype_font_name_to_list,
+  EnumFontFamiliesExW (dc, &font_match_pattern,
+                      (FONTENUMPROCW) add_opentype_font_name_to_list,
                       (LPARAM) &list, 0);
   release_frame_dc (f, dc);
 
@@ -604,12 +604,12 @@ uniscribe_encode_char (struct font *font, int c)
 */
 
 \f
-/* Callback function for EnumFontFamiliesEx.
+/* Callback function for EnumFontFamiliesExW.
    Adds the name of opentype fonts to a Lisp list (passed in as the
    lParam arg). */
 static int CALLBACK ALIGN_STACK
-add_opentype_font_name_to_list (ENUMLOGFONTEX *logical_font,
-				NEWTEXTMETRICEX *physical_font,
+add_opentype_font_name_to_list (ENUMLOGFONTEXW *logical_font,
+				NEWTEXTMETRICEXW *physical_font,
 				DWORD font_type, LPARAM list_object)
 {
   Lisp_Object* list = (Lisp_Object *) list_object;
@@ -680,7 +680,7 @@ add_opentype_font_name_to_list (ENUMLOGFONTEX *logical_font,
    OTF_SPEC is in the format
      (script lang [(gsub_feature ...)|nil] [(gpos_feature ...)]?) */
 int
-uniscribe_check_otf (LOGFONT *font, Lisp_Object otf_spec)
+uniscribe_check_otf (LOGFONTW *font, Lisp_Object otf_spec)
 {
   Lisp_Object script, lang, rest;
   Lisp_Object features[2];
@@ -724,7 +724,7 @@ uniscribe_check_otf (LOGFONT *font, Lisp_Object otf_spec)
   /* Set up graphics context so we can use the font.  */
   f = XFRAME (selected_frame);
   context = get_frame_dc (f);
-  check_font = CreateFontIndirect (font);
+  check_font = CreateFontIndirectW (font);
   old_font = SelectObject (context, check_font);
 
   /* Everything else is contained within otf_spec so should get

bug#19910: 24.4; Japanese font names are decoded incorrectly in Cygwin's emacs-w32 in LANG=ja_JP.UTF-8

[Eli Zaretskii]

> Date: Sat, 28 Feb 2015 00:22:00 +0900
> From: Fujii Hironori <fujii.hironori@gmail.com>
> Cc: 19910@debbugs.gnu.org
> 
> On Fri, Feb 20, 2015 at 8:21 PM, Eli Zaretskii <eliz@gnu.org> wrote:
> > The problem is in w32font.c: it should call the "wide" (a.k.a.
> > "Unicode") APIs, and then decode strings using utf-16le, like we do in
> > w32fns.c with encoding strings we pass to w32 GUI APIs.
> 
> Unicode API patch is attached. Could you review it?
> Should I use GetProcAddress for Windows 9x?

Thanks.

However, this goes too far: there's no need to replace all the
functions with "wide" versions, only those functions that return font
name strings from the system.  For example, I don't think
CreateFontIndirect needs to be switched to Unicode, does it?  And CRT
functions like _wcslwr and swprintf that work on wchar_t arguments
aren't supported on Windows 9X, AFAIK, so we cannot call them.  (One
reason for using the minimum number of "wide" APIs is that we don't
have good ways of testing the development code on Windows 9X.)

And yes, for Windows 9X you will need to call these functions through
function pointers, after assigning them with GetProcAddress, as
w32font.c does elsewhere.

I would actually suggest to have a Cygwin-only branches of the code,
where you can freely call the "wide" APIs without bothering about
Windows 9X, since that's what the Cygwin-w32 build does elsewhere, and
since this is a Cygwin-specific problem due to the difference between
file-name encoding and the locale emulated by Cygwin.  There are a
bunch of macros like GUI_STR and GUI_ENCODE_FILE near the end of
w32term.h that can be used to minimize #ifdef's to the absolute
minimum.

bug#19910: 24.4; Japanese font names are decoded incorrectly in Cygwin's emacs-w32 in LANG=ja_JP.UTF-8

[Fujii Hironori]

Thank you for reviewing my patch, Eli.

On Sat, Feb 28, 2015 at 1:03 AM, Eli Zaretskii <eliz@gnu.org> wrote:
> However, this goes too far: there's no need to replace all the
> functions with "wide" versions, only those functions that return font
> name strings from the system.  For example, I don't think
> CreateFontIndirect needs to be switched to Unicode, does it?  And CRT
> functions like _wcslwr and swprintf that work on wchar_t arguments
> aren't supported on Windows 9X, AFAIK, so we cannot call them.  (One
> reason for using the minimum number of "wide" APIs is that we don't
> have good ways of testing the development code on Windows 9X.)

This is the code:

|    862  hfont = CreateFontIndirect (&logfont);
| (...)
|    912 = DECODE_SYSTEM (build_string (logfont.lfFaceName));

logfont.lfFaceName is ANSI text and DECODE_SYSTEM is the problem.
CreateFontIndirect should be wide.

> I would actually suggest to have a Cygwin-only branches of the code,
> where you can freely call the "wide" APIs without bothering about
> Windows 9X, since that's what the Cygwin-w32 build does elsewhere, and
> since this is a Cygwin-specific problem due to the difference between
> file-name encoding and the locale emulated by Cygwin.  There are a
> bunch of macros like GUI_STR and GUI_ENCODE_FILE near the end of
> w32term.h that can be used to minimize #ifdef's to the absolute
> minimum.

If this approach is used, structs such as LOGFONT and ENUMLOGFONTEX
should be ranemed to GUI_FN(LOGFONT) and GUI_FN(ENUMLOGFONTEX).
This looks ugly.

The best way to solve this is defining _UNICODE.
Defining _UNICODE is already filed, but closed as wontfix.

#265 - Build error with _UNICODE on w32. - GNU bug report logs
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=265

If Bug#265 is resolved, this bug (Bug#19910) will be resolved automatically.
And, _UNICODE macro can be used not only for Cygwin, but also NTEmacs.

bug#19910: 24.4; Japanese font names are decoded incorrectly in Cygwin's emacs-w32 in LANG=ja_JP.UTF-8

[Eli Zaretskii]

> Date: Sat, 28 Feb 2015 21:14:00 +0900
> From: Fujii Hironori <fujii.hironori@gmail.com>
> Cc: 19910@debbugs.gnu.org
> 
> > I would actually suggest to have a Cygwin-only branches of the code,
> > where you can freely call the "wide" APIs without bothering about
> > Windows 9X, since that's what the Cygwin-w32 build does elsewhere, and
> > since this is a Cygwin-specific problem due to the difference between
> > file-name encoding and the locale emulated by Cygwin.  There are a
> > bunch of macros like GUI_STR and GUI_ENCODE_FILE near the end of
> > w32term.h that can be used to minimize #ifdef's to the absolute
> > minimum.
> 
> If this approach is used, structs such as LOGFONT and ENUMLOGFONTEX
> should be ranemed to GUI_FN(LOGFONT) and GUI_FN(ENUMLOGFONTEX).
> This looks ugly.

We use it in quite a few places in Emacs, so ugly or not, this is a
kind of de-facto standard for resolving these issues.  More
importantly, it doesn't run the risk of breaking Emacs on Windows 9X.

> The best way to solve this is defining _UNICODE.
> Defining _UNICODE is already filed, but closed as wontfix.
> 
> #265 - Build error with _UNICODE on w32. - GNU bug report logs
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=265
> 
> If Bug#265 is resolved, this bug (Bug#19910) will be resolved automatically.
> And, _UNICODE macro can be used not only for Cygwin, but also NTEmacs.

Most, if not all, of the issues which could motivate someone to use
_UNICODE were meanwhile fixed, so reviving that now makes very little
sense.  In particular, the native Windows build already uses the
Unicode APIs wherever feasible.  (The particular issue discussed in
this thread doesn't exist in the native build, AFAIU, because
DECODE_SYSTEM does its job there.)

Thanks.

bug#19910: 24.4; Japanese font names are decoded incorrectly in Cygwin's emacs-w32 in LANG=ja_JP.UTF-8

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Fri, 20 Feb 2015 19:39:55 +0900
>> From: Fujii Hironori <fujii.hironori@gmail.com>
>> 
>> (font-family-list) returns incorrectly decoded Japanese font names.
>> My locale-coding-system is utf-8-unix.
>> 
>> If I do (setq locale-coding-system 'cp932), it returns correct font names.
>> But, locale-coding-system is used in other places (e.g. M-x term and M-x man).
>> locale-coding-system must be utf-8 in my Emacs.
>
> The problem is in w32font.c: it should call the "wide" (a.k.a.
> "Unicode") APIs, and then decode strings using utf-16le, like we do in
> w32fns.c with encoding strings we pass to w32 GUI APIs.

That was 5 years ago.  Is any of this still an issue on recent
versions of Emacs?

Best regards,
Stefan Kangas

bug#19910: 24.4; Japanese font names are decoded incorrectly in Cygwin's emacs-w32 in LANG=ja_JP.UTF-8

[Eli Zaretskii]

> From: Stefan Kangas <stefan@marxist.se>
> Cc: Fujii Hironori <fujii.hironori@gmail.com>,  19910@debbugs.gnu.org
> Date: Sun, 01 Dec 2019 09:21:56 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Date: Fri, 20 Feb 2015 19:39:55 +0900
> >> From: Fujii Hironori <fujii.hironori@gmail.com>
> >> 
> >> (font-family-list) returns incorrectly decoded Japanese font names.
> >> My locale-coding-system is utf-8-unix.
> >> 
> >> If I do (setq locale-coding-system 'cp932), it returns correct font names.
> >> But, locale-coding-system is used in other places (e.g. M-x term and M-x man).
> >> locale-coding-system must be utf-8 in my Emacs.
> >
> > The problem is in w32font.c: it should call the "wide" (a.k.a.
> > "Unicode") APIs, and then decode strings using utf-16le, like we do in
> > w32fns.c with encoding strings we pass to w32 GUI APIs.
> 
> That was 5 years ago.  Is any of this still an issue on recent
> versions of Emacs?

I don't think anything's changed in that department, so the problem
should still be there.

However, I have an idea of a much simpler fix for this, but I need a
volunteer who has this problem to test a patch I'd like to write to
fix this.  Anyone?