bug#24162: 25.0.50; utf8 filenames with ornamented characters in MS-Windows

[Eli Zaretskii <eliz@gnu.org>]

[Please keep the bug address on the CC list.]

> From: "cschr" <cschr@freenet.de>
> Date: Sat, 6 Aug 2016 13:30:18 +0200
> 
> Honestly I am not an emacs expert yet - it could well be that my language
> configuration is not as it should be.   Maybe you could give me a hint how
> to set it, or direct me to the right web page ?

You shouldn't need to do anything, the defaults should "just work" for
you.  They do work for me.  However, using MSYS2 or Cygwin builds of
various utilities might be a problem, and could very well cause
incompatibilities.  The only way around that is to use native ports of
those utilities.

> Im indeed using the MSYS2 build of the GNU touch utility.

That doesn't tell enough about the nature of the binary.  What does
the following command display inside Emacs?

  M-: (w32-application-type (executable-find "touch")) RET

> But the strange thing is: I also downloaded and used emacs via MSYS2
> - and it had the same problem !

Emacs downloaded from MSYS2 is a native Windows program.  I'm not so
sure about touch.exe; the above command should tell.

> I had had an earlier issue with ornamented characters which was even worse:
> dired sometimes displayed files with ornamented characters incorrectly and
> was unable to open them (but this did not happen always - I never found out
> when it did and when not!).

Could it be that the files that cannot be displayed were created by
some external utilities?  Or by old versions of Emacs?  What does the
Windows Explorer show when you try to display the folders with those
file names?

> I had resolved this one successfully by putting
> the following into my init.el - since then the issue never occurred again:
>  (prefer-coding-system 'utf-8)
> (set-default-coding-systems 'utf-8)
> (set-terminal-coding-system 'utf-8)
> (set-keyboard-coding-system 'utf-8)
> ;; backwards compatibility as default-buffer-file-coding-system
> ;; is deprecated in 23.2.
> (if (boundp 'buffer-file-coding-system)
>     (setq-default buffer-file-coding-system 'utf-8)
>   (setq default-buffer-file-coding-system 'utf-8))
> ;; Treat clipboard input as UTF-8 string first; compound text next, etc.
> (setq x-select-request-type '(UTF8_STRING COMPOUND_TEXT TEXT STRING))

These are all wrong, you should remove them from your init file.  The
correct defaults for the various coding-systems are set up on Windows
automatically, and you shouldn't try changing them, as long as you are
using the native Windows build of Emacs.

> what I now just tried instead is: 
> 1. I took out all "...-coding-system" statements from my init.el and
> restarted emacs
> 2. I started dired and looked at the folder "täst.txt" - result: now the
> ornamented character is displayed uncorrectly and the file cannot be opened
> anymore

Can you display that folder in the Windows Explorer?  If so, does it
show correct file names?  If the file names are corrupted in the
Explorer, you have somehow created that folder with file names in the
wrong encoding.

> 3. I executed again > C:\Programme\Emacs-24.5-Win64\bin\emacsclientw.exe -n
> -a C:\Programme\Emacs-24.5-Win64\bin\runemacs.exe -c "täst.txt" - result:
> now this file opened properly

As expected.

> I now also did another test with a minimum init.el which just had
> "(server-start)" - everything worked fine - but this does not mean that the
> dired-problem is not there, because this one only occurs sometimes

As long as you use native Windows programs to create files, and as
long as your Emacs doesn't have those wrong settings of coding
systems, you should not see such problems.  At least not in Emacs 24.5
and later.

> So is this a config problem, and if yes:  can you give me a hint how to get
> the right setup in order to cover all 3 problems ?

The advice is (a) don't change any of the coding-systems in your init
file, and (b) use only native Windows utilities outside of Emacs to
create/modify files whose names include non-ASCII characters.