Re: [Emacs-diffs] trunk r114483: * term/ns-win.el (ns-initialize-window-system): Set locale-coding-system

Re: [Emacs-diffs] trunk r114483: * term/ns-win.el (ns-initialize-window-system): Set locale-coding-system

[Stefan Monnier]

> --- a/lisp/term/ns-win.el	2013-09-18 04:36:34 +0000
> +++ b/lisp/term/ns-win.el	2013-09-29 17:09:39 +0000
> @@ -912,6 +912,13 @@
>    ;; FIXME: This will surely lead to "MODIFIED OUTSIDE CUSTOM" warnings.
>    (menu-bar-mode (if (get-lisp-resource nil "Menus") 1 -1))
 
> +  ;; For NS nothing except UTF-8 makes sense.
> +  (add-hook 'before-init-hook
> +            #'(lambda ()
> +                (setq locale-coding-system 'utf-8-unix)
> +                (setq default-process-coding-system
> +                      '(utf-8-unix . utf-8-unix))))
> +
>    ;; OS X Lion introduces PressAndHold, which is unsupported by this port.
>    ;; See this thread for more details:
>    ;; http://lists.gnu.org/archive/html/emacs-devel/2011-06/msg00505.html

Does this apply to any Darwin build and only to Darwin builds?
What about tty-only Darwin builds?  And what about GNUstep builds?


        Stefan

Re: [Emacs-diffs] trunk r114483: * term/ns-win.el (ns-initialize-window-system): Set locale-coding-system

[Jan Djärv]

Hello. 

30 sep 2013 kl. 20:36 skrev Stefan Monnier <monnier@IRO.UMontreal.CA>:

> 
> Does this apply to any Darwin build and only to Darwin builds?

No. 

> What about tty-only Darwin builds?

What about them?  They can't be started from the Dock. 

> And what about GNUstep builds?

Applies to them. 

   Jan D. 

Re: [Emacs-diffs] trunk r114483: * term/ns-win.el (ns-initialize-window-system): Set locale-coding-system

[Stefan Monnier]

>> What about tty-only Darwin builds?
> What about them?  They can't be started from the Dock. 

But they should also use utf-8 for locale-coding-system and friends,
regardless of LANG settings, IIUC.

>> And what about GNUstep builds?
> Applies to them.

But should it?

IIUC this "utf-8 everywhere" is a property of the OS (Darwin) not of the
GUI/toolkit.  So it should apply to tty-only uses under Darwin and it
should not apply to GNUstep.


        Stefan

Re: [Emacs-diffs] trunk r114483: * term/ns-win.el (ns-initialize-window-system): Set locale-coding-system

[Jan Djärv]

Hello.

1 okt 2013 kl. 00:38 skrev Stefan Monnier <monnier@iro.umontreal.ca>:

>>> What about tty-only Darwin builds?
>> What about them?  They can't be started from the Dock. 
> 
> But they should also use utf-8 for locale-coding-system and friends,
> regardless of LANG settings, IIUC.

No they should not IMHO.  As they must be started from a teminal, LANG is either correct or set to another value for some purpose.

> 
>>> And what about GNUstep builds?
>> Applies to them.
> 
> But should it?
> 
> IIUC this "utf-8 everywhere" is a property of the OS (Darwin) not of the
> GUI/toolkit.  So it should apply to tty-only uses under Darwin and it
> should not apply to GNUstep.
> 

Makes sense, fixed in trunk.

	Jan D.

Re: [Emacs-diffs] trunk r114483: * term/ns-win.el (ns-initialize-window-system): Set locale-coding-system

[Stefan Monnier]

> No they should not IMHO.  As they must be started from a teminal, LANG is
> either correct or set to another value for some purpose.

IIUC the encoding used under Darwin for filenames is always utf-8.
More to the point it uses NFD canonicalization, performed by the OS when
needed; so pretending these are "byte sequences" that can use any
encoding you feel like, as is the case in POSIX, is not an option.

I guess it's OK to obey the user who wants to shot himself in the foot.
But I do wonder: is LANG really always set in a terminal?  IF so, why
and by whom?  Will LANG also be set properly if you use some other
terminal than Terminal.app (e.g. let's say you run `emacs -nw' inside
an Emacs.app started from the GUI)?

>> IIUC this "utf-8 everywhere" is a property of the OS (Darwin) not of the
>> GUI/toolkit.  So it should apply to tty-only uses under Darwin and it
>> should not apply to GNUstep.
> Makes sense, fixed in trunk.

Thanks,


        Stefan

Re: [Emacs-diffs] trunk r114483: * term/ns-win.el (ns-initialize-window-system): Set locale-coding-system

[Stephen J. Turnbull]

Stefan Monnier writes:

 > IIUC the encoding used under Darwin for filenames is always utf-8.

In practice, yes, in theory, no.  *Darwin* is just a BSD *nix kernel
(more precisely, FreeBSD kernel functions wrapped around a Mach
microkernel), and at that level, they're just byte sequences.  Mac OS
X defaults to a file system which enforces UTF-8, that's all.  But
consider NFS mounts (whatever the remote system wants) and removable
media (most often VFAT with some bogus Microsoft encoding).

 > More to the point it uses NFD canonicalization, performed by the OS
 > when needed;

Sort of true, for values of "when needed" == "when the actual
filesystem is HFS+".

Re: [Emacs-diffs] trunk r114483: * term/ns-win.el (ns-initialize-window-system): Set locale-coding-system

[Stefan Monnier]

>> IIUC the encoding used under Darwin for filenames is always utf-8.
> In practice, yes, in theory, no.  *Darwin* is just a BSD *nix kernel
> (more precisely, FreeBSD kernel functions wrapped around a Mach
> microkernel), and at that level, they're just byte sequences.  Mac OS
> X defaults to a file system which enforces UTF-8, that's all.  But
> consider NFS mounts (whatever the remote system wants) and removable
> media (most often VFAT with some bogus Microsoft encoding).

Good to know, thanks.  I had the impression that the utf-8 coding was
imposed at the API level rather than the filesystem level.


        Stefan

Re: [Emacs-diffs] trunk r114483: * term/ns-win.el (ns-initialize-window-system): Set locale-coding-system

[Andreas Schwab]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Good to know, thanks.  I had the impression that the utf-8 coding was
> imposed at the API level rather than the filesystem level.

The HFS+ filesystem actually uses UTF-16.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: [Emacs-diffs] trunk r114483: * term/ns-win.el (ns-initialize-window-system): Set locale-coding-system

[Stephen J. Turnbull]

Andreas Schwab writes:
 > Stefan Monnier <monnier@iro.umontreal.ca> writes:
 > 
 > > Good to know, thanks.  I had the impression that the utf-8 coding was
 > > imposed at the API level rather than the filesystem level.
 > 
 > The HFS+ filesystem actually uses UTF-16.

Sure, but the kernel VFS functions expect Unicode in UTF-8 format, and
output NFD.  I think that's what's relevant here, rather than what
happens inside the HFS+ drivers and on the hardware.

Re: [Emacs-diffs] trunk r114483: * term/ns-win.el (ns-initialize-window-system): Set locale-coding-system

[Stefan Monnier]

>> Good to know, thanks.  I had the impression that the utf-8 coding was
>> imposed at the API level rather than the filesystem level.
> The HFS+ filesystem actually uses UTF-16.

So, IIUC this UTF-16 is only used "on disk" but is converted to UTF-8 in
the hfsplus driver so the rest of the kernel sees UTF-8 coming from
HFS+ partitions.


        Stefan

Re: [Emacs-diffs] trunk r114483: * term/ns-win.el (ns-initialize-window-system): Set locale-coding-system

[Jan Djärv]

Hello.

2 okt 2013 kl. 02:43 skrev Stefan Monnier <monnier@iro.umontreal.ca>:

>> No they should not IMHO.  As they must be started from a teminal, LANG is
>> either correct or set to another value for some purpose.
> 
> IIUC the encoding used under Darwin for filenames is always utf-8.
> More to the point it uses NFD canonicalization, performed by the OS when
> needed; so pretending these are "byte sequences" that can use any
> encoding you feel like, as is the case in POSIX, is not an option.

As pointed out, how about remote mounts?  I don't know.

> 
> I guess it's OK to obey the user who wants to shot himself in the foot.
> But I do wonder: is LANG really always set in a terminal?  IF so, why
> and by whom?  Will LANG also be set properly if you use some other
> terminal than Terminal.app (e.g. let's say you run `emacs -nw' inside
> an Emacs.app started from the GUI)?

It is Terminal.app that sets it based on system settings for language.
For other emulators, I don't know.  I don't know of any other emulators BTW.
For emacs -nw inside emacs started from GUI, LANG will not be set.

But that goes for many X11 setups as well. If LANG is set in /etc/profile or ~/.profile, it won't be set if you start Emacs from a panel and the parent session manager has not LANG set.


	Jan D.

Re: [Emacs-diffs] trunk r114483: * term/ns-win.el (ns-initialize-window-system): Set locale-coding-system

[Stefan Monnier]

> But that goes for many X11 setups as well. If LANG is set in /etc/profile or
> ~/.profile, it won't be set if you start Emacs from a panel and the parent
> session manager has not LANG set.

So maybe the better solution is to default to utf-8 if/when LANG is not set,
rather than try to distinguish the GUI-vs-tty case.


        Stefan

Re: [Emacs-diffs] trunk r114483: * term/ns-win.el (ns-initialize-window-system): Set locale-coding-system

[Jan Djärv]

Hi.

3 okt 2013 kl. 03:53 skrev Stefan Monnier <monnier@iro.umontreal.ca>:

>> But that goes for many X11 setups as well. If LANG is set in /etc/profile or
>> ~/.profile, it won't be set if you start Emacs from a panel and the parent
>> session manager has not LANG set.
> 
> So maybe the better solution is to default to utf-8 if/when LANG is not set,
> rather than try to distinguish the GUI-vs-tty case.
> 

... and LC_ALL and LC_CTYPE.  I think it is too much effort for practcally no benefit.

	Jan D.