bug#35671: 'glibc-utf8-locales' should include C.UTF-8 locale

bug#35671: 'glibc-utf8-locales' should include C.UTF-8 locale

[Ludovic Courtès]

Hello,

I noticed that on Debian 9, ‘guix-daemon.service’ (systemd) runs under
the C.UTF-8 locale by default, even if the machine is otherwise
configured with another locale.  Consequently, ‘guix substitute’ keeps
complaining about locales and all that, which is terrible.

Thus, I think ‘glibc-utf8-locales’ should provide that locale.

Ludo’.

bug#35671: 'glibc-utf8-locales' should include C.UTF-8 locale

[Meiyo Peng]

Hi Ludovic,

Ludovic Courtès writes:

> I noticed that on Debian 9, ‘guix-daemon.service’ (systemd) runs under
> the C.UTF-8 locale by default, even if the machine is otherwise
> configured with another locale.  Consequently, ‘guix substitute’ keeps
> complaining about locales and all that, which is terrible.
>
> Thus, I think ‘glibc-utf8-locales’ should provide that locale.

And the zh_CN.UTF-8, zh_TW.UTF-8 please.


--
Meiyo Peng
https://www.pengmeiyu.com/

bug#35671: 'glibc-utf8-locales' should include C.UTF-8 locale

[Ludovic Courtès]

Hello,

Ludovic Courtès <ludovic.courtes@inria.fr> skribis:

> I noticed that on Debian 9, ‘guix-daemon.service’ (systemd) runs under
> the C.UTF-8 locale by default, even if the machine is otherwise
> configured with another locale.  Consequently, ‘guix substitute’ keeps
> complaining about locales and all that, which is terrible.

It turns out that the “C.UTF-8” locale doesn’t exist upstream in glibc:

  https://bugzilla.redhat.com/show_bug.cgi?id=902094

But anyway, I think we should just set “LC_ALL=en_US.utf8” in the
.service file: that’s what we do on Guix System, and the choice of a
locale doesn’t matter since ‘guix substitute’ honors the client’s
locale.

Meiyo Peng <meiyo@riseup.net> skribis:

> Ludovic Courtès writes:

[...]

>> Thus, I think ‘glibc-utf8-locales’ should provide that locale.
>
> And the zh_CN.UTF-8, zh_TW.UTF-8 please.

‘glibc-utf8-locales’ has always been described as an *arbitrary* sample
of UTF-8 locales (info "(guix) Application Setup").  Initially its main
purpose was to use it in build processes, but we’ve come to more or less
recommend it for users, which is not great because we know it only works
for some users.

I built a ‘glibc-utf8-locales’ package that provides all the supported
UTF-8 locales, and only UTF-8 locales:

--8<---------------cut here---------------start------------->8---
$ guix size /gnu/store/rh7iq1kwma7ir96mvzvqcg1v50yi05yp-glibc-utf8-locales-2.28
store item                                                       total    self
/gnu/store/rh7iq1kwma7ir96mvzvqcg1v50yi05yp-glibc-utf8-locales-2.28   855.7   855.7 100.0%
total: 855.7 MiB
$ ls -1d /gnu/store/rh7iq1kwma7ir96mvzvqcg1v50yi05yp-glibc-utf8-locales-2.28/lib/locale/2.28/*.utf8 |wc -l
312
$ guix size glibc-locales
store item                                                       total    self
/gnu/store/acl2wxzzkkcjv74rlqswdf9p8pwddlmk-glibc-locales-2.28     916.7   916.7 100.0%
total: 916.7 MiB
--8<---------------cut here---------------end--------------->8---

As you can see, it’s almost the same size as ‘glibc-locales’, so there’s
no point in having such a ‘glibc-utf8-locales’ package.

In Guix System, the (gnu system locale) module generates just the needed
locales.  I’m thinking we should do the same at the ‘guix package’
level.  For example, ‘guix package --install-locales’ would
automatically install whatever $LANG or $LC_* refers to.

Thoughts?

Thanks,
Ludo’.

bug#35671: 'glibc-utf8-locales' should include C.UTF-8 locale

[Ludovic Courtès]

Hi!

Ludovic Courtès <ludo@gnu.org> skribis:

> Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
>
>> I noticed that on Debian 9, ‘guix-daemon.service’ (systemd) runs under
>> the C.UTF-8 locale by default, even if the machine is otherwise
>> configured with another locale.  Consequently, ‘guix substitute’ keeps
>> complaining about locales and all that, which is terrible.
>
> It turns out that the “C.UTF-8” locale doesn’t exist upstream in glibc:
>
>   https://bugzilla.redhat.com/show_bug.cgi?id=902094
>
> But anyway, I think we should just set “LC_ALL=en_US.utf8” in the
> .service file: that’s what we do on Guix System, and the choice of a
> locale doesn’t matter since ‘guix substitute’ honors the client’s
> locale.

Done in 489d6c0dc03f1d95a3fefa6301f36cac9b50b2c3.

I got the syntax from
<https://www.freedesktop.org/software/systemd/man/systemd.service.html>
but I wouldn’t mind if someone would double-check!

Thanks,
Ludo’.

bug#35671: 'glibc-utf8-locales' should include C.UTF-8 locale

[Meiyo Peng]

Hi Ludovic,

Ludovic Courtès writes:

> But anyway, I think we should just set “LC_ALL=en_US.utf8” in the
> .service file: that’s what we do on Guix System, and the choice of a
> locale doesn’t matter since ‘guix substitute’ honors the client’s
> locale.

That's what I did to my guix-daemon.service and guix-publish.service
files on a foreign distro after getting locale warnings.

> Meiyo Peng <meiyo@riseup.net> skribis:
>
>> Ludovic Courtès writes:
>
> [...]
>
>>> Thus, I think ‘glibc-utf8-locales’ should provide that locale.
>>
>> And the zh_CN.UTF-8, zh_TW.UTF-8 please.
>
> ‘glibc-utf8-locales’ has always been described as an *arbitrary* sample
> of UTF-8 locales (info "(guix) Application Setup").  Initially its main
> purpose was to use it in build processes, but we’ve come to more or less
> recommend it for users, which is not great because we know it only works
> for some users.

I was surprised last week (maybe the week before that) when I found out
that glibc-utf8-locales only provides locales for de_DE, el_GR, en_US,
fr_FR, tr_TR.  And in our manual, we tell people to install
glibc-utf8-locales after installing Guix on a foreign distro in order to
solve locale problems.  The name "glibc-utf8-locales" indicates it
provides ALL UTF-8 locales while in fact it does not.  This caused me a
trouble because I have to set the locale of Emacs to zh_CN.UTF-8 in
order to enable the ibus input method in Emacs.  (This is a known
problem of Emacs.  Locale has to be set to one of CJK locale in order to
enable external input methods.)  Since the zh_CN.UTF-8 locale is not in
$GUIX_LOCPATH, ibus does not work in Emacs.  I ended up installing the
"glibc-locales" package and solved the problem.

> I built a ‘glibc-utf8-locales’ package that provides all the supported
> UTF-8 locales, and only UTF-8 locales:
>
> --8<---------------cut here---------------start------------->8---
> $ guix size /gnu/store/rh7iq1kwma7ir96mvzvqcg1v50yi05yp-glibc-utf8-locales-2.28
> store item                                                       total    self
> /gnu/store/rh7iq1kwma7ir96mvzvqcg1v50yi05yp-glibc-utf8-locales-2.28   855.7   855.7 100.0%
> total: 855.7 MiB
> $ ls -1d /gnu/store/rh7iq1kwma7ir96mvzvqcg1v50yi05yp-glibc-utf8-locales-2.28/lib/locale/2.28/*.utf8 |wc -l
> 312
> $ guix size glibc-locales
> store item                                                       total    self
> /gnu/store/acl2wxzzkkcjv74rlqswdf9p8pwddlmk-glibc-locales-2.28     916.7   916.7 100.0%
> total: 916.7 MiB
> --8<---------------cut here---------------end--------------->8---
>
> As you can see, it’s almost the same size as ‘glibc-locales’, so there’s
> no point in having such a ‘glibc-utf8-locales’ package.
>
> In Guix System, the (gnu system locale) module generates just the needed
> locales.  I’m thinking we should do the same at the ‘guix package’
> level.  For example, ‘guix package --install-locales’ would
> automatically install whatever $LANG or $LC_* refers to.
>
> Thoughts?

I think it would be better to simply tell people to install the
"glibc-locales" package.  There may be a case where different
applications are set to different locales.  For example, my system is
set to the en_US.UTF-8 locale but my Emacs is set to the zh_CN.UTF-8
locale.  So $LC_* only refers to en_US.UTF-8 if guix tries to detect the
locale.  If ‘guix package --install-locales’ only install what $LANG or
$LC_* refers to, zh_CN.UTF-8 won't be installed for me.

We are unable to detect what locales a user will need later.  Installing
a full package like "glibc-locales" or letting the user explicitly
specify what locales they need can avoid a lot of special problems.


--
Meiyo Peng
https://www.pengmeiyu.com/

bug#35671: 'glibc-utf8-locales' should include C.UTF-8 locale

[Ludovic Courtès]

Hi,

Meiyo Peng <meiyo@riseup.net> skribis:

> I was surprised last week (maybe the week before that) when I found out
> that glibc-utf8-locales only provides locales for de_DE, el_GR, en_US,
> fr_FR, tr_TR.  And in our manual, we tell people to install
> glibc-utf8-locales after installing Guix on a foreign distro in order to
> solve locale problems.  The name "glibc-utf8-locales" indicates it
> provides ALL UTF-8 locales while in fact it does not.  This caused me a
> trouble because I have to set the locale of Emacs to zh_CN.UTF-8 in
> order to enable the ibus input method in Emacs.  (This is a known
> problem of Emacs.  Locale has to be set to one of CJK locale in order to
> enable external input methods.)  Since the zh_CN.UTF-8 locale is not in
> $GUIX_LOCPATH, ibus does not work in Emacs.  I ended up installing the
> "glibc-locales" package and solved the problem.

To be fair, the manual has always recommended ‘glibc-locales’, not
‘glibc-utf8-locales’, and the latter is explicitly described as “limited
to a few UTF-8 locales”:

  https://www.gnu.org/software/guix/manual/en/html_node/Application-Setup.html

The hint in (guix ui) is less clear, though.

> I think it would be better to simply tell people to install the
> "glibc-locales" package.  There may be a case where different
> applications are set to different locales.  For example, my system is
> set to the en_US.UTF-8 locale but my Emacs is set to the zh_CN.UTF-8
> locale.  So $LC_* only refers to en_US.UTF-8 if guix tries to detect the
> locale.  If ‘guix package --install-locales’ only install what $LANG or
> $LC_* refers to, zh_CN.UTF-8 won't be installed for me.

That command could take a parameter, of course.

See also:
<https://lists.gnu.org/archive/html/guix-devel/2019-05/msg00285.html>.

Thanks for your feedback!

Ludo’.

bug#35671: 'glibc-utf8-locales' should include C.UTF-8 locale

[Meiyo Peng]

Hi Ludovic,

Ludovic Courtès writes:

> Meiyo Peng <meiyo@riseup.net> skribis:
>
>> I was surprised last week (maybe the week before that) when I found out
>> that glibc-utf8-locales only provides locales for de_DE, el_GR, en_US,
>> fr_FR, tr_TR.  And in our manual, we tell people to install
>> glibc-utf8-locales after installing Guix on a foreign distro in order to
>> solve locale problems.  The name "glibc-utf8-locales" indicates it
>> provides ALL UTF-8 locales while in fact it does not.  This caused me a
>> trouble because I have to set the locale of Emacs to zh_CN.UTF-8 in
>> order to enable the ibus input method in Emacs.  (This is a known
>> problem of Emacs.  Locale has to be set to one of CJK locale in order to
>> enable external input methods.)  Since the zh_CN.UTF-8 locale is not in
>> $GUIX_LOCPATH, ibus does not work in Emacs.  I ended up installing the
>> "glibc-locales" package and solved the problem.
>
> To be fair, the manual has always recommended ‘glibc-locales’, not
> ‘glibc-utf8-locales’, and the latter is explicitly described as “limited
> to a few UTF-8 locales”:
>
>   https://www.gnu.org/software/guix/manual/en/html_node/Application-Setup.html

You are right.  My mistake.  But when I saw the text:

#+begin_quote
  Note that the glibc-locales package contains data for all the locales
  supported by the GNU libc and weighs in at around 110 MiB.
  Alternatively, the glibc-utf8-locales is smaller but limited to a
  few UTF-8 locales.
#+end_quote

I thought the glibc-utf8-locales should satisfy my requirements since
all my locales are in UTF-8, and it's preferred if disk space is
limited.  The name of glibc-utf8-locales is misleading.  In fact, the
"few" UTF-8 locales it provides are too few.  We'd better modify the
text a little bit if we are going to reach a broader user base.

> The hint in (guix ui) is less clear, though.
>
>> I think it would be better to simply tell people to install the
>> "glibc-locales" package.  There may be a case where different
>> applications are set to different locales.  For example, my system is
>> set to the en_US.UTF-8 locale but my Emacs is set to the zh_CN.UTF-8
>> locale.  So $LC_* only refers to en_US.UTF-8 if guix tries to detect the
>> locale.  If ‘guix package --install-locales’ only install what $LANG or
>> $LC_* refers to, zh_CN.UTF-8 won't be installed for me.
>
> That command could take a parameter, of course.

That makes sense.


--
Meiyo Peng
https://www.pengmeiyu.com/

bug#35671: guix-daemon cannot find UTF-8 locale out-of-the-box on foreign distros

[Maxim Cournoyer]

Hello,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi!
>
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Ludovic Courtès <ludovic.courtes@inria.fr> skribis:
>>
>>> I noticed that on Debian 9, ‘guix-daemon.service’ (systemd) runs under
>>> the C.UTF-8 locale by default, even if the machine is otherwise
>>> configured with another locale.  Consequently, ‘guix substitute’ keeps
>>> complaining about locales and all that, which is terrible.
>>
>> It turns out that the “C.UTF-8” locale doesn’t exist upstream in glibc:
>>
>>   https://bugzilla.redhat.com/show_bug.cgi?id=902094
>>
>> But anyway, I think we should just set “LC_ALL=en_US.utf8” in the
>> .service file: that’s what we do on Guix System, and the choice of a
>> locale doesn’t matter since ‘guix substitute’ honors the client’s
>> locale.
>
> Done in 489d6c0dc03f1d95a3fefa6301f36cac9b50b2c3.
>
> I got the syntax from
> <https://www.freedesktop.org/software/systemd/man/systemd.service.html>
> but I wouldn’t mind if someone would double-check!

Closing, then :-).

Maxim