bug#36076: Manual should clarify that glibc-utf8-locales is needed by default on foreign distros

[Giovanni Biscuolo <g@xelera.eu>]

[-- Attachment #1: Type: text/plain, Size: 5771 bytes --]

Hello Jack,

AFAIU you have found a bug, but not in the manual :-)

The manual states that glibc-utf8-locales is an alternative to
glibc-locales, limited to a few UTF-8 locales; this is how it should be.

The issue is that glibc-locales package does not create symlinks to the
"normalized codeset" like glibc-utf8-locales does (see below for
details): **all* normalized codes like en_US.utf8 are missing.

If I'm right we should change the issue subject to something like
"glibc-locales does not generate normalized codesets", but I'm
not used to debbugs so I'll let this part to others **if applicable**

As a temporary (or permanent :-) ) workaround you could add this in the
env of guix-daemon adding this line to
"/etc/systemd/system/guix-daemon.service":

--8<---------------cut here---------------start------------->8---
Environment=LC_ALL=en_US.UTF-8
--8<---------------cut here---------------end--------------->8---

You should also use en_US.UTF-8 instead of en_US.utf8 in your user env.

Jack Hill <jackhill@jackhill.us> writes:

[...]

>> --8<---------------cut here---------------start------------->8---
>> $ ls -lah /var/guix/profiles/per-user/giovanni/guix-profile/lib/locale/2.28/ | grep en_US
>> lrwxrwxrwx 33 root root   84 Jan  1  1970 en_US -> /gnu/store/acl2wxzzkkcjv74rlqswdf9p8pwddlmk-glibc-locales-2.28/lib/locale/2.28/en_US
>> lrwxrwxrwx 33 root root   94 Jan  1  1970 en_US.utf8 -> /gnu/store/94k5w17z54w25lgp90czdqfv9m4hwzhq-glibc-utf8-locales-2.28/lib/locale/2.28/en_US.utf8
>> dr-xr-xr-x  3 root root 4.0K Jan  1  1970 en_US.UTF-8
>> --8<---------------cut here---------------end--------------->8---
>>
>> So AFAIU glibc-locales provides both en_US.UTF-8 and en_US.utf8 that is
>> used by guix-daemon.
>
> That's not how I read the above lines. It looks like en_US.utf8 is a link 
> to the 94k5…glib-utf8-locales-2.28 store item.

You are right, I overlooked that: I removed glibc-utf8-locales from my
user profile, I just have glibc-locales now:

--8<---------------cut here---------------start------------->8---
~$ ls -lah /var/guix/profiles/per-user/giovanni/guix-profile/lib/locale/2.28/ | grep en_US
dr-xr-xr-x   3 root root 4.0K Jan  1  1970 en_US
dr-xr-xr-x   3 root root 4.0K Jan  1  1970 en_US.UTF-8
--8<---------------cut here---------------end--------------->8---

"en_US.utf8" is missing; that "incidentally" is also the default locale
of my user, so:

--8<---------------cut here---------------start------------->8---
~$ guix package -I
guile: warning: failed to install locale
--8<---------------cut here---------------end--------------->8---

Setting "LANG=en_US.UTF-8" fixes the problem.

In fact make-glibc-locales in base.scm "just" installs all locales in
"not normalized codeset" with:

--8<---------------cut here---------------start------------->8---
[...]
             (replace 'build
               (lambda _
                 (invoke "make" "localedata/install-locales"
                         "-j" (number->string (parallel-job-count)))))
[...]
--8<---------------cut here---------------end--------------->8---

while make-glibc-utf8-locales generates a limited list locales via localedef in
"normalized codeset" and creates symlinks for "not normalized codeset"

--8<---------------cut here---------------start------------->8---
                     (for-each (lambda (locale)
                                 (define file
                                   ;; Use the "normalized codeset" by
                                   ;; default--e.g., "en_US.utf8".
                                   (string-append localedir "/" locale ".utf8"))

                                 (invoke "localedef" "--no-archive"
                                         "--prefix" localedir
                                         "-i" locale
                                         "-f" "UTF-8" file)

                                 ;; For backward compatibility with Guix
                                 ;; <= 0.8.3, add "xx_YY.UTF-8".
                                 (symlink (string-append locale ".utf8")
                                          (string-append localedir "/"
                                                         locale ".UTF-8")))

                               ;; These are the locales commonly used for
                               ;; tests---e.g., in Guile's i18n tests.
                               '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
--8<---------------cut here---------------end--------------->8---

I don't understand if "normalized codeset" are mandatory or facultative
(see info guix "Locales"), but AFAIU are missing from the making of
glibc localedata/install-locales; for sure they are used by default by a
lot of users in their locale env, and in the default locale of
guix-daemon.

AFAIU one possible workaround is to write a patch that normalizes [1]
all the codesets

--8<---------------cut here---------------start------------->8---
by applying the following rules:

1. Remove all characters besides numbers and letters.
2. Fold letters to lowercase.
3. If the same only contains digits prepend the string "iso".
--8<---------------cut here---------------end--------------->8---

and creates a symlink to not normalized ones

An alternative could be to create share/locale/locale.alias with
"normalized codes" aliases

Sorry I'm still not able to propose such patch for make-glibc-locales.

To deprecate "normalized codeset" is not an alternative IMHO

HTH! Gio'.

[...]

[1] https://www.gnu.org/software/libc/manual/html_node/Using-gettextized-software.html#Using-gettextized-software

-- 
Giovanni Biscuolo

Xelera IT Infrastructures

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]