[bug#44075] [PATCH] gnu: Add make-glibc-locales-collection.

[bug#44075] [PATCH] gnu: Add make-glibc-locales-collection.

[Efraim Flashner]

* gnu/packages/base.scm (make-glibc-locales-collection): New macro.
(en_us-glibc-locales): New variable.
---
 gnu/packages/base.scm | 73 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 72 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index c83775d8ee..41d3aaf865 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -62,7 +62,8 @@
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
   #:export (glibc
-            libiconv-if-needed))
+            libiconv-if-needed
+            make-custom-glibc-locales))
 
 ;;; Commentary:
 ;;;
@@ -1106,6 +1107,69 @@ to the @code{share/locale} sub-directory of this package.")
                                         ,(version-major+minor
                                           (package-version glibc)))))))))))
 
+(define* (make-glibc-locales-collection
+           glibc
+           #:optional (locales
+                        '(list "en_US.utf8" "en_US.ISO-8859-1")))
+                        ;; This list for testing
+                        ;'(list "el_GR.UTF-8" "en_US.utf8" "he_IL.ISO-8859-8" "ja_JP.EUC-JP" "zh_CN.GB18030" "zh_CN.GBK" "hy_AM.ARMSCII-8")))
+  (package
+    (name "glibc-locales-collection")
+    (version (package-version glibc))
+    (source #f)
+    (build-system trivial-build-system)
+    (arguments
+     `(#:modules ((guix build utils))
+       #:builder
+       (begin
+         (use-modules (guix build utils))
+
+         (let* ((libc      (assoc-ref %build-inputs "glibc"))
+                (gzip      (assoc-ref %build-inputs "gzip"))
+                (out       (assoc-ref %outputs "out"))
+                (localedir (string-append out "/lib/locale/"
+                                          ,(version-major+minor version))))
+           ;; 'localedef' needs 'gzip'.
+           (setenv "PATH" (string-append libc "/bin:" gzip "/bin"))
+
+           (mkdir-p localedir)
+           (for-each
+             (lambda (locale)
+               (let* ((contains-dot? (string-index locale #\.))
+                      (encoding-type (substring locale (1+ contains-dot?)))
+                      (raw-locale    (substring locale 0 contains-dot?))
+                      (utf8? (or (number? (string-contains locale ".utf8"))
+                                 (number? (string-contains locale ".UTF-8"))))
+                      (file (if utf8?
+                              (string-append localedir "/" raw-locale ".utf8")
+                              (if (string-contains locale ".ISO")
+                                (string-append localedir "/" raw-locale)
+                                (string-append localedir "/" locale)))))
+
+                 (invoke "localedef" "--no-archive"
+                         "--prefix" localedir
+                         "-i" raw-locale
+                         "-f" (if (equal? "utf8" encoding-type)
+                                "UTF-8"
+                                encoding-type)
+                         file)
+
+                 ;; Is it utf8 or UTF-8? NO ONE KNOWS!
+                 (when utf8?
+                   (symlink (string-append raw-locale ".utf8")
+                            (string-append localedir "/"
+                                           raw-locale ".UTF-8")))))
+             ,locales)
+           #t))))
+    (native-inputs `(("glibc" ,glibc)
+                     ("gzip" ,gzip)))
+    (synopsis "Customizable collection of locales")
+    (description
+     "This package provides a custom collection of locales useful for
+providing exactly the locales requested when size matters.")
+    (home-page (package-home-page glibc))
+    (license (package-license glibc))))
+
 (define-public (make-glibc-utf8-locales glibc)
   (package
     (name "glibc-utf8-locales")
@@ -1161,6 +1225,13 @@ test environments.")
 (define-public glibc-utf8-locales
   (make-glibc-utf8-locales glibc))
 
+(define-public en_us-glibc-locales
+  (package
+    (inherit (make-glibc-locales-collection
+               glibc
+               '(list "en_US.utf8" "en_US.ISO-8859-1")))
+    (name "en-us-glibc-locales")))
+
 ;; Packages provided to ease use of binaries linked against the previous libc.
 (define-public glibc-locales-2.29
   (package (inherit (make-glibc-locales glibc-2.29))
-- 
2.28.0

[bug#44075] [PATCH] gnu: Add make-glibc-locales-collection.

[Miguel Ángel Arruga Vivas]

[-- Attachment #1: Type: text/plain, Size: 1273 bytes --]

Hi Efraim,

I've been taking a look into your patch.  One issue are the comments
about utf8 and UTF-8, as the issue is already explained in
make-glibc[-utf8]-locales.

Other point is:

Efraim Flashner <efraim@flashner.co.il> writes:
> +(define* (make-glibc-locales-collection
> +           glibc
> +           #:optional (locales
> +                        '(list "en_US.utf8" "en_US.ISO-8859-1")))
> (... Removed for clarity ...)
> +             ,locales)

I would have used list there like (list ,@locales) or '(,@locales), this
looks a bit odd to my eyes at least.  I'd expect this kind of calling code:

(let ((locales '("de_CH.utf8" ... "de_DE.utf8"))
      (my-glibc ...))
  (make-glibc-locales-collection myglibc locales))

Enforcing an extra quotation for no real reason on the calling site, as
strings are self-evaluating objects, and the use of the symbol list,
whose meaning depends on other context of execution, doesn't seem
necessary.  Even worse, my example would raise an error as "de_CH.utf8"
is not a procedure.

What do you think about replacing make-glibc-utf8-locales with a call of
the new function (using that code) ensuring that the generated
derivation stays the same for that case (i.e. it's optimized for the
UTF-8 case)?

Happy hacking!
Miguel


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 658 bytes --]

[bug#44075] [PATCH] gnu: Add make-glibc-locales-collection.

[Efraim Flashner]

[-- Attachment #1.1: Type: text/plain, Size: 3051 bytes --]

On Mon, Oct 19, 2020 at 03:17:54PM +0200, Miguel Ángel Arruga Vivas wrote:
> Hi Efraim,
> 
> I've been taking a look into your patch.  One issue are the comments
> about utf8 and UTF-8, as the issue is already explained in
> make-glibc[-utf8]-locales.

Thanks for taking a look. For the utf8 vs UTF-8 there are a couple of
comments in the code:
The above phase does not install locales with names using
the "normalized codeset."  Thus, create symlinks like:
en_US.utf8 -> en_US.UTF-8

and also:
For backward compatibility with Guix
<= 0.8.3, add "xx_YY.UTF-8".

When I check on one of my Debian boxes:
$localectl list-locales
C.UTF-8
en_US.utf8

I've learned that 'C.UTF-8' isn't from upstream, Debian has a patch for
it. Having written this patch a week or so ago and returning to it I
can't tell you which is the correct one. I think it's best to offer both
so it's not confusing which is correct. That works for the logic for
accepting either in the list of locales, but I'm concerned that skipping
the symlink to the other one could cause problems.

> 
> Other point is:
> 
> Efraim Flashner <efraim@flashner.co.il> writes:
> > +(define* (make-glibc-locales-collection
> > +           glibc
> > +           #:optional (locales
> > +                        '(list "en_US.utf8" "en_US.ISO-8859-1")))
> > (... Removed for clarity ...)
> > +             ,locales)
> 
> I would have used list there like (list ,@locales) or '(,@locales), this
> looks a bit odd to my eyes at least.  I'd expect this kind of calling code:
> 
> (let ((locales '("de_CH.utf8" ... "de_DE.utf8"))
>       (my-glibc ...))
>   (make-glibc-locales-collection myglibc locales))
> 
> Enforcing an extra quotation for no real reason on the calling site, as
> strings are self-evaluating objects, and the use of the symbol list,
> whose meaning depends on other context of execution, doesn't seem
> necessary.  Even worse, my example would raise an error as "de_CH.utf8"
> is not a procedure.

My scheme-foo isn't terribly strong, sometimes I just hack at it until
the code does what I want, and this is one of those times. I've changed
it so that locales takes a list and not an item that is a list.

> What do you think about replacing make-glibc-utf8-locales with a call of
> the new function (using that code) ensuring that the generated
> derivation stays the same for that case (i.e. it's optimized for the
> UTF-8 case)?

This is what I originally wanted to do, but there's a glibc-locales
buried in the bootstrap path so it's not so easy to just swap it out. I
can make the change in core-updates. I'll play around with it and see if
I can come out with the same derivation using a different function, but
I'm not expecting it to turn out identical.

> 
> Happy hacking!
> Miguel
> 



-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #1.2: v2-0001-gnu-Add-make-glibc-locales-collection.patch --]
[-- Type: text/plain, Size: 4646 bytes --]

From 3c2dfe702ad1c62b2f73a1f1036da3bcfa007c15 Mon Sep 17 00:00:00 2001
From: Efraim Flashner <efraim@flashner.co.il>
Date: Thu, 1 Oct 2020 22:50:32 +0300
Subject: [PATCH v2] gnu: Add make-glibc-locales-collection.

* gnu/packages/base.scm (make-glibc-locales-collection): New macro.
(en_us-glibc-locales): New variable.
---
 gnu/packages/base.scm | 73 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 72 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index c83775d8ee..4ea31c2ab6 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -62,7 +62,8 @@
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
   #:export (glibc
-            libiconv-if-needed))
+            libiconv-if-needed
+            make-custom-glibc-locales))
 
 ;;; Commentary:
 ;;;
@@ -1106,6 +1107,69 @@ to the @code{share/locale} sub-directory of this package.")
                                         ,(version-major+minor
                                           (package-version glibc)))))))))))
 
+(define* (make-glibc-locales-collection
+           glibc
+           #:optional (locales
+                        (list "en_US.utf8" "en_US.ISO-8859-1")))
+                        ;; This list for testing
+                        ;(list "el_GR.UTF-8" "en_US.utf8" "he_IL.ISO-8859-8" "ja_JP.EUC-JP" "zh_CN.GB18030" "zh_CN.GBK" "hy_AM.ARMSCII-8")))
+  (package
+    (name "glibc-locales-collection")
+    (version (package-version glibc))
+    (source #f)
+    (build-system trivial-build-system)
+    (arguments
+     `(#:modules ((guix build utils))
+       #:builder
+       (begin
+         (use-modules (guix build utils))
+
+         (let* ((libc      (assoc-ref %build-inputs "glibc"))
+                (gzip      (assoc-ref %build-inputs "gzip"))
+                (out       (assoc-ref %outputs "out"))
+                (localedir (string-append out "/lib/locale/"
+                                          ,(version-major+minor version))))
+           ;; 'localedef' needs 'gzip'.
+           (setenv "PATH" (string-append libc "/bin:" gzip "/bin"))
+
+           (mkdir-p localedir)
+           (for-each
+             (lambda (locale)
+               (let* ((contains-dot? (string-index locale #\.))
+                      (encoding-type (substring locale (1+ contains-dot?)))
+                      (raw-locale    (substring locale 0 contains-dot?))
+                      (utf8? (or (number? (string-contains locale ".utf8"))
+                                 (number? (string-contains locale ".UTF-8"))))
+                      (file (if utf8?
+                              (string-append localedir "/" raw-locale ".utf8")
+                              (if (string-contains locale ".ISO")
+                                (string-append localedir "/" raw-locale)
+                                (string-append localedir "/" locale)))))
+
+                 (invoke "localedef" "--no-archive"
+                         "--prefix" localedir
+                         "-i" raw-locale
+                         "-f" (if (equal? "utf8" encoding-type)
+                                "UTF-8"
+                                encoding-type)
+                         file)
+
+                 ;; Is it utf8 or UTF-8? NO ONE KNOWS!
+                 (when utf8?
+                   (symlink (string-append raw-locale ".utf8")
+                            (string-append localedir "/"
+                                           raw-locale ".UTF-8")))))
+             (list ,@locales))
+           #t))))
+    (native-inputs `(("glibc" ,glibc)
+                     ("gzip" ,gzip)))
+    (synopsis "Customizable collection of locales")
+    (description
+     "This package provides a custom collection of locales useful for
+providing exactly the locales requested when size matters.")
+    (home-page (package-home-page glibc))
+    (license (package-license glibc))))
+
 (define-public (make-glibc-utf8-locales glibc)
   (package
     (name "glibc-utf8-locales")
@@ -1161,6 +1225,13 @@ test environments.")
 (define-public glibc-utf8-locales
   (make-glibc-utf8-locales glibc))
 
+(define-public en_us-glibc-locales
+  (package
+    (inherit (make-glibc-locales-collection
+               glibc
+               (list "en_US.utf8" "en_US.ISO-8859-1")))
+    (name "en-us-glibc-locales")))
+
 ;; Packages provided to ease use of binaries linked against the previous libc.
 (define-public glibc-locales-2.29
   (package (inherit (make-glibc-locales glibc-2.29))
-- 
2.28.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#44075] [PATCH] gnu: Add make-glibc-locales-collection.

[Miguel Ángel Arruga Vivas]

Hello,

Efraim Flashner <efraim@flashner.co.il> writes:
> Thanks for taking a look.

No problem at all, I was thinking about something along these lines too,
but then I saw your patch. :-)

> For the utf8 vs UTF-8 there are a couple of comments in the code:
> The above phase does not install locales with names using
> the "normalized codeset."  Thus, create symlinks like:
> en_US.utf8 -> en_US.UTF-8
>
> and also:
> For backward compatibility with Guix
> <= 0.8.3, add "xx_YY.UTF-8".

Yes, what I mean is that the comments along the code may need to be
clarified, but adding a "nobody knows" doesn't add much information.
The actual source[1] says that the correct value is utf8, following
the rules for the 'normalized codeset' naming, that I copy here:

  1. Remove all characters besides numbers and letters.
  2. Fold letters to lowercase.
  3. If the same only contains digits prepend the string ‘"iso"’.

We should stick to that naming regarding libc locales (note to self),
even though we keep the links for compatibility.  That includes the
other locale at en_us-glibc-locales.

>> What do you think about replacing make-glibc-utf8-locales with a call of
>> the new function (using that code) ensuring that the generated
>> derivation stays the same for that case (i.e. it's optimized for the
>> UTF-8 case)?
>
> This is what I originally wanted to do, but there's a glibc-locales
> buried in the bootstrap path so it's not so easy to just swap it out.

I guess you mean glibc-utf8-locales, if not I'm missing where that
reference could come from.  That's why I insist on leaving exactly the
same derivation.

> I can make the change in core-updates. I'll play around with it and
> see if I can come out with the same derivation using a different
> function, but I'm not expecting it to turn out identical.

The colour of my lisp-rate belt isn't even close to some you can see
around here but I could bring you some help if you want, because I think
it's easier to do it without any change in current packages than it
might sound.  Not the definitions in scheme, of course, I mean the
derivations the daemon actually sees.

Said this, I've seen that [single-]locale-directory does mostly the
same, and there is a build-locale function in gnu/build/locale.scm... so
I'm starting to see as a better idea to clean up glibc-utf8-locales up
in core-updates using the latter, as it would lead to cleaner code for
the builder.

Could you check that and tell if you consider feasible what I propose?

Happy hacking!
Miguel

[1] info "(libc)Using gettextized software"

[bug#44075] [PATCH] gnu: Add make-glibc-locales-collection.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 3730 bytes --]

On Mon, Oct 19, 2020 at 07:43:27PM +0200, Miguel Ángel Arruga Vivas wrote:
> Hello,
> 
> Efraim Flashner <efraim@flashner.co.il> writes:
> > Thanks for taking a look.
> 
> No problem at all, I was thinking about something along these lines too,
> but then I saw your patch. :-)

Your patch was what got me to actually send mine to guix-patches :)

> > For the utf8 vs UTF-8 there are a couple of comments in the code:
> > The above phase does not install locales with names using
> > the "normalized codeset."  Thus, create symlinks like:
> > en_US.utf8 -> en_US.UTF-8
> >
> > and also:
> > For backward compatibility with Guix
> > <= 0.8.3, add "xx_YY.UTF-8".
> 
> Yes, what I mean is that the comments along the code may need to be
> clarified, but adding a "nobody knows" doesn't add much information.

That is a good point. I'll change it to something actually useful,
mentioning that there's a convention but that people often get it wrong.

> The actual source[1] says that the correct value is utf8, following
> the rules for the 'normalized codeset' naming, that I copy here:
> 
>   1. Remove all characters besides numbers and letters.
>   2. Fold letters to lowercase.
>   3. If the same only contains digits prepend the string ‘"iso"’.
> 
> We should stick to that naming regarding libc locales (note to self),
> even though we keep the links for compatibility.  That includes the
> other locale at en_us-glibc-locales.

I agree it should be renamed to en-us. I have some bikeshedding to think
about for en-us-glibc-locales vs glibc-locales-en-us. And I'll also test
out one for glibc-locales-guile-tests.

> >> What do you think about replacing make-glibc-utf8-locales with a call of
> >> the new function (using that code) ensuring that the generated
> >> derivation stays the same for that case (i.e. it's optimized for the
> >> UTF-8 case)?
> >
> > This is what I originally wanted to do, but there's a glibc-locales
> > buried in the bootstrap path so it's not so easy to just swap it out.
> 
> I guess you mean glibc-utf8-locales, if not I'm missing where that
> reference could come from.  That's why I insist on leaving exactly the
> same derivation.

I meant in commencement.scm. There's a glibc-utf8-locales-final near the
definition for %boot5-inputs.

> > I can make the change in core-updates. I'll play around with it and
> > see if I can come out with the same derivation using a different
> > function, but I'm not expecting it to turn out identical.
> 
> The colour of my lisp-rate belt isn't even close to some you can see
> around here but I could bring you some help if you want, because I think
> it's easier to do it without any change in current packages than it
> might sound.  Not the definitions in scheme, of course, I mean the
> derivations the daemon actually sees.
> 
> Said this, I've seen that [single-]locale-directory does mostly the
> same, and there is a build-locale function in gnu/build/locale.scm... so
> I'm starting to see as a better idea to clean up glibc-utf8-locales up
> in core-updates using the latter, as it would lead to cleaner code for
> the builder.
> 
> Could you check that and tell if you consider feasible what I propose?

I hadn't seen gnu/build/locales.scm. I'll have to see what we can do
about using it. We normally don't allow cross-over from the build side
to the packages side.

> Happy hacking!
> Miguel
> 
> [1] info "(libc)Using gettextized software"

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#44075] [PATCH] gnu: Add make-glibc-locales-collection.

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 574 bytes --]

Hi!

Efraim Flashner <efraim@flashner.co.il> skribis:

> * gnu/packages/base.scm (make-glibc-locales-collection): New macro.
> (en_us-glibc-locales): New variable.

Cool!

A while back I posted the attached patch, which reuses existing code
from (gnu system locale) and (gnu packages base).  I think I used it
just to see how much space all the UTF-8 locales take, compared to all
the locales.

Would it make sense to borrow from that?

At any rate, it’s the kind of package that’ll be more useful if/when we
have parameterized packages.

Ludo’.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 3868 bytes --]

diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index c83775d8ee..3fc43b04da 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -53,6 +53,8 @@
   #:use-module (gnu packages python)
   #:use-module (gnu packages gettext)
   #:use-module (guix utils)
+  #:use-module (guix gexp)
+  #:use-module (guix modules)
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
@@ -62,6 +64,8 @@
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
   #:export (glibc
+            %default-utf8-locales
+            make-glibc-utf8-locales
             libiconv-if-needed))
 
 ;;; Commentary:
@@ -1106,7 +1110,12 @@ to the @code{share/locale} sub-directory of this package.")
                                         ,(version-major+minor
                                           (package-version glibc)))))))))))
 
-(define-public (make-glibc-utf8-locales glibc)
+(define %default-utf8-locales
+  '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+
+(define* (make-glibc-utf8-locales glibc #:optional
+                                  (locales %default-utf8-locales)
+                                  (locale-file #f))
   (package
     (name "glibc-utf8-locales")
     (version (package-version glibc))
@@ -1145,10 +1154,17 @@ to the @code{share/locale} sub-directory of this package.")
 
                                ;; These are the locales commonly used for
                                ;; tests---e.g., in Guile's i18n tests.
-                               '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+                               ,(if locale-file
+                                    `(call-with-input-file
+                                         (assoc-ref %build-inputs "locale-file")
+                                       read)
+                                    `',locales))
                      #t))))
     (native-inputs `(("glibc" ,glibc)
-                     ("gzip" ,gzip)))
+                     ("gzip" ,gzip)
+                     ,@(if locale-file
+                           `(("locale-file" ,locale-file))
+                           '())))
     (synopsis "Small sample of UTF-8 locales")
     (description
      "This package provides a small sample of UTF-8 locales mostly useful in
@@ -1169,6 +1185,40 @@ test environments.")
   (package (inherit (make-glibc-utf8-locales glibc-2.29))
            (name "glibc-utf8-locales-2.29")))
 
+(define (glibc-supported-locales libc)
+  ((module-ref (resolve-interface '(gnu system locale)) ;FIXME: hack
+               'glibc-supported-locales)
+   libc))
+
+(define* (make-glibc-utf8-locales/full #:optional (glibc glibc))
+  (define utf8-locales
+    (computed-file "glibc-supported-utf8-locales.scm"
+                   #~(begin
+                       (use-modules (srfi srfi-1)
+                                    (ice-9 match)
+                                    (ice-9 pretty-print))
+
+                       (define locales
+                         (call-with-input-file
+                             #+(glibc-supported-locales glibc)
+                           read))
+
+                       (define utf8-locales
+                         (filter-map (match-lambda
+                                       ((name . "UTF-8")
+                                        (if (string-suffix? ".UTF-8" name)
+                                            (string-drop-right name 6)
+                                            name))
+                                       (_ #f))
+                                     locales))
+
+                       (call-with-output-file #$output
+                         (lambda (port)
+                           (pretty-print utf8-locales port))))))
+
+  (make-glibc-utf8-locales glibc #:locale-file utf8-locales))
+
+\f
 (define-public which
   (package
     (name "which")

[bug#44075] [PATCH] gnu: Add make-glibc-locales-collection.

[Miguel Ángel Arruga Vivas]

[-- Attachment #1: Type: text/plain, Size: 5827 bytes --]

Hello,

Efraim Flashner <efraim@flashner.co.il> writes:
> Your patch was what got me to actually send mine to guix-patches :)

Thanks. :-)

> That is a good point. I'll change it to something actually useful,
> mentioning that there's a convention but that people often get it wrong.

Great, the more knowledge, the easier to overcome that common
misunderstanding. :-)

> I agree it should be renamed to en-us. I have some bikeshedding to think
> about for en-us-glibc-locales vs glibc-locales-en-us. And I'll also test
> out one for glibc-locales-guile-tests.

I meant the ISO-8859-1, the locale creation should be en_EN.iso88591,
the locale name can be en_EN.88591 too...

Although in that case shouldn't be glibc-en-us-locales?  I'd like green
lights for the bike shed, please. ;-)

> I hadn't seen gnu/build/locales.scm. I'll have to see what we can do
> about using it. We normally don't allow cross-over from the build side
> to the packages side.

It is already being used in the build of glibc-locales and I guess it
would be harmless to move it to guix build if that is a problem, but I
think it can be used as it is right now, even I didn't check yet.  In
this for-each(gnu/packages/base.scm:1084), for example:
------------------------->8-------------begin----------------------
(for-each (lambda (directory)
            (let*-values (((base)
                           (basename directory))
                          ((name codeset)
                           (locale->name+codeset base))
                          ((normalized)
                           (normalize-codeset codeset)))
              (unless (string=? codeset normalized)
                (symlink base
                         (string-append (dirname directory)
                                        "/" name "."
                                        normalized)))))
          locales)
-------------------------8<--------------end-----------------------

I would write something like this in the builder code:
------------------------->8-------------begin----------------------
(define (locale-builder directory)
  (lambda (locale)
    (let-values (((name codeset)
                  (locale->name+codeset)))
      (if codeset
          (build-locale name
                        #:directory directory
                        #:codeset (normalize-codeset codeset))
          (build-locale name
                        #:directory directory
                        #:codeset "utf8")))))
(let ...
  (for-each (locale-builder localedir) '(,@locales))
  ...)
-------------------------8<--------------end-----------------------

I leave you here the function I was hacking apart (please, do not
confuse with working code ;-)) if you want to take any idea other from
it:
------------------------->8-------------begin----------------------
(define*-public (make-glibc-selected-locales
                 glibc
                 #:key
                 (suffix "-utf8-locales")
                 (locales %default-locales)
                 (alias %default-alias))
  (package
    (name (string-append (package-name glibc) suffix))
    (version (package-version glibc))
    (source #f)
    (build-system trivial-build-system)
    (arguments
     `(#:modules ((guix build utils) (gnu build locale))
       #:builder
       (begin
         (use-modules (guix build utils) (gnu build locale))
         ;; Print the common alias for locale
         (define (print-locale-alias locale)
           (let-values (((name codeset)
                         (locale->name+codeset)))
             (if codeset
                 (or (string-equal codeset (normalize-codeset codeset))
                     (format #t "~a.~a ~a.~a~%" name codeset
                             name (normalize-codeset codeset)))
                 (format #t "~a.UTF-8 ~a.utf8" name name))))

         ;; Generate a function that builds the locale into directory.
         (define (locale-builder directory)
           (lambda (locale)
             (let-values (((name codeset)
                           (locale->name+codeset)))
               (if codeset
                   (build-locale name
                                 #:directory directory
                                 #:codeset (normalize-codeset codeset))
                   (build-locale name
                                 #:directory directory
                                 #:codeset "utf8")))))

         ;; Do the job.
         (let* ((libc      (assoc-ref %build-inputs "glibc"))
                (gzip      (assoc-ref %build-inputs "gzip"))
                (out       (assoc-ref %outputs "out"))
                (localedir (string-append out "/lib/locale/"
                                          ,(version-major+minor version)))
                (alias     (string-append localedir "/locale.alias"))
                (locales   '(,@locales)))

           ;; 'localedef' needs 'gzip'.
           (setenv "PATH" (string-append libc "/bin:" gzip "/bin"))

           (mkdir-p localedir)
           ;; Generate each locale provided.
           (for-each (locale-builder localedir) locales)
           ;; Generate alias file.
           (with-output-to-file alias
             (lambda ()
               (format #t "~a~%" "# Aliases for common codeset names.")
               (for-each print-locale-alias locales)
               (format #t "~a~%" "# Other aliases.")
               (for-each (lambda (line)
                           (format #t "~a~%" line))
                         '(,@alias)))))))
    (native-inputs `(("glibc" ,glibc)
                     ("gzip" ,gzip)))
    (synopsis "Configured sample of locales")
    (description
     "TODO.")
    (home-page (package-home-page glibc))
    (license (package-license glibc))))
-------------------------8<--------------end-----------------------

Happy hacking,
Miguel

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 658 bytes --]

[bug#44075] [PATCH] gnu: Add make-glibc-locales-collection.

[Miguel Ángel Arruga Vivas]

[-- Attachment #1.1: Type: text/plain, Size: 943 bytes --]

Hi Efraim and Ludo!

This patch could be an starting point to even master, it allows to
extend easily glibc-utf8-locales to the desired ones.

I've used the following steps to test it[1]:
------------------------------>8----------------------------------
$ echo '(define-module (t) #:use-module (gnu packages base))
(define-public glibc-utf8-test
  (make-glibc-utf8-locales glibc #:locales (list "es_ES")
                           #:name "glibc-utf8-test"))
glibc-utf8-test' > tmp/t.scm
$ ./pre-inst-env guix build -f tmp/t.scm
$ ./pre-inst-env guix package -L tmp --show=glibc-utf8-test
------------------------------8<----------------------------------

Something like glibc-utf8-<language>-locales could be easily implemented
on top of this, and I think it could be helpful for the end user too.
WDYT?

Happy hacking!
Miguel

[1] After checking that ./pre-inst-env guix build glibc-utf8-locales
didn't rebuild anything at all, of course.


[-- Attachment #1.2: utf8-locales.patch --]
[-- Type: text/x-patch, Size: 3755 bytes --]

From b0d2bfbe20a0a48a23a8dd4f14c0acce4ef5842f Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miguel=20=C3=81ngel=20Arruga=20Vivas?=
 <rosen644835@gmail.com>
Date: Wed, 4 Nov 2020 15:48:45 +0100
Subject: [PATCH] gnu: base: Add optional locales keyword to
 make-glibc-utf8-locales.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

* gnu/packages/base.scm (make-glibc-utf8-locales): Add keyword parameter
locales with the old value as default.
[arguments]: Use locales value.

Co-authored-by: Efraim Flashner <efraim@flashner.co.il>
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
---
 gnu/packages/base.scm | 31 +++++++++++++++++++++++--------
 1 file changed, 23 insertions(+), 8 deletions(-)

diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index c83775d8ee..5170a77270 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -52,13 +52,16 @@
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
   #:use-module (gnu packages gettext)
+  #:use-module (guix i18n)
   #:use-module (guix utils)
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system trivial)
+  #:use-module (ice-9 format)
   #:use-module (ice-9 match)
+  #:use-module (ice-9 optargs)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
   #:export (glibc
@@ -1106,9 +1109,16 @@ to the @code{share/locale} sub-directory of this package.")
                                         ,(version-major+minor
                                           (package-version glibc)))))))))))
 
-(define-public (make-glibc-utf8-locales glibc)
+(define %default-utf8-locales
+  ;; These are the locales commonly used for tests---e.g., in Guile's i18n
+  ;; tests.
+  '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+(define*-public (make-glibc-utf8-locales glibc #:key
+                                         (locales %default-utf8-locales)
+                                         (name "glibc-utf8-locales"))
+  (define default-locales? (equal? locales %default-utf8-locales))
   (package
-    (name "glibc-utf8-locales")
+    (name name)
     (version (package-version glibc))
     (source #f)
     (build-system trivial-build-system)
@@ -1142,17 +1152,22 @@ to the @code{share/locale} sub-directory of this package.")
                                  (symlink (string-append locale ".utf8")
                                           (string-append localedir "/"
                                                          locale ".UTF-8")))
-
-                               ;; These are the locales commonly used for
-                               ;; tests---e.g., in Guile's i18n tests.
-                               '("de_DE" "el_GR" "en_US" "fr_FR" "tr_TR"))
+                               ',locales)
                      #t))))
     (native-inputs `(("glibc" ,glibc)
                      ("gzip" ,gzip)))
-    (synopsis "Small sample of UTF-8 locales")
+    (synopsis (if default-locales?
+                  (P_ "Small sample of UTF-8 locales")
+                  (P_ "Customized sample of UTF-8 locales")))
     (description
-     "This package provides a small sample of UTF-8 locales mostly useful in
+     (if default-locales?
+         (P_ "This package provides a small sample of UTF-8 locales mostly useful in
 test environments.")
+         (format #f (P_ "This package provides the following UTF-8 locales:
+@itemize
+~{@item ~a~%~}
+@end itemize~%")
+                 locales)))
     (home-page (package-home-page glibc))
     (license (package-license glibc))))
 
-- 
2.28.0


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 658 bytes --]

[bug#44075] [PATCH] gnu: Add make-glibc-locales-collection.

[Ludovic Courtès]

Hi,

Miguel Ángel Arruga Vivas <rosen644835@gmail.com> skribis:

> From b0d2bfbe20a0a48a23a8dd4f14c0acce4ef5842f Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Miguel=20=C3=81ngel=20Arruga=20Vivas?=
>  <rosen644835@gmail.com>
> Date: Wed, 4 Nov 2020 15:48:45 +0100
> Subject: [PATCH] gnu: base: Add optional locales keyword to
>  make-glibc-utf8-locales.
> MIME-Version: 1.0
> Content-Type: text/plain; charset=UTF-8
> Content-Transfer-Encoding: 8bit
>
> * gnu/packages/base.scm (make-glibc-utf8-locales): Add keyword parameter
> locales with the old value as default.
> [arguments]: Use locales value.
>
> Co-authored-by: Efraim Flashner <efraim@flashner.co.il>
> Co-authored-by: Ludovic Courtès <ludo@gnu.org>

Looks reasonable to me.

The next question is how we’ll use it.  I think it’s an excellent case
for parameterized packages, for which I’ve proposed a rough design
before, but we need someone to champion on that one.

Thanks,
Ludo’.

[bug#44075] [PATCH] gnu: Add make-glibc-locales-collection.

[Ludovic Courtès]

Hi,

Ludovic Courtès <ludo@gnu.org> skribis:

> Miguel Ángel Arruga Vivas <rosen644835@gmail.com> skribis:
>
>> From b0d2bfbe20a0a48a23a8dd4f14c0acce4ef5842f Mon Sep 17 00:00:00 2001
>> From: =?UTF-8?q?Miguel=20=C3=81ngel=20Arruga=20Vivas?=
>>  <rosen644835@gmail.com>
>> Date: Wed, 4 Nov 2020 15:48:45 +0100
>> Subject: [PATCH] gnu: base: Add optional locales keyword to
>>  make-glibc-utf8-locales.
>> MIME-Version: 1.0
>> Content-Type: text/plain; charset=UTF-8
>> Content-Transfer-Encoding: 8bit
>>
>> * gnu/packages/base.scm (make-glibc-utf8-locales): Add keyword parameter
>> locales with the old value as default.
>> [arguments]: Use locales value.
>>
>> Co-authored-by: Efraim Flashner <efraim@flashner.co.il>
>> Co-authored-by: Ludovic Courtès <ludo@gnu.org>
>
> Looks reasonable to me.

I think it’s OK for ‘master’.

> The next question is how we’ll use it.  I think it’s an excellent case
> for parameterized packages, for which I’ve proposed a rough design
> before, but we need someone to champion on that one.

It’s not there yet but there’s hope:

  https://lists.gnu.org/archive/html/guix-devel/2020-11/msg00312.html

Ludo’.

[bug#44075] [PATCH] gnu: Add make-glibc-locales-collection.

[bdju--- via Guix-patches via]

Could the en_SE.utf8 locale be added? For those who want their system in English, but with YMD date and 24 hour time.

[bug#44075] [PATCH] gnu: Add make-glibc-locales-collection.

[Leo Famulari]

On Thu, Dec 24, 2020 at 09:38:51AM +0000, bdju--- via Guix-patches via wrote:
> Could the en_SE.utf8 locale be added? For those who want their system in English, but with YMD date and 24 hour time.

Is this contained in the glibc-locales package?

The glibc-utf8-locales package is basically just a small package variant
for testing and we are planning to remove it completely.

[bug#44075] [PATCH] gnu: Add make-glibc-locales-collection.

[Leo Famulari]

On Thu, Dec 24, 2020 at 05:04:41PM -0500, Leo Famulari wrote:
> On Thu, Dec 24, 2020 at 09:38:51AM +0000, bdju--- via Guix-patches via wrote:
> > Could the en_SE.utf8 locale be added? For those who want their system in English, but with YMD date and 24 hour time.
> 
> Is this contained in the glibc-locales package?
> 
> The glibc-utf8-locales package is basically just a small package variant
> for testing and we are planning to remove it completely.

I read more of this thread and realized I misunderstood its point.
Please disregard my previous message.

[bug#44075] [PATCH] gnu: Add make-glibc-locales-collection.

[Miguel Ángel Arruga Vivas]

Hi,

Ludovic Courtès <ludo@gnu.org> writes:

> I think it’s OK for ‘master’.

Oops, sorry, I forgot completely about this. :-(

I've just pushed as 0d3f2716a8 to master.

Thank you!
Miguel