bug#67044: C.utf8 locale cannot be built

bug#67044: C.utf8 locale cannot be built

[Tomas Volf]

[-- Attachment #1: Type: text/plain, Size: 2061 bytes --]

Hi,

when trying to build a system with C.utf8 locale, I end up with the following
error:

    building /gnu/store/v6jma6kmwywr509n4y0vypchnh4y5s3m-locale-2.35.drv...
    building locale 'C.utf8'...
    [error] LC_MONETARY: value for field `mon_decimal_point' must not be an empty string
    [error] no output file produced because errors were issued
    Backtrace:
               2 (primitive-load "/gnu/store/ccv2qfrqxk166ysg6anrzj1kz4h?")
    In ice-9/boot-9.scm:
       285:13  1 (for-each #<procedure 7fffeef5c540 at ice-9/eval.scm:3?> ?)
    In guix/build/utils.scm:
        812:6  0 (invoke "localedef" "--no-archive" "--prefix" "/gnu/st?" ?)
    
    guix/build/utils.scm:812:6: In procedure invoke:
    ERROR:
      1. &invoke-error:
          program: "localedef"
          arguments: ("--no-archive" "--prefix" "/gnu/store/08rlginv27b9v1ba4n94plp7lmxjihja-locale-2.35/2.35" "-i" "C" "-f" "UTF-8" "/gnu/store/08rlginv27b9v1ba4n94plp7lmxjihja-locale-2.35/2.35/C.utf8")
          exit-status: 4
          term-signal: #f
          stop-signal: #f
    builder for `/gnu/store/v6jma6kmwywr509n4y0vypchnh4y5s3m-locale-2.35.drv' failed with exit code 1
    build of /gnu/store/v6jma6kmwywr509n4y0vypchnh4y5s3m-locale-2.35.drv failed
    View build log at '/var/log/guix/drvs/v6/jma6kmwywr509n4y0vypchnh4y5s3m-locale-2.35.drv.gz'.
    cannot build derivation `/gnu/store/g47g7zqs5la6qpfmn6q1zgbhp291l1ha-system.drv': 1 dependencies couldn't be built
    guix system: error: build of `/gnu/store/g47g7zqs5la6qpfmn6q1zgbhp291l1ha-system.drv' failed

This seems to be a known problem in 2.35,
https://sourceware.org/bugzilla/show_bug.cgi?id=28861 .  On the page there is
also a workaround, and that is to compile with the locales with -c.

So that would be one solution until we update to 2.36 or higher.  I do not see a
way to override this (add the -c) from the operating-system definition.

Tomas Volf

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#67044: C.utf8 locale cannot be built

[Ludovic Courtès]

Hi Tomas!

Tomas Volf <~@wolfsden.cz> skribis:

> when trying to build a system with C.utf8 locale, I end up with the following
> error:
>
>     building /gnu/store/v6jma6kmwywr509n4y0vypchnh4y5s3m-locale-2.35.drv...
>     building locale 'C.utf8'...
>     [error] LC_MONETARY: value for field `mon_decimal_point' must not be an empty string
>     [error] no output file produced because errors were issued

[...]

> This seems to be a known problem in 2.35,
> https://sourceware.org/bugzilla/show_bug.cgi?id=28861 .  On the page there is
> also a workaround, and that is to compile with the locales with -c.
>
> So that would be one solution until we update to 2.36 or higher.  I do not see a
> way to override this (add the -c) from the operating-system definition.

We could/should fix this in (gnu system locale).

Now, it would also be nice if C.utf8 were built-in, shipped with the
‘glibc’ package we have (to me that’s the whole point of C.utf8).  We
should fix that now in ‘core-updates’.  Ideas on how to do that?

Thanks,
Ludo’.

bug#67044: C.utf8 locale cannot be built

[Tomas Volf]

[-- Attachment #1: Type: text/plain, Size: 2870 bytes --]

On 2023-11-27 23:02:26 +0100, Ludovic Courtès wrote:
> Hi Tomas!

Hi! :)

> 
> Tomas Volf <~@wolfsden.cz> skribis:
> 
> > when trying to build a system with C.utf8 locale, I end up with the following
> > error:
> >
> >     building /gnu/store/v6jma6kmwywr509n4y0vypchnh4y5s3m-locale-2.35.drv...
> >     building locale 'C.utf8'...
> >     [error] LC_MONETARY: value for field `mon_decimal_point' must not be an empty string
> >     [error] no output file produced because errors were issued
> 
> [...]
> 
> > This seems to be a known problem in 2.35,
> > https://sourceware.org/bugzilla/show_bug.cgi?id=28861 .  On the page there is
> > also a workaround, and that is to compile with the locales with -c.
> >
> > So that would be one solution until we update to 2.36 or higher.  I do not see a
> > way to override this (add the -c) from the operating-system definition.
> 
> We could/should fix this in (gnu system locale).

That is currently not possible I am afraid, since %default-locale-definitions is
global, not per-version, and glibc-2.33 is installed by default.

> 
> Now, it would also be nice if C.utf8 were built-in, shipped with the
> ‘glibc’ package we have (to me that’s the whole point of C.utf8).  We
> should fix that now in ‘core-updates’.  Ideas on how to do that?

After short research, I do have an idea.  My knowledge of Guix's internals is
not good enough (yet? :)) to implement it though.  And I am not even sure it
should be done.  Anyway here it goes:

1. Add a phase after 'install that builds and installs the C.utf8 locale, as
documented here[0].
2. Make glibc package add the directory into GUIX_LOCPATH.  Since it accepts :
separated directories, it should be possible, however I am unsure how.

I think that should do it.  However, I am not sure what the benefit would be.
The base locale is C, anything else (like C.utf8) is extra, and user needs to
modify LANG to get it working anyway.  So installing it via the glibc-locales
seems fine enough.

In my opinion, the correct long-term approach here is to just add C.utf8 into
%default-locale-definitions.  That however cannot be done until glibc-2.33 is
dropped from %default-locale-libcs.

For the time being however, using C.utf8 is solvable[1] from the
operating-system definition, with the exception of compiling the locale.  About
which is this issue and the fix is trivial[2].

I am not sure this issue is worth overthinking it.

0: https://www.gnu.org/software/libc/manual/html_node/Running-make-install.html
1: https://emacs.ch/@graywolf/111404592336140803
2: https://git.sr.ht/~graywolf/guix/commit/1e94b59a7b27d44435f321083a01242bdf16c566



Let me know what you think :)

Tomas

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#67044: C.utf8 locale cannot be built

[Ludovic Courtès]

Hello!

Tomas Volf <~@wolfsden.cz> skribis:

> On 2023-11-27 23:02:26 +0100, Ludovic Courtès wrote:

[...]

>> Now, it would also be nice if C.utf8 were built-in, shipped with the
>> ‘glibc’ package we have (to me that’s the whole point of C.utf8).  We
>> should fix that now in ‘core-updates’.  Ideas on how to do that?
>
> After short research, I do have an idea.  My knowledge of Guix's internals is
> not good enough (yet? :)) to implement it though.  And I am not even sure it
> should be done.  Anyway here it goes:
>
> 1. Add a phase after 'install that builds and installs the C.utf8 locale, as
> documented here[0].
> 2. Make glibc package add the directory into GUIX_LOCPATH.  Since it accepts :
> separated directories, it should be possible, however I am unsure how.

I decided to give it a go:

  https://issues.guix.gnu.org/67686

Please do chime in and let me know what you think!

Ludo’.

[bug#67686] bug#67044: C.utf8 locale cannot be built

[Tomas Volf]

[-- Attachment #1: Type: text/plain, Size: 2399 bytes --]

Hi :)

On 2023-12-07 11:27:04 +0100, Ludovic Courtès wrote:
> [..]
>
> I decided to give it a go:
>
>   https://issues.guix.gnu.org/67686
>
> Please do chime in and let me know what you think!

Thanks to the detailed cover letter, now I understand the benefit, so I agree it
would make sense.  I looked over the implementation, and it looks fine to me (I
am not sure if I am qualified to do the review though :) ).

I just have few notes:

1.

> (glibc-2.35)[arguments]: Delete ‘install-utf8-c-locale’ phase.

I do think 2.35 should install the locale as well.  That would require to change

    (invoke (string-append bin "/localedef")
            "--no-archive" "--prefix" locale
            "-i" "C" "-f" "UTF-8"
            (string-append locale "/C.UTF-8")))))

into

    (invoke (string-append bin "/localedef")
            "-c" "--no-archive" "--prefix" locale
            "-i" "C" "-f" "UTF-8"
            (string-append locale "/C.UTF-8")))))

however I think that is fine.  I am using locale built like that and it works
well.  What is more, from the discussion under the other issue[0], that is
exactly what is done during normal glibc build:

> It turns out we ignore errors during the glibc build (--quiet -c).

After that the drop of 'install-utf8-c-locale can be moved into some other
version < 2.35.

2.

I still believe it makes sense to add the -c also into the locale builder,
because my understanding is that this change will not allow using (locale
"C.utf8") in the operating-system definition (since that would still try to
build it, and fail).

If you are not opposed to the idea, I can send a patch if you would prefer not
to do it yourself.

3.

> I suspect libc builds an additional ‘localedef’ for the build machine but I’m
> not sure where it is, hmm…

I looked around a bit, and I am not sure that is true.  There seems to be only
./locale/localedef created.  However, there is localedef inside gcc-toolchain's
bin directory, and, of course, in the build glibc.  I am not sure what are the
version requirements here, but I would expect at least the one provided by glibc
to be usable.



Have a nice day,
Tomas

0: https://sourceware.org/bugzilla/show_bug.cgi?id=28845

--
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#67044: [bug#67686] bug#67044: C.utf8 locale cannot be built

[Ludovic Courtès]

Hi Tomas,

Tomas Volf <~@wolfsden.cz> skribis:

>> (glibc-2.35)[arguments]: Delete ‘install-utf8-c-locale’ phase.
>
> I do think 2.35 should install the locale as well.  That would require to change
>
>     (invoke (string-append bin "/localedef")
>             "--no-archive" "--prefix" locale
>             "-i" "C" "-f" "UTF-8"
>             (string-append locale "/C.UTF-8")))))
>
> into
>
>     (invoke (string-append bin "/localedef")
>             "-c" "--no-archive" "--prefix" locale
>             "-i" "C" "-f" "UTF-8"
>             (string-append locale "/C.UTF-8")))))
>
> however I think that is fine.  I am using locale built like that and it works
> well.  What is more, from the discussion under the other issue[0], that is
> exactly what is done during normal glibc build:
>
>> It turns out we ignore errors during the glibc build (--quiet -c).
>
> After that the drop of 'install-utf8-c-locale can be moved into some other
> version < 2.35.

I’m a bit wary of using ‘-c’ (aka. ‘--force’) unconditionally as this
could hide real problems.

But more importantly, I think it won’t matter whether glibc 2.35 ships
C.UTF-8 since it’s no longer going to be used, except for building old
locale data via ‘locale-libcs’.

> 2.
>
> I still believe it makes sense to add the -c also into the locale builder,
> because my understanding is that this change will not allow using (locale
> "C.utf8") in the operating-system definition (since that would still try to
> build it, and fail).
>
> If you are not opposed to the idea, I can send a patch if you would prefer not
> to do it yourself.

No you’re right, we could add ‘-c’ to the code in (gnu system locale),
though perhaps it would be safer to do so only in the 2.35 + C.UTF-8
case.

(We can do that independently of this patch.)

> 3.
>
>> I suspect libc builds an additional ‘localedef’ for the build machine but I’m
>> not sure where it is, hmm…
>
> I looked around a bit, and I am not sure that is true.

In the meantime I found that this is wrong indeed:

  https://issues.guix.gnu.org/67686#11

Thanks for your feedback!

Ludo’.

bug#67044: [bug#67686] bug#67044: C.utf8 locale cannot be built

[Tomas Volf]

[-- Attachment #1: Type: text/plain, Size: 851 bytes --]

On 2023-12-09 15:46:58 +0100, Ludovic Courtès wrote:
> > I still believe it makes sense to add the -c also into the locale builder,
> > because my understanding is that this change will not allow using (locale
> > "C.utf8") in the operating-system definition (since that would still try to
> > build it, and fail).
> >
> > If you are not opposed to the idea, I can send a patch if you would prefer not
> > to do it yourself.
>
> No you’re right, we could add ‘-c’ to the code in (gnu system locale),
> though perhaps it would be safer to do so only in the 2.35 + C.UTF-8
> case.
>
> (We can do that independently of this patch.)

My attempt at that can be found here: https://issues.guix.gnu.org/67735

Tomas

--
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]