Formatted output with locale

Formatted output with locale

[dev]

Hi there,
I have problems to get a proper formatting using the (ice-9 format)
module. In my code, I need to deal with monetary figures, but I fail
to get the correct format for my German locale.
Let's say I have one hundred thousand Euros. The correct format to output
this in de_DE.utf-8 would be
100.000,00 EUR
(with 2 decimals)

Now I tried:
(setlocale LC_ALL "de_DE.UTF-8")
(use-modules (ice-9 format))
(define a 100000.00)
(format #t "~12,2h EUR~%" a)
(format #t "~,,12$ EUR~%" a)
(format #t "~12,2f EUR~%" a)

But this is what Guile gives to me:
   100.000,0 EUR
   100000.00 EUR
   100000.00 EUR

None is correct. What did I miss? 
-Martin

Fwd: Formatted output with locale

[Vladimir Zhbanov]

Martin,
I'm sorry for the private reply, my fault :-(
Forward to the list.

---------- Forwarded message ----------
From: Vladimir Zhbanov <vzhbanov@gmail.com>
Date: Mon, 14 Nov 2016 09:03:19 +0300
Subject: Re: Formatted output with locale
To: dev@famic.de

Hi Martin,

On 11/13/16, dev@famic.de <dev@famic.de> wrote:
> Hi there,
> I have problems to get a proper formatting using the (ice-9 format)
> module. In my code, I need to deal with monetary figures, but I fail
> to get the correct format for my German locale.
> Let's say I have one hundred thousand Euros. The correct format to output
> this in de_DE.utf-8 would be
> 100.000,00 EUR
> (with 2 decimals)
>
> Now I tried:
> (setlocale LC_ALL "de_DE.UTF-8")
> (use-modules (ice-9 format))
> (define a 100000.00)
> (format #t "~12,2h EUR~%" a)
> (format #t "~,,12$ EUR~%" a)
> (format #t "~12,2f EUR~%" a)
>
> But this is what Guile gives to me:
>    100.000,0 EUR
>    100000.00 EUR
>    100000.00 EUR
>
> None is correct. What did I miss?
> -Martin

Looking at the `format' code for option `h' I see the procedure
`number->locale-string' is used.

OK, let's do some experiments:

scheme@(guile-user)> (setlocale LC_ALL)
$2 = "ru_RU.UTF-8"
scheme@(guile-user)> ,use (ice-9 i18n)
scheme@(guile-user)> (number->locale-string 10.0 5)
$3 = "10,0"
scheme@(guile-user)> (number->locale-string .00003 2)
$4 = "3,0e"
scheme@(guile-user)> (number->locale-string .00003 1)
$5 = "3,0"
scheme@(guile-user)> (number->locale-string .00003 3)
$6 = "3,0e-"
scheme@(guile-user)> (number->locale-string .0 3)
$7 = "-0,0"
scheme@(guile-user)> (number->locale-string .0 10)
$8 = "-0,0"
scheme@(guile-user)> (number->locale-string .0 1)
$9 = "-0,0"

Now for the procedure monetary-amount->locale-string:

scheme@(guile-user)> (monetary-amount->locale-string .0 #t)
$10 = "-0.0 RUB "
scheme@(guile-user)> (monetary-amount->locale-string 100000.0 #t)
$11 = "100\xa0000.0 RUB "
scheme@(guile-user)> (monetary-amount->locale-string .00003 #t)
$12 = "3.0e RUB "

And my guile version:

scheme@(guile-user)> (version)
$13 = "2.0.11"


It seems `number->locale-string' is broken. If I do, e.g.
  (number->locale-string .00003 10)
it yields
  "3,0e-5"

If the last arg decreases, it just trims the number of
characters after comma, which is wrong in this case.

Output for zero is funny, too.

Regards,
  Vladimir

Re: Formatted output with locale

[Ludovic Courtès]

Hi!

dev@famic.de skribis:

> I have problems to get a proper formatting using the (ice-9 format)
> module. In my code, I need to deal with monetary figures, but I fail
> to get the correct format for my German locale.
> Let's say I have one hundred thousand Euros. The correct format to output
> this in de_DE.utf-8 would be
> 100.000,00 EUR
> (with 2 decimals)
>
> Now I tried:
> (setlocale LC_ALL "de_DE.UTF-8")
> (use-modules (ice-9 format))
> (define a 100000.00)
> (format #t "~12,2h EUR~%" a)
> (format #t "~,,12$ EUR~%" a)
> (format #t "~12,2f EUR~%" a)
>
> But this is what Guile gives to me:
>    100.000,0 EUR
>    100000.00 EUR
>    100000.00 EUR

It seems to work as advertised for me:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(ice-9 i18n)
scheme@(guile-user)> (number->locale-string 10000.01 2 (make-locale LC_ALL "fr_FR.utf8"))
$13 = "10 000,01"
scheme@(guile-user)> (number->locale-string 10000.01 2 (make-locale LC_ALL "de_DE.utf8"))
$14 = "10.000,01"
scheme@(guile-user)> ,use(ice-9 format)
scheme@(guile-user)> (setlocale LC_ALL "de_DE.utf8")
$15 = "de_DE.utf8"
scheme@(guile-user)> (format #f "~12,2h" 10000.01)
$16 = "   10.000,01"
--8<---------------cut here---------------end--------------->8---

That’s on GNU/Linux (glibc 2.24).

Note that number formatting data comes from the C library.  What C
library do you use?

Thanks,
Ludo’.

Re: Formatted output with locale

[Eli Zaretskii]

> From: ludo@gnu.org (Ludovic Courtès)
> Date: Thu, 17 Nov 2016 12:02:06 +0100
> 
> It seems to work as advertised for me:
> 
> --8<---------------cut here---------------start------------->8---
> scheme@(guile-user)> ,use(ice-9 i18n)
> scheme@(guile-user)> (number->locale-string 10000.01 2 (make-locale LC_ALL "fr_FR.utf8"))
> $13 = "10 000,01"
> scheme@(guile-user)> (number->locale-string 10000.01 2 (make-locale LC_ALL "de_DE.utf8"))
> $14 = "10.000,01"
> scheme@(guile-user)> ,use(ice-9 format)
> scheme@(guile-user)> (setlocale LC_ALL "de_DE.utf8")
> $15 = "de_DE.utf8"
> scheme@(guile-user)> (format #f "~12,2h" 10000.01)
> $16 = "   10.000,01"
> --8<---------------cut here---------------end--------------->8---
> 
> That’s on GNU/Linux (glibc 2.24).

It works for me as well, on MS-Windows (although I need to use a
non-UTF-8 codeset in the locales).

Re: Formatted output with locale

[Vladimir Zhbanov]

On Thu, Nov 17, 2016 at 12:02:06PM +0100, Ludovic Courtès wrote:
> Hi!
> 
> dev@famic.de skribis:
> 
> > I have problems to get a proper formatting using the (ice-9 format)
> > module. In my code, I need to deal with monetary figures, but I fail
> > to get the correct format for my German locale.
> > Let's say I have one hundred thousand Euros. The correct format to output
> > this in de_DE.utf-8 would be
> > 100.000,00 EUR
> > (with 2 decimals)
> >
> > Now I tried:
> > (setlocale LC_ALL "de_DE.UTF-8")
> > (use-modules (ice-9 format))
> > (define a 100000.00)
> > (format #t "~12,2h EUR~%" a)
> > (format #t "~,,12$ EUR~%" a)
> > (format #t "~12,2f EUR~%" a)
> >
> > But this is what Guile gives to me:
> >    100.000,0 EUR
> >    100000.00 EUR
> >    100000.00 EUR
> 
> It seems to work as advertised for me:
> 
> --8<---------------cut here---------------start------------->8---
> scheme@(guile-user)> ,use(ice-9 i18n)
> scheme@(guile-user)> (number->locale-string 10000.01 2 (make-locale LC_ALL "fr_FR.utf8"))
> $13 = "10 000,01"
> scheme@(guile-user)> (number->locale-string 10000.01 2 (make-locale LC_ALL "de_DE.utf8"))
> $14 = "10.000,01"
> scheme@(guile-user)> ,use(ice-9 format)
> scheme@(guile-user)> (setlocale LC_ALL "de_DE.utf8")
> $15 = "de_DE.utf8"
> scheme@(guile-user)> (format #f "~12,2h" 10000.01)
> $16 = "   10.000,01"
> --8<---------------cut here---------------end--------------->8---

What does it output if you type
  (number->locale-string 10000.00 2 (make-locale LC_ALL "fr_FR.utf8"))

that is with two zeros after the point?

I would expect the same two zeros if the second argument is
2... Am I wrong?

> That’s on GNU/Linux (glibc 2.24).
> 
> Note that number formatting data comes from the C library.  What C
> library do you use?

My glibc version is 2.19.

Thanks

-- 
  Vladimir

Re: Formatted output with locale

[Ludovic Courtès]

Vladimir Zhbanov <vzhbanov@gmail.com> skribis:

> On Thu, Nov 17, 2016 at 12:02:06PM +0100, Ludovic Courtès wrote:
>> Hi!
>> 
>> dev@famic.de skribis:
>> 
>> > I have problems to get a proper formatting using the (ice-9 format)
>> > module. In my code, I need to deal with monetary figures, but I fail
>> > to get the correct format for my German locale.
>> > Let's say I have one hundred thousand Euros. The correct format to output
>> > this in de_DE.utf-8 would be
>> > 100.000,00 EUR
>> > (with 2 decimals)
>> >
>> > Now I tried:
>> > (setlocale LC_ALL "de_DE.UTF-8")
>> > (use-modules (ice-9 format))
>> > (define a 100000.00)
>> > (format #t "~12,2h EUR~%" a)
>> > (format #t "~,,12$ EUR~%" a)
>> > (format #t "~12,2f EUR~%" a)
>> >
>> > But this is what Guile gives to me:
>> >    100.000,0 EUR
>> >    100000.00 EUR
>> >    100000.00 EUR
>> 
>> It seems to work as advertised for me:
>> 
>> --8<---------------cut here---------------start------------->8---
>> scheme@(guile-user)> ,use(ice-9 i18n)
>> scheme@(guile-user)> (number->locale-string 10000.01 2 (make-locale LC_ALL "fr_FR.utf8"))
>> $13 = "10 000,01"
>> scheme@(guile-user)> (number->locale-string 10000.01 2 (make-locale LC_ALL "de_DE.utf8"))
>> $14 = "10.000,01"
>> scheme@(guile-user)> ,use(ice-9 format)
>> scheme@(guile-user)> (setlocale LC_ALL "de_DE.utf8")
>> $15 = "de_DE.utf8"
>> scheme@(guile-user)> (format #f "~12,2h" 10000.01)
>> $16 = "   10.000,01"
>> --8<---------------cut here---------------end--------------->8---
>
> What does it output if you type
>   (number->locale-string 10000.00 2 (make-locale LC_ALL "fr_FR.utf8"))
>
> that is with two zeros after the point?

I get this:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (number->locale-string 10000.00 2 (make-locale LC_ALL "fr_FR.utf8"))
$1 = "10 000,0"
scheme@(guile-user)> (number->locale-string 10000.00 4  (make-locale LC_ALL "fr_FR.utf8"))              
$2 = "10 000,0"
--8<---------------cut here---------------end--------------->8---

… and that’s definitely a bug.

Could you send it to bug-guile@gnu.org so we keep track of it?

>> That’s on GNU/Linux (glibc 2.24).
>> 
>> Note that number formatting data comes from the C library.  What C
>> library do you use?
>
> My glibc version is 2.19.

It might be responsible for the incorrect thousand and fraction
separators you observe in the de_DE output.  To dig further, you can
check the low-level info provided by ‘nl_langinfo’:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(ice-9 i18n)
scheme@(guile-user)> (define l (make-locale LC_ALL "de_DE.utf8"))
scheme@(guile-user)> (locale-digit-grouping l)
$2 = (3 3 . #-1#)
scheme@(guile-user)> (locale-decimal-point l)
$3 = ","
scheme@(guile-user)> (locale-thousands-separator l)
$4 = "."
--8<---------------cut here---------------end--------------->8---

Thanks,
Ludo’.

Re: Formatted output with locale

[Vladimir Zhbanov]

[-- Attachment #1: Type: text/plain, Size: 4928 bytes --]

Hi Ludovic,

...

>> What does it output if you type
>>   (number->locale-string 10000.00 2 (make-locale LC_ALL "fr_FR.utf8"))
>>
>> that is with two zeros after the point?
>
> I get this:
>
> --8<---------------cut here---------------start------------->8---
> scheme@(guile-user)> (number->locale-string 10000.00 2 (make-locale LC_ALL
> "fr_FR.utf8"))
> $1 = "10 000,0"
> scheme@(guile-user)> (number->locale-string 10000.00 4  (make-locale LC_ALL
> "fr_FR.utf8"))
> $2 = "10 000,0"
> --8<---------------cut here---------------end--------------->8---
>
> … and that’s definitely a bug.
>
> Could you send it to bug-guile@gnu.org so we keep track of it?

I don't know how to do this, sorry. I see bugs are numbered
there. Is there a special form for bug submission? Or just will an
email do? And how to answer the mail already being there if I'm
not subscribed?

>
>>> That’s on GNU/Linux (glibc 2.24).
>>>
>>> Note that number formatting data comes from the C library.  What C
>>> library do you use?
>>
>> My glibc version is 2.19.
>
> It might be responsible for the incorrect thousand and fraction
> separators you observe in the de_DE output.  To dig further, you can
> check the low-level info provided by ‘nl_langinfo’:
>
> --8<---------------cut here---------------start------------->8---
> scheme@(guile-user)> ,use(ice-9 i18n)
> scheme@(guile-user)> (define l (make-locale LC_ALL "de_DE.utf8"))
> scheme@(guile-user)> (locale-digit-grouping l)
> $2 = (3 3 . #-1#)
> scheme@(guile-user)> (locale-decimal-point l)
> $3 = ","
> scheme@(guile-user)> (locale-thousands-separator l)
> $4 = "."
> --8<---------------cut here---------------end--------------->8---


My Scheme output is as follows:
--------------------------------8<--------------------------------
scheme@(guile-user)> ,use (ice-9 i18n)
scheme@(guile-user)> (define l (make-locale LC_ALL "ru_RU.UTF-8"))
scheme@(guile-user)> (locale-digit-grouping l)
$2 = (3 3 . #-1#)
scheme@(guile-user)> (locale-decimal-point l)
$3 = ","
scheme@(guile-user)> (locale-thousands-separator l)
$4 = "\xa0"
-------------------------------->8--------------------------------

(The last result is weird on my system though it's another story.)

I don't think all this somehow affects my output. I seem to have
found several bugs in number->locale-string().


1) Superfluous minus sign in output of zero:

--------------------------------8<--------------------------------
scheme@(guile-user)> (number->locale-string 0)
$5 = "0"
scheme@(guile-user)> (number->locale-string 0.0)
$6 = "-0,0"
-------------------------------->8--------------------------------

For this case a patch against the master branch is attached.


2) Wrong output for numbers less than 0.001

--------------------------------8<--------------------------------
scheme@(guile-user)> (number->locale-string 0.0009)
$7 = "9,0e-4"
scheme@(guile-user)> (number->locale-string 0.0009 2)
$8 = "9,0e"
scheme@(guile-user)> (number->locale-string 0.0009 1)
$9 = "9,0"
scheme@(guile-user)> (number->locale-string 0.0009 0)
$10 = "9"
-------------------------------->8--------------------------------

I don't really know how to proceed here. My naive approach is to
not modify the decimal part in such cases. The solution is in the
second patch.

3) Should output for the below cases differ?
--------------------------------8<--------------------------------
scheme@(guile-user)> (number->locale-string 10 2)
$2 = "10"
scheme@(guile-user)> (number->locale-string 10.0 2)
$3 = "10,0"
-------------------------------->8--------------------------------

4) Definitely a bug with different fraction-digits:
--------------------------------8<--------------------------------
scheme@(guile-user)> (number->locale-string 10.0 2)
$3 = "10,0"
scheme@(guile-user)> (number->locale-string 10.0 4)
$4 = "10,0"
-------------------------------->8--------------------------------

I think, the decimal parts should be zero padded to get desired
result.

5) Eventually, I don't really understand why substring() is used
to cut decimals:

--------------------------------8<--------------------------------
scheme@(guile-user)> (number->locale-string 0.99 0)
$8 = "0"
scheme@(guile-user)> (number->locale-string 0.99 1)
$9 = "0,9"
scheme@(guile-user)> (number->locale-string 0.99 2)
$10 = "0,99"
-------------------------------->8--------------------------------

I would prefer some rounding here in order to have the result as
follows:

--Desired results---------------8<-------------------------Begin--
scheme@(guile-user)> (number->locale-string 0.99 0)
$8 = "1"
scheme@(guile-user)> (number->locale-string 0.99 1)
$9 = "1,0"
scheme@(guile-user)> (number->locale-string 0.99 2)
$10 = "0,99"
--Desired results--------------->8---------------------------End--

Regards,
  Vladimir

[-- Attachment #2: 0001-Fix-output-of-0.0-in-number-locale-string.patch --]
[-- Type: application/octet-stream, Size: 936 bytes --]

From 679acd3803b07e63a1d3a221d6b13eee969a745e Mon Sep 17 00:00:00 2001
From: Vladimir Zhbanov <vzhbanov@gmail.com>
Date: Wed, 23 Nov 2016 12:56:09 +0300
Subject: [PATCH 1/2] Fix output of 0.0 in number->locale-string().

---
 module/ice-9/i18n.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/module/ice-9/i18n.scm b/module/ice-9/i18n.scm
index 1d12dd0..5e7e1cb 100644
--- a/module/ice-9/i18n.scm
+++ b/module/ice-9/i18n.scm
@@ -387,7 +387,7 @@ number of fractional digits to be displayed."
                                    (substring dec 0 fraction-digits)
                                    dec))))))
 
-    (let* ((external-repr (number->string (if (> number 0)
+    (let* ((external-repr (number->string (if (>= number 0)
                                               number
                                               (- number))))
            (int+dec   (string-split external-repr #\.))
-- 
2.1.4


[-- Attachment #3: 0002-Fix-number-locale-string-for-numbers-less-than-.001.patch --]
[-- Type: application/octet-stream, Size: 2091 bytes --]

From 3c2e718eff6ab1d8f529185896a4ea841122e0f0 Mon Sep 17 00:00:00 2001
From: Vladimir Zhbanov <vzhbanov@gmail.com>
Date: Wed, 23 Nov 2016 14:26:55 +0300
Subject: [PATCH 2/2] Fix number->locale-string() for numbers less than .001.

For such numbers, internally used number->string can return strings
containing exponent, such as, say "9.0e-4".  Just cutting of the decimal
part in this case is wrong, so the solution presented here is just to
use decimal part unchanged in such cases.
---
 module/ice-9/i18n.scm | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/module/ice-9/i18n.scm b/module/ice-9/i18n.scm
index 5e7e1cb..0c4d9f6 100644
--- a/module/ice-9/i18n.scm
+++ b/module/ice-9/i18n.scm
@@ -378,14 +378,17 @@ number of fractional digits to be displayed."
                 (else         "")))
          (decimal-part
           (lambda (dec)
-            (if (or (string=? dec "") (eq? 0 fraction-digits))
-                ""
-                (string-append (locale-decimal-point locale)
-                               (if (and (integer? fraction-digits)
-                                        (< fraction-digits
-                                           (string-length dec)))
-                                   (substring dec 0 fraction-digits)
-                                   dec))))))
+            (if (string-index dec #\-)
+                ;; For strings like "1.0e-4"
+                (string-append (locale-decimal-point locale) dec)
+                (if (or (string=? dec "") (eq? 0 fraction-digits))
+                 ""
+                 (string-append (locale-decimal-point locale)
+                                (if (and (integer? fraction-digits)
+                                         (< fraction-digits
+                                            (string-length dec)))
+                                    (substring dec 0 fraction-digits)
+                                    dec)))))))
 
     (let* ((external-repr (number->string (if (>= number 0)
                                               number
-- 
2.1.4