i18n test fail, guile-2.0.0

i18n test fail, guile-2.0.0

[David Fang]

Hi,

With guile-2.0.0, I see some i18n test failures on i686- and x86_64 
-pc-linux-gnu:

------------>8 snip 8<------------
Testing /home/davidfang/local/src/guile/guile-2.0.0/build32/meta/guile ...
with 
GUILE_LOAD_PATH=/home/davidfang/local/src/guile/guile-2.0.0/test-suite
Running 00-initial-env.test
Running alist.test
Running and-let-star.test
Running arbiters.test
Running arrays.test
Running asm-to-bytecode.test
Running bit-operations.test
Running bitvectors.test
Running brainfuck.test
Running bytevectors.test
Running c-api.test
Running chars.test
Running common-list.test
Running continuations.test
Running control.test
Running coverage.test
Running curried-definitions.test
Running ecmascript.test
Running elisp-compiler.test
Running elisp-reader.test
Running elisp.test
Running eval.test
UNRESOLVED: eval.test: stacks: stack involving a subr
UNRESOLVED: eval.test: stacks: stack involving a gsubr
UNRESOLVED: eval.test: stacks: arguments of a gsubr stack frame
Running exceptions.test
Running filesys.test
Running fluids.test
Running foreign.test
Running format.test
Running fractions.test
Running ftw.test
Running future.test
Running gc.test
Running getopt-long.test
Running goops.test
Running guardians.test
UNRESOLVED: guardians.test: g2-saved
UNRESOLVED: guardians.test: g2-garbage saved
Running hash.test
Running hooks.test
Running i18n.test
ERROR: i18n.test: number->locale-string: French: integer - arguments: 
((out-of-range #f "Value out of range ~S to ~S: ~S" (0 6 7) (7)))
ERROR: i18n.test: number->locale-string: French: fraction - arguments: 
((out-of-range #f "Value out of range ~S to ~S: ~S" (0 4 5) (5)))
ERROR: i18n.test: number->locale-string: French: fraction, 1 digit - 
arguments: ((out-of-range #f "Value out of range ~S to ~S: ~S" (0 4 5) 
(5)))
Running import.test
Running interp.test
Running keywords.test
Running list.test
Running load.test
Running match.test
Running modules.test
Running net-db.test
Running numbers.test
Running optargs.test
Running options.test
Running poe.test
Running popen.test
Running ports.test
Running posix.test
Running print.test
Running procprop.test
Running procs.test
Running q.test
Running r4rs.test
Running r5rs_pitfall.test
Running r6rs-arithmetic-bitwise.test
Running r6rs-arithmetic-fixnums.test
UNRESOLVED: r6rs-arithmetic-fixnums.test: fx+/carry: simple
UNRESOLVED: r6rs-arithmetic-fixnums.test: fx-/carry: simple
UNRESOLVED: r6rs-arithmetic-fixnums.test: fx*/carry: simple
Running r6rs-arithmetic-flonums.test
Running r6rs-base.test
Running r6rs-conditions.test
Running r6rs-control.test
Running r6rs-enums.test
Running r6rs-eval.test
Running r6rs-exceptions.test
Running r6rs-files.test
Running r6rs-hashtables.test
Running r6rs-lists.test
Running r6rs-ports.test
Running r6rs-records-inspection.test
Running r6rs-records-procedural.test
Running r6rs-records-syntactic.test
Running r6rs-unicode.test
UNRESOLVED: r6rs-unicode.test: string-foldcase: case folding expands 
string
Running ramap.test
Running rdelim.test
Running reader.test
Running receive.test
Running regexp.test
Running rnrs-libraries.test
Running session.test
Running signals.test
Running socket.test
Running srcprop.test
UNRESOLVED: srcprop.test: set-source-property!: setting the breakpoint 
property works
UNRESOLVED: srcprop.test: set-source-properties!: setting the breakpoint 
property works
Running srfi-1.test
Running srfi-10.test
Running srfi-11.test
Running srfi-13.test
Running srfi-14.test
Running srfi-19.test
Running srfi-26.test
Running srfi-27.test
Running srfi-31.test
Running srfi-34.test
Running srfi-35.test
Running srfi-37.test
Running srfi-38.test
Running srfi-39.test
Running srfi-4.test
Running srfi-42.test
Running srfi-45.test
Running srfi-6.test
Running srfi-60.test
Running srfi-67.test
Running srfi-69.test
Running srfi-88.test
Running srfi-9.test
Running statprof.test
Running strings.test
Running structs.test
Running sxml.fold.test
Running sxml.match.test
Running sxml.simple.test
Running sxml.ssax.test
Running sxml.transform.test
Running sxml.xpath.test
WARNING: (test-suite sxml-xpath): imported module (sxml xpath) overrides 
core binding `filter'
Running symbols.test
Running syncase.test
Running syntax.test
UNRESOLVED: syntax.test: letrec: bindings: initial bindings are undefined
UNRESOLVED: syntax.test: letrec*: bindings: initial bindings are undefined
Running texinfo.docbook.test
Running texinfo.serialize.test
Running texinfo.string-utils.test
Running texinfo.test
Running threads.test
Running time.test
Running tree-il.test
UNRESOLVED: tree-il.test: warnings: unused-toplevel: used by macro
Running version.test
UNRESOLVED: version.test: version reporting works
Running vlist.test
Running weaks.test
Running web-http.test
Running web-request.test
Running web-response.test
Running web-uri.test

Totals for this test run:
passes:                 34318
failures:               0
unexpected passes:      0
expected failures:      31
unresolved test cases:  15
untested test cases:    5
unsupported test cases: 9
errors:                 3

FAIL: check-guile
==================================
1 of 1 test failed
Please report to bug-guile@gnu.org
==================================
make[3]: *** [check-TESTS] Error 1
make[3]: Leaving directory 
`/home/davidfang/local/src/guile/guile-2.0.0/build32'
make[2]: *** [check-am] Error 2
make[2]: Leaving directory 
`/home/davidfang/local/src/guile/guile-2.0.0/build32'
make[1]: *** [check-recursive] Error 1
make[1]: Leaving directory 
`/home/davidfang/local/src/guile/guile-2.0.0/build32'
make: *** [check] Error 2
------------>8 snip 8<------------

% locale
LANG=en_US
LC_CTYPE="en_US"
LC_NUMERIC="en_US"
LC_TIME="en_US"
LC_COLLATE="en_US"
LC_MONETARY="en_US"
LC_MESSAGES="en_US"
LC_PAPER="en_US"
LC_NAME="en_US"
LC_ADDRESS="en_US"
LC_TELEPHONE="en_US"
LC_MEASUREMENT="en_US"
LC_IDENTIFICATION="en_US"
LC_ALL=



Let me know if you need any more information.



Fang


David Fang
http://www.csl.cornell.edu/~fang/
http://www.achronix.com/

Re: i18n test fail, guile-2.0.0

[Ludovic Courtès]

Hi,

David Fang <fang@csl.cornell.edu> writes:

> With guile-2.0.0, I see some i18n test failures on i686- and x86_64
> -pc-linux-gnu:

Which glibc version?  (See “ldd --version”.)

> ERROR: i18n.test: number->locale-string: French: integer - arguments:
> ((out-of-range #f "Value out of range ~S to ~S: ~S" (0 6 7) (7)))
> ERROR: i18n.test: number->locale-string: French: fraction - arguments:
> ((out-of-range #f "Value out of range ~S to ~S: ~S" (0 4 5) (5)))
> ERROR: i18n.test: number->locale-string: French: fraction, 1 digit - 
> arguments: ((out-of-range #f "Value out of range ~S to ~S: ~S" (0 4 5)
> (5)))

What does “locale -a | grep FR” say on your system?

Can you try this and report the backtrace?

   (let ((fr (make-locale LC_ALL "fr_FR.utf8")))
     (number->locale-string 123456 #t fr))

(If necessary, replace “fr_FR.utf8” with the locale name you found
above.)

Thanks for the report,
Ludo’.

Re: i18n test fail, guile-2.0.0

[David Fang]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3277 bytes --]

> Hi,
>
> David Fang <fang@csl.cornell.edu> writes:
>
>> With guile-2.0.0, I see some i18n test failures on i686- and x86_64
>> -pc-linux-gnu:
>
> Which glibc version?  (See ÿÿldd --versionÿÿ.)

%ldd `which guile-2.0`
         libguile-2.0.so.22 => /home/davidfang/local/lib/libguile-2.0.so.22 (0x00167000)
         libgc.so.1 => /home/davidfang/local/lib/libgc.so.1 (0x008bf000)
         libpthread.so.0 => /lib/tls/libpthread.so.0 (0x005c8000)
         libdl.so.2 => /lib/libdl.so.2 (0x004bf000)
         libffi.so.5 => /home/davidfang/local/lib/libffi.so.5 (0x0057c000)
         libunistring.so.0 => /home/davidfang/local/lib/libunistring.so.0 (0x00366000)
         libgmp.so.3 => /usr/lib/sse2/libgmp.so.3 (0x00989000)
         libltdl.so.3 => /usr/lib/libltdl.so.3 (0x00ee7000)
         libcrypt.so.1 => /lib/libcrypt.so.1 (0x0755a000)
         libm.so.6 => /lib/tls/libm.so.6 (0x0049a000)
         libc.so.6 => /lib/tls/libc.so.6 (0x005da000)
         /lib/ld-linux.so.2 (0x0034f000)

and

% ldd --version `which guile-2.0`
ldd (GNU libc) 2.3.4
Copyright (C) 2005 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR 
PURPOSE.
Written by Roland McGrath and Ulrich Drepper.


>> ERROR: i18n.test: number->locale-string: French: integer - arguments:
>> ((out-of-range #f "Value out of range ~S to ~S: ~S" (0 6 7) (7)))
>> ERROR: i18n.test: number->locale-string: French: fraction - arguments:
>> ((out-of-range #f "Value out of range ~S to ~S: ~S" (0 4 5) (5)))
>> ERROR: i18n.test: number->locale-string: French: fraction, 1 digit -
>> arguments: ((out-of-range #f "Value out of range ~S to ~S: ~S" (0 4 5)
>> (5)))
>
> What does ÿÿlocale -a | grep FRÿÿ say on your system?

% locale -a | grep FR
br_FR
br_FR@euro
br_FR.iso88591
br_FR.iso885915@euro
br_FR.utf8
fr_FR
fr_FR@euro
fr_FR.iso88591
fr_FR.iso885915@euro
fr_FR.utf8
oc_FR
oc_FR.iso88591
oc_FR.utf8

> Can you try this and report the backtrace?
>
>   (let ((fr (make-locale LC_ALL "fr_FR.utf8")))
>     (number->locale-string 123456 #t fr))
>
> (If necessary, replace ÿÿfr_FR.utf8ÿÿ with the locale name you found
> above.)

scheme@(guile-user)> (use-modules (ice-9 i18n))
scheme@(guile-user)> (let ((fr (make-locale LC_ALL "fr_FR.utf8"))) 
(number->locale-string 123456 #t fr))
ERROR: In procedure substring:
ERROR: Value out of range 0 to 6: 7

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,bt
In ice-9/i18n.scm:
    403:21  2 (number->locale-string 123456 #t #<locale 9e21960>)
    263:19  1 (%number-integer-part "123456" (-1 -1 -1 -1 -1 -1 . #) "")
In unknown file:
            0 (substring "123456" 0 7)
scheme@(guile-user) [1]> ,q
scheme@(guile-user)> (let ((fr (make-locale LC_ALL "fr_FR.iso88591"))) 
(number->locale-string 123456 #t fr))
ERROR: In procedure substring:
ERROR: Value out of range 0 to 6: 7

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,q


Does this help?


> Thanks for the report,
> Ludoÿÿ.
>

David Fang
http://www.csl.cornell.edu/~fang/
http://www.achronix.com/

Re: i18n test fail, guile-2.0.0

[Ludovic Courtès]

Hi,

(Cc: Bruno for additional insight.)

David Fang <fang@csl.cornell.edu> writes:

> % ldd --version `which guile-2.0`
> ldd (GNU libc) 2.3.4

Ouch, that’s very old.

>    263:19  1 (%number-integer-part "123456" (-1 -1 -1 -1 -1 -1 . #) "")

That’s the problem: the locale contains erroneous digit grouping
information.  It should be a circular list of ‘3’, like this:

  scheme@(ice-9 i18n)> (locale-digit-grouping (make-locale LC_ALL "fr_FR"))
  $5 = (3 . #0#)

Likewise, the thousand separator (last argument in the frame above) is
the empty string, whereas it should be a space:

  scheme@(ice-9 i18n)> (locale-thousands-separator (make-locale LC_ALL "fr_FR"))
  $7 = " "

I think it’s a bug in that version of glibc or its locale data.  I
suppose our nl_langinfo wrapper could filter out negative grouping
values since that doesn’t seem to have any meaning.  Thoughts?

Just to make sure can you try this variant:

  scheme@(ice-9 i18n)> (setlocale LC_ALL "fr_FR.utf8")
  $10 = "fr_FR.utf8"
  scheme@(ice-9 i18n)> (locale-digit-grouping)
  $12 = (3 . #0#)

Thanks,
Ludo’.

Re: i18n test fail, guile-2.0.0

[Bruno Haible]

Ludovic Courtès wrote:
> >    263:19  1 (%number-integer-part "123456" (-1 -1 -1 -1 -1 -1 . #) "")
> 
> That’s the problem: the locale contains erroneous digit grouping
> information.  It should be a circular list of ‘3’, like this:

You may be misunderstanding the glibc format of the grouping information
returned by the nl_langinfo function. As you can see
  - from glibc/stdio-common/printf_fp.c, function __guess_grouping,
  - from glibc/stdio-common/vfprintf.c, function group_number,
when the first value in the array is zero or negative or 127, it means
"no grouping should be done".

The POSIX description of 'grouping' for localeconv() [1] and the description in
glibc's manual [2] don't mention negative numbers, but yes, sometimes you need
to look into the source code to understand glibc specific details.

Bruno

[1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/localeconv.html
[2] http://www.gnu.org/software/libc/manual/html_node/General-Numeric.html

-- 
In memoriam Henri Salmide <http://en.wikipedia.org/wiki/Henri_Salmide>

Re: i18n test fail, guile-2.0.0

[Ludovic Courtès]

Hi Bruno,

Thanks for the quick reply.

Bruno Haible <bruno@clisp.org> writes:

> Ludovic Courtès wrote:
>> >    263:19  1 (%number-integer-part "123456" (-1 -1 -1 -1 -1 -1 . #) "")
>> 
>> That’s the problem: the locale contains erroneous digit grouping
>> information.  It should be a circular list of ‘3’, like this:
>
> You may be misunderstanding the glibc format of the grouping information
> returned by the nl_langinfo function. As you can see
>   - from glibc/stdio-common/printf_fp.c, function __guess_grouping,
>   - from glibc/stdio-common/vfprintf.c, function group_number,
> when the first value in the array is zero or negative or 127, it means
> "no grouping should be done".
>
> The POSIX description of 'grouping' for localeconv() [1] and the description in
> glibc's manual [2] don't mention negative numbers, but yes, sometimes you need
> to look into the source code to understand glibc specific details.

Yes, the description just mentions CHAR_MAX, which seems to be honored
in recent glibcs and Darwins.

I’ll submit a patch later today.

Thanks,
Ludo’.

Re: i18n test fail, guile-2.0.0

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 719 bytes --]

Hi,

Bruno Haible <bruno@clisp.org> writes:

> Ludovic Courtès wrote:
>> >    263:19  1 (%number-integer-part "123456" (-1 -1 -1 -1 -1 -1 . #) "")
>> 
>> That’s the problem: the locale contains erroneous digit grouping
>> information.  It should be a circular list of ‘3’, like this:
>
> You may be misunderstanding the glibc format of the grouping information
> returned by the nl_langinfo function. As you can see
>   - from glibc/stdio-common/printf_fp.c, function __guess_grouping,
>   - from glibc/stdio-common/vfprintf.c, function group_number,
> when the first value in the array is zero or negative or 127, it means
> "no grouping should be done".

David: can you try this patch?


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1248 bytes --]

diff --git a/libguile/i18n.c b/libguile/i18n.c
index 14dc9b9..c51df4a 100644
--- a/libguile/i18n.c
+++ b/libguile/i18n.c
@@ -1564,11 +1564,14 @@ SCM_DEFINE (scm_nl_langinfo, "nl-langinfo", 1, 1, 0,
 	  {
 	    char *p;
 
-	    /* In this cases, the result is to be interpreted as a list of
-	       numbers.  If the last item is `CHARS_MAX', it has the special
-	       meaning "no more grouping".  */
+	    /* In this cases, the result is to be interpreted as a list
+	       of numbers.  If the last item is `CHAR_MAX' or a negative
+	       number, it has the special meaning "no more grouping"
+	       (negative numbers aren't specified in POSIX but can be
+	       used by glibc; see
+	       <http://lists.gnu.org/archive/html/bug-guile/2011-02/msg00159.html>).  */
 	    result = SCM_EOL;
-	    for (p = c_result; (*p != '\0') && (*p != CHAR_MAX); p++)
+	    for (p = c_result; (*p > 0) && (*p != CHAR_MAX); p++)
 	      result = scm_cons (SCM_I_MAKINUM ((int) *p), result);
 
 	    {
@@ -1576,7 +1579,7 @@ SCM_DEFINE (scm_nl_langinfo, "nl-langinfo", 1, 1, 0,
 
 	      result = scm_reverse_x (result, SCM_EOL);
 
-	      if (*p != CHAR_MAX)
+	      if (*p == 0)
 		{
 		  /* Cyclic grouping information.  */
 		  if (last_pair != SCM_EOL)

[-- Attachment #3: Type: text/plain, Size: 142 bytes --]


The i18n.test tests that used to raise an error should now fail (because
glibc provides invalid locale data anyway.)

Thanks,
Ludo’.

Re: i18n test fail, guile-2.0.0

[David Fang]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1618 bytes --]

> Bruno Haible <bruno@clisp.org> writes:
>
>> Ludovic Courtès wrote:
>>>>    263:19  1 (%number-integer-part "123456" (-1 -1 -1 -1 -1 -1 . #) "")
>>>
>>> Thatÿÿs the problem: the locale contains erroneous digit grouping
>>> information.  It should be a circular list of ÿÿ3ÿÿ, like this:
>>
>> You may be misunderstanding the glibc format of the grouping information
>> returned by the nl_langinfo function. As you can see
>>   - from glibc/stdio-common/printf_fp.c, function __guess_grouping,
>>   - from glibc/stdio-common/vfprintf.c, function group_number,
>> when the first value in the array is zero or negative or 127, it means
>> "no grouping should be done".
>
> David: can you try this patch?


Hi,
 	After applying patch and re-running tests:

Running i18n.test
FAIL: i18n.test: number->locale-string: French: integer
FAIL: i18n.test: number->locale-string: French: fraction
FAIL: i18n.test: number->locale-string: French: fraction, 1 digit

I then tried:

% meta/uninstalled-env guile
GNU Guile 2.0.0
Copyright (C) 1995-2011 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> (use-modules (ice-9 i18n))
scheme@(guile-user)> (let ((fr (make-locale LC_ALL "fr_FR.utf8"))) 
(number->locale-string 123456 #t fr))
$1 = "123456"

Does this match your expectations?

Fang


David Fang
http://www.csl.cornell.edu/~fang/
http://www.achronix.com/

Re: i18n test fail, guile-2.0.0

[Ludovic Courtès]

Hi David,

Thanks for testing!

David Fang <fang@csl.cornell.edu> writes:

> scheme@(guile-user)> (use-modules (ice-9 i18n))
> scheme@(guile-user)> (let ((fr (make-locale LC_ALL "fr_FR.utf8")))
> (number->locale-string 123456 #t fr))
> $1 = "123456"
>
> Does this match your expectations?

It does.

(That still means the libc/locale data is broken, but at least it not
“our” problem.  ;-))

Thanks,
Ludo’.