bug#13342: Errors trying to build Guile 2.0.7

bug#13342: Errors trying to build Guile 2.0.7

[Peter Teeson]

[-- Attachment #1: Type: text/plain, Size: 2023 bytes --]

My environment is Mac Pro 4,1 Intel, MacOS 10.7.5 Lion
All required Guile libs are newly installed in /usr/local/

I get the following 2 errors, (A & B below) trying to build Guile.

(A) I solved this one by installing GNU readline-6.2.
Prior to that it must have used Apple's deadline which didn't have the rl_get_keymap_name
function.

You might wish to make a test for that function in configure and maybe make GNU readline a required lib?

-------- Tail Log for Make -----
…...
make  all-am
  CC       readline.lo
readline.c: In function 'init_bouncing_parens':
readline.c:445: warning: implicit declaration of function 'rl_get_keymap_name'
readline.c:445: warning: passing argument 1 of 'strncmp' makes pointer from integer without a cast
  CCLD     libguilereadline-v-18.la
Undefined symbols for architecture x86_64:
  "_rl_get_keymap_name", referenced from:
      _scm_init_readline in readline.o
ld: symbol(s) not found for architecture x86_64
collect2: ld returned 1 exit status
make[3]: *** [libguilereadline-v-18.la] Error 1
make[2]: *** [all] Error 2
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
Gandalf:guile-2.0.7 pteeson$

----------------- readline.c:445 --------
  if (strncmp (rl_get_keymap_name (rl_get_keymap ()), "vi", 2))
=========================================================


(B) I ran Make Check after Make as I always do and this error is reported
--------------- Make Check --------------------
…..
PASS: test-asmobs
bad return from expression `(f-sum -1 2000 -30000 40000000000)': expected 39999971999; got 39999972255
FAIL: test-ffi
PASS: test-list
….
PASS: test-pthread-create-secondary
==================================
1 of 29 tests failed
Please report to bug-guile@gnu.org
==================================
make[5]: *** [check-TESTS] Error 1
make[4]: *** [check-am] Error 2
make[3]: *** [check] Error 2
make[2]: *** [check-recursive] Error 1
make[1]: *** [check-recursive] Error 1
make: *** [check] Error 2




[-- Attachment #2: Type: text/html, Size: 3089 bytes --]

bug#13342: Errors trying to build Guile 2.0.7

[Ludovic Courtès]

Hi!

Peter Teeson <pteeson@me.com> skribis:

> I get the following 2 errors, (A & B below) trying to build Guile.
>
> (A) I solved this one by installing GNU readline-6.2.
> Prior to that it must have used Apple's deadline which didn't have the rl_get_keymap_name
> function.

Yes, that’s because you were using the Readline “compatibility” layer of
libeditline, which is shipped as part of MacOS.

> You might wish to make a test for that function in configure and maybe make GNU readline a required lib?

There are tests checking for GNU Readline.  The problem is that
libeditline attempts to replicate Readline’s API, but not quite, hence
the problem.  That’s why ‘README’ insists on using GNU Readline.

> bad return from expression `(f-sum -1 2000 -30000 40000000000)': expected 39999971999; got 39999972255
> FAIL: test-ffi

This is a known issue when building Guile with LLVM/Clang:

  http://bugs.gnu.org/10015
  http://bugs.gnu.org/10681

It would be great if you could investigate.

Thanks,
Ludo’.

bug#13342: Errors trying to build Guile 2.0.7

[Peter Teeson]

Hi Ludo':
Thanks for your reply and your explanation.

On 2013-01-03, at 10:22 AM, Ludovic Courtès wrote:
>> (A) I solved this one by installing GNU readline-6.2.
>> Prior to that it must have used Apple's deadline which didn't have the rl_get_keymap_name
>> function.
> 
> Yes, that’s because you were using the Readline “compatibility” layer of
> libeditline, which is shipped as part of MacOS.

>> You might wish to make a test for that function in configure and maybe make GNU readline a required lib?
> 
> There are tests checking for GNU Readline.  The problem is that
> libeditline attempts to replicate Readline’s API, but not quite, hence
> the problem.  That’s why ‘README’ insists on using GNU Readline.
Before I did/do anything I always read the README ;}
Somehow I missed the README insisting - I still don't see it there;}
There is a cryptic statement on page 1
"It will also use the libreadline library if it is available." 

and buried later on at line 413 (using BBEdit)
"guile-readline:
       The glue code for using GNU readline with Guile.  This
       will be build when configure can find a recent enough readline
       library on your system."

I agree there should be something mentioning the Mac situation and insisting on on GNU Readline. Where do you think it should be explicitly mentioned? In the Special Instructions For Some Systems section?

>> bad return from expression `(f-sum -1 2000 -30000 40000000000)': expected 39999971999; got 39999972255
>> FAIL: test-ffi
> 
> This is a known issue when building Guile with LLVM/Clang:
> 
> http://bugs.gnu.org/10015
> http://bugs.gnu.org/10681
> 
> It would be great if you could investigate.

I'd be glad to try and help but please give me pointers on how to approach this.
This will be a learning exercise for me as I am familiar with Xcode and it's debugger (GDB and LLDB) but have only used it developing Cocoa apps.

Care to suggest how I should approach things?

thanks and respect….

Peter

bug#13342: Errors trying to build Guile 2.0.7

[Peter Teeson]

Hi Ludo':
Thanks for your reply and your explanation.

On 2013-01-03, at 10:22 AM, Ludovic Courtès wrote:
>> (A) I solved this one by installing GNU readline-6.2.
>> Prior to that it must have used Apple's deadline which didn't have the rl_get_keymap_name
>> function.
> 
> Yes, that’s because you were using the Readline “compatibility” layer of
> libeditline, which is shipped as part of MacOS.

>> You might wish to make a test for that function in configure and maybe make GNU readline a required lib?
> 
> There are tests checking for GNU Readline.  The problem is that
> libeditline attempts to replicate Readline’s API, but not quite, hence
> the problem.  That’s why ‘README’ insists on using GNU Readline.
Before I did/do anything I always read the README ;}
Somehow I missed the README insisting - I still don't see it there;}
There is a cryptic statement on page 1
"It will also use the libreadline library if it is available." 

and buried later on at line 413 (using BBEdit)
"guile-readline:
      The glue code for using GNU readline with Guile.  This
      will be build when configure can find a recent enough readline
      library on your system."

I agree there should be something mentioning the Mac situation and insisting on on GNU Readline. Where do you think it should be explicitly mentioned? In the Special Instructions For Some Systems section?

>> bad return from expression `(f-sum -1 2000 -30000 40000000000)': expected 39999971999; got 39999972255
>> FAIL: test-ffi
> 
> This is a known issue when building Guile with LLVM/Clang:
> 
> http://bugs.gnu.org/10015
> http://bugs.gnu.org/10681
> 
> It would be great if you could investigate.

I'd be glad to try and help but please give me pointers on how to approach this.
This will be a learning exercise for me as I am familiar with Xcode and it's debugger (GDB and LLDB) but have only used it developing Cocoa apps.

Care to suggest how I should approach things?

thanks and respect….

Peter

bug#13342: Errors trying to build Guile 2.0.7

[Ludovic Courtès]

Hi,

Peter Teeson <pteeson@me.com> skribis:

> On 2013-01-03, at 10:22 AM, Ludovic Courtès wrote:
>>> (A) I solved this one by installing GNU readline-6.2.
>>> Prior to that it must have used Apple's deadline which didn't have the rl_get_keymap_name
>>> function.
>> 
>> Yes, that’s because you were using the Readline “compatibility” layer of
>> libeditline, which is shipped as part of MacOS.
>
>>> You might wish to make a test for that function in configure and maybe make GNU readline a required lib?
>> 
>> There are tests checking for GNU Readline.  The problem is that
>> libeditline attempts to replicate Readline’s API, but not quite, hence
>> the problem.  That’s why ‘README’ insists on using GNU Readline.
> Before I did/do anything I always read the README ;}
> Somehow I missed the README insisting - I still don't see it there;}
> There is a cryptic statement on page 1
> "It will also use the libreadline library if it is available." 
>
> and buried later on at line 413 (using BBEdit)
> "guile-readline:
>       The glue code for using GNU readline with Guile.  This
>       will be build when configure can find a recent enough readline
>       library on your system."
>
> I agree there should be something mentioning the Mac situation and insisting on on GNU Readline. Where do you think it should be explicitly mentioned? In the Special Instructions For Some Systems section?

Well, you’re right that it doesn’t really insist.  ;-)  It does mention
GNU Readline though, which is not what was being used here.

I’m reluctant to adding anything special about libeditline, since
they’re really failing to be compatible with Readline here, and it’s
something to be fixed in libeditline.

>>> bad return from expression `(f-sum -1 2000 -30000 40000000000)': expected 39999971999; got 39999972255
>>> FAIL: test-ffi
>> 
>> This is a known issue when building Guile with LLVM/Clang:
>> 
>> http://bugs.gnu.org/10015
>> http://bugs.gnu.org/10681
>> 
>> It would be great if you could investigate.
>
> I'd be glad to try and help but please give me pointers on how to approach this.

I’d first add printfs in test_ffi_sum (in test-ffi-lib.c) to see if it
gets the right arguments.  Depending on the result, I’d try to call
test_ffi_sum from C with the same arguments to see if it’s an
integer-wrapping issue.

I’ll try a Clang build when time permits to see what happens.

Thanks,
Ludo’.

bug#13386: Fwd: bug#13342: Errors trying to build Guile 2.0.7

[Peter Teeson]

[-- Attachment #1: Type: text/plain, Size: 2822 bytes --]



Begin forwarded message:

> From: Peter Teeson <pteeson@me.com>
> Subject: Re: bug#13342: Errors trying to build Guile 2.0.7
> Date: 7 January, 2013 8:21:59 PM EST
> To: Ludovic Courtès <ludo@gnu.org>
> 
> Hi Ludo:
> 
> On 2013-01-04, at 12:23 PM, Ludovic Courtès wrote:
>>>>> bad return from expression `(f-sum -1 2000 -30000 40000000000)': expected 39999971999; got 39999972255
>>>>> FAIL: test-ffi
>>>> 
>>>> This is a known issue when building Guile with LLVM/Clang:
>>>> 
>>>> http://bugs.gnu.org/10015
>>>> http://bugs.gnu.org/10681
>>>> 
>>>> It would be great if you could investigate.
> 
> Here is the result of my investigation - do you agree?
> My hypothesis is that the scheme interpreter is not calculating the sum correctly based on the following:
> 
> (0) First observe this
> "In Apple's version of GCC, both cc and gcc are actually symbolic links to the llvm-gcc compiler.  Similarly, c++ and g++ are links to llvm-g++."
> 
> (1) Also we note that 39999972255 - 39999971999 = 256!
> 
> (2) This program
> //
> //  main.c
> //  testffi
> //
> //  Created by Peter Teeson on 13-01-07.
> //  Copyright (c) 2013 PHT Software. All rights reserved.
> //
> 
> #include <stdio.h>
> #include <inttypes.h>
> 
> int64_t test_ffi_sum (int8_t a, int16_t b,
>                           int32_t c, int64_t d);
> int64_t test_ffi_sum (int8_t a, int16_t b,
>                       int32_t c, int64_t d)
> {
>     printf("int64 d %" PRId64 " %#llX \n", d,d);
>     printf("int32 c %" PRId32 " %#X \n", c,c);
>     printf("int16 b %" PRId16 " %#X \n", b,b);
>     printf("int08 a %" PRId8 " %#X \n", a,a);
> 
>     int64_t sum = d + c + b + a;
>     printf("int64 sum %" PRId64 " %#llX \n", sum,sum);
> 
>     return sum;
> }
> int main(int argc, const char * argv[])
> {
>     test_ffi_sum(-1, 2000, -30000, 40000000000);
>     return 0;
> }
> 
> (3) produces this output
> 
> int64 d 40000000000 0X9502F9000 
> int32 c -30000 0XFFFF8AD0 
> int16 b 2000 0X7D0 
> int08 a -1 0XFFFFFFFF 
> int64 sum 39999971999 0X9502F229F 
> 
> (4) This function in  /guile-2.0.7/test-suite/standalone/test-ffi 
> ;;
> ;; Multiple int args of differing types
> ;;
> (define f-sum
>   (pointer->procedure int64 (dynamic-func "test_ffi_sum" lib)
>                       (list int8 int16 int32 int64)))
> (test (f-sum -1 2000 -30000 40000000000)
>       (+ -1 2000 -30000 40000000000))
> 
> might be the culprit and I am guessing that it is this expression
> 
> (+ -1 2000 -30000 40000000000) 
> 
> which the scheme interpreter is calculating incorrectly. Probably related to -1;
> 
> Since I am not familiar with the scheme/guile language I can't go any further than this without help.
> Let me know if I can do more.
> 
> respect…..
> 
> Peter


[-- Attachment #2: Type: text/html, Size: 17689 bytes --]

bug#13342: [PARTIALLY SOLVED] bug#13342: Errors trying to build Guile 2.0.7

[Peter Teeson]

[-- Attachment #1: Type: text/plain, Size: 1453 bytes --]

Hi Ludo:
I think the reason that we get this error is that the "a" argument is declared to be only 8 bits wide
but is passed in as -1 which is correctly represented as 0xFFFFFFFF

However an 8-bit representation is 0xFF which is only 255.

Here is how I tracked it down:
(0) Took the printf's from my test program and placed them in 

scm_t_int64 test_ffi_sum (scm_t_int8 a, scm_t_int16 b,
                          scm_t_int32 c, scm_t_int64 d);
scm_t_int64 test_ffi_sum (scm_t_int8 a, scm_t_int16 b,
                          scm_t_int32 c, scm_t_int64 d)
{
    scm_t_int64 sum;
    printf("scm_t_int64 d %" "ll" "d" " %#llX \n", d,d);
    printf("scm_t_int32 c %" "d" " %#X \n", c,c);
    printf("scm_t_int16 b %" "hd" " %#X \n", b,b);
    printf("scm_t_int8 a %" "hh" "d" " %#X \n", a,a);
    sum = d + c + b + a;
    printf("scm_t_int64 sum %" "ll" "d" " %#llX \n", sum,sum);

    a = -1; //  NOTE:re-assign of a!!!!
    printf("scm_t_int16 a %" "hd" " %#X \n", a,a);

    return d + c + b + a;
}

(1) Did make and make check with the following result

scm_t_int64 d 40000000000 0X9502F9000 
scm_t_int32 c -30000 0XFFFF8AD0 
scm_t_int16 b 2000 0X7D0 
scm_t_int8 a -1 0XFF 
scm_t_int64 sum 39999972255 0X9502F239F 
scm_t_int16 a -1 0XFFFFFFFF 
PASS: test-ffi


(2) Observe that the test passed. 
So I think my original suspicion seems to have been confirmed.

I do not know where to go from here but await your further instructions.

respect

Peter

[-- Attachment #2: Type: text/html, Size: 5458 bytes --]

bug#13342: Errors trying to build Guile 2.0.7 [PARTIALLY SOLVED]

[Peter Teeson]

[-- Attachment #1: Type: text/plain, Size: 1489 bytes --]

Hi Ludo:

I believe my conjecture that -1 was correct based on the following:

(0) I took the printf statements from my test program and placed them in 
test-ffi-sum:

scm_t_int64 test_ffi_sum (scm_t_int8 a, scm_t_int16 b,
                          scm_t_int32 c, scm_t_int64 d);
scm_t_int64 test_ffi_sum (scm_t_int8 a, scm_t_int16 b,
                          scm_t_int32 c, scm_t_int64 d)
{
    scm_t_int64 sum;
    printf("scm_t_int64 d %" "ll" "d" " %#llX \n", d,d);
    printf("scm_t_int32 c %" "d" " %#X \n", c,c);
    printf("scm_t_int16 b %" "hd" " %#X \n", b,b);
    printf("scm_t_int8 a %" "hh" "d" " %#X \n", a,a);

    sum = d + c + b + a;
    printf("scm_t_int64 sum %" "ll" "d" " %#llX \n", sum,sum);

    a = -1; // NOTE re-assinging of a!
    printf("scm_t_int16 a %" "hd" " %#X \n", a,a);

    return d + c + b + a;
}

(1) Ran make 

Making all in standalone
make  all-am
  CC       libtest_ffi_la-test-ffi-lib.lo
  CCLD     libtest-ffi.la

and then make check

scm_t_int64 d 40000000000 0X9502F9000 
scm_t_int32 c -30000 0XFFFF8AD0 
scm_t_int16 b 2000 0X7D0 
scm_t_int8 a -1 0XFF 
scm_t_int64 sum 39999972255 0X9502F239F 
scm_t_int16 a -1 0XFFFFFFFF 
PASS: test-ffi

(2) Since scm_t_int8 is only 8 bits wide it's maximum value is 0xFF <=> 255
Reassigning "a" to be -1 allowed it to pass the test.

(3) Therefore I conclude that this is why the test fails (correctly IMHO).
You can only get a litre of milk into a litre jar!

(4) Where do we go from here?

respect

Peter

[-- Attachment #2: Type: text/html, Size: 5601 bytes --]

bug#13342: [PARTIALLY SOLVED] bug#13342: Errors trying to build Guile 2.0.7

[Ludovic Courtès]

Hi Peter,

Sorry for the late reply, and thanks for investigating.

I reproduced the bug on GNU/Linux with Clang 3.1.

Peter Teeson <peter.teeson@me.com> skribis:

> I think the reason that we get this error is that the "a" argument is declared to be only 8 bits wide
> but is passed in as -1 which is correctly represented as 0xFFFFFFFF
>
> However an 8-bit representation is 0xFF which is only 255.

This is part of the issue.

Consider this example:

--8<---------------cut here---------------start------------->8---
#include <stdint.h>

int64_t
test_sum (int8_t a, int64_t b)
{
  return a + b;
}
--8<---------------cut here---------------end--------------->8---

When compiled with GCC 4.6, the assembly is:

--8<---------------cut here---------------start------------->8---
test_sum:
.LFB0:
	.cfi_startproc
	movsbq	%dil, %rdi
	leaq	(%rdi,%rsi), %rax
	ret
	.cfi_endproc
--8<---------------cut here---------------end--------------->8---

With Clang 3.1, it is:

--8<---------------cut here---------------start------------->8---
test_sum:                               # @test_sum
	.cfi_startproc
# BB#0:
	movslq	%edi, %rax
	addq	%rsi, %rax
	ret
--8<---------------cut here---------------end--------------->8---

The ‘movsbq’ emitted by GCC arranges to keep only the 8 LSBs.  Clang
does no such thing, thus keeping all the bits of the first operand in
the addition.

I looked at Section 3.2.3 (“Parameter Passing”) of the SysV ABI x86_64
PS but couldn’t find any evidence as to what the correct behavior is.

However, on the caller side, both compilers emit the same code.  This
program:

--8<---------------cut here---------------start------------->8---
#include <stdint.h>

extern int64_t test_sum (int8_t a, int64_t b);

int64_t
foo (void)
{
  return test_sum (-1, 123132);
}
--8<---------------cut here---------------end--------------->8---

leads to the following assembly with both compilers:

--8<---------------cut here---------------start------------->8---
foo:                                    # @foo
	.cfi_startproc
	movl	$-1, %edi
	movl	$123132, %esi           # imm = 0x1E0FC
	jmp	test_sum                # TAILCALL
--8<---------------cut here---------------end--------------->8---

(And as we’ve seen, libffi does the same.)

So that seems to indicate that the callee code generated by Clang is
incorrect as it fails to discard the 24 MSBs of its first argument.

Could you report it as a Clang/LLVM bug (keeping 13342@debbugs.gnu.org
Cc’d)?

Thanks!

Ludo’.

bug#13342: [PARTIALLY SOLVED] bug#13342: Errors trying to build Guile 2.0.7

[Peter Teeson]

[-- Attachment #1: Type: text/plain, Size: 2244 bytes --]

Hi Ludo:
MacOS 10.7.5 Intel Dual CPU quad cores 

Many thanks for your reply. A nice demo but, no disrespect, maybe not really the issue.
Looking at the generated code from each compiler was going to be my next step after
taking a look at the stack and regs to see what got passed/pushed/popped in the call to test_sum.
Which is why I posted about how can I debug/step through the interpreter. 
I still don't understand how to do that even though I read through the manual.

My experiment in building guile with both compilers indicate they both produce the same error.
bad return from expression `(f-sum -1 2000 -30000 40000000000)': expected 39999971999; got 39999972255
I am still of the opinion that it's an interpreter issue and not a compiler one. 

I ask for help in confirming my understanding of this scheme code from the file 
/test-suite/standalone/test-ffi
;;
;; Multiple int args of differing types
;;
(define f-sum
  (pointer->procedure int64 (dynamic-func "test_ffi_sum" lib)
                      (list int8 int16 int32 int64)))
(test (f-sum -1 2000 -30000 40000000000)  ;; This calls the c function f-sum
      (+ -1 2000 -30000 40000000000))	     ;;  This executes the scheme interpreter to do the sum

Is my understanding correct??????

If so please explain to me how the interpreter knows that the -1 in this expression is an int8?
My suspicion is that it thinks it's an int16 and that's why we see the difference!!

I don't know how to pass an int8 of -1 to the interpreter in REPL mode.
Otherwise I'd try that.

============= from Terminal REPL session =============
Gandalf:guile-2.0.7 pteeson$ guile
GNU Guile 2.0.7
<snipped comments>

scheme@(guile-user)> (+ -1 2000 -30000 40000000000)
$1 = 39999971999
scheme@(guile-user)> ,q

which matches the fail message value.
bad return from expression `(f-sum -1 2000 -30000 40000000000)': expected 39999971999; got 39999972255

================= from http://pubs.opengroup.org/onlinepubs/009695399/basedefs/stdint.h.html =========== 
Limits of exact-width integer types
Minimum values of exact-width signed integer types:
{INTN_MIN}
	Exactly -(2 N-1)
Maximum values of exact-width signed integer types:
{INTN_MAX}
	Exactly 2N-1 -1
==========================================



[-- Attachment #2: Type: text/html, Size: 3573 bytes --]

bug#13342: [PARTIALLY SOLVED] bug#13342: Errors trying to build Guile 2.0.7

[Mark H Weaver]

Hi Ludovic,

Thanks for looking into this!  I think I understand the problem now.

ludo@gnu.org (Ludovic Courtès) writes:
> Consider this example:
>
> #include <stdint.h>
>
> int64_t
> test_sum (int8_t a, int64_t b)
> {
>   return a + b;
> }
>
> When compiled with GCC 4.6, the assembly is:
>
> test_sum:
> .LFB0:
> 	.cfi_startproc
> 	movsbq	%dil, %rdi
> 	leaq	(%rdi,%rsi), %rax
> 	ret
> 	.cfi_endproc
>
> With Clang 3.1, it is:
>
> test_sum:                               # @test_sum
> 	.cfi_startproc
> # BB#0:
> 	movslq	%edi, %rax
> 	addq	%rsi, %rax
> 	ret
>
> The ‘movsbq’ emitted by GCC arranges to keep only the 8 LSBs.  Clang
> does no such thing, thus keeping all the bits of the first operand in
> the addition.

This is the key revelation, although I've reached a different conclusion
about where the bug is.

> I looked at Section 3.2.3 (“Parameter Passing”) of the SysV ABI x86_64
> PS but couldn’t find any evidence as to what the correct behavior is.

I read the same section, and although it is not as clear as I'd prefer,
my interpretation is that the caller is responsible for sign-extending
signed chars to ints.  This is also consistent with something I vaguely
remember reading in K&R long ago, namely that 'char' and 'short'
arguments are coerced to 'int' before making a function call.

Clang strictly requires callers to sign-extend, whereas GCC is tolerant
of callers who fail to do so.  IMO, both behaviors are permitted by the
ABI.

The problem is that libffi does *not* sign-extend arguments passed in
registers when making calls, which is IMO a bug that has gone (mostly)
unnoticed because of the tolerance and ubiquity of GCC.

> However, on the caller side, both compilers emit the same code.  This
> program:
>
> #include <stdint.h>
>
> extern int64_t test_sum (int8_t a, int64_t b);
>
> int64_t
> foo (void)
> {
>   return test_sum (-1, 123132);
> }
>
> leads to the following assembly with both compilers:
>
> foo:                                    # @foo
> 	.cfi_startproc
> 	movl	$-1, %edi
> 	movl	$123132, %esi           # imm = 0x1E0FC
> 	jmp	test_sum                # TAILCALL
>
> (And as we’ve seen, libffi does the same.)

No, libffi does *not* do the same.  Take a look at the relevant code:

  https://github.com/atgreen/libffi/blob/master/src/x86/ffi64.c#L488

As you can see in lines 487 and 488, arguments passed in registers are
never sign-extended, but rather zero-extended.  The register values are
then copied whole in the darwin-specific assembly stub:

  https://github.com/atgreen/libffi/blob/master/src/x86/darwin64.S#L61

Interestingly, arguments passed on the stack *are* sign-extended:

  https://github.com/atgreen/libffi/blob/master/src/x86/darwin64.S#L120

* * * * *

In summary, I think this is a bug in libffi.

Note that it has already been reported that the libffi testsuite shows
many failures on OS X Lion, and the failures appear to be related to
this precise issue:

  http://sourceware.org/ml/libffi-discuss/2012/msg00162.html

The libffi maintainer wrote "I'm going to chalk this up to compiler
bugs", based on his observation that the tests worked properly when
compiled with -O0.  I think it's time to raise this issue again on the
libffi-discuss mailing list.

In any case, it's certainly not a bug in Guile.  The bug is either in
LLVM/Clang or libffi, depending on how one chooses to interpret the
x86-64 API.

    Regards,
      Mark

bug#13342: [PARTIALLY SOLVED] bug#13342: Errors trying to build Guile 2.0.7

[Peter Teeson]

Hi Mark:
Thanks for your comments and I agree I was wrong.
Ludovic pointed out GCC dropping off the MSB 24-bits whereas Clang does not.
You pointed out the lack of sign extension in the libffi implementation.

I would think it would be easier to make the change to libffi 
rather than expect changes to be made to the compilers.
But that's just my opinion.

Anyway glad that you wizards have nailed it.

OK if I bow out now and let others fix it?

respect….

Peter

bug#13342: [PARTIALLY SOLVED] bug#13342: Errors trying to build Guile 2.0.7

[Ludovic Courtès]

retitle 13342 Clang, the FFI, and 8-bit signed integers
merge 13342 13386
thanks

Hello!

Mark H Weaver <mhw@netris.org> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:

[...]

>> I looked at Section 3.2.3 (“Parameter Passing”) of the SysV ABI x86_64
>> PS but couldn’t find any evidence as to what the correct behavior is.
>
> I read the same section, and although it is not as clear as I'd prefer,
> my interpretation is that the caller is responsible for sign-extending
> signed chars to ints.

What exactly in that section makes you think so?

> Clang strictly requires callers to sign-extend, whereas GCC is tolerant
> of callers who fail to do so.  IMO, both behaviors are permitted by the
> ABI.
>
> The problem is that libffi does *not* sign-extend arguments passed in
> registers when making calls, which is IMO a bug that has gone (mostly)
> unnoticed because of the tolerance and ubiquity of GCC.

Oh, I see.

>> However, on the caller side, both compilers emit the same code.  This
>> program:
>>
>> #include <stdint.h>
>>
>> extern int64_t test_sum (int8_t a, int64_t b);
>>
>> int64_t
>> foo (void)
>> {
>>   return test_sum (-1, 123132);
>> }
>>
>> leads to the following assembly with both compilers:
>>
>> foo:                                    # @foo
>> 	.cfi_startproc
>> 	movl	$-1, %edi
>> 	movl	$123132, %esi           # imm = 0x1E0FC
>> 	jmp	test_sum                # TAILCALL
>>
>> (And as we’ve seen, libffi does the same.)
>
> No, libffi does *not* do the same.  Take a look at the relevant code:
>
>   https://github.com/atgreen/libffi/blob/master/src/x86/ffi64.c#L488

Indeed!

[...]

> In summary, I think this is a bug in libffi.
>
> Note that it has already been reported that the libffi testsuite shows
> many failures on OS X Lion, and the failures appear to be related to
> this precise issue:

Note that it’s not just Darwin: I tested with Clang on GNU/Linux.

>   http://sourceware.org/ml/libffi-discuss/2012/msg00162.html
>
> The libffi maintainer wrote "I'm going to chalk this up to compiler
> bugs", based on his observation that the tests worked properly when
> compiled with -O0.  I think it's time to raise this issue again on the
> libffi-discuss mailing list.

Would you like to email them?

Anyway, thanks for the thorough investigation!

Ludo’.

bug#13342: [PARTIALLY SOLVED] bug#13342: Errors trying to build Guile 2.0.7

[Peter Teeson]

Hi:
Do you also need to retitle and merge 10015 and 10681?

These were the similar bugs you mentioned when I first reported the issue.

Peter

On 2013-01-28, at 7:56 AM, Ludovic Courtès wrote:
> retitle 13342 Clang, the FFI, and 8-bit signed integers
> merge 13342 13386
> thanks

bug#13342: [PARTIALLY SOLVED] bug#13342: Errors trying to build Guile 2.0.7

[Ludovic Courtès]

merge 13342 10015 10681
thanks

Peter Teeson <peter.teeson@me.com> skribis:

> Do you also need to retitle and merge 10015 and 10681?

Indeed, thanks!

Ludo’.

bug#13342: Errors trying to build Guile 2.0.7

[Mark H Weaver]

FYI, I've posted about this to the libffi-discuss mailing list.

http://sourceware.org/ml/libffi-discuss/2013/msg00012.html

    Mark

bug#13342: Errors trying to build Guile 2.0.7

[Ludovic Courtès]

tags 13342 notabug
thanks

Hello!

Mark H Weaver <mhw@netris.org> skribis:

> FYI, I've posted about this to the libffi-discuss mailing list.
>
> http://sourceware.org/ml/libffi-discuss/2013/msg00012.html

Thanks!  So, according to
<http://gcc.gnu.org/ml/gcc/2013-01/msg00449.html>, this is an LLVM bug.

Peter: would you like to report it over at LLVM?

Anyway, closing this bug now.  Thanks!

Ludo’.

bug#13386: bug#13342: closed (Re: bug#13342: Errors trying to build Guile 2.0.7) , bug#13386: closed (Re: bug#13342: Errors trying to build Guile 2.0.7)

[Peter Teeson]

Hi guys:
Thanks for closing off those bugs.
I have read all the emails you linked in your close emails and 
completely understand the conversation. 

Specially the comment by Michael Matz.
That clearly confirms that it is an LLVM compiler bug.

Good job both of you for digging down deeper and making things so clear.
At least now we know.

Would you please report this over at LLVM since you did the work to discover it.

Thanks again for your great patience and kindness in helping my fumbling efforts.

respect…..

Peter

bug#13342: closed (Re: bug#13342: Errors trying to build Guile 2.0.7) , bug#13386: closed (Re: bug#13342: Errors trying to build Guile 2.0.7)

[Mark H Weaver]

Hi Peter,

Peter Teeson <pteeson@me.com> writes:
> Would you please report this over at LLVM since you did the work to
> discover it.

Okay, I'll take care of it.

I'm also working with the libffi folks to work around this problem in
the interim.  The next libffi release (due out fairly soon) will
interface properly with the existing LLVM-compiled code on your system.

Thanks again for helping us track this down.

   Regards,
     Mark

bug#13386: bug#13342: closed (Re: bug#13342: Errors trying to build Guile 2.0.7) , bug#13386: closed (Re: bug#13342: Errors trying to build Guile 2.0.7)

[Peter Teeson]

Hi Mark:
I read the thread on libffi-discuss.
When you think things are ready to test I'll be glad to do so on my Mac setup.

At present my boot system is MacOS 10.7.5 Lion.
But I can boot into previous versions (Leopard 10.5, Snow Leopard 10.6)

Just so you know….

Peter
On 2013-01-31, at 3:20 PM, Mark H Weaver wrote:
> I'm also working with the libffi folks to work around this problem in
> the interim.  The next libffi release (due out fairly soon) will
> interface properly with the existing LLVM-compiled code on your system.

bug#13386: bug#13342: closed (Re: bug#13342: Errors trying to build Guile 2.0.7) , bug#13386: closed (Re: bug#13342: Errors trying to build Guile 2.0.7)

[Mark H Weaver]

[-- Attachment #1: Type: text/plain, Size: 417 bytes --]

Hi Peter,

Peter Teeson <pteeson@me.com> wrote:
> When you think things are ready to test I'll be glad to do so on my
> Mac setup.

I believe that this libffi patch will fix the problem.  Would you like
to test it?  It would also be helpful if you could run "make check" on
both libffi and Guile and let us know of any failures.  There's no need
to test on older versions of the OS.

    Thanks again!
        Mark



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: SIGN_EXTEND_SINT_ARGS.patch --]
[-- Type: text/x-diff, Size: 1178 bytes --]

diff --git a/src/x86/ffi64.c b/src/x86/ffi64.c
index b8a823d..2014af2 100644
--- a/src/x86/ffi64.c
+++ b/src/x86/ffi64.c
@@ -484,8 +484,25 @@ ffi_call (ffi_cif *cif, void (*fn)(void), void *rvalue, void **avalue)
 		{
 		case X86_64_INTEGER_CLASS:
 		case X86_64_INTEGERSI_CLASS:
-		  reg_args->gpr[gprcount] = 0;
-		  memcpy (&reg_args->gpr[gprcount], a, size < 8 ? size : 8);
+		  /* Sign-extend integer arguments passed in general
+		     purpose registers, to cope with the fact that
+		     LLVM incorrectly assumes that this will be done
+		     (the x86-64 PS ABI does not specify this). */
+		  switch (arg_types[i]->type)
+		    {
+		    case FFI_TYPE_SINT8:
+		      *(SINT64 *)&reg_args->gpr[gprcount] = (SINT64) *((SINT8 *) a);
+		      break;
+		    case FFI_TYPE_SINT16:
+		      *(SINT64 *)&reg_args->gpr[gprcount] = (SINT64) *((SINT16 *) a);
+		      break;
+		    case FFI_TYPE_SINT32:
+		      *(SINT64 *)&reg_args->gpr[gprcount] = (SINT64) *((SINT32 *) a);
+		      break;
+		    default:
+		      reg_args->gpr[gprcount] = 0;
+		      memcpy (&reg_args->gpr[gprcount], a, size < 8 ? size : 8);
+		    }
 		  gprcount++;
 		  break;
 		case X86_64_SSE_CLASS:

bug#13342: closed (Re: bug#13342: Errors trying to build Guile 2.0.7) , bug#13386: closed (Re: bug#13342: Errors trying to build Guile 2.0.7)

[Peter Teeson]

[-- Attachment #1: Type: text/plain, Size: 959 bytes --]

Hi Mark:
The short answer is:
make  check-am
…...
make  check-TESTS
PASS: test-system-cmds
…...
PASS: test-asmobs
PASS: test-ffi
PASS: test-list
……
===================
All 29 tests passed
===================


In more detail the following steps:
Unzipped libffi-3.0.11.tar.gz
Downloaded libffi-3.0.11-includedir-1.patch
Applied that patch
Had to manually apply your diff file patch

(I'll explain in more detail if you wish)

Built and installed libffi in the usual manner.

Then did the same process for guile-2.0.7.tar.gz
did ./configure && make && make check
got the above results.

Here's the summary
Totals for this test run:
passes:                 35268
failures:               0
unexpected passes:      0
expected failures:      6
unresolved test cases:  34
untested test cases:    1
unsupported test cases: 9
errors:                 0

PASS: check-guile


Anything else I can do to help?

Peter





[-- Attachment #2: Type: text/html, Size: 2405 bytes --]

bug#13386: bug#13342: closed (Re: bug#13342: Errors trying to build Guile 2.0.7) , bug#13386: closed (Re: bug#13342: Errors trying to build Guile 2.0.7)

[Mark H Weaver]

Peter Teeson <pteeson@me.com> writes:
> …...
> PASS: test-asmobs
> PASS: test-ffi
> PASS: test-list
> ……
> ===================
> All 29 tests passed
> ===================

Excellent!  We can now submit the libffi patch upstream.

    Thanks,
      Mark