Now that SCM type is a union...

Now that SCM type is a union...

[rixed]

Between stable-2.0 and master a patch changed the C representation
of the SCM type so that it is now a union.

This code :

static SCM foo = SCM_UNSPECIFIED;

now expands to something similar to :

static SCM foo = (SCM) { ... };

This form (casting a struct or union initializer while initializing a
global identifier) is for some reason invalid when gcc is called with
"-std=c99" (I was about to say : "was invalid in c99", but who really
knowns?) nor "-std=gnu99" (although it works when std is set to c89 or
gnu89).

I tried to get rid of the cast to (SCM) in tags.h but the compilation
then fails since some code relies on the cast to SCM.

So, lets suppose I have an app written in c99 that I want to extend with
guile, how could I compile it ?

Re: Now that SCM type is a union...

[Ken Raeburn]

On Aug 12, 2011, at 08:44, rixed@happyleptic.org wrote:
> Between stable-2.0 and master a patch changed the C representation
> of the SCM type so that it is now a union.
> 
> This code :
> 
> static SCM foo = SCM_UNSPECIFIED;
> 
> now expands to something similar to :
> 
> static SCM foo = (SCM) { ... };
> 
> This form (casting a struct or union initializer while initializing a
> global identifier) is for some reason invalid when gcc is called with
> "-std=c99" (I was about to say : "was invalid in c99", but who really
> knowns?) nor "-std=gnu99" (although it works when std is set to c89 or
> gnu89).

("Is for some reason invalid" is not very descriptive; please include specific compiler messages.)

That syntax -- the parenthesized type followed by a list of initializers in braces -- is called a "compound literal" (technically not a cast expression) and was added in C99.  The value of a compound literal is an anonymous object with a value given by the initializer elements.  (It's an lvalue, and can even have its address taken.)  That means it's not a "constant expression" and cannot be used for static initialization, under C99 rules.

The GCC documentation says that a GCC extension permits such initialization.  From that I'd kind of expect it to work in gnu89 or gnu99 modes, and maybe not c89, but apparently that's not how they did it...  It wouldn't surprise me if it has to do with how either compound literals or constant expressions are handled in the compiler making it hard to support them as both addressable anonymous objects and constant initializers, and for C99 conformance they'd have to go with the former.  I guess the GCC documentation needs fixing.

> I tried to get rid of the cast to (SCM) in tags.h but the compilation
> then fails since some code relies on the cast to SCM.

Yes, to be used as a value in an expression, rather than an initializer for a variable, it needs to use the compound-literal syntax with the parenthesized type name.

> So, lets suppose I have an app written in c99 that I want to extend with
> guile, how could I compile it ?

Have an initialization function which stuffs SCM_UNSPECIFIED into a bunch of uninitialized SCM variables before making them accessible from other code?  Yeah, it's kind of unfortunate.  Maybe we need a separate set of macros for static initialization using SCM_UNDEFINED, SCM_BOOL_F, and friends?

It looks like Guile is compiled in the default (gnu89?) mode, not C99.  It has a few places where static variables are initialized using SCM_BOOL_F, which will have the same problem.

Ken

Re: Now that SCM type is a union...

[rixed]

-[ Sat, Aug 13, 2011 at 01:26:03AM -0400, Ken Raeburn ]----
> ("Is for some reason invalid" is not very descriptive; please include specific compiler messages.)

Sorry ; the specific error message was, you guesssed it right:
"initializer element is not constant".

> That syntax -- the parenthesized type followed by a list of initializers in braces -- is called a "compound literal" (technically not a cast expression) and was added in C99.  The value of a compound literal is an anonymous object with a value given by the initializer elements.  (It's an lvalue, and can even have its address taken.)  That means it's not a "constant expression" and cannot be used for static initialization, under C99 rules.

Yes thank you for clarifying this. In this context of initializing a
global variable my brain failed to recognize a compound literal.
It all make sense now, both why it's used and why gcc is complaining.
Now it seams indeed wrong to use an anonymous lvalue as an initializer.

> The GCC documentation says that a GCC extension permits such initialization.

Even if it was the case, would it be a good idea for header files that
are supposed to be included in user programs to rely on a gcc extension ?

> From that I'd kind of expect it to work in gnu89 or gnu99 modes, and maybe not c89, but apparently that's not how they did it...

It actualy works with gnu89 or c89, but neither gnu99 nor c99 :

---[ foobar.c ]---

struct foo { int a; };
struct foo bar = (struct foo){ 1 };

---[ /foobar.c ]---

make foobar.o CFLAGS="-std=gnu99"
foobar.c:2:26: error: initializer element is not constant

same goes for c99 ; while:

make foobar.o CFLAGS="-std=c89"

works.

> Have an initialization function which stuffs SCM_UNSPECIFIED into a bunch of uninitialized SCM variables before making them accessible from other code?  Yeah, it's kind of unfortunate.

Especially unfortunate if one do not want to rely on gcc specific
constructor to initialize it automagically ; and it makes programs
that were working with former versions of guile fails to compile which
is not so nice.

> Maybe we need a separate set of macros for static initialization using SCM_UNDEFINED, SCM_BOOL_F, and friends?

I'd like it. Or maybe instead of checking for a PREHISTORIC_COMPILER
the header files may check for actual compiler settings (ie. if C99
conformance is required then don't use the union) ; if the underlying
encoding of the alternative is the same, of course.

> It looks like Guile is compiled in the default (gnu89?) mode, not C99.  It has a few places where static variables are initialized using SCM_BOOL_F, which will have the same problem.

Yes, whenever c99 becomes the default (it's only 12 years old now).

Re: Now that SCM type is a union...

[Andy Wingo]

On Sat 13 Aug 2011 07:26, Ken Raeburn <raeburn@raeburn.org> writes:

> That syntax -- the parenthesized type followed by a list of initializers
> in braces -- is called a "compound literal" (technically not a cast
> expression) and was added in C99.  The value of a compound literal is an
> anonymous object with a value given by the initializer elements.  (It's
> an lvalue, and can even have its address taken.)  That means it's not a
> "constant expression" and cannot be used for static initialization,
> under C99 rules.

I only have a draft copy of C99, from 7 September 2007, but it does
permit constant expressions to appear outside function bodies.  How
could that happen except for in initializers?  I do see the language in
the GCC docs though; it's confusing.  I was under the impression that
they would be constant expressions, but perhaps I was mistaken.

I will take a look at this issue soonish, but your help (and Cedric's)
in debugging it is most appreciated :)  I would love to keep the union
as the "normal" SCM definition, but that might not be possible.

Regards,

Andy
-- 
http://wingolog.org/

Re: Now that SCM type is a union...

[Ken Raeburn]

On Aug 13, 2011, at 08:23, Andy Wingo wrote:
> I only have a draft copy of C99, from 7 September 2007, but it does
> permit constant expressions to appear outside function bodies.  How
> could that happen except for in initializers?  I do see the language in
> the GCC docs though; it's confusing.  I was under the impression that
> they would be constant expressions, but perhaps I was mistaken.

It is confusing.  There's an example in the final spec which I think indicates what's happening:

  int *p = (int []){2, 4}; // definition at file scope, outside any fn

The compound literal produces an anonymous object (with static storage duration, in this case) of type "int[2]"; in this context, its initialization elements have to be constants.  Through the standard conversion of array to pointer-to-element, the value of this expression becomes an "int *" which is used to initialize p (allowed because the object has static storage duration and thus its address is a constant).

When array/pointer conversions aren't involved, you might assume you could just initialize a variable statically, but since the compound literal creates an object, an lvalue, it's more like saying:

  static int anon = 3; // "anonymous" obj with constant initializer
  int x = anon; // not allowed in C99

You could make "anon" const, but it wouldn't change anything, under C99 rules.  Note, too, that the unnamed object created doesn't need to be "const" -- changing p[0] above is perfectly valid.  Though if you use const in the type, then the compiler is permitted to combine identical values and store only one copy.

Also:
 * We should expect some Guile applications to be in C++.  What versions of the C++ spec should Guile support?
 * Shouldn't there be testing to catch this?  (C89 mode, C99 mode, different C++ specs, enabling various compiler warnings -- for whatever compiler is in use -- and make them fatal, any interesting ways one might want to use libguile in an application that might stress compatibility issues.)  I mean automated testing, not just Cedric. :-)

> I will take a look at this issue soonish, but your help (and Cedric's)
> in debugging it is most appreciated :)  I would love to keep the union
> as the "normal" SCM definition, but that might not be possible.

Regardless of the validity, there are popular compilers out there now which do not support this, when used in modes people may need or want to use.  The installed headers need to adhere to higher standards in terms of portability problems and warnings than the library source, where we can dictate some of the compiler options.

Ken

Re: Now that SCM type is a union...

[Andy Wingo]

Hi Ken,

On Sun 14 Aug 2011 00:00, Ken Raeburn <raeburn@raeburn.org> writes:

>  * We should expect some Guile applications to be in C++.  What
>  versions of the C++ spec should Guile support?

Dunno.  What do other languages have to do with Guile's C interface?
Specifically what does C++ have to do with this?  (Serious question.)

>  * Shouldn't there be testing to catch this?  (C89 mode, C99 mode,
>  different C++ specs, enabling various compiler warnings -- for
>  whatever compiler is in use -- and make them fatal, any interesting
>  ways one might want to use libguile in an application that might
>  stress compatibility issues.)  I mean automated testing, not just
>  Cedric. :-)

Perhaps :)  Interested volunteers are welcome to set this up :)

>> I will take a look at this issue soonish, but your help (and Cedric's)
>> in debugging it is most appreciated :)  I would love to keep the union
>> as the "normal" SCM definition, but that might not be possible.
>
> Regardless of the validity, there are popular compilers out there now
> which do not support this, when used in modes people may need or want
> to use.  The installed headers need to adhere to higher standards in
> terms of portability problems and warnings than the library source,
> where we can dictate some of the compiler options.

Agreed.  And AFAIK there are some ABI differences for returning unions
versus returning uintptr_t, so unfortunately it can't be a question of
ifdefs in Guile's C interface, it seems.  I'll work on this, but if you
have any suggestions as to the proper fix, they are most welcome.

Andy
-- 
http://wingolog.org/

Re: Now that SCM type is a union...

[Ken Raeburn]

On Aug 14, 2011, at 16:10, Andy Wingo wrote:
>> * We should expect some Guile applications to be in C++.  What
>> versions of the C++ spec should Guile support?
> 
> Dunno.  What do other languages have to do with Guile's C interface?
> Specifically what does C++ have to do with this?  (Serious question.)

Most languages, nothing; they can't directly use the C headers, even if there's another way to tell, for example, the GNU Ada compiler what the Guile ABI looks like.  C++ is another matter -- at least, the languages are sufficiently compatible that people will try.  And we've already got the 'extern "C"' bits in libguile.h to help support it.

The C++ compiler will treat the calls to extern "C" functions specially, using different name-mangling techniques (and I think different function-calling interfaces are even possible, but I'm not sure).  But the header still needs to be parsed as C++, and that's not always a strict superset of the latest and greatest C spec.


> 
>> * Shouldn't there be testing to catch this?  (C89 mode, C99 mode,
>> different C++ specs, enabling various compiler warnings -- for
>> whatever compiler is in use -- and make them fatal, any interesting
>> ways one might want to use libguile in an application that might
>> stress compatibility issues.)  I mean automated testing, not just
>> Cedric. :-)
> 
> Perhaps :)  Interested volunteers are welcome to set this up :)

I've done it before for other projects... you just need to add to the test suite a few files to be compiled with certain compilers and compiler options -- perhaps repeatedly with different options -- and make sure they compile.  If you want to get really fancy, link and run some trivial programs, but generally compiling with lots of warning options is good enough, at least for C.  For GCC, for example, I'd start with "gcc -Wall -Wextra -Werror -pedantic" and "g++ -Wall -Wextra -Werror -pedantic", and add more interesting options from there, then multiply that by the various "-std" options available; the Solaris compiler has a different syntax for warning options, but it too can be told to enable various warnings and make them errors.

I'll see about setting up something simple...

> 
>>> I will take a look at this issue soonish, but your help (and Cedric's)
>>> in debugging it is most appreciated :)  I would love to keep the union
>>> as the "normal" SCM definition, but that might not be possible.
>> 
>> Regardless of the validity, there are popular compilers out there now
>> which do not support this, when used in modes people may need or want
>> to use.  The installed headers need to adhere to higher standards in
>> terms of portability problems and warnings than the library source,
>> where we can dictate some of the compiler options.
> 
> Agreed.  And AFAIK there are some ABI differences for returning unions
> versus returning uintptr_t, so unfortunately it can't be a question of
> ifdefs in Guile's C interface, it seems.  I'll work on this, but if you
> have any suggestions as to the proper fix, they are most welcome.

I don't see anything really clean... separate macros (same value, different syntax) for static initialization vs use in run-time evaluated expressions might be one way to keep the union type.  For non-GNU C89, the latter could expand to the return value of a function call, which sucks for performance but keeps the API and ABI intact.  For GCC in pedantic C89 mode, the __extension__ keyword will tell it to shut up about intentionally-used GNU extensions.  There are other places where the union value won't be useable -- for example, case labels in switch statements need integer values.

If you want one macro for static initialization and for normal expressions, I think it's got to be an arithmetic or pointer type....

Re: Now that SCM type is a union...

[Ken Raeburn]

On Aug 14, 2011, at 22:04, Ken Raeburn wrote:
>>> * Shouldn't there be testing to catch this?  (C89 mode, C99 mode,
>>> different C++ specs, enabling various compiler warnings -- for
>>> whatever compiler is in use -- and make them fatal, any interesting
>>> ways one might want to use libguile in an application that might
>>> stress compatibility issues.)  I mean automated testing, not just
>>> Cedric. :-)
>> 
>> Perhaps :)  Interested volunteers are welcome to set this up :)
> 
> I've done it before for other projects... you just need to add to the test suite a few files to be compiled with certain compilers and compiler options -- perhaps repeatedly with different options -- and make sure they compile.  If you want to get really fancy, link and run some trivial programs, but generally compiling with lots of warning options is good enough, at least for C.  For GCC, for example, I'd start with "gcc -Wall -Wextra -Werror -pedantic" and "g++ -Wall -Wextra -Werror -pedantic", and add more interesting options from there, then multiply that by the various "-std" options available; the Solaris compiler has a different syntax for warning options, but it too can be told to enable various warnings and make them errors.
> 
> I'll see about setting up something simple...

Oh, hm, automake... on second thought, this might take a while....

Ken

Re: Now that SCM type is a union...

[Andy Wingo]

On Fri 12 Aug 2011 05:44, rixed@happyleptic.org writes:

> Between stable-2.0 and master a patch changed the C representation
> of the SCM type so that it is now a union.

As a result of this constant-expression issue, I have reverted those
patches.  Now `master' uses the same SCM representation as 2.0.

Regards,

Andy
-- 
http://wingolog.org/