ABI incompatibilities with MinGW GCC 4.7.0

ABI incompatibilities with MinGW GCC 4.7.0

[Eli Zaretskii]

See http://sourceforge.net/mailarchive/message.php?msg_id=29376223 and
the following discussion (which is still unfolding) for the details.

The upshot of this, AFAIU, is that the MinGW GCC 4.7.0 should NOT be
used for building Emacs on Windows with any of the optional libraries,
such as image libraries, GnuTLS, libxml2, etc., because _all_ of those
libraries were compiled with versions of GCC before 4.7.0, and are now
ABI incompatible with code compiled by 4.7.0.

(Actually, you cannot safely use the MinGW GCC 4.7.0 for building
Emacs on Windows at all for now, because (a) there's no MinGW runtime
available that is compatible with the new ABI, and (b) you must link
with libxpm.dll, which was compiled by an older GCC.)

I sincerely hope that this incompatibility will either be reverted or
turns out as an insignificant one, because otherwise we will be facing
a lot of subtle and hard to reproduce bugs in the Windows build.  The
only alternative, if this issue is not resolved, is to not upgrade to
GCC 4.7.

Re: ABI incompatibilities with MinGW GCC 4.7.0

[joakim]

Eli Zaretskii <eliz@gnu.org> writes:

> See http://sourceforge.net/mailarchive/message.php?msg_id=29376223 and
> the following discussion (which is still unfolding) for the details.
>
> The upshot of this, AFAIU, is that the MinGW GCC 4.7.0 should NOT be
> used for building Emacs on Windows with any of the optional libraries,
> such as image libraries, GnuTLS, libxml2, etc., because _all_ of those
> libraries were compiled with versions of GCC before 4.7.0, and are now
> ABI incompatible with code compiled by 4.7.0.
>
> (Actually, you cannot safely use the MinGW GCC 4.7.0 for building
> Emacs on Windows at all for now, because (a) there's no MinGW runtime
> available that is compatible with the new ABI, and (b) you must link
> with libxpm.dll, which was compiled by an older GCC.)
>
> I sincerely hope that this incompatibility will either be reverted or
> turns out as an insignificant one, because otherwise we will be facing
> a lot of subtle and hard to reproduce bugs in the Windows build.  The
> only alternative, if this issue is not resolved, is to not upgrade to
> GCC 4.7.

Does this also affect the mingw cross-compiler, on Fedora for example?
AFAICS the Fedora mingw libraries are all compiled with the correct
versions, but I'm not sure about the ABI incompatibility you describe.

-- 
Joakim Verona

Re: ABI incompatibilities with MinGW GCC 4.7.0

[Eli Zaretskii]

> From: joakim@verona.se
> Cc: emacs-devel@gnu.org
> Date: Fri, 08 Jun 2012 11:42:41 +0200
> 
> Does this also affect the mingw cross-compiler, on Fedora for example?

I have no idea, sorry.  I don't know who or how produces the cross
tools.  Better ask on the relevant forums, where the maintainers of
those packages dwell.

Re: ABI incompatibilities with MinGW GCC 4.7.0

[Jason Rumney]

Eli Zaretskii <eliz@gnu.org> writes:

> The upshot of this, AFAIU, is that the MinGW GCC 4.7.0 should NOT be
> used for building Emacs on Windows with any of the optional libraries,
> such as image libraries, GnuTLS, libxml2, etc., because _all_ of those
> libraries were compiled with versions of GCC before 4.7.0, and are now
> ABI incompatible with code compiled by 4.7.0.

Has anyone actually tried?

As far as I can see, it should only affect interfaces that use bitfields, and
C++.  All the libraries that Emacs uses are Free software, so can be
rebuilt with the same compiler if neccesary.

Re: ABI incompatibilities with MinGW GCC 4.7.0

[Eli Zaretskii]

> From: Jason Rumney <jasonr@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Sat, 09 Jun 2012 11:10:20 +0800
> 
> Has anyone actually tried?

I don't know.  I didn't.  I still hope that the MinGW GCC maintainer
will respond to the thread with the authoritative information about
the ABI changes.

> As far as I can see, it should only affect interfaces that use bitfields, and
> C++.

That's the only specific ABI changes mentioned in the announcement,
yes.  But the announcement said "in particular", which might mean
there are other incompatible ABI changes; my question about that did
not get any responses except "don't know".  In addition, the MinGW GCC
maintainer said in a GCC forum that there are other ABI
incompatibilities beyond the bitfields, which he will publish "later";
AFAICS, that didn't happen yet.

> All the libraries that Emacs uses are Free software, so can be
> rebuilt with the same compiler if neccesary.

Of course.  But if the incompatibilities are real, and cannot be
removed by using some compiler switches, it would mean that we will
need 2 incompatible versions of each library, and will have to cope with
bugs caused by users who install the wrong version.

Re: ABI incompatibilities with MinGW GCC 4.7.0

[Achim Gratz]

Eli Zaretskii writes:
> See http://sourceforge.net/mailarchive/message.php?msg_id=29376223 and
> the following discussion (which is still unfolding) for the details.

The first of these is a red herring.  You always needed to know whether
all libraries you link to were produced with '-mms-bitfields' or
'-mno-ms-bitfields' anyway ever since that option was introduced.  So
the default changes with 4.7.0, but you can just as easily chose the
former default.

The second change only affects C++ programs AFAICS and it makes
(__thiscall) the default which has been introduced with 4.6.2 (I think).
I can only assume that you can still override it with (__stdcall), but
that means changes to the source.  There is a disturbing lack of
consideration for backwards compatibility and I would have expected that
the ABI version is bumped (so one could specify the old default with,
say, -mabi=...).  If there's really no way to get the old default back
without qualifying all functions in the source, I'd consider that a
defect that needs to be fixed for 4.7.1.

> (Actually, you cannot safely use the MinGW GCC 4.7.0 for building
> Emacs on Windows at all for now, because (a) there's no MinGW runtime
> available that is compatible with the new ABI, and (b) you must link
> with libxpm.dll, which was compiled by an older GCC.)

I still think that simply adding '-mno-ms-bitfields' to the build is all
you need for Emacs


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: ABI incompatibilities with MinGW GCC 4.7.0

[Eli Zaretskii]

> From: Achim Gratz <Stromeko@nexgo.de>
> Date: Sat, 09 Jun 2012 14:06:48 +0200
> 
> Eli Zaretskii writes:
> > See http://sourceforge.net/mailarchive/message.php?msg_id=29376223 and
> > the following discussion (which is still unfolding) for the details.
> 
> The first of these is a red herring.

I very much hope you are right, but I'm not sure where your optimism
comes from.

> You always needed to know whether all libraries you link to were
> produced with '-mms-bitfields' or '-mno-ms-bitfields' anyway ever
> since that option was introduced.  So the default changes with
> 4.7.0, but you can just as easily chose the former default.

When one builds a library, one usually follows the build instructions
(unless they are the maintainers, which is rarely the case for a
Windows build).  These instructions mostly boil down to

  ./configure && make && make check && make install

or something similar.  When a library is built this way, the GCC
options used are the default ones, plus whatever the package-specific
Makefile's set.  You can guess what would be the probability of having
'-mms-bitfields' among those switches; I can _tell_ you that I never
ever saw them when I built libraries (some of which are recommended
for optional Emacs features).

So in practice, I submit that most, if not all, of the precompiled
libraries out there were build with the equivalent of
'-mno-ms-bitfields', and therefore the new default is actually, not
just theoretically, different.

But what bothers me most is that no one said this is the only change
that affects C programs.

> > (Actually, you cannot safely use the MinGW GCC 4.7.0 for building
> > Emacs on Windows at all for now, because (a) there's no MinGW runtime
> > available that is compatible with the new ABI, and (b) you must link
> > with libxpm.dll, which was compiled by an older GCC.)
> 
> I still think that simply adding '-mno-ms-bitfields' to the build is all
> you need for Emacs

If we know the libraries out there are not built with GCC 4.7.x, then
this is indeed the way to go.  But what about people who like to build
all their libraries themselves? if they use GCC 4.7 to build their
libraries, and don't make a point of using '-mno-ms-bitfields' when
they do, we cannot let them build Emacs with '-mno-ms-bitfields', can
we?

And then there's the issue of other ABI changes, if there are any.
That is the really disturbing part, because the bitfields issue rarely
if at all affects real-life code.

> There is a disturbing lack of consideration for backwards
> compatibility and I would have expected that the ABI version is
> bumped (so one could specify the old default with, say, -mabi=...).
> If there's really no way to get the old default back without
> qualifying all functions in the source, I'd consider that a defect
> that needs to be fixed for 4.7.1.

I agree.  But this list is not concerned with maintaining GCC, it uses
GCC to build Emacs.  I posted the info here to try to proactively
avoid subtle problems this ABI change could produce, if people haste
to upgrade.

Re: ABI incompatibilities with MinGW GCC 4.7.0

[Jason Rumney]

Eli Zaretskii <eliz@gnu.org> writes:

>> I still think that simply adding '-mno-ms-bitfields' to the build is all
>> you need for Emacs
>
> If we know the libraries out there are not built with GCC 4.7.x, then
> this is indeed the way to go.  But what about people who like to build
> all their libraries themselves? if they use GCC 4.7 to build their
> libraries, and don't make a point of using '-mno-ms-bitfields' when
> they do, we cannot let them build Emacs with '-mno-ms-bitfields', can
> we?

The GTK binaries available for Windows have been built with
-mms-bitfields for some years now, and the image libraries contained
within them have worked without problem with Emacs for all that time. So
I think the choice of whether to build with or without that flag is a
non-issue for Emacs.

> And then there's the issue of other ABI changes, if there are any.
> That is the really disturbing part, because the bitfields issue rarely
> if at all affects real-life code.

It is somewhat disturbing that the MinGW-GCC maintainers themselves are
unsure about other ABI changes, but I doubt that any of them will affect
pure C code except maybe in more rare corner cases like the bitfield
issue.  If we were using C++, it might be another story, but for
interfaces to third-party libraries that are distributed separately in
binary form, C++ has always been a bad choice.

Re: ABI incompatibilities with MinGW GCC 4.7.0

[Eli Zaretskii]

> From: Jason Rumney <jasonr@gnu.org>
> Cc: Achim Gratz <Stromeko@nexgo.de>,  emacs-devel@gnu.org
> Date: Sat, 09 Jun 2012 22:44:01 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> I still think that simply adding '-mno-ms-bitfields' to the build is all
> >> you need for Emacs
> >
> > If we know the libraries out there are not built with GCC 4.7.x, then
> > this is indeed the way to go.  But what about people who like to build
> > all their libraries themselves? if they use GCC 4.7 to build their
> > libraries, and don't make a point of using '-mno-ms-bitfields' when
> > they do, we cannot let them build Emacs with '-mno-ms-bitfields', can
> > we?
> 
> The GTK binaries available for Windows have been built with
> -mms-bitfields for some years now, and the image libraries contained
> within them have worked without problem with Emacs for all that time. So
> I think the choice of whether to build with or without that flag is a
> non-issue for Emacs.

What is true for some libraries might be false for others.  It all
depends whether a library triggers the code generation where this
switch makes the difference.

> > And then there's the issue of other ABI changes, if there are any.
> > That is the really disturbing part, because the bitfields issue rarely
> > if at all affects real-life code.
> 
> It is somewhat disturbing that the MinGW-GCC maintainers themselves are
> unsure about other ABI changes, but I doubt that any of them will affect
> pure C code except maybe in more rare corner cases like the bitfield
> issue.

A I said, I hope you are right.  But until we _know_ for sure, I think
it's prudent to tell people to stay away of the new version, which was
the sole purpose of my OP.  If you are saying that people can safely
disregard this issue, then I very much disagree.

Re: ABI incompatibilities with MinGW GCC 4.7.0

[Achim Gratz]

Eli Zaretskii writes:
> I very much hope you are right, but I'm not sure where your optimism
> comes from.

Simple.  I know the default has been '-mno-ms-bitfields' and there are
libraries that you can optionally compile with '-mms-bitfields' or that
are even built that way by default (because they need to be linked to MS
stuff more often than not).  This particular trap is not something that
has been created with gcc-4.7, it has existed for years.

> So in practice, I submit that most, if not all, of the precompiled
> libraries out there were build with the equivalent of
> '-mno-ms-bitfields', and therefore the new default is actually, not
> just theoretically, different.

I'll grant you that, however you can go back and forth as often as you
wish until all your libraries live at only one side of the chasm.

> But what bothers me most is that no one said this is the only change
> that affects C programs.

It's said to be the only _binary_ incompatibility.  Without further
information to the contrary, I'll believe the gcc folks.

> If we know the libraries out there are not built with GCC 4.7.x, then
> this is indeed the way to go.  But what about people who like to build
> all their libraries themselves? if they use GCC 4.7 to build their
> libraries, and don't make a point of using '-mno-ms-bitfields' when
> they do, we cannot let them build Emacs with '-mno-ms-bitfields', can
> we?

As I said, you already have that problem today.  You must make sure that
_all_ your libraries are built one way or the other, but never mixed —
or that the compiler generated bitfield layout is never exposed in a
public interface, which is possible, but even harder to ascertain.

If there's a bug here it is that there is a default for this switch at
all, which only helps to forget that you have to make a choice and stick
to it.

> I agree.  But this list is not concerned with maintaining GCC, it uses
> GCC to build Emacs.  I posted the info here to try to proactively
> avoid subtle problems this ABI change could produce, if people haste
> to upgrade.

Appreciated.  I don't know if it's possible to look at a library and
decide which way it's been compiled, but if there is, configure should
be amended to check it for MinGW compiles and set the switch
appropriately.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptation for Waldorf Blofeld V1.15B11:
http://Synth.Stromeko.net/Downloads.html#WaldorfSDada

Re: ABI incompatibilities with MinGW GCC 4.7.0

[Eli Zaretskii]

> From: Achim Gratz <Stromeko@nexgo.de>
> Date: Sat, 09 Jun 2012 18:19:27 +0200
> 
> > But what bothers me most is that no one said this is the only change
> > that affects C programs.
> 
> It's said to be the only _binary_ incompatibility.

Where did you see that?  I don't think I have, but maybe I missed
something.

Thanks.

Re: ABI incompatibilities with MinGW GCC 4.7.0

[Achim Gratz]

Eli Zaretskii writes:
>> It's said to be the only _binary_ incompatibility.
>
> Where did you see that?  I don't think I have, but maybe I missed
> something.

I may be reading too much into the release blurb, but it says "binary
incompatibility notice" and then lists only the two items discussed, of
which the second cannot apply to C programs (so it must be the C++ ABI
change) — that leaves the first item for the C ABI change only.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Terratec KOMPLEXER:
http://Synth.Stromeko.net/Downloads.html#KomplexerWaves