Re: MinGW build fixes

Re: MinGW build fixes

[carlo.bramix]

Hello!

>> I don't understand why this fixes anything, since the `_ptr' variables
>> are declared as `SCM_API' just like the non-`_ptr' variables.
>
> Indeed.  My guess is that it's because the DLL import/export mechanism
> works for atomic data - i.e. chars, ints, pointers etc - but not for
> structs.

False.
A DLL can export functions, variables, structures, unions and classes.
After few fixes that I'm preparing to share, guile already compiles and it works perfectly in cygwin, which uses DLL too: although  it virtualizes a lot, cygwin cannot do miracles, it's always windows based and if you add -mno-cygwin it becames identical to mingw.
What we could suspect about it is the implementation of -export-dynamic under your mingw: which version are you using?
I'm using the latest maintenance releases of gcc 3.4.5 and binutils 2.18.50 without troubles.
Actually the troubles with mingw are elsewhere... may I ask how you made it working?
Unless it has been committed recently, guile cannot be even compiled well because it is missing the support for CreateFileMapping(), plus other fixes around the code!
Are you using another posix compatibility layer? But I could not see any change in your patch for configure script...
Anyways, I would not corrupt guile because an isolated compiler bug (my opinion).

Sincerely,

Carlo Bramini.

---------- Initial Header -----------

From      : guile-devel-bounces+carlo.bramix=libero.it@gnu.org
To          : ludo@gnu.org
Cc          : guile-devel@gnu.org
Date      : Sat, 27 Jun 2009 10:29:54 +0100
Subject : Re: MinGW build fixes

> ludo@gnu.org (Ludovic Courtès) writes:
>
> >>  SCM_API struct scm_t_cell_type_statistics scm_i_master_freelist;
> >> +SCM_API struct scm_t_cell_type_statistics *scm_i_master_freelist_ptr;
> >>  SCM_API struct scm_t_cell_type_statistics scm_i_master_freelist2;
> >> +SCM_API struct scm_t_cell_type_statistics *scm_i_master_freelist2_ptr;
> >
> > I don't understand why this fixes anything, since the `_ptr' variables
> > are declared as `SCM_API' just like the non-`_ptr' variables.
>
> Indeed.  My guess is that it's because the DLL import/export mechanism
> works for atomic data - i.e. chars, ints, pointers etc - but not for
> structs.  Or at least, not in the same way - it could be that some
> extra magic incantation is needed somewhere.
>
> (For example, there's no problem with scm_cells_allocated, which is
> also referenced by the inline scm_cell code.)
>
> I haven't managed to find a reference for this, though.  Also this
> restriction could be one imposed by MSVC/Windows, or it could be a
> restriction in MinGW's emulation of that.
>
> I guess the thing to do would be to email the MinGW people, so I'll do
> that.
>
> > Also, it adds an indirection, which may impact performance.
>
> Yes.  So I guess I should make these diffs #ifdef __MINGW32__.
>
> > Other than that, I think this is good news!
>
> Me too.  On the one hand Windows isn't a free platform, so it can't be
> our prime concern.  On the other, I think it's good for us to grow the
> possible audience for Guile as much as possible.
>
> (Guile is already in Cygwin, of course, but it's an older version and
> AFAIK the build was not straightforward.  So these fixes are about
> making Windows builds work out of the upstream box.)
>
> Regards,
>         Neil
>
>
>

Re: MinGW build fixes

[Neil Jerram]

"carlo.bramix" <carlo.bramix@libero.it> writes:

> Hello!

Hi Carlo!

>>> I don't understand why this fixes anything, since the `_ptr'
>>> variables are declared as `SCM_API' just like the non-`_ptr'
>>> variables.
>>  Indeed.  My guess is that it's because the DLL import/export
>> mechanism works for atomic data - i.e. chars, ints, pointers etc -
>> but not for structs.
>
> False.  A DLL can export functions, variables, structures, unions and
> classes.  After few fixes that I'm preparing to share, guile already
> compiles and it works perfectly in cygwin, which uses DLL too:

Great!  I know there are already several emails from you that I
haven't responded to yet; please be assured that I do plan to.

> although it virtualizes a lot, cygwin cannot do miracles, it's always
> windows based and if you add -mno-cygwin it becames identical to
> mingw.

I think you mean when compiling in Cygwin, and that the -mno-cygwin is
a GCC flag; is that right?

>  What we could suspect about it is the implementation of
> -export-dynamic under your mingw: which version are you using?

I'm using the MinGW cross compiler on GNU/Linux (Debian lenny):

neil@arudy:~$ i586-mingw32msvc-gcc --version
i586-mingw32msvc-gcc (GCC) 4.2.1-sjlj (mingw32-2)

The configure invocation is:

    CPPFLAGS=-I/usr/local/i586-mingw32msvc/include \
    LDFLAGS=-L/usr/local/i586-mingw32msvc/lib \
    GUILE_FOR_BUILD=/usr/local/bin/guile \
    ./configure --prefix=/usr/local/i586-mingw32msvc --host=i586-mingw32msvc

You can see the logs of my builds, for the Guile master and
branch_release-1-8 branches, at
http://www.ossau.uklinux.net/guile/snapshots, or at
http://autobuild.josefsson.org/guile/.

>  I'm
> using the latest maintenance releases of gcc 3.4.5 and binutils
> 2.18.50 without troubles.  Actually the troubles with mingw are
> elsewhere... may I ask how you made it working?  Unless it has been
> committed recently, guile cannot be even compiled well because it is
> missing the support for CreateFileMapping(), plus other fixes around
> the code!

What version of Guile have you been trying to build?  My attempt to
build the master branch (i.e. similar to the 1.9.0 pre-release)
currently fails at

./guile-snarf -o objcodes.x objcodes.c -DHAVE_CONFIG_H -DBUILDING_LIBGUILE=1 -I.. -I.. -I../lib -I../lib -I/usr/local/i586-mingw32msvc/include -g -O2
objcodes.c:26:22: error: sys/mman.h: No such file or directory

which I guess might be the same problem as "missing the support for
CreateFileMapping".

The branch_release-1-8 branch (i.e. similar to 1.8.x releases) seems
fine. though.

>  Are you using another posix compatibility layer? But I
> could not see any change in your patch for configure script...

I hope my answers above cover this; let me know if not.

> Anyways, I would not corrupt guile because an isolated compiler bug
> (my opinion).

Worst case is that I'll put the MinGW changes in #ifdef __MINGW32__.
That's still corruption in a sense, but only a very little.

Regards,
        Neil

MinGW build fixes

[Neil Jerram]

[-- Attachment #1: Type: text/plain, Size: 177 bytes --]

The patch below allows a complete MinGW build of branch_release-1-8; I
haven't tried actually running the built .exe and .dlls yet.

Comments appreciated as usual!

     Neil



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-MinGW-build-fixes.patch --]
[-- Type: text/x-diff, Size: 7419 bytes --]

From ed7189a28292cd9ba8fcb5d18d2dcd817061935a Mon Sep 17 00:00:00 2001
From: Neil Jerram <neil@ossau.uklinux.net>
Date: Sat, 27 Jun 2009 00:33:06 +0100
Subject: [PATCH] MinGW build fixes

- Need to use GUILE_FOR_BUILD for autogeneration steps during the
  build.

- MinGW can't export/import struct variables from/to a DLL, so wrap
  the ones needed by inline.h in exported/imported pointer variables.

- The dummy scm_init_* and scm_c_init_* functions in the empty
  libraries for SRFIs 4, 13 and 14 clash with declarations with the
  same names in libguile.h that are marked (in the MinGW build) with
  __declspec (dllimport).  These libraries are empty because their
  content was moved into libguile some time ago, and I think these
  functions are needed only so that the libraries appear to contain
  something.  They used also to be needed for the (load-extension ...)
  forms in the corresponding .scm modules, but those forms were not
  needed anymore, and so were removed, when everything moved into
  libguile.

* doc/ref/Makefile.am (autoconf-macros.texi): Use GUILE_FOR_BUILD to
  generate this.

* libguile/gc-freelist.c (scm_i_master_freelist_ptr,
  scm_i_master_freelist2_ptr): New API variables.

* libguile/gc.h (scm_i_freelist_ptr, scm_i_freelist2_ptr,
  scm_i_master_freelist_ptr, scm_i_master_freelist2_ptr): New global
  variable declarations.

* libguile/inline.h (scm_cell): Use scm_i_freelist_ptr and
  scm_i_master_freelist_ptr instead of scm_i_freelist and
  scm_i_master_freelist.
  (scm_double_cell): Use scm_i_freelist2_ptr and
  scm_i_master_freelist2_ptr instead of scm_i_freelist2 and
  scm_i_master_freelist2.

* libguile/threads.c (scm_i_freelist_ptr, scm_i_freelist2_ptr): New
  global variables.

* srfi/srfi-13.c (scm_init_srfi_13_no_clash_with_libguile): Renamed
  from scm_init_srfi_13.
  (scm_init_srfi_13_14_no_clash_with_libguile): Same.

* srfi/srfi-14.c (scm_init_srfi_14_no_clash_with_libguile): Same.
  (scm_c_init_srfi_14_no_clash_with_libguile): Same.

* srfi/srfi-4.c (scm_init_srfi_4_no_clash_with_libguile): Same.

* srfi/srfi-60.c (SCM_MIN): Define here instead of including
  private-gc.h.
---
 doc/ref/Makefile.am    |    2 +-
 libguile/gc-freelist.c |    2 ++
 libguile/gc.h          |    4 ++++
 libguile/inline.h      |    8 ++++----
 libguile/threads.c     |    2 ++
 srfi/srfi-13.c         |    7 +++++--
 srfi/srfi-14.c         |    7 +++++--
 srfi/srfi-4.c          |    4 +++-
 srfi/srfi-60.c         |    2 +-
 9 files changed, 27 insertions(+), 11 deletions(-)

diff --git a/doc/ref/Makefile.am b/doc/ref/Makefile.am
index b60f2ff..46958b6 100644
--- a/doc/ref/Makefile.am
+++ b/doc/ref/Makefile.am
@@ -86,7 +86,7 @@ include $(top_srcdir)/am/pre-inst-guile
 
 autoconf.texi: autoconf-macros.texi
 autoconf-macros.texi: $(top_srcdir)/guile-config/guile.m4
-	$(preinstguiletool)/snarf-guile-m4-docs $(top_srcdir)/guile-config/guile.m4 \
+	GUILE="$(GUILE_FOR_BUILD)" $(top_srcdir)/scripts/snarf-guile-m4-docs $(top_srcdir)/guile-config/guile.m4 \
 	  > $(srcdir)/$@
 
 MAINTAINERCLEANFILES = autoconf-macros.texi
diff --git a/libguile/gc-freelist.c b/libguile/gc-freelist.c
index c8824df..4b38992 100644
--- a/libguile/gc-freelist.c
+++ b/libguile/gc-freelist.c
@@ -29,6 +29,8 @@
 
 scm_t_cell_type_statistics scm_i_master_freelist;
 scm_t_cell_type_statistics scm_i_master_freelist2;
+scm_t_cell_type_statistics *scm_i_master_freelist_ptr = &scm_i_master_freelist;
+scm_t_cell_type_statistics *scm_i_master_freelist2_ptr = &scm_i_master_freelist2;
 
 
 
diff --git a/libguile/gc.h b/libguile/gc.h
index 3cedeb4..3b60689 100644
--- a/libguile/gc.h
+++ b/libguile/gc.h
@@ -273,9 +273,13 @@ SCM_API size_t scm_max_segment_size;
 #define SCM_SET_FREELIST_LOC(key,ptr) scm_i_pthread_setspecific ((key), (ptr))
 #define SCM_FREELIST_LOC(key) ((SCM *) scm_i_pthread_getspecific (key))
 SCM_API scm_i_pthread_key_t scm_i_freelist;
+SCM_API scm_i_pthread_key_t *scm_i_freelist_ptr;
 SCM_API scm_i_pthread_key_t scm_i_freelist2;
+SCM_API scm_i_pthread_key_t *scm_i_freelist2_ptr;
 SCM_API struct scm_t_cell_type_statistics scm_i_master_freelist;
+SCM_API struct scm_t_cell_type_statistics *scm_i_master_freelist_ptr;
 SCM_API struct scm_t_cell_type_statistics scm_i_master_freelist2;
+SCM_API struct scm_t_cell_type_statistics *scm_i_master_freelist2_ptr;
 
 
 SCM_API unsigned long scm_gc_cells_swept;
diff --git a/libguile/inline.h b/libguile/inline.h
index eae1e22..de2a50a 100644
--- a/libguile/inline.h
+++ b/libguile/inline.h
@@ -109,10 +109,10 @@ SCM
 scm_cell (scm_t_bits car, scm_t_bits cdr)
 {
   SCM z;
-  SCM *freelist = SCM_FREELIST_LOC (scm_i_freelist);
+  SCM *freelist = SCM_FREELIST_LOC (*scm_i_freelist_ptr);
 
   if (scm_is_null (*freelist))
-    z = scm_gc_for_newcell (&scm_i_master_freelist, freelist);
+    z = scm_gc_for_newcell (scm_i_master_freelist_ptr, freelist);
   else
     {
       z = *freelist;
@@ -180,10 +180,10 @@ scm_double_cell (scm_t_bits car, scm_t_bits cbr,
 		 scm_t_bits ccr, scm_t_bits cdr)
 {
   SCM z;
-  SCM *freelist = SCM_FREELIST_LOC (scm_i_freelist2);
+  SCM *freelist = SCM_FREELIST_LOC (*scm_i_freelist2_ptr);
 
   if (scm_is_null (*freelist))
-    z = scm_gc_for_newcell (&scm_i_master_freelist2, freelist);
+    z = scm_gc_for_newcell (scm_i_master_freelist2_ptr, freelist);
   else
     {
       z = *freelist;
diff --git a/libguile/threads.c b/libguile/threads.c
index 3d6df11..ff8771d 100644
--- a/libguile/threads.c
+++ b/libguile/threads.c
@@ -1680,6 +1680,8 @@ scm_dynwind_critical_section (SCM mutex)
 /*** Initialization */
 
 scm_i_pthread_key_t scm_i_freelist, scm_i_freelist2;
+scm_i_pthread_key_t *scm_i_freelist_ptr = &scm_i_freelist;
+scm_i_pthread_key_t *scm_i_freelist2_ptr = &scm_i_freelist2;
 scm_i_pthread_mutex_t scm_i_misc_mutex;
 
 #if SCM_USE_PTHREAD_THREADS
diff --git a/srfi/srfi-13.c b/srfi/srfi-13.c
index dd5ce9b..8655885 100644
--- a/srfi/srfi-13.c
+++ b/srfi/srfi-13.c
@@ -29,12 +29,15 @@
 
 #include <srfi/srfi-13.h>
 
+void scm_init_srfi_13_no_clash_with_libguile (void);
+void scm_init_srfi_13_14_no_clash_with_libguile (void);
+
 void
-scm_init_srfi_13 (void)
+scm_init_srfi_13_no_clash_with_libguile (void)
 {
 }
 
 void
-scm_init_srfi_13_14 (void)
+scm_init_srfi_13_14_no_clash_with_libguile (void)
 {
 }
diff --git a/srfi/srfi-14.c b/srfi/srfi-14.c
index 1a7297b..76cea87 100644
--- a/srfi/srfi-14.c
+++ b/srfi/srfi-14.c
@@ -23,12 +23,15 @@
 
 #include <srfi/srfi-14.h>
 
+void scm_init_srfi_14_no_clash_with_libguile (void);
+void scm_c_init_srfi_14_no_clash_with_libguile (void);
+
 void
-scm_init_srfi_14 (void)
+scm_init_srfi_14_no_clash_with_libguile (void)
 {
 }
 
 void
-scm_c_init_srfi_14 (void)
+scm_c_init_srfi_14_no_clash_with_libguile (void)
 {
 }
diff --git a/srfi/srfi-4.c b/srfi/srfi-4.c
index f40c6b3..f2e29e2 100644
--- a/srfi/srfi-4.c
+++ b/srfi/srfi-4.c
@@ -28,8 +28,10 @@
 
 #include <srfi/srfi-4.h>
 
+void scm_init_srfi_4_no_clash_with_libguile (void);
+
 void
-scm_init_srfi_4 (void)
+scm_init_srfi_4_no_clash_with_libguile (void)
 {
 }
 
diff --git a/srfi/srfi-60.c b/srfi/srfi-60.c
index e173369..a91cb50 100644
--- a/srfi/srfi-60.c
+++ b/srfi/srfi-60.c
@@ -22,9 +22,9 @@
 #endif
 
 #include <libguile.h>
-#include <libguile/private-gc.h>  /* for SCM_MIN */
 #include <srfi/srfi-60.h>
 
+#define SCM_MIN(A, B) ((A) < (B) ? (A) : (B))
 
 SCM_DEFINE (scm_srfi60_log2_binary_factors, "log2-binary-factors", 1, 0, 0,
             (SCM n),
-- 
1.5.6.5

Re: MinGW build fixes

[Ludovic Courtès]

Hello!

Neil Jerram <neil@ossau.uklinux.net> writes:

> - MinGW can't export/import struct variables from/to a DLL, so wrap
>   the ones needed by inline.h in exported/imported pointer variables.

[...]

> * libguile/gc.h (scm_i_freelist_ptr, scm_i_freelist2_ptr,
>   scm_i_master_freelist_ptr, scm_i_master_freelist2_ptr): New global
>   variable declarations.
>
> * libguile/inline.h (scm_cell): Use scm_i_freelist_ptr and
>   scm_i_master_freelist_ptr instead of scm_i_freelist and
>   scm_i_master_freelist.
>   (scm_double_cell): Use scm_i_freelist2_ptr and
>   scm_i_master_freelist2_ptr instead of scm_i_freelist2 and
>   scm_i_master_freelist2.
>
> * libguile/threads.c (scm_i_freelist_ptr, scm_i_freelist2_ptr): New
>   global variables.

[...]

>  SCM_API struct scm_t_cell_type_statistics scm_i_master_freelist;
> +SCM_API struct scm_t_cell_type_statistics *scm_i_master_freelist_ptr;
>  SCM_API struct scm_t_cell_type_statistics scm_i_master_freelist2;
> +SCM_API struct scm_t_cell_type_statistics *scm_i_master_freelist2_ptr;

I don't understand why this fixes anything, since the `_ptr' variables
are declared as `SCM_API' just like the non-`_ptr' variables.

Also, it adds an indirection, which may impact performance.

Other than that, I think this is good news!

Thanks,
Ludo'.

Re: MinGW build fixes

[Neil Jerram]

ludo@gnu.org (Ludovic Courtès) writes:

>>  SCM_API struct scm_t_cell_type_statistics scm_i_master_freelist;
>> +SCM_API struct scm_t_cell_type_statistics *scm_i_master_freelist_ptr;
>>  SCM_API struct scm_t_cell_type_statistics scm_i_master_freelist2;
>> +SCM_API struct scm_t_cell_type_statistics *scm_i_master_freelist2_ptr;
>
> I don't understand why this fixes anything, since the `_ptr' variables
> are declared as `SCM_API' just like the non-`_ptr' variables.

Indeed.  My guess is that it's because the DLL import/export mechanism
works for atomic data - i.e. chars, ints, pointers etc - but not for
structs.  Or at least, not in the same way - it could be that some
extra magic incantation is needed somewhere.

(For example, there's no problem with scm_cells_allocated, which is
also referenced by the inline scm_cell code.)

I haven't managed to find a reference for this, though.  Also this
restriction could be one imposed by MSVC/Windows, or it could be a
restriction in MinGW's emulation of that.

I guess the thing to do would be to email the MinGW people, so I'll do
that.

> Also, it adds an indirection, which may impact performance.

Yes.  So I guess I should make these diffs #ifdef __MINGW32__.

> Other than that, I think this is good news!

Me too.  On the one hand Windows isn't a free platform, so it can't be
our prime concern.  On the other, I think it's good for us to grow the
possible audience for Guile as much as possible.

(Guile is already in Cygwin, of course, but it's an older version and
AFAIK the build was not straightforward.  So these fixes are about
making Windows builds work out of the upstream box.)

Regards,
        Neil