Reconsideration of MinGW work

Reconsideration of MinGW work

[Neil Jerram]

I've been making gradual progress on MinGW cross building, but I've
reached a point where I'm no longer sure that this is worthwhile.  This
email explains why, and invites comments from anyone interested in this
- especially from anyone who is really trying to use Guile on Windows.

First, I've found that completing a successful build (i.e. autogen.sh,
configure and make) is not at all the end of the story; it's only the
first part of what is really needed - because at runtime some key pieces
of function can still be missing, or can behave differently on
MinGW/Windows than on Linux/POSIX.  Two examples of this are operations
on file descriptors - where on MinGW/Windows different functions must be
called for sockets than for real files - and regular expressions, which
are not supported by the MinGW/Windows C library.

Therefore the definition of "success" also needs to include a successful
"make check", or some equivalent of that.  Using MSYS and MinGW on
Windows, I assume that "make check" can be run just as easily as
"make".  When cross-building with i586-mingw32msvc-* tools on Linux,
"make check" can be run (in principle) if you also have Wine installed,
and that's the setup that I've been using recently.

Second, though, it turns out that using i586-mingw32msvc-* and Wine on
Linux unfortunately does not give the same results as MSYS and MinGW on
Windows.  For example I've found that system(NULL) throws a SIGSEGV
under Wine, but for Carlo Bramix, working on Windows, that wasn't a
problem; and instead, for Carlo, there were other problems that I don't
see with a Linux cross build.

This is hardly surprising.  Although Wine aims to emulate the Windows
runtime DLLs precisely, of course it is only software and so can have
bugs.  But it is an extra hassle in practice.

Third, I reviewed my own need for Guile on Windows - and in fact I've
been perfectly happily using the Cygwin version for some time now.  So
actually I don't currently need a MinGW version - and maybe that would
be true for other Guile on Windows users too.

Looking forwards, supporting a Cygwin build of Guile will always be much
easier than supporting a MinGW build, because the whole point of Cygwin
is to provide an emulation on Windows of the POSIX API, and that's what
most of the Guile code assumes.

Fourth, I've realized that I significantly misunderstood MinGW's
objective.  Its true objective is to provide free software tools for
building native Windows programs to run with the Win32 runtime DLLs, and
the MinGW website is actually very clear about this.  That means that it
is 100% targeting the API provided by the Win32 DLLs.  But somehow or
other I had got the idea that it was also a bit Cygwin-ish, in trying to
provide pieces of the Linux/POSIX API that aren't provided by those
DLLs.

That last idea is completely wrong, which means that trying to build a
POSIX-assuming project for MinGW is always going to be hard.

Fifth, and now thinking more of the future with 1.9/2.0, I wondered if
it would be better to address MinGW porting problems within Gnulib,
instead of putting lots of #ifdef __MINGW32__ into Guile's own code.
And in fact yes, based on a small and completely unscientific sample of
Google results, it seems the Gnulib guys do regard MinGW compatibility
as part of their remit.

So putting #ifdef __MINGW32__ stuff into Guile is probably unhelpful
anyway, because it would be better to contribute to Gnulib instead.  And
it's possible that Gnulib might already have what we need.

And finally, I noticed that there already claims to be a MinGW port of
Guile 1.8, here:

http://sourceforge.net/projects/mingw/files/
http://sourceforge.net/projects/mingw/files/MSYS%20guile/guile-1.8.7-1/guile-1.8.7-1-msys.RELEASE_NOTES/download

Therefore I'm inclined to conclude the following.

- Overall, it isn't as important as I had been thinking to get a
  complete MinGW build of Guile.  I personally plan to concentrate more
  now on 1.8 - 1.9/2.0 compatibility, and on the manual.

- As far as future development is concerned, including the current
  "master" branch, MinGW portability fixes should be directed at Gnulib
  if possible, instead of done directly in the Guile code.

- As far as 1.8.x is concerned, I'm not sure if there's any need out
  there that isn't already met either by Cygwin or by the MinGW port
  linked above.

Anyone still reading?  Congratulations if so - and please let me know
what you think!

Regards,
      Neil

Re: Reconsideration of MinGW work

[Grant Rettke]

On Sun, Mar 21, 2010 at 3:51 PM, Neil Jerram <neil@ossau.uklinux.net> wrote:
> This email explains why, and invites comments from anyone interested in this
> - especially from anyone who is really trying to use Guile on Windows.

I've decided only to run UNIX origin software on Cygwin as there is
always some "gotcha" that makes any other approach not worth the
effort.

Re: Reconsideration of MinGW work

[Ken Raeburn]

On Mar 21, 2010, at 16:51, Neil Jerram wrote:
> First, I've found that completing a successful build (i.e. autogen.sh,
> configure and make) is not at all the end of the story; it's only the
> first part of what is really needed - because at runtime some key pieces
> of function can still be missing, or can behave differently on
> MinGW/Windows than on Linux/POSIX.  Two examples of this are operations
> on file descriptors - where on MinGW/Windows different functions must be
> called for sockets than for real files - and regular expressions, which
> are not supported by the MinGW/Windows C library.

Yes... you then also need to decide if Guile is exposing GNU/POSIX functionality, whatever the native OS functionality is, or some abstraction... e.g., looking at file descriptors, do you try to make sockets look like files, expose the differing underlying functionality to the Scheme coder, or make them distinct object types with some APIs that can handle both (and may do so with identical or different code, depending on the OS)?

I can imagine some ways in which distinguishing them would be beneficial.  You could code getpeername to reject arguments that aren't socket objects (instead of calling into the kernel to see if it works), and similarly for fstat and sockets.  On the other hand, the program will be started up with file descriptors on objects of types you don't know initially (on UNIX), so you still have to handle that.

> Therefore the definition of "success" also needs to include a successful
> "make check", or some equivalent of that.  Using MSYS and MinGW on
> Windows, I assume that "make check" can be run just as easily as
> "make".  When cross-building with i586-mingw32msvc-* tools on Linux,
> "make check" can be run (in principle) if you also have Wine installed,
> and that's the setup that I've been using recently.

As an aside, one nice thing about the DejaGnu setup we had at Cygnus was that you could easily(?) define how to run tests in cross-build setups, and it could include "copy these files over to the target machine described in this config file and then use this command to invoke the program over there".  This was mostly used for embedded systems testing, with target boards connected via serial ports.  Once you've got the framework in place, defining new target systems and ways of communicating with them can be relatively easy.

Having just bought a Lemote Yeelong notebook at LibrePlanet this weekend, it has already occurred to me that cross-compilation from a faster machine (like my 8-core home Mac) targeting the Yeelong would be desirable; running tests too would be better still, when a machine is available.

> Second, though, it turns out that using i586-mingw32msvc-* and Wine on
> Linux unfortunately does not give the same results as MSYS and MinGW on
> Windows.  For example I've found that system(NULL) throws a SIGSEGV
> under Wine, but for Carlo Bramix, working on Windows, that wasn't a
> problem; and instead, for Carlo, there were other problems that I don't
> see with a Linux cross build.

You *are* reporting these bugs, aren't you? :-)

> Anyone still reading?  Congratulations if so - and please let me know
> what you think!

I think cross-compilation and cross-testing is a good thing to be able to do.  Obviously cross-testing requires some additional setup -- as a trivial example, the test suite code isn't going to know the name or IP address of your Windows or Yeelong machine.  I know it's hard to set up, though, and especially hard to maintain if few people actually use it.  Perhaps having build farms available with multiple platform types can help there.

One nagging concern I've got about my Guile-Emacs project is the seemingly narrow focus of active Guile developers as far as platforms are concerned.  I'm one of, what, two or three people testing the development versions on Mac OS X now and then, and most of the rest of the work is on x86 or x86-64 GNU/Linux systems, it seems?  But Emacs works on a lot more systems (including MinGW, for people who don't want all of Cygwin), and saying "hey, we can change Emacs to be Guile-based on x86 GNU/Linux systems; too bad about all the other platforms" wouldn't go over terribly well.

For a random Scheme implementation, it's okay to pick the set of platforms you want to support, and drop whatever's inconvenient.  But if you want to be the official extension language for the GNU project, used by (theoretically) lots of GNU packages, you've got to support all the platforms the developers of those platforms want to support, if you possibly can.  I think that includes both Cygwin and MinGW, and probably not just supporting whatever subset can be mapped into POSIX functions via Gnulib.  We can probably punt on VMS, though....

Ken

Re: Reconsideration of MinGW work

[Andy Wingo]

Hi!

On Mon 22 Mar 2010 02:28, Ken Raeburn <raeburn@raeburn.org> writes:

> I think cross-compilation and cross-testing is a good thing to be able
> to do.

Totally agreed. I'd like to start compiling Guile for ARM devices now.

> Perhaps having build farms available with multiple platform types can
> help there.

There has been the Debian build farms (and will be once 2.0 is out):

  http://packages.debian.org/sid/guile-1.8

There are the nixos builders, which track git, and build on linux, fbsd,
and darwin:

  http://hydra.nixos.org/jobset/gnu/guile-master

> One nagging concern I've got about my Guile-Emacs project is the
> seemingly narrow focus of active Guile developers as far as platforms
> are concerned. I'm one of, what, two or three people testing the
> development versions on Mac OS X now and then, and most of the rest of
> the work is on x86 or x86-64 GNU/Linux systems, it seems?

So, point taken, sorta; but there are the builders above, I support mac
installations of Guile as well (intel 10.4 and 10.5 right now), Ludovic
regularly builds on sparc systems, and there are only about 6 active
committers anyway!

> But Emacs works on a lot more systems (including MinGW, for people who
> don't want all of Cygwin), and saying "hey, we can change Emacs to be
> Guile-based on x86 GNU/Linux systems; too bad about all the other
> platforms" wouldn't go over terribly well.
>
> For a random Scheme implementation, it's okay to pick the set of
> platforms you want to support, and drop whatever's inconvenient. But if
> you want to be the official extension language for the GNU project, used
> by (theoretically) lots of GNU packages, you've got to support all the
> platforms the developers of those platforms want to support, if you
> possibly can. I think that includes both Cygwin and MinGW, and probably
> not just supporting whatever subset can be mapped into POSIX functions
> via Gnulib. We can probably punt on VMS, though....

Heh, regarding VMS :)

Seriously though, I think we should drive the gnulib approach as far as
it can go, and only then make concessions in the areas where a gnulib
solution is inappropriate.

Cheers,

Andy
-- 
http://wingolog.org/

Re: Reconsideration of MinGW work

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 1841 bytes --]


Ken Raeburn <raeburn@raeburn.org> writes:

> One nagging concern I've got about my Guile-Emacs project is the
> seemingly narrow focus of active Guile developers as far as platforms
> are concerned.  I'm one of, what, two or three people testing the
> development versions on Mac OS X now and then, and most of the rest of
> the work is on x86 or x86-64 GNU/Linux systems, it seems?  But Emacs
> works on a lot more systems (including MinGW, for people who don't
> want all of Cygwin), and saying "hey, we can change Emacs to be
> Guile-based on x86 GNU/Linux systems; too bad about all the other
> platforms" wouldn't go over terribly well.

I test on NetBSD, and in theory care about not only i386 and amd64 but
also sparc64.  But I have not had a lot of spare time lately to hack on
guile.  I am running autobuilds on list.ir.bbn.com (NetBSD amd64):

  http://autobuild.josefsson.org/guile/

and it looked like some non-portable assumptions have crept in:

  http://autobuild.josefsson.org/guile/log-201003220603936147000.txt

> For a random Scheme implementation, it's okay to pick the set of
> platforms you want to support, and drop whatever's inconvenient.  But
> if you want to be the official extension language for the GNU project,
> used by (theoretically) lots of GNU packages, you've got to support
> all the platforms the developers of those platforms want to support,
> if you possibly can.  I think that includes both Cygwin and MinGW, and
> probably not just supporting whatever subset can be mapped into POSIX
> functions via Gnulib.  We can probably punt on VMS, though....

The target set definitely ought to include cygwin, but the GNU project
has a bias for Free and/or POSIX operating systems so I am willing to
forgo getting upset about lack of mingw support.  But surely we should
be happy if someone provides it.


[-- Attachment #2: Type: application/pgp-signature, Size: 194 bytes --]

Re: Reconsideration of MinGW work

[Neil Jerram]

Ken Raeburn <raeburn@raeburn.org> writes:

> Yes... you then also need to decide if Guile is exposing GNU/POSIX
> functionality, whatever the native OS functionality is, or some
> abstraction...

Ideally, yes, I think.  In other words, I think it's preferable if Guile
provides the same function to applications on all platforms.  (Emacs
shows how fantastically useful this is.)

> Having just bought a Lemote Yeelong notebook at LibrePlanet [...]

Aside: I was wondering about buying one of those too, but haven't yet
because of performance concerns.  Can it compile Guile successfully, and
if so how long does it take?

> You *are* reporting these bugs, aren't you? :-)

Err... not yet.  So thanks for the reminder.  But I just checked, and
their (i.e. Wine's) bugzilla requires creating an account, so I'm not
sure I'll bother.  (Is that really irresponsible of me?)

> I think cross-compilation and cross-testing is a good thing to be able
> to do.  Obviously cross-testing requires some additional setup -- as a
> trivial example, the test suite code isn't going to know the name or
> IP address of your Windows or Yeelong machine.  I know it's hard to
> set up, though, and especially hard to maintain if few people actually
> use it.  Perhaps having build farms available with multiple platform
> types can help there.

I agree that this is useful, but it is, as you say, difficult to set up.
Personally, to the extent that I continue working on this, I think I'm
happy to limit myself to what can be achieved with emulation (i.e. with
Wine) on a single Linux box.  At least so far as regular builds are
concerned; of course I'd also expect occasionally to copy the built DLLs
over to Windows and try them there.

> One nagging concern I've got about my Guile-Emacs project is the
> seemingly narrow focus of active Guile developers as far as platforms
> are concerned.  I'm one of, what, two or three people testing the
> development versions on Mac OS X now and then, and most of the rest of
> the work is on x86 or x86-64 GNU/Linux systems, it seems?

Yes, but this isn't intentional, just a matter of limited resources.

On the other hand, in the Windows case, it seems there is independent
effort being put in to the problem downstream.  I wonder why that's
downstream - perhaps because we're not good enough at accepting patches
when they're offered?

On the MacOS side I haven't been closely involved recently, but I think
- for all platforms other than the really mainstream ones that you've
mentioned above - we need people who are authoritative about what the
solution for a given problem is.  If I see a patch for MacOS that seems
to be done at the right place in the build, and obviously doesn't
inappropriately impact other platforms, and the submitter is
authoritative about it being the right solution for MacOS, it's easy to
say "OK, I believe you, and there's no risk, so let's commit it".  But
in the conversations that I remember there hasn't been that
authoritative tone; it's more like I've needed to work out half of the
solution myself, which I'm not in a position to do.  So then things get
left sitting, etc.  And similarly for other less mainstream platforms.

> For a random Scheme implementation, it's okay to pick the set of
> platforms you want to support, and drop whatever's inconvenient.

Which is absolutely not my intention...

>  But if you want to be the official extension language for the GNU
> project, used by (theoretically) lots of GNU packages, you've got to
> support all the platforms the developers of those platforms want to
> support, if you possibly can.

I agree.

>  I think that includes both Cygwin and MinGW,

This is a key point.  My previous email hinted that Cygwin support might
be enough, and several people have commented that it isn't.  So I'm
happy now that that hint was wrong.  For me, then, the questions are
whether we need any more Cygwin/MinGW support for 1.8.x than what
already exists, and how, structurally speaking, we do Cygwin/MinGW
support for 1.9/2.0 and future releases.

Having said that, I don't feel I understand technically why a MinGW
version of Guile, including all necessary emulation code, would run
faster or be less bloated than a Cygwin Guile.  Can someone explain
that?

> and probably not just supporting whatever subset can be mapped into
> POSIX functions via Gnulib.

I don't think I understand your point here.  Gnulib isn't a fixed point.
If we can add emulation code into Guile, can't we just as easily add it
into Gnulib?

Regards,
       Neil

Re: Reconsideration of MinGW work

[Ken Raeburn]

On Mar 22, 2010, at 20:04, Neil Jerram wrote:
> Ken Raeburn <raeburn@raeburn.org> writes:
>> Yes... you then also need to decide if Guile is exposing GNU/POSIX
>> functionality, whatever the native OS functionality is, or some
>> abstraction...
> 
> Ideally, yes, I think.  In other words, I think it's preferable if Guile
> provides the same function to applications on all platforms.  (Emacs
> shows how fantastically useful this is.)

But is that POSIX, or something more abstract, when POSIX and other platforms differ in significant ways?

>> Having just bought a Lemote Yeelong notebook at LibrePlanet [...]
> 
> Aside: I was wondering about buying one of those too, but haven't yet
> because of performance concerns.  Can it compile Guile successfully, and
> if so how long does it take?

Don't know yet; I've been having network problems until tonight.  But I plan to try it out, maybe later this week.

>> You *are* reporting these bugs, aren't you? :-)
> 
> Err... not yet.  So thanks for the reminder.  But I just checked, and
> their (i.e. Wine's) bugzilla requires creating an account, so I'm not
> sure I'll bother.  (Is that really irresponsible of me?)

It is kind of a pain, isn't it?  But with all the spam flying around these days, it seems to me that fewer and fewer projects/sites accept anonymous email or web submissions.

> I agree that this is useful, but it is, as you say, difficult to set up.
> Personally, to the extent that I continue working on this, I think I'm
> happy to limit myself to what can be achieved with emulation (i.e. with
> Wine) on a single Linux box.  At least so far as regular builds are
> concerned; of course I'd also expect occasionally to copy the built DLLs
> over to Windows and try them there.

I believe NetBSD (and maybe some of the GNU/Linux distros?) has made progress in cross-compiling for the OS on one architecture, hosted on the OS running on another architecture.  (And I'm not just talking about 32-bit and 64-bit variants of the same architecture -- this might be x86 to sparc, for example.)  So a first cut at something like this might just take as little on the Guile side as setting up the test suite to copy binaries and test scripts over to the target machine with scp, then run a test over ssh.  Hmm....

>> One nagging concern I've got about my Guile-Emacs project is the
>> seemingly narrow focus of active Guile developers as far as platforms
>> are concerned.  I'm one of, what, two or three people testing the
>> development versions on Mac OS X now and then, and most of the rest of
>> the work is on x86 or x86-64 GNU/Linux systems, it seems?
> 
> Yes, but this isn't intentional, just a matter of limited resources.

I'm glad to see the other non-Linux people were paying attention. ;-)  I guess I overstated the situation, but I have on occasion gotten the feeling that only GNU/Linux systems were getting tested as part of the main development work, with "porting" back to other systems happening later, when Greg or I or someone else found problems.  (I can certainly understand not trying on every system on the planet.  But *occasionally* it feels like only one is really used.)  But, we are paying attention, when we've got time, and those of you doing most of the core development work are doing awesome stuff, and listening to us when platform issues come up, so I shouldn't be complaining too much....

The build farms are a good improvement; we should try to get as much variety there as we can.  And maybe some sort of alert (bot messages to #guile ?) when a build fails...

> On the other hand, in the Windows case, it seems there is independent
> effort being put in to the problem downstream.  I wonder why that's
> downstream - perhaps because we're not good enough at accepting patches
> when they're offered?

We could ask and see.  And maybe get someone to figure out how to fit Cygwin and/or MinGW into the build farm (or some automatic build-and-test setup), and keep an eye on things, while we help them get their porting changes integrated.

> On the MacOS side I haven't been closely involved recently, but I think
> - for all platforms other than the really mainstream ones that you've
> mentioned above - we need people who are authoritative about what the
> solution for a given problem is.  If I see a patch for MacOS that seems
> to be done at the right place in the build, and obviously doesn't
> inappropriately impact other platforms, and the submitter is
> authoritative about it being the right solution for MacOS, it's easy to
> say "OK, I believe you, and there's no risk, so let's commit it".  But
> in the conversations that I remember there hasn't been that
> authoritative tone; it's more like I've needed to work out half of the
> solution myself, which I'm not in a position to do.  So then things get
> left sitting, etc.  And similarly for other less mainstream platforms.

I don't mind trying to sound authoritative sometimes. :-)  And I deal pretty well with the UNIXy bits of Mac OS X.  But sometimes I'm unsure how things are intended to work on the Guile side.

>> and probably not just supporting whatever subset can be mapped into
>> POSIX functions via Gnulib.
> 
> I don't think I understand your point here.  Gnulib isn't a fixed point.
> If we can add emulation code into Guile, can't we just as easily add it
> into Gnulib?

I had the impression that Gnulib was about emulating the GNU/Linux/POSIX type API on other platforms, not providing an abstraction.  But it looks like I'm wrong; at the very least, there are non-POSIX utility functions as well as replacements for missing POSIX routines.  Maybe that is the way to go, then....

Ken

Re: Reconsideration of MinGW work

[Andy Wingo]

On Tue 23 Mar 2010 07:59, Ken Raeburn <raeburn@raeburn.org> writes:

> The build farms are a good improvement; we should try to get as much
> variety there as we can. And maybe some sort of alert (bot messages to
> #guile ?) when a build fails...

Subscribe to guile-commits@gnu.org :)

Andy
-- 
http://wingolog.org/

guile on lemote (was Re: Reconsideration of MinGW work)

[Ken Raeburn]

On Mar 22, 2010, at 20:04, Neil Jerram wrote:
>> Having just bought a Lemote Yeelong notebook at LibrePlanet [...]
> 
> Aside: I was wondering about buying one of those too, but haven't yet
> because of performance concerns.  Can it compile Guile successfully, and
> if so how long does it take?

It appears that released versions of Guile are packaged for the machine already.

I checked out the current git source to build; configuring took 3.5 minutes just to remind me that I needed to fetch the GC library and install it.  I had 7.2-alpha2 sitting around, so copied it over and installed it.

Unfortunately libunistring failed one of its tests when I was installing it.  At first glance, I'm not sure if it's a library bug or a testcase bug.

I supplied the --with-libunistring-prefix option, which got converted to a "-R/foo/bar/lib" option for the compiler... which gcc says it doesn't recognize, though it keeps on going.

Configure reports that readline is too old; I've got 5.2 installed.  Actually, I have the runtime support but not the "-dev" package installed, so there's no .so to link against, only a .so.5 for already-linked programs, so the error message is wrong.  I installed the -dev package.

A successful configure run took four minutes.  Building the whole thing took 99 minutes.  It's a single-core system and the build was almost entirely CPU-bound, so I didn't try a parallel build.

In my Guile build, "make check" got to sxml.ssax.test, where it seems to have hung in semaphore wait in the GC library.

I'm retrying with 7.2-alpha4; 100 minutes this time.  It compiled the C code okay again, but it had been working on compiling psyntax-pp.scm for over 25 CPU minutes when I wandered away.  Based on timestamps, it looks like it took 29 minutes to compile, and then boot-9.scm took another 11, though later stuff built more quickly.  Running tests... after 5m18s, it looks pretty happy; some expected failures and untested or unsupported cases, 1051 unresolved test cases, and 12292 passes.

So, yeah, it looks like it can build the current Guile code, albeit slowly.  But, if you've got the core and compiler-related stuff already built and are just hacking on other Scheme code, it seems okay.

Ken

Re: guile on lemote

[Neil Jerram]

Ken Raeburn <raeburn@raeburn.org> writes:


> I checked out the current git source to build; configuring took 3.5
> minutes just to remind me that I needed to fetch the GC library and
> install it.  I had 7.2-alpha2 sitting around, so copied it over and
> installed it.
[...]
> A successful configure run took four minutes.  Building the whole
> thing took 99 minutes.  It's a single-core system and the build was
> almost entirely CPU-bound, so I didn't try a parallel build.

Thanks, Ken.  I think that tells me that I wouldn't have been able to
use the lemote efficiently for Guile development.  Let's hope the next
revision of the Loongson chip gets closer.

Regards,
      Neil

Re: Reconsideration of MinGW work

[Peter Brett]

Neil Jerram <neil@ossau.uklinux.net> writes:

> I've been making gradual progress on MinGW cross building, but I've
> reached a point where I'm no longer sure that this is worthwhile.  This
> email explains why, and invites comments from anyone interested in this
> - especially from anyone who is really trying to use Guile on Windows.

We get people coming to the gEDA user mailing list on a regular basis
saying, "Where can I find a version of gEDA for Windows?" and the
Windows builds we've put out have been generally well-received.  Since
Guile is one of our core dependencies, lack of Windows support in Guile
would mean that we wouldn't be able to provide a Windows build at all
(we already had massive problems at the start of the Guile 1.8.x series
with GMP portability, or lack thereof, and this meant that it took
almost three years after 1.8 became the supported stable release for us
to be able to stop supporting 1.6).

Cygwin isn't an option, unfortunately; we think it's totally
reasonable for users to want to use the native windowing system, not to
mention the fact that Cygwin is *dog slow*.

I imagine that other applications that link libguile would provide
similar feedback.

Regards,

                                Peter

-- 
Peter Brett <peter@peter-b.co.uk>
Remote Sensing Research Group
Surrey Space Centre

Re: Reconsideration of MinGW work

[Andy Wingo]

Hi Peter (& Neil & co),

On Mon 22 Mar 2010 09:10, Peter Brett <peter@peter-b.co.uk> writes:

> Neil Jerram <neil@ossau.uklinux.net> writes:
>
>> I've been making gradual progress on MinGW cross building, but I've
>> reached a point where I'm no longer sure that this is worthwhile.  This
>> email explains why, and invites comments from anyone interested in this
>> - especially from anyone who is really trying to use Guile on Windows.
>
> We get people coming to the gEDA user mailing list on a regular basis
> saying, "Where can I find a version of gEDA for Windows?" and the
> Windows builds we've put out have been generally well-received.  Since
> Guile is one of our core dependencies, lack of Windows support in Guile
> would mean that we wouldn't be able to provide a Windows build at all
> (we already had massive problems at the start of the Guile 1.8.x series
> with GMP portability, or lack thereof, and this meant that it took
> almost three years after 1.8 became the supported stable release for us
> to be able to stop supporting 1.6).

As Neil mentioned, hopefully we can get there via the gnulib path; it
does seem to be the right thing to do GNU-wise in this case. Guile's
code would stay the same, as much as possible, and Gnulib would have
mingw support in it, so that if you compile Guile on mingw, the Gnulib
compat layer gets compiled in.

Sounds acceptable, no? Granted it might take a little while to get it
right, but hopefully not three years.

Andy
-- 
http://wingolog.org/

Re: Reconsideration of MinGW work

[Linas Vepstas]

On 22 March 2010 14:00, Andy Wingo <wingo@pobox.com> wrote:
> Hi Peter (& Neil & co),
>
> On Mon 22 Mar 2010 09:10, Peter Brett <peter@peter-b.co.uk> writes:
>>
>> We get people coming to the gEDA user mailing list on a regular basis
>> saying, "Where can I find a version of gEDA for Windows?" and the
>> Windows builds we've put out have been generally well-received.  Since
>> Guile is one of our core dependencies, lack of Windows support in Guile
>> would mean that we wouldn't be able to provide a Windows build at all

FWIW, somehow gnucash (which uses guile) runs on windows, not sure how.

My pet peeve with mingw is the lack of ready-to-go regex. This is completely
unrelated to guile; I have another project that made the mistake of assuming
that regex "just worked" on windows, and I've been bitched at ever since.
Getting regex into mingw and having it "just work" would be excellent.

--linas

Re: Reconsideration of MinGW work

[Neil Jerram]

Linas Vepstas <linasvepstas@gmail.com> writes:

> My pet peeve with mingw is the lack of ready-to-go regex. This is completely
> unrelated to guile; I have another project that made the mistake of assuming
> that regex "just worked" on windows, and I've been bitched at ever
> since.

Gnulib has a regex library.

> Getting regex into mingw and having it "just work" would be excellent.

But that won't happen, because that's not part of MinGW's mission.  All
MinGW is trying to do is provide a free software toolset for building
native Windows programs - which at runtime will link to the standard set
of DLLs that M$ provides.  Those DLLs don't provide regex, so the MinGW
header files don't provide regex either.

      Neil

Re: Reconsideration of MinGW work

[Neil Jerram]

Peter Brett <peter@peter-b.co.uk> writes:

> Neil Jerram <neil@ossau.uklinux.net> writes:
>
>> I've been making gradual progress on MinGW cross building, but I've
>> reached a point where I'm no longer sure that this is worthwhile.  This
>> email explains why, and invites comments from anyone interested in this
>> - especially from anyone who is really trying to use Guile on Windows.
>
> We get people coming to the gEDA user mailing list on a regular basis
> saying, "Where can I find a version of gEDA for Windows?" and the
> Windows builds we've put out have been generally well-received.  Since
> Guile is one of our core dependencies, lack of Windows support in Guile
> would mean that we wouldn't be able to provide a Windows build at all
> (we already had massive problems at the start of the Guile 1.8.x series
> with GMP portability, or lack thereof, and this meant that it took
> almost three years after 1.8 became the supported stable release for us
> to be able to stop supporting 1.6).

Hi Peter,

I'm sorry, I didn't mean my email to suggest dropping Windows support.
It was more about what more (if anything) is needed for 1.8 releases,
and how to handle Windows support in future, and how those points relate
to a line of work that I've been spending time on recently.

Where are you getting your MinGW guile from at the moment?  Is it that
MinGW SF page that I mentioned, or somewhere else?

> Cygwin isn't an option, unfortunately; we think it's totally
> reasonable for users to want to use the native windowing system, not to
> mention the fact that Cygwin is *dog slow*.

OK, understood.  I'd like to understand more about what makes Cygwin
slow, though, in order to see why a MinGW Guile wouldn't end up being
equally slow.  But in the interim I'm happy to accept that MinGW is
needed.

Regards,
      Neil

Re: Reconsideration of MinGW work

[Ludovic Courtès]

Hello,

I was under the impression that Windows users would often prefer
software that requires “only” MinGW, as opposed to the more heavyweight
Cygwin.  From that point of view a MinGW port seems to be useful,
especially if Guile does part of the OS abstraction job that Cygwin
does.

I agree that portability tricks should primarily be directed to Gnulib.

Thanks,
Ludo’.

Re: Reconsideration of MinGW work

[Neil Jerram]

ludo@gnu.org (Ludovic Courtès) writes:

> Hello,
>
> I was under the impression that Windows users would often prefer
> software that requires “only” MinGW, as opposed to the more heavyweight
> Cygwin.  From that point of view a MinGW port seems to be useful,
> especially if Guile does part of the OS abstraction job that Cygwin
> does.

Yes, agreed now.  It was wrong of me to suggest that MinGW support might
not be necessary...

> I agree that portability tricks should primarily be directed to Gnulib.

Thanks,
       Neil

Re:Reconsideration of MinGW work

[carlo.bramix]

Hello!

> First, I've found that completing a successful build (i.e. autogen.sh,
> configure and make) is not at all the end of the story; it's only the
> first part of what is really needed - because at runtime some key pieces
> of function can still be missing, or can behave differently on
> MinGW/Windows than on Linux/POSIX.  Two examples of this are operations
> on file descriptors - where on MinGW/Windows different functions must be
> called for sockets than for real files - and regular expressions, which
> are not supported by the MinGW/Windows C library.

Unfortunately, the network is one of the common problems when porting. It could be resoved with some work and with some "tricks" if someone wants.
Did you mean "regex" with "regular expressions"? There are two of these libraries at mingw downloads but, unfortunately, I was not able to make them working: I had to take original sources and I recompiled myself.

> Second, though, it turns out that using i586-mingw32msvc-* and Wine on
> Linux unfortunately does not give the same results as MSYS and MinGW on
> Windows.  For example I've found that system(NULL) throws a SIGSEGV
> under Wine, but for Carlo Bramix, working on Windows, that wasn't a
> problem; and instead, for Carlo, there were other problems that I don't
> see with a Linux cross build.

Yes, it seems to be a bug of WINE.
Look the sources of _wsystem() function at:

http://source.winehq.org/git/wine.git/?a=blob;f=dlls/msvcrt/process.c;h=0b1eb01d2728b4df9e7d12a457dd3065bed1f1d1;hb=HEAD

As you can see, you get a SIGSEGV because the parameter "cmd" (which is NULL in sources of GUILE) is used immediately in strlenW().
Hopefully, this bug does not exist in the sources of ReactOS:

http://svn.reactos.org/svn/reactos/trunk/reactos/lib/sdk/crt/process/_system.c?view=markup

> This is hardly surprising.  Although Wine aims to emulate the Windows
> runtime DLLs precisely, of course it is only software and so can have
> bugs.  But it is an extra hassle in practice.
>
> Third, I reviewed my own need for Guile on Windows - and in fact I've
> been perfectly happily using the Cygwin version for some time now.  So
> actually I don't currently need a MinGW version - and maybe that would
> be true for other Guile on Windows users too.
>
> Looking forwards, supporting a Cygwin build of Guile will always be much
> easier than supporting a MinGW build, because the whole point of Cygwin
> is to provide an emulation on Windows of the POSIX API, and that's what
> most of the Guile code assumes.

I have not tried to compile latest GUILE 1.9.9 on CYGWIN but I will try it in the lunch pause tomorrow.
I'm quite confident it will work because I had not particular problems on previous versions.

> Fourth, I've realized that I significantly misunderstood MinGW's
> objective.  Its true objective is to provide free software tools for
> building native Windows programs to run with the Win32 runtime DLLs, and
> the MinGW website is actually very clear about this.  That means that it
> is 100% targeting the API provided by the Win32 DLLs.  But somehow or
> other I had got the idea that it was also a bit Cygwin-ish, in trying to
> provide pieces of the Linux/POSIX API that aren't provided by those
> DLLs.
>
> That last idea is completely wrong, which means that trying to build a
> POSIX-assuming project for MinGW is always going to be hard.

Cygwin is the easier and quickest way for getting a posix software working on Windows, at least for a developer: configure, make, make install and it is done.
For an user, things may be a bit different and this is the reason because I started to port many free software and libraries to mingw with msys utilities (bash and typical unix tools).
Just for making an example, a simple graphical "Hello world!" application needs an absurd, huge amount of software, including a slave X server.
Instead, everything compiled with mingw is a plain win32 application that can run directly on the Windows Desktopand few DLLs, well integrated in the system and running at a native speed that you will never reach in cygwin.
Although many efforts have been made, cygwin acts more similar to virtual machine to me.
I'm not trying to change the decisions of the team in any way, I just wanted to show you why true win32 applications should be prefered to the ones made with cygwin (if this is possible to do, of course!).

> Fifth, and now thinking more of the future with 1.9/2.0, I wondered if
> it would be better to address MinGW porting problems within Gnulib,
> instead of putting lots of #ifdef __MINGW32__ into Guile's own code.
> And in fact yes, based on a small and completely unscientific sample of
> Google results, it seems the Gnulib guys do regard MinGW compatibility
> as part of their remit.
>
> So putting #ifdef __MINGW32__ stuff into Guile is probably unhelpful
> anyway, because it would be better to contribute to Gnulib instead.  And
> it's possible that Gnulib might already have what we need.
>
> And finally, I noticed that there already claims to be a MinGW port of
> Guile 1.8, here:
>
> http://sourceforge.net/projects/mingw/files/
> http://sourceforge.net/projects/mingw/files/MSYS%20guile/guile-1.8.7-1/guile-1.8.7-1-msys.RELEASE_NOTES/download
>

Me too, I made a working GUILE 1.8.6 that I'm currently using and until now it worked fine; afterall, I'm trying to build GUILE on Windows since version 1.8.3 :P

> Therefore I'm inclined to conclude the following.
>
> - Overall, it isn't as important as I had been thinking to get a
>   complete MinGW build of Guile.  I personally plan to concentrate more
>   now on 1.8 - 1.9/2.0 compatibility, and on the manual.
>
> - As far as future development is concerned, including the current
>   "master" branch, MinGW portability fixes should be directed at Gnulib
>   if possible, instead of done directly in the Guile code.

For a project as complex as guile, probably this sounds to be a good solution.

Sincerely,

Carlo Bramini.

Re: Reconsideration of MinGW work

[Neil Jerram]

"carlo.bramix" <carlo.bramix@libero.it> writes:

> Hello!

Hi Carlo!

> Unfortunately, the network is one of the common problems when
> porting. It could be resoved with some work and with some "tricks" if
> someone wants.

Indeed.  I know that I have patches pending for this.  I also wonder if
the MinGW Guile port at the SF page that I cited includes those tricks.
(I will take a look.)

>  Did you mean "regex" with "regular expressions"?

Yes.

> There
> are two of these libraries at mingw downloads but, unfortunately, I
> was not able to make them working: I had to take original sources and
> I recompiled myself.

Again, I wonder if the advertised MinGW Guile port has regex support.

>> Second, though, it turns out that using i586-mingw32msvc-* and Wine
>> on Linux unfortunately does not give the same results as MSYS and
>> MinGW on Windows.  For example I've found that system(NULL) throws a
>> SIGSEGV under Wine, but for Carlo Bramix, working on Windows, that
>> wasn't a problem; and instead, for Carlo, there were other problems
>> that I don't see with a Linux cross build.
>
> Yes, it seems to be a bug of WINE.  Look the sources of _wsystem()
> function at:
>
> http://source.winehq.org/git/wine.git/?a=blob;f=dlls/msvcrt/process.c;h=0b1eb01d2728b4df9e7d12a457dd3065bed1f1d1;hb=HEAD

Thanks.

> I have not tried to compile latest GUILE 1.9.9 on CYGWIN but I will
> try it in the lunch pause tomorrow.  I'm quite confident it will work
> because I had not particular problems on previous versions.

Thanks, that's good to know.

> DLLs, well integrated in the system and running at a native speed that
> you will never reach in cygwin.  Although many efforts have been made,
> cygwin acts more similar to virtual machine to me.

But why?  I don't doubt that this is true - because many people have
said this, and I've seen myself that Cygwin applications seem slow.  But
why, technically speaking, is it true?

>  I'm not trying to
> change the decisions of the team in any way, I just wanted to show you
> why true win32 applications should be prefered to the ones made with
> cygwin (if this is possible to do, of course!).

Thanks.  I accept that now.

> Me too, I made a working GUILE 1.8.6 that I'm currently using and
> until now it worked fine; afterall, I'm trying to build GUILE on
> Windows since version 1.8.3 :P

Yes.  I appreciate your efforts, and I'm sorry it's taken a while for us
to get everything needed upstream.

>> - As far as future development is concerned, including the current
>> "master" branch, MinGW portability fixes should be directed at
>> Gnulib if possible, instead of done directly in the Guile code.
>
> For a project as complex as guile, probably this sounds to be a good
> solution.

Many thanks for your comments.

      Neil