guile-2.0 on mingw: the sequel

guile-2.0 on mingw: the sequel

[Panicz Maciej Godek]

[-- Attachment #1: Type: text/plain, Size: 2869 bytes --]

Recently I've had a little more time to experiment
with guile-2.0 on mingw32. Following the advice of
Eli Zaretskii from
http://lists.gnu.org/archive/html/guile-user/2013-06/msg00028.html,
I compiled guile-2.0.9 without thread support,
and modified my demos so they no longer use theads
explicitly (they only use SDL timers).

I managed to go through the compilation process,
and (after modifying meta/Makefile.am according to Eli's
patch) to make install.

When I ran guile-2.0, I got the following message:
===
Throw without catch before boot:
Throw to key misc-error with args ("primitive-load-path" "Unable to find
file ~S in load path" ("ice-9/boot-9") #f)Aborting.

This application has requested the Runtime to terminate it in an unusual
way.
Please contact the application's support team for more information.
Cannot exit gracefully when init is in progress; aborting.
===

Supplying GUILE_LOAD_PATH=/usr/share/guile/2.0 explicitly
helped a little, but only during the first run -- it
compiled some files from that directory and proceeded to
the prompt.
When I tired later, I got a lot of messages like
===
;;; compiling C:/MinGW/share/guile/2.0\ice-9\vlist.scm
;;; it seems C:/MinGW/share/guile/2.0\ice-9\vlist.scm
;;; is part of the compiler; skipping auto-compilation
===

and a single instance of the following message
===
;;; WARNING: compilation of C:/MinGW/share/guile/2.0\system\vm\vm.scm
failed:
;;; ERROR: no code for module (system vm vm)
===

and it even displayed the welcome message, but when I tried
to (+ 2 3), I got:
===
While compiling expression:
ERROR: In procedure module-lookup: Unbound variable: compile
===

What turned out the most painful, however, was that I wasn't
able to run a program linked to libguile, not even the simplest
one, like
===
#include <stdio.h>

int main() {
  scm_init_guile();
  return 0;
}
===

which caused an abort with the following message:
===
Throw without catch before boot:
Throw to key stack-overflow with args (#f "Stack overflow" #f #f)Aborting.

This application has requested [...]
===

When I tried a similar piece of code, but with
scm_with_guile instead, I had the second part of
the message ("The application has requested ...")
in a "Microsoft Visual C++ Runtime Library" popup.

Exporting GUILE_LOAD_PATH didn't help at all

The Windows environment doesn't seem to be particularly
developer-friendly (or maybe I just don't know the right
tools), but I'd truly appreciate some help. (My goal is
to provide a binary distribution of the SLAYER framework
that I've presented before; it would allow programmers
to develop portable multimedia applications in Guile
Scheme. The development version of SLAYER is available
at https://puszcza.gnu.org.ua/projects/slayer )

Also, it would be helpful if someone could point
to the git revision which is known to build under
windows.

Thanks in advance,
M.

[-- Attachment #2: Type: text/html, Size: 4115 bytes --]

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

> Date: Thu, 22 Aug 2013 22:25:02 +0200
> From: Panicz Maciej Godek <godek.maciek@gmail.com>
> 
> I managed to go through the compilation process,
> and (after modifying meta/Makefile.am according to Eli's
> patch) to make install.

This compilation process includes compiling all the Scheme files that
come with Guile.  Since you say it succeeded, I don't understand what
you say later, see below.  Do you see a lot of *.go files in the
directory where you built Guile?

> When I ran guile-2.0, I got the following message:
> ===
> Throw without catch before boot:
> Throw to key misc-error with args ("primitive-load-path" "Unable to find
> file ~S in load path" ("ice-9/boot-9") #f)Aborting.

This seems to say that Guile cannot find its Scheme files.

> Supplying GUILE_LOAD_PATH=/usr/share/guile/2.0 explicitly
> helped a little

If that helped to avoid the failure to load, you probably didn't
specify a correct --prefix at configure time, or your "make install"
somehow didn't DTRT.

> but only during the first run -- it compiled some files from that
> directory and proceeded to the prompt.

This is what I don't understand: which files it needed to compile, and
why?  The compilation of Scheme files is part of the build process,
which you say you ran successfully to completion.  What am I missing?

> The Windows environment doesn't seem to be particularly
> developer-friendly (or maybe I just don't know the right
> tools)

Which developer-friendly features do you lack?  Perhaps it's a matter
of installing or configuring what is already available out there.

> but I'd truly appreciate some help.

Let's start with the basics.  Please describe:

 . Which build of what version of the GC library did you use, and
   where did you get the Windows build of that library?  Likewise for
   other build dependencies, like libunistring, libiconv, etc. --
   please tell where you got each one of them.

 . How did you configure Guile?  If you used any --prefix argument,
   please tell to which Windows directory does that prefix map on your
   system?

 . Did you see any warning or error messages during the build, and if
   so, please show them.

 . Did you see all the Scheme files being byte-compiled?

 . What command(s) did you use to install the built Guile?

 . How do you run Guile after installing it?  In particular, do you
   run it from the MSYS Bash or from the Windows cmd window?

IOW, you didn't tell enough details about the build and the usage to
be able to guess what possible problems could be in your way.

FWIW, I've successfully ran Guile from the Windows prompt after
building it, and successfully built GNU Make with Guile support.  So
it's definitely possible to do that with MinGW.

Re: guile-2.0 on mingw: the sequel

[Panicz Maciej Godek]

[-- Attachment #1: Type: text/plain, Size: 3875 bytes --]

2013/8/23 Eli Zaretskii <eliz@gnu.org>

>
> > I managed to go through the compilation process,
> > and (after modifying meta/Makefile.am according to Eli's
> > patch) to make install.
>
> This compilation process includes compiling all the Scheme files that
> come with Guile.  Since you say it succeeded, I don't understand what
> you say later, see below.  Do you see a lot of *.go files in the
> directory where you built Guile?
>

When I enter the build directory and type
$ find ./ -name '*.go'
it seems that every .scm file from `modules' directory has its .go
counterpart,


> > When I ran guile-2.0, I got the following message:
> > ===
> > Throw without catch before boot:
> > Throw to key misc-error with args ("primitive-load-path" "Unable to find
> > file ~S in load path" ("ice-9/boot-9") #f)Aborting.
>
> This seems to say that Guile cannot find its Scheme files.
>
> > Supplying GUILE_LOAD_PATH=/usr/share/guile/2.0 explicitly
> > helped a little
>
> If that helped to avoid the failure to load, you probably didn't
> specify a correct --prefix at configure time, or your "make install"
> somehow didn't DTRT.
>
>
I ./configured with --prefix=/usr (which is at the same mount point
as / on my mingw configuration). When it comes to make install,
I don't know what TRT would be


> > but only during the first run -- it compiled some files from that
> > directory and proceeded to the prompt.
>
> This is what I don't understand: which files it needed to compile, and
> why?  The compilation of Scheme files is part of the build process,
> which you say you ran successfully to completion.  What am I missing?
>
> I don't know either.



> > but I'd truly appreciate some help.
>
> Let's start with the basics.  Please describe:
>
>  . Which build of what version of the GC library did you use, and
>    where did you get the Windows build of that library?  Likewise for
>    other build dependencies, like libunistring, libiconv, etc. --
>    please tell where you got each one of them.
>
>
gc-7.2d compiled from the source from Hans Boehm's web site
(I don't remember whether I finally configured it with pthreads
or with windows threads, but I tried both for sure),
gmp-5.2.1 from the source (I think it is shared version),
libffi-3.0.13 from the source
libunistring-0.9.3 from the source
libiconv-1.14 and a few others bundled with mingw
Everything built from the source was configured with --prefix=/usr

 . How did you configure Guile?  If you used any --prefix argument,
>    please tell to which Windows directory does that prefix map on your
>    system?
>

Both / and /usr point to C:\MinGW\
Additionally, guile has been compiled with --program-suffix=-2.0


>  . Did you see any warning or error messages during the build, and if
>    so, please show them.
>
>
I can try to run the compilation for the second time, but
surely there were no errors that would stop make from
carrying on


>  . Did you see all the Scheme files being byte-compiled?
>
> Yes


>  . What command(s) did you use to install the built Guile?
>
>
./configure --without-threads --prefix=/usr
make
make install


>  . How do you run Guile after installing it?  In particular, do you
>    run it from the MSYS Bash or from the Windows cmd window?
>
> So far I've been running it only in msys bash

IOW, you didn't tell enough details about the build and the usage to
> be able to guess what possible problems could be in your way.
>
> I'm eager to tell whatever is needed, if it could eventually
help to get the process right.

FWIW, I've successfully ran Guile from the Windows prompt after
> building it, and successfully built GNU Make with Guile support.  So
> it's definitely possible to do that with MinGW.
>
>
And would it be possible to distribute the binary release somehow?
(in particular, the libguile.dll + scm/go modules that would be loaded
from current directory)

[-- Attachment #2: Type: text/html, Size: 6951 bytes --]

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

> Date: Fri, 23 Aug 2013 11:29:27 +0200
> From: Panicz Maciej Godek <godek.maciek@gmail.com>
> Cc: "guile-user@gnu.org" <guile-user@gnu.org>
> 
> When I enter the build directory and type
> $ find ./ -name '*.go'
> it seems that every .scm file from `modules' directory has its .go
> counterpart,

Are they also present in the tree where you installed Guile after
building it?

> I ./configured with --prefix=/usr (which is at the same mount point
> as / on my mingw configuration). When it comes to make install,
> I don't know what TRT would be

Configuring with --prefix=/usr is a mistake: there's no /usr on your
system, as far as MinGW programs are concerned.

You need to understand the difference between MinGW and MSYS programs
and their respective setups.  MSYS, like Cygwin, emulates a Posix
system, so it supports /usr etc.  It does so by mounting some Windows
directory (in your case, it seems to be C:/MinGW) as /.  But the Guile
you compiled is _not_ an MSYS program, it is a native Windows program,
so it doesn't know about this mount.  Since the build process records
the /usr prefix in the binary, the resulting Guile executables will
look for their files starting from the /usr directory on the current
drive, and that directory doesn't exist.

You need to reconfigure Guile with --prefix=c:/MinGW (assuming that is
where you want your installation tree to be rooted), and then rebuild
everything.  I would suggest "make distclean" followed by a new
configure command with the right prefix and then "make" and "make
install".

> gc-7.2d compiled from the source from Hans Boehm's web site
> (I don't remember whether I finally configured it with pthreads
> or with windows threads, but I tried both for sure),
> gmp-5.2.1 from the source (I think it is shared version),
> libffi-3.0.13 from the source
> libunistring-0.9.3 from the source
> libiconv-1.14 and a few others bundled with mingw
> Everything built from the source was configured with --prefix=/usr

See above: you will therefore need to reconfigure and rebuild those as
well, using the correct Windows absolute directory name as prefix.

> Both / and /usr point to C:\MinGW\

Right, and this is the source of your problems.

> And would it be possible to distribute the binary release somehow?
> (in particular, the libguile.dll + scm/go modules that would be loaded
> from current directory)

Yes, it should be possible.  What I usually do for that install to a
separate empty directory, like this:

  make install prefix=d:/test

(you can use any other empty directory as value of "prefix"), then
create a tarball or a zip file from everything under d:/test.

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

Panicz Maciej Godek <godek.maciek@gmail.com> writes:

> 2013/8/23 Eli Zaretskii <eliz@gnu.org>
>
>     
>     > I managed to go through the compilation process,
>     > and (after modifying meta/Makefile.am according to Eli's
>     > patch) to make install.
>     
>     
>     This compilation process includes compiling all the Scheme files
>     that
>     come with Guile. Since you say it succeeded, I don't understand
>     what
>     you say later, see below. Do you see a lot of *.go files in the
>     directory where you built Guile?
>
> When I enter the build directory and type
> $ find ./ -name '*.go'
> it seems that every .scm file from `modules' directory has its .go
> counterpart,

Note that setting GUILE_LOAD_PATH is not enough.  You must also set
GUILE_LOAD_COMPILED_PATH to include the top directory where the .go
files are installed.  Specifically, both of these should point to a
directory that contains "ice-9", so that
$GUILE_LOAD_PATH/ice-9/boot-9.scm exists, and so does
$GUILE_LOAD_COMPILED_PATH/ice-9/boot-9.go.

A better solution is to provide a proper --prefix, so that the files are
in the default load paths in the built Guile.

Of course, ideally we would allow the entire Guile install to be freely
relocatable, as Windows users have come to expect.  This would require
some help from a Windows developer.

Someone on IRC (who prefers to stay anonymous) once outlined this plan
to me:

  infer paths in the usual way - by saving the hInstance handle that it
  gets in DllMain(), then using GetModuleFileName () on that handle to
  obtain location of the libguile-whatever.dll, then checking if that dll
  is in a "bin" directory, and taking parent directory of that "bin"
  directory as the prefix, and then appending things like "/etc/config.d"
  and "/share/guile/stuff" to that prefix

  Well, an easy way to fix this is to separate the fix into two parts. One
  part is about getting the prefix at runtime (on POSIX you can just use a
  mockup that gets it from some envvar; later you'll be able to add a
  W32-only implementation that does what i said earlier), and the other
  makes guile use that prefix to construct paths it needs

If there are no objections to this approach, I'd be willing to work with
a Windows developer to make it happen.

    Regards,
      Mark

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

> From: Mark H Weaver <mhw@netris.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  "guile-user\@gnu.org" <guile-user@gnu.org>
> Date: Fri, 23 Aug 2013 11:13:19 -0400
> 
> Of course, ideally we would allow the entire Guile install to be freely
> relocatable, as Windows users have come to expect.  This would require
> some help from a Windows developer.

Why should this feature be limited to Windows builds of Guile?  Why
not make it work for Posix platforms as well?  Some other projects
already do, they use argv[0] and PATH search to find the place where
the executable is installed, and record relative file names from
there.

Re: guile-2.0 on mingw: the sequel

[Panicz Maciej Godek]

[-- Attachment #1: Type: text/plain, Size: 824 bytes --]

2013/8/23 Eli Zaretskii <eliz@gnu.org>

> Configuring with --prefix=/usr is a mistake: there's no /usr on your
> system, as far as MinGW programs are concerned.
> [...]
>

yes, it seems that this was the reason. So now the guile
interpreter runs properly, but I still have a problem with
running a program that, having been linked against
libguile, would call scm_with_guile or scm_init_guile, e.g.
===
#include <libguile.h>
void hi(void *unused) { scm_puts("hello!\n", scm_current_output_port()); }
int main() {
   scm_with_guile(hi, NULL);
  return 0;
}
===

I compile it under msys in c:/guile-2.0/bin (i.e. prefix/bin) using
$ gcc -o hello.exe hello.c -I ../include/guile/2.0 -lguile-2.0

but I still get the same error as before ("throw without catch
with stack-overflow key")

Any ideas what could be wrong this time?

[-- Attachment #2: Type: text/html, Size: 1449 bytes --]

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

> Date: Fri, 23 Aug 2013 22:14:43 +0200
> From: Panicz Maciej Godek <godek.maciek@gmail.com>
> Cc: "guile-user@gnu.org" <guile-user@gnu.org>
> 
> > Configuring with --prefix=/usr is a mistake: there's no /usr on your
> > system, as far as MinGW programs are concerned.
> > [...]
> >
> 
> yes, it seems that this was the reason. So now the guile
> interpreter runs properly, but I still have a problem with
> running a program that, having been linked against
> libguile, would call scm_with_guile or scm_init_guile, e.g.
> ===
> #include <libguile.h>
> void hi(void *unused) { scm_puts("hello!\n", scm_current_output_port()); }
> int main() {
>    scm_with_guile(hi, NULL);
>   return 0;
> }
> ===

Does this work on other platforms?  (I know almost nothing about
linking C programs with libguile; maybe your program has a bug, or you
are missing some function call necessary for this to work?)

> I compile it under msys in c:/guile-2.0/bin (i.e. prefix/bin) using
> $ gcc -o hello.exe hello.c -I ../include/guile/2.0 -lguile-2.0

There should be no need to compile under MSYS, you can compile from
the Windows cmd window.  I don't think that's the reason for your
problems, but it's worth a try.

Re: guile-2.0 on mingw: the sequel

[Panicz Maciej Godek]

[-- Attachment #1: Type: text/plain, Size: 1521 bytes --]

2013/8/24 Eli Zaretskii <eliz@gnu.org>

> > yes, it seems that this was the reason. So now the guile
> > interpreter runs properly, but I still have a problem with
> > running a program that, having been linked against
> > libguile, would call scm_with_guile or scm_init_guile, e.g.
> > ===
> > #include <libguile.h>
> > void hi(void *unused) { scm_puts("hello!\n", scm_current_output_port());
> }
> > int main() {
> >    scm_with_guile(hi, NULL);
> >   return 0;
> > }
> > ===
>
> Does this work on other platforms?  (I know almost nothing about
> linking C programs with libguile; maybe your program has a bug, or you
> are missing some function call necessary for this to work?)
>
>
It's the simplest example I could come up with.
The same code executes properly (i.e. displays "hello!") on linux.
I don't know whether windows requires any additional steps for that
to work.

> I compile it under msys in c:/guile-2.0/bin (i.e. prefix/bin) using
> > $ gcc -o hello.exe hello.c -I ../include/guile/2.0 -lguile-2.0
>
> There should be no need to compile under MSYS, you can compile from
> the Windows cmd window.  I don't think that's the reason for your
> problems, but it's worth a try.
>

I did, but the result's still the same (i.e. stack overflow exception).
I could try linking it with libguile-1.8 and see if that works, but
I''d be truly grateful if anyone who's had some experience with
libguile-2.0 linkage on mingw could help me (if such person exists :D)

Still, thanks for pushing my work ahead a little :)

[-- Attachment #2: Type: text/html, Size: 2419 bytes --]

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

> Date: Sat, 24 Aug 2013 10:05:13 +0200
> From: Panicz Maciej Godek <godek.maciek@gmail.com>
> Cc: "guile-user@gnu.org" <guile-user@gnu.org>
> 
> 2013/8/24 Eli Zaretskii <eliz@gnu.org>
> 
> > > yes, it seems that this was the reason. So now the guile
> > > interpreter runs properly, but I still have a problem with
> > > running a program that, having been linked against
> > > libguile, would call scm_with_guile or scm_init_guile, e.g.
> > > ===
> > > #include <libguile.h>
> > > void hi(void *unused) { scm_puts("hello!\n", scm_current_output_port());
> > }
> > > int main() {
> > >    scm_with_guile(hi, NULL);
> > >   return 0;
> > > }
> > > ===
> >
> > Does this work on other platforms?  (I know almost nothing about
> > linking C programs with libguile; maybe your program has a bug, or you
> > are missing some function call necessary for this to work?)
> >
> >
> It's the simplest example I could come up with.
> The same code executes properly (i.e. displays "hello!") on linux.

I guess you didn't configure without threads on GNU/Linux, did you?
If not, I suggest to try that, my impression is that Guile without
threads is not used too much on Posix platforms.

Anyway, I looked into this a bit.  I can confirm that the simple
program you mentioned the first time aborts due to "stack overflow"
(the blurb about "application requesting runtime to terminate in an
unusual way" is displayed when the application calls 'abort').
However, I think this is a bogus result of what was supposed to be a
test for stack overflow.  Here are the details:

If I set a breakpoint in 'abort', I see this backtrace:

     #0  0x004c58d0 in abort ()
     #1  0x004257ff in pre_init_throw ()
     #2  0x0045903e in vm_regular_engine ()
     #3  0x0045aca7 in scm_call_with_vm ()
     #4  0x004254df in scm_throw ()
     #5  0x00404f37 in scm_error_scm ()
     #6  0x00404fa7 in scm_error ()
     #7  0x004391ef in scm_report_stack_overflow ()
     #8  0x0045abf1 in scm_c_vm_run ()
     #9  0x0045af17 in scm_load_compiled_with_vm ()
     #10 0x0043c1e0 in scm_primitive_load_path ()
     #11 0x00404af6 in scm_load_startup_files ()
     #12 0x00404e5c in scm_i_init_guile ()
     #13 0x00402344 in scm_i_init_thread_for_guile ()
     #14 0x00402519 in scm_init_guile ()
     #15 0x004013c0 in main () at guile-hello.c:13

The stack check, SCM_CHECK_STACK, called by scm_c_vm_run, tests the
current stack pointer against the stack base and stack limit.  What I
see is that the stack base of the main (and only) thread is zero,
which of course will always fail any such test.

Stepping through the initialization code code in scm_i_init_guile
(specifically, guilify_self_1, where the first thread is recorded,
including its stack base), it turns out that GC_get_stack_base, which
is implemented in libgc, returns zero as the stack base.  The rest is,
as they say, history.  I didn't have a debug build of libgc to step
into its code, so I couldn't see what is going on there, and libgc has
too many different implementations of GC_get_stack_base to guess from
the sources which one is actually compiled in, and what goes wrong
there.  Perhaps you could step into GC_get_stack_base that on your
system.

Alternatively, a possible workaround is to disable stack checking,
using scm_debug_options.  I'm quite sure there's nothing wrong with
the stack.

In any case, the Guile that is built _is_ functional: I've built GNU
Make with Guile support using it, and the resulting make.exe passes
all the tests that exercise the Guile interface, which include
invoking the Guile interpreter to evaluate expressions.

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

Eli Zaretskii <eliz@gnu.org> writes:

> I guess you didn't configure without threads on GNU/Linux, did you?
> If not, I suggest to try that, my impression is that Guile without
> threads is not used too much on Posix platforms.

Hydra, a continuous integration system, runs Guile's "make check" with
threads disabled on several POSIX platforms, so there's no need for
Panicz to do this test.

  http://hydra.nixos.org/jobset/gnu/guile-2-0#tabs-status

> Anyway, I looked into this a bit.  I can confirm that the simple
> program you mentioned the first time aborts due to "stack overflow"
[...]
> [...] it turns out that GC_get_stack_base, which
> is implemented in libgc, returns zero as the stack base.

Thanks for looking into this, Eli!

This raises the question: what's the relevant difference between
Panicz's simple 'main' and Guile's 'main' (in libguile/guile.c) that
causes one to (apparently) initialize the stack base properly, where the
other fails?  It would be worthwhile to find out.

    Thanks,
      Mark

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

> From: Mark H Weaver <mhw@netris.org>
> Cc: Panicz Maciej Godek <godek.maciek@gmail.com>,  guile-user@gnu.org
> Date: Sun, 25 Aug 2013 12:59:43 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I guess you didn't configure without threads on GNU/Linux, did you?
> > If not, I suggest to try that, my impression is that Guile without
> > threads is not used too much on Posix platforms.
> 
> Hydra, a continuous integration system, runs Guile's "make check" with
> threads disabled on several POSIX platforms, so there's no need for
> Panicz to do this test.

Sorry, I disagree.  "make check" is not the issue here: Guile passes
it on my machine with flying colors (see my reports back then).  And
yet a simple program shown by Panicz aborts due to "stack overflow".
I was suggesting to configure Guile on GNU/Linux without threads and
run that same program: perhaps it will also fail.

Re: guile-2.0 on mingw: the sequel

[Ludovic Courtès]

Eli Zaretskii <eliz@gnu.org> skribis:

>> From: Mark H Weaver <mhw@netris.org>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  "guile-user\@gnu.org" <guile-user@gnu.org>
>> Date: Fri, 23 Aug 2013 11:13:19 -0400
>> 
>> Of course, ideally we would allow the entire Guile install to be freely
>> relocatable, as Windows users have come to expect.  This would require
>> some help from a Windows developer.
>
> Why should this feature be limited to Windows builds of Guile?  Why
> not make it work for Posix platforms as well?  Some other projects
> already do, they use argv[0] and PATH search to find the place where
> the executable is installed, and record relative file names from
> there.

This is not completely reliable when using argv[0] because argv[0] can
be anything, though this can be work around on Linux-based systems with
/proc/self/exe as done in
<http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/patches/guile-relocatable.patch>.

Here’s a proposal: add a --enable-relocatable configure option, have it
default to yes on MinGW and Cygwin (assuming there’s a method that works
reliably on these), and default to no elsewhere; alternately, have it
default to no everywhere.

WDYT?

Thanks,
Ludo’.

Re: guile-2.0 on mingw: the sequel

[Ludovic Courtès]

Hi Eli,

Eli Zaretskii <eliz@gnu.org> skribis:

>> From: Mark H Weaver <mhw@netris.org>
>> Cc: Panicz Maciej Godek <godek.maciek@gmail.com>,  guile-user@gnu.org
>> Date: Sun, 25 Aug 2013 12:59:43 -0400
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > I guess you didn't configure without threads on GNU/Linux, did you?
>> > If not, I suggest to try that, my impression is that Guile without
>> > threads is not used too much on Posix platforms.
>> 
>> Hydra, a continuous integration system, runs Guile's "make check" with
>> threads disabled on several POSIX platforms, so there's no need for
>> Panicz to do this test.
>
> Sorry, I disagree.  "make check" is not the issue here: Guile passes
> it on my machine with flying colors (see my reports back then).  And
> yet a simple program shown by Panicz aborts due to "stack overflow".
> I was suggesting to configure Guile on GNU/Linux without threads and
> run that same program: perhaps it will also fail.

In test-suite/standalone, there are several tests that look similar (or
more complex) than this test program–e.g., test-scm-with-guile.c.

Do you think there’s any relevant difference here?

Thanks,
Ludo’.

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Mark H Weaver <mhw@netris.org>
>> Cc: Panicz Maciej Godek <godek.maciek@gmail.com>,  guile-user@gnu.org
>> Date: Sun, 25 Aug 2013 12:59:43 -0400
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > I guess you didn't configure without threads on GNU/Linux, did you?
>> > If not, I suggest to try that, my impression is that Guile without
>> > threads is not used too much on Posix platforms.
>> 
>> Hydra, a continuous integration system, runs Guile's "make check" with
>> threads disabled on several POSIX platforms, so there's no need for
>> Panicz to do this test.
>
> Sorry, I disagree.  "make check" is not the issue here: Guile passes
> it on my machine with flying colors (see my reports back then).  And
> yet a simple program shown by Panicz aborts due to "stack overflow".
> I was suggesting to configure Guile on GNU/Linux without threads and
> run that same program: perhaps it will also fail.

Okay, I stand corrected.

     Thanks,
       Mark

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Mark H Weaver <mhw@netris.org>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  "guile-user\@gnu.org" <guile-user@gnu.org>
>> Date: Fri, 23 Aug 2013 11:13:19 -0400
>> 
>> Of course, ideally we would allow the entire Guile install to be freely
>> relocatable, as Windows users have come to expect.  This would require
>> some help from a Windows developer.
>
> Why should this feature be limited to Windows builds of Guile?  Why
> not make it work for Posix platforms as well?  Some other projects
> already do, they use argv[0] and PATH search to find the place where
> the executable is installed, and record relative file names from
> there.

Well, that's not generally how things are done on POSIX systems, and
furthermore I don't see how this could be made to work reliably on
POSIX.  The location of the Guile executable is sometimes not found in
either argv[0] or PATH.  For example, if you run the following script:

--8<---------------cut here---------------start------------->8---
#!/usr/local/bin/guile -s
!#

(format #t "~s~%" (program-arguments))
--8<---------------cut here---------------end--------------->8---

argv[0] will point to the script, not the Guile executable.  I suppose
we could detect if argv[0] points to a script and read the shebang (and
iterate, if the shebang points to another script), but even that doesn't
cover all cases.  It's possible to pass any arbitrary value for argv[0]
using the 'exec' family of system calls.

    Regards,
      Mark

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

Mark H Weaver <mhw@netris.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: Mark H Weaver <mhw@netris.org>
>>> Cc: Eli Zaretskii <eliz@gnu.org>,  "guile-user\@gnu.org" <guile-user@gnu.org>
>>> Date: Fri, 23 Aug 2013 11:13:19 -0400
>>> 
>>> Of course, ideally we would allow the entire Guile install to be freely
>>> relocatable, as Windows users have come to expect.  This would require
>>> some help from a Windows developer.
>>
>> Why should this feature be limited to Windows builds of Guile?  Why
>> not make it work for Posix platforms as well?  Some other projects
>> already do, they use argv[0] and PATH search to find the place where
>> the executable is installed, and record relative file names from
>> there.
>
> Well, that's not generally how things are done on POSIX systems, and
> furthermore I don't see how this could be made to work reliably on
> POSIX.  The location of the Guile executable is sometimes not found in
> either argv[0] or PATH.  For example, if you run the following script:
>
> #!/usr/local/bin/guile -s
> !#
>
> (format #t "~s~%" (program-arguments))
>
> argv[0] will point to the script, not the Guile executable.  I suppose
> we could detect if argv[0] points to a script and read the shebang (and
> iterate, if the shebang points to another script), but even that doesn't
> cover all cases.  It's possible to pass any arbitrary value for argv[0]
> using the 'exec' family of system calls.

I forgot to mention a much more common case that would cause problems.

Remember that Guile is a library, not just an executable.  So argv[0]
could point to any arbitrary executable that's linked with libguile.
There are many different mechanisms available to do the linking,
including static linking, LD_LIBRARY_PATH, rpaths, etc.  Trying to
portably figure out which copy of libguile was used, and where it is
located, is a can of worms, to put it mildly.

Having said that, we should probably come up with a solution for
relocatable builds on MacOS X as well.

    Regards,
      Mark

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

> From: Mark H Weaver <mhw@netris.org>
> Cc: godek.maciek@gmail.com,  guile-user@gnu.org
> Date: Sun, 25 Aug 2013 15:50:34 -0400
> 
> Well, that's not generally how things are done on POSIX systems

As I said, some GNU projects already do that.  GCC and GDB are two
examples.

> furthermore I don't see how this could be made to work reliably on
> POSIX.

It's true that this is not 100% bulletproof on Posix, but it's close.
And when it doesn't work, we didn't lose anything, we are just back to
where we are now, right?

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

> From: Mark H Weaver <mhw@netris.org>
> Cc: godek.maciek@gmail.com,  guile-user@gnu.org
> Date: Sun, 25 Aug 2013 15:56:53 -0400
> 
> Remember that Guile is a library, not just an executable.  So argv[0]
> could point to any arbitrary executable that's linked with libguile.

We can provide an API for passing to the library the root of its
installation.

> Having said that, we should probably come up with a solution for
> relocatable builds on MacOS X as well.

Indeed.

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

> Date: Sun, 25 Aug 2013 23:33:51 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: guile-user@gnu.org
> 
> > From: Mark H Weaver <mhw@netris.org>
> > Cc: godek.maciek@gmail.com,  guile-user@gnu.org
> > Date: Sun, 25 Aug 2013 15:56:53 -0400
> > 
> > Remember that Guile is a library, not just an executable.  So argv[0]
> > could point to any arbitrary executable that's linked with libguile.
> 
> We can provide an API for passing to the library the root of its
> installation.

And btw, how is this different from GCC looking for its libgcc or GDB
looking for its Python scripts?

An executable linked with libguile will either be in /usr/bin or
somesuch, i.e. close to /usr/lib where libguile lives; or it will be
in some random place under the user's home directory, in which case
either libguile is in the default place, or it is near the binary.
The latter case is precisely the additional feature where looking for
the library nearby will be a benefit.

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

ludo@gnu.org (Ludovic Courtès) writes:

> Eli Zaretskii <eliz@gnu.org> skribis:
>
>>> From: Mark H Weaver <mhw@netris.org>
>>> Cc: Eli Zaretskii <eliz@gnu.org>,  "guile-user\@gnu.org" <guile-user@gnu.org>
>>> Date: Fri, 23 Aug 2013 11:13:19 -0400
>>> 
>>> Of course, ideally we would allow the entire Guile install to be freely
>>> relocatable, as Windows users have come to expect.  This would require
>>> some help from a Windows developer.
>>
>> Why should this feature be limited to Windows builds of Guile?  Why
>> not make it work for Posix platforms as well?  Some other projects
>> already do, they use argv[0] and PATH search to find the place where
>> the executable is installed, and record relative file names from
>> there.
>
> This is not completely reliable when using argv[0] because argv[0] can
> be anything, though this can be work around on Linux-based systems with
> /proc/self/exe as done in
> <http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/patches/guile-relocatable.patch>.

This finds the executable, but as I noted in a recent message, that
doesn't reliably lead to finding the location of libguile.  I think
you're assuming that the executable is the standard 'guile' executable,
or at least that it's installed in the same prefix as libguile.

> Here’s a proposal: add a --enable-relocatable configure option, have it
> default to yes on MinGW and Cygwin (assuming there’s a method that works
> reliably on these), and default to no elsewhere; alternately, have it
> default to no everywhere.
>
> WDYT?

In theory, I like this idea.  I just don't see how it can be made to
work reliably on POSIX systems.

    Thanks,
      Mark

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Mark H Weaver <mhw@netris.org>
>> Cc: godek.maciek@gmail.com,  guile-user@gnu.org
>> Date: Sun, 25 Aug 2013 15:56:53 -0400
>> 
>> Remember that Guile is a library, not just an executable.  So argv[0]
>> could point to any arbitrary executable that's linked with libguile.
>
> We can provide an API for passing to the library the root of its
> installation.

I suppose, but that assumes that the main program knows the location of
the libguile installation it's linked to.  How would it know this?  In
the common case on POSIX, the dynamic linker takes care of finding a
suitable copy of libguile, drawing from sources such as /etc/ld.so.conf,
/etc/ld.so.conf.d/*, LD_LIBRARY_PATH, rpaths, etc.  How can the main
program know?

> And btw, how is this different from GCC looking for its libgcc or GDB
> looking for its Python scripts?

GCC and GDB are programs, not libraries.  Finding out the location of
the current executable is a much easier problem than finding out the
install prefix of a particular library.

> An executable linked with libguile will either be in /usr/bin or
> somesuch, i.e. close to /usr/lib where libguile lives; or it will be
> in some random place under the user's home directory, in which case
> either libguile is in the default place, or it is near the binary.
> The latter case is precisely the additional feature where looking for
> the library nearby will be a benefit.

You're making a lot of very dubious assumptions here.  I'm uncomfortable
advertising a new feature for Guile that is impossible to implement
robustly.  If it has to make assumptions such as "the libguile library
is probably near the binary", it is likely to fail in many cases.

> It's true that this is not 100% bulletproof on Posix, but it's close.

So far, I've not seen a solution that's anywhere near close to being
correct on POSIX.  All I've seen is a bunch of very dubious guesses.

> And when it doesn't work, we didn't lose anything, we are just back to
> where we are now, right?

I disagree.  If we advertise a new feature that cannot work without
making dubious assumptions, then we're making promises we can't keep.
That's a step in the wrong direction, IMO.

    Regards,
      Mark

Re: guile-2.0 on mingw: the sequel

[dsmich]

---- Mark H Weaver <mhw@netris.org> wrote: 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Mark H Weaver <mhw@netris.org>
> >> Cc: godek.maciek@gmail.com,  guile-user@gnu.org
> >> Date: Sun, 25 Aug 2013 15:56:53 -0400
> >> 
> >> Remember that Guile is a library, not just an executable.  So argv[0]
> >> could point to any arbitrary executable that's linked with libguile.
> >
> > We can provide an API for passing to the library the root of its
> > installation.
> 
> I suppose, but that assumes that the main program knows the location of
> the libguile installation it's linked to.  How would it know this?  In
> the common case on POSIX, the dynamic linker takes care of finding a
> suitable copy of libguile, drawing from sources such as /etc/ld.so.conf,
> /etc/ld.so.conf.d/*, LD_LIBRARY_PATH, rpaths, etc.  How can the main
> program know?
> 
> > And btw, how is this different from GCC looking for its libgcc or GDB
> > looking for its Python scripts?
> 
> GCC and GDB are programs, not libraries.  Finding out the location of
> the current executable is a much easier problem than finding out the
> install prefix of a particular library.
> 
> > An executable linked with libguile will either be in /usr/bin or
> > somesuch, i.e. close to /usr/lib where libguile lives; or it will be
> > in some random place under the user's home directory, in which case
> > either libguile is in the default place, or it is near the binary.
> > The latter case is precisely the additional feature where looking for
> > the library nearby will be a benefit.
> 
> You're making a lot of very dubious assumptions here.  I'm uncomfortable
> advertising a new feature for Guile that is impossible to implement
> robustly.  If it has to make assumptions such as "the libguile library
> is probably near the binary", it is likely to fail in many cases.
> 
> > It's true that this is not 100% bulletproof on Posix, but it's close.
> 
> So far, I've not seen a solution that's anywhere near close to being
> correct on POSIX.  All I've seen is a bunch of very dubious guesses.

Linux (well, glibc) has dladdr http://linux.die.net/man/3/dladdr which could
be used I guess.  But that's not POSIX.

On windows, I think you call GetModuleFileName() with the handle that was passed to DllMain().
Is it possible for libguile to do that?

But there are fundamental differences in the way applications/libraries/packages are places on windows and POSIX and POSIX-like systems.

Seems like most windows apps install everything into their own subtree.  The main application is not even on the system PATH!

On a POSIXy system, there are shared places for config files, libraries, documentation, and so on.  It's just different.  I don't see why we would need to be "relocatable" here.

My $.02
  -Dale

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

> From: ludo@gnu.org (Ludovic Courtès)
> Date: Sun, 25 Aug 2013 20:59:03 +0200
> 
> > Why should this feature be limited to Windows builds of Guile?  Why
> > not make it work for Posix platforms as well?  Some other projects
> > already do, they use argv[0] and PATH search to find the place where
> > the executable is installed, and record relative file names from
> > there.
> 
> This is not completely reliable when using argv[0] because argv[0] can
> be anything, though this can be work around on Linux-based systems with
> /proc/self/exe as done in
> <http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/patches/guile-relocatable.patch>.

Indeed.

> Here’s a proposal: add a --enable-relocatable configure option, have it
> default to yes on MinGW and Cygwin (assuming there’s a method that works
> reliably on these), and default to no elsewhere; alternately, have it
> default to no everywhere.
> 
> WDYT?

Sounds good to me, thanks.

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

> From: Mark H Weaver <mhw@netris.org>
> Cc: godek.maciek@gmail.com,  guile-user@gnu.org
> Date: Sun, 25 Aug 2013 17:42:27 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Mark H Weaver <mhw@netris.org>
> >> Cc: godek.maciek@gmail.com,  guile-user@gnu.org
> >> Date: Sun, 25 Aug 2013 15:56:53 -0400
> >> 
> >> Remember that Guile is a library, not just an executable.  So argv[0]
> >> could point to any arbitrary executable that's linked with libguile.
> >
> > We can provide an API for passing to the library the root of its
> > installation.
> 
> I suppose, but that assumes that the main program knows the location of
> the libguile installation it's linked to.  How would it know this?

We are talking about the situation where libguile is _not_ installed
in the usual places.  Why would a program _not_ know where that is?

> > And btw, how is this different from GCC looking for its libgcc or GDB
> > looking for its Python scripts?
> 
> GCC and GDB are programs, not libraries.  Finding out the location of
> the current executable is a much easier problem than finding out the
> install prefix of a particular library.

The issue is how to find the auxiliary files _given_ the location of
the executable, not how to find where the executable itself lives.

> > And when it doesn't work, we didn't lose anything, we are just back to
> > where we are now, right?
> 
> I disagree.  If we advertise a new feature that cannot work without
> making dubious assumptions, then we're making promises we can't keep.
> That's a step in the wrong direction, IMO.

My turn to disagree.

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

> Date: Sun, 25 Aug 2013 19:24:17 -0400
> From:  <dsmich@roadrunner.com>
> Cc: guile-user@gnu.org
> 
> On windows, I think you call GetModuleFileName() with the handle that was passed to DllMain().
> Is it possible for libguile to do that?

I see no reasons why not.

> Seems like most windows apps install everything into their own subtree.  The main application is not even on the system PATH!

That's not true.  On my machine, there's a single Posix-like tree
rooted at d:/usr, with everything below it.

Many (most?) installers will install the shared libraries into a
single system directory (e.g., C:/WINDOWS/system32), unless you
override the installation defaults, something the majority of users
won't do.

> On a POSIXy system, there are shared places for config files, libraries, documentation, and so on.  It's just different.  I don't see why we would need to be "relocatable" here.

One reason to be "relocatable" is to be able to easily run
applications from the build or source tree, while a different version
is installed on the system.

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Sun, 25 Aug 2013 19:24:17 -0400
>> From:  <dsmich@roadrunner.com>
>> Cc: guile-user@gnu.org
>> 
>> On windows, I think you call GetModuleFileName() with the handle that was passed to DllMain().
>> Is it possible for libguile to do that?
>
> I see no reasons why not.

Yes, that much we can agree on.  Since the Windows API provides a
reliable method to tell us the location of the libguile library, we can
certainly use that to find the rest of the Guile installation.

>> On a POSIXy system, there are shared places for config files,
>> libraries, documentation, and so on.  It's just different.  I don't
>> see why we would need to be "relocatable" here.
>
> One reason to be "relocatable" is to be able to easily run
> applications from the build or source tree, while a different version
> is installed on the system.

We already have a solution for this.  Just run "meta/guile" from the
build directory, or more generally "meta/uninstalled-env".

>> From: Mark H Weaver <mhw@netris.org>
>> Cc: godek.maciek@gmail.com,  guile-user@gnu.org
>> Date: Sun, 25 Aug 2013 17:42:27 -0400
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> > We can provide an API for passing to the library the root of its
>> > installation.
>> 
>> I suppose, but that assumes that the main program knows the location of
>> the libguile installation it's linked to.  How would it know this?
>
> We are talking about the situation where libguile is _not_ installed
> in the usual places.  Why would a program _not_ know where that is?

I explained that in the part of my message that you omitted from your
quotation.

I can think of one notable case where the program would know where
libguile is located: when the program bundles its own copy of Guile, and
therefore assumes that libguile and the program are always moved
together as an atomic unit.

This practice is frowned upon on most POSIX systems (and rightfully so),
but I guess it's fairly common on Windows and MacOS X systems.

In cases where libguile is installed independently, and where libguile
has been moved since the program was built, there's no reason why the
program would know where libguile is located.  Every time the program is
run, the dynamic linker searches for libguile in many different places.
The set of places it looks depends on the platform.

>> > And btw, how is this different from GCC looking for its libgcc or GDB
>> > looking for its Python scripts?
>> 
>> GCC and GDB are programs, not libraries.  Finding out the location of
>> the current executable is a much easier problem than finding out the
>> install prefix of a particular library.
>
> The issue is how to find the auxiliary files _given_ the location of
> the executable, not how to find where the executable itself lives.

Yes, and those executables (GCC and GDB) can reasonably assume that
their auxilliary files are always installed in the same prefix as the
executables, since they are part of the same package.

However, libguile and some-program-that-uses-libguile are _not_ part of
the same package.  There's no reason to assume that they'll be installed
in the same prefix, and they could be moved independently of each other.

When the program is compiled, of course it needs to know where libguile
is located, but since we're talking about making these packages
relocatable, there's no guarantee that libguile will be in the same
place as it was when the program was compiled.

>> > And when it doesn't work, we didn't lose anything, we are just back to
>> > where we are now, right?
>> 
>> I disagree.  If we advertise a new feature that cannot work without
>> making dubious assumptions, then we're making promises we can't keep.
>> That's a step in the wrong direction, IMO.
>
> My turn to disagree.

Okay, again: How exactly do you suggest we determine the location of the
libguile installation from an arbitrary executable that links to it?

     Mark

Re: guile-2.0 on mingw: the sequel

[Mark H Weaver]

I wrote:

> I can think of one notable case where the program would know where
> libguile is located: when the program bundles its own copy of Guile, and
> therefore assumes that libguile and the program are always moved
> together as an atomic unit.
>
> This practice is frowned upon on most POSIX systems (and rightfully so),
> but I guess it's fairly common on Windows and MacOS X systems.

For what it's worth, I _do_ think that we should support this use case,
and therefore I agree that it would probably be useful to have the API
you suggested, where the program can tell libguile where libguile is
located.

However, the only case where this API could reasonably be used is when
libguile is bundled with the program in question, because that's the
only case where the program would reliably know the location of the
libguile that the dynamic linker found.

    Regards,
      Mark

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

> From: Mark H Weaver <mhw@netris.org>
> Cc: dsmich@roadrunner.com,  guile-user@gnu.org
> Date: Mon, 26 Aug 2013 01:56:19 -0400
> 
> Okay, again: How exactly do you suggest we determine the location of the
> libguile installation from an arbitrary executable that links to it?

Like this:

  Step 1: provide an API for the executable to tell that; failing that
  Step 2: look in ../share relative to the executable's argv[0]; finally
  Step 3: fall back on the default recorded at build time

Step 2 will have a Windows wrinkle, whereby we can look in ..share
relative to where Guile's shared library lives.

Re: guile-2.0 on mingw: the sequel

[Eli Zaretskii]

> From: Mark H Weaver <mhw@netris.org>
> Cc: Panicz Maciej Godek <godek.maciek@gmail.com>,  guile-user@gnu.org
> Date: Sun, 25 Aug 2013 12:59:43 -0400
> 
> > Anyway, I looked into this a bit.  I can confirm that the simple
> > program you mentioned the first time aborts due to "stack overflow"
> [...]
> > [...] it turns out that GC_get_stack_base, which
> > is implemented in libgc, returns zero as the stack base.
> 
> Thanks for looking into this, Eli!
> 
> This raises the question: what's the relevant difference between
> Panicz's simple 'main' and Guile's 'main' (in libguile/guile.c) that
> causes one to (apparently) initialize the stack base properly, where the
> other fails?  It would be worthwhile to find out.

Thanks for the suggestion, here's some follow-up:

Comparison between libguile/guile.exe and the test program shows that
the former goes through a different initialization process, calling
libgc as part of it.  Here's the call stack from guilify_self_1, when
called from libguile/guile.exe:

     Breakpoint 1, guilify_self_1 (base=base@entry=0x28fe8c) at threads.c:533
     533     {
     (gdb) n
     541       t.pthread = scm_i_pthread_self ();
     (gdb) n
     542       t.handle = SCM_BOOL_F;
     (gdb) p t.pthread
     $1 = 0
     (gdb) bt
     #0  guilify_self_1 (base=base@entry=0x28fe8c) at threads.c:542
     #1  0x004330b7 in scm_threads_prehistory (base=base@entry=0x28fe8c)
	 at threads.c:2176
     #2  0x00412c2d in scm_i_init_guile (base=base@entry=0x28fe8c) at init.c:386
     #3  0x0043249c in scm_i_init_thread_for_guile (base=0x28fe8c,
	 parent=<optimized out>) at threads.c:835
     #4  scm_i_init_thread_for_guile (base=0x28fe8c, parent=<optimized out>)
	 at threads.c:814
     #5  0x004324c4 in with_guile_and_parent (base=0x28fe8c, data=0x28feb4)
	 at threads.c:901
     #6  0x709cae6f in ?? () from D:\usr\bin\libgc-1.dll
     #7  0x004326cc in scm_i_with_guile_and_parent (parent=<optimized out>,
	 data=0x28fee0, data@entry=0x28feb0,
	 func=func@entry=0x412abc <invoke_main_func>) at threads.c:951
     #8  scm_with_guile (func=func@entry=0x412abc <invoke_main_func>,
	 data=data@entry=0x28fee0) at threads.c:957
     #9  0x00412beb in scm_boot_guile (argc=argc@entry=1,
	 argv=argv@entry=0x28c6970,
	 main_func=main_func@entry=0x4013d4 <inner_main>,
	 closure=closure@entry=0x0) at init.c:320
     #10 0x004c5eab in main (argc=1, argv=0x28c6970) at guile.c:108
     (gdb) p t.base
     $2 = (SCM_STACKITEM *) 0x28fe8c

while the latter does not go through libgc:

     Breakpoint 1, guilify_self_1 (base=base@entry=0x2deff0c) at threads.c:533
     533     {
     (gdb) bt
     #0  guilify_self_1 (base=base@entry=0x2deff0c) at threads.c:533
     #1  0x00402f5f in scm_threads_prehistory (base=base@entry=0x2deff0c)
	 at threads.c:2176
     #2  0x00404b89 in scm_i_init_guile (base=base@entry=0x2deff0c) at init.c:386
     #3  0x00402344 in scm_i_init_thread_for_guile (base=0x2deff0c,
	 parent=<optimized out>) at threads.c:835
     #4  scm_i_init_thread_for_guile (base=0x2deff0c, parent=<optimized out>)
	 at threads.c:814
     #5  0x00402519 in scm_init_guile () at threads.c:869
     #6  0x004013c0 in main () at guile-hello.c:13
     (gdb) p base->mem_base
     $1 = (void *) 0x0

In the former case, scm_i_with_guile_and_parent does this:

     static void *
     scm_i_with_guile_and_parent (void *(*func)(void *), void *data, SCM parent)
     {
       struct with_guile_args args;

       args.func = func;
       args.data = data;
       args.parent = parent;

       return GC_call_with_stack_base (with_guile_and_parent, &args);
     }        ^^^^^^^^^^^^^^^^^^^^^^^

and the call to GC_call_with_stack_base correctly initializes the
stack base:

     GC_API void * GC_CALL GC_call_with_stack_base(GC_stack_base_func fn, void *arg)
     {
	 struct GC_stack_base base;
	 void *result;

	 base.mem_base = (void *)&base; <<<<<<<<<<<<<<<<<<<<<<<<<<

I have verified in the debugger that base.mem_base gets a good value
here.

By contrast, in the test program, GC_call_with_stack_base is never
called.  Instead, GC_get_stack_base is called, which returns zero.
The reason for that seems to be that GC_get_stack_base is called
_before_ GC_init:

     (gdb) break GC_get_stack_base
     Breakpoint 1 at 0x4bbed8
     (gdb) break GC_init
     Breakpoint 2 at 0x4bbfa8
     (gdb) r
     Starting program: D:\usr\eli\utils\guile-2.0.9\guile-hello.exe
     [New Thread 11200.0x39ac]

     Breakpoint 1, 0x004bbed8 in GC_get_stack_base ()
     (gdb) bt
     #0  0x004bbed8 in GC_get_stack_base ()
     #1  0x00402508 in scm_init_guile () at threads.c:868
     #2  0x004013c0 in main () at guile-hello.c:13
     (gdb) c
     Continuing.

     Breakpoint 2, 0x004bbfa8 in GC_init ()
     (gdb) bt
     #0  0x004bbfa8 in GC_init ()
     #1  0x0043d53b in scm_storage_prehistory () at gc.c:653
     #2  0x00404b81 in scm_i_init_guile (base=base@entry=0x2deff0c) at init.c:385
     #3  0x00402344 in scm_i_init_thread_for_guile (base=0x2deff0c,
	 parent=<optimized out>) at threads.c:835
     #4  scm_i_init_thread_for_guile (base=0x2deff0c, parent=<optimized out>)
	 at threads.c:814
     #5  0x00402519 in scm_init_guile () at threads.c:869
     #6  0x004013c0 in main () at guile-hello.c:13
     (gdb)

(Stepping through GC_init shows that GC_setpagesize is called and
returns the correct value: 0x1000.  But it is called too late.)

This happens because scm_init_guile calls GC_get_stack_base without
verifying that libgc was initialized.

Changing the test program to call GC_init at the beginning, like this:

     #include <stdio.h>
     #include <guile/2.0/libguile.h>
     #include <gc/gc.h>

     int
     main (void)
     {
       GC_init ();
       scm_init_guile ();
       return 0;
     }

passes the stack overflow test, but crashes further down the Guile
initialization path:

     Program received signal SIGSEGV, Segmentation fault.
     0x0042c66d in symbol_lookup_assoc_fn (obj=0x2f5870, alist=0x1e4d50,
	 closure=0x0) at symbols.c:176
     176           SCM sym = SCM_CAAR (alist);
     (gdb) bt
     #0  0x0042c66d in symbol_lookup_assoc_fn (obj=0x2f5870, alist=0x1e4d50,
	 closure=0x0) at symbols.c:176
     #1  0x0046a045 in weak_bucket_assoc (table=table@entry=0x2d8fd8,
	 buckets=buckets@entry=0x2d4000, bucket_index=bucket_index@entry=2165,
	 hash_fn=hash_fn@entry=0x42c110 <symbol_lookup_hash_fn>,
	 assoc=assoc@entry=0x42c63c <symbol_lookup_assoc_fn>,
	 object=object@entry=0x2f5870, closure=closure@entry=0x0) at hashtab.c:214
     #2  0x0046a80e in scm_hash_fn_create_handle_x (table=0x2d8fd8, obj=0x2f5870,
	 init=init@entry=0x904,
	 hash_fn=hash_fn@entry=0x42c110 <symbol_lookup_hash_fn>,
	 assoc_fn=assoc_fn@entry=0x42c63c <symbol_lookup_assoc_fn>,
	 closure=closure@entry=0x0) at hashtab.c:698
     #3  0x0042c207 in intern_symbol (symbol=<optimized out>) at symbols.c:195
     #4  scm_i_str2symbol (str=0x4dd028 <scm_logbit_p__name_string_raw_cell>)
	 at symbols.c:218
     #5  0x0042c440 in scm_string_to_symbol (
	 string=string@entry=0x4dd028 <scm_logbit_p__name_string_raw_cell>)
	 at symbols.c:323
     #6  0x0041d0b5 in scm_init_numbers () at ../libguile/numbers.x:48
     #7  0x00404cb9 in scm_i_init_guile (base=base@entry=0x2deff0c) at init.c:453
     #8  0x00402348 in scm_i_init_thread_for_guile (base=0x2deff0c,
	 parent=<optimized out>) at threads.c:835
     #9  scm_i_init_thread_for_guile (base=0x2deff0c, parent=<optimized out>)
	 at threads.c:814
     #10 0x0040251d in scm_init_guile () at threads.c:869
     #11 0x004013c5 in main () at guile-hello.c:15
     (gdb)

Any further ideas?

Re: guile-2.0 on mingw: the sequel

[Panicz Maciej Godek]

[-- Attachment #1: Type: text/plain, Size: 1757 bytes --]

Hi,
firstly -- thanks a lot for the efforts for straightening the matters.
I'm having a busy time right now, but I will try to apply your
solution ASAP, and so I need you to know how much I appreciate it!


> > > I guess you didn't configure without threads on GNU/Linux, did you?
> > > If not, I suggest to try that, my impression is that Guile without
> > > threads is not used too much on Posix platforms.
> >
> > Hydra, a continuous integration system, runs Guile's "make check" with
> > threads disabled on several POSIX platforms, so there's no need for
> > Panicz to do this test.
>
> Sorry, I disagree.  "make check" is not the issue here: Guile passes
> it on my machine with flying colors (see my reports back then).  And
> yet a simple program shown by Panicz aborts due to "stack overflow".
> I was suggesting to configure Guile on GNU/Linux without threads and
> run that same program: perhaps it will also fail.
>

So I eventually tried that (after having set up a virtual box especially
for that purpose), and everything went smoothly -- the "hello" message
was displayed without any complaints, despite the lack of threads.

In the meantime I've been still trying to build guile+bdw-gc with
thread support, and to spot the exact place which causes the error
message to pop out, but it's terribly slow (all the components of
libguile and guile.exe get rebuilt from scratch, in spite of the fact
that the error occurs at the guile-procedures.texi generation stage),
and I've been just placing some output messages here and there
throughout procedures in libgc/win32_threads.c (I wonder if there's
any way to configure the program, whether it be libgc or guile,
 so that it would print the C backtrace when the popup appears)

Best regards,
M.

[-- Attachment #2: Type: text/html, Size: 2674 bytes --]