Guile 1.8 success on `i386-apple-darwin9.6.0'

Guile 1.8 success on `i386-apple-darwin9.6.0'

[Ludovic Courtès]

Hello,

For the record, I successfully tested a recent `branch_release-1-8'
snapshot on `i386-apple-darwin9.6.0' (Mac OS X "Leopard").  The only
quirk was the linker complaining when creating libguile:

  libtool: link: gcc -dynamiclib  -o .libs/libguile.17.dylib  .libs/libguile_la-alist.o .libs/libguile_la-arbiters.o .libs/libguile_la-async.o .libs/libguile_la-backtrace.o .libs/libguile_la-boolean.o .libs/libguile_la-chars.o .libs/libguile_la-continuations.o .libs/libguile_la-convert.o .libs/libguile_la-debug.o .libs/libguile_la-deprecation.o .libs/libguile_la-deprecated.o .libs/libguile_la-discouraged.o .libs/libguile_la-dynwind.o .libs/libguile_la-environments.o .libs/libguile_la-eq.o .libs/libguile_la-error.o .libs/libguile_la-eval.o .libs/libguile_la-evalext.o .libs/libguile_la-extensions.o .libs/libguile_la-feature.o .libs/libguile_la-fluids.o .libs/libguile_la-fports.o .libs/libguile_la-futures.o .libs/libguile_la-gc.o .libs/libguile_la-gc-mark.o .libs/libguile_la-gc-segment.o .libs/libguile_la-gc-malloc.o .libs/libguile_la-gc-card.o .libs/libguile_la-gc-freelist.o .libs/libguile_la-gc_os_dep.o .libs/libguile_la-gdbint.o .libs/libguile_la-gh_data.o .libs/libguile_la-gh_eval.o .libs/libguile_la-gh_funcs.o .libs/libguile_la-gh_init.o .libs/libguile_la-gh_io.o .libs/libguile_la-gh_list.o .libs/libguile_la-gh_predicates.o .libs/libguile_la-goops.o .libs/libguile_la-gsubr.o .libs/libguile_la-guardians.o .libs/libguile_la-hash.o .libs/libguile_la-hashtab.o .libs/libguile_la-hooks.o .libs/libguile_la-i18n.o .libs/libguile_la-init.o .libs/libguile_la-inline.o .libs/libguile_la-ioext.o .libs/libguile_la-keywords.o .libs/libguile_la-lang.o .libs/libguile_la-list.o .libs/libguile_la-load.o .libs/libguile_la-macros.o .libs/libguile_la-mallocs.o .libs/libguile_la-modules.o .libs/libguile_la-numbers.o .libs/libguile_la-objects.o .libs/libguile_la-objprop.o .libs/libguile_la-options.o .libs/libguile_la-pairs.o .libs/libguile_la-ports.o .libs/libguile_la-print.o .libs/libguile_la-procprop.o .libs/libguile_la-procs.o .libs/libguile_la-properties.o .libs/libguile_la-random.o .libs/libguile_la-rdelim.o .libs/libguile_la-read.o .libs/libguile_la-root.o .libs/libguile_la-rw.o .libs/libguile_la-scmsigs.o .libs/libguile_la-script.o .libs/libguile_la-simpos.o .libs/libguile_la-smob.o .libs/libguile_la-sort.o .libs/libguile_la-srcprop.o .libs/libguile_la-stackchk.o .libs/libguile_la-stacks.o .libs/libguile_la-stime.o .libs/libguile_la-strings.o .libs/libguile_la-srfi-4.o .libs/libguile_la-srfi-13.o .libs/libguile_la-srfi-14.o .libs/libguile_la-strorder.o .libs/libguile_la-strports.o .libs/libguile_la-struct.o .libs/libguile_la-symbols.o .libs/libguile_la-threads.o .libs/libguile_la-null-threads.o .libs/libguile_la-throw.o .libs/libguile_la-values.o .libs/libguile_la-variable.o .libs/libguile_la-vectors.o .libs/libguile_la-version.o .libs/libguile_la-vports.o .libs/libguile_la-weaks.o .libs/libguile_la-ramap.o .libs/libguile_la-unif.o .libs/dynl.o .libs/filesys.o .libs/posix.o .libs/net_db.o .libs/socket.o .libs/regex-posix.o   -lgmp -lm -lltdl    -install_name  /var/root/soft/lib/libguile.17.dylib -compatibility_version 21 -current_version 21.0 -Wl,-single_module
  ld warning: codegen in ___gmpn_popcount (offset 0x00000007) prevents image from loading in dyld shared cache
  ld warning: codegen in ___gmpn_popcount (offset 0x0000000E) prevents image from loading in dyld shared cache
  ld warning: codegen in ___gmpn_popcount (offset 0x00000015) prevents image from loading in dyld shared cache
  ld warning: codegen in ___gmpn_hamdist (offset 0x00000007) prevents image from loading in dyld shared cache
  ld warning: codegen in ___gmpn_hamdist (offset 0x0000000E) prevents image from loading in dyld shared cache
  ld warning: codegen in ___gmpn_hamdist (offset 0x00000015) prevents image from loading in dyld shared cache
  ld: absolute addressing (perhaps -mdynamic-no-pic) used in ___gmpn_add_n from /usr/local/lib/libgmp.a(add_n.o) not allowed in slidable image. Use '-read_only_relocs suppress' to enable text relocs
  collect2: ld returned 1 exit status

This was easily solved by following ld's advice:

  make LDFLAGS='-Wl,-read_only_relocs -Wl,suppress'

The compiler was Apple's GCC:

  # gcc --version
  i686-apple-darwin9-gcc-4.0.1 (GCC) 4.0.1 (Apple Inc. build 5488)
  Copyright (C) 2005 Free Software Foundation, Inc.

Thanks,
Ludo'.

Re: Guile 1.8 success on `i386-apple-darwin9.6.0'

[Ludovic Courtès]

The good news is that `master' also builds and tests fine on this
platform with these two patches:

  http://git.savannah.gnu.org/cgit/guile.git/commit/?id=6cc323e2ff4e555d58e115032016a50ef15a1948

  http://git.savannah.gnu.org/cgit/guile.git/commit/?id=7ca96180f00800414a9cf855e5ca4dceb9baca07
    (The calibrated stack limit on this machine is 45771, i.e., slightly
     more than on GNU/Linux.)

However, this was with `--disable-error-on-warning' because of a problem
with GCC's visibility attribute:

  ../../libguile/async.c: In function 'scm_i_setup_sleep':
  ../../libguile/async.c:277: warning: internal and protected visibility attributes not supported in this configuration; ignored

We can't use Gnulib's `visiblity' module to fix that because the
attribute appears in public headers, which are potentially processed by
compilers other than the one that built Guile.

One possibility would be to move internal things in internal headers
that are not installed, but it's annoying.  Some "#ifdef" magic would be
best, but I don't know of any such tricks.  Ideas?

Thanks,
Ludo'.

Re: Guile 1.8 success on `i386-apple-darwin9.6.0'

[Neil Jerram]

ludo@gnu.org (Ludovic Courtès) writes:

> The good news is that `master' also builds and tests fine on this
> platform with these two patches:

Indeed.  Following the fixes that we did for MacOS earlier in the
1.8.x series, it's good to know that something else hasn't regressed.

>   http://git.savannah.gnu.org/cgit/guile.git/commit/?id=6cc323e2ff4e555d58e115032016a50ef15a1948
>
>   http://git.savannah.gnu.org/cgit/guile.git/commit/?id=7ca96180f00800414a9cf855e5ca4dceb9baca07

I'm not sure about moving stack-limit-calibration.scm from TESTS to
BUILT_SOURCES.  The point of putting it in TESTS was to help with
cross-compiling.  When cross-compiling, my understanding is that
`make' should be run in a build host environment, and `make check' in
a target host environment.  stack-limit-calibration.scm should be
calculated in the target host environment, so it makes better sense to
do it as part of `make check' than as part of `make'.

>     (The calibrated stack limit on this machine is 45771, i.e., slightly
>      more than on GNU/Linux.)

Isn't that 2.5 times more?  (i.e. not "slightly" :-))  I believe the
GNU/Linux limit is 20000.

> However, this was with `--disable-error-on-warning' because of a problem
> with GCC's visibility attribute:
>
>   ../../libguile/async.c: In function 'scm_i_setup_sleep':
>   ../../libguile/async.c:277: warning: internal and protected visibility attributes not supported in this configuration; ignored
>
> We can't use Gnulib's `visiblity' module to fix that because the
> attribute appears in public headers, which are potentially processed by
> compilers other than the one that built Guile.
>
> One possibility would be to move internal things in internal headers
> that are not installed, but it's annoying.  Some "#ifdef" magic would be
> best, but I don't know of any such tricks.  Ideas?

Moving internal things into non-installed headers feels like the best
thing to me.

Regards,
        Neil

Re: Guile 1.8 success on `i386-apple-darwin9.6.0'

[Ludovic Courtès]

Hi Neil,

Neil Jerram <neil@ossau.uklinux.net> writes:

> I'm not sure about moving stack-limit-calibration.scm from TESTS to
> BUILT_SOURCES.  The point of putting it in TESTS was to help with
> cross-compiling.  When cross-compiling, my understanding is that
> `make' should be run in a build host environment, and `make check' in
> a target host environment.  stack-limit-calibration.scm should be
> calculated in the target host environment, so it makes better sense to
> do it as part of `make check' than as part of `make'.

Hmm, right.  OTOH, my impression was that tools like Scratchbox had
taken the world over, meaning that cross-compilation usually takes place
as "native" compilation in an emulated target environment.

The thing is that `stack-limit-calibration.scm' is really needed in
cases like this.  Maybe we could detect in `configure' whether we are
cross-compiling and conditionalize build of
`stack-limit-calibration.scm' on that?

>>     (The calibrated stack limit on this machine is 45771, i.e., slightly
>>      more than on GNU/Linux.)
>
> Isn't that 2.5 times more?  (i.e. not "slightly" :-))  I believe the
> GNU/Linux limit is 20000.

It's been 40000 since commit 32c8ae20.

> Moving internal things into non-installed headers feels like the best
> thing to me.

There are internal declarations in essentially every `.h' (88 files).
Presumably we'd have to put them in a single `.h' rather than creating
88 new headers.  The drawback is that declarations would thus live in a
header whose name does not match the name of files where the definitions
occur.

Thanks,
Ludo'.

Re: Guile 1.8 success on `i386-apple-darwin9.6.0'

[Neil Jerram]

ludo@gnu.org (Ludovic Courtès) writes:

> Hi Neil,

Hi again!

> Neil Jerram <neil@ossau.uklinux.net> writes:
>
>> I'm not sure about moving stack-limit-calibration.scm from TESTS to
>> BUILT_SOURCES.  The point of putting it in TESTS was to help with
>> cross-compiling.  When cross-compiling, my understanding is that
>> `make' should be run in a build host environment, and `make check' in
>> a target host environment.  stack-limit-calibration.scm should be
>> calculated in the target host environment, so it makes better sense to
>> do it as part of `make check' than as part of `make'.
>
> Hmm, right.  OTOH, my impression was that tools like Scratchbox had
> taken the world over, meaning that cross-compilation usually takes place
> as "native" compilation in an emulated target environment.

Ah, OK.  If you feel fairly confident about that, I'm happy to defer
to you on this; I don't really understand cross-compiling that well,
but I think what you say is consistent with the scratchbox builds that
I've done.

> The thing is that `stack-limit-calibration.scm' is really needed in
> cases like this.

That is pretty compelling!

> Maybe we could detect in `configure' whether we are
> cross-compiling and conditionalize build of
> `stack-limit-calibration.scm' on that?

No, that sounds too tricky.  I'm persuaded now that your change is
good.

Just one nit: I think there's now only 1 piece of Automake magic being
relied on, so you could update that text (in Makefile.am) and remove
the "2. ".

>>>     (The calibrated stack limit on this machine is 45771, i.e., slightly
>>>      more than on GNU/Linux.)
>>
>> Isn't that 2.5 times more?  (i.e. not "slightly" :-))  I believe the
>> GNU/Linux limit is 20000.
>
> It's been 40000 since commit 32c8ae20.

Ah, OK, didn't notice that; thanks.

>> Moving internal things into non-installed headers feels like the best
>> thing to me.
>
> There are internal declarations in essentially every `.h' (88 files).
> Presumably we'd have to put them in a single `.h' rather than creating
> 88 new headers.  The drawback is that declarations would thus live in a
> header whose name does not match the name of files where the definitions
> occur.

I agree that that wouldn't be nice.  But how about:

- preserving all the installed header names as they are now (even
  though we could maybe make a case for only preserving "libguile.h")

- splitting individual headers one-by-one, only when we have a need to
  do so:

  - public header name xyz.h (unchanged)

  - private header name _xyz.h

  - the private header #includes the public header

  - in libguile/*.c, globally replace "xyz.h" with "_xyz.h". 

That feels nice enough to me.  What do you think?

Regards,
        Neil

Re: Guile 1.8 success on `i386-apple-darwin9.6.0'

[Ludovic Courtès]

Good morning!

Neil Jerram <neil@ossau.uklinux.net> writes:

> ludo@gnu.org (Ludovic Courtès) writes:

>> Maybe we could detect in `configure' whether we are
>> cross-compiling and conditionalize build of
>> `stack-limit-calibration.scm' on that?
>
> No, that sounds too tricky.  I'm persuaded now that your change is
> good.

Alright.

> Just one nit: I think there's now only 1 piece of Automake magic being
> relied on, so you could update that text (in Makefile.am) and remove
> the "2. ".

Right, I did this:

  http://git.savannah.gnu.org/cgit/guile.git/commit/?id=0fe95f9c4ce063781e79a15bc123c57c33ef9755

>> There are internal declarations in essentially every `.h' (88 files).
>> Presumably we'd have to put them in a single `.h' rather than creating
>> 88 new headers.  The drawback is that declarations would thus live in a
>> header whose name does not match the name of files where the definitions
>> occur.
>
> I agree that that wouldn't be nice.  But how about:
>
> - preserving all the installed header names as they are now (even
>   though we could maybe make a case for only preserving "libguile.h")
>
> - splitting individual headers one-by-one, only when we have a need to
>   do so:
>
>   - public header name xyz.h (unchanged)
>
>   - private header name _xyz.h

So IIUC you're advocating the creation of 88 new header files, right?
To me it looks like it would way too much overhead, especially since
each private header would contain few declarations.

>   - the private header #includes the public header
>
>   - in libguile/*.c, globally replace "xyz.h" with "_xyz.h". 
>
> That feels nice enough to me.  What do you think?

I think I'd prefer the single-private-header option, but I'm not 100%
convinced either.

Actually there's yet another option: enclose internal declarations in
"#ifdef LIBGUILE_IN_LIBGUILE" or similar, which we only define when
compiling Guile itself.  This is what Glibc does with, e.g.,
`__LIBC_INTERNAL_MATH_INLINES' and what GMP does with
`__GMP_WITHIN_GMP'.  I think I like it better.

Thanks,
Ludo'.

Re: Guile 1.8 success on `i386-apple-darwin9.6.0'

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 797 bytes --]


ludo@gnu.org (Ludovic Courtès) writes:

> Hmm, right.  OTOH, my impression was that tools like Scratchbox had
> taken the world over, meaning that cross-compilation usually takes place
> as "native" compilation in an emulated target environment.

I don't think that's true at all.  It could be that for running Linux on
arm pdas that's what most people do, but for the far more general case
there is normal cross compiling as autoconf has supported for years.

I am working on a project that does cross builds of a lot of software;
none of it uses scratchbox.

I can certainly see the point of something like scratchbox, to ease the
process and work around software that has non-cross-clean build systems.
But I wouldn't say it's time to give up on the normal/traditional way.


[-- Attachment #2: Type: application/pgp-signature, Size: 193 bytes --]

Re: Guile 1.8 success on `i386-apple-darwin9.6.0'

[Ludovic Courtès]

Hi Greg,

Greg Troxel <gdt@ir.bbn.com> writes:

> I don't think that's true at all.  It could be that for running Linux on
> arm pdas that's what most people do, but for the far more general case
> there is normal cross compiling as autoconf has supported for years.
>
> I am working on a project that does cross builds of a lot of software;
> none of it uses scratchbox.

You may well have more experience than I have.

> I can certainly see the point of something like scratchbox, to ease the
> process and work around software that has non-cross-clean build systems.
> But I wouldn't say it's time to give up on the normal/traditional way.

How would you handle this particular case in a "cross-clean" way?  The
problem is that we need a Guile to compile the compiler.

I think it's not unusual to use just-built binaries to produce some
intermediate source files, especially in the area of interpreters and
compilers.  How do others handle it?

IIRC GCC stage N uses `xgcc' from stage N-1 in the non-cross case.  How
does it work in the cross-compilation case?

Thanks,
Ludo'.

Re: Guile 1.8 success on `i386-apple-darwin9.6.0'

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 3861 bytes --]


ludo@gnu.org (Ludovic Courtès) writes:

> Hi Greg,
>
> Greg Troxel <gdt@ir.bbn.com> writes:
>
>> I don't think that's true at all.  It could be that for running Linux on
>> arm pdas that's what most people do, but for the far more general case
>> there is normal cross compiling as autoconf has supported for years.
>>
>> I am working on a project that does cross builds of a lot of software;
>> none of it uses scratchbox.
>
> You may well have more experience than I have.

Or maybe just different...

I don't really understand the compiler.

>> I can certainly see the point of something like scratchbox, to ease the
>> process and work around software that has non-cross-clean build systems.
>> But I wouldn't say it's time to give up on the normal/traditional way.
>
> How would you handle this particular case in a "cross-clean" way?  The
> problem is that we need a Guile to compile the compiler.

Do we just need a (reasonable) guile, or do we need to run the target
just-built guile itself?  I saw some discussion about finding stack
offsets, and that's perhaps different.

I will use the terms 'host' and 'target' to describe the system one is
doing the build on, and the the system that the resulting guile runs on.
autoconf would call this 'build' and 'host'; 'host' refers to the
machine for which a compiler is built, and target to what the compiler
outputs.  But we aren't really building a compiler in that sense, maybe.

> I think it's not unusual to use just-built binaries to produce some
> intermediate source files, especially in the area of interpreters and
> compilers.  How do others handle it?

Yes, this is normal.  I'll point out that some of my experience comes
From the netbsd build system (to build the OS), and that experience
affects my opinions.  In NetBSD, basically all builds are cross, even if
host==target, in that the host toolchain is used to build the NetBSD
toolchain with the desired target, and then that is used to compile the
system.  One can build for other architectures, and from different host
platforms in the same way.

One of the things that has to be worked out to make a cross system
function is the notion of 'host tool', which is a program that is
compiled for the host.  An example from netbsd is the time zone
compiler, which shows up as nbzic in tools/bin.  There's only one such
binary even if you build for sparc64, i386, and alpha on an amd64 host.
There are three compilers, though; each runs on amd64 and produces
separate output.

> IIRC GCC stage N uses `xgcc' from stage N-1 in the non-cross case.  How
> does it work in the cross-compilation case?

There's a separate bootstrapping problem for compilers, which I think is
about building gcc with host cc, and then building gcc with gcc, so that
you get a gcc-compiled binary in the end.  With cross, I think you have
to build a gcc with target=host and then use that to build gcc with
target=target, but I'm not sure.


So, building guile probably needs either to build guile as a host tool

  if host != target, preferably in an objdir, and then that can be used.

  take a --with-guile that points to a working host guile, and people
  doing cross builds will have to build guile first.  This is not really
  all that differnet from having to build a cross gcc first, and would
  be ok with me (as a cross user - my system doesn't have guile, but if
  it worked that way it would be fine).  This can be the default
  behavior if autoconf's build and host (my host and target) differ.

I suppose the host=target case can use the same guile as the tool and
the output, because the compiler is additional to the interpreter.

If people use scratchbox, then the build is apparently not cross, even
if the gcc that is invoked is cross.  So this shouldn't hurt.

[-- Attachment #2: Type: application/pgp-signature, Size: 193 bytes --]

Re: Guile 1.8 success on `i386-apple-darwin9.6.0'

[Neil Jerram]

ludo@gnu.org (Ludovic Courtès) writes:

> Good morning!

Hello again!

>> Just one nit: I think there's now only 1 piece of Automake magic being
>> relied on, so you could update that text (in Makefile.am) and remove
>> the "2. ".
>
> Right, I did this:
>
>   http://git.savannah.gnu.org/cgit/guile.git/commit/?id=0fe95f9c4ce063781e79a15bc123c57c33ef9755

Thanks, that looks good.

> So IIUC you're advocating the creation of 88 new header files, right?

Potentially, yes. :-)

> I think I'd prefer the single-private-header option, but I'm not 100%
> convinced either.
>
> Actually there's yet another option: enclose internal declarations in
> "#ifdef LIBGUILE_IN_LIBGUILE" or similar, which we only define when
> compiling Guile itself.  This is what Glibc does with, e.g.,
> `__LIBC_INTERNAL_MATH_INLINES' and what GMP does with
> `__GMP_WITHIN_GMP'.  I think I like it better.

That sounds fine to me too - so I guess we should choose this
approach.  Although I would find "LIBGUILE_INTERNAL" more intuitive
than "LIBGUILE_IN_LIBGUILE".

Regards,
        Neil

Dealing with cross-compilation

[Ludovic Courtès]

Hi,

Greg Troxel <gdt@ir.bbn.com> writes:

> So, building guile probably needs either to build guile as a host tool
>
>   if host != target, preferably in an objdir, and then that can be used.
>
>   take a --with-guile that points to a working host guile, and people
>   doing cross builds will have to build guile first.

Right, we could use an already installed Guile 1.8/1.9 when
cross-compiling.  That means we have to make sure the compiler can run
on top of 1.8.  I think this is currently the case.  Andy?

Ludo'.

Re: Dealing with cross-compilation

[Andy Wingo]

Hi,

On Tue 31 Mar 2009 09:10, ludo@gnu.org (Ludovic Courtès) writes:

> Greg Troxel <gdt@ir.bbn.com> writes:
>
>> So, building guile probably needs either to build guile as a host tool
>>
>>   if host != target, preferably in an objdir, and then that can be used.
>>
>>   take a --with-guile that points to a working host guile, and people
>>   doing cross builds will have to build guile first.
>
> Right, we could use an already installed Guile 1.8/1.9 when
> cross-compiling.  That means we have to make sure the compiler can run
> on top of 1.8.  I think this is currently the case.  Andy?

The compiler won't work on 1.8, I don't think. But you could install a
1.9 on the host, and that would work -- modulo some endianness issues
that would need to be sorted out for a host guilec to be able to
cross-compile to the target.

Andy
-- 
http://wingolog.org/