i guess we're frozen & stuff

i guess we're frozen & stuff

[Andy Wingo]

Hello Guilers,

Today's the 10th, we release on the 15th, ergo we're frozen.

What does this mean? Well I think the dilly is that we shouldn't be
pushing to master, except to fix things that don't work at all, and to
update documentation, NEWS, etc.

What if you don't have push access? Well give Guile a try with your
programs, run some benchmarks against 1.9.1, give the tires a good kick.

From my part, I wrote a summary of what I've been up to here:
http://wingolog.org/archives/2009/08/10/goto-

Happy hacking!

Andy

-- 
http://wingolog.org/

Re: i guess we're frozen & stuff

[Mike Gran]

> From: Andy Wingo <wingo@pobox.com>
> 
> Hello Guilers,

Hi Andy, 

> 
> Today's the 10th, we release on the 15th, ergo we're frozen.
> 

Hmmm.  I should have been watching the clock.  I've left master in
an odd state w.r.t. strings and chars.  The storage is available for
non-8-bit strings and chars, but, nothing can be done with them.

> What does this mean? Well I think the dilly is that we shouldn't be
> pushing to master, except to fix things that don't work at all, and to
> update documentation, NEWS, etc.

Do test-suite tests count?

> From my part, I wrote a summary of what I've been up to here:
> http://wingolog.org/archives/2009/08/10/goto-

Nice!

Thanks,

Mike

Re: i guess we're frozen & stuff

[Andy Wingo]

Hi Mike,

On Mon 10 Aug 2009 23:08, Mike Gran <spk121@yahoo.com> writes:

>> Today's the 10th, we release on the 15th, ergo we're frozen.
>
> Hmmm.  I should have been watching the clock.  I've left master in
> an odd state w.r.t. strings and chars.  The storage is available for
> non-8-bit strings and chars, but, nothing can be done with them.

Well, I think a certain amount of disrespect for temporal prescriptions
is healthy. Should you manage to push something tomorrow that makes
things monotonically more awesome, I don't think anyone would object :)

The positive effect of this cyclic rhythm is that it allows us to
coordinate our efforts -- we're managing our expectations. The corollary
being, don't mess up Ludovic's expectations of a great release with a
half-baked push :)

>> What does this mean? Well I think the dilly is that we shouldn't be
>> pushing to master, except to fix things that don't work at all, and to
>> update documentation, NEWS, etc.
>
> Do test-suite tests count?

Tests are welcome!

Cheers,

Andy
-- 
http://wingolog.org/

Re: i guess we're frozen & stuff

[Ludovic Courtès]

Mike Gran <spk121@yahoo.com> writes:

> Hmmm.  I should have been watching the clock.  I've left master in
> an odd state w.r.t. strings and chars.  The storage is available for
> non-8-bit strings and chars, but, nothing can be done with them.

I don't think it's a serious issue.

Just a small comment:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (%string-dump "foo")
$1 = ((string . "foo") (start . 0) (length . 3) (shared . #f) (stringbuf . #<unknown-type (0x30227 . 0x6f6f66) @ 0x7f1c59c99b00>) (stringbuf-chars . "foo") (stringbuf-length . 3) (stringbuf-shared . #f) (stringbuf-inline . #t) (stringbuf-wide . #f))
--8<---------------cut here---------------end--------------->8---

The stringbuf object cannot be printed (as shown above), has no public
type predicate, etc., so I think we'd better not return it to Scheme
code.  Can you change that in `%string-dump'?

Note for later: it'd be great if we had micro-benchmarks to compare the
former and the new string implementations (on Latin-1 strings, that is),
and to compare Latin-1 and wide strings in the new implementation.

Thanks,
Ludo'.

Re: i guess we're frozen & stuff

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 997 bytes --]


  Today's the 10th, we release on the 15th, ergo we're frozen.

  What does this mean? Well I think the dilly is that we shouldn't be
  pushing to master, except to fix things that don't work at all, and to
  update documentation, NEWS, etc.

  What if you don't have push access? Well give Guile a try with your
  programs, run some benchmarks against 1.9.1, give the tires a good kick.

Sorry, I've been distracted.  Is this what we call 'master', and what I
am running autobuilds against, along with the 1.8 branch?  I will try to
find time to install libunistring and get it to build.

Have there been recent reports of success of 1.9.x on platforms other
than GNU/Linux?  Guile has been quite portable in the past and it's
surely still very close if not there, and it would be a shame if 2.0 had
issues.  I realize this is hard for people to test if they don't have
the platform, but it would be really good to have testing on at least
one normal BSD and also Darwin. 


[-- Attachment #2: Type: application/pgp-signature, Size: 193 bytes --]

Re: i guess we're frozen & stuff

[Ken Raeburn]

[-- Attachment #1: Type: text/plain, Size: 3445 bytes --]

On Aug 11, 2009, at 07:34, Greg Troxel wrote:
> Have there been recent reports of success of 1.9.x on platforms other
> than GNU/Linux?  Guile has been quite portable in the past and it's
> surely still very close if not there, and it would be a shame if 2.0  
> had
> issues.  I realize this is hard for people to test if they don't have
> the platform, but it would be really good to have testing on at least
> one normal BSD and also Darwin.


The Mac build off of "master" fails for me currently in srfi-13.c,  
with the comparison-always-false warning Greg discussed.  I hacked  
around that, but then guile-readline doesn't build:

Making all in guile-readline
../libguile/guile-snarf -o readline.x ../../guile-readline/readline.c - 
DHAVE_CONFIG_H  -I. -I.. -I../../guile-readline/.. -I../../guile- 
readline/lib -I./lib  -g -O2
In file included from ../../guile-readline/readline.c:29:
../../guile-readline/../libguile.h:25:17: error: gmp.h: No such file  
or directory
In file included from ../../guile-readline/../libguile.h:95,
                  from ../../guile-readline/readline.c:29:
../../guile-readline/../libguile/strings.h:26:21: error: uniconv.h: No  
such file or directory

Neither the path specified for libgmp nor the path specified for  
libunistring at configure time is included here.

I don't think any of this is Mac-specific; I'm surprised that it works  
on GNU/Linux systems.  Perhaps I'm building it in ways that are  
unusual for the other developers (build dir != src dir, libgmp and  
guile-1.8 installed in the same place, libgmp and libunistring  
installed in different nonstandard directories)?

If I use CPPFLAGS=... and LDFLAGS=... instead of --with-libfoo-prefix  
configure options to specify paths to find libgmp and libunistring,  
the tests still pick old, installed Guile headers (which this time  
I've poisoned to highlight the problem) from those locations instead  
of the in-tree versions:

Making all in test-suite
Making all in standalone
../../libguile/guile-snarf -o test-asmobs-lib.x ../../../test-suite/ 
standalone/test-asmobs-lib.c -DHAVE_CONFIG_H -I. -I../../../test-suite/ 
standalone -I../.. -I/opt/local/include -I/Users/raeburn/dev/guile/ 
libunistring-0.9.1/I/include -g -O2 -I../../.. -I../../../lib -I../../ 
lib -I../..
In file included from /opt/local/include/libguile.h:30,
                  from ../../../test-suite/standalone/test-asmobs- 
lib.c:23:
/opt/local/include/libguile/__scm.h:3:2: error: #error Poison!

I might be building Guile as part of a larger package  
(*cough*Emacs*cough*) that wants to include stuff from the same system  
directories (e.g., for MacPorts, pkgsrc, whatever) where an old  
version of Guile is installed, and thus Guile gets passed CPPFLAGS/ 
LDFLAGS settings that add that old version to the search paths.  So I  
think the CPPFLAGS/LDFLAGS version needs to be made to work, as well  
as the --with-libfoo-prefix version.

With the attached patch, I can get guile to build with CPPFLAGS= and  
LDFLAGS= ... someone more familiar than I am with automake will have  
to fix the guile-readline stuff.  Even with my patch, it fails its  
tests:

make  check-TESTS
Testing /Users/raeburn/dev/guile/git/guile/b3/meta/guile ...
with GUILE_LOAD_PATH=/Users/raeburn/dev/guile/git/guile/test-suite
ERROR: In procedure make_objcode_by_mmap:
ERROR: bad header on object file: "GOOF-0.6-LE-4---"
FAIL: check-guile

Still looking into that....

Ken


[-- Attachment #2: guile-master.diff --]
[-- Type: application/octet-stream, Size: 1043 bytes --]

--- a/test-suite/standalone/Makefile.am
+++ b/test-suite/standalone/Makefile.am
@@ -30,17 +30,20 @@ EXTRA_DIST =
 
 TESTS_ENVIRONMENT = "${top_builddir}/meta/uninstalled-env"
 
+## Check for headers in $(srcdir) and bulid dir before $(CPPFLAGS), which
+## may point us to an old, installed version of guile.
+AM_CPPFLAGS = -I$(top_srcdir) -I$(top_builddir) \
+	      -I$(top_srcdir)/lib -I$(top_builddir)/lib
+
 test_cflags =					\
-  -I$(top_srcdir)/test-suite/standalone		\
-  -I$(top_srcdir) -I$(top_builddir)		\
-  -I$(top_srcdir)/lib -I$(top_builddir)/lib	\
+  -I$(top_srcdir)/test-suite/standalone -I.	\
   $(EXTRA_DEFS) $(GUILE_CFLAGS) $(GCC_CFLAGS)
 
 AM_LDFLAGS = $(GUILE_CFLAGS)
 
-snarfcppopts =								\
-  $(DEFS) $(DEFAULT_INCLUDES) $(CPPFLAGS) $(CFLAGS) -I$(top_srcdir)	\
-  -I$(top_srcdir)/lib -I$(top_builddir)/lib -I$(top_builddir)
+snarfcppopts =								      \
+  -I$(top_srcdir) -I$(top_srcdir)/lib -I$(top_builddir)/lib -I$(top_builddir) \
+  -I. $(DEFS) $(DEFAULT_INCLUDES) $(CPPFLAGS) $(CFLAGS)
 
 SUFFIXES = .x
 .c.x:

[-- Attachment #3: Type: text/plain, Size: 1 bytes --]

Re: i guess we're frozen & stuff

[Ken Raeburn]

On Aug 11, 2009, at 09:59, Ken Raeburn wrote:
> ERROR: In procedure make_objcode_by_mmap:
> ERROR: bad header on object file: "GOOF-0.6-LE-4---"

Ah, that was an old compiled file cached away in $HOME/.cache/ 
guile/... that I needed to delete in order to make the tests pass.

I re-ran the test suite for the 1.9.1 release, and it creates a file  
with "GOOF-0.6" under ~/.cache/guile/ccache/1.9/... whereas the  
current master code creates (and requires?) a file with "GOOF-0.9"  
under the same prefix (and both appending the full pathname).  So if I  
had built and tested guile-1.9.1 and then I update to 1.9.2 in the  
same directory (e.g., stripping off the version number so it can be  
found as "guile" in my project tree), it will fail its tests, if I  
don't know to get rid of this hidden cache file, and its pathname is  
not actually mentioned in the error message.  That's kind of bad.

It appears that the word size and endianness is also encoded into the  
header.  Is this a good idea, when people can share home directories  
across machines of different architectures, and even run mixed-size  
binaries on a single system (or mixed-architecture, in some cases)?

Ken

Re: i guess we're frozen & stuff

[Ludovic Courtès]

Hi,

Ken Raeburn <raeburn@raeburn.org> writes:

> It appears that the word size and endianness is also encoded into the
> header.  Is this a good idea, when people can share home directories
> across machines of different architectures, and even run mixed-size
> binaries on a single system (or mixed-architecture, in some cases)?

Currently `.go' files cannot be shared across heterogeneous
architectures, which is why the endianness and word size are encoded in
`.go' headers.

Thanks,
Ludo'.

Re: i guess we're frozen & stuff

[Ken Raeburn]

On Aug 11, 2009, at 11:36, Ludovic Courtès wrote:
>> It appears that the word size and endianness is also encoded into the
>> header.  Is this a good idea, when people can share home directories
>> across machines of different architectures, and even run mixed-size
>> binaries on a single system (or mixed-architecture, in some cases)?
>
> Currently `.go' files cannot be shared across heterogeneous
> architectures, which is why the endianness and word size are encoded  
> in
> `.go' headers.

That's fine... but shouldn't we maybe use different directories, then,  
so runs on different platforms don't collide?

Ken

Re: i guess we're frozen & stuff

[Andy Wingo]

On Tue 11 Aug 2009 16:45, Ken Raeburn <raeburn@raeburn.org> writes:

> On Aug 11, 2009, at 09:59, Ken Raeburn wrote:
>> ERROR: In procedure make_objcode_by_mmap:
>> ERROR: bad header on object file: "GOOF-0.6-LE-4---"
>
> Ah, that was an old compiled file cached away in $HOME/.cache/
> guile/... that I needed to delete in order to make the tests pass.

This problem has been fixed.

Andy
-- 
http://wingolog.org/

Re: i guess we're frozen & stuff

[Ludovic Courtès]

Ken Raeburn <raeburn@raeburn.org> writes:

> ../../guile-readline/../libguile/strings.h:26:21: error: uniconv.h: No
> such file or directory
>
> Neither the path specified for libgmp nor the path specified for
> libunistring at configure time is included here.

Arrrrg, that's what I feared with the inclusion of libunistring headers
in Guile's public headers.

Mike: can you fix it one way or another?  Solutions are:

  1. Fix `guile.pc' to have "-I${libunistring}", and fix guile-readline
     CPPFLAGS as well.

  2. Or just don't include <uniconv.h> in public headers.  This means
     you have to declare, say, `scm_t_string_failed_conversion_handler'
     as a replacement for `iconv_ilseq_handler'.

I would prefer #2 for reasons explained in another message.

Thanks in advance!

Ludo'.

Re: i guess we're frozen & stuff

[Andy Wingo]

Hi Ken,

On Tue 11 Aug 2009 15:59, Ken Raeburn <raeburn@raeburn.org> writes:

> Perhaps I'm building [Guile] in ways that are unusual for the other
> developers (build dir != src dir, libgmp and guile-1.8 installed in
> the same place, libgmp and libunistring installed in different
> nonstandard directories)?

Yes, I think this is the case. Your bug reports are appreciated.

Andy
-- 
http://wingolog.org/

Re: i guess we're frozen & stuff

[Andy Wingo]

Hi Ken,

 On Tue 11 Aug 2009 15:59, Ken Raeburn <raeburn@raeburn.org> writes:

> Perhaps I'm building [Guile] in ways that are unusual for the other
> developers (build dir != src dir, libgmp and guile-1.8 installed in
> the same place, libgmp and libunistring installed in different
> nonstandard directories)?

I think I fixed the case in which $prefix was not in LD_LIBRARY_PATH or
the like. Can you update us on the status of your build issues?

Andy
-- 
http://wingolog.org/

Re: i guess we're frozen & stuff

[Ken Raeburn]

On Sep 16, 2009, at 15:00, Andy Wingo wrote:
> Hi Ken,
>
> On Tue 11 Aug 2009 15:59, Ken Raeburn <raeburn@raeburn.org> writes:
>
>> Perhaps I'm building [Guile] in ways that are unusual for the other
>> developers (build dir != src dir, libgmp and guile-1.8 installed in
>> the same place, libgmp and libunistring installed in different
>> nonstandard directories)?
>
> I think I fixed the case in which $prefix was not in LD_LIBRARY_PATH  
> or
> the like. Can you update us on the status of your build issues?

Frequently failing. :-(
On the bright side, using the --with-foo-prefix options, I haven't  
seen indications yet of the include- and load-path problems I saw  
before.

Today (f60e892), it can't compile i18n.c at line 799 on my Mac because  
it tries to deference scm_t_locale which points to an incomplete type.

Backing up to bcccf04, it builds okay, but lots of regexp tests fail  
with "illegal byte sequence" errors.  With "make -k check", that's the  
only error reported.

With f60e892, if I disable the defining of USE_GNU_LOCALE_API, I still  
get the regexp test errors, and srfi-18.test fails, but only sometimes:

Running srfi-18.test
WARNING: (srfi srfi-18): imported module (srfi srfi-34) overrides core  
binding `raise'
throw from within critical section.
error key: foo
/bin/sh: line 1: 74711 Abort trap              ${dir}$tst
FAIL: check-guile

I haven't checked if srfi-18.test has intermittent failures in the  
older revision.  It might also matter that I was sometimes piping the  
test output through "more", which could've conceivably blocked process  
output for a while, changing the timing of events between threads.

On GNU/Linux (RHEL 4), I had to download, build and install the GC  
library, and took the stable one linked to from the web page for the  
package.  But that didn't declare GC_dump, so I had to disable  
warnings in the build.  After that, building, installing and testing  
*seemed* to go just fine.

I'm seeing another weird effect though -- sometimes some of the tests  
outputs are getting reported twice.  In another test I just did on my  
Mac, the repetition isn't starting on line boundaries, though, so it  
looks like it's actually the output getting corrupted, vs re-running  
or re-reporting tests for no obvious reason:

Running socket.test
WARNING: (srfi srfi-18): imported module (srfi srfi-34) overrides core  
binding `raise'
WARNING: (test-srfi-69): imported module (srfi srfi-69) overrides core  
binding `make-hash-table'
WARNING: (test-srfi-69): imported module (srfi srfi-69) overrides core  
binding `hash-table?'
exp" "illegal byte sequence" #f ("a<C7>")))
ERROR: regexp.test: regexp-quote: regexp/extended: (string "aX" 200 # 
\<C8> "a<C8>" "a<C8>") - arguments:
  ((regular-expression-syntax "make-regexp" "illegal byte sequence" #f  
("a<C8>")))

I've seen the duplicated output on GNU/Linux too.  In reproducing  
this, again, it seems to help to pipe the output through "more", and  
follow it slowly (i.e., so presumably the pipe will fill and output  
will occasionally be blocked).

Ken

Re: i guess we're frozen & stuff

[Mike Gran]

On Fri, 2009-09-25 at 17:59 -0400, Ken Raeburn wrote:
> Today (f60e892), it can't compile i18n.c at line 799 on my Mac because  
> it tries to deference scm_t_locale which points to an incomplete type.

Yeah.  I figured that would break something.  I'm trying to find a
decent solution for the problem discussed in
http://lists.gnu.org/archive/html/guile-devel/2009-09/msg00167.html and
http://savannah.gnu.org/bugs/?27520

> 
> Backing up to bcccf04, it builds okay, but lots of regexp tests fail  
> with "illegal byte sequence" errors.  With "make -k check", that's the  
> only error reported.

This does surprise me.  I thought I had fixed that problem. Does it pass
if you replace the current test-suite/tests/regexp.test with the 1.8.x
version of the regexp test?

Thanks,

Mike

Re: i guess we're frozen & stuff

[Ken Raeburn]

On Sep 26, 2009, at 11:45, Mike Gran wrote:
>> Backing up to bcccf04, it builds okay, but lots of regexp tests fail
>> with "illegal byte sequence" errors.  With "make -k check", that's  
>> the
>> only error reported.
>
> This does surprise me.  I thought I had fixed that problem. Does it  
> pass
> if you replace the current test-suite/tests/regexp.test with the 1.8.x
> version of the regexp test?

It passes the regexp tests, yes, but the i18n tests complain about  
case conversions in Turkish.

Ken

Re: i guess we're frozen & stuff

[Mike Gran]

> > This does surprise me.  I thought I had fixed that problem. Does it  
> > pass
> > if you replace the current test-suite/tests/regexp.test with the 1.8.x
> > version of the regexp test?

Hmmm.  Maybe I'll just revert regexp.test.  

> 
> It passes the regexp tests, yes, but the i18n tests complain about  
> case conversions in Turkish.

Yeah.  The code that is necessary to make those Turkish tests pass is
that code in the USE_GNU_LOCALE_API section of i18n.c that won't compile
on your Darwin box.  Until I can get a better fix from the upstream
libunistring or hack around the problem in i18n.c, those tests will
fail.

Thanks,

Mike

Re: i guess we're frozen & stuff

[Ludovic Courtès]

Hi Ken,

Ken Raeburn <raeburn@raeburn.org> writes:

> Running srfi-18.test
> WARNING: (srfi srfi-18): imported module (srfi srfi-34) overrides core
> binding `raise'
> throw from within critical section.
> error key: foo
> /bin/sh: line 1: 74711 Abort trap              ${dir}$tst
> FAIL: check-guile

This must be related to http://savannah.gnu.org/bugs/?27457 .
Contributions welcome!  ;-)

> On GNU/Linux (RHEL 4), I had to download, build and install the GC
> library, and took the stable one linked to from the web page for the
> package.  But that didn't declare GC_dump, so I had to disable
> warnings in the build.

Which version is that?  The declaration appears to be present at least
in 7.1 and current CVS.

Thanks,
Ludo’.

Re: i guess we're frozen & stuff

[Ken Raeburn]

On Sep 26, 2009, at 17:02, Ludovic Courtès wrote:
>> Running srfi-18.test
>> WARNING: (srfi srfi-18): imported module (srfi srfi-34) overrides  
>> core
>> binding `raise'
>> throw from within critical section.
>> error key: foo
>> /bin/sh: line 1: 74711 Abort trap              ${dir}$tst
>> FAIL: check-guile
>
> This must be related to http://savannah.gnu.org/bugs/?27457 .
> Contributions welcome!  ;-)

Oh, fun... yeah, I may take a look.

>> On GNU/Linux (RHEL 4), I had to download, build and install the GC
>> library, and took the stable one linked to from the web page for the
>> package.  But that didn't declare GC_dump, so I had to disable
>> warnings in the build.
>
> Which version is that?  The declaration appears to be present at least
> in 7.1 and current CVS.

I got version 6.8.  It had the function, just not the declaration.

Ken

Re: i guess we're frozen & stuff

[Ludovic Courtès]

Hi,

Ken Raeburn <raeburn@raeburn.org> writes:

> I got version 6.8.  It had the function, just not the declaration.

You should use 7.x.  In fact, IIRC, 6.x is not detected by ‘configure’
because it has no ‘bdw-gc.pc’.

Thanks,
Ludo’.

Re: i guess we're frozen & stuff

[Ken Raeburn]

On Sep 27, 2009, at 05:10, Ludovic Courtès wrote:
>> I got version 6.8.  It had the function, just not the declaration.
>
> You should use 7.x.  In fact, IIRC, 6.x is not detected by ‘configure’
> because it has no ‘bdw-gc.pc’.

True, though the error message from the configure script itself  
suggested I could use BDW_GC_* environment variables instead, and  
didn't mention a required minimum version number, so I took the  
"older, more stable version" recommended on the web site for the GC  
library.  Yes, I obviously failed to check Guile's own README file. :-)

If we need 7.x, should we check GC_VERSION_MAJOR>=7 at compile time?

Ken

Re: i guess we're frozen & stuff

[Ludovic Courtès]

Hello!

Ken Raeburn <raeburn@raeburn.org> writes:

> If we need 7.x, should we check GC_VERSION_MAJOR>=7 at compile time?

Actually, there’s infrastructure in <libguile/bdw-gc.h> to support 6.x,
which is why you were able to compile most things.

So 7.x is not /strictly/ required, but I prefer to mark it as required,
because there have been performance improvements and possibly other
useful changes, and I don’t want to maintain the compatibility layer
(7.x is already old!).

Now, I don’t feel like checking GC_VERSION_MAJOR >= 7 for now, because
checking for ‘bdw-gc.pc’ should already do the trick, and those who
forcibly use 6.x do it at their own risk.  :-)

Thanks,
Ludo’.

Re: i guess we're frozen & stuff

[Neil Jerram]

ludo@gnu.org (Ludovic Courtès) writes:

> Hi Ken,
>
> Ken Raeburn <raeburn@raeburn.org> writes:
>
>> Running srfi-18.test
>> WARNING: (srfi srfi-18): imported module (srfi srfi-34) overrides core
>> binding `raise'
>> throw from within critical section.
>> error key: foo
>> /bin/sh: line 1: 74711 Abort trap              ${dir}$tst
>> FAIL: check-guile
>
> This must be related to http://savannah.gnu.org/bugs/?27457 .
> Contributions welcome!  ;-)

I will start looking at this later this evening.  (Unless you're already
investigating - in which case please let me know!)

Regards,
        Neil

Re: i guess we're frozen & stuff

[Ludovic Courtès]

Hi,

Neil Jerram <neil@ossau.uklinux.net> writes:

> ludo@gnu.org (Ludovic Courtès) writes:

>> This must be related to http://savannah.gnu.org/bugs/?27457 .
>> Contributions welcome!  ;-)
>
> I will start looking at this later this evening.  (Unless you're already
> investigating - in which case please let me know!)

I won’t look into it in the next days, so go ahead!  ;-)

(Note that you can assign the bug to yourself in the bug tracker, if you
want.)

Thanks,
Ludo’.

Re: i guess we're frozen & stuff

[Neil Jerram]

ludo@gnu.org (Ludovic Courtès) writes:

> Hi,
>
> Neil Jerram <neil@ossau.uklinux.net> writes:
>
>> ludo@gnu.org (Ludovic Courtès) writes:
>
>>> This must be related to http://savannah.gnu.org/bugs/?27457 .
>>> Contributions welcome!  ;-)
>>
>> I will start looking at this later this evening.  (Unless you're already
>> investigating - in which case please let me know!)
>
> I won’t look into it in the next days, so go ahead!  ;-)

Well...  I don't see the throw from critical section problem.  Instead,
after a while, I see a hang, with one thread doing:

#0  0xb7f06424 in __kernel_vsyscall ()
#1  0xb7acd255 in sem_wait@@GLIBC_2.1 () from /lib/i686/cmov/libpthread.so.0
#2  0xb7dc2018 in GC_stop_world () from /usr/lib/libgc.so.1

and all the others:

#0  0xb7f06424 in __kernel_vsyscall ()
#1  0xb7b05837 in sigsuspend () from /lib/i686/cmov/libc.so.6
#2  0xb7dc222b in GC_suspend_handler_inner () from /usr/lib/libgc.so.1
#3  0xb7dc22b5 in GC_suspend_handler () from /usr/lib/libgc.so.1
#4  <signal handler called>

In theory, each other thread must have called sem_post(), and the number
of those sem_post()s should be the same as the number of times that the
GC_stop_world thread calls sem_wait(), and so the GC_stop_world thread
shouldn't be waiting.

I wonder if there's a way that the pthread_kill(p->id, SIG_SUSPEND) in
GC_stop_world can appear to succeed (by returning 0), but the signalled
thread doesn't get the signal, or dies before it does the sem_post()?

Regarding the throw from critical section problem, I guess I'm not
seeing this because of not running on a multi-core machine.  Can someone
who does see this problem

- run under GDB
- set a breakpoint on the fprintf (stderr, "throw from within critical
  section.\n") line in throw.c
- post the thread backtraces (thread apply all bt), when this breakpoint
  is hit?

Thanks.

        Neil

Bug #27457 (“Threads, mutexes, and critical sections”)

[Ludovic Courtès]

Hi Neil,

Neil Jerram <neil@ossau.uklinux.net> writes:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Hi,
>>
>> Neil Jerram <neil@ossau.uklinux.net> writes:
>>
>>> ludo@gnu.org (Ludovic Courtès) writes:
>>
>>>> This must be related to http://savannah.gnu.org/bugs/?27457 .
>>>> Contributions welcome!  ;-)
>>>
>>> I will start looking at this later this evening.  (Unless you're already
>>> investigating - in which case please let me know!)
>>
>> I won’t look into it in the next days, so go ahead!  ;-)
>
> Well...  I don't see the throw from critical section problem.  Instead,
> after a while, I see a hang, with one thread doing:
>
> #0  0xb7f06424 in __kernel_vsyscall ()
> #1  0xb7acd255 in sem_wait@@GLIBC_2.1 () from /lib/i686/cmov/libpthread.so.0
> #2  0xb7dc2018 in GC_stop_world () from /usr/lib/libgc.so.1
>
> and all the others:
>
> #0  0xb7f06424 in __kernel_vsyscall ()
> #1  0xb7b05837 in sigsuspend () from /lib/i686/cmov/libc.so.6
> #2  0xb7dc222b in GC_suspend_handler_inner () from /usr/lib/libgc.so.1
> #3  0xb7dc22b5 in GC_suspend_handler () from /usr/lib/libgc.so.1
> #4  <signal handler called>
>
> In theory, each other thread must have called sem_post(), and the number
> of those sem_post()s should be the same as the number of times that the
> GC_stop_world thread calls sem_wait(), and so the GC_stop_world thread
> shouldn't be waiting.

I should have mentioned this one:
http://thread.gmane.org/gmane.comp.programming.garbage-collection.boehmgc/3185 .

This is normally somewhat fixed in current BDW-GC CVS, but Guile itself
may have troubles of its own dealing with cancellation.

The critical section issue should be orthogonal, though.

> Regarding the throw from critical section problem, I guess I'm not
> seeing this because of not running on a multi-core machine.  Can someone
> who does see this problem
>
> - run under GDB
> - set a breakpoint on the fprintf (stderr, "throw from within critical
>   section.\n") line in throw.c
> - post the thread backtraces (thread apply all bt), when this breakpoint
>   is hit?

Here it is:

--8<---------------cut here---------------start------------->8---
Breakpoint 1, scm_ithrow (key=0x707060, args=0x404, noreturn=1) at throw.c:746
(gdb) thread apply all bt

Thread 564 (Thread 0x7f90c0d20950 (LWP 18603)):
#0  0x00007f90c6f3077a in times () from /nix/store/s88vdfglm94x7jn0vqm24pqhq460s0c7-glibc-2.9/lib/libc.so.6
#1  0x00007f90c85308cc in scm_c_get_internal_run_time () at stime.c:106
#2  0x00007f90c85439ec in vm_debug_engine (vp=0x7f90c0d1f860, program=0x7f90c0d1f860, argv=0x0, nargs=<value optimized out>) at vm-engine.c:59
#3  0x00007f90c853a751 in scm_c_with_throw_handler (tag=0x8e90c0, body=<value optimized out>, body_data=0x7f90c0d1fa50, handler=<value optimized out>, handler_data=<value optimized out>, lazy_catch_p=<value optimized out>) at throw.c:318
#4  0x00007f90c853a845 in scm_with_throw_handler (key=0x7f90c0d1f860, thunk=<value optimized out>, handler=0xc18da0) at throw.c:665
#5  0x00007f90c8545861 in vm_debug_engine (vp=0x703500, program=<value optimized out>, argv=<value optimized out>, nargs=<value optimized out>) at vm-i-system.c:605
#6  0x00007f90c853abc5 in scm_c_catch (tag=<value optimized out>, body=<value optimized out>, body_data=<value optimized out>, handler=<value optimized out>, handler_data=<value optimized out>, pre_unwind_handler=0, pre_unwind_handler_data=0x7f90c0d1fd60) at throw.c:243
#7  0x00007f90c853ad4e in scm_catch_with_pre_unwind_handler (key=0x7f90c0d1f860, thunk=<value optimized out>, handler=0xa2c900, pre_unwind_handler=0x204) at throw.c:627
#8  0x00007f90c853939b in really_launch (d=0x7fff794861b0) at threads.c:801
#9  0x00007f90c84d04aa in c_body (d=0x7f90c0d20000) at continuations.c:318
#10 0x00007f90c853abc5 in scm_c_catch (tag=<value optimized out>, body=<value optimized out>, body_data=<value optimized out>, handler=<value optimized out>, handler_data=<value optimized out>, pre_unwind_handler=0x7f90c8539fc0 <scm_handle_by_message_noexit>, pre_unwind_handler_data=0x0) at throw.c:243
#11 0x00007f90c84d08d7 in scm_i_with_continuation_barrier (body=<value optimized out>, body_data=<value optimized out>, handler=0x7f90c84d04c0 <c_handler>, handler_data=0x7f90c0d20000, pre_unwind_handler=<value optimized out>, pre_unwind_handler_data=<value optimized out>) at continuations.c:294
#12 0x00007f90c84d0970 in scm_c_with_continuation_barrier (func=<value optimized out>, data=<value optimized out>) at continuations.c:336
#13 0x00007f90c8539972 in scm_i_with_guile_and_parent (func=<value optimized out>, data=<value optimized out>, parent=<value optimized out>) at threads.c:709
#14 0x00007f90c8539a13 in launch_thread (d=0x7fff794861b0) at threads.c:814
#15 0x00007f90c7ffddb2 in GC_inner_start_routine (sb=<value optimized out>, arg=<value optimized out>) at ../pthread_support.c:1153
#16 0x00007f90c7ff7af5 in GC_call_with_stack_base (fn=0x66b61bdc, arg=0x7f90c0d1f860) at ../misc.c:1323
#17 0x00007f90c7dcaf97 in start_thread () from /nix/store/s88vdfglm94x7jn0vqm24pqhq460s0c7-glibc-2.9/lib/libpthread.so.0
#18 0x00007f90c6f6156d in clone () from /nix/store/s88vdfglm94x7jn0vqm24pqhq460s0c7-glibc-2.9/lib/libc.so.6

Thread 2 (Thread 0x7f90c61c0950 (LWP 18001)):
#0  0x00007f90c7dd191b in read () from /nix/store/s88vdfglm94x7jn0vqm24pqhq460s0c7-glibc-2.9/lib/libpthread.so.0
#1  0x00007f90c85182b3 in signal_delivery_thread (data=<value optimized out>) at scmsigs.c:164
#2  0x00007f90c853abc5 in scm_c_catch (tag=<value optimized out>, body=<value optimized out>, body_data=<value optimized out>, handler=<value optimized out>, handler_data=<value optimized out>, pre_unwind_handler=0, pre_unwind_handler_data=0x0) at throw.c:243
#3  0x00007f90c853adb4 in scm_internal_catch (tag=0x2c, body=0x7f90c61bfb8f, body_data=0x1, handler=0xffffffffffffffff, handler_data=0x7f90c8217520) at throw.c:257
#4  0x00007f90c85391ce in really_spawn (d=0x7fff79486dd0) at threads.c:891
#5  0x00007f90c84d04aa in c_body (d=0x7f90c61c0000) at continuations.c:318
#6  0x00007f90c853abc5 in scm_c_catch (tag=<value optimized out>, body=<value optimized out>, body_data=<value optimized out>, handler=<value optimized out>, handler_data=<value optimized out>, pre_unwind_handler=0x7f90c8539fc0 <scm_handle_by_message_noexit>, pre_unwind_handler_data=0x0) at throw.c:243
#7  0x00007f90c84d08d7 in scm_i_with_continuation_barrier (body=<value optimized out>, body_data=<value optimized out>, handler=0x7f90c84d04c0 <c_handler>, handler_data=0x7f90c61c0000, pre_unwind_handler=<value optimized out>, pre_unwind_handler_data=<value optimized out>) at continuations.c:294
#8  0x00007f90c84d0970 in scm_c_with_continuation_barrier (func=<value optimized out>, data=<value optimized out>) at continuations.c:336
#9  0x00007f90c8539972 in scm_i_with_guile_and_parent (func=<value optimized out>, data=<value optimized out>, parent=<value optimized out>) at threads.c:709
#10 0x00007f90c85399e3 in spawn_thread (d=0x7fff79486dd0) at threads.c:903
#11 0x00007f90c7ffddb2 in GC_inner_start_routine (sb=<value optimized out>, arg=<value optimized out>) at ../pthread_support.c:1153
#12 0x00007f90c7ff7af5 in GC_call_with_stack_base (fn=0xfffffffffffffe00, arg=0x7f90c61bfb8f) at ../misc.c:1323
#13 0x00007f90c7dcaf97 in start_thread () from /nix/store/s88vdfglm94x7jn0vqm24pqhq460s0c7-glibc-2.9/lib/libpthread.so.0
#14 0x00007f90c6f6156d in clone () from /nix/store/s88vdfglm94x7jn0vqm24pqhq460s0c7-glibc-2.9/lib/libc.so.6

Thread 1 (Thread 0x7f90c89bd6f0 (LWP 17980)):
#0  scm_ithrow (key=0x707060, args=0x404, noreturn=1) at throw.c:746
#1  0x00007f90c84f00a0 in scm_i_gsubr_apply_list (self=0x707060, args=<value optimized out>) at gsubr.c:316
#2  0x00007f90c8545861 in vm_debug_engine (vp=0x703f00, program=<value optimized out>, argv=<value optimized out>, nargs=<value optimized out>) at vm-i-system.c:605
#3  0x00007f90c84e4b90 in scm_c_with_fluid (fluid=0x6602c0, value=0x4, cproc=0x7f90c84e49e0 <apply_thunk>, cdata=0xc106c0) at fluids.c:382
#4  0x00007f90c8545861 in vm_debug_engine (vp=0x703f00, program=<value optimized out>, argv=<value optimized out>, nargs=<value optimized out>) at vm-i-system.c:605
#5  0x00007f90c853abc5 in scm_c_catch (tag=<value optimized out>, body=<value optimized out>, body_data=<value optimized out>, handler=<value optimized out>, handler_data=<value optimized out>, pre_unwind_handler=0, pre_unwind_handler_data=0x7fff79486100) at throw.c:243
#6  0x00007f90c853ad4e in scm_catch_with_pre_unwind_handler (key=0x707060, thunk=<value optimized out>, handler=0xc10620, pre_unwind_handler=0x204) at throw.c:627
#7  0x00007f90c84f00a0 in scm_i_gsubr_apply_list (self=0x707060, args=<value optimized out>) at gsubr.c:316
#8  0x00007f90c8545ba6 in vm_debug_engine (vp=0x703f00, program=<value optimized out>, argv=<value optimized out>, nargs=<value optimized out>) at vm-i-system.c:689
#9  0x00007f90c8545861 in vm_debug_engine (vp=0x703f00, program=<value optimized out>, argv=<value optimized out>, nargs=<value optimized out>) at vm-i-system.c:605
#10 0x00007f90c84e4b90 in scm_c_with_fluid (fluid=0x6c68e0, value=0x4, cproc=0x7f90c84e49e0 <apply_thunk>, cdata=0x8e80c0) at fluids.c:382
#11 0x00007f90c8545861 in vm_debug_engine (vp=0x703f00, program=<value optimized out>, argv=<value optimized out>, nargs=<value optimized out>) at vm-i-system.c:605
#12 0x00007f90c84d4df3 in scm_dynamic_wind (in_guard=0x8e8280, thunk=0x8e82a0, out_guard=0x8e81c0) at dynwind.c:112
#13 0x00007f90c8545861 in vm_debug_engine (vp=0x703f00, program=<value optimized out>, argv=<value optimized out>, nargs=<value optimized out>) at vm-i-system.c:605
#14 0x00007f90c853d60f in scm_vm_apply (vm=0x6ff1f0, program=0x8e84c0, args=0x404) at vm.c:406
#15 0x00007f90c8545861 in vm_debug_engine (vp=0x703f00, program=<value optimized out>, argv=<value optimized out>, nargs=<value optimized out>) at vm-i-system.c:605
#16 0x00007f90c84d1122 in scm_sys_start_stack (id=0x104, thunk=0x8cfe20) at debug.c:483
#17 0x00007f90c8545ba6 in vm_debug_engine (vp=0x703f00, program=<value optimized out>, argv=<value optimized out>, nargs=<value optimized out>) at vm-i-system.c:689
#18 0x00007f90c853abc5 in scm_c_catch (tag=<value optimized out>, body=<value optimized out>, body_data=<value optimized out>, handler=<value optimized out>, handler_data=<value optimized out>, pre_unwind_handler=0, pre_unwind_handler_data=0x7fff79486c30) at throw.c:243
#19 0x00007f90c853ad4e in scm_catch_with_pre_unwind_handler (key=0x707060, thunk=<value optimized out>, handler=0x8cfe60, pre_unwind_handler=0x204) at throw.c:627
#20 0x00007f90c84f00a0 in scm_i_gsubr_apply_list (self=0x707060, args=<value optimized out>) at gsubr.c:316
#21 0x00007f90c8545861 in vm_debug_engine (vp=0x703f00, program=<value optimized out>, argv=<value optimized out>, nargs=<value optimized out>) at vm-i-system.c:605
#22 0x00007f90c84d1122 in scm_sys_start_stack (id=0x104, thunk=0x8cc0a0) at debug.c:483
#23 0x00007f90c8545861 in vm_debug_engine (vp=0x703f00, program=<value optimized out>, argv=<value optimized out>, nargs=<value optimized out>) at vm-i-system.c:605
#24 0x00007f90c853abc5 in scm_c_catch (tag=<value optimized out>, body=<value optimized out>, body_data=<value optimized out>, handler=<value optimized out>, handler_data=<value optimized out>, pre_unwind_handler=0x7f90c853a590 <scm_handle_by_proc>, pre_unwind_handler_data=0x7fff794871f0) at throw.c:243
#25 0x00007f90c853ad4e in scm_catch_with_pre_unwind_handler (key=0x707060, thunk=<value optimized out>, handler=0xa58fc0, pre_unwind_handler=0x8c5de0) at throw.c:627
#26 0x00007f90c84f00a0 in scm_i_gsubr_apply_list (self=0x707060, args=<value optimized out>) at gsubr.c:316
#27 0x00007f90c8545861 in vm_debug_engine (vp=0x703f00, program=<value optimized out>, argv=<value optimized out>, nargs=<value optimized out>) at vm-i-system.c:605
#28 0x00007f90c84d4df3 in scm_dynamic_wind (in_guard=0xa58120, thunk=0xa58100, out_guard=0xa580e0) at dynwind.c:112
#29 0x00007f90c8545861 in vm_debug_engine (vp=0x703f00, program=<value optimized out>, argv=<value optimized out>, nargs=<value optimized out>) at vm-i-system.c:605
#30 0x00007f90c84db73c in ceval (x=0x8b52c0, env=0x8b5220) at eval.i.c:1136
#31 0x00007f90c84e353a in scm_eval_x (exp=0x89b650, module_or_state=0x804720) at eval.c:4095
#32 0x00007f90c85195ff in scm_shell (argc=2, argv=0x7fff79487ae8) at script.c:771
#33 0x00007f90c84f611f in invoke_main_func (body_data=0x7fff794879e0) at init.c:378
#34 0x00007f90c84d04aa in c_body (d=0x7fff79487910) at continuations.c:318
#35 0x00007f90c853abc5 in scm_c_catch (tag=<value optimized out>, body=<value optimized out>, body_data=<value optimized out>, handler=<value optimized out>, handler_data=<value optimized out>, pre_unwind_handler=0x7f90c8539fc0 <scm_handle_by_message_noexit>, pre_unwind_handler_data=0x0) at throw.c:243
#36 0x00007f90c84d08d7 in scm_i_with_continuation_barrier (body=<value optimized out>, body_data=<value optimized out>, handler=0x7f90c84d04c0 <c_handler>, handler_data=0x7fff79487910, pre_unwind_handler=<value optimized out>, pre_unwind_handler_data=<value optimized out>) at continuations.c:294
#37 0x00007f90c84d0970 in scm_c_with_continuation_barrier (func=<value optimized out>, data=<value optimized out>) at continuations.c:336
#38 0x00007f90c8539972 in scm_i_with_guile_and_parent (func=<value optimized out>, data=<value optimized out>, parent=<value optimized out>) at threads.c:709
#39 0x00007f90c84f60d5 in scm_boot_guile (argc=<value optimized out>, argv=<value optimized out>, main_func=0x1, closure=0x1) at init.c:361
#40 0x0000000000400b50 in main (argc=7368800, argv=0x404) at guile.c:70
--8<---------------cut here---------------end--------------->8---

Hope this helps,
Ludo’.

(no subject)

[Neil Jerram]

ludo@gnu.org (Ludovic Courtès) writes:

> I should have mentioned this one:
> http://thread.gmane.org/gmane.comp.programming.garbage-collection.boehmgc/3185 .
>
> This is normally somewhat fixed in current BDW-GC CVS, but Guile itself
> may have troubles of its own dealing with cancellation.

Thanks.  I thought already about cancellation, but I couldn't see
anything in the test program that would do that.  Do you know of
anything in that program, or in BDW-GC, that does a pthread_cancel?

> The critical section issue should be orthogonal, though.

Yes.

> Here it is:

Thanks, and thanks to Ken also.

I've cherry-picked the following branch_release-1-8 fixes.

commit 499c43b03225abb8d3af9087b7630d523b74e13a
Author: Neil Jerram <neil@ossau.uklinux.net>
Date:   Thu Mar 5 20:03:33 2009 +0000

    Avoid throw from critical section, given invalid sigaction call

(This is for a different critical section problem, but which still
occurs (prior to this commit) in master.)

commit 3009b5d557e1ebfd828fd0de9b1da5abb2f6ec9a
Author: Neil Jerram <neil@ossau.uklinux.net>
Date:   Tue Mar 10 23:55:31 2009 +0000

    Fix spurious `throw from within critical section' errors

(This is the commit that I hope will fix the errors that you're seeing.)

Can you see how the test runs now?

Also, can I check my thinking on one other fix, and how it
interacts with BDW-GC?

commit 2bfcaf2605f8366d8c708c148bde5313b88497e0
Author: Neil Jerram <neil@ossau.uklinux.net>
Date:   Wed Mar 4 23:45:11 2009 +0000

    Lock ordering: don't allocate when in critical section (scm_sigaction_for_thread)

With BDW-GC I believe that allocation in a critical section is no longer
a problem, specifically because

- the stop-the-world mechanism uses signals and sigsuspend - whereas
  Guile GC used mutexes - and hence there are no concerns about lock
  ordering

- it doesn't matter if there are threads waiting for the critical
  section when a GC happens, because the signal mechanism will still
  interrupt them.

Is that right?

Thanks!
        Neil

Re: Bug #27457 (“Threads, mutexes, and critical sections”)

[Ludovic Courtès]

Hi Neil,

Neil Jerram <neil@ossau.uklinux.net> writes:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> I should have mentioned this one:
>> http://thread.gmane.org/gmane.comp.programming.garbage-collection.boehmgc/3185 .
>>
>> This is normally somewhat fixed in current BDW-GC CVS, but Guile itself
>> may have troubles of its own dealing with cancellation.
>
> Thanks.  I thought already about cancellation, but I couldn't see
> anything in the test program that would do that.  Do you know of
> anything in that program, or in BDW-GC, that does a pthread_cancel?

Hmm no, sorry for the confusion.  The srfi-18 test this was taken from
did use cancellation, though.

> I've cherry-picked the following branch_release-1-8 fixes.
>
> commit 499c43b03225abb8d3af9087b7630d523b74e13a
> Author: Neil Jerram <neil@ossau.uklinux.net>
> Date:   Thu Mar 5 20:03:33 2009 +0000
>
>     Avoid throw from critical section, given invalid sigaction call
>
> (This is for a different critical section problem, but which still
> occurs (prior to this commit) in master.)

Good that you noticed it was missing from ‘master’.  (BTW ‘signals.test’
should be changed to GPLv3+ and have a ‘define-module’ clause.)

> commit 3009b5d557e1ebfd828fd0de9b1da5abb2f6ec9a
> Author: Neil Jerram <neil@ossau.uklinux.net>
> Date:   Tue Mar 10 23:55:31 2009 +0000
>
>     Fix spurious `throw from within critical section' errors
>
> (This is the commit that I hope will fix the errors that you're seeing.)
>
> Can you see how the test runs now?

The abort () is apparently never reached with this patch.

It took me some time to understand this:

-  if (scm_i_critical_section_level)
+  if (thread->critical_section_level)
     {
       fprintf (stderr, "continuation invoked from within critical section.\n");
       abort ();

Critical sections concern all the threads in guile mode.  Thus I would
think that the question “are the threads in a critical section?” cannot
be answered by looking at a per-thread ‘critical_section_level’.

However, it really seems that the intent was really to forbid ‘throw’s
when the *current thread* is holding the critical section mutex, so the
patch is correct but I find the name ‘critical_section_level’ is
slightly misleading.

What do you think?

> Also, can I check my thinking on one other fix, and how it
> interacts with BDW-GC?
>
> commit 2bfcaf2605f8366d8c708c148bde5313b88497e0
> Author: Neil Jerram <neil@ossau.uklinux.net>
> Date:   Wed Mar 4 23:45:11 2009 +0000
>
>     Lock ordering: don't allocate when in critical section (scm_sigaction_for_thread)
>
> With BDW-GC I believe that allocation in a critical section is no longer
> a problem, specifically because
>
> - the stop-the-world mechanism uses signals and sigsuspend - whereas
>   Guile GC used mutexes - and hence there are no concerns about lock
>   ordering
>
> - it doesn't matter if there are threads waiting for the critical
>   section when a GC happens, because the signal mechanism will still
>   interrupt them.
>
> Is that right?

Yes.

BTW, rest assured: thanks to libgc we won’t get any Helgrind report!
:-)

Thanks,
Ludo’.

(no subject)

[Neil Jerram]

ludo@gnu.org (Ludovic Courtès) writes:

>>> This is normally somewhat fixed in current BDW-GC CVS, but Guile itself
>>> may have troubles of its own dealing with cancellation.
>>
>> Thanks.  I thought already about cancellation, but I couldn't see
>> anything in the test program that would do that.  Do you know of
>> anything in that program, or in BDW-GC, that does a pthread_cancel?
>
> Hmm no, sorry for the confusion.  The srfi-18 test this was taken from
> did use cancellation, though.

OK, thanks for clarifying that.  I guess there must be something other
than cancellation lurking.

> Good that you noticed it was missing from ‘master’.  (BTW ‘signals.test’
> should be changed to GPLv3+ and have a ‘define-module’ clause.)

Thanks, done.

>> commit 3009b5d557e1ebfd828fd0de9b1da5abb2f6ec9a
>> Author: Neil Jerram <neil@ossau.uklinux.net>
>> Date:   Tue Mar 10 23:55:31 2009 +0000
>>
>>     Fix spurious `throw from within critical section' errors
>>
>> (This is the commit that I hope will fix the errors that you're seeing.)
>>
>> Can you see how the test runs now?
>
> The abort () is apparently never reached with this patch.
>
> It took me some time to understand this:
>
> -  if (scm_i_critical_section_level)
> +  if (thread->critical_section_level)
>      {
>        fprintf (stderr, "continuation invoked from within critical section.\n");
>        abort ();
>
> Critical sections concern all the threads in guile mode.  Thus I would
> think that the question “are the threads in a critical section?” cannot
> be answered by looking at a per-thread ‘critical_section_level’.

That's not the question here.  The question is "is _this_ thread in a
critical section?"  There's no reason (AFAIK) why thread A shouldn't
throw while thread B is in a critical section.

Note that access to the critical section is controlled by a mutex, not
by (scm_i_)critical_section_level.  (scm_i_)critical_section_level only
exists to catch genuine "throw from critical section" errors.

> However, it really seems that the intent was really to forbid ‘throw’s
> when the *current thread* is holding the critical section mutex, so the
> patch is correct but I find the name ‘critical_section_level’ is
> slightly misleading.

The tricky detail here is that scm_i_critical_section_mutex is
initialized as a recursive mutex, so the implication is that it's
possible for a thread to enter the critical section multiple times.
Hence we need some kind of "level", instead of just a boolean
"in_critical_section".

> What do you think?

I have no objection to a more intuitive name, if you can suggest one.

>> Is that right?
>
> Yes.

Thanks.

> BTW, rest assured: thanks to libgc we won’t get any Helgrind report!
> :-)

I assume you mean because most of the helgrind reports involved the
heap_mutex, which no longer exists.  Presumably we could still (in
theory) get helgrind reports for other mutexes?

Regards,
        Neil

Re: Bug #27457 (“Threads, mutexes, and critical sections”)

[Ken Raeburn]

On Sep 30, 2009, at 16:59, Neil Jerram wrote:
> I've cherry-picked the following branch_release-1-8 fixes.

With these fixes, indeed, I can't reproduce the problem any more; I  
left the test program running for over an hour, whereas before it  
would trigger the problem almost immediately.
Thanks!

Ken

(no subject)

[Neil Jerram]

Ken Raeburn <raeburn@raeburn.org> writes:

> On Sep 30, 2009, at 16:59, Neil Jerram wrote:
>> I've cherry-picked the following branch_release-1-8 fixes.
>
> With these fixes, indeed, I can't reproduce the problem any more; I
> left the test program running for over an hour, whereas before it
> would trigger the problem almost immediately.
> Thanks!

Great news; thanks for trying and confirming that.

      Neil

Re: i guess we're frozen & stuff

[Ken Raeburn]

On Sep 28, 2009, at 18:42, Neil Jerram wrote:
> Regarding the throw from critical section problem, I guess I'm not
> seeing this because of not running on a multi-core machine.

You're on a single-core machine?  How quaint! ;-)

>  Can someone
> who does see this problem
>
> - run under GDB
> - set a breakpoint on the fprintf (stderr, "throw from within critical
>  section.\n") line in throw.c
> - post the thread backtraces (thread apply all bt), when this  
> breakpoint
>  is hit?

My Mac shows the problem almost instantly.  The requested stack trace  
is below.

I also note that according to gdb, scm-i_critical_section_level is  
zero, though this breakpoint is only reachable if it's nonzero;  
presumably another running thread changed it?

Thread 3 (process 53308 thread 0x1cb):
#0  0x973b5286 in mach_msg_trap ()
#1  0x973bca7c in mach_msg ()
#2  0x973d92ee in mach_port_deallocate ()
#3  0x002a5470 in GC_delete_thread ()
#4  0x002a59dd in GC_unregister_my_thread ()
#5  0x973e8b9a in _pthread_exit ()
#6  0x001d73d0 in really_launch (d=0xbfffe8b0) at ../../libguile/ 
threads.c:804
#7  0x0015d712 in c_body (d=0xbfffe8b0) at ../../libguile/ 
continuations.c:318
#8  0x001d9b95 in scm_c_catch (tag=0x0, body=0x15d700 <c_body>,  
body_data=0xb0080e64, handler=0x15d720 <c_handler>,  
handler_data=0xb0080e64, pre_unwind_handler=0,  
pre_unwind_handler_data=0x0) at ../../libguile/throw.c:243
#9  0x0015d6dd in scm_i_with_continuation_barrier (body=0,  
body_data=0x0, handler=0, handler_data=0x0, pre_unwind_handler=0,  
pre_unwind_handler_data=0x0) at ../../libguile/continuations.c:294
#10 0x0015d78e in scm_c_with_continuation_barrier (func=0, data=0x0)  
at ../../libguile/continuations.c:336
#11 0x001d71d9 in scm_i_with_guile_and_parent (func=0x1d7340  
<really_launch>, data=0x1031358, parent=0x973b5286) at ../../libguile/ 
threads.c:709
#12 0x001d742a in launch_thread (d=0x555720) at ../../libguile/ 
threads.c:814
#13 0x002a5ddd in GC_inner_start_routine ()
#14 0x002a0a2c in GC_call_with_stack_base ()
#15 0x002a5e62 in GC_start_routine ()
#16 0x973e6155 in _pthread_start ()
#17 0x973e6012 in thread_start ()

Thread 2 (process 53308 thread 0x2003):
#0  0x973d8f9a in read$UNIX2003 ()
#1  0x001b115a in signal_delivery_thread (data=0x0) at ../../libguile/ 
scmsigs.c:164
#2  0x001d9b95 in scm_c_catch (tag=0xc0003, body=0x1b1100  
<signal_delivery_thread>, body_data=0x0, handler=0x1da840  
<scm_handle_by_message>, handler_data=0x20a8c4,  
pre_unwind_handler=0xc0003, pre_unwind_handler_data=0xc0003) at ../../ 
libguile/throw.c:243
#3  0x001d9d0d in scm_internal_catch (tag=0xc0003, body=0xc0003,  
body_data=0xc0003, handler=0xc0003, handler_data=0xc0003) at ../../ 
libguile/throw.c:257
#4  0x001d74d6 in really_spawn (d=0xb0080b00) at ../../libguile/ 
threads.c:891
#5  0x0015d712 in c_body (d=0xb0080b00) at ../../libguile/ 
continuations.c:318
#6  0x001d9b95 in scm_c_catch (tag=0xc0003, body=0x15d700 <c_body>,  
body_data=0xb0102e64, handler=0x15d720 <c_handler>,  
handler_data=0xb0102e64, pre_unwind_handler=0xc0003,  
pre_unwind_handler_data=0xc0003) at ../../libguile/throw.c:243
#7  0x0015d6dd in scm_i_with_continuation_barrier (body=0xc0003,  
body_data=0xc0003, handler=0xc0003, handler_data=0xc0003,  
pre_unwind_handler=0xc0003, pre_unwind_handler_data=0xc0003) at ../../ 
libguile/continuations.c:294
#8  0x0015d78e in scm_c_with_continuation_barrier (func=0xc0003,  
data=0xc0003) at ../../libguile/continuations.c:336
#9  0x001d71d9 in scm_i_with_guile_and_parent (func=0x1d7440  
<really_spawn>, data=0x10275d8, parent=0x973d8f9a) at ../../libguile/ 
threads.c:709
#10 0x001d754a in spawn_thread (d=0x555be0) at ../../libguile/ 
threads.c:903
#11 0x002a5ddd in GC_inner_start_routine ()
#12 0x002a0a2c in GC_call_with_stack_base ()
#13 0x002a5e62 in GC_start_routine ()
#14 0x973e6155 in _pthread_start ()
#15 0x973e6012 in thread_start ()

Thread 1 (process 53308 local thread 0x3b07):
#0  scm_ithrow (key=0x59ad50, args=0x404, noreturn=1) at ../../ 
libguile/throw.c:746
#1  0x001dadda in scm_throw (key=0x59ad50, args=0x404) at ../../ 
libguile/throw.c:722
#2  0x00181c51 in gsubr_apply_raw (proc=0x56fe90, argc=<value  
temporarily unavailable, due to optimizations>, argv=0xbfffe21c)  
at ../../libguile/gsubr.c:212
#3  0x00181e91 in scm_i_gsubr_apply_list (self=0x56fe90,  
args=0x102f210) at ../../libguile/gsubr.c:316
#4  0x0016d25b in scm_dapply (proc=<value temporarily unavailable, due  
to optimizations>, arg1=0x59ad50, args=0x102f210) at eval.i.c:1903
#5  0x001e7659 in vm_debug_engine (vp=0x596fa0, program=<value  
temporarily unavailable, due to optimizations>, argv=0x0, nargs=<value  
temporarily unavailable, due to optimizations>) at vm-i-system.c:605
#6  0x00172d71 in scm_call_0 (proc=0x1004090) at ../../libguile/eval.c: 
3113
#7  0x001750ce in scm_c_with_fluid (fluid=0x216920, value=0x216920,  
cproc=0x175020 <apply_thunk>, cdata=0x216920) at ../../libguile/ 
fluids.c:382
#8  0x00175110 in scm_with_fluid (fluid=0x216920, value=0x216920,  
thunk=0x216920) at ../../libguile/fluids.c:369
#9  0x0016d7cc in scm_dapply (proc=<value temporarily unavailable, due  
to optimizations>, arg1=0x102f2c8, args=<value temporarily  
unavailable, due to optimizations>) at eval.i.c:1793
#10 0x001e7659 in vm_debug_engine (vp=0x596fa0, program=<value  
temporarily unavailable, due to optimizations>, argv=0x0, nargs=<value  
temporarily unavailable, due to optimizations>) at vm-i-system.c:605
#11 0x00172d71 in scm_call_0 (proc=0x1004020) at ../../libguile/eval.c: 
3113
#12 0x001d9b95 in scm_c_catch (tag=0x216920, body=0x1da0c0  
<scm_body_thunk>, body_data=0xbfffe7b8, handler=0x1da0e0  
<scm_handle_by_proc>, handler_data=0xbfffe7d8,  
pre_unwind_handler=0x216920 <scm_i_critical_section_level>,  
pre_unwind_handler_data=0x216920) at ../../libguile/throw.c:243
#13 0x001da279 in scm_catch_with_pre_unwind_handler (key=0x104,  
thunk=0x1004020, handler=0x1004000, pre_unwind_handler=0x204) at ../../ 
libguile/throw.c:627
#14 0x00181c51 in gsubr_apply_raw (proc=0x56ffb0, argc=<value  
temporarily unavailable, due to optimizations>, argv=0xbfffe85c)  
at ../../libguile/gsubr.c:212
#15 0x00181efc in scm_i_gsubr_apply_list (self=0x56ffb0,  
args=0x102f380) at ../../libguile/gsubr.c:316
#16 0x0016d25b in scm_dapply (proc=<value temporarily unavailable, due  
to optimizations>, arg1=0x104, args=0x102f380) at eval.i.c:1903
#17 0x001ecfa7 in vm_debug_engine (vp=0x596fa0, program=<value  
temporarily unavailable, due to optimizations>, argv=0x0, nargs=<value  
temporarily unavailable, due to optimizations>) at vm-i-system.c:689
#18 0x001ddf1d in scm_load_compiled_with_vm (file=0x216920) at ../../ 
libguile/vm.c:627
#19 0x0016d07a in scm_dapply (proc=0x2a4f9a, arg1=0x10b26b0,  
args=0x404) at eval.i.c:1804
#20 0x001e7659 in vm_debug_engine (vp=0x596fa0, program=<value  
temporarily unavailable, due to optimizations>, argv=0x0, nargs=<value  
temporarily unavailable, due to optimizations>) at vm-i-system.c:605
#21 0x00172d71 in scm_call_0 (proc=0x10137e0) at ../../libguile/eval.c: 
3113
#22 0x001750ce in scm_c_with_fluid (fluid=0x216920, value=0x216920,  
cproc=0x175020 <apply_thunk>, cdata=0x216920) at ../../libguile/ 
fluids.c:382
#23 0x00175110 in scm_with_fluid (fluid=0x216920, value=0x216920,  
thunk=0x216920) at ../../libguile/fluids.c:369
#24 0x0016d7cc in scm_dapply (proc=<value temporarily unavailable, due  
to optimizations>, arg1=0x1047540, args=<value temporarily  
unavailable, due to optimizations>) at eval.i.c:1793
#25 0x001e7659 in vm_debug_engine (vp=0x596fa0, program=<value  
temporarily unavailable, due to optimizations>, argv=0x0, nargs=<value  
temporarily unavailable, due to optimizations>) at vm-i-system.c:605
#26 0x00172d71 in scm_call_0 (proc=0x1013820) at ../../libguile/eval.c: 
3113
#27 0x001622d3 in scm_dynamic_wind (in_guard=0x1013810,  
thunk=0x1013820, out_guard=0x10137f0) at ../../libguile/dynwind.c:112
#28 0x0016d7cc in scm_dapply (proc=<value temporarily unavailable, due  
to optimizations>, arg1=0x1047580, args=<value temporarily  
unavailable, due to optimizations>) at eval.i.c:1793
#29 0x001e7659 in vm_debug_engine (vp=0x596fa0, program=<value  
temporarily unavailable, due to optimizations>, argv=0xbffff18c,  
nargs=<value temporarily unavailable, due to optimizations>) at vm-i- 
system.c:605
#30 0x00170add in ceval (x=0x404, env=0x1047c78) at eval.i.c:1247
#31 0x00171726 in ceval (x=<value temporarily unavailable, due to  
optimizations>, env=0x1047c78) at eval.i.c:330
#32 0x00172f2a in scm_primitive_eval_x (exp=0x1046288) at ../../ 
libguile/eval.c:4060
#33 0x00172f93 in scm_eval_x (exp=0x216920, module_or_state=0x10118a0)  
at ../../libguile/eval.c:4095
#34 0x001b360f in scm_shell (argc=3, argv=0xbffff590) at ../../ 
libguile/script.c:771
#35 0x001892a1 in invoke_main_func (body_data=0xbffff530) at ../../ 
libguile/init.c:378
#36 0x0015d712 in c_body (d=0xbffff4a4) at ../../libguile/ 
continuations.c:318
#37 0x001d9b95 in scm_c_catch (tag=0x216920, body=0x15d700 <c_body>,  
body_data=0xbffff4a4, handler=0x15d720 <c_handler>,  
handler_data=0xbffff4a4, pre_unwind_handler=0x216920  
<scm_i_critical_section_level>, pre_unwind_handler_data=0x216920)  
at ../../libguile/throw.c:243
#38 0x0015d6dd in scm_i_with_continuation_barrier (body=0x216920  
<scm_i_critical_section_level>, body_data=0x216920, handler=0x216920  
<scm_i_critical_section_level>, handler_data=0x216920,  
pre_unwind_handler=0x216920 <scm_i_critical_section_level>,  
pre_unwind_handler_data=0x216920) at ../../libguile/continuations.c:294
#39 0x0015d78e in scm_c_with_continuation_barrier (func=0x216920  
<scm_i_critical_section_level>, data=0x216920) at ../../libguile/ 
continuations.c:336
#40 0x001d71d9 in scm_i_with_guile_and_parent (func=0x189260  
<invoke_main_func>, data=0x558dc8, parent=0x1) at ../../libguile/ 
threads.c:709
#41 0x001d7239 in scm_with_guile (func=0x216920  
<scm_i_critical_section_level>, data=0x216920) at ../../libguile/ 
threads.c:688
#42 0x0018923a in scm_boot_guile (argc=2189600, argv=0x216920,  
main_func=0x216920 <scm_i_critical_section_level>, closure=0x216920)  
at ../../libguile/init.c:361
#43 0x00001ff1 in main (argc=2189600, argv=0x216920) at ../../libguile/ 
guile.c:70
(gdb)

Ken

Re: i guess we're frozen & stuff

[Ken Raeburn]

On Sep 25, 2009, at 17:59, Ken Raeburn wrote:
> On Sep 16, 2009, at 15:00, Andy Wingo wrote:
>> Hi Ken,
>>
>> On Tue 11 Aug 2009 15:59, Ken Raeburn <raeburn@raeburn.org> writes:
>>
>>> Perhaps I'm building [Guile] in ways that are unusual for the other
>>> developers (build dir != src dir, libgmp and guile-1.8 installed in
>>> the same place, libgmp and libunistring installed in different
>>> nonstandard directories)?
>>
>> I think I fixed the case in which $prefix was not in  
>> LD_LIBRARY_PATH or
>> the like. Can you update us on the status of your build issues?
>
> Frequently failing. :-(
> On the bright side, using the --with-foo-prefix options, I haven't  
> seen indications yet of the include- and load-path problems I saw  
> before.

The LD_LIBRARY_PATH issue does seem to be present still, actually.   
(Sadly, I'm running into enough problems that I'm losing track of  
which is which if I'm not careful.)  In a fresh Mac OS X build:

% ./I/bin/guile
Backtrace:
In ../../module/ice-9/boot-9.scm:
2363: 29* [#<program 100bb30 at ../../module/ice-9/boot-9.scm:2361:12  
()>]
[...]
In unknown file:
    ?: 46* [primitive-load-path "srfi/srfi-1" #f]
In ../../module/srfi/srfi-1.scm:
  223: 47* [#<program 100b160 ()>]
In unknown file:
    ?: 48* [load-extension "libguile-srfi-srfi-1-v-4" "scm_init_srfi_1"]

ERROR: In procedure dynamic-link:
ERROR: file: "libguile-srfi-srfi-1-v-4", message: "file not found"
%

After I set LD_LIBRARY_PATH *and* added a symlink using the .so suffix  
instead of .dylib, it worked.  Either by itself was not sufficient.

This is with commit 43b03fb from today, plus two patches: Disabling  
the glibc-specific locale code, and using the 1.8 version of  
regexp.test.

Ken

Re: i guess we're frozen & stuff

[Neil Jerram]

Ken Raeburn <raeburn@raeburn.org> writes:

> With the attached patch, I can get guile to build with CPPFLAGS= and
> LDFLAGS= ... someone more familiar than I am with automake will have
> to fix the guile-readline stuff.

The patch looks good to me, so I've committed it now.  Thanks!

    Neil

Re: i guess we're frozen & stuff

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 17760 bytes --]


I built libunistring and tried to build master.  Some tests failed, and
also the build failed due to warnings (post to follow).  I think all the
failed tests are locale related.

"gmake check" output on NetBSD/amd64 5.0 with libunistring 0.9.1:

Making check in lib
gmake[1]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/lib'
gmake  check-recursive
gmake[2]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/lib'
gmake[3]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/lib'
gmake[3]: Nothing to be done for `check-am'.
gmake[3]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/lib'
gmake[2]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/lib'
gmake[1]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/lib'
Making check in meta
gmake[1]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/meta'
gmake[1]: Nothing to be done for `check'.
gmake[1]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/meta'
Making check in libguile
gmake[1]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/libguile'
gmake  check-am
gmake[2]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/libguile'
gmake[2]: Nothing to be done for `check-am'.
gmake[2]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/libguile'
gmake[1]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/libguile'
Making check in guile-readline
gmake[1]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/guile-readline'
gmake  check-recursive
gmake[2]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/guile-readline'
Making check in ice-9
gmake[3]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/guile-readline/ice-9'
gmake[3]: Nothing to be done for `check'.
gmake[3]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/guile-readline/ice-9'
gmake[3]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/guile-readline'
gmake[3]: Nothing to be done for `check-am'.
gmake[3]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/guile-readline'
gmake[2]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/guile-readline'
gmake[1]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/guile-readline'
Making check in emacs
gmake[1]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/emacs'
gmake[1]: Nothing to be done for `check'.
gmake[1]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/emacs'
Making check in srfi
gmake[1]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/srfi'
gmake  check-am
gmake[2]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/srfi'
gmake[2]: Nothing to be done for `check-am'.
gmake[2]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/srfi'
gmake[1]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/srfi'
Making check in doc
gmake[1]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/doc'
Making check in ref
gmake[2]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/doc/ref'
gmake  check-am
gmake[3]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/doc/ref'
gmake[3]: Nothing to be done for `check-am'.
gmake[3]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/doc/ref'
gmake[2]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/doc/ref'
Making check in tutorial
gmake[2]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/doc/tutorial'
gmake[2]: Nothing to be done for `check'.
gmake[2]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/doc/tutorial'
Making check in goops
gmake[2]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/doc/goops'
gmake[2]: Nothing to be done for `check'.
gmake[2]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/doc/goops'
Making check in r5rs
gmake[2]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/doc/r5rs'
gmake[2]: Nothing to be done for `check'.
gmake[2]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/doc/r5rs'
gmake[2]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/doc'
gmake[2]: Nothing to be done for `check-am'.
gmake[2]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/doc'
gmake[1]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/doc'
Making check in examples
gmake[1]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/examples'
gmake[1]: Nothing to be done for `check'.
gmake[1]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/examples'
Making check in test-suite
gmake[1]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/test-suite'
Making check in standalone
gmake[2]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/test-suite/standalone'
gmake  check-am
gmake[3]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/test-suite/standalone'
gmake  test-num2integral test-round test-list test-unwind test-conversion test-scm-c-read test-scm-take-locale-symbol test-with-guile-module test-scm-with-guile test-system-cmds test-bad-identifiers test-require-extension test-asmobs test-fast-slot-ref test-use-srfi test-extensions
gmake[4]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/test-suite/standalone'
gmake[4]: `test-num2integral' is up to date.
gmake[4]: `test-round' is up to date.
gmake[4]: `test-list' is up to date.
gmake[4]: `test-unwind' is up to date.
gmake[4]: `test-conversion' is up to date.
gmake[4]: `test-scm-c-read' is up to date.
gmake[4]: `test-scm-take-locale-symbol' is up to date.
gmake[4]: `test-with-guile-module' is up to date.
gmake[4]: `test-scm-with-guile' is up to date.
gmake[4]: Nothing to be done for `test-system-cmds'.
gmake[4]: Nothing to be done for `test-bad-identifiers'.
gmake[4]: Nothing to be done for `test-require-extension'.
gmake[4]: Nothing to be done for `test-asmobs'.
gmake[4]: `test-fast-slot-ref' is up to date.
gmake[4]: `test-use-srfi' is up to date.
gmake[4]: Nothing to be done for `test-extensions'.
gmake[4]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/test-suite/standalone'
gmake  check-TESTS
gmake[4]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/test-suite/standalone'
PASS: test-system-cmds
PASS: test-bad-identifiers
PASS: test-require-extension
PASS: test-num2integral
PASS: test-round
PASS: test-asmobs
PASS: test-list
PASS: test-unwind
PASS: test-conversion
PASS: test-fast-slot-ref
PASS: test-use-srfi
PASS: test-scm-c-read
PASS: test-scm-take-locale-symbol
PASS: test-extensions
PASS: test-with-guile-module
PASS: test-scm-with-guile
===================
All 16 tests passed
===================
gmake[4]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/test-suite/standalone'
gmake[3]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/test-suite/standalone'
gmake[2]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/test-suite/standalone'
gmake[2]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/test-suite'
gmake[2]: Nothing to be done for `check-am'.
gmake[2]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/test-suite'
gmake[1]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/test-suite'
Making check in benchmark-suite
gmake[1]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/benchmark-suite'
gmake[1]: Nothing to be done for `check'.
gmake[1]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/benchmark-suite'
Making check in lang
gmake[1]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/lang'
gmake[1]: Nothing to be done for `check'.
gmake[1]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/lang'
Making check in am
gmake[1]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/am'
gmake[1]: Nothing to be done for `check'.
gmake[1]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/am'
Making check in module
gmake[1]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/module'
gmake[1]: Nothing to be done for `check'.
gmake[1]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/module'
Making check in testsuite
gmake[1]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/testsuite'
gmake  check-TESTS
gmake[2]: Entering directory `/home/gdt/BUILD-GUILE-master/guile/testsuite'
running `./t-basic-contructs.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-basic-contructs.scm
running `./t-global-bindings.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-global-bindings.scm
running `./t-catch.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-catch.scm
running `./t-call-cc.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-call-cc.scm
running `./t-closure.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-closure.scm
running `./t-closure2.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-closure2.scm
running `./t-closure3.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-closure3.scm
running `./t-closure4.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-closure4.scm
running `./t-do-loop.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-do-loop.scm
running `./t-literal-integers.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-literal-integers.scm
running `./t-macros.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-macros.scm
running `./t-macros2.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-macros2.scm
running `./t-map.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-map.scm
running `./t-or.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-or.scm
running `./t-proc-with-setter.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-proc-with-setter.scm
running `./t-quasiquote.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-quasiquote.scm
running `./t-values.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-values.scm
running `./t-records.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-records.scm
running `./t-match.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-match.scm
running `./t-mutual-toplevel-defines.scm'...  reading...  compiling...  running...  reading...  ok
PASS: t-mutual-toplevel-defines.scm
===================
All 20 tests passed
===================
gmake[2]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/testsuite'
gmake[1]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile/testsuite'
gmake[1]: Entering directory `/home/gdt/BUILD-GUILE-master/guile'
gmake  check-TESTS
gmake[2]: Entering directory `/home/gdt/BUILD-GUILE-master/guile'
Testing /home/gdt/BUILD-GUILE-master/guile/meta/guile ...
with GUILE_LOAD_PATH=/home/gdt/BUILD-GUILE-master/guile/test-suite
Running alist.test
Running and-let-star.test
Running arbiters.test
Running asm-to-bytecode.test
Running bit-operations.test
Running bytevectors.test
FAIL: bytevectors.test: 2.9 Operations on Strings: string->utf8 [latin-1]
FAIL: bytevectors.test: 2.9 Operations on Strings: utf8->string [latin-1]
Running c-api.test
Running chars.test
Running common-list.test
Running compiler.test
Running continuations.test
Running dynamic-scope.test
Running elisp.test
Running environments.test
Running eval.test
Running exceptions.test
Running filesys.test
Running format.test
Running fractions.test
Running ftw.test
Running gc.test
Running getopt-long.test
Running goops.test
Running guardians.test
Running hash.test
Running hooks.test
Running i18n.test
FAIL: i18n.test: locale objects: make-locale with unknown locale
UNRESOLVED: i18n.test: text collation (French): string-locale<?
UNRESOLVED: i18n.test: text collation (French): char-locale<?
UNRESOLVED: i18n.test: text collation (French): string-locale-ci=?
UNRESOLVED: i18n.test: text collation (French): string-locale-ci<>?
UNRESOLVED: i18n.test: text collation (French): char-locale-ci<>?
UNRESOLVED: i18n.test: number parsing: locale-string->inexact (French)
FAIL: i18n.test: nl-langinfo et al.: locale-day (1 arg)
FAIL: i18n.test: nl-langinfo et al.: locale-day (2 args)
FAIL: i18n.test: nl-langinfo et al.: locale-day (2 args, using `%global-locale')
UNRESOLVED: i18n.test: nl-langinfo et al.: locale-day (French)
UNRESOLVED: i18n.test: nl-langinfo et al.: locale-day (French, using `%global-locale')
UNRESOLVED: i18n.test: nl-langinfo et al.: default locale
Running import.test
Running interp.test
Running list.test
Running load.test
Running modules.test
Running numbers.test
Running optargs.test
Running options.test
Running pairs.test
Running poe.test
Running popen.test
Running ports.test
Running posix.test
Running procprop.test
UNRESOLVED: procprop.test: procedure-arity: apply
Running q.test
Running r4rs.test
Running r5rs_pitfall.test
Running r6rs-ports.test
Running ramap.test
Running reader.test
Running receive.test
Running regexp.test
Running socket.test

Some deprecated features have been used.  Set the environment
variable GUILE_WARN_DEPRECATED to "detailed" and rerun the
program to get more information.  Set it to "no" to suppress
this message.
WARNING: (srfi srfi-18): imported module (srfi srfi-34) overrides core binding `raise'
WARNING: (test-srfi-69): imported module (srfi srfi-69) overrides core binding `make-hash-table'
WARNING: (test-srfi-69): imported module (srfi srfi-69) overrides core binding `hash-table?'
Running alist.test
Running and-let-star.test
Running arbiters.test
Running asm-to-bytecode.test
Running bit-operations.test
Running bytevectors.test
FAIL: bytevectors.test: 2.9 Operations on Strings: string->utf8 [latin-1]
FAIL: bytevectors.test: 2.9 Operations on Strings: utf8->string [latin-1]
Running c-api.test
Running chars.test
Running common-list.test
Running compiler.test
Running continuations.test
Running dynamic-scope.test
Running elisp.test
Running environments.test
Running eval.test
Running exceptions.test
Running filesys.test
Running format.test
Running fractions.test
Running ftw.test
Running gc.test
Running getopt-long.test
Running goops.test
Running guardians.test
Running hash.test
Running hooks.test
Running i18n.test
FAIL: i18n.test: locale objects: make-locale with unknown locale
UNRESOLVED: i18n.test: text collation (French): string-locale<?
UNRESOLVED: i18n.test: text collation (French): char-locale<?
UNRESOLVED: i18n.test: text collation (French): string-locale-ci=?
UNRESOLVED: i18n.test: text collation (French): string-locale-ci<>?
UNRESOLVED: i18n.test: text collation (French): char-locale-ci<>?
UNRESOLVED: i18n.test: number parsing: locale-string->inexact (French)
FAIL: i18n.test: nl-langinfo et al.: locale-day (1 arg)
FAIL: i18n.test: nl-langinfo et al.: locale-day (2 args)
FAIL: i18n.test: nl-langinfo et al.: locale-day (2 args, using `%global-locale')
UNRESOLVED: i18n.test: nl-langinfo et al.: locale-day (French)
UNRESOLVED: i18n.test: nl-langinfo et al.: locale-day (French, using `%global-locale')
UNRESOLVED: i18n.test: nl-langinfo et al.: default locale
Running import.test
Running interp.test
Running list.test
Running load.test
Running modules.test
Running numbers.test
Running optargs.test
Running options.test
Running pairs.test
Running poe.test
Running popen.test
Running ports.test
Running posix.test
Running procprop.test
UNRESOLVED: procprop.test: procedure-arity: apply
Running q.test
Running r4rs.test
Running r5rs_pitfall.test
Running r6rs-ports.test
Running ramap.test
Running reader.test
Running receive.test
Running regexp.test
Running socket.test
Running sort.test
Running srcprop.test
Running srfi-1.test
Running srfi-10.test
Running srfi-11.test
Running srfi-13.test
Running srfi-14.test
UNRESOLVED: srfi-14.test: Latin-1 (8-bit charset): char-set:letter (membership)
UNRESOLVED: srfi-14.test: Latin-1 (8-bit charset): char-set:letter (size)
UNRESOLVED: srfi-14.test: Latin-1 (8-bit charset): char-set:lower-case (size)
UNRESOLVED: srfi-14.test: Latin-1 (8-bit charset): char-set:upper-case (size)
UNRESOLVED: srfi-14.test: Latin-1 (8-bit charset): char-set:punctuation (membership)
Running srfi-17.test
Running srfi-18.test
Running srfi-19.test
ERROR: srfi-19.test: SRFI date/time library: string->date understands days and months - arguments: ((misc-error string->date "TIME-ERROR type ~A: ~S" (bad-date-template-string ("Invalid string for " #<program priv:locale-long-weekday->index (string)>)) #f))
ERROR: srfi-19.test: SRFI date/time library: string->date works on Sunday - arguments: ((misc-error string->date "TIME-ERROR type ~A: ~S" (bad-date-template-string ("Invalid string for " #<program priv:locale-abbr-weekday->index (string)>)) #f))
Running srfi-26.test
Running srfi-31.test
Running srfi-34.test
Running srfi-35.test
Running srfi-37.test
Running srfi-39.test
Running srfi-4.test
Running srfi-6.test
Running srfi-60.test
Running srfi-69.test
Running srfi-88.test
Running srfi-9.test
Running srfi-98.test
Running streams.test
Running strings.test
Running structs.test
Running symbols.test
Running syncase.test
Running syntax.test
Running threads.test
Running time.test
Running tree-il.test
Running unif.test
Running vectors.test
Running version.test
Running weaks.test

Totals for this test run:
passes:                 12382
failures:               6
unexpected passes:      0
expected failures:      24
unresolved test cases:  15
untested test cases:    1
unsupported test cases: 11
errors:                 2


Some deprecated features have been used.  Set the environment
variable GUILE_WARN_DEPRECATED to "detailed" and rerun the
program to get more information.  Set it to "no" to suppress
this message.
FAIL: check-guile
==================================
1 of 1 tests failed
Please report to bug-guile@gnu.org
==================================
gmake[2]: *** [check-TESTS] Error 1
gmake[2]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile'
gmake[1]: *** [check-am] Error 2
gmake[1]: Leaving directory `/home/gdt/BUILD-GUILE-master/guile'
gmake: *** [check-recursive] Error 1


[-- Attachment #2: Type: application/pgp-signature, Size: 193 bytes --]

Re: i guess we're frozen & stuff

[Mike Gran]

On Tue, 2009-08-11 at 08:29 -0400, Greg Troxel wrote:
> I built libunistring and tried to build master.  Some tests failed, and
> also the build failed due to warnings (post to follow).  I think all the
> failed tests are locale related.
> 

A possible cause of some of this may be when I replaced toupper() with
uc_toupper, leading to a confusion between locale-dependent case
handling and Unicode codepoint case handling.  Probably should revert
the uc_toupper and uc_tolower calls for now until the conversion is more
complete.

Odd that I didn't see these errors, though.

-Mike

Re: i guess we're frozen & stuff

[Ludovic Courtès]

Greg Troxel <gdt@ir.bbn.com> writes:

> FAIL: bytevectors.test: 2.9 Operations on Strings: string->utf8 [latin-1]
> FAIL: bytevectors.test: 2.9 Operations on Strings: utf8->string [latin-1]

Does libunistring's "make check" pass on this platform?

> FAIL: i18n.test: locale objects: make-locale with unknown locale

Can you try this (run "./meta/guile"):

  (use-modules (ice-9 i18n))
  (make-locale LC_ALL "does-not-exist")

... and report the result?

> FAIL: i18n.test: nl-langinfo et al.: locale-day (1 arg)

What does "(map locale-day (map 1+ (iota 7)))" return?

> FAIL: i18n.test: nl-langinfo et al.: locale-day (2 args)

What does this return?

  (map (lambda (day)
         (locale-day day (make-locale LC_ALL "C")))
       (map 1+ (iota 7)))

> FAIL: i18n.test: nl-langinfo et al.: locale-day (2 args, using `%global-locale')

and:

  (map (lambda (day)
         (locale-day day %global-locale))
       (map 1+ (iota 7)))

> FAIL: bytevectors.test: 2.9 Operations on Strings: string->utf8 [latin-1]
> FAIL: bytevectors.test: 2.9 Operations on Strings: utf8->string [latin-1]

Again, might be a libunistring issue.

And why were some test files evaluated twice?!

> ERROR: srfi-19.test: SRFI date/time library: string->date understands days and months - arguments: ((misc-error string->date "TIME-ERROR type ~A: ~S" (bad-date-template-string ("Invalid string for " #<program priv:locale-long-weekday->index (string)>)) #f))

What does this do:

  (use-modules (srfi srfi-19))
  (let ((d (string->date "Saturday, December 9, 2006"
                                            "~A, ~B ~d, ~Y")))
    (date->time-utc (make-date (date-nanosecond d)
                               (date-second d)
                               (date-minute d)
                               (date-hour d)
                               (date-day d)
                               (date-month d)
                               (date-year d)
                               0)))

> ERROR: srfi-19.test: SRFI date/time library: string->date works on Sunday - arguments: ((misc-error string->date "TIME-ERROR type ~A: ~S" (bad-date-template-string ("Invalid string for " #<program priv:locale-abbr-weekday->index (string)>)) #f))

And this:

  (use-modules (srfi srfi-19))
  (let* ((str  "Sun, 05 Jun 2005 18:33:00 +0200")
         (date (string->date str "~a, ~d ~b ~Y ~H:~M:~S ~z")))
   (date->string date))


I also noticed another problem (x86_64-unknown-linux-gnu):

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (string=? "\u0100\u0101\x0102" "\u0100\u0101\x0102")

Backtrace:
In unknown file:
   ?: 0* [#<vm 7fc03c042ff0> #<program 7fc0349061c0 at <unknown port>:4:0 ()>]
   5: 1* [#<program 7fc0349061c0 at <unknown port>:4:0 ()>]
   ?: 2* [string=? "\u0100\u0101\x0102" "\u0100\u0101\x0102"]

ERROR: In procedure string=?:
ERROR: Invalid read access of chars of wide string: "\u0100\u0101\x0102"
--8<---------------cut here---------------end--------------->8---

It's not a regression strictly speaking, but still.  Mike?  :-)

Thanks,
Ludo'.

Re: i guess we're frozen & stuff

[Mike Gran]

On Tue, 2009-08-11 at 17:54 +0200, Ludovic Courtès wrote:

> I also noticed another problem (x86_64-unknown-linux-gnu):
> 
> --8<---------------cut here---------------start------------->8---
> scheme@(guile-user)> (string=? "\u0100\u0101\x0102" "\u0100\u0101\x0102")
> 
> Backtrace:
> In unknown file:
>    ?: 0* [#<vm 7fc03c042ff0> #<program 7fc0349061c0 at <unknown port>:4:0 ()>]
>    5: 1* [#<program 7fc0349061c0 at <unknown port>:4:0 ()>]
>    ?: 2* [string=? "\u0100\u0101\x0102" "\u0100\u0101\x0102"]
> 
> ERROR: In procedure string=?:
> ERROR: Invalid read access of chars of wide string: "\u0100\u0101\x0102"
> --8<---------------cut here---------------end--------------->8---
> 
> It's not a regression strictly speaking, but still.  Mike?  :-)

All of srfi-13.c isn't ready to operate on non-8-bit strings.  string=?
calls MY_VALIDATE_SUBSTRING_SPEC_COPY, which calls scm_i_string_chars.
scm_i_string_chars either returns a pointer to the Latin-1 strings or
errors with the error you saw.

-Mike

Re: i guess we're frozen & stuff

[Ludovic Courtès]

Mike Gran <spk121@yahoo.com> writes:

> All of srfi-13.c isn't ready to operate on non-8-bit strings.  string=?
> calls MY_VALIDATE_SUBSTRING_SPEC_COPY, which calls scm_i_string_chars.
> scm_i_string_chars either returns a pointer to the Latin-1 strings or
> errors with the error you saw.

OK, thanks for the explanation.

Apparently the disassembler needs to be taught Unicode:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,c x
Disassembly of #<objcode 7feee66c9840>:

   0    (load-symbol "\x01")            ;; .
   6    (add)                           
   7    (link-now)                      
   8    (variable-ref)                  
   9    (return)                        

scheme@(guile-user)> ,c y
Disassembly of #<objcode 7feee66cbc20>:

   0    (load-symbol "\x01")            ;; .
   6    (sub)                           
   7    (link-now)                      
   8    (variable-ref)                  
   9    (return)                        

scheme@(guile-user)> ,c a
Disassembly of #<objcode 7feee66bf160>:


Backtrace:
In system/base/compile.scm:
 244: 0  [decompile-fold # # #f ...]
In language/assembly/decompile-bytecode.scm:
  37: 1  [decompile-bytecode # #f ()]
In language/assembly/decompile-bytecode.scm:
  88: 2  [lp ((load-symbol "\x01"))]
In language/assembly/decompile-bytecode.scm:
 100: 3  [# {97}]
In unknown file:
   ?: 4* [opcode->instruction {97}]

ERROR: In procedure opcode->instruction:
ERROR: Wrong type argument in position 1 (expecting INSTRUCTION_P): 97
scheme@(guile-user)> ,c d
Disassembly of #<objcode 7feee66d0f40>:

   0    (load-symbol "\x01")            ;; .
   6    (not)                           
   7    (link-now)                      
   8    (variable-ref)                  
   9    (return)                        

scheme@(guile-user)> ,c c
Disassembly of #<objcode 7feee66dcf40>:


Backtrace:
In system/base/compile.scm:
 244: 0  [decompile-fold # # #f ...]
In language/assembly/decompile-bytecode.scm:
  37: 1  [decompile-bytecode # #f ()]
In language/assembly/decompile-bytecode.scm:
  88: 2  [lp ((load-symbol "\x01"))]
In language/assembly/decompile-bytecode.scm:
 100: 3  [# {99}]
In unknown file:
   ?: 4* [opcode->instruction {99}]

ERROR: In procedure opcode->instruction:
ERROR: Wrong type argument in position 1 (expecting INSTRUCTION_P): 99
scheme@(guile-user)> ,c abcd
Disassembly of #<objcode 7feee66ca380>:

   0    (load-symbol "\x01abc")         ;; .abc
   9    (not)                           
  10    (link-now)                      
  11    (variable-ref)                  
  12    (return)                        

--8<---------------cut here---------------end--------------->8---

Ideas on how to fix this?

Thanks,
Ludo'.

Re: i guess we're frozen & stuff

[Mike Gran]

> From: Ludovic Courtès ludo@gnu.org

> Apparently the disassembler needs to be taught Unicode:

There's a disassembler?  

In the assembler, the string, symbol, keyword, and define are changed 
from

--
3 bytes:      string length
LENGTH bytes: one byte chars
--

to 

--
3 bytes:      string length
1 bytes:      bytes per char
LENGTH*BYTES_PER_CHAR bytes: character array
--

where the character array is either and 8-bit chars array or a 
32-bit native endian unsigned integer array.

so, the reverse op needs to make it into the disassembler for string,
keyword, symbol, and define.  But, I've never checked out the 
disassembler before.  I'll look at it tonight.

Thanks,

-Mike

Re: i guess we're frozen & stuff

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 1108 bytes --]


  Does libunistring's "make check" pass on this platform?

should have tried that.  testlocale.c fails on line 38 with:

test-locale.c:38: error: expected ')' before numeric constant

expanding that with cpp gets me the rather stunning:

  extern int (* verify_function__ (void)) [(!!sizeof (struct { unsigned int verify_error_if_negative_size__: (sizeof (void *)0 == sizeof (void *)) ? 1 : -1; }))];

which I'll rewrap to:

  extern int (* verify_function__ (void))
      [(!!sizeof (struct {
          unsigned int verify_error_if_negative_size__ : (sizeof (void *)0 == sizeof (void *)) ? 1 : -1; }
      ))];

which seems odd but not necessarily wrong.  The original line was
  verify (sizeof NULL == sizeof (void *));


So it seems that NULL is expanding to (void *) 0, and "sizeof (void *)
0" is not legit.  AFAIK sizeof is specified to work on variables and
types, and NULL is neither a variable nor a type.

Is NULL something else on Linux?

Fixing that so sizeof(NULL) gets this to pass with only one failure:

  test-striconveh.c:389: assertion failed
  Abort (core dumped)
  FAIL: test-striconveh

[-- Attachment #2: Type: application/pgp-signature, Size: 193 bytes --]

Re: i guess we're frozen & stuff

[Ken Raeburn]

On Aug 11, 2009, at 13:04, Greg Troxel wrote:
> So it seems that NULL is expanding to (void *) 0, and "sizeof (void *)
> 0" is not legit.  AFAIK sizeof is specified to work on variables and
> types, and NULL is neither a variable nor a type.

No, sizeof should work fine on expression values as well.  I'm not  
quite sure about the question of syntax here -- sizeof's operand may  
be "an expression or the parenthesized name of a type".  (So "sizeof  
var" is just a special case of the expression version, and doesn't  
require parentheses.)  But if the expression starts with a  
parenthesized type because it's a cast... looking at the grammar, I'd  
think it would be valid, but that would have implications for code  
such as "sizeof (unsigned long) + 3" ... is that a single expression  
(3UL) you're taking the size of, or a sum involving the size of a type?

Certainly "sizeof ((void *)0)", with the extra parens, works, and I  
think I've nearly always seen pointer versions of "NULL" using the  
enclosing parentheses, perhaps because of just this issue.  Assuming  
"sizeof (TYPE) EXPR" isn't valid, I'd call it a defect in your  
system's definition of NULL, though I wouldn't go so far as to call it  
non-compliant.  One could also argue that an expression provided to  
sizeof should always be parenthesized unless you know the syntax of it  
won't be altered by sticking "sizeof" in front, e.g., "sizeof(3+4)"  
instead of "sizeof 3+4".

However, they're testing for a POSIX 2008 requirement that C99 and  
POSIX 2004 implementations need not meet, namely that NULL be of type  
"void *" instead of any null pointer constant (e.g., "0").  I think  
requiring POSIX 2008 support for Guile and anything that builds on it  
seems like a bad idea.  I haven't looked at the libunistring code to  
see why it might be relevant, but it seems like a pretty gratuitous  
imposition to me.  The only benefit of it I can see is that a variadic  
function can then take NULL as an argument without casting to char*;  
is that worth refusing to support other systems?

> Is NULL something else on Linux?

I'm not sure if it's GNU libc or GCC, but I'm getting "((void *)0)".

Ken

Re: i guess we're frozen & stuff

[Ludovic Courtès]

Ken Raeburn <raeburn@raeburn.org> writes:

> However, they're testing for a POSIX 2008 requirement that C99 and
> POSIX 2004 implementations need not meet, namely that NULL be of type
> "void *" instead of any null pointer constant (e.g., "0").  I think
> requiring POSIX 2008 support for Guile and anything that builds on it
> seems like a bad idea.  I haven't looked at the libunistring code to
> see why it might be relevant, but it seems like a pretty gratuitous
> imposition to me.  The only benefit of it I can see is that a variadic
> function can then take NULL as an argument without casting to char*;
> is that worth refusing to support other systems?

I didn't know it was a POSIX 2008 requirement.  Then indeed, we should
discuss this with the libunistring folks.

Thanks,
Ludo'.

Re: i guess we're frozen & stuff

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 1494 bytes --]


ludo@gnu.org (Ludovic Courtès) writes:

> Ken Raeburn <raeburn@raeburn.org> writes:
>
>> However, they're testing for a POSIX 2008 requirement that C99 and
>> POSIX 2004 implementations need not meet, namely that NULL be of type
>> "void *" instead of any null pointer constant (e.g., "0").  I think
>> requiring POSIX 2008 support for Guile and anything that builds on it
>> seems like a bad idea.  I haven't looked at the libunistring code to
>> see why it might be relevant, but it seems like a pretty gratuitous
>> imposition to me.  The only benefit of it I can see is that a variadic
>> function can then take NULL as an argument without casting to char*;
>> is that worth refusing to support other systems?
>
> I didn't know it was a POSIX 2008 requirement.  Then indeed, we should
> discuss this with the libunistring folks.

gnulib upstream claims that while they are testing for POSIX 2008, it's
only to behave right if it isn't met.  But the test code fails when NULL
is '(void *)0'.  As I read a reply on a netbsd list, C99 says that NULL
is a constant with value 0 or that vast to void *, and "(void *)0" seems
to meet that requirement quite nicely.

I wonder if it's a gcc bug that 'sizeof (void *) 0' fails.

Regardless, guile aims to be far more portable than POSIX 2008 (which I
had not even heard of until today, and at work I'm one of the chief
ranters making people read posix instead of assuming the way their linux
box behaves is the spec).

[-- Attachment #2: Type: application/pgp-signature, Size: 193 bytes --]

Re: i guess we're frozen & stuff

[Ludovic Courtès]

Greg Troxel <gdt@ir.bbn.com> writes:

> I wonder if it's a gcc bug that 'sizeof (void *) 0' fails.

My understanding of Section A.2.1 of C99 is that both this and
"sizeof ((void *) 0)" are syntactically invalid:

  (6.5.1) primary-expression:
                identifier
                constant
                string-literal
                ( expression )

  (6.5.2) postfix-expression:
                primary-expression
                postfix-expression [ expression ]
                postfix-expression ( argument-expression-listopt )
                postfix-expression . identifier
                postfix-expression -> identifier
                postfix-expression ++
                postfix-expression --
                ( type-name ) { initializer-list }
                ( type-name ) { initializer-list , }

  [...]

  (6.5.3) unary-expression:
                 postfix-expression
                 ++ unary-expression
                 -- unary-expression
                 unary-operator cast-expression
                 sizeof unary-expression
                 sizeof ( type-name )

  (6.5.3) unary-operator: one of
                & * + - ~        !

  (6.5.4) cast-expression:
                 unary-expression
                 ( type-name ) cast-expression


Do you have pointers to the discussions you've had with the Gnulib and
NetBSD people?

Thanks,
Ludo'.

Re: i guess we're frozen & stuff

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 542 bytes --]


http://mail-index.netbsd.org/tech-userlevel/2009/08/thread1.html#002545

http://savannah.gnu.org/support/?106973

My conclusion so far is that "(void *)0" is a legal value for NULL (even
given as an example in POSIX 2008 glossary), and that just because an
expression E evaluates to an integer constant one can not assume that
"sizeof E" is valid.

gnulib has changed to sizeof(NULL) and I think NetBSD is going to change
NULL to add the () as well (to make it less likely people will have
trouble, not because the current value is wrong).


[-- Attachment #2: Type: application/pgp-signature, Size: 193 bytes --]

Re: i guess we're frozen & stuff

[Ludovic Courtès]

Hello!

Greg Troxel <gdt@ir.bbn.com> writes:

> My conclusion so far is that "(void *)0" is a legal value for NULL

Whether "legal" or not, under-parenthesized expression-like macros are
likely to cause troubles one day or another.

Thanks for your work with the upstream people!

Ludo'.

Re: i guess we're frozen & stuff

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 404 bytes --]


ludo@gnu.org (Ludovic Courtès) writes:

>> My conclusion so far is that "(void *)0" is a legal value for NULL
>
> Whether "legal" or not, under-parenthesized expression-like macros are
> likely to cause troubles one day or another.
>
> Thanks for your work with the upstream people!

I now have a pointer to something in POSIX 2008 that says
expression-like macros have to be parenthesized.

[-- Attachment #2: Type: application/pgp-signature, Size: 193 bytes --]

Re: i guess we're frozen & stuff

[Ludovic Courtès]

Greg Troxel <gdt@ir.bbn.com> writes:

>   Does libunistring's "make check" pass on this platform?
>
> should have tried that.  testlocale.c fails on line 38 with:

Can you report it upstream?

> So it seems that NULL is expanding to (void *) 0, and "sizeof (void *)
> 0" is not legit.  AFAIK sizeof is specified to work on variables and
> types, and NULL is neither a variable nor a type.
>
> Is NULL something else on Linux?

On GNU it's "#define NULL ((void *)0)".

Can you try this:

  echo '#include <stdlib.h>' | cpp -dM - | grep NULL

(assuming cpp(1) is from GCC.)

Thanks,
Ludo'.

Re: i guess we're frozen & stuff

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 48 bytes --]


The essential bit is:

#define NULL (void *)0


[-- Attachment #2: Type: application/pgp-signature, Size: 193 bytes --]

Re: i guess we're frozen & stuff

[Ludovic Courtès]

Greg Troxel <gdt@ir.bbn.com> writes:

> #define NULL (void *)0

This is under-parenthesized!

Two actions to be taken:

  1. Report a bug to your favorite libc maintainers.  ;-)

  2. Ask libunistring folks at
     http://savannah.gnu.org/projects/libunistring/ to replace
     "sizeof NULL" by "sizeof (NULL)" in `test-locale.c'.

Thanks,
Ludo'.

Re: i guess we're frozen & stuff

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 703 bytes --]


ludo@gnu.org (Ludovic Courtès) writes:

> Greg Troxel <gdt@ir.bbn.com> writes:
>
>> #define NULL (void *)0
>
> This is under-parenthesized!
>
> Two actions to be taken:
>
>   1. Report a bug to your favorite libc maintainers.  ;-)

I think that's a legal value for NULL per C99.  Interestingly it has not
apparently caused other trouble.  We'll see what others say, and I can
see the point that having it parenthesized is defensive even if it's
legal as is.

>   2. Ask libunistring folks at
>      http://savannah.gnu.org/projects/libunistring/ to replace
>      "sizeof NULL" by "sizeof (NULL)" in `test-locale.c'.

I did, and this has been committed to their repo for gnulib.

[-- Attachment #2: Type: application/pgp-signature, Size: 193 bytes --]

Re: i guess we're frozen & stuff

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 3273 bytes --]


requested tests follow


list gdt 52 ~/BUILD-GUILE-master/guile > ./meta/guile
;;; note: autocompilation is enabled, set GUILE_AUTO_COMPILE=0
;;;       or pass the --no-autocompile argument to disable.
;;; compiling /home/gdt/BUILD-GUILE-master/guile/guile-readline/ice-9/readline.scm
;;; compiled /home/gdt/.cache/guile/ccache/1.9/home/gdt/BUILD-GUILE-master/guile/guile-readline/ice-9/readline.scm.go
Guile Scheme interpreter 0.5 on Guile 1.9.1
Copyright (C) 2001-2008 Free Software Foundation, Inc.

Enter `,help' for help.
scheme@(guile-user)> (use-modules (ice-9 i18n))
scheme@(guile-user)> (make-locale LC_ALL "does-not-exist")
#<locale 7f7ffc1d9ca0>

scheme@(guile-user)> (map locale-day (map 1+ (iota 7)))
("Sun" "Mon" "Tue" "Wed" "Thu" "Fri" "Sat")

scheme@(guile-user)> (map (lambda (day)
         (locale-day day (make-locale LC_ALL "C")))
       (map 1+ (iota 7)))
("Sun" "Mon" "Tue" "Wed" "Thu" "Fri" "Sat")

scheme@(guile-user)> (map (lambda (day)
         (locale-day day %global-locale))
       (map 1+ (iota 7)))
("Sun" "Mon" "Tue" "Wed" "Thu" "Fri" "Sat")


scheme@(guile-user)> (use-modules (srfi srfi-19))
  (let ((d (string->date "Saturday, December 9, 2006"
scheme@(guile-user)>                                             "~A, ~B ~d, ~Y")))
    (date->time-utc (make-date (date-nanosecond d)
                               (date-second d)
                               (date-minute d)
                               (date-hour d)
                               (date-day d)
                               (date-month d)
                               (date-year d)
                               0)))

Backtrace:
In unknown file:
   ?: 0* [#<vm 7f7ffc833ff0> #<program 7f7ffc6dd8c0 at standard input:34:2 ()>]
In standard input:
  35: 1* [#<program 7f7ffc6dd8c0 at standard input:34:2 ()>]
In srfi/srfi-19.scm:
1462: 2  [string->date "Saturday, December 9, 2006" "~A, ~B ~d, ~Y"]
In srfi/srfi-19.scm:
1438: 3  [priv:string->date # 0 "~A, ~B ~d, ~Y" ...]
In srfi/srfi-19.scm:
 176: 4  [priv:time-error string->date bad-date-template-string ...]
In unknown file:
   ?: 5* [throw misc-error misc-error ...]

ERROR: In procedure string->date:
ERROR: TIME-ERROR type bad-date-template-string: ("Invalid string for " #<program priv:locale-long-weekday->index (string)>)


scheme@(guile-user)> (use-modules (srfi srfi-19))
  (let* ((str  "Sun, 05 Jun 2005 18:33:00 +0200")
         (date (string->date str "~a, ~d ~b ~Y ~H:~M:~S ~z")))
   (date->string date))

scheme@(guile-user)> 
Backtrace:
In unknown file:
   ?: 0* [#<vm 7f7ffc833ff0> #<program 7f7ffc14c600 at standard input:45:2 ()>]
In standard input:
  47: 1* [#<program 7f7ffc14c600 at standard input:45:2 ()>]
In srfi/srfi-19.scm:
1462: 2  [string->date "Sun, 05 Jun 2005 18:33:00 +0200" "~a, ~d ~b ~Y ~H:~M:~S ~z"]
In srfi/srfi-19.scm:
1438: 3  [priv:string->date # 0 "~a, ~d ~b ~Y ~H:~M:~S ~z" ...]
In srfi/srfi-19.scm:
 176: 4  [priv:time-error string->date bad-date-template-string ...]
In unknown file:
   ?: 5* [throw misc-error misc-error ...]

ERROR: In procedure string->date:
ERROR: TIME-ERROR type bad-date-template-string: ("Invalid string for " #<program priv:locale-abbr-weekday->index (string)>)

[-- Attachment #2: Type: application/pgp-signature, Size: 193 bytes --]

i18n issues on NetBSD

[Ludovic Courtès]

Greg Troxel <gdt@ir.bbn.com> writes:

> scheme@(guile-user)> (make-locale LC_ALL "does-not-exist")
> #<locale 7f7ffc1d9ca0>

And `(setlocale LC_ALL "does-not-exist")'?

It looks like setlocale(3) doesn't behave as specified by POSIX
(http://www.opengroup.org/onlinepubs/9699919799/functions/setlocale.html).

> scheme@(guile-user)> (map locale-day (map 1+ (iota 7)))
> ("Sun" "Mon" "Tue" "Wed" "Thu" "Fri" "Sat")

It looks like `(nl-langinfo DAY_1)' doesn't DTRT per
http://www.opengroup.org/onlinepubs/9699919799/basedefs/langinfo.h.html .

Just to make sure, can you try this program (assume the `en_GB' locale
is available):

--8<---------------cut here---------------start------------->8---
#include <langinfo.h>
#include <nl_types.h>
#include <locale.h>

#include <stdio.h>
#include <assert.h>


int
main (int argc, char *argv[])
{
  char *s;

  s = setlocale (LC_ALL, "C");
  assert (s != NULL);

  printf ("DAY_1: %s\n", nl_langinfo (DAY_1));

  s = setlocale (LC_ALL, "en_GB");
  assert (s != NULL);

  printf ("DAY_1: %s\n", nl_langinfo (DAY_1));

  return 0;
}
--8<---------------cut here---------------end--------------->8---

> ERROR: In procedure string->date:
> ERROR: TIME-ERROR type bad-date-template-string: ("Invalid string for " #<program priv:locale-long-weekday->index (string)>)

This is actually the same problem.

Thanks,
Ludo'.

Re: i18n issues on NetBSD

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 770 bytes --]


scheme@(guile-user)> (setlocale LC_ALL "does-not-exist")  
"C"



list gdt 8 ~ > cat l.c
#include <langinfo.h>
#include <nl_types.h>
#include <locale.h>

#include <stdio.h>
#include <assert.h>


int
main (int argc, char *argv[])
{
  char *s;

  s = setlocale (LC_ALL, "C");
  assert (s != NULL);

  printf ("DAY_1: %s\n", nl_langinfo (DAY_1));

  s = setlocale (LC_ALL, "en_GB");
  assert (s != NULL);

  printf ("DAY_1: %s\n", nl_langinfo (DAY_1));

  return 0;
}
list gdt 9 ~ > cc -o l l.c
list gdt 10 ~ > ./l
DAY_1: Sun
DAY_1: Sun



But if I run this on a linux box I get

gdt 78 ~ > ./l
DAY_1: Sunday
l: l.c:20: main: Assertion `s != ((void *)0)' failed.
Aborted


I'm afraid I don't really understand i18n details.

[-- Attachment #2: Type: application/pgp-signature, Size: 193 bytes --]

Re: i18n issues on NetBSD

[Ludovic Courtès]

Greg Troxel <gdt@ir.bbn.com> writes:

> scheme@(guile-user)> (setlocale LC_ALL "does-not-exist")  
> "C"

This looks wrong to me:

  Upon successful completion, setlocale() shall return the string
  associated with the specified category for the new locale.  Otherwise,
  setlocale() shall return a null pointer and the locale of the process
  is not changed.

  http://www.opengroup.org/onlinepubs/9699919799/functions/setlocale.html

Here we do not expect "successful completion", so it "shall return a
null pointer".

Can you submit a bug report against NetBSD's C library?

> list gdt 8 ~ > cat l.c
> #include <langinfo.h>
> #include <nl_types.h>
> #include <locale.h>
>
> #include <stdio.h>
> #include <assert.h>
>
>
> int
> main (int argc, char *argv[])
> {
>   char *s;
>
>   s = setlocale (LC_ALL, "C");
>   assert (s != NULL);
>
>   printf ("DAY_1: %s\n", nl_langinfo (DAY_1));
>
>   s = setlocale (LC_ALL, "en_GB");
>   assert (s != NULL);
>
>   printf ("DAY_1: %s\n", nl_langinfo (DAY_1));
>
>   return 0;
> }
> list gdt 9 ~ > cc -o l l.c
> list gdt 10 ~ > ./l
> DAY_1: Sun
> DAY_1: Sun

Both look wrong to me: it returns the abbreviated name of the day
instead of the full name of the day
(http://www.opengroup.org/onlinepubs/9699919799/basedefs/langinfo.h.html).

Can you report a bug for this one, too?

Thanks,
Ludo'.

Re: i guess we're frozen & stuff

[Juhani Viheräkoski]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 989 bytes --]

Hi,

For me the build of the current master fails with:

libtool: compile:  gcc -DHAVE_CONFIG_H -DBUILDING_LIBGUILE=1 -I.. -I.. 
-I../lib -I../lib -pthread -Wall -Wmissing-prototypes -Werror 
-fvisibility=hidden -O3 -g -march=athlon-xp -MT libguile_la-strings.lo -MD 
-MP -MF .deps/libguile_la-strings.Tpo -c strings.c  -fPIC -DPIC -o 
.libs/libguile_la-strings.o
cc1: warnings being treated as errors
strings.c: In function ‘scm_string_append’:
strings.c:1300: error: ‘data’ may be used uninitialized in this 
function
make[3]: *** [libguile_la-strings.lo] Error 1

It is easy to disable -Werror or set the offending variable to NULL, but 
with my limited knowledge of Guile internals it seems that this 
uninitialized pointer is indeed passed to scm_i_make_string or 
scm_i_make_wide_string which do not seem to reserve space for it and 
even dereference the pointer. Might be just me, but it seems there is 
something weird going on here..

-- 
Juhani

Re: i guess we're frozen & stuff

[Mike Gran]

> From: Juhani Viheräkoski moonshine@kapsi.fi

> Hi,
> 
> For me the build of the current master fails with:
> 
> libtool: compile:  gcc -DHAVE_CONFIG_H -DBUILDING_LIBGUILE=1 -I.. -I.. 
-I../lib 
> -I../lib -pthread -Wall -Wmissing-prototypes -Werror -fvisibility=hidden -O3 
-g 
> -march=athlon-xp -MT libguile_la-strings.lo -MD -MP -MF 
> .deps/libguile_la-strings.Tpo -c strings.c  -fPIC -DPIC -o 
> .libs/libguile_la-strings.o
> cc1: warnings being treated as errors
> strings.c: In function ‘scm_string_append’:
> strings.c:1300: error: ‘data’ may be used uninitialized in this function
> make[3]: *** [libguile_la-strings.lo] Error 1
> 
> It is easy to disable -Werror or set the offending variable to NULL, but with 
my 
> limited knowledge of Guile internals it seems that this uninitialized pointer 
is 
> indeed passed to scm_i_make_string or scm_i_make_wide_string which do not seem 

> to reserve space for it and even dereference the pointer. Might be just me, 
but 
> it seems there is something weird going on here..
> 

This one's okay, I think.  The allocation is made in make_stringbuf and `data' 
is set to point to its buffer.

We'll have to set it to NULL in scm_string_append and other places to quiet 
warnings.

> -- Juhani

Thanks,

-Mike

Re: i guess we're frozen & stuff

[Ludovic Courtès]

Mike Gran <spk121@yahoo.com> writes:

>> From: Juhani Viheräkoski moonshine@kapsi.fi

[...]

>> strings.c: In function ‘scm_string_append’:
>> strings.c:1300: error: ‘data’ may be used uninitialized in this function
>> make[3]: *** [libguile_la-strings.lo] Error 1

[...]

> This one's okay, I think.  The allocation is made in make_stringbuf and `data' 
> is set to point to its buffer.
>
> We'll have to set it to NULL in scm_string_append and other places to quiet 
> warnings.

Yes.  Or use "union { char *narrow; scm_t_wchar *wide; }" to make things
obvious.

Thanks,
Ludo'.

Re: i guess we're frozen & stuff

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 2126 bytes --]


guile master fails to build on NetBSD because it passes characters to
tolower, which is specified to take an int.  It's really a macro, and
this is a messy situation.  The language weenies I've talked to about
this think that in this case it's the program that passes a char that's
wrong.  Hence the following patch.  Note that because tolower is
specified to take an int, this promotion to int is what would happen if
it really were a function, so this "can't be wrong" :-)

See

  http://www.opengroup.org/onlinepubs/000095399/functions/tolower.html

and the statement about the argument being of type int and being of a
restricted set of values.


diff --git a/libguile/strings.c b/libguile/strings.c
index c3ea8b8..437cedc 100644
--- a/libguile/strings.c
+++ b/libguile/strings.c
@@ -1427,8 +1427,8 @@ unistring_escapes_to_guile_escapes (char **bufp, size_t *lenp)
           /* Convert \u00NN to \xNN */
           after[j] = '\\';
           after[j + 1] = 'x';
-          after[j + 2] = tolower (before[i + 4]);
-          after[j + 3] = tolower (before[i + 5]);
+          after[j + 2] = tolower ((int) before[i + 4]);
+          after[j + 3] = tolower ((int) before[i + 5]);
           i += 6;
           j += 4;
         }
@@ -1440,12 +1440,12 @@ unistring_escapes_to_guile_escapes (char **bufp, size_t *lenp)
           /* Convert \U00NNNNNN to \UNNNNNN */
           after[j] = '\\';
           after[j + 1] = 'U';
-          after[j + 2] = tolower (before[i + 4]);
-          after[j + 3] = tolower (before[i + 5]);
-          after[j + 4] = tolower (before[i + 6]);
-          after[j + 5] = tolower (before[i + 7]);
-          after[j + 6] = tolower (before[i + 8]);
-          after[j + 7] = tolower (before[i + 9]);
+          after[j + 2] = tolower ((int) before[i + 4]);
+          after[j + 3] = tolower ((int) before[i + 5]);
+          after[j + 4] = tolower ((int) before[i + 6]);
+          after[j + 5] = tolower ((int) before[i + 7]);
+          after[j + 6] = tolower ((int) before[i + 8]);
+          after[j + 7] = tolower ((int) before[i + 9]);
           i += 10;
           j += 8;
         }

[-- Attachment #2: Type: application/pgp-signature, Size: 193 bytes --]

unsigned char confusion

[Greg Troxel]

[-- Attachment #1: Type: text/plain, Size: 870 bytes --]


In srfi-13.c line 25222, SCM_MAKE_CHAR is called with an argument that
is an unsigned char.  This leads to:

cc1: warnings being treated as errors
srfi-13.c: In function 'string_titlecase_x':
srfi-13.c:2522: warning: comparison is always false due to limited range of data type
srfi-13.c:2522: warning: comparison is always false due to limited range of data type

This is because SCM_MAKE_CHAR (in libguile/chars.h) has a bizarre
conditional that checks the argument for < 0, and if so casts it to
unsigned char.  Otherwise it does not cast.  There is no comment that
explains what the point is.  Fairly obviously this is an attempt to
avoid sign extension during SCM_MAKE_ITAG8.  The value is then cast to
uintptr_t which is also unsigned, but sign extension would set more
bits.

So, I think the cast to unsigned char should just always be there, without the test.


[-- Attachment #2: Type: application/pgp-signature, Size: 193 bytes --]

Re: unsigned char confusion

[Mike Gran]

On Tue, 2009-08-11 at 09:39 -0400, Greg Troxel wrote:
> In srfi-13.c line 25222, SCM_MAKE_CHAR is called with an argument that
> is an unsigned char.  This leads to:
> 
> cc1: warnings being treated as errors
> srfi-13.c: In function 'string_titlecase_x':
> srfi-13.c:2522: warning: comparison is always false due to limited range of data type
> srfi-13.c:2522: warning: comparison is always false due to limited range of data type
> 
> This is because SCM_MAKE_CHAR (in libguile/chars.h) has a bizarre
> conditional that checks the argument for < 0, and if so casts it to
> unsigned char.  Otherwise it does not cast.  There is no comment that
> explains what the point is.  Fairly obviously this is an attempt to
> avoid sign extension during SCM_MAKE_ITAG8.  The value is then cast to
> uintptr_t which is also unsigned, but sign extension would set more
> bits.
> 
> So, I think the cast to unsigned char should just always be there, without the test.
> 

Yeah, that was me.   In the move to Unicode, I'm trying to get to a
point where the underlying storage of characters is uint32.  I was
trying to come up with a macro that would cast all of char, unsigned
char, and uint32 to uint32, since SCM_MAKE_CHAR is used in each of those
cases in the code.  If SCM_MAKE_CHAR receives something negative, it is
from a signed char.

For portability, it might be best if SCM_MAKE_CHAR becomes an inline
function that takes int32, since the top bit of uint32 isn't used in
encoding Unicode codepoints anyway.  That would cover all those cases.

Or, to save the macro, it could become

#define SCM_MAKE_CHAR(x)                                              \
  (((scm_t_int32) (x) < 0)                                            \
   ? SCM_MAKE_ITAG8 ((scm_t_bits) (unsigned char) (x), scm_tc8_char)  \
   : SCM_MAKE_ITAG8 ((scm_t_bits) (x), scm_tc8_char))

Yeah, and better comments for that as well.

-Mike

Re: i guess we're frozen & stuff

[Ken Raeburn]

Just checking: I assume we don't care about binary compatibility  
between 1.9.x unstable releases?  Hence changing function signatures  
and structure contents, and deleting symbols, while not changing the  
library's major version number, is okay?

Ken

Re: i guess we're frozen & stuff

[Ludovic Courtès]

Ken Raeburn <raeburn@raeburn.org> writes:

> Just checking: I assume we don't care about binary compatibility
> between 1.9.x unstable releases?  Hence changing function signatures
> and structure contents, and deleting symbols, while not changing the
> library's major version number, is okay?

Yes.

Ludo'.

Merging ‘bdw-gc-static-alloc’

[Ludovic Courtès]

Hello!

Following the inlining of stringbuf storage
(ba54a2026beaadb4e7566d4b9e2c9e4c7cd793e6), the ‘bdw-gc-static-alloc'
branch introduces a small source-level incompatibility.  Namely, the
following no longer works:

  const char foo[] = "foo";
  SCM_SYMBOL (s_foo, foo);

Because of the macrology that’s used to statically allocate stringbufs [0],
the string has to be a literal:

  SCM_SYMBOL (s_foo, "foo");

The use of non-literal strings occurred only in ‘eval.c’ in Guile proper [1].

The question is: should we merge ‘bdw-gc-static-alloc’ despite this
incompatibility (it can be argued that this won’t hurt many people and
can be easily worked around)?

If the answer is “no”, we still have the option of merging other aspects
of the ‘bdw-gc-static-alloc’ branch, such as static allocation of subrs.

What do you think?

Thanks,
Ludo’.

[0] http://git.sv.gnu.org/gitweb/?p=guile.git;a=blobdiff;f=libguile/snarf.h;h=9eaccf60c14d825e657718e6f75257e293649f84;hp=c3113e1a7a977f95eb922e924855244dfe4d6cc2;hb=5f236208d0d864546e59afa0f5a11c9b3ba14b10;hpb=f0eb5ae6c173aed35965b0561897fda1d8ff0db1
[1] http://git.sv.gnu.org/gitweb/?p=guile.git;a=blobdiff;f=libguile/eval.c;h=e58c05410c2831d4061eb92b4ee2883c9203c919;hp=59db42976242e12fd41995e5e1fa5d169bd3b9bf;hb=5f236208d0d864546e59afa0f5a11c9b3ba14b10;hpb=d7e7a02a6251c8ed4f76933d9d30baeee3f599c0

Re: Merging ‘bdw-gc-static-alloc’

[Andy Wingo]

On Fri 02 Oct 2009 02:15, ludo@gnu.org (Ludovic Courtès) writes:

> Because of the macrology that’s used to statically allocate stringbufs [0],
> the string has to be a literal:
>
>   SCM_SYMBOL (s_foo, "foo");
>
> What do you think?

Need more information :) How widespread is the non-literal usage? Could
you grep the source packages of the Guile users that you have identified
for occurrences of this idiom?

Andy
-- 
http://wingolog.org/

Re: Merging ‘bdw-gc-static-alloc’

[Ludovic Courtès]

Hi,

Andy Wingo <wingo@pobox.com> writes:

> On Fri 02 Oct 2009 02:15, ludo@gnu.org (Ludovic Courtès) writes:
>
>> Because of the macrology that’s used to statically allocate stringbufs [0],
>> the string has to be a literal:
>>
>>   SCM_SYMBOL (s_foo, "foo");
>>
>> What do you think?
>
> Need more information :) How widespread is the non-literal usage? Could
> you grep the source packages of the Guile users that you have identified
> for occurrences of this idiom?

Apparently this is going to be rare.  I’ve found no occurrences here:

  http://www.google.fr/codesearch?q=SCM_%28GLOBAL_%29%3FSYMBOL&hl=fr&btnG=Rechercher+du+code

“grep -r -E \
  'SCM_(GLOBAL_)?SYMBOL[[:blank:]]*\([^,]+,[[:blank:]]*[^"]+\)' \
  gnucash-2.2.9 guile-avahi guile-cairo-1.4.0 \
  guile-gnome-platform-2.16.1 gnutls mailutils-2.1 snd-9.4”
doesn’t show any occurrence either.

So I’d be tempted to merge this in.

What do you think?

Ludo’.

Re: Merging ‘bdw-gc-static-alloc’

[Andy Wingo]

EHLO

On Fri 23 Oct 2009 11:16, ludo@gnu.org (Ludovic Courtès) writes:

> Andy Wingo <wingo@pobox.com> writes:
>
>> On Fri 02 Oct 2009 02:15, ludo@gnu.org (Ludovic Courtès) writes:
>>
>>> Because of the macrology that’s used to statically allocate stringbufs [0],
>>> the string has to be a literal:
>>>
>>>   SCM_SYMBOL (s_foo, "foo");
>>>
>>> What do you think?
>>
>> Need more information :) How widespread is the non-literal usage? Could
>> you grep the source packages of the Guile users that you have identified
>> for occurrences of this idiom?
>
> Apparently this is going to be rare.  I’ve found no occurrences here:
>
> So I’d be tempted to merge this in.
>
> What do you think?

Roll on, merge train, roll on!

I have finished case-lambda, but am rebasing and cleaning up patches I
haven't pushed yet. So it's a race. Gentlemen, start your git rebase &
push!

A
-- 
http://wingolog.org/

Re: Merging ‘bdw-gc-static-alloc’

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 592 bytes --]

Hello!

I merged that branch in ‘master’.

... and soon after I noticed this glitch with ‘SCM_DEFINE’: when
‘SCM_SUPPORT_STATIC_ALLOCATION’ is defined, it requires a previous C
declaration of the subr being defined.  For instance:

  SCM_DEFINE (foo, "foo", 1, 0, 0, (SCM x), "") { ... }

must now be preceded by:

  extern SCM foo (SCM);

This is not a problem in Guile core where we compile with
‘-Wmissing-prototypes’ and provide public declarations for all
‘SCM_DEFINE’d functions, but it may be a problem in projects that don’t.

This is easily remedied this way:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 624 bytes --]

--- a/libguile/snarf.h
+++ b/libguile/snarf.h
@@ -101,14 +101,15 @@ SCM_SNARF_DOCS(primitive, FNAME, PRIMNAME, ARGLIST, REQ, OPT, VAR, DOCSTRING)
 /* Static subr allocation.  */
 #define SCM_DEFINE(FNAME, PRIMNAME, REQ, OPT, VAR, ARGLIST, DOCSTRING)	\
 SCM_SYMBOL (scm_i_paste (FNAME, __name), PRIMNAME);			\
-SCM_SNARF_HERE(								\
+SCM_SNARF_HERE(							\
   static const char scm_i_paste (s_, FNAME) [] = PRIMNAME;		\
+  extern SCM FNAME ARGLIST;						\
   SCM_IMMUTABLE_SUBR (scm_i_paste (FNAME, __subr),			\
 		      scm_i_paste (FNAME, __name),			\
 		      REQ, OPT, VAR, &FNAME);				\
   SCM FNAME ARGLIST							\

[-- Attachment #3: Type: text/plain, Size: 679 bytes --]


However, there are potentially 2 problems with this:

  1. The declaration could conflict with a previous, slightly different
     one.

     Apparently, a given function declaration and the same one decorated
     with GCC function attributes are considered the same, so this
     should be OK.  However, I’m slightly concerned about MSVC’s
     __declspec: does it work if it sees:

       __declspec(dllexport) extern SCM foo (SCM);
       extern SCM foo (SCM);

     If it works, that probably means this point is moot.

  2. The automatically added ‘extern’ declaration makes
     ‘-Wmissing-prototypes’ useless, which is annoying.

Thoughts?

Thanks,
Ludo’.

(no subject)

[Neil Jerram]

ludo@gnu.org (Ludovic Courtès) writes:

>      should be OK.  However, I’m slightly concerned about MSVC’s
>      __declspec: does it work if it sees:
>
>        __declspec(dllexport) extern SCM foo (SCM);
>        extern SCM foo (SCM);

I don't know for sure, but I know I've seen "function redeclared with
different linkage" warnings in the past, and this could well be one of
the situations that generates such warnings.

But what about using SCM_API instead of extern?  Wouldn't that always
generate the right code?  (At least for building guile itself.)

>      If it works, that probably means this point is moot.
>
>   2. The automatically added ‘extern’ declaration makes
>      ‘-Wmissing-prototypes’ useless, which is annoying.

I guess it depends if the primitives in question are designed to be used
only from Scheme, or also from other C files.  If they're only for
Scheme, it's a nice feature that SCM_DEFINE does everything needed and
the developer doesn't need a matching prototype.  If they're for other C
files, the missing prototype warning could be helpful.

Then again, the developer will get some kind of warning anyway when
trying to use the function from another file.  So I think the balance is
in favour of SCM_DEFINE including the prototype.

   Neil

Re: Merging ‘bdw-gc-static-alloc’

[Ludovic Courtès]

Hi Neil,

Neil Jerram <neil@ossau.uklinux.net> writes:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>>      should be OK.  However, I’m slightly concerned about MSVC’s
>>      __declspec: does it work if it sees:
>>
>>        __declspec(dllexport) extern SCM foo (SCM);
>>        extern SCM foo (SCM);
>
> I don't know for sure, but I know I've seen "function redeclared with
> different linkage" warnings in the past, and this could well be one of
> the situations that generates such warnings.

Right.

> But what about using SCM_API instead of extern?  Wouldn't that always
> generate the right code?  (At least for building guile itself.)

You’re right.  It’ll work for building Guile itself, but most probably
fail when building external libraries, because it will have
SCM_API==dllimport whereas the symbols of the lib being built want
dllexport.

I have no idea how to fix this since third party libraries typically
have their own ‘SCM_API’-like macro for that purpose.

Well, we could just disable ‘SCM_SUPPORT_STATIC_ALLOCATION’ altogether
for “Woe32” builds.

What do you think?

In the meantime, I’ve pushed this:
http://git.savannah.gnu.org/cgit/guile.git/commit/?id=f4767979987530bb9d88dde6ebe61cdac6f756b9 .

> I guess it depends if the primitives in question are designed to be used
> only from Scheme, or also from other C files.  If they're only for
> Scheme, it's a nice feature that SCM_DEFINE does everything needed and
> the developer doesn't need a matching prototype.  If they're for other C
> files, the missing prototype warning could be helpful.

Yes, although ideally functions only for Scheme would be either static
or with internal linkage, which calls for an ‘SCM_DEFINE_LOCAL’ macro or
similar.

> Then again, the developer will get some kind of warning anyway when
> trying to use the function from another file.  So I think the balance is
> in favour of SCM_DEFINE including the prototype.

Yes.

Thanks,
Ludo’.

(no subject)

[Neil Jerram]

ludo@gnu.org (Ludovic Courtès) writes:

> Hi Neil,
>
> Neil Jerram <neil@ossau.uklinux.net> writes:
>
>> But what about using SCM_API instead of extern?  Wouldn't that always
>> generate the right code?  (At least for building guile itself.)
>
> You’re right.  It’ll work for building Guile itself, but most probably
> fail when building external libraries, because it will have
> SCM_API==dllimport whereas the symbols of the lib being built want
> dllexport.
>
> I have no idea how to fix this since third party libraries typically
> have their own ‘SCM_API’-like macro for that purpose.
>
> Well, we could just disable ‘SCM_SUPPORT_STATIC_ALLOCATION’ altogether
> for “Woe32” builds.
>
> What do you think?

We're talking here about macros that are always used in .c files, aren't
we?  (As opposed to headers that an application using the library would
include.)

So on Woe32, we actually always want dllexport.

i.e. if the header has
SCM_API SCM scm_frobate (SCM arg);

and the SCM_DEFINE in .c expands to something that includes
__declspec(dllexport) SCM scm_frobate (SCM arg);

that should be good, because SCM_API will be __declspec(dllexport) when
building the library.

In the Scheme-only case, that does mean exporting something that doesn't
need exporting - but either we can just live with that, or later add
your suggestion of SCM_DEFINE_LOCAL.

Regards,
        Neil

(no subject)

[Tanisha Villalobos]

Hi again,

Here is Tanisha Villalobos. I write to you because we are accepting your m=
ortgage application.
Our office confirms you can get a $220.000 lo=C0n for a $252.00 per month =
payment.
Approval process will take 1 minute, so please fill out the form on our we=
bsite:


http://lockian-growth.refi-talk.com


Thank you.

Best Regards Tanisha Villalobos
First Account Manager





_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

(no subject)

[Karen Ferguson]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1: Type: text/plain; charset="iso-5748-8", Size: 401 bytes --]

Hi again,

Here is Karen Ferguson. I wite you because we are accepting your mortgage application.
Our office confirms you can get a $220.000 loÀn for a $352.00 per month payment.
Approval process will take 1 minute, so please fill out the form on our website:

http://methodic.moneyintelligent.info/j8/o0o.php?jq1=101



Thank you.

Best Regards Karen Ferguson
First Account Manager



[-- Attachment #2: Type: text/plain, Size: 143 bytes --]

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/guile-devel

(no subject)

[Dirk Herrmann]

Clinton Ebadi wrote:
> > I don't care how memoization works.  I understand there are
> > optimization issues.  Optimizing isn't important to me, or
> > I would not use an interpretive language as my "extension
> > language".  If you add a disabling mechanism, then you'll
> > have the best of all worlds.  You can memoize/optimize to
> > your heart's content, and I'll disable it for my purposes.
> > If that is not practical and Guile becomes, essentially,
> > an semi-compiled language with constraints on where
> > defines may happen, then my goose will be cooked.
>
> Not to mention how Guile wouldn't be Scheme anymore. Why can't code be
> memiozied and compiled incrementally?

First, why wouldn't guile be scheme any more if conditional definitions
are disallowed?  They are disallowed by R5RS, so actually, if you
understand "being scheme" as "conforming to R5RS" then currently guile is
not scheme and disallowing conditional definitions will bring it one step
closer to "being scheme" :-)

Second, certainly can code be memoized and compiled incrementally.
However, this would have to be done every time the code was read in again,
since in an expression like (if <condition> (define <foo> <bar)) the value
of <condition> may change every time the code is read in.  That is, we
would reduce our possibility to pre-compile code.  Pre compiling code,
however, is one of our options to speed up guile's starting time (aside
from optimizing guile's module system...).

Best regards, 
Dirk Herrmann



_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: (no subject)

[Clinton Ebadi]

On Saturday 09 November 2002 03:29, Dirk Herrmann wrote:
> > Not to mention how Guile wouldn't be Scheme anymore. Why can't code be
> > memiozied and compiled incrementally?
>
> First, why wouldn't guile be scheme any more if conditional definitions
> are disallowed?  They are disallowed by R5RS, so actually, if you
> understand "being scheme" as "conforming to R5RS" then currently guile is
> not scheme and disallowing conditional definitions will bring it one step
> closer to "being scheme" :-)

I was referring to the part where you would have to reload your code if a 
binding changed and the whole (define define define-private) etc. not 
working.

-- 
http://unknownlamer.org
Truth lies in loneliness
When hope is long gone by


_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

(no subject)

[Han-Wen]

Subject: status: separation of expansion/optimization/memoization/execution
In-Reply-To: <Pine.GSO.4.05.10208022349150.5209-100000@sallust.ida.ing.tu-bs.de>
References: <Pine.GSO.4.05.10208022349150.5209-100000@sallust.ida.ing.tu-bs.de>
X-Mailer: VM 7.05 under Emacs 21.2.1
Reply-To: hanwen@cs.uu.nl
FCC: ~/persoonlijk/Mail/sent

dirk@sallust.ida.ing.tu-bs.de writes:
> Hi folks,
>  [ eval hacking ]

I'm just wondering -- Some time ago Keisuke Nishida (sp?) was working on
a byte compiler for GUILE that offered major speedups in many
cases. Is that still being pursued?

--
Han-Wen Nienhuys   |   hanwen@cs.uu.nl   |   http://www.cs.uu.nl/~hanwen 

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

Re: (no subject)

[Rob Browning]

Han-Wen <hanwen@cs.uu.nl> writes:

> I'm just wondering -- Some time ago Keisuke Nishida (sp?) was
> working on a byte compiler for GUILE that offered major speedups in
> many cases. Is that still being pursued?

Not at the moment.  He had to stop working on that (at least for a
while), and I can't recall if he though he'd be able to get back to
it.

Regardless, I think the general plan was to first get the evaluation
process cleaned up a bit and then step back and consider what comes
next.  There as been discussion of scm->c, jit, lightning-based-jit,
gcc-front-end-based[1], and traditional byte compilation.  Once the
evaluation process better separates memoization and execution from
each other and from the other steps (a la Dirk's current work).  It
should be a lot easier to explore the possibilities.

[1] which with the addition of -foptimiize-sibling is now at least one
    step more likely to be feasible, though perhaps still not the
    right idea.  I'm still not sure that gcc's intermediate language
    has all the features you'd really need.

    Interesting related link:
      http://gcc.gnu.org/frontends.html -- see FLIM -- a realy simple
      example gcc front end, and Ksi, a compiler from a very thin sexp
      representation of gcc's intermedate trees to machine code, and
      Gont a high level language that spits out forms to be processed
      by Ksi on the back-end.

-- 
Rob Browning
rlb @defaultvalue.org, @linuxdevel.com, and @debian.org
Previously @cs.utexas.edu
GPG=1C58 8B2C FB5E 3F64 EA5C  64AE 78FE E5FE F0CB A0AD

_______________________________________________
Guile-devel mailing list
Guile-devel@gnu.org
http://mail.gnu.org/mailman/listinfo/guile-devel

(no subject)

[±è°æÀÏ]

[-- Attachment #1: Type: text/html, Size: 4702 bytes --]