bug#10780: 24.0.93 fails to build with 'Invalid function: "DEAD"'

bug#10780: 24.0.93 fails to build with 'Invalid function: "DEAD"'

[Dan Horák]

Hi,

emacs 24.0.93 (and 24.0.92) fails to build on ppc and s390 (32-bit
big-endians) with

make[3]: Entering directory
`/builddir/build/BUILD/emacs-24.0.93/build-gtk/lisp' 
cd /builddir/build/BUILD/emacs-24.0.93/lisp; subdirs=`(find . -type d -print)`; for file in $subdirs; do case $file in */.* | */.*/* | */=* | */cedet* ) ;; *) wins="$wins $file" ;; esac; done; \
for file in $wins; do \
   /builddir/build/BUILD/emacs-24.0.93/update-subdirs $file; \
done;
make[3]: Leaving directory `/builddir/build/BUILD/emacs-24.0.93/build-gtk/lisp'
if test "no" = "yes"; then \
  ln -f temacs bootstrap-emacs; \
else \
  `/bin/pwd`/temacs --batch --load loadup bootstrap || exit 1; \
  mv -f emacs bootstrap-emacs; \
fi
Loading loadup.el (source)...
Using load-path (/builddir/build/BUILD/emacs-24.0.93/lisp /builddir/build/BUILD/emacs-24.0.93/lisp/emacs-lisp /builddir/build/BUILD/emacs-24.0.93/lisp/language /builddir/build/BUILD/emacs-24.0.93/lisp/international /builddir/build/BUILD/emacs-24.0.93/lisp/textmodes)
Loading emacs-lisp/byte-run (source)...
Loading emacs-lisp/backquote (source)...
Loading subr (source)...
Invalid function: "DEAD"
make[2]: Leaving directory `/builddir/build/BUILD/emacs-24.0.93/build-gtk/src'
make[2]: *** [bootstrap-emacs] Error 1
make[1]: Leaving directory `/builddir/build/BUILD/emacs-24.0.93/build-gtk'
make[1]: *** [src] Error 2
make: *** [bootstrap] Error 2
RPM build errors:
error: Bad exit status from /var/tmp/rpm-tmp.SbWTXU (%build)
    Bad exit status from /var/tmp/rpm-tmp.SbWTXU (%build)
Child returncode was: 1
EXCEPTION: Command failed. See logs for output.
...

Build environment was Fedora 17 with gcc 4.7, but also fails on RHEL-6.
Builds on ppc64 and s390x were successful.

bug#10780: 24.0.93 fails to build with 'Invalid function: "DEAD"'

[Glenn Morris]

Dan Horák wrote:

> Invalid function: "DEAD"

Maybe see http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10749

bug#10749: 24.0.93 fails to build with 'Invalid function: "DEAD"'

[Paul Eggert]

I can't reproduce the problem on my bigendian 32-bit host
(Solaris 10, 32-bit sparc, Sun C 5.12 2011/11/16),
so the problem can't be just the 32-bit bigendianness.

The symptoms are those of a failure during conservative
garbage collection, and my suspicion is that the
conservative marking isn't picking up some register that's
hiding in a setjmp buffer or whatnot.

Suppose you do a "make clean" and then
"make CPPFLAGS=-DGC_LISP_OBJECT_ALIGNMENT=2" and/or
"make CPPFLAGS=-DGC_LISP_OBJECT_ALIGNMENT=1".
Does that work around the problem?

Likewise, suppose you do a "make clean"
and then rebuild with compiler optimization disabled.
Does that work around the problem?

I'll CC: this to bug 10749, since GC_LISP_OBJECT_ALIGNMENT=1
may work around its problems too.

bug#10749: 24.0.93 fails to build with 'Invalid function: "DEAD"'

[Dan Horák]

Paul Eggert píše v Út 14. 02. 2012 v 23:24 -0800: 
> I can't reproduce the problem on my bigendian 32-bit host
> (Solaris 10, 32-bit sparc, Sun C 5.12 2011/11/16),
> so the problem can't be just the 32-bit bigendianness.
> 
> The symptoms are those of a failure during conservative
> garbage collection, and my suspicion is that the
> conservative marking isn't picking up some register that's
> hiding in a setjmp buffer or whatnot.
> 
> Suppose you do a "make clean" and then
> "make CPPFLAGS=-DGC_LISP_OBJECT_ALIGNMENT=2" and/or
> "make CPPFLAGS=-DGC_LISP_OBJECT_ALIGNMENT=1".
> Does that work around the problem?
> 
> Likewise, suppose you do a "make clean"
> and then rebuild with compiler optimization disabled.
> Does that work around the problem?
> 
> I'll CC: this to bug 10749, since GC_LISP_OBJECT_ALIGNMENT=1
> may work around its problems too.

I have to do some more testing but now it seems that using
--with-wide-int option for configure provokes the "Invalid function"
behaviour. When the option is not given, the build is successful, when
given, then I see a segfault in the non-bootstrap build.


Dan

bug#10780: 24.0.93 fails to build with 'Invalid function: "DEAD"'

[Paul Eggert]

On 02/14/2012 11:39 PM, Dan Horák wrote:
> I have to do some more testing but now it seems that using
> --with-wide-int option for configure provokes the "Invalid function"
> behaviour.

Thanks for looking into it.  I configured --with-wide-int on
my bigendian 32-bit host and could not reproduce the problem.
My host is a Solaris 10 sparc (32-bit).  I tried with both
Sun C 5.12 SunOS_sparc 2011/11/16 (cc -xO4) and with
GCC 3.4.3 (csl-sol210-3_4-branch+sol_rpath) (gcc -O2).

Since you built --with-wide-int, can you please also try
compiling from scratch, with
"make CPPFLAGS=-DGC_LISP_OBJECT_ALIGNMENT=4"?
It could be that your compilers claim to be aligning 64-bit
integers on 8-byte boundaries, but are occasionally actually
aligning them on 4-byte boundaries.  This messes up conservative
garbage collection big-time and would explain your symptoms.

bug#10780: 24.0.93 fails to build with 'Invalid function: "DEAD"'

[Andreas Schwab]

Paul Eggert <eggert@cs.ucla.edu> writes:

> It could be that your compilers claim to be aligning 64-bit
> integers on 8-byte boundaries, but are occasionally actually
> aligning them on 4-byte boundaries.  This messes up conservative
> garbage collection big-time and would explain your symptoms.

What about local variables in registers?  They are split between two
registers, but mark_stack does not handle that.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#10780: 24.0.93 fails to build with 'Invalid function: "DEAD"'

[Paul Eggert]

On 02/15/2012 02:09 AM, Andreas Schwab wrote:

> What about local variables in registers?  They are split between two
> registers, but mark_stack does not handle that.

True.  Teerrroooo.  Thanks for diagnosing that.

I pushed the following fix into the trunk.  It's needed
regardless of this particular bug.  Dan, can you please
check whether it indeed fixes the bug for you?  Thanks.


Fix crash due to non-contiguous EMACS_INT (Bug#10780).
* lisp.h (VALBITS): Move definition up, so that USE_LSB_TAG can use it.
(USE_LSB_TAG): Do not define if UINTPTR_MAX >> VALBITS == 0.
It's useless in that case, and it can cause problems on hosts
that allocate halves of EMACS_INT values separately.
Reported by Dan Horák.  Diagnosed by Andreas Schwab in
<http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10780#30>.
* mem-limits.h (EXCEEDS_LISP_PTR): Define to 0 on hosts where
UINTPTR_MAX >> VALBITS == 0.  This is required by the above change;
it avoids undefined behavior on hosts where shifting right by more
than the word width has undefined behavior.
=== modified file 'src/lisp.h'
--- src/lisp.h	2012-01-19 07:21:25 +0000
+++ src/lisp.h	2012-02-20 16:27:49 +0000
@@ -168,6 +168,10 @@
 #define GCTYPEBITS 3
 #endif

+#ifndef VALBITS
+#define VALBITS (BITS_PER_EMACS_INT - GCTYPEBITS)
+#endif
+
 #ifndef NO_DECL_ALIGN
 # ifndef DECL_ALIGN
 #  if HAVE_ATTRIBUTE_ALIGNED
@@ -191,7 +195,15 @@
      || defined DARWIN_OS || defined __sun)
 /* We also need to be able to specify mult-of-8 alignment on static vars.  */
 # if defined DECL_ALIGN
-#  define USE_LSB_TAG
+/* mark_maybe_object assumes that EMACS_INT values are contiguous,
+   but this is not true on some hosts where EMACS_INT is wider than a pointer,
+   as they may allocate the halves of an EMACS_INT separately.
+   On these hosts USE_LSB_TAG is not needed because the top bits of an
+   EMACS_INT are unused, so define USE_LSB_TAG only on hosts where it
+   might be useful.  */
+#  if UINTPTR_MAX >> VALBITS != 0
+#   define USE_LSB_TAG
+#  endif
 # endif
 #endif

@@ -309,11 +321,6 @@
     Lisp_Fwd_Kboard_Obj,	/* Fwd to a Lisp_Object field of kboards.  */
   };

-/* These values are overridden by the m- file on some machines.  */
-#ifndef VALBITS
-#define VALBITS (BITS_PER_EMACS_INT - GCTYPEBITS)
-#endif
-
 #ifdef USE_LISP_UNION_TYPE

 #ifndef WORDS_BIGENDIAN

=== modified file 'src/mem-limits.h'
--- src/mem-limits.h	2012-01-19 07:21:25 +0000
+++ src/mem-limits.h	2012-02-20 22:53:46 +0000
@@ -34,7 +34,7 @@
 #endif

 extern char *start_of_data (void);
-#if defined USE_LSB_TAG
+#if defined USE_LSB_TAG || UINTPTR_MAX >> VALBITS == 0
 #define EXCEEDS_LISP_PTR(ptr) 0
 #elif defined DATA_SEG_BITS
 #define EXCEEDS_LISP_PTR(ptr) \

bug#10780: 24.0.93 fails to build with 'Invalid function: "DEAD"'

[Dan Horák]

Paul Eggert píše v Po 20. 02. 2012 v 15:14 -0800: 
> On 02/15/2012 02:09 AM, Andreas Schwab wrote:
> 
> > What about local variables in registers?  They are split between two
> > registers, but mark_stack does not handle that.
> 
> True.  Teerrroooo.  Thanks for diagnosing that.
> 
> I pushed the following fix into the trunk.  It's needed
> regardless of this particular bug.  Dan, can you please
> check whether it indeed fixes the bug for you?  Thanks.

yes, this change fixed the issue for me. Thanks to all involved.

> 
> Fix crash due to non-contiguous EMACS_INT (Bug#10780).
> * lisp.h (VALBITS): Move definition up, so that USE_LSB_TAG can use it.
> (USE_LSB_TAG): Do not define if UINTPTR_MAX >> VALBITS == 0.
> It's useless in that case, and it can cause problems on hosts
> that allocate halves of EMACS_INT values separately.
> Reported by Dan Horák.  Diagnosed by Andreas Schwab in
> <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10780#30>.
> * mem-limits.h (EXCEEDS_LISP_PTR): Define to 0 on hosts where
> UINTPTR_MAX >> VALBITS == 0.  This is required by the above change;
> it avoids undefined behavior on hosts where shifting right by more
> than the word width has undefined behavior.
> === modified file 'src/lisp.h'
> --- src/lisp.h	2012-01-19 07:21:25 +0000
> +++ src/lisp.h	2012-02-20 16:27:49 +0000
> @@ -168,6 +168,10 @@
>  #define GCTYPEBITS 3
>  #endif
> 
> +#ifndef VALBITS
> +#define VALBITS (BITS_PER_EMACS_INT - GCTYPEBITS)
> +#endif
> +
>  #ifndef NO_DECL_ALIGN
>  # ifndef DECL_ALIGN
>  #  if HAVE_ATTRIBUTE_ALIGNED
> @@ -191,7 +195,15 @@
>       || defined DARWIN_OS || defined __sun)
>  /* We also need to be able to specify mult-of-8 alignment on static vars.  */
>  # if defined DECL_ALIGN
> -#  define USE_LSB_TAG
> +/* mark_maybe_object assumes that EMACS_INT values are contiguous,
> +   but this is not true on some hosts where EMACS_INT is wider than a pointer,
> +   as they may allocate the halves of an EMACS_INT separately.
> +   On these hosts USE_LSB_TAG is not needed because the top bits of an
> +   EMACS_INT are unused, so define USE_LSB_TAG only on hosts where it
> +   might be useful.  */
> +#  if UINTPTR_MAX >> VALBITS != 0
> +#   define USE_LSB_TAG
> +#  endif
>  # endif
>  #endif
> 
> @@ -309,11 +321,6 @@
>      Lisp_Fwd_Kboard_Obj,	/* Fwd to a Lisp_Object field of kboards.  */
>    };
> 
> -/* These values are overridden by the m- file on some machines.  */
> -#ifndef VALBITS
> -#define VALBITS (BITS_PER_EMACS_INT - GCTYPEBITS)
> -#endif
> -
>  #ifdef USE_LISP_UNION_TYPE
> 
>  #ifndef WORDS_BIGENDIAN
> 
> === modified file 'src/mem-limits.h'
> --- src/mem-limits.h	2012-01-19 07:21:25 +0000
> +++ src/mem-limits.h	2012-02-20 22:53:46 +0000
> @@ -34,7 +34,7 @@
>  #endif
> 
>  extern char *start_of_data (void);
> -#if defined USE_LSB_TAG
> +#if defined USE_LSB_TAG || UINTPTR_MAX >> VALBITS == 0
>  #define EXCEEDS_LISP_PTR(ptr) 0
>  #elif defined DATA_SEG_BITS
>  #define EXCEEDS_LISP_PTR(ptr) \
> 
> 

bug#10780: 24.0.93 fails to build with 'Invalid function: "DEAD"'

[Dan Horák]

Paul Eggert píše v St 15. 02. 2012 v 01:15 -0800: 
> On 02/14/2012 11:39 PM, Dan Horák wrote:
> > I have to do some more testing but now it seems that using
> > --with-wide-int option for configure provokes the "Invalid function"
> > behaviour.
> 
> Thanks for looking into it.  I configured --with-wide-int on
> my bigendian 32-bit host and could not reproduce the problem.
> My host is a Solaris 10 sparc (32-bit).  I tried with both
> Sun C 5.12 SunOS_sparc 2011/11/16 (cc -xO4) and with
> GCC 3.4.3 (csl-sol210-3_4-branch+sol_rpath) (gcc -O2).
> 
> Since you built --with-wide-int, can you please also try
> compiling from scratch, with
> "make CPPFLAGS=-DGC_LISP_OBJECT_ALIGNMENT=4"?
> It could be that your compilers claim to be aligning 64-bit
> integers on 8-byte boundaries, but are occasionally actually
> aligning them on 4-byte boundaries.  This messes up conservative
> garbage collection big-time and would explain your symptoms.

The builds (on s390, haven't tried ppc yet) were successful with
GC_LISP_OBJECT_ALIGNMENT set to 1, 2 and 4. Only when it's not set, the
build fails with the "Invalid function" error message.