bug#45214: guile segfaults on 32-bit big-endian targets

bug#45214: guile segfaults on 32-bit big-endian targets

[John Paul Adrian Glaubitz]

Hello!

As reported in [1], guile segfaults on 32-bit big-endian targets with:

guild compile --target="hppa-unknown-linux-gnu"                \
          -W0 -O1              \
  -L "/<<PKGBUILDDIR>>/module"                         \
  -L "/<<PKGBUILDDIR>>/guile-readline"			\
  -o "language/tree-il/eta-expand.go" "../module/language/tree-il/eta-expand.scm"
make[4]: *** [Makefile:1927: language/tree-il/eta-expand.go] Segmentation fault
make[4]: *** Waiting for unfinished jobs....
make[4]: *** [Makefile:1927: language/tree-il/debug.go] Segmentation fault
make[4]: *** [Makefile:1927: language/tree-il/analyze.go] Segmentation fault
still going
make[4]: *** [Makefile:1927: language/tree-il/effects.go] Segmentation fault
make[4]: Leaving directory '/<<PKGBUILDDIR>>/bootstrap'
make[3]: *** [Makefile:1851: all-recursive] Error 1
make[3]: Leaving directory '/<<PKGBUILDDIR>>'
make[2]: *** [Makefile:1737: all] Error 2
make[2]: Leaving directory '/<<PKGBUILDDIR>>'
make[1]: *** [debian/rules:192: override_dh_auto_build] Error 2
make[1]: Leaving directory '/<<PKGBUILDDIR>>'
make: *** [debian/rules:104: build-arch] Error 2
dpkg-buildpackage: error: debian/rules build-arch subprocess returned exit status 2

Full build log in [2]. This affects hppa, m68k and powerpc (32-bit).

Backtrace:

#0  0x006e3e9c in scm_sum (x=x@entry=0x0, y=0x6) at numbers.c:7569
7569      else if (SCM_BIGP (x))
[Current thread is 1 (Thread 0xf7c34480 (LWP 529732))]
(gdb) bt
#0  0x006e3e9c in scm_sum (x=x@entry=0x0, y=0x6) at numbers.c:7569
#1  0x006cd498 in add_immediate (a=0x0, b=<optimized out>) at intrinsics.c:80
#2  0x00749818 in vm_regular_engine (thread=0xf7971e00) at vm-engine.c:1583
#3  0x0074c2f4 in scm_call_n (proc=proc@entry=0xeffbf148, argv=argv@entry=0x0, 
nargs=nargs@entry=0) at vm.c:1608
#4  0x006a70fc in scm_call_0 (proc=proc@entry=0xeffbf148) at eval.c:490
#5  0x006d39d8 in scm_primitive_load_path (args=<optimized out>) at load.c:1259
#6  0x006d3f28 in scm_c_primitive_load_path (filename=filename@entry=0x776458 
"ice-9/boot-9") at load.c:1275
#7  0x006c9310 in scm_load_startup_files () at init.c:222
#8  0x006c96a8 in scm_i_init_guile (base=base@entry=0xffecfb18) at init.c:505
#9  0x0073b858 in scm_i_init_thread_for_guile (base=base@entry=0xffecfb18, 
dynamic_state=dynamic_state@entry=0x0) at threads.c:570
#10 0x0073ba1c in scm_i_init_thread_for_guile (dynamic_state=0x0, 
base=0xffecfb18) at threads.c:677
#11 with_guile (base=0xffecfb18, data=0xffecfb38) at threads.c:638
#12 0x003e17e4 in GC_call_with_stack_base () from 
/usr/lib/powerpc-linux-gnu/libgc.so.1
#13 0x0073bfb8 in scm_i_with_guile (dynamic_state=<optimized out>, 
data=0xffecfb28, func=0x6c90c0 <invoke_main_func>) at threads.c:688
#14 scm_with_guile (func=func@entry=0x6c90c0 <invoke_main_func>, 
data=data@entry=0xffecfb58) at threads.c:694
#15 0x006c93c8 in scm_boot_guile (argc=argc@entry=16, 
argv=argv@entry=0xffecfe14, main_func=main_func@entry=0x800950 <inner_main>, 
closure=closure@entry=0x0) at init.c:290
#16 0x00800754 in main (argc=16, argv=0xffecfe14) at guile.c:95
(gdb)

I assume the crash has got something to do how values are packed and unpacked
into the SCM object type. I have not been able to find the problem yet myself,
unfortunately which is why I am reporting this issue here.

For anyone willing to debug this, access to a fast POWER machine running a
big-endian Debian unstable can be obtained through the GCC compile farm [3].

Thanks,
Adrian

> [1] https://lists.gnu.org/archive/html/guile-devel/2020-12/msg00003.html
> [2] https://buildd.debian.org/status/fetch.php?pkg=guile-3.0&arch=hppa&ver=3.0.4-3&stamp=1607546304&raw=0
> [3] https://gcc.gnu.org/wiki/CompileFarm

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

bug#45214: guile segfaults on 32-bit big-endian targets

[John David Anglin]

Hi,

I created this debian PR earlier today:
Bug#977223: Acknowledgement (guile-3.0: FTBFS on hppa - segmentation faults)

In the above, the segmentation fault occurs during exception handling on hppa.  The exception is more
or less handled on powerpc:

../meta/build-env                \
guild compile --target="powerpc-unknown-linux-gnu"                \
          -W0 -O1              \
  -L "/<<PKGBUILDDIR>>/module"                         \
  -L "/<<PKGBUILDDIR>>/guile-readline"            \
  -o "language/tree-il/compile-cps.go" "../module/language/tree-il/compile-cps.scm"
Backtrace:
          11 (_ #<procedure efa84650 at ice-9/eval.scm:330:13 ()> # #)
          10 (apply-smob/0 #<thunk efa7cba0>)
In ice-9/eval.scm:
    657:8  9 (_ #(#(#f) ("prompt") #<procedure efacfcb0 at ice-9/?> ?))
    619:8  8 (_ #(#(#<directory (guile-user) f00566e0>)))
    155:9  7 (_ #(#(#<directory (guild) efe0daa0> ("compile" # ?)) #))
    619:8  6 (_ #(#(#<directory (srfi srfi-1) efcc4780> #<proc?> ?) ?))
    163:9  5 (_ #(#(#<directory (scripts compile) ef13ae60> "p?" ?) ?))
In unknown file:
           4 (_ #<procedure eebf7cb0 at ice-9/eval.scm:330:13 ()> # #)
In system/base/target.scm:
     65:6  3 (with-target "powerpc-unknown-linux-gnu" #<procedure ee?>)
In system/base/compile.scm:
    187:6  2 (compile-file "../module/ice-9/rdelim.scm" #:output-file ?)
    56:12  1 (call-once #<procedure ee49af40 at system/base/compile.?>)
In unknown file:
           0 (_ #<procedure ee413dc0 at ice-9/eval.scm:330:13 ()> # #)

ERROR: thunk may only be entered once: ~a #<procedure ee49af40 at system/base/compile.scm:66:5 ()>
make[4]: *** [Makefile:1927: ice-9/rdelim.go] Error 1

Trying to bisect.

Regards,
Dave

On 2020-12-12 4:26 p.m., John Paul Adrian Glaubitz wrote:
> Hello!
>
> As reported in [1], guile segfaults on 32-bit big-endian targets with:
>
> guild compile --target="hppa-unknown-linux-gnu"                \
>           -W0 -O1              \
>   -L "/<<PKGBUILDDIR>>/module"                         \
>   -L "/<<PKGBUILDDIR>>/guile-readline"			\
>   -o "language/tree-il/eta-expand.go" "../module/language/tree-il/eta-expand.scm"
> make[4]: *** [Makefile:1927: language/tree-il/eta-expand.go] Segmentation fault
> make[4]: *** Waiting for unfinished jobs....
> make[4]: *** [Makefile:1927: language/tree-il/debug.go] Segmentation fault
> make[4]: *** [Makefile:1927: language/tree-il/analyze.go] Segmentation fault
> still going
> make[4]: *** [Makefile:1927: language/tree-il/effects.go] Segmentation fault
> make[4]: Leaving directory '/<<PKGBUILDDIR>>/bootstrap'
> make[3]: *** [Makefile:1851: all-recursive] Error 1
> make[3]: Leaving directory '/<<PKGBUILDDIR>>'
> make[2]: *** [Makefile:1737: all] Error 2
> make[2]: Leaving directory '/<<PKGBUILDDIR>>'
> make[1]: *** [debian/rules:192: override_dh_auto_build] Error 2
> make[1]: Leaving directory '/<<PKGBUILDDIR>>'
> make: *** [debian/rules:104: build-arch] Error 2
> dpkg-buildpackage: error: debian/rules build-arch subprocess returned exit status 2
>
> Full build log in [2]. This affects hppa, m68k and powerpc (32-bit).
>
> Backtrace:
>
> #0  0x006e3e9c in scm_sum (x=x@entry=0x0, y=0x6) at numbers.c:7569
> 7569      else if (SCM_BIGP (x))
> [Current thread is 1 (Thread 0xf7c34480 (LWP 529732))]
> (gdb) bt
> #0  0x006e3e9c in scm_sum (x=x@entry=0x0, y=0x6) at numbers.c:7569
> #1  0x006cd498 in add_immediate (a=0x0, b=<optimized out>) at intrinsics.c:80
> #2  0x00749818 in vm_regular_engine (thread=0xf7971e00) at vm-engine.c:1583
> #3  0x0074c2f4 in scm_call_n (proc=proc@entry=0xeffbf148, argv=argv@entry=0x0, 
> nargs=nargs@entry=0) at vm.c:1608
> #4  0x006a70fc in scm_call_0 (proc=proc@entry=0xeffbf148) at eval.c:490
> #5  0x006d39d8 in scm_primitive_load_path (args=<optimized out>) at load.c:1259
> #6  0x006d3f28 in scm_c_primitive_load_path (filename=filename@entry=0x776458 
> "ice-9/boot-9") at load.c:1275
> #7  0x006c9310 in scm_load_startup_files () at init.c:222
> #8  0x006c96a8 in scm_i_init_guile (base=base@entry=0xffecfb18) at init.c:505
> #9  0x0073b858 in scm_i_init_thread_for_guile (base=base@entry=0xffecfb18, 
> dynamic_state=dynamic_state@entry=0x0) at threads.c:570
> #10 0x0073ba1c in scm_i_init_thread_for_guile (dynamic_state=0x0, 
> base=0xffecfb18) at threads.c:677
> #11 with_guile (base=0xffecfb18, data=0xffecfb38) at threads.c:638
> #12 0x003e17e4 in GC_call_with_stack_base () from 
> /usr/lib/powerpc-linux-gnu/libgc.so.1
> #13 0x0073bfb8 in scm_i_with_guile (dynamic_state=<optimized out>, 
> data=0xffecfb28, func=0x6c90c0 <invoke_main_func>) at threads.c:688
> #14 scm_with_guile (func=func@entry=0x6c90c0 <invoke_main_func>, 
> data=data@entry=0xffecfb58) at threads.c:694
> #15 0x006c93c8 in scm_boot_guile (argc=argc@entry=16, 
> argv=argv@entry=0xffecfe14, main_func=main_func@entry=0x800950 <inner_main>, 
> closure=closure@entry=0x0) at init.c:290
> #16 0x00800754 in main (argc=16, argv=0xffecfe14) at guile.c:95
> (gdb)
>
> I assume the crash has got something to do how values are packed and unpacked
> into the SCM object type. I have not been able to find the problem yet myself,
> unfortunately which is why I am reporting this issue here.
>
> For anyone willing to debug this, access to a fast POWER machine running a
> big-endian Debian unstable can be obtained through the GCC compile farm [3].
>
> Thanks,
> Adrian
>
>> [1] https://lists.gnu.org/archive/html/guile-devel/2020-12/msg00003.html
>> [2] https://buildd.debian.org/status/fetch.php?pkg=guile-3.0&arch=hppa&ver=3.0.4-3&stamp=1607546304&raw=0
>> [3] https://gcc.gnu.org/wiki/CompileFarm


-- 
John David Anglin  dave.anglin@bell.net

bug#45214: guile segfaults on 32-bit big-endian targets

[John Paul Adrian Glaubitz]

Hi!

On 12/12/20 11:00 PM, John David Anglin wrote:
> Trying to bisect.

Bisecting lead me to this which I guess is just painting over the real
problem:

(sid_powerpc-dchroot)glaubitz@perotto:~/guile-git$ git bisect good
a68c80c747a2a8ec92fa84684ebd60b4ecb7ffa0 is the first bad commit
commit a68c80c747a2a8ec92fa84684ebd60b4ecb7ffa0
Author: Andy Wingo <wingo@pobox.com>
Date:   Mon May 11 15:22:29 2020 +0200

    Switch to baseline compiler for bootstrap/

    * bootstrap/Makefile.am (GUILE_OPTIMIZATIONS): Add -Ono-cps so that we
      use the baseline compiler when bootstrapping.

 bootstrap/Makefile.am | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
(sid_powerpc-dchroot)glaubitz@perotto:~/guile-git$

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

bug#45214: guile segfaults on 32-bit big-endian targets

[John David Anglin]

On 2020-12-12 4:26 p.m., John Paul Adrian Glaubitz wrote:
> I assume the crash has got something to do how values are packed and unpacked
> into the SCM object type. I have not been able to find the problem yet myself,
> unfortunately which is why I am reporting this issue here.
I see this in scm.h:

/* The 0?: constructions makes sure that the code is never executed, and
   that there is no performance hit.  However, the alternative is
   compiled, and does generate a warning when used with the wrong
   pointer type.  We use a volatile pointer type to avoid warnings from
   clang.

   The Tru64 and ia64-hp-hpux11.23 compilers fail on `case (0?0=0:x)'
   statements, so for them type-checking is disabled.  */
# if defined __DECC || defined __HP_cc
#  define SCM_UNPACK(x) ((scm_t_bits) (x))
# else
#  define SCM_UNPACK(x) ((scm_t_bits) (0? (*(volatile SCM *)0=(x)): x))
# endif

Regards,
Dave

-- 
John David Anglin  dave.anglin@bell.net

bug#45214: guile segfaults on 32-bit big-endian targets

[John David Anglin]

On 2020-12-12 5:21 p.m., John David Anglin wrote:
> On 2020-12-12 4:26 p.m., John Paul Adrian Glaubitz wrote:
>> I assume the crash has got something to do how values are packed and unpacked
>> into the SCM object type. I have not been able to find the problem yet myself,
>> unfortunately which is why I am reporting this issue here.
> I see this in scm.h:
>
> /* The 0?: constructions makes sure that the code is never executed, and
>    that there is no performance hit.  However, the alternative is
>    compiled, and does generate a warning when used with the wrong
>    pointer type.  We use a volatile pointer type to avoid warnings from
>    clang.
>
>    The Tru64 and ia64-hp-hpux11.23 compilers fail on `case (0?0=0:x)'
>    statements, so for them type-checking is disabled.  */
> # if defined __DECC || defined __HP_cc
> #  define SCM_UNPACK(x) ((scm_t_bits) (x))
> # else
> #  define SCM_UNPACK(x) ((scm_t_bits) (0? (*(volatile SCM *)0=(x)): x))
> # endif
Also just before, there is:
/* But as external interface, we define SCM, which may, according to the
   desired level of type checking, be defined in several ways.  */
#if (SCM_DEBUG_TYPING_STRICTNESS == 2)
  typedef union SCM { struct { scm_t_bits n; } n; } SCM;
# define SCM_UNPACK(x) ((x).n.n)
# define SCM_PACK(x) ((SCM) { { (scm_t_bits) (x) } })
#elif (SCM_DEBUG_TYPING_STRICTNESS == 1)
/* This is the default, which provides an intermediate level of compile
   time type checking while still resulting in very efficient code.  */
  typedef struct scm_unused_struct { char scm_unused_field; } *SCM;

The fault on hppa appears to be at strictness 1.

Regards,
Dave

-- 
John David Anglin  dave.anglin@bell.net

bug#45214: guile segfaults on 32-bit big-endian targets

[John David Anglin]

On 2020-12-12 5:30 p.m., John David Anglin wrote:
> On 2020-12-12 5:21 p.m., John David Anglin wrote:
>> On 2020-12-12 4:26 p.m., John Paul Adrian Glaubitz wrote:
>>> I assume the crash has got something to do how values are packed and unpacked
>>> into the SCM object type. I have not been able to find the problem yet myself,
>>> unfortunately which is why I am reporting this issue here.
>> I see this in scm.h:
>>
>> /* The 0?: constructions makes sure that the code is never executed, and
>>    that there is no performance hit.  However, the alternative is
>>    compiled, and does generate a warning when used with the wrong
>>    pointer type.  We use a volatile pointer type to avoid warnings from
>>    clang.
>>
>>    The Tru64 and ia64-hp-hpux11.23 compilers fail on `case (0?0=0:x)'
>>    statements, so for them type-checking is disabled.  */
>> # if defined __DECC || defined __HP_cc
>> #  define SCM_UNPACK(x) ((scm_t_bits) (x))
>> # else
>> #  define SCM_UNPACK(x) ((scm_t_bits) (0? (*(volatile SCM *)0=(x)): x))
>> # endif
> Also just before, there is:
> /* But as external interface, we define SCM, which may, according to the
>    desired level of type checking, be defined in several ways.  */
> #if (SCM_DEBUG_TYPING_STRICTNESS == 2)
>   typedef union SCM { struct { scm_t_bits n; } n; } SCM;
> # define SCM_UNPACK(x) ((x).n.n)
> # define SCM_PACK(x) ((SCM) { { (scm_t_bits) (x) } })
> #elif (SCM_DEBUG_TYPING_STRICTNESS == 1)
> /* This is the default, which provides an intermediate level of compile
>    time type checking while still resulting in very efficient code.  */
>   typedef struct scm_unused_struct { char scm_unused_field; } *SCM;
>
> The fault on hppa appears to be at strictness 1.
Guess, I should have copied the whole bit:

The last option is:
#else
/* This should be used as a fall back solution for machines on which
   casting to a pointer may lead to loss of bit information, e. g. in
   the three least significant bits.  */
  typedef scm_t_bits SCM;
# define SCM_UNPACK(x) (x)
# define SCM_PACK(x) ((SCM) (x))
#endif

I think strictness 1 may be problematic on hppa.

Dave

-- 
John David Anglin  dave.anglin@bell.net

bug#45214: guile segfaults on 32-bit big-endian targets

[John Paul Adrian Glaubitz]

On 12/12/20 11:44 PM, John David Anglin wrote:
> Guess, I should have copied the whole bit:
> 
> The last option is:
> #else
> /* This should be used as a fall back solution for machines on which
>    casting to a pointer may lead to loss of bit information, e. g. in
>    the three least significant bits.  */
>   typedef scm_t_bits SCM;
> # define SCM_UNPACK(x) (x)
> # define SCM_PACK(x) ((SCM) (x))
> #endif
> 
> I think strictness 1 may be problematic on hppa.

And m68k and powerpc most likely.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

bug#45214: guile segfaults on 32-bit big-endian targets

[John David Anglin]

On 2020-12-12 5:47 p.m., John Paul Adrian Glaubitz wrote:
> And m68k and powerpc most likely.
This could also be a bug on 32-bit targets:
  CC       libguile_3.0_la-hash.lo
../../guile/libguile/hash.c: In function 'narrow_string_hash':
../../guile/libguile/hash.c:118:34: warning: left shift count >= width of type [
-Wshift-count-overflow]
  118 |       ret = (((unsigned long) c) << 32) | b;
\
      |                                  ^~
../../guile/libguile/hash.c:128:3: note: in expansion of macro 'JENKINS_LOOKUP3_
HASHWORD2'
  128 |   JENKINS_LOOKUP3_HASHWORD2 (str, len, ret);
      |   ^~~~~~~~~~~~~~~~~~~~~~~~~
../../guile/libguile/hash.c: In function 'wide_string_hash':
../../guile/libguile/hash.c:118:34: warning: left shift count >= width of type [-Wshift-count-overflow]
  118 |       ret = (((unsigned long) c) << 32) | b;                            \
      |                                  ^~

Regards,
Dave

-- 
John David Anglin  dave.anglin@bell.net

bug#45214: guile segfaults on 32-bit big-endian targets

[John David Anglin]

On 2020-12-12 5:47 p.m., John Paul Adrian Glaubitz wrote:
>> I think strictness 1 may be problematic on hppa.
> And m68k and powerpc most likely.
I did a build of the current debian package on hppa with type strictness set to 0.  With that, I get the same fault
that you reported for powerpc.   SCM_BIGP(x) faults when x is 0.

I updated debian PR with additional analysis:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=977223

Regards,
Dave

-- 
John David Anglin  dave.anglin@bell.net

bug#45214: guile segfaults on 32-bit big-endian targets

[John Paul Adrian Glaubitz]

On 12/13/20 7:06 PM, John David Anglin wrote:
> I did a build of the current debian package on hppa with type strictness set to 0.  With that, I get the same fault
> that you reported for powerpc.   SCM_BIGP(x) faults when x is 0.
> 
> I updated debian PR with additional analysis:
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=977223

See also: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=45214

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913