bug#42771: smalltalk fails to build

["Miguel Ángel Arruga Vivas" <rosen644835@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 4059 bytes --]

Hi,

I've been investigating just a bit about this.

Ludovic Courtès <ludo@gnu.org> writes:

> (1) investigate why the test is failing (I think it’s a single test
> failure here),

Currently there are two main problems with 3.2.5:

  - Integer multiplication overflow handling invokes undefined behavior,
    which is "cleaned up" by the compiler.  This is why
    [ 100 fact / 99 fact ] returns <new> 0, and it's solved with the
    first patch---trimmed down from the upstream patch, to avoid
    conflicts.

   - ANSI test suite fails with errors like these:
----------------------------------8<-----------------------------------
--- /dev/null   2020-12-24 20:38:33.836725540 +0000
+++ /tmp/guix-build-smalltalk-3.2.5.drv-0/smalltalk-3.2.5/tests/testsuite.dir/at-groups/47/stderr       2020-12-28 09:25:57.283891452 +0000
@@ -0,0 +1,13 @@
+gst: Aborted
+gst: Error occurred while not in byte code interpreter!!
+/tmp/guix-build-smalltalk-3.2.5.drv-0/smalltalk-3.2.5/libgst/.libs/libgst.so.7(+0x72d97)[0x7ffff7f5ed97]
+/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/libc.so.6(+0x36b20)[0x7ffff7b3bb20]
+/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/libc.so.6(gsignal+0xca)[0x7ffff7b3baba]
+/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/libc.so.6(abort+0x165)[0x7ffff7b3cbf5]
+/tmp/guix-build-smalltalk-3.2.5.drv-0/smalltalk-3.2.5/libgst/.libs/libgst.so.7(+0x2c936)[0x7ffff7f18936]
+/gnu/store/yrwirrblml57nwga1aza6rg3l9s8qga0-libsigsegv-2.12/lib/libsigsegv.so.2(+0x128c)[0x7ffff7ee728c]
+/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/lib/libc.so.6(+0x36b20)[0x7ffff7b3bb20]
+/tmp/guix-build-smalltalk-3.2.5.drv-0/smalltalk-3.2.5/libgst/.libs/libgst.so.7(+0x569f0)[0x7ffff7f429f0]
+/tmp/guix-build-smalltalk-3.2.5.drv-0/smalltalk-3.2.5/libgst/.libs/libgst.so.7(+0x72919)[0x7ffff7f5e919]
+/tmp/guix-build-smalltalk-3.2.5.drv-0/smalltalk-3.2.5/libgst/.libs/libgst.so.7(+0x2e4c7)[0x7ffff7f1a4c7]
+/tmp/guix-build-smalltalk-3.2.5.drv-0/smalltalk-3.2.5/tests/testsuite.dir/at-groups/47/test-source: line 20: 21205 Aborted                 $TIMEOUT gst $image_path -f $abs_srcdir/AnsiRun.st ArrayANSITest
stdout:
./testsuite.at:83: exit code was 134, expected 0
47. testsuite.at:83: 47. ArrayANSITest (testsuite.at:83): FAILED (testsuite.at:83)
---------------------------------->8-----------------------------------

>      (2) try to determine whether it’s serious or not,

The first one is pretty serious.  The second one might as bad as the
first one or may be a flaw on the tests and could be omitted.

>      (3) see if we can work around it with reasonable effort, and if
>      not, skip just this test.

This only should be the case when the problem is on the test side:
either it's using something we don't want to provide, as side channels,
or the check itself is buggy; never when the test is working properly
because we would be delivering buggy software after we have been warned
about it.

I cannot spot where the second error was fixed, but 3.2.91 as provided
with the second patch doesn't manifest it.

>   2. I think we should just have ‘smalltalk’ (latest release) and
>      ‘smalltalk-next’ (VCS snapshot).  Having an extra package for the
>      release candidate is not really useful IMO, and not something we
>      generally do.

The last change made to GNU Smalltalk's master branch was more than 2
years ago, the latest change affecting executable paths was February
2017.  IMHO, providing a patched release candidate (or even the
snapshot) seems the best option until the project release a new version,
as in fact we aren't providing currently a useful package.

Happy hacking!
Miguel

PS: Its build isn't reproducible:
  - gst.im has inside some kind of timestamp.  This probably needs
    changes on the source code.
  - package.xml and kernel/ folders contain different timestamps inside
    dot-star files---a wrapper to zip or something like
    reset-gzip-timestamps could solve this issue.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: fix-integer.patch --]
[-- Type: text/x-patch, Size: 7860 bytes --]

From c4385abde62bba4c634a7a874c2f431451909ec2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miguel=20=C3=81ngel=20Arruga=20Vivas?=
 <rosen644835@gmail.com>
Date: Mon, 28 Dec 2020 10:36:48 +0100
Subject: [PATCH 1/2] gnu: smalltalk: Fix integer multiplication overflow.

* gnu/packages/patches/smalltalk-multiplication-overflow.patch: Patch
from upstream commit 72ada189aba0283c551ead16635c1983968080b8.
* gnu/packages/smalltalk.scm (smalltalk): Use patch and link with gmp
and lightning libraries instead of the included source.
---
 .../smalltalk-multiplication-overflow.patch   | 121 ++++++++++++++++++
 gnu/packages/smalltalk.scm                    |  43 ++++++-
 2 files changed, 159 insertions(+), 5 deletions(-)
 create mode 100644 gnu/packages/patches/smalltalk-multiplication-overflow.patch

diff --git a/gnu/packages/patches/smalltalk-multiplication-overflow.patch b/gnu/packages/patches/smalltalk-multiplication-overflow.patch
new file mode 100644
index 0000000000..7a0b4d02f7
--- /dev/null
+++ b/gnu/packages/patches/smalltalk-multiplication-overflow.patch
@@ -0,0 +1,121 @@
+Extracted from this commit without the ChangeLog to avoid conflicts:
+http://git.savannah.gnu.org/cgit/smalltalk.git/commit/?id=72ada189aba0283c551ead16635c1983968080b8
+
+The upstream commit message is
+From 72ada189aba0283c551ead16635c1983968080b8 Mon Sep 17 00:00:00 2001
+From: Holger Hans Peter Freyther <holger@moiji-mobile.com>
+Date: Sat, 7 Nov 2015 18:09:31 +0100
+Subject: libgst: Add alternative multiplication overflow check
+
+Apple clang on OSX and the version on FreeBSD optimize the
+multiplication check away. Clang introduced a family of
+builtins to do the multiplication and check for the overflow
+and GCC made the API usable. For clang we would need to know
+if intptr_t is of type int, long int, long long int and
+then use the smul, smull smulll.
+Luckily clang is adopting the better interface and this is
+what we are starting to use now. This means the new code
+will be used on GCC5 (and later) and some future versions of
+clang.
+
+2015-11-07  Holger Hans Peter Freyther  <holger@freyther.de>
+
+	* build-aux/overflow-builtins.m4: Add new macro.
+	* configure.ac: Use GST_C_OVERFLOW_BUILTINS macro.
+
+2015-11-07  Holger Hans Peter Freyther  <holger@freyther.de>
+
+	* interp.inl: Add alternative mul_with_check implementation.
+---
+ ChangeLog                      |  5 +++++
+ build-aux/overflow-builtins.m4 | 23 +++++++++++++++++++++++
+ configure.ac                   |  1 +
+ libgst/ChangeLog               |  4 ++++
+ libgst/interp.inl              | 22 ++++++++++++++++++++++
+ 5 files changed, 55 insertions(+)
+ create mode 100644 build-aux/overflow-builtins.m4
+
+diff --git a/build-aux/overflow-builtins.m4 b/build-aux/overflow-builtins.m4
+new file mode 100644
+index 00000000..9d050196
+--- /dev/null
++++ b/build-aux/overflow-builtins.m4
+@@ -0,0 +1,23 @@
++dnl Check whether the host supports synchronization builtins.
++
++AC_DEFUN([GST_C_OVERFLOW_BUILTINS], [
++  AC_REQUIRE([AC_CANONICAL_HOST])
++  AC_CACHE_CHECK([whether the host supports __builtin_mul_overflow],
++                 gst_cv_have_builtin_mul_overflow, [
++    save_CFLAGS="$CFLAGS"
++    case $host in
++      i?86-apple-darwin*) ;;
++      i?86-*-*) CFLAGS="$CFLAGS -march=i486" ;;
++    esac
++    AC_LINK_IFELSE([AC_LANG_PROGRAM([[int foovar = 0;]], [[
++if (__builtin_mul_overflow(44444, 55555, &foovar))
++	return 23;]])],
++		   [gst_cv_have_builtin_mul_overflow=yes],
++		   [gst_cv_have_builtin_mul_overflow=no])
++    CFLAGS="$save_CFLAGS"
++  ])
++  if test $gst_cv_have_builtin_mul_overflow = yes; then
++    AC_DEFINE(HAVE_OVERFLOW_BUILTINS, 1,
++              [Define to 1 if the host supports __builtin_*_overflow builtins])
++  fi
++])
+diff --git a/configure.ac b/configure.ac
+index e789be45..0bac23ef 100644
+--- a/configure.ac
++++ b/configure.ac
+@@ -243,6 +243,7 @@ GST_C_SYNC_BUILTINS
+ if test $gst_cv_have_sync_fetch_and_add = no; then
+   AC_MSG_ERROR([Synchronization primitives not found, please use a newer compiler.])
+ fi
++GST_C_OVERFLOW_BUILTINS
+ 
+ GST_LOCK
+ AC_SYS_LARGEFILE
+diff --git a/libgst/interp.inl b/libgst/interp.inl
+index e18e27c7..dbc631bc 100644
+--- a/libgst/interp.inl
++++ b/libgst/interp.inl
+@@ -159,6 +159,27 @@ sub_with_check (OOP op1, OOP op2, mst_Boolean *overflow)
+ OOP
+ mul_with_check (OOP op1, OOP op2, mst_Boolean *overflow)
+ {
++#ifdef HAVE_OVERFLOW_BUILTINS
++  intptr_t a = TO_INT (op1);
++  intptr_t b = TO_INT (op2);
++  intptr_t result;
++
++  if (__builtin_mul_overflow(a, b, &result))
++    {
++       *overflow = true;
++       return FROM_INT(0);
++    }
++
++
++  if (result < MIN_ST_INT || result > MAX_ST_INT)
++    {
++       *overflow = true;
++       return FROM_INT(0);
++    }
++
++  *overflow = false;
++  return FROM_INT(result);
++#else
+   intptr_t a = TO_INT (op1);
+   intptr_t b = TO_INT (op2);
+   intmax_t result = (intmax_t)a * b;
+@@ -188,6 +209,7 @@ mul_with_check (OOP op1, OOP op2, mst_Boolean *overflow)
+     }
+ 
+   return FROM_INT (0);
++#endif
+ }
+ 
+ /* State of the random generator.
+-- 
+2.29.2
+
diff --git a/gnu/packages/smalltalk.scm b/gnu/packages/smalltalk.scm
index 5d35f563e2..8c152cfd04 100644
--- a/gnu/packages/smalltalk.scm
+++ b/gnu/packages/smalltalk.scm
@@ -26,6 +26,8 @@
   #:use-module (guix download)
   #:use-module (guix build-system cmake)
   #:use-module (guix build-system gnu)
+  #:use-module (gnu packages)
+  #:use-module (gnu packages assembly)
   #:use-module (gnu packages audio)
   #:use-module (gnu packages autotools)
   #:use-module (gnu packages base)
@@ -36,6 +38,7 @@
   #:use-module (gnu packages libffi)
   #:use-module (gnu packages libsigsegv)
   #:use-module (gnu packages linux)
+  #:use-module (gnu packages multiprecision)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages pulseaudio)
   #:use-module (gnu packages xorg))
@@ -51,18 +54,48 @@
                           version ".tar.xz"))
       (sha256
        (base32
-        "1k2ssrapfzhngc7bg1zrnd9n2vyxp9c9m70byvsma6wapbvib6l1"))))
+        "1k2ssrapfzhngc7bg1zrnd9n2vyxp9c9m70byvsma6wapbvib6l1"))
+      ;; XXX: To be removed with the next release of Smalltalk.
+      (patches (search-patches "smalltalk-multiplication-overflow.patch"))))
     (build-system gnu-build-system)
     (native-inputs
-     `(("libffi" ,libffi)
+     `(("pkg-config" ,pkg-config)
+       ;; XXX: To be removed with the next release of Smalltalk.
+       ("autoconf" ,autoconf)
+       ("automake" ,automake)
+       ("libtool" ,libtool)))
+    ;; XXX: Missing optional dependencies:
+    ;;  - glib
+    ;;  - gtk+-2
+    ;;  - mesa
+    ;;  - SDL
+    ;;  - sqlite
+    ;;  - zlib
+    (inputs
+     `(("gmp" ,gmp)
+       ("libffi" ,libffi)
        ("libltdl" ,libltdl)
        ("libsigsegv" ,libsigsegv)
-       ("pkg-config" ,pkg-config)))
-    (inputs
-     `(("zip" ,zip)))
+       ("lightning" ,lightning)
+       ("zip" ,zip)))
     (arguments
      `(#:phases
        (modify-phases %standard-phases
+         ;; XXX: To be removed with the next release of Smalltalk.
+         ;; The overflow patch modifies configure.ac, therefore remove
+         ;; old configure script and enforce an autoreconf.
+         (add-before 'bootstrap 'remove-unpatched-configure
+           (lambda _
+             (delete-file "configure")
+             #t))
+         ;; XXX: To be removed with the next release of Smalltalk.
+         ;; We don't want to regenerate the info files.
+         (add-after 'build 'keep-generated-info-manual
+           (lambda _
+             (for-each (lambda (file)
+                         (invoke "touch" file))
+                       (find-files "doc" "\\.info"))
+             #t))
          (add-before 'configure 'fix-libc
            (lambda _
              (let ((libc (assoc-ref %build-inputs "libc")))
-- 
2.29.2


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: update-to-91.patch --]
[-- Type: text/x-patch, Size: 1357 bytes --]

From 46809e6c265280571ff024c749c448c74af79c1a Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Miguel=20=C3=81ngel=20Arruga=20Vivas?=
 <rosen644835@gmail.com>
Date: Mon, 28 Dec 2020 11:42:31 +0100
Subject: [PATCH 2/2] gnu: smalltalk: Update to version 3.2.91.

* gnu/packages/smalltalk.scm (smalltalk): Update to 3.2.91.

Co-Authored-By: Holger Peters <holger.peters@posteo.de>
---
 gnu/packages/smalltalk.scm | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/smalltalk.scm b/gnu/packages/smalltalk.scm
index 8c152cfd04..c7c56f87b4 100644
--- a/gnu/packages/smalltalk.scm
+++ b/gnu/packages/smalltalk.scm
@@ -46,15 +46,15 @@
 (define-public smalltalk
   (package
     (name "smalltalk")
-    (version "3.2.5")
+    (version "3.2.91")
     (source
      (origin
       (method url-fetch)
-      (uri (string-append "mirror://gnu/smalltalk/smalltalk-"
+      (uri (string-append "https://alpha.gnu.org/gnu/smalltalk/smalltalk-"
                           version ".tar.xz"))
       (sha256
        (base32
-        "1k2ssrapfzhngc7bg1zrnd9n2vyxp9c9m70byvsma6wapbvib6l1"))
+        "1zb2h5cbz1cwybqjl24lflw359lwj7sjvvhwb4x6miypzhwq4qh0"))
       ;; XXX: To be removed with the next release of Smalltalk.
       (patches (search-patches "smalltalk-multiplication-overflow.patch"))))
     (build-system gnu-build-system)
-- 
2.29.2