[bug#32221] [PATCH 0/5] MariaDB updates

[bug#32221] [PATCH 0/5] MariaDB updates

[Marius Bakke]

These patches gives us more control of the MariaDB package.  In
particular, we now invoke the test runner directly, so we can pass
custom arguments; there's a phase for disabling plugins in an attempt
to fix the armhf build; we now use system zlib, pcre, xz and snappy;
and the package is ~37 MiB smaller thanks to a new "static" output.

I've verified that 'qtbase' builds against this MariaDB.

Marius Bakke (5):
  gnu: mariadb: Disable plugin that fails on armhf.
  gnu: mariadb: Run the full test suite.
  gnu: mariadb: Update to 10.1.34.
  gnu: mariadb: Remove some bundled libraries.
  gnu: mariadb: Move static libraries to separate output.

 gnu/packages/databases.scm | 104 ++++++++++++++++++++++++++++++++++---
 1 file changed, 97 insertions(+), 7 deletions(-)

-- 
2.18.0

[bug#32221] [PATCH 1/5] gnu: mariadb: Disable plugin that fails on armhf.

[Marius Bakke]

* gnu/packages/databases.scm (mariadb)[arguments]: Add 'disable-plugins' phase.
---
 gnu/packages/databases.scm | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index b9ae9ee15..c27381ac2 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -665,6 +665,19 @@ Language.")
          "-DINSTALL_SHAREDIR=share/mysql")
        #:phases
        (modify-phases %standard-phases
+         (add-before 'configure 'disable-plugins
+           (lambda _
+             (let ((disable-plugin (lambda (name)
+                                     (call-with-output-file
+                                         (string-append "plugin/" name
+                                                        "/CMakeLists.txt")
+                                       (lambda (port)
+                                         (format port "\n")))))
+                   (disabled-plugins '(;; FIXME: On armhf-linux, this plugin
+                                       ;; triggers a GCC ICE.  Disable for now.
+                                       "semisync")))
+               (for-each disable-plugin disabled-plugins)
+               #t)))
          (add-before
           'configure 'pre-configure
           (lambda _
-- 
2.18.0

[bug#32221] [PATCH 2/5] gnu: mariadb: Run the full test suite.

[Marius Bakke]

* gnu/packages/databases.scm (mariadb)[arguments]: Override 'check' phase.
Add phase 'adjust-tests'.  Disable one more plugin.
[properties]: New field.
---
 gnu/packages/databases.scm | 35 ++++++++++++++++++++++++++++++++++-
 1 file changed, 34 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index c27381ac2..9e4adc340 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -665,6 +665,29 @@ Language.")
          "-DINSTALL_SHAREDIR=share/mysql")
        #:phases
        (modify-phases %standard-phases
+         (add-after 'unpack 'adjust-tests
+           (lambda _
+             (let ((disabled-tests
+                    '(;; These fail because root@hostname == root@localhost in
+                      ;; the build environment, causing a user count mismatch.
+                      ;; See <https://jira.mariadb.org/browse/MDEV-7761>.
+                      "main.join_cache"
+                      "main.explain_non_select"
+                      "roles.acl_statistics"))
+
+                   ;; This file contains a list of known-flaky tests for this
+                   ;; release.  Append our own list.
+                   (unstable-tests (open-file "mysql-test/unstable-tests" "a")))
+               (for-each (lambda (test)
+                           (format unstable-tests "~a : ~a\n"
+                                   test "Disabled in Guix"))
+                         disabled-tests)
+               (close-port unstable-tests)
+
+               (substitute* "mysql-test/mysql-test-run.pl"
+                 (("/bin/ls") (which "ls"))
+                 (("/bin/sh") (which "sh")))
+               #t)))
          (add-before 'configure 'disable-plugins
            (lambda _
              (let ((disable-plugin (lambda (name)
@@ -675,7 +698,9 @@ Language.")
                                          (format port "\n")))))
                    (disabled-plugins '(;; FIXME: On armhf-linux, this plugin
                                        ;; triggers a GCC ICE.  Disable for now.
-                                       "semisync")))
+                                       "semisync"
+                                       ;; XXX: Causes a test failure.
+                                       "disks")))
                (for-each disable-plugin disabled-plugins)
                #t)))
          (add-before
@@ -683,6 +708,11 @@ Language.")
           (lambda _
             (setenv "CONFIG_SHELL" (which "sh"))
             #t))
+         (replace 'check
+           (lambda _
+             (with-directory-excursion "mysql-test"
+               (invoke "./mtr" "--parallel" (number->string (parallel-job-count))
+                       "--verbose" "--skip-test-list=unstable-tests"))))
          (add-after
           'install 'post-install
           (lambda* (#:key outputs #:allow-other-keys)
@@ -708,6 +738,9 @@ Language.")
        ("openssl" ,openssl)
        ("pcre" ,pcre)
        ("zlib" ,zlib)))
+    ;; The test suite is very resource intensive and can take more than three
+    ;; hours on a x86_64 system.  Give slow and busy machines some leeway.
+    (properties '((timeout . 43200)))        ;12 hours
     (home-page "https://mariadb.org/")
     (synopsis "SQL database server")
     (description
-- 
2.18.0

[bug#32221] [PATCH 3/5] gnu: mariadb: Update to 10.1.34.

[Marius Bakke]

* gnu/packages/databases.scm (mariadb): Update to 10.1.34.
---
 gnu/packages/databases.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 9e4adc340..3b767664a 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -627,7 +627,7 @@ Language.")
 (define-public mariadb
   (package
     (name "mariadb")
-    (version "10.1.33")
+    (version "10.1.34")
     (source (origin
               (method url-fetch)
               (uri (string-append "https://downloads.mariadb.org/f/"
@@ -635,7 +635,7 @@ Language.")
                                   name "-" version ".tar.gz"))
               (sha256
                (base32
-                "0bax748j4srsyhw5cs5jvwigndh0zwmf4r2cjvhja31ckx8jqccl"))))
+                "0j2mdpyvj41vkq2rwrzky88b7170hzz6gy2vb2bc1447s2gp3q67"))))
     (build-system cmake-build-system)
     (arguments
      '(#:configure-flags
-- 
2.18.0

[bug#32221] [PATCH 4/5] gnu: mariadb: Remove some bundled libraries.

[Marius Bakke]

* gnu/packages/databases.scm (mariadb)[source](snippet, modules): New fields.
[arguments]: Add explicit #:configure-flags for system libraries.  Add
'unbundle' phase.  Remove 'pre-configure' phase.
[inputs]: Add SNAPPY and XZ.
---
 gnu/packages/databases.scm | 43 ++++++++++++++++++++++++++++++++------
 1 file changed, 37 insertions(+), 6 deletions(-)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 3b767664a..de1db2f93 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -635,7 +635,16 @@ Language.")
                                   name "-" version ".tar.gz"))
               (sha256
                (base32
-                "0j2mdpyvj41vkq2rwrzky88b7170hzz6gy2vb2bc1447s2gp3q67"))))
+                "0j2mdpyvj41vkq2rwrzky88b7170hzz6gy2vb2bc1447s2gp3q67"))
+              (modules '((guix build utils)))
+              (snippet
+               '(begin
+                  (delete-file-recursively "storage/tokudb/PerconaFT/third_party")
+                  (for-each (lambda (file)
+                              (unless (string-suffix? "CMakeLists.txt" file)
+                                (delete-file file)))
+                            (append (find-files "pcre") (find-files "zlib")))
+                  #t))))
     (build-system cmake-build-system)
     (arguments
      '(#:configure-flags
@@ -649,6 +658,11 @@ Language.")
          ;; For now, disable the features that that use libarchive (xtrabackup).
          "-DWITH_LIBARCHIVE=OFF"
 
+         ;; Ensure the system libraries are used.
+         "-DWITH_JEMALLOC=yes"
+         "-DWITH_PCRE=system"
+         "-DWITH_ZLIB=system"
+
          "-DDEFAULT_CHARSET=utf8"
          "-DDEFAULT_COLLATION=utf8_general_ci"
          "-DMYSQL_DATADIR=/var/lib/mysql"
@@ -665,6 +679,26 @@ Language.")
          "-DINSTALL_SHAREDIR=share/mysql")
        #:phases
        (modify-phases %standard-phases
+         (add-after 'unpack 'unbundle
+           (lambda _
+             ;; The bundled PCRE in MariaDB has a patch that was upstreamed
+             ;; in version 8.34.  Unfortunately the upstream patch behaves
+             ;; slightly differently and the build system fails to detect it.
+             ;; See <https://bugs.exim.org/show_bug.cgi?id=2173>.
+             ;; XXX: Consider patching PCRE instead.
+             (substitute* "cmake/pcre.cmake"
+               ((" OR NOT PCRE_STACK_SIZE_OK") ""))
+
+             (substitute* "storage/tokudb/PerconaFT/ft/CMakeLists.txt"
+               ;; Remove dependency on these CMake targets.
+               ((" build_lzma build_snappy") ""))
+
+             (substitute* "storage/tokudb/PerconaFT/CMakeLists.txt"
+               ;; This file checks that the bundled sources are present and
+               ;; declares build procedures for them.  We don't need that.
+               (("^include\\(TokuThirdParty\\)") ""))
+
+             #t))
          (add-after 'unpack 'adjust-tests
            (lambda _
              (let ((disabled-tests
@@ -703,11 +737,6 @@ Language.")
                                        "disks")))
                (for-each disable-plugin disabled-plugins)
                #t)))
-         (add-before
-          'configure 'pre-configure
-          (lambda _
-            (setenv "CONFIG_SHELL" (which "sh"))
-            #t))
          (replace 'check
            (lambda _
              (with-directory-excursion "mysql-test"
@@ -737,6 +766,8 @@ Language.")
        ("ncurses" ,ncurses)
        ("openssl" ,openssl)
        ("pcre" ,pcre)
+       ("snappy" ,snappy)
+       ("xz" ,xz)
        ("zlib" ,zlib)))
     ;; The test suite is very resource intensive and can take more than three
     ;; hours on a x86_64 system.  Give slow and busy machines some leeway.
-- 
2.18.0

[bug#32221] [PATCH 5/5] gnu: mariadb: Move static libraries to separate output.

[Marius Bakke]

* gnu/packages/databases.scm (mariadb)[outputs]: New field.
[arguments]: Add 'move-static-libs' phase.
---
 gnu/packages/databases.scm | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index de1db2f93..cbc530bec 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -646,6 +646,7 @@ Language.")
                             (append (find-files "pcre") (find-files "zlib")))
                   #t))))
     (build-system cmake-build-system)
+    (outputs '("out" "static"))
     (arguments
      '(#:configure-flags
        '("-DBUILD_CONFIG=mysql_release"
@@ -742,6 +743,18 @@ Language.")
              (with-directory-excursion "mysql-test"
                (invoke "./mtr" "--parallel" (number->string (parallel-job-count))
                        "--verbose" "--skip-test-list=unstable-tests"))))
+         (add-after 'install 'move-static-libs
+           ;; Move ~37 MiB worth of static libraries to a separate output.
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let ((lib (string-append (assoc-ref outputs "out") "/lib"))
+                   (slib (string-append (assoc-ref outputs "static") "/lib")))
+               (mkdir-p slib)
+               (with-directory-excursion lib
+                 (for-each (lambda (ar)
+                             (link ar (string-append slib "/" (basename ar)))
+                             (delete-file ar))
+                           (find-files "." "\\.a$"))
+                 #t))))
          (add-after
           'install 'post-install
           (lambda* (#:key outputs #:allow-other-keys)
-- 
2.18.0

bug#32221: [PATCH 0/5] MariaDB updates

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 845 bytes --]

Marius Bakke <mbakke@fastmail.com> writes:

> These patches gives us more control of the MariaDB package.  In
> particular, we now invoke the test runner directly, so we can pass
> custom arguments; there's a phase for disabling plugins in an attempt
> to fix the armhf build; we now use system zlib, pcre, xz and snappy;
> and the package is ~37 MiB smaller thanks to a new "static" output.
>
> I've verified that 'qtbase' builds against this MariaDB.
>
> Marius Bakke (5):
>   gnu: mariadb: Disable plugin that fails on armhf.
>   gnu: mariadb: Run the full test suite.
>   gnu: mariadb: Update to 10.1.34.
>   gnu: mariadb: Remove some bundled libraries.
>   gnu: mariadb: Move static libraries to separate output.

I've pushed this patchset with slight modifications:

* Also unbundled YaSSL
* Deleted static library instead separate output

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]