[bug#39881] Add couchdb

[bug#39881] Add couchdb

[Lars-Dominik Braun]

[-- Attachment #1: Type: text/plain, Size: 734 bytes --]

Hi,

these two patches add couchdb. First, an upgrade to erlang is essential, since
our version is quite outdated and has known issues with couchdb. I’m only
bumping to a newer maintenance release of 21.x for now. `guix refresh
--list-dependent` lists only elixir, which builds fine with the new version and
emacs-erlang, build from the same sources, seems to work too.

The second patch adds couchdb. I’ve got almost the entire test suite up and
running. It is very extensive and takes a lot of time (compared to
compilation). Some patching is required to unbundle Erlangs ERTS, so an update
to Erlang in the future might break 'install due to the hard-coded version. Any
thoughts?

Future work: Shepherd service.

Cheers,
Lars


[-- Attachment #2: 0001-gnu-erlang-Update-to-21.3.8.13.patch --]
[-- Type: text/x-diff, Size: 1538 bytes --]

From c30c7d3ff91b8a442b48249af37dc05954fbb486 Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <ldb@leibniz-psychology.org>
Date: Tue, 3 Mar 2020 09:41:08 +0100
Subject: [PATCH 1/2] gnu: erlang: Update to 21.3.8.13

* gnu/packages/erlang.scm (erlang): Update to 21.3.8.13
---
 gnu/packages/erlang.scm | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/erlang.scm b/gnu/packages/erlang.scm
index 4e12bb4813..865895da78 100644
--- a/gnu/packages/erlang.scm
+++ b/gnu/packages/erlang.scm
@@ -40,7 +40,7 @@
 (define-public erlang
   (package
     (name "erlang")
-    (version "21.0.5")
+    (version "21.3.8.13")
     (source (origin
               (method git-fetch)
               ;; The tarball from http://erlang.org/download contains many
@@ -52,7 +52,7 @@
               (file-name (git-file-name name version))
               (sha256
                (base32
-                "0gv83i5ybj1z3ykbbldjzf7dbfjszp84c0yzrpshj611b9wp0176"))
+                "1dj37vk712dx76y25g13na24wbpn7a5ddmlpf4n51gm10sib54wj"))
               (patches (search-patches "erlang-man-path.patch"))))
     (build-system gnu-build-system)
     (native-inputs
@@ -68,7 +68,7 @@
                                (version-major+minor version) ".tar.gz"))
            (sha256
             (base32
-             "0cgv23q0215f6lvmhj4w9skx5m29khjs9mb890635s3yp520xgqh"))))))
+             "0wm1dg1psv1n3gpiwyms06yhsryrnr28p455fp0l1ak8hdf4nipm"))))))
     (inputs
      `(("ncurses" ,ncurses)
        ("openssl" ,openssl)
-- 
2.20.1


[-- Attachment #3: 0002-gnu-Add-couchdb.patch --]
[-- Type: text/x-diff, Size: 5390 bytes --]

From e3df880d9aa5cbdc983c1a5497cde536f1a13fe6 Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <ldb@leibniz-psychology.org>
Date: Tue, 3 Mar 2020 10:38:40 +0100
Subject: [PATCH 2/2] gnu: Add couchdb

* gnu/packages/databases.scm (couchdb): New variable
---
 gnu/packages/databases.scm | 93 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 93 insertions(+)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index d4891841e4..a4ccb599f6 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -74,14 +74,17 @@
   #:use-module (gnu packages cyrus-sasl)
   #:use-module (gnu packages dbm)
   #:use-module (gnu packages emacs)
+  #:use-module (gnu packages erlang)
   #:use-module (gnu packages flex)
   #:use-module (gnu packages gcc)
   #:use-module (gnu packages gettext)
   #:use-module (gnu packages glib)
   #:use-module (gnu packages gnupg)
+  #:use-module (gnu packages gnuzilla)
   #:use-module (gnu packages guile)
   #:use-module (gnu packages time)
   #:use-module (gnu packages golang)
+  #:use-module (gnu packages icu4c)
   #:use-module (gnu packages jemalloc)
   #:use-module (gnu packages language)
   #:use-module (gnu packages libevent)
@@ -3368,3 +3371,93 @@ The drivers officially supported by @code{libdbi} are:
 @end itemize")
     (home-page "http://libdbi-drivers.sourceforge.net/")
     (license license:lgpl2.1+)))
+
+(define-public couchdb
+  (package
+    (name "couchdb")
+    (version "3.0.0")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append
+             "mirror://apache/couchdb/source/" version
+             "/apache-couchdb-" version ".tar.gz"))
+       (sha256
+        (base32
+         "1nbz2vafzhp9jv8xna8cfnf99jwn22xs4ydzm426qx7yf0dbn2fi"))
+       (modules '((guix build utils)))
+       ;; remove pre-built binary, will be rebuilt
+       (snippet '(begin (delete-file "bin/rebar") #t))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'fix-paths
+           (lambda* (#:key inputs #:allow-other-keys)
+             (substitute* "src/couch/rebar.config.script"
+               (("/usr/include/mozjs-60")
+                (string-append (assoc-ref inputs "mozjs") "/include/mozjs-60")))
+             (substitute* "Makefile"
+               ;; disable python-black code formatting
+               (("check: all python-black") "check: all")
+               ;; disable venv creation and package installation
+               (("python3 -m venv .venv") "true")
+               (("\\.venv/bin/python3 -m pip install -r requirements.txt") "true")
+               ;; do not test elixir, depends on `mix` and several external dependencies
+               (("@\\$\\(MAKE\\) elixir") "")
+               ;; use system python
+               (("\\.venv/bin/python3")
+                (string-append (assoc-ref inputs "python") "/bin/python3")))
+             #t))
+         (add-after 'unpack 'set-env
+           (lambda* (#:key inputs #:allow-other-keys)
+             ;; make sure rebar finds a C compiler
+             (setenv "CC" (string-append (assoc-ref inputs "gcc") "/bin/gcc"))
+             #t))
+         (replace 'configure
+           (lambda* (#:key inputs #:allow-other-keys)
+             (invoke "./configure" "--skip-deps" "--spidermonkey-version" "60")
+             #t))
+         (replace 'build
+           (lambda _
+             ;; create a release build
+             (invoke "make" "release")
+             #t))
+         (replace 'install
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             ;; couchdb tries to bundle erts, use our own copy
+             (let ((erts-version "10.3.5.9"))
+               (delete-file-recursively
+                (string-append "rel/couchdb/erts-" erts-version))
+               (substitute* '("rel/couchdb/bin/couchdb" "rel/couchdb/bin/remsh")
+                 (("ROOTDIR=\\$\\{ERTS_BIN_DIR%/\\*\\}")
+                  (string-append "ROOTDIR=" (assoc-ref outputs "out")))
+                 (("BINDIR=\"[^\"]+\"")
+                  (string-append
+                   "BINDIR=" (assoc-ref inputs "erlang")
+                   "/lib/erlang/erts-" erts-version "/bin")))
+               (copy-recursively "rel/couchdb" (assoc-ref outputs "out")))
+             #t)))))
+    (native-inputs
+     ;; for tests
+     `(("python" ,python)
+       ("python-nose" ,python-nose)
+       ("python-requests" ,python-requests)
+       ("python-hypothesis" ,python-hypothesis)
+       ("curl" ,curl)
+       ;; for documentation
+       ("python-sphinx" ,python-sphinx)))
+    (inputs
+     `(("erlang" ,erlang)
+       ("icu4c" ,icu4c)
+       ("openssl" ,openssl)
+       ("mozjs" ,mozjs-60)))
+    (home-page "https://couchdb.apache.org/")
+    (synopsis "Document-oriented NoSQL database, implemented in Erlang")
+    (description "CouchDB is a database that completely embraces the web.  Store
+your data with JSON documents.  Access your documents with your web browser, via
+HTTP.  Query, combine, and transform your documents with JavaScript.  CouchDB
+works well with modern web and mobile apps.  You can distribute your data,
+efficiently using CouchDB’s incremental replication.  CouchDB supports
+master-master setups with automatic conflict detection.")
+    (license license:asl2.0)))
-- 
2.20.1

[bug#39881] Add couchdb

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 320 bytes --]

I've pushed the erlang update for now. I haven't taken a look at the
couchdb package at all as of yet.


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#39881] Add couchdb

[Efraim Flashner]

[-- Attachment #1.1: Type: text/plain, Size: 1157 bytes --]

I touched up the couchdb patch a bit and there were a couple of things
that stood out to me. The test suite passed for me without any problems.

There are a couple of bundled fonts as well as bundled javascript inside
the share directory, which also makes it to the output.

Looking at %out/lib/ it looks like there are a bunch of other libraries
bundled with couchdb. I don't know much about erlang but I assume all
these libraries should be unbundled. Looking inside erlang I see there
are also a lot of what looks like bundled libraries so I'm not really
sure how that goes.

On the other hand, after talking to you on IRC and looking at how Debian
takes erlang¹ and seems to split it into dozens of package outputs I'm
far more uncertain of how it works.

Anyway, take a look at the changes I made and see if anything/everything
works :)

¹ https://sources.debian.org/src/erlang/1:23.0%7Erc1+dfsg-1/debian/patterns/


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #1.2: 0001-gnu-Add-couchdb.patch --]
[-- Type: text/plain, Size: 5797 bytes --]

From 238b40d1da1e486392c69840ad9439e5bbb60c4a Mon Sep 17 00:00:00 2001
From: Lars-Dominik Braun <ldb@leibniz-psychology.org>
Date: Tue, 3 Mar 2020 10:38:40 +0100
Subject: [PATCH] gnu: Add couchdb.

* gnu/packages/databases.scm (couchdb): New variable.

---
 gnu/packages/databases.scm | 91 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 90 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index c8d12f43b5..063292c68a 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -74,15 +74,17 @@
   #:use-module (gnu packages cyrus-sasl)
   #:use-module (gnu packages dbm)
   #:use-module (gnu packages emacs)
+  #:use-module (gnu packages erlang)
   #:use-module (gnu packages flex)
   #:use-module (gnu packages gcc)
   #:use-module (gnu packages gettext)
   #:use-module (gnu packages glib)
   #:use-module (gnu packages gnupg)
+  #:use-module (gnu packages gnuzilla)
   #:use-module (gnu packages guile)
-  #:use-module (gnu packages time)
   #:use-module (gnu packages golang)
   #:use-module (gnu packages jemalloc)
+  #:use-module (gnu packages icu4c)
   #:use-module (gnu packages language)
   #:use-module (gnu packages libevent)
   #:use-module (gnu packages linux)
@@ -110,6 +112,7 @@
   #:use-module (gnu packages tcl)
   #:use-module (gnu packages terminals)
   #:use-module (gnu packages textutils)
+  #:use-module (gnu packages time)
   #:use-module (gnu packages tls)
   #:use-module (gnu packages valgrind)
   #:use-module (gnu packages web)
@@ -3403,3 +3406,89 @@ The drivers officially supported by @code{libdbi} are:
 PostreSQL, SQLite, ODBC and MySQL.")
     (home-page "http://soci.sourceforge.net/")
     (license license:boost1.0)))
+
+(define-public couchdb
+  (package
+    (name "couchdb")
+    (version "3.0.0")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append
+              "mirror://apache/couchdb/source/" version
+              "/apache-couchdb-" version ".tar.gz"))
+       (sha256
+        (base32
+         "1nbz2vafzhp9jv8xna8cfnf99jwn22xs4ydzm426qx7yf0dbn2fi"))
+       (modules '((guix build utils)))
+       ;; TODO: Unbundle fonts, others from share/docs/html/_static, share/www, share/server
+       ;; Remove pre-built binary, will be rebuilt.
+       (snippet '(begin (delete-file "bin/rebar") #t))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:make-flags '("release"
+                      "CC=gcc")
+       #:configure-flags (list "--skip-deps"
+                               "--spidermonkey-version"
+                               ,(version-major (package-version mozjs-60)))
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'fix-paths
+           (lambda* (#:key inputs #:allow-other-keys)
+             (substitute* "src/couch/rebar.config.script"
+               (("/usr/include/mozjs-60")
+                (string-append (assoc-ref inputs "mozjs") "/include/mozjs-60")))
+             (substitute* "Makefile"
+               ;; Disable python-black code formatting.
+               (("check: all python-black") "check: all")
+               ;; Disable venv creation and package installation.
+               (("python3 -m venv .venv") "true")
+               (("\\.venv/bin/python3 -m pip install -r requirements.txt") "true")
+               ;; Do not test elixir, depends on `mix` and several external dependencies.
+               (("@\\$\\(MAKE\\) elixir") "")
+               ;; Use system python.
+               (("\\.venv/bin/python3") (which "python3")))
+             #t))
+         (replace 'configure
+           (lambda* (#:key (configure-flags '()) #:allow-other-keys)
+             (apply invoke "./configure" configure-flags)))
+         (replace 'install
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             ;; CouchDB tries to bundle erts, use our own copy.
+             (let ((out          (assoc-ref outputs "out"))
+                   (erts-version "10.3.5.9"))
+               (delete-file-recursively
+                 (string-append "rel/couchdb/erts-" erts-version))
+               (substitute* '("rel/couchdb/bin/couchdb"
+                              "rel/couchdb/bin/remsh")
+                 (("ROOTDIR=\\$\\{ERTS_BIN_DIR%/\\*\\}")
+                  (string-append "ROOTDIR=" out))
+                 (("BINDIR=\"[^\"]+\"")
+                  (string-append
+                    "BINDIR=" (assoc-ref inputs "erlang")
+                    "/lib/erlang/erts-" erts-version "/bin")))
+               (copy-recursively "rel/couchdb" out))
+             #t)))))
+    (native-inputs
+     ;; for tests
+     `(("curl" ,curl)
+       ("python" ,python)
+       ("python-hypothesis" ,python-hypothesis)
+       ("python-nose" ,python-nose)
+       ("python-requests" ,python-requests)
+       ;; for documentation
+       ("python-sphinx" ,python-sphinx)))
+    (inputs
+     `(("erlang" ,erlang)
+       ("icu4c" ,icu4c)
+       ("mozjs" ,mozjs-60)
+       ("openssl" ,openssl)))
+    (home-page "https://couchdb.apache.org/")
+    (synopsis "Document-oriented NoSQL database, implemented in Erlang")
+    (description "CouchDB is a database that completely embraces the web.  Store
+your data with JSON documents.  Access your documents with your web browser, via
+HTTP.  Query, combine, and transform your documents with JavaScript.  CouchDB
+works well with modern web and mobile apps.  You can distribute your data,
+efficiently using CouchDB’s incremental replication.  CouchDB supports
+master-master setups with automatic conflict detection.")
+    (license license:asl2.0)))
-- 
2.25.1


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#39881] Add couchdb

[Lars-Dominik Braun]

[-- Attachment #1: Type: text/plain, Size: 2031 bytes --]

Hi Efraim,

> There are a couple of bundled fonts as well as bundled javascript inside
> the share directory, which also makes it to the output.
we can probably unbundle fontawesome, but not their own font. The web frontend
(fauxton) depends on NPM and a bunch of JavaScript packages – nothing we can do
about that.

> Looking at %out/lib/ it looks like there are a bunch of other libraries
> bundled with couchdb. I don't know much about erlang but I assume all
> these libraries should be unbundled. Looking inside erlang I see there
> are also a lot of what looks like bundled libraries so I'm not really
> sure how that goes.
As far as I see there is a mix between bundled external libraries and those
internal to couchdb. External libraries in src/ are:

- bear (https://github.com/boundary/bear)
- dreyfus (https://github.com/cloudant-labs/dreyfus)
- folsom (https://github.com/boundary/folsom)
- hyper (https://github.com/GameAnalytics/hyper)
- jiffy (https://github.com/davisp/jiffy)
- meck (https://github.com/eproxus/meck)
- mochiweb (https://github.com/mochi/mochiweb/)
- proper (https://github.com/proper-testing/proper)
- rebar (https://github.com/rebar/rebar)
- recon (https://github.com/ferd/recon)
- snappy (https://github.com/skunkwerks/snappy-erlang-nif)

All modules are also mirrored by the Apache project here
https://github.com/apache?q=couchdb&type=&language=
so it might be possible to build most of them “out of tree”, i.e. create
separate packages and somehow “link” to them. I’ll look into that when I have
some time, but I’m not an Erlang guru. In the meantime I’ll be maintaining the
package over here[1], since the current state is not good enough for inclusion
into guix.

> Anyway, take a look at the changes I made and see if anything/everything
> works :)
Yep, I can confirm the server and web interface are working.

Thanks for the review,
Lars

[1] https://github.com/leibniz-psychology/guix-zpid/blob/master/zpid/packages/couchdb.scm

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

[bug#39881] Add couchdb

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 215 bytes --]

On Thu, Mar 12, 2020 at 10:07:26AM +0100, Lars-Dominik Braun wrote:
> Yep, I can confirm the server and web interface are working.

Alright, what is the status of this patch? Does it need more work? Or is
it ready?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#39881] Add couchdb

[Lars-Dominik Braun]

[-- Attachment #1: Type: text/plain, Size: 286 bytes --]

Hi,

> Alright, what is the status of this patch? Does it need more work? Or is
> it ready?
I don’t have time to work on this any more. The package works fine, but
probably does not satisfy guix’ requirements regarding bundled and precompiled
(JavaScript) sources.

Lars


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

bug#39881: Add couchdb

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 489 bytes --]

On Fri, May 29, 2020 at 08:28:51AM +0200, Lars-Dominik Braun wrote:
> Hi,
> 
> > Alright, what is the status of this patch? Does it need more work? Or is
> > it ready?
> I don’t have time to work on this any more. The package works fine, but
> probably does not satisfy guix’ requirements regarding bundled and precompiled
> (JavaScript) sources.

Oh well, that's too bad. I'll close this ticket and anyone should feel
free to reopen it if they'd like to continue the work.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]