[bug#47081] [PATCH 0/5] gnu: Remove mongodb.

[bug#47081] [PATCH 0/5] gnu: Remove mongodb.

[Léo Le Bouter via Guix-patches via]

mongodb 3.4.10 has unpatched CVEs and mongodb 3.4.24 has some files in the
release tarball under the SSPL, therefore we cannot provide mongodb while
upholding to good security standards.

Léo Le Bouter (5):
  gnu: Remove mongo-tools.
  doc: Remove mongodb-service-type.
  tests: databases: Remove mongodb test.
  services: Remove mongodb service.
  gnu: Remove mongodb.

 doc/guix.texi              |  28 -----
 gnu/packages/databases.scm | 252 -------------------------------------
 gnu/services/databases.scm |  88 -------------
 gnu/tests/databases.scm    |  83 ------------
 4 files changed, 451 deletions(-)

-- 
2.30.2

[bug#47081] [PATCH 1/5] gnu: Remove mongo-tools.

[Léo Le Bouter via Guix-patches via]

Preparing for mongodb removal, first it's dependents must be removed.

* gnu/packages/databases.scm (mongo-tools): Remove.
---
 gnu/packages/databases.scm | 115 -------------------------------------
 1 file changed, 115 deletions(-)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 437449174f..2330f9fd26 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -3640,121 +3640,6 @@ transforms idiomatic python function calls to well-formed SQL queries.")
 the SQL language using a syntax that reflects the resulting query.")
     (license license:asl2.0)))
 
-(define-public mongo-tools
-  (package
-    (name "mongo-tools")
-    (version "3.4.0")
-    (source
-     (origin (method git-fetch)
-             (uri (git-reference
-                   (url "https://github.com/mongodb/mongo-tools")
-                   (commit (string-append "r" version))))
-             (file-name (git-file-name name version))
-             (sha256
-              (base32
-               "1bcsz5cvj39a7nsxsfqmz9igrw33j6yli9kffigqyscs52amw7x1"))))
-    (build-system go-build-system)
-    (arguments
-     `(#:import-path "github.com/mongodb/mongo-tools"
-       #:modules ((srfi srfi-1)
-                  (guix build go-build-system)
-                  (guix build utils))
-       #:install-source? #f
-       #:phases
-       (let ((all-tools
-              '("bsondump" "mongodump" "mongoexport" "mongofiles"
-                "mongoimport" "mongooplog" "mongorestore"
-                "mongostat" "mongotop")))
-         (modify-phases %standard-phases
-           (add-after 'unpack 'delete-bundled-source-code
-             (lambda _
-               (delete-file-recursively
-                "src/github.com/mongodb/mongo-tools/vendor")
-               #t))
-           (add-after 'delete-bundled-source-code 'patch-source
-             (lambda _
-               ;; Remove a redundant argument that causes compilation to fail.
-               (substitute*
-                   "src/github.com/mongodb/mongo-tools/mongorestore/filepath.go"
-                 (("skipping restore of system.profile collection\", db)")
-                  "skipping restore of system.profile collection\")"))
-               #t))
-           (replace 'build
-             (lambda _
-               (for-each (lambda (tool)
-                           (let ((command
-                                  `("go" "build"
-                                    ;; This is where the tests expect to find the
-                                    ;; executables
-                                    "-o" ,(string-append
-                                           "src/github.com/mongodb/mongo-tools/bin/"
-                                           tool)
-                                    "-v"
-                                    "-tags=\"ssl sasl\""
-                                    "-ldflags"
-                                    "-extldflags=-Wl,-z,now,-z,relro"
-                                    ,(string-append
-                                      "src/github.com/mongodb/mongo-tools/"
-                                      tool "/main/" tool ".go"))))
-                             (simple-format #t "build: running ~A\n"
-                                            (string-join command))
-                             (apply invoke command)))
-                         all-tools)
-               #t))
-           (replace 'check
-             (lambda _
-               (with-directory-excursion "src"
-                 (for-each (lambda (tool)
-                             (invoke
-                              "go" "test" "-v"
-                              (string-append "github.com/mongodb/mongo-tools/"
-                                             tool)))
-                           all-tools))
-               #t))
-           (replace 'install
-             (lambda* (#:key outputs #:allow-other-keys)
-               (for-each (lambda (tool)
-                           (install-file
-                            (string-append "src/github.com/mongodb/mongo-tools/bin/"
-                                           tool)
-                            (string-append (assoc-ref outputs "out")
-                                           "/bin")))
-                         all-tools)
-               #t))))))
-    (native-inputs
-     `(("go-github.com-howeyc-gopass" ,go-github.com-howeyc-gopass)
-       ("go-github.com-jessevdk-go-flags" ,go-github.com-jessevdk-go-flags)
-       ("go-golang-org-x-crypto" ,go-golang-org-x-crypto)
-       ("go-gopkg.in-mgo.v2" ,go-gopkg.in-mgo.v2)
-       ("go-gopkg.in-tomb.v2" ,go-gopkg.in-tomb.v2)
-       ("go-github.com-nsf-termbox-go" ,go-github.com-nsf-termbox-go)
-       ("go-github.com-smartystreets-goconvey" ,go-github.com-smartystreets-goconvey)))
-    (home-page "https://github.com/mongodb/mongo-tools")
-    (synopsis "Various tools for interacting with MongoDB and BSON")
-    (description
-     "This package includes a collection of tools related to MongoDB.
-@table @code
-@item bsondump
-Display BSON files in a human-readable format
-@item mongoimport
-Convert data from JSON, TSV or CSV and insert them into a collection
-@item mongoexport
-Write an existing collection to CSV or JSON format
-@item mongodump/mongorestore
-Dump MongoDB backups to disk in the BSON format
-@item mongorestore
-Read MongoDB backups in the BSON format, and restore them to a live database
-@item mongostat
-Monitor live MongoDB servers, replica sets, or sharded clusters
-@item mongofiles
-Read, write, delete, or update files in GridFS
-@item mongooplog
-Replay oplog entries between MongoDB servers
-@item mongotop
-Monitor read/write activity on a mongo server
-@end table")
-    (license license:asl2.0)))
-
 ;; There are many wrappers for this in other languages. When touching, please
 ;; be sure to ensure all dependencies continue to build.
 (define-public apache-arrow
-- 
2.30.2

[bug#47081] [PATCH 2/5] doc: Remove mongodb-service-type.

[Léo Le Bouter via Guix-patches via]

* doc/guix.texi (mongodb-service-type): Remove.
---
 doc/guix.texi | 28 ----------------------------
 1 file changed, 28 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 77aafafd97..bd51f7bf81 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -19804,34 +19804,6 @@ Additional command line options to pass to @code{memcached}.
 @end table
 @end deftp
 
-@subsubheading MongoDB
-
-@defvr {Scheme Variable} mongodb-service-type
-This is the service type for @uref{https://www.mongodb.com/, MongoDB}.
-The value for the service type is a @code{mongodb-configuration} object.
-@end defvr
-
-@lisp
-(service mongodb-service-type)
-@end lisp
-
-@deftp {Data Type} mongodb-configuration
-Data type representing the configuration of mongodb.
-
-@table @asis
-@item @code{mongodb} (default: @code{mongodb})
-The MongoDB package to use.
-
-@item @code{config-file} (default: @code{%default-mongodb-configuration-file})
-The configuration file for MongoDB.
-
-@item @code{data-directory} (default: @code{"/var/lib/mongodb"})
-This value is used to create the directory, so that it exists and is
-owned by the mongodb user.  It should match the data-directory which
-MongoDB is configured to use through the configuration file.
-@end table
-@end deftp
-
 @subsubheading Redis
 
 @defvr {Scheme Variable} redis-service-type
-- 
2.30.2

[bug#47081] [PATCH 3/5] tests: databases: Remove mongodb test.

[Léo Le Bouter via Guix-patches via]

* gnu/tests/databases.scm (%test-mongodb, %mongodb-os, run-mongodb-test):
Remove.
---
 gnu/tests/databases.scm | 83 -----------------------------------------
 1 file changed, 83 deletions(-)

diff --git a/gnu/tests/databases.scm b/gnu/tests/databases.scm
index e831d69f5a..4bfe4ee282 100644
--- a/gnu/tests/databases.scm
+++ b/gnu/tests/databases.scm
@@ -30,7 +30,6 @@
   #:use-module (guix gexp)
   #:use-module (guix store)
   #:export (%test-memcached
-            %test-mongodb
             %test-postgresql
             %test-mysql))
 
@@ -127,88 +126,6 @@
    (description "Connect to a running MEMCACHED server.")
    (value (run-memcached-test))))
 
-(define %mongodb-os
-  (operating-system
-    (inherit
-     (simple-operating-system
-      (service dhcp-client-service-type)
-      (service mongodb-service-type)))
-    (packages (cons* mongodb
-                     %base-packages))))
-
-(define* (run-mongodb-test #:optional (port 27017))
-  "Run tests in %MONGODB-OS, forwarding PORT."
-  (define os
-    (marionette-operating-system
-     %mongodb-os
-     #:imported-modules '((gnu services herd)
-                          (guix combinators))))
-
-  (define vm
-    (virtual-machine
-     (operating-system os)
-     (memory-size 1024)
-     (disk-image-size (* 1024 (expt 2 20)))
-     (port-forwardings `((27017 . ,port)))))
-
-  (define test
-    (with-imported-modules '((gnu build marionette))
-      #~(begin
-          (use-modules (srfi srfi-11) (srfi srfi-64)
-                       (gnu build marionette)
-                       (ice-9 popen)
-                       (ice-9 rdelim))
-
-          (define marionette
-            (make-marionette (list #$vm)))
-
-          (mkdir #$output)
-          (chdir #$output)
-
-          (test-begin "mongodb")
-
-          (test-assert "service running"
-            (marionette-eval
-             '(begin
-                (use-modules (gnu services herd))
-                (match (start-service 'mongodb)
-                  (#f #f)
-                  (('service response-parts ...)
-                   (match (assq-ref response-parts 'running)
-                     ((pid) (number? pid))))))
-             marionette))
-
-          (test-eq "test insert"
-            0
-            (system* (string-append #$mongodb "/bin/mongo")
-                     "test"
-                     "--eval"
-                     "db.testCollection.insert({data: 'test-data'})"))
-
-          (test-equal "test find"
-            "test-data"
-            (let* ((port (open-pipe*
-                          OPEN_READ
-                          (string-append #$mongodb "/bin/mongo")
-                          "test"
-                          "--quiet"
-                          "--eval"
-                          "db.testCollection.findOne().data"))
-                   (output (read-line port))
-                   (status (close-pipe port)))
-              output))
-
-          (test-end)
-          (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
-
-  (gexp->derivation "mongodb-test" test))
-
-(define %test-mongodb
-  (system-test
-   (name "mongodb")
-   (description "Connect to a running MONGODB server.")
-   (value (run-mongodb-test))))
-
 \f
 ;;;
 ;;; The PostgreSQL service.
-- 
2.30.2

[bug#47081] [PATCH 4/5] services: Remove mongodb service.

[Léo Le Bouter via Guix-patches via]

* gnu/services/databases.scm (mongodb-configuration, mongodb-configuration?,
mongodb-configuration-mongodb, mongodb-configuration-config-file,
mongodb-configuration-data-directory, mongodb-service-type,
%default-mongodb-configuration-file, %mongodb-accounts, mongodb-activation,
mongodb-shepherd-service): Remove.
---
 gnu/services/databases.scm | 88 --------------------------------------
 1 file changed, 88 deletions(-)

diff --git a/gnu/services/databases.scm b/gnu/services/databases.scm
index 979f3dd6c8..a841e7a50e 100644
--- a/gnu/services/databases.scm
+++ b/gnu/services/databases.scm
@@ -79,13 +79,6 @@
             memcached-configuration-udp-port
             memcached-configuration-additional-options
 
-            mongodb-configuration
-            mongodb-configuration?
-            mongodb-configuration-mongodb
-            mongodb-configuration-config-file
-            mongodb-configuration-data-directory
-            mongodb-service-type
-
             mysql-service
             mysql-service-type
             mysql-configuration
@@ -521,87 +514,6 @@ created after the PostgreSQL database is started.")))
                                           (const %memcached-accounts))))
                 (default-value (memcached-configuration))))
 
-\f
-;;;
-;;; MongoDB
-;;;
-
-(define %default-mongodb-configuration-file
-  (plain-file
-   "mongodb.yaml"
-   "# GNU Guix: MongoDB default configuration file
-processManagement:
-  pidFilePath: /var/run/mongodb/pid
-storage:
-  dbPath: /var/lib/mongodb
-"))
-
-
-(define-record-type* <mongodb-configuration>
-  mongodb-configuration make-mongodb-configuration
-  mongodb-configuration?
-  (mongodb             mongodb-configuration-mongodb
-                       (default mongodb))
-  (config-file         mongodb-configuration-config-file
-                       (default %default-mongodb-configuration-file))
-  (data-directory      mongodb-configuration-data-directory
-                       (default "/var/lib/mongodb")))
-
-(define %mongodb-accounts
-  (list (user-group (name "mongodb") (system? #t))
-        (user-account
-         (name "mongodb")
-         (group "mongodb")
-         (system? #t)
-         (comment "Mongodb server user")
-         (home-directory "/var/lib/mongodb")
-         (shell (file-append shadow "/sbin/nologin")))))
-
-(define mongodb-activation
-  (match-lambda
-    (($ <mongodb-configuration> mongodb config-file data-directory)
-     #~(begin
-         (use-modules (guix build utils))
-         (let ((user (getpwnam "mongodb")))
-           (for-each
-            (lambda (directory)
-              (mkdir-p directory)
-              (chown directory
-                     (passwd:uid user) (passwd:gid user)))
-            '("/var/run/mongodb" #$data-directory)))))))
-
-(define mongodb-shepherd-service
-  (match-lambda
-    (($ <mongodb-configuration> mongodb config-file data-directory)
-     (shepherd-service
-      (provision '(mongodb))
-      (documentation "Run the Mongodb daemon.")
-      (requirement '(user-processes loopback))
-      (start #~(make-forkexec-constructor
-                `(,(string-append #$mongodb "/bin/mongod")
-                  "--config"
-                  ,#$config-file)
-                #:user "mongodb"
-                #:group "mongodb"
-                #:pid-file "/var/run/mongodb/pid"
-                #:log-file "/var/log/mongodb.log"))
-      (stop #~(make-kill-destructor))))))
-
-(define mongodb-service-type
-  (service-type
-   (name 'mongodb)
-   (description "Run the MongoDB document database server.")
-   (extensions
-    (list (service-extension shepherd-root-service-type
-                             (compose list
-                                      mongodb-shepherd-service))
-          (service-extension activation-service-type
-                             mongodb-activation)
-          (service-extension account-service-type
-                             (const %mongodb-accounts))))
-   (default-value
-     (mongodb-configuration))))
-
 \f
 ;;;
 ;;; MySQL.
-- 
2.30.2

[bug#47081] [PATCH 5/5] gnu: Remove mongodb.

[Léo Le Bouter via Guix-patches via]

mongodb 3.4.10 has unpatched CVEs and mongodb 3.4.24 has some files in the
release tarball under the SSPL, therefore we cannot provide mongodb while
upholding to good security standards.

* gnu/packages/databases.scm (mongodb): Remove.
---
 gnu/packages/databases.scm | 137 -------------------------------------
 1 file changed, 137 deletions(-)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 2330f9fd26..5249aa0d10 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -615,143 +615,6 @@ replacement for the code@{python-memcached} library.")
 (define-public python2-pylibmc
   (package-with-python2 python-pylibmc))
 
-;; There is a point at which mongodb switched to the Server Side Public
-;; License (SSPL), which is not a FOSS license.  As such, be careful
-;; when updating this package.
-(define-public mongodb
-  (package
-    (name "mongodb")
-    (version "3.4.24")
-    (source (origin
-              (method url-fetch)
-              (uri (string-append "https://github.com/mongodb/mongo/archive/r"
-                                  version ".tar.gz"))
-              (file-name (string-append name "-" version ".tar.gz"))
-              (sha256
-               (base32 "0y1669sqj8wyf0y0njhxs4qhn1qzjhrs2h2qllya5samxrlrjhkg"))
-              (modules '((guix build utils)))
-              (snippet
-               '(begin
-                  (for-each (lambda (dir)
-                              (delete-file-recursively
-                                (string-append "src/third_party/" dir)))
-                            '("pcre-8.42" "scons-2.5.0" "snappy-1.1.3"
-                              "valgrind-3.11.0" "wiredtiger"
-                              "yaml-cpp-0.6.2" "zlib-1.2.11"))
-                  #t))))
-    (build-system scons-build-system)
-    (inputs
-     `(("openssl" ,openssl-1.0)
-       ("pcre" ,pcre)
-        ,@(match (%current-system)
-            ((or "x86_64-linux" "aarch64-linux" "mips64el-linux")
-             `(("wiredtiger" ,wiredtiger)))
-            (_ `()))
-       ("yaml-cpp" ,yaml-cpp)
-       ("zlib" ,zlib)
-       ("snappy" ,snappy)))
-    (native-inputs
-     `(("valgrind" ,valgrind)
-       ("perl" ,perl)
-       ("python" ,python-2)
-       ("python2-pymongo" ,python2-pymongo)
-       ("python2-pyyaml" ,python2-pyyaml)
-       ("tzdata" ,tzdata-for-tests)))
-    (arguments
-     `(#:scons ,scons-python2
-       #:phases
-       (let ((common-options
-              `(;; "--use-system-tcmalloc" TODO: Missing gperftools
-                "--use-system-pcre"
-                ;; wiredtiger is 64-bit only
-                ,,(if (any (cute string-prefix? <> (or (%current-target-system)
-                                                       (%current-system)))
-                           '("i686-linux" "armhf-linux"))
-                    ``"--wiredtiger=off"
-                    ``"--use-system-wiredtiger")
-                ;; TODO
-                ;; build/opt/mongo/db/fts/unicode/string.o failed: Error 1
-                ;; --use-system-boost
-                "--use-system-snappy"
-                "--use-system-zlib"
-                "--use-system-valgrind"
-                ;; "--use-system-stemmer" TODO: Missing relevant package
-                "--use-system-yaml"
-                "--disable-warnings-as-errors"
-                ,(format #f "--jobs=~a" (parallel-job-count))
-                "--ssl")))
-         (modify-phases %standard-phases
-           (add-after 'unpack 'patch
-             (lambda _
-               ;; Remove use of GNU extensions in parse_number_test.cpp, to
-               ;; allow compiling with GCC 7 or later
-               ;; https://jira.mongodb.org/browse/SERVER-28063
-               (substitute* "src/mongo/base/parse_number_test.cpp"
-                 (("0xabcab\\.defdefP-10")
-                  "687.16784283419838"))
-               #t))
-           (add-after 'unpack 'scons-propagate-environment
-             (lambda _
-               ;; Modify the SConstruct file to arrange for
-               ;; environment variables to be propagated.
-               (substitute* "SConstruct"
-                 (("^env = Environment\\(")
-                  "env = Environment(ENV=os.environ, "))
-               #t))
-           (add-after 'unpack 'create-version-file
-             (lambda _
-               (call-with-output-file "version.json"
-                 (lambda (port)
-                   (display ,(simple-format #f "{
-    \"version\": \"~A\"
-}" version) port)))
-               #t))
-           (replace 'build
-             (lambda _
-               (apply invoke `("scons"
-                               ,@common-options
-                               "mongod" "mongo" "mongos"))))
-           (replace 'check
-             (lambda* (#:key tests? inputs #:allow-other-keys)
-               (setenv "TZDIR"
-                       (string-append (assoc-ref inputs "tzdata")
-                                      "/share/zoneinfo"))
-               (when tests?
-                 ;; Note that with the tests, especially the unittests, the
-                 ;; build can take up to ~45GB of space, as many tests are
-                 ;; individual executable files, with some being hundreds of
-                 ;; megabytes in size.
-                 (apply invoke `("scons" ,@common-options "dbtest" "unittests"))
-                 (substitute* "build/unittests.txt"
-                   ;; TODO: Don't run the async_stream_test, as it hangs
-                   (("^build\\/opt\\/mongo\\/executor\\/async\\_stream\\_test\n$")
-                    "")
-                   ;; TODO: This test fails
-                   ;; Expected 0UL != disks.size() (0 != 0) @src/mongo/util/procparser_test.cpp:476
-                   (("^build\\/opt\\/mongo\\/util\\/procparser\\_test\n$")
-                    ""))
-                 (invoke "python" "buildscripts/resmoke.py"
-                         "--suites=dbtest,unittests"
-                         (format #f  "--jobs=~a" (parallel-job-count))))
-               #t))
-           (replace 'install
-             (lambda* (#:key outputs #:allow-other-keys)
-               (let ((bin (string-append (assoc-ref outputs "out") "/bin")))
-                 (install-file "mongod" bin)
-                 (install-file "mongos" bin)
-                 (install-file "mongo" bin))
-               #t))))))
-    (home-page "https://www.mongodb.org/")
-    (synopsis "High performance and high availability document database")
-    (description
-     "Mongo is a high-performance, high availability, schema-free
-document-oriented database.  A key goal of MongoDB is to bridge the gap
-between key/value stores (which are fast and highly scalable) and traditional
-RDBMS systems (which are deep in functionality).")
-    (license (list license:agpl3
-                   ;; Some parts are licensed under the Apache License
-                   license:asl2.0))))
-
 (define-public mycli
   (package
     (name "mycli")
-- 
2.30.2

[bug#47081] [PATCH v2 1/6] gnu: Remove go-gopkg.in-mgo.v2.

[Léo Le Bouter via Guix-patches via]

Preparing for mongodb removal, first it's dependents must be removed.

* gnu/packages/databases.scm (go-gopkg.in-mgo.v2): Remove.
---
 gnu/packages/databases.scm | 46 --------------------------------------
 1 file changed, 46 deletions(-)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 8705cd2be4..437449174f 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -193,52 +193,6 @@
 either single machines or networked clusters.")
     (license license:gpl3+)))
 
-(define-public go-gopkg.in-mgo.v2
-  (package
-    (name "go-gopkg.in-mgo.v2")
-    (version "2016.08.01")
-    (source (origin
-              (method git-fetch)
-              (uri (git-reference
-                    (url "https://github.com/go-mgo/mgo")
-                    (commit (string-append "r" version))))
-              (file-name (git-file-name name version))
-              (sha256
-               (base32
-                "0rwbi1z63w43b0z9srm8m7iz1fdwx7bq7n2mz862d6liiaqa59jd"))))
-    (build-system go-build-system)
-    (arguments
-     `(#:import-path "gopkg.in/mgo.v2"
-       ;; TODO: The tests fail as MongoDB fails to start
-       ;; Error parsing command line: unrecognised option '--chunkSize'
-       #:tests? #f
-       #:phases
-       (modify-phases %standard-phases
-         (delete 'reset-gzip-timestamps)
-         (add-before 'check 'start-mongodb
-           (lambda* (#:key tests? #:allow-other-keys)
-             (when tests?
-               (with-directory-excursion "src/gopkg.in/mgo.v2"
-                 (invoke "make" "startdb")))
-             #t))
-         (add-after 'check 'stop'mongodb
-           (lambda* (#:key tests? #:allow-other-keys)
-             (when tests?
-               (with-directory-excursion "src/gopkg.in/mgo.v2"
-                 (invoke "make" "stopdb")))
-             #t)))))
-    (native-inputs
-     `(("go-gopkg.in-check.v1" ,go-gopkg.in-check.v1)
-       ("mongodb" ,mongodb)
-       ("daemontools" ,daemontools)))
-    (synopsis "@code{mgo} offers a rich MongoDB driver for Go.")
-    (description
-     "@code{mgo} (pronounced as mango) is a MongoDB driver for the Go language.
-It implements a rich selection of features under a simple API following
-standard Go idioms.")
-    (home-page "https://labix.org/mgo")
-    (license license:bsd-2)))
-
 (define-public ephemeralpg
   (package
     (name "ephemeralpg")
-- 
2.30.2

[bug#47081] [PATCH v2 2/6] gnu: Remove mongo-tools.

[Léo Le Bouter via Guix-patches via]

Preparing for mongodb removal, first it's dependents must be removed.

* gnu/packages/databases.scm (mongo-tools): Remove.
---
 gnu/packages/databases.scm | 115 -------------------------------------
 1 file changed, 115 deletions(-)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 437449174f..2330f9fd26 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -3640,121 +3640,6 @@ transforms idiomatic python function calls to well-formed SQL queries.")
 the SQL language using a syntax that reflects the resulting query.")
     (license license:asl2.0)))
 
-(define-public mongo-tools
-  (package
-    (name "mongo-tools")
-    (version "3.4.0")
-    (source
-     (origin (method git-fetch)
-             (uri (git-reference
-                   (url "https://github.com/mongodb/mongo-tools")
-                   (commit (string-append "r" version))))
-             (file-name (git-file-name name version))
-             (sha256
-              (base32
-               "1bcsz5cvj39a7nsxsfqmz9igrw33j6yli9kffigqyscs52amw7x1"))))
-    (build-system go-build-system)
-    (arguments
-     `(#:import-path "github.com/mongodb/mongo-tools"
-       #:modules ((srfi srfi-1)
-                  (guix build go-build-system)
-                  (guix build utils))
-       #:install-source? #f
-       #:phases
-       (let ((all-tools
-              '("bsondump" "mongodump" "mongoexport" "mongofiles"
-                "mongoimport" "mongooplog" "mongorestore"
-                "mongostat" "mongotop")))
-         (modify-phases %standard-phases
-           (add-after 'unpack 'delete-bundled-source-code
-             (lambda _
-               (delete-file-recursively
-                "src/github.com/mongodb/mongo-tools/vendor")
-               #t))
-           (add-after 'delete-bundled-source-code 'patch-source
-             (lambda _
-               ;; Remove a redundant argument that causes compilation to fail.
-               (substitute*
-                   "src/github.com/mongodb/mongo-tools/mongorestore/filepath.go"
-                 (("skipping restore of system.profile collection\", db)")
-                  "skipping restore of system.profile collection\")"))
-               #t))
-           (replace 'build
-             (lambda _
-               (for-each (lambda (tool)
-                           (let ((command
-                                  `("go" "build"
-                                    ;; This is where the tests expect to find the
-                                    ;; executables
-                                    "-o" ,(string-append
-                                           "src/github.com/mongodb/mongo-tools/bin/"
-                                           tool)
-                                    "-v"
-                                    "-tags=\"ssl sasl\""
-                                    "-ldflags"
-                                    "-extldflags=-Wl,-z,now,-z,relro"
-                                    ,(string-append
-                                      "src/github.com/mongodb/mongo-tools/"
-                                      tool "/main/" tool ".go"))))
-                             (simple-format #t "build: running ~A\n"
-                                            (string-join command))
-                             (apply invoke command)))
-                         all-tools)
-               #t))
-           (replace 'check
-             (lambda _
-               (with-directory-excursion "src"
-                 (for-each (lambda (tool)
-                             (invoke
-                              "go" "test" "-v"
-                              (string-append "github.com/mongodb/mongo-tools/"
-                                             tool)))
-                           all-tools))
-               #t))
-           (replace 'install
-             (lambda* (#:key outputs #:allow-other-keys)
-               (for-each (lambda (tool)
-                           (install-file
-                            (string-append "src/github.com/mongodb/mongo-tools/bin/"
-                                           tool)
-                            (string-append (assoc-ref outputs "out")
-                                           "/bin")))
-                         all-tools)
-               #t))))))
-    (native-inputs
-     `(("go-github.com-howeyc-gopass" ,go-github.com-howeyc-gopass)
-       ("go-github.com-jessevdk-go-flags" ,go-github.com-jessevdk-go-flags)
-       ("go-golang-org-x-crypto" ,go-golang-org-x-crypto)
-       ("go-gopkg.in-mgo.v2" ,go-gopkg.in-mgo.v2)
-       ("go-gopkg.in-tomb.v2" ,go-gopkg.in-tomb.v2)
-       ("go-github.com-nsf-termbox-go" ,go-github.com-nsf-termbox-go)
-       ("go-github.com-smartystreets-goconvey" ,go-github.com-smartystreets-goconvey)))
-    (home-page "https://github.com/mongodb/mongo-tools")
-    (synopsis "Various tools for interacting with MongoDB and BSON")
-    (description
-     "This package includes a collection of tools related to MongoDB.
-@table @code
-@item bsondump
-Display BSON files in a human-readable format
-@item mongoimport
-Convert data from JSON, TSV or CSV and insert them into a collection
-@item mongoexport
-Write an existing collection to CSV or JSON format
-@item mongodump/mongorestore
-Dump MongoDB backups to disk in the BSON format
-@item mongorestore
-Read MongoDB backups in the BSON format, and restore them to a live database
-@item mongostat
-Monitor live MongoDB servers, replica sets, or sharded clusters
-@item mongofiles
-Read, write, delete, or update files in GridFS
-@item mongooplog
-Replay oplog entries between MongoDB servers
-@item mongotop
-Monitor read/write activity on a mongo server
-@end table")
-    (license license:asl2.0)))
-
 ;; There are many wrappers for this in other languages. When touching, please
 ;; be sure to ensure all dependencies continue to build.
 (define-public apache-arrow
-- 
2.30.2

[bug#47081] [PATCH v2 3/6] doc: Remove mongodb-service-type.

[Léo Le Bouter via Guix-patches via]

* doc/guix.texi (mongodb-service-type): Remove.
---
 doc/guix.texi | 28 ----------------------------
 1 file changed, 28 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 77aafafd97..bd51f7bf81 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -19804,34 +19804,6 @@ Additional command line options to pass to @code{memcached}.
 @end table
 @end deftp
 
-@subsubheading MongoDB
-
-@defvr {Scheme Variable} mongodb-service-type
-This is the service type for @uref{https://www.mongodb.com/, MongoDB}.
-The value for the service type is a @code{mongodb-configuration} object.
-@end defvr
-
-@lisp
-(service mongodb-service-type)
-@end lisp
-
-@deftp {Data Type} mongodb-configuration
-Data type representing the configuration of mongodb.
-
-@table @asis
-@item @code{mongodb} (default: @code{mongodb})
-The MongoDB package to use.
-
-@item @code{config-file} (default: @code{%default-mongodb-configuration-file})
-The configuration file for MongoDB.
-
-@item @code{data-directory} (default: @code{"/var/lib/mongodb"})
-This value is used to create the directory, so that it exists and is
-owned by the mongodb user.  It should match the data-directory which
-MongoDB is configured to use through the configuration file.
-@end table
-@end deftp
-
 @subsubheading Redis
 
 @defvr {Scheme Variable} redis-service-type
-- 
2.30.2

[bug#47081] [PATCH v2 4/6] tests: databases: Remove mongodb test.

[Léo Le Bouter via Guix-patches via]

* gnu/tests/databases.scm (%test-mongodb, %mongodb-os, run-mongodb-test):
Remove.
---
 gnu/tests/databases.scm | 83 -----------------------------------------
 1 file changed, 83 deletions(-)

diff --git a/gnu/tests/databases.scm b/gnu/tests/databases.scm
index e831d69f5a..4bfe4ee282 100644
--- a/gnu/tests/databases.scm
+++ b/gnu/tests/databases.scm
@@ -30,7 +30,6 @@
   #:use-module (guix gexp)
   #:use-module (guix store)
   #:export (%test-memcached
-            %test-mongodb
             %test-postgresql
             %test-mysql))
 
@@ -127,88 +126,6 @@
    (description "Connect to a running MEMCACHED server.")
    (value (run-memcached-test))))
 
-(define %mongodb-os
-  (operating-system
-    (inherit
-     (simple-operating-system
-      (service dhcp-client-service-type)
-      (service mongodb-service-type)))
-    (packages (cons* mongodb
-                     %base-packages))))
-
-(define* (run-mongodb-test #:optional (port 27017))
-  "Run tests in %MONGODB-OS, forwarding PORT."
-  (define os
-    (marionette-operating-system
-     %mongodb-os
-     #:imported-modules '((gnu services herd)
-                          (guix combinators))))
-
-  (define vm
-    (virtual-machine
-     (operating-system os)
-     (memory-size 1024)
-     (disk-image-size (* 1024 (expt 2 20)))
-     (port-forwardings `((27017 . ,port)))))
-
-  (define test
-    (with-imported-modules '((gnu build marionette))
-      #~(begin
-          (use-modules (srfi srfi-11) (srfi srfi-64)
-                       (gnu build marionette)
-                       (ice-9 popen)
-                       (ice-9 rdelim))
-
-          (define marionette
-            (make-marionette (list #$vm)))
-
-          (mkdir #$output)
-          (chdir #$output)
-
-          (test-begin "mongodb")
-
-          (test-assert "service running"
-            (marionette-eval
-             '(begin
-                (use-modules (gnu services herd))
-                (match (start-service 'mongodb)
-                  (#f #f)
-                  (('service response-parts ...)
-                   (match (assq-ref response-parts 'running)
-                     ((pid) (number? pid))))))
-             marionette))
-
-          (test-eq "test insert"
-            0
-            (system* (string-append #$mongodb "/bin/mongo")
-                     "test"
-                     "--eval"
-                     "db.testCollection.insert({data: 'test-data'})"))
-
-          (test-equal "test find"
-            "test-data"
-            (let* ((port (open-pipe*
-                          OPEN_READ
-                          (string-append #$mongodb "/bin/mongo")
-                          "test"
-                          "--quiet"
-                          "--eval"
-                          "db.testCollection.findOne().data"))
-                   (output (read-line port))
-                   (status (close-pipe port)))
-              output))
-
-          (test-end)
-          (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
-
-  (gexp->derivation "mongodb-test" test))
-
-(define %test-mongodb
-  (system-test
-   (name "mongodb")
-   (description "Connect to a running MONGODB server.")
-   (value (run-mongodb-test))))
-
 \f
 ;;;
 ;;; The PostgreSQL service.
-- 
2.30.2

[bug#47081] [PATCH v2 5/6] services: Remove mongodb service.

[Léo Le Bouter via Guix-patches via]

* gnu/services/databases.scm (mongodb-configuration, mongodb-configuration?,
mongodb-configuration-mongodb, mongodb-configuration-config-file,
mongodb-configuration-data-directory, mongodb-service-type,
%default-mongodb-configuration-file, %mongodb-accounts, mongodb-activation,
mongodb-shepherd-service): Remove.
---
 gnu/services/databases.scm | 88 --------------------------------------
 1 file changed, 88 deletions(-)

diff --git a/gnu/services/databases.scm b/gnu/services/databases.scm
index 979f3dd6c8..a841e7a50e 100644
--- a/gnu/services/databases.scm
+++ b/gnu/services/databases.scm
@@ -79,13 +79,6 @@
             memcached-configuration-udp-port
             memcached-configuration-additional-options
 
-            mongodb-configuration
-            mongodb-configuration?
-            mongodb-configuration-mongodb
-            mongodb-configuration-config-file
-            mongodb-configuration-data-directory
-            mongodb-service-type
-
             mysql-service
             mysql-service-type
             mysql-configuration
@@ -521,87 +514,6 @@ created after the PostgreSQL database is started.")))
                                           (const %memcached-accounts))))
                 (default-value (memcached-configuration))))
 
-\f
-;;;
-;;; MongoDB
-;;;
-
-(define %default-mongodb-configuration-file
-  (plain-file
-   "mongodb.yaml"
-   "# GNU Guix: MongoDB default configuration file
-processManagement:
-  pidFilePath: /var/run/mongodb/pid
-storage:
-  dbPath: /var/lib/mongodb
-"))
-
-
-(define-record-type* <mongodb-configuration>
-  mongodb-configuration make-mongodb-configuration
-  mongodb-configuration?
-  (mongodb             mongodb-configuration-mongodb
-                       (default mongodb))
-  (config-file         mongodb-configuration-config-file
-                       (default %default-mongodb-configuration-file))
-  (data-directory      mongodb-configuration-data-directory
-                       (default "/var/lib/mongodb")))
-
-(define %mongodb-accounts
-  (list (user-group (name "mongodb") (system? #t))
-        (user-account
-         (name "mongodb")
-         (group "mongodb")
-         (system? #t)
-         (comment "Mongodb server user")
-         (home-directory "/var/lib/mongodb")
-         (shell (file-append shadow "/sbin/nologin")))))
-
-(define mongodb-activation
-  (match-lambda
-    (($ <mongodb-configuration> mongodb config-file data-directory)
-     #~(begin
-         (use-modules (guix build utils))
-         (let ((user (getpwnam "mongodb")))
-           (for-each
-            (lambda (directory)
-              (mkdir-p directory)
-              (chown directory
-                     (passwd:uid user) (passwd:gid user)))
-            '("/var/run/mongodb" #$data-directory)))))))
-
-(define mongodb-shepherd-service
-  (match-lambda
-    (($ <mongodb-configuration> mongodb config-file data-directory)
-     (shepherd-service
-      (provision '(mongodb))
-      (documentation "Run the Mongodb daemon.")
-      (requirement '(user-processes loopback))
-      (start #~(make-forkexec-constructor
-                `(,(string-append #$mongodb "/bin/mongod")
-                  "--config"
-                  ,#$config-file)
-                #:user "mongodb"
-                #:group "mongodb"
-                #:pid-file "/var/run/mongodb/pid"
-                #:log-file "/var/log/mongodb.log"))
-      (stop #~(make-kill-destructor))))))
-
-(define mongodb-service-type
-  (service-type
-   (name 'mongodb)
-   (description "Run the MongoDB document database server.")
-   (extensions
-    (list (service-extension shepherd-root-service-type
-                             (compose list
-                                      mongodb-shepherd-service))
-          (service-extension activation-service-type
-                             mongodb-activation)
-          (service-extension account-service-type
-                             (const %mongodb-accounts))))
-   (default-value
-     (mongodb-configuration))))
-
 \f
 ;;;
 ;;; MySQL.
-- 
2.30.2

[bug#47081] [PATCH v2 5/6] services: Remove mongodb service.

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 811 bytes --]


Léo Le Bouter via Guix-patches via <guix-patches@gnu.org> writes:

> * gnu/services/databases.scm (mongodb-configuration, mongodb-configuration?,
> mongodb-configuration-mongodb, mongodb-configuration-config-file,
> mongodb-configuration-data-directory, mongodb-service-type,
> %default-mongodb-configuration-file, %mongodb-accounts, mongodb-activation,
> mongodb-shepherd-service): Remove.
> ---
>  gnu/services/databases.scm | 88 --------------------------------------
>  1 file changed, 88 deletions(-)

I think it would be better to squash this commit in with the two
previous commits, so there just one commit to remove the service,
including the tests and documentation. That's a better "atomic" change,
since all three bits (service itself, test and docs) are one thing in my
opinion.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

[bug#47081] [PATCH v2 5/6] services: Remove mongodb service.

[Léo Le Bouter via Guix-patches via]

[-- Attachment #1: Type: text/plain, Size: 651 bytes --]

On Sun, 2021-03-14 at 14:54 +0000, Christopher Baines wrote:
> I think it would be better to squash this commit in with the two
> previous commits, so there just one commit to remove the service,
> including the tests and documentation. That's a better "atomic"
> change,
> since all three bits (service itself, test and docs) are one thing in
> my
> opinion.

Oh no, then all the individual commits and ordering work (so no single
commit breaks GNU Guix or makes it inconsistent) is useless now :-(

I will try squashing ASAP, but what do you recommend as title then? It
seems to me this kind of violates GNU commit message guidelines.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#47081] [PATCH v2 5/6] services: Remove mongodb service.

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1190 bytes --]


Léo Le Bouter <lle-bout@zaclys.net> writes:

> On Sun, 2021-03-14 at 14:54 +0000, Christopher Baines wrote:
>> I think it would be better to squash this commit in with the two
>> previous commits, so there just one commit to remove the service,
>> including the tests and documentation. That's a better "atomic"
>> change,
>> since all three bits (service itself, test and docs) are one thing in
>> my
>> opinion.
>
> Oh no, then all the individual commits and ordering work (so no single
> commit breaks GNU Guix or makes it inconsistent) is useless now :-(

I don't think there's going to be any problems with combining the
commits I'm suggesting. Given they're contiguous, there's no chance of
it breaking something, right?

> I will try squashing ASAP, but what do you recommend as title then? It
> seems to me this kind of violates GNU commit message guidelines.

I'm no expert at writing commit messages in the Guix/GNU style, but
maybe take a look at the commit that added the service [1]. I think
"services: Remove MongoDB." would be a reasonable title.

1: https://git.savannah.gnu.org/cgit/guix.git/commit/?id=5266ff719e274056cb3e2b9740183f0063177255

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 987 bytes --]

bug#47081: [PATCH v2 5/6] services: Remove mongodb service.

[Léo Le Bouter via Guix-patches via]

[-- Attachment #1: Type: text/plain, Size: 102 bytes --]

I squashed and pushed as 097cf21d1d34f4f66a403c3fb6e15b6709c4dd8a.

Hope everything is alright :-D

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#47081] [PATCH v2 6/6] gnu: Remove mongodb.

[Léo Le Bouter via Guix-patches via]

mongodb 3.4.10 has unpatched CVEs and mongodb 3.4.24 has some files in the
release tarball under the SSPL, therefore we cannot provide mongodb while
upholding to good security standards.

* gnu/packages/databases.scm (mongodb): Remove.
---
 gnu/packages/databases.scm | 137 -------------------------------------
 1 file changed, 137 deletions(-)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 2330f9fd26..5249aa0d10 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -615,143 +615,6 @@ replacement for the code@{python-memcached} library.")
 (define-public python2-pylibmc
   (package-with-python2 python-pylibmc))
 
-;; There is a point at which mongodb switched to the Server Side Public
-;; License (SSPL), which is not a FOSS license.  As such, be careful
-;; when updating this package.
-(define-public mongodb
-  (package
-    (name "mongodb")
-    (version "3.4.24")
-    (source (origin
-              (method url-fetch)
-              (uri (string-append "https://github.com/mongodb/mongo/archive/r"
-                                  version ".tar.gz"))
-              (file-name (string-append name "-" version ".tar.gz"))
-              (sha256
-               (base32 "0y1669sqj8wyf0y0njhxs4qhn1qzjhrs2h2qllya5samxrlrjhkg"))
-              (modules '((guix build utils)))
-              (snippet
-               '(begin
-                  (for-each (lambda (dir)
-                              (delete-file-recursively
-                                (string-append "src/third_party/" dir)))
-                            '("pcre-8.42" "scons-2.5.0" "snappy-1.1.3"
-                              "valgrind-3.11.0" "wiredtiger"
-                              "yaml-cpp-0.6.2" "zlib-1.2.11"))
-                  #t))))
-    (build-system scons-build-system)
-    (inputs
-     `(("openssl" ,openssl-1.0)
-       ("pcre" ,pcre)
-        ,@(match (%current-system)
-            ((or "x86_64-linux" "aarch64-linux" "mips64el-linux")
-             `(("wiredtiger" ,wiredtiger)))
-            (_ `()))
-       ("yaml-cpp" ,yaml-cpp)
-       ("zlib" ,zlib)
-       ("snappy" ,snappy)))
-    (native-inputs
-     `(("valgrind" ,valgrind)
-       ("perl" ,perl)
-       ("python" ,python-2)
-       ("python2-pymongo" ,python2-pymongo)
-       ("python2-pyyaml" ,python2-pyyaml)
-       ("tzdata" ,tzdata-for-tests)))
-    (arguments
-     `(#:scons ,scons-python2
-       #:phases
-       (let ((common-options
-              `(;; "--use-system-tcmalloc" TODO: Missing gperftools
-                "--use-system-pcre"
-                ;; wiredtiger is 64-bit only
-                ,,(if (any (cute string-prefix? <> (or (%current-target-system)
-                                                       (%current-system)))
-                           '("i686-linux" "armhf-linux"))
-                    ``"--wiredtiger=off"
-                    ``"--use-system-wiredtiger")
-                ;; TODO
-                ;; build/opt/mongo/db/fts/unicode/string.o failed: Error 1
-                ;; --use-system-boost
-                "--use-system-snappy"
-                "--use-system-zlib"
-                "--use-system-valgrind"
-                ;; "--use-system-stemmer" TODO: Missing relevant package
-                "--use-system-yaml"
-                "--disable-warnings-as-errors"
-                ,(format #f "--jobs=~a" (parallel-job-count))
-                "--ssl")))
-         (modify-phases %standard-phases
-           (add-after 'unpack 'patch
-             (lambda _
-               ;; Remove use of GNU extensions in parse_number_test.cpp, to
-               ;; allow compiling with GCC 7 or later
-               ;; https://jira.mongodb.org/browse/SERVER-28063
-               (substitute* "src/mongo/base/parse_number_test.cpp"
-                 (("0xabcab\\.defdefP-10")
-                  "687.16784283419838"))
-               #t))
-           (add-after 'unpack 'scons-propagate-environment
-             (lambda _
-               ;; Modify the SConstruct file to arrange for
-               ;; environment variables to be propagated.
-               (substitute* "SConstruct"
-                 (("^env = Environment\\(")
-                  "env = Environment(ENV=os.environ, "))
-               #t))
-           (add-after 'unpack 'create-version-file
-             (lambda _
-               (call-with-output-file "version.json"
-                 (lambda (port)
-                   (display ,(simple-format #f "{
-    \"version\": \"~A\"
-}" version) port)))
-               #t))
-           (replace 'build
-             (lambda _
-               (apply invoke `("scons"
-                               ,@common-options
-                               "mongod" "mongo" "mongos"))))
-           (replace 'check
-             (lambda* (#:key tests? inputs #:allow-other-keys)
-               (setenv "TZDIR"
-                       (string-append (assoc-ref inputs "tzdata")
-                                      "/share/zoneinfo"))
-               (when tests?
-                 ;; Note that with the tests, especially the unittests, the
-                 ;; build can take up to ~45GB of space, as many tests are
-                 ;; individual executable files, with some being hundreds of
-                 ;; megabytes in size.
-                 (apply invoke `("scons" ,@common-options "dbtest" "unittests"))
-                 (substitute* "build/unittests.txt"
-                   ;; TODO: Don't run the async_stream_test, as it hangs
-                   (("^build\\/opt\\/mongo\\/executor\\/async\\_stream\\_test\n$")
-                    "")
-                   ;; TODO: This test fails
-                   ;; Expected 0UL != disks.size() (0 != 0) @src/mongo/util/procparser_test.cpp:476
-                   (("^build\\/opt\\/mongo\\/util\\/procparser\\_test\n$")
-                    ""))
-                 (invoke "python" "buildscripts/resmoke.py"
-                         "--suites=dbtest,unittests"
-                         (format #f  "--jobs=~a" (parallel-job-count))))
-               #t))
-           (replace 'install
-             (lambda* (#:key outputs #:allow-other-keys)
-               (let ((bin (string-append (assoc-ref outputs "out") "/bin")))
-                 (install-file "mongod" bin)
-                 (install-file "mongos" bin)
-                 (install-file "mongo" bin))
-               #t))))))
-    (home-page "https://www.mongodb.org/")
-    (synopsis "High performance and high availability document database")
-    (description
-     "Mongo is a high-performance, high availability, schema-free
-document-oriented database.  A key goal of MongoDB is to bridge the gap
-between key/value stores (which are fast and highly scalable) and traditional
-RDBMS systems (which are deep in functionality).")
-    (license (list license:agpl3
-                   ;; Some parts are licensed under the Apache License
-                   license:asl2.0))))
-
 (define-public mycli
   (package
     (name "mycli")
-- 
2.30.2

Why [bug#47081] Remove mongodb?

[zimoun]

Hi Léo,

On Fri, 12 Mar 2021 at 01:56, Léo Le Bouter <lle-bout@zaclys.net> wrote:

> mongodb 3.4.10 has unpatched CVEs and mongodb 3.4.24 has some files in the
> release tarball under the SSPL, therefore we cannot provide mongodb while
> upholding to good security standards.

[...]

>  doc/guix.texi              |  28 -----
>  gnu/packages/databases.scm | 252 -------------------------------------
>  gnu/services/databases.scm |  88 -------------
>  gnu/tests/databases.scm    |  83 ------------
>  4 files changed, 451 deletions(-)

Could you wait more than 4 days between the patch submission and
effectively pushing it?

Well, you updated mongodb from 3.4.10 to 3.4.24 on the March 10th,
submitted a patch series for the removal on the March 12th and pushed on
the March 16th.  In the meantime, the update has been reverted on the
March 11th because of license issue, IIUC.


If the removal for security reasons had been discussed on IRC, it could
be nice to point the discussion here.  Otherwise, open a discussion on
the topic on guix-devel or bug-guix.  The full removal is a radical
solution (especially, it should be done with 2 commits: service+doc and
then package; well another story).


All the best,
simon

Re: Why [bug#47081] Remove mongodb?

[Léo Le Bouter]

[-- Attachment #1: Type: text/plain, Size: 1206 bytes --]

On Wed, 2021-03-17 at 17:56 +0100, zimoun wrote:
> If the removal for security reasons had been discussed on IRC, it
> could
> be nice to point the discussion here.  Otherwise, open a discussion
> on
> the topic on guix-devel or bug-guix.  The full removal is a radical
> solution (especially, it should be done with 2 commits: service+doc
> and
> then package; well another story).

https://issues.guix.gnu.org/47081 - some of it there: 
https://logs.guix.gnu.org/guix/2021-03-12.log#001752

Efraim, Cbaines, Lfam was involved there and shown no big objections

> 
> Well, you updated mongodb from 3.4.10 to 3.4.24 on the March 10th,
> submitted a patch series for the removal on the March 12th and pushed
> on
> the March 16th.  In the meantime, the update has been reverted on the
> March 11th because of license issue, IIUC.
> 

The security update was reverted, then the revert was reverted due to
debate on licensing which turns out reverting the revert was actually
wrong because some specific files were under SSPL, at that point we
were shipping SSPL code which is nonfree, so the removal is also that.

Nonfree code + security issue made it kind of stressful

Léo

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Why [bug#47081] Remove mongodb?

[zimoun]

On Wed, 17 Mar 2021 at 18:09, Léo Le Bouter <lle-bout@zaclys.net> wrote:
> On Wed, 2021-03-17 at 17:56 +0100, zimoun wrote:
>> If the removal for security reasons had been discussed on IRC, it
>> could
>> be nice to point the discussion here.  Otherwise, open a discussion
>> on
>> the topic on guix-devel or bug-guix.  The full removal is a radical
>> solution (especially, it should be done with 2 commits: service+doc
>> and
>> then package; well another story).
>
> https://issues.guix.gnu.org/47081 - some of it there: 
> https://logs.guix.gnu.org/guix/2021-03-12.log#001752
>
> Efraim, Cbaines, Lfam was involved there and shown no big objections

Thanks.


>> Well, you updated mongodb from 3.4.10 to 3.4.24 on the March 10th,
>> submitted a patch series for the removal on the March 12th and pushed
>> on
>> the March 16th.  In the meantime, the update has been reverted on the
>> March 11th because of license issue, IIUC.
>> 
>
> The security update was reverted, then the revert was reverted due to
> debate on licensing which turns out reverting the revert was actually
> wrong because some specific files were under SSPL, at that point we
> were shipping SSPL code which is nonfree, so the removal is also that.

AFAIT, 3.4.10 is released under GNU AGPL 3.0 and Apache 2.0.  This
version had been released before the October 16th, 2018.  Could you
point which code is non-free?

IMHO, this claim about non-free code is wrong.  The last versions with
an acceptable license seem 4.0.3 or 4.1.4, I guess.

I am not against removing MongoBD.  I am just saying that the removal
deserves at least a message on guix-devel and maybe a --news entry.

Other said, it deserves more than 6 days between the “oh there is
security vulnerabilities” and the full removal.  When one uses a version
from 2017 as 3.4.10 is, one knows that it can have security
vulnerabilities.

I am not complaining about the commit itself, but I am complaining by
the way of doing the thing.


All the best,
simon

Re: Why [bug#47081] Remove mongodb?

[Léo Le Bouter]

[-- Attachment #1: Type: text/plain, Size: 1435 bytes --]

On Wed, 2021-03-17 at 18:56 +0100, zimoun wrote:
> AFAIT, 3.4.10 is released under GNU AGPL 3.0 and Apache 2.0.  This
> version had been released before the October 16th, 2018.  Could you
> point which code is non-free?
> 
> IMHO, this claim about non-free code is wrong.  The last versions
> with
> an acceptable license seem 4.0.3 or 4.1.4, I guess.

It's not wrong, look at 2f9132e2e0b1e01398a01a32972e87f45ec2f7a6, we
were shipping 3.4.24 before the removal, not 3.4.10.

> I am not against removing MongoBD.  I am just saying that the removal
> deserves at least a message on guix-devel and maybe a --news entry.
> 
> Other said, it deserves more than 6 days between the “oh there is
> security vulnerabilities” and the full removal.  When one uses a
> version
> from 2017 as 3.4.10 is, one knows that it can have security
> vulnerabilities.
> 
> I am not complaining about the commit itself, but I am complaining by
> the way of doing the thing.

I agree, will do differently in the future, no one mentionned it during
all discussions, but if it was I would've, 3-4 days did not give you
time to comment so I'll wait longer maybe re-re-revert the revert to
restore 3.4.10 instead so we get rid of the non-free code issue. Does
anyone actually use MongoDB on GNU Guix? Some people don't look at
versions or when they were released and just trust GNU Guix.

> 
> All the best,
> simon

Léo

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Why [bug#47081] Remove mongodb?

[zimoun]

On Wed, 17 Mar 2021 at 19:16, Léo Le Bouter <lle-bout@zaclys.net> wrote:
> On Wed, 2021-03-17 at 18:56 +0100, zimoun wrote:
>> AFAIT, 3.4.10 is released under GNU AGPL 3.0 and Apache 2.0.  This
>> version had been released before the October 16th, 2018.  Could you
>> point which code is non-free?
>> 
>> IMHO, this claim about non-free code is wrong.  The last versions
>> with
>> an acceptable license seem 4.0.3 or 4.1.4, I guess.
>
> It's not wrong, look at 2f9132e2e0b1e01398a01a32972e87f45ec2f7a6, we
> were shipping 3.4.24 before the removal, not 3.4.10.

It is exactly what I am complaining!  It is not possible to follow.

The version before the March 10th is 3.4.10.  This version is free and
from 2017; with security vulnerabilities but everything is fine.

Then less than 6 days later, the package is updated to 3.4.24 which is a
non-free version.  So reverted to 3.4.10.  So re-reverted to 3.4.24.
And last, removed.

It shows exactly my point.  The correct and polite way of doing the
thing is first to examine the issue at hand (3.4.10 is old with security
vulnerabilities), then propose a fix (e.g., the removal), wait feedback,
and complete.

Whatever, now it is done.  And as I said, I am not against the removal.


All the best,
simon

Re: Why [bug#47081] Remove mongodb?

[Léo Le Bouter]

[-- Attachment #1: Type: text/plain, Size: 515 bytes --]

The issue with 3.4.24 / 3.4.10 is that Efraim reverted the commit then
it was briefly discussed on IRC and Efraim thought I was right about
the licensing being fine on 3.4.24 and reverted their revert commit,
after some actual checking in the tarball grepping for license headers
I found out I was wrong and instead of reverting the revert of the
revert of Efraim the next change was removal because of other reasons.

Besides the openssl issue I think the commit message laid out these
things quite well.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Why [bug#47081] Remove mongodb?

[Léo Le Bouter]

[-- Attachment #1: Type: text/plain, Size: 1112 bytes --]

On Wed, 2021-03-17 at 19:51 +0100, zimoun wrote:
> It shows exactly my point.  The correct and polite way of doing the
> thing is first to examine the issue at hand (3.4.10 is old with
> security
> vulnerabilities), then propose a fix (e.g., the removal), wait
> feedback,
> and complete.

Actually we did not know pushing a security fix with 3.4.24 was not
fine, from quick auditing I have made 3.4.24 would still be under AGPL
so it would be fine to upgrade, turns out not since some files inside
are under SSPL but that was discovered way later, even when Efraim had
doubt and reverted my commit we had a debate and Efraim bought my
arguing even though I was wrong and they were right, if for every
security issue I have to ask feedback I may not ship them in a timely
manner, so that's also why they tend to be pushed faster than usual..
we may want to establish a clear process here. I usually create issues
for things I need help on, if I can do it myself and feel confident, I
just push, I can be wrong of course and always sorry for issues, I fix
them shortly in next commits if any.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Why [bug#47081] Remove mongodb?

[zimoun]

On Wed, 17 Mar 2021 at 20:11, Léo Le Bouter <lle-bout@zaclys.net> wrote:
> On Wed, 2021-03-17 at 19:51 +0100, zimoun wrote:
>> It shows exactly my point.  The correct and polite way of doing the
>> thing is first to examine the issue at hand (3.4.10 is old with
>> security
>> vulnerabilities), then propose a fix (e.g., the removal), wait
>> feedback,
>> and complete.
>
> Actually we did not know pushing a security fix with 3.4.24 was not
> fine, from quick auditing I have made 3.4.24 would still be under AGPL
> so it would be fine to upgrade, turns out not since some files inside
> are under SSPL but that was discovered way later, even when Efraim had

Later means here only hours.

> doubt and reverted my commit we had a debate and Efraim bought my
> arguing even though I was wrong and they were right, if for every
> security issue I have to ask feedback I may not ship them in a timely
> manner, so that's also why they tend to be pushed faster than usual..

Haste is not speed.

> we may want to establish a clear process here. I usually create issues
> for things I need help on, if I can do it myself and feel confident, I
> just push, I can be wrong of course and always sorry for issues, I fix
> them shortly in next commits if any.

I really appreciate your valuable work. I have the impression you think
that you have to push as fast as you can, whatever if it is the right
fix.  If I might, first please avoid to burn out and second do not
worry, the world will not explode because of a security vulnerability in
Guix.  Maybe one day when Guix will dominate the world, soon! :-)

I am not convinced that the regular Guix user is upgrading their package
set twice a day; maybe once a week at best and more probably time to
time.  Guix is rooted in The Right Thing™ and sometimes it means delay
to think what the right thing really is.  Therefore, the process is
already clear: go via guix-patch for non-trivial changes and wait
feedback.

At the end, I cannot express better what Tobias wrote:

   <https://yhetil.org/guix/87ft0un7ma.fsf@nckx>

or Leo:

   <https://yhetil.org/guix/YFEDt/PUd2ZeC6/F@jasmine.lan>
   

All the best,
simon

Re: Why [bug#47081] Remove mongodb?

[Ludovic Courtès]

Hi Léo,

Léo Le Bouter <lle-bout@zaclys.net> skribis:

> On Wed, 2021-03-17 at 17:56 +0100, zimoun wrote:
>> If the removal for security reasons had been discussed on IRC, it
>> could
>> be nice to point the discussion here.  Otherwise, open a discussion
>> on
>> the topic on guix-devel or bug-guix.  The full removal is a radical
>> solution (especially, it should be done with 2 commits: service+doc
>> and
>> then package; well another story).
>
> https://issues.guix.gnu.org/47081 - some of it there: 

Removing a package and its services is not something to do lightly: it
breaks user configs with no recourse.

We must insist on getting more opinions on such matters, and I think
there just wasn’t enough feedback here.  I understand it can be
frustrating to wait for input, but in such a case, please do.  This
project has always strove for consensus.

Remember that the opinion of those who’ve been taking care of security
issues in Guix for years, those who’ve been maintaining MongoDB, those
who wrote the service and its tests, are invaluable; they must have a
say.  I insist: humbly solicit and wait for their feedback.

Now, how do we move forward?  IMO we must look for available options
before we remove MongoDB.  Are there forks of the original
freely-licensed code base maintained around?  That sounds likely.  Are
there backports of the security fixes?  What do the previous
contributors to this code think—Chris, Efraim, Marius, Arun?

Léo, please get involved in reaching consensus on a solution.

Ludo’.

Re: Why [bug#47081] Remove mongodb?

[Léo Le Bouter]

[-- Attachment #1: Type: text/plain, Size: 2860 bytes --]

Hello!

> Removing a package and its services is not something to do lightly:
> it
> breaks user configs with no recourse.
> 
> We must insist on getting more opinions on such matters, and I think
> there just wasn’t enough feedback here.  I understand it can be
> frustrating to wait for input, but in such a case, please do.  This
> project has always strove for consensus.
> 
> Remember that the opinion of those who’ve been taking care of
> security
> issues in Guix for years, those who’ve been maintaining MongoDB,
> those
> who wrote the service and its tests, are invaluable; they must have a
> say.  I insist: humbly solicit and wait for their feedback.
> 

I understand, and I did not think it was a light thing to do, no one
mentionned anything we should do for the remove, so I actually do not
know how we handle that but the security/non-free code thing put some
urge into the situation, apologizes for moving on and pushing without
waiting for more feedback, few people gave their feedback on IRC and by
email and that's why I felt more confident doing the actual change.

> Now, how do we move forward?  IMO we must look for available options
> before we remove MongoDB.  Are there forks of the original
> freely-licensed code base maintained around?  That sounds likely.  

I never heard of any and after some searches even before I pushed the
remove commit it remained inconclusive on whether we can rely on a
fork.

> Are
> there backports of the security fixes? 

Ubuntu Focal maintains a package still but to me they still don't have
all the fixes, see: https://packages.ubuntu.com/focal/mongodb-server

All in all, I don't think we should keep a package in more-than-
maintenance mode when the upstream has decided to change the license,
they are uncooperative and making our work harder so I think we should
remove the package. It's not like we are an LTS distro like Ubuntu
Focal that absolutely must keep a package until the end of the support
cycle. It may break configs yes, but actually this had to happen, at
the same time they changed to a problematic nonfree license and openssl
1.1.1 is not supported on 3.4.x (Ubuntu uses 3.6.8 instead which also
is under AGPL but more recent than our 3.4.10 we had so supports
openssl 1.1.1 with some patches they made). I'm not particularily
sympathetic to MongoDB. Also are there actually people using the
mongodb service on GNU Guix?

> What do the previous
> contributors to this code think—Chris, Efraim, Marius, Arun?

Chris voiced their opinion saying they didnt mind removing the package,
I think Efraim said that on IRC also but I am not sure, so let's wait
for their input here.

> 
> Léo, please get involved in reaching consensus on a solution.

CC'd them, of course, again, sorry.

> Ludo’.

Léo



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Why [bug#47081] Remove mongodb?

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 3835 bytes --]

On Sun, Mar 21, 2021 at 11:15:32PM +0100, Léo Le Bouter wrote:
> Hello!
> 
> > Removing a package and its services is not something to do lightly:
> > it
> > breaks user configs with no recourse.
> > 
> > We must insist on getting more opinions on such matters, and I think
> > there just wasn’t enough feedback here.  I understand it can be
> > frustrating to wait for input, but in such a case, please do.  This
> > project has always strove for consensus.
> > 
> > Remember that the opinion of those who’ve been taking care of
> > security
> > issues in Guix for years, those who’ve been maintaining MongoDB,
> > those
> > who wrote the service and its tests, are invaluable; they must have a
> > say.  I insist: humbly solicit and wait for their feedback.
> > 
> 
> I understand, and I did not think it was a light thing to do, no one
> mentionned anything we should do for the remove, so I actually do not
> know how we handle that but the security/non-free code thing put some
> urge into the situation, apologizes for moving on and pushing without
> waiting for more feedback, few people gave their feedback on IRC and by
> email and that's why I felt more confident doing the actual change.
> 
> > Now, how do we move forward?  IMO we must look for available options
> > before we remove MongoDB.  Are there forks of the original
> > freely-licensed code base maintained around?  That sounds likely.  
> 
> I never heard of any and after some searches even before I pushed the
> remove commit it remained inconclusive on whether we can rely on a
> fork.
> 
> > Are
> > there backports of the security fixes? 
> 
> Ubuntu Focal maintains a package still but to me they still don't have
> all the fixes, see: https://packages.ubuntu.com/focal/mongodb-server
> 
> All in all, I don't think we should keep a package in more-than-
> maintenance mode when the upstream has decided to change the license,
> they are uncooperative and making our work harder so I think we should
> remove the package. It's not like we are an LTS distro like Ubuntu
> Focal that absolutely must keep a package until the end of the support
> cycle. It may break configs yes, but actually this had to happen, at
> the same time they changed to a problematic nonfree license and openssl
> 1.1.1 is not supported on 3.4.x (Ubuntu uses 3.6.8 instead which also
> is under AGPL but more recent than our 3.4.10 we had so supports
> openssl 1.1.1 with some patches they made). I'm not particularily
> sympathetic to MongoDB. Also are there actually people using the
> mongodb service on GNU Guix?
> 
> > What do the previous
> > contributors to this code think—Chris, Efraim, Marius, Arun?
> 
> Chris voiced their opinion saying they didnt mind removing the package,
> I think Efraim said that on IRC also but I am not sure, so let's wait
> for their input here.
> 
> > 
> > Léo, please get involved in reaching consensus on a solution.
> 
> CC'd them, of course, again, sorry.
> 
> > Ludo’.
> 
> Léo
> 

I don't have a strong opinion. I had hoped they'd return to a free
license but that doesn't seem to be the case. I see it a bit more from a
selfish angle, I'd rather drop packages like mongodb which are
unsupported or effectively dead upstream AND I don't use to free up
resources for other packages but I'd rather not take away a package that
someone else is actually using.

Given limited developer time, I would personally rather spend my own
developer time porting gourmet (last release 2014) to python3 than
porting mongodb to openssl-1.1.



-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Why [bug#47081] Remove mongodb?

[Ludovic Courtès]

Hi Léo,

Léo Le Bouter <lle-bout@zaclys.net> skribis:

>> Removing a package and its services is not something to do lightly:
>> it
>> breaks user configs with no recourse.
>> 
>> We must insist on getting more opinions on such matters, and I think
>> there just wasn’t enough feedback here.  I understand it can be
>> frustrating to wait for input, but in such a case, please do.  This
>> project has always strove for consensus.
>> 
>> Remember that the opinion of those who’ve been taking care of
>> security
>> issues in Guix for years, those who’ve been maintaining MongoDB,
>> those
>> who wrote the service and its tests, are invaluable; they must have a
>> say.  I insist: humbly solicit and wait for their feedback.
>> 
>
> I understand, and I did not think it was a light thing to do, no one
> mentionned anything we should do for the remove, so I actually do not
> know how we handle that but the security/non-free code thing put some
> urge into the situation, apologizes for moving on and pushing without
> waiting for more feedback, few people gave their feedback on IRC and by
> email and that's why I felt more confident doing the actual change.

Sure, now you know.  :-) For package removal, we have to wait for
feedback, pinging people if needed, and waiting longer than
usual—security pressure or not.  Removing a package can only happen if
there’s some consensus.

Thanks for your reply!

Ludo’.

Re: Why [bug#47081] Remove mongodb?

[Jack Hill]

I don't have anything to add with respect to the process for package 
removeal, but for the completeness of the thread I'd like the observe that 
one of the packages that was removed (mongo-tools) was broken for over a 
year: https://issues.guix.gnu.org/39637

For the reasons Efraim pointed out, I think that package was unlikely to 
be fixed, so I'm okay with it being removed.

Best,
Jack

Re: Why [bug#47081] Remove mongodb?

[Léo Le Bouter]

[-- Attachment #1: Type: text/plain, Size: 679 bytes --]

Sorry for duplicated email,

On Wed, 2021-03-17 at 17:56 +0100, zimoun wrote:
> If the removal for security reasons had been discussed on IRC, it
> could
> be nice to point the discussion here.  Otherwise, open a discussion
> on
> the topic on guix-devel or bug-guix.  The full removal is a radical
> solution (especially, it should be done with 2 commits: service+doc
> and
> then package; well another story).

Another thing is that openssl 1.1.1 on non-SSPL mongodb doesnt work and
we are working on removal of openssl 1.0.x which will removed all it's
dependents and mongodb is one so it was inevitably going to be removed
anyway.

> All the best,
> simon

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]