* [bug#37412] [PATCH 0/2] Add package and service for the Guix Data Service. @ 2019-09-15 18:18 Christopher Baines 2019-09-15 18:21 ` [bug#37412] [PATCH 1/2] gnu: Add guix-data-service Christopher Baines 2019-09-22 12:03 ` Christopher Baines 0 siblings, 2 replies; 17+ messages in thread From: Christopher Baines @ 2019-09-15 18:18 UTC (permalink / raw) To: 37412 [-- Attachment #1: Type: text/plain, Size: 538 bytes --] These patches add a package definition, and service type for the Guix Data Service, as well as a system test. Christopher Baines (2): gnu: Add guix-data-service. services: Add the Guix Data Service. gnu/local.mk | 2 + gnu/packages/web.scm | 83 +++++++++++++++++ gnu/services/guix.scm | 204 ++++++++++++++++++++++++++++++++++++++++++ gnu/tests/guix.scm | 173 +++++++++++++++++++++++++++++++++++ 4 files changed, 462 insertions(+) create mode 100644 gnu/services/guix.scm create mode 100644 gnu/tests/guix.scm [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 962 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. 2019-09-15 18:18 [bug#37412] [PATCH 0/2] Add package and service for the Guix Data Service Christopher Baines @ 2019-09-15 18:21 ` Christopher Baines 2019-09-15 18:21 ` [bug#37412] [PATCH 2/2] services: Add the Guix Data Service Christopher Baines ` (3 more replies) 2019-09-22 12:03 ` Christopher Baines 1 sibling, 4 replies; 17+ messages in thread From: Christopher Baines @ 2019-09-15 18:21 UTC (permalink / raw) To: 37412 * gnu/packages/web.scm (guix-data-service): New variable. --- gnu/packages/web.scm | 83 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm index 39475bd6a8..30adc75aa1 100644 --- a/gnu/packages/web.scm +++ b/gnu/packages/web.scm @@ -3973,6 +3973,89 @@ CDF, Atom 0.3, and Atom 1.0 feeds.") (define-public python2-feedparser (package-with-python2 python-feedparser)) +(define-public guix-data-service + (let ((commit "6a9f11fb94fbfd969396dd4d9f768a0c9ec67734") + (revision "1")) + (package + (name "guix-data-service") + (version (string-append "0.0.1-" revision "." (string-take commit 7))) + (source (origin + (method git-fetch) + (uri (git-reference + (url "https://git.savannah.gnu.org/git/guix/data-service.git") + (commit commit))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "08kfi74dp8h1i03gfmjqfv07ff7m93psnh6xmrgwzr0ja5cdqjri")))) + (build-system gnu-build-system) + (arguments + '(#:tests? #f ; TODO Tests require PostgreSQL + #:modules ((guix build utils) + (guix build gnu-build-system) + (ice-9 rdelim) + (ice-9 popen)) + #:phases + (modify-phases %standard-phases + (add-after 'install 'wrap-executable + (lambda* (#:key inputs outputs #:allow-other-keys) + (let* ((out (assoc-ref outputs "out")) + (bin (string-append out "/bin")) + (guile (assoc-ref inputs "guile")) + (guile-effective-version + (read-line + (open-pipe* OPEN_READ + (string-append guile "/bin/guile") + "-c" "(display (effective-version))"))) + (scm (string-append out "/share/guile/site/" + guile-effective-version)) + (go (string-append out "/lib/guile/" + guile-effective-version + "/site-ccache"))) + (for-each + (lambda (file) + (wrap-program (string-append bin "/" file) + `("PATH" ":" prefix + (,bin)) + `("GUILE_LOAD_PATH" ":" prefix + (,scm ,(getenv "GUILE_LOAD_PATH"))) + `("GUILE_LOAD_COMPILED_PATH" ":" prefix + (,go ,(getenv "GUILE_LOAD_COMPILED_PATH"))))) + '("guix-data-service" + "guix-data-service-process-branch-updated-email" + "guix-data-service-process-job" + "guix-data-service-process-jobs" + "guix-data-service-query-build-servers")) + #t)))))) + (inputs + `(("guile" ,guile-2.2) + ("guix" ,guix) + ("guile-fibers" ,guile-fibers) + ("guile-json" ,guile-json-3) + ("guile-email" ,guile-email) + ("guile-squee" ,guile-squee) + ("postgresql" ,postgresql) + ("sqitch" ,sqitch))) + (native-inputs + `(("autoconf" ,autoconf) + ("automake" ,automake) + ("pkg-config" ,pkg-config))) + (native-search-paths + ;; guile-git requires this to be set + (list (search-path-specification + (variable "GIT_SSL_CAINFO") + (file-type 'regular) + (separator #f) + (files '("etc/ssl/certs/ca-certificates.crt"))))) + (synopsis "Store and provide data about GNU Guix") + (description + "The Guix Data Service stores data about GNU Guix, and provides this +through a web interface. It supports listening to the guix-commits mailing +list to find out about new revisions, then loads the data from these in to a +PostgreSQL database.") + (home-page "https://data.guix.gnu.org/") + (license license:agpl3+)))) + (define-public gumbo-parser (package (name "gumbo-parser") -- 2.23.0 ^ permalink raw reply related [flat|nested] 17+ messages in thread
* [bug#37412] [PATCH 2/2] services: Add the Guix Data Service. 2019-09-15 18:21 ` [bug#37412] [PATCH 1/2] gnu: Add guix-data-service Christopher Baines @ 2019-09-15 18:21 ` Christopher Baines 2019-09-16 7:53 ` Ludovic Courtès 2019-09-15 20:45 ` [bug#37412] [PATCH 1/2] gnu: Add guix-data-service Alex ter Weele ` (2 subsequent siblings) 3 siblings, 1 reply; 17+ messages in thread From: Christopher Baines @ 2019-09-15 18:21 UTC (permalink / raw) To: 37412 * gnu/services/guix.scm: New file. * gnu/tests/guix.scm: New file. * gnu/local.mk (GNU_SYSTEM_MODULES): Add both new files. --- gnu/local.mk | 2 + gnu/services/guix.scm | 204 ++++++++++++++++++++++++++++++++++++++++++ gnu/tests/guix.scm | 173 +++++++++++++++++++++++++++++++++++ 3 files changed, 379 insertions(+) create mode 100644 gnu/services/guix.scm create mode 100644 gnu/tests/guix.scm diff --git a/gnu/local.mk b/gnu/local.mk index 3f32b9cbf2..b30afdd585 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -532,6 +532,7 @@ GNU_SYSTEM_MODULES = \ %D%/services/authentication.scm \ %D%/services/games.scm \ %D%/services/getmail.scm \ + %D%/services/guix.scm \ %D%/services/kerberos.scm \ %D%/services/lirc.scm \ %D%/services/virtualization.scm \ @@ -596,6 +597,7 @@ GNU_SYSTEM_MODULES = \ %D%/tests/desktop.scm \ %D%/tests/dict.scm \ %D%/tests/docker.scm \ + %D%/tests/guix.scm \ %D%/tests/monitoring.scm \ %D%/tests/nfs.scm \ %D%/tests/install.scm \ diff --git a/gnu/services/guix.scm b/gnu/services/guix.scm new file mode 100644 index 0000000000..65cffce2c0 --- /dev/null +++ b/gnu/services/guix.scm @@ -0,0 +1,204 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2019 Christopher Baines <mail@cbaines.net> +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify +;;; it under the terms of the GNU General Public License as published by +;;; the Free Software Foundation, either version 3 of the License, or +;;; (at your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, +;;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. + +(define-module (gnu services guix) + #:use-module (ice-9 match) + #:use-module (guix gexp) + #:use-module (guix records) + #:use-module ((gnu packages base) + #:select (glibc-utf8-locales)) + #:use-module (gnu packages admin) + #:use-module (gnu packages web) + #:use-module (gnu services) + #:use-module (gnu services base) + #:use-module (gnu services admin) + #:use-module (gnu services shepherd) + #:use-module (gnu services getmail) + #:use-module (gnu system shadow) + #:export (<guix-data-service-configuration> + guix-data-service-configuration + guix-data-service-configuration? + + guix-data-service-type)) + +;;;; Commentary: +;;; +;;; This module implements a service that to run instances of the Guix Data +;;; Service, which provides data about Guix over time. +;;; +;;;; Code: + +(define-record-type* <guix-data-service-configuration> + guix-data-service-configuration make-guix-data-service-configuration + guix-data-service-configuration? + (package guix-data-service-package + (default guix-data-service)) + (user guix-data-service-configuration-user + (default "guix-data-service")) + (group guix-data-service-configuration-group + (default "guix-data-service")) + (port guix-data-service-port + (default 8765)) + (host guix-data-service-host + (default "127.0.0.1")) + (getmail-idle-mailboxes + guix-data-service-getmail-idle-mailboxes + (default #f)) + (commits-getmail-retriever-configuration + guix-data-service-commits-getmail-retriever-configuration + (default #f))) + +(define (guix-data-service-profile-packages config) + "Return the guix-data-service package, this will populate the +ca-certificates.crt file in the system profile." + (list + (guix-data-service-package config))) + +(define (guix-data-service-shepherd-services config) + (match config + (($ <guix-data-service-configuration> package user group + port host) + (list + (shepherd-service + (documentation "Guix Data Service web server") + (provision '(guix-data-service)) + (requirement '(postgres networking)) + (start #~(make-forkexec-constructor + (list #$(file-append package + "/bin/guix-data-service") + "--pid-file=/var/run/guix-data-service/pid" + #$(string-append "--port=" (number->string port)) + #$(string-append "--host=" host) + ;; Perform any database migrations when the + ;; service is started + "--update-database") + + #:user #$user + #:group #$group + #:pid-file "/var/run/guix-data-service/pid" + ;; Allow time for migrations to run + #:pid-file-timeout 60 + #:environment-variables + `(,(string-append + "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale") + "LC_ALL=en_US.utf8") + #:log-file "/var/log/guix-data-service/web.log")) + (stop #~(make-kill-destructor))) + + (shepherd-service + (documentation "Guix Data Service process jobs") + (provision '(guix-data-service-process-jobs)) + (requirement '(postgres + networking + ;; Require guix-data-service, as that the database + ;; migrations are handled through this service + guix-data-service)) + (start #~(make-forkexec-constructor + (list + #$(file-append package + "/bin/guix-data-service-process-jobs")) + #:user #$user + #:group #$group + #:environment-variables + `("HOME=/var/lib/guix-data-service" + "GIT_SSL_CAINFO=/etc/ssl/certs/ca-certificates.crt" + ,(string-append + "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale") + "LC_ALL=en_US.utf8") + #:log-file "/var/log/guix-data-service/process-jobs.log")) + (stop #~(make-kill-destructor))))))) + +(define (guix-data-service-activation config) + #~(begin + (use-modules (guix build utils)) + + (define %user (getpw "guix-data-service")) + + (mkdir-p "/var/log/guix-data-service") + + ;; Allow writing the PID file + (mkdir-p "/var/run/guix-data-service") + (chown "/var/run/guix-data-service" + (passwd:uid %user) + (passwd:gid %user)))) + +(define (guix-data-service-account config) + (match config + (($ <guix-data-service-configuration> package user group) + (list (user-group + (name group) + (system? #t)) + (user-account + (name user) + (group group) + (system? #t) + (comment "Guix Data Service user") + (home-directory "/var/lib/guix-data-service") + (shell (file-append shadow "/sbin/nologin"))))))) + +(define (guix-data-service-getmail-configuration config) + (match config + (($ <guix-data-service-configuration> package user group + port host + #f #f) + '()) + (($ <guix-data-service-configuration> package user group + port host + getmail-idle-mailboxes + commits-getmail-retriever-configuration) + (list + (getmail-configuration + (name 'guix-data-service) + (user user) + (group group) + (directory "/var/lib/getmail/guix-data-service") + (rcfile + (getmail-configuration-file + (retriever commits-getmail-retriever-configuration) + (destination + (getmail-destination-configuration + (type "MDA_external") + (path (file-append + package + "/bin/guix-data-service-process-branch-updated-email")))) + (options + (getmail-options-configuration + (read-all #f) + (delivered-to #f) + (received #f))))) + (idle getmail-idle-mailboxes)))))) + +(define guix-data-service-type + (service-type + (name 'guix-data-service) + (extensions + (list + (service-extension profile-service-type + guix-data-service-profile-packages) + (service-extension shepherd-root-service-type + guix-data-service-shepherd-services) + (service-extension activation-service-type + guix-data-service-activation) + (service-extension account-service-type + guix-data-service-account) + (service-extension getmail-service-type + guix-data-service-getmail-configuration))) + (default-value + (guix-data-service-configuration)) + (description + "Run an instance of the Guix Data Service."))) diff --git a/gnu/tests/guix.scm b/gnu/tests/guix.scm new file mode 100644 index 0000000000..6139e31cf0 --- /dev/null +++ b/gnu/tests/guix.scm @@ -0,0 +1,173 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2019 Christopher Baines <mail@cbaines.net> +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. + +(define-module (gnu tests guix) + #:use-module (gnu tests) + #:use-module (gnu system) + #:use-module (gnu system file-systems) + #:use-module (gnu system shadow) + #:use-module (gnu system vm) + #:use-module (gnu services) + #:use-module (gnu services guix) + #:use-module (gnu services databases) + #:use-module (gnu services shepherd) + #:use-module (gnu services networking) + #:use-module (gnu packages databases) + #:use-module (guix packages) + #:use-module (guix modules) + #:use-module (guix records) + #:use-module (guix gexp) + #:use-module (guix store) + #:use-module (guix utils) + #:use-module (ice-9 match) + #:export (%test-guix-data-service)) + +\f +;;; +;;; Guix Data Service +;;; + +(define guix-data-service-initial-database-setup-service + (let ((user "guix_data_service") + (name "guix_data_service")) + (define start-gexp + #~(lambda () + (let ((pid (primitive-fork)) + (postgres (getpwnam "postgres"))) + (if (eq? pid 0) + (dynamic-wind + (const #t) + (lambda () + (setgid (passwd:gid postgres)) + (setuid (passwd:uid postgres)) + (primitive-exit + (if (and + (zero? + (system* #$(file-append postgresql "/bin/createuser") + #$user)) + (zero? + (system* #$(file-append postgresql "/bin/createdb") + "-O" #$user #$name))) + 0 + 1))) + (lambda () + (primitive-exit 1))) + (zero? (cdr (waitpid pid))))))) + + (shepherd-service + (requirement '(postgres)) + (provision '(guix-data-service-initial-database-setup)) + (start start-gexp) + (stop #~(const #f)) + (respawn? #f) + (one-shot? #t) + (documentation "Setup Guix Data Service database.")))) + +(define %guix-data-service-os + (simple-operating-system + (service dhcp-client-service-type) + (service postgresql-service-type + (postgresql-configuration + (config-file + (postgresql-config-file + (hba-file + (plain-file "pg_hba.conf" + " +local all all trust +host all all 127.0.0.1/32 trust +host all all ::1/128 trust")))))) + (service guix-data-service-type + (guix-data-service-configuration + (host "0.0.0.0"))) + (simple-service 'guix-data-service-database-setup + shepherd-root-service-type + (list guix-data-service-initial-database-setup-service)))) + +(define (run-guix-data-service-test) + (define os + (marionette-operating-system + %guix-data-service-os + #:imported-modules '((gnu services herd) + (guix combinators)))) + + (define forwarded-port 8080) + + (define vm + (virtual-machine + (operating-system os) + (memory-size 1024) + (port-forwardings `((,forwarded-port . 8765))))) + + (define test + (with-imported-modules '((gnu build marionette)) + #~(begin + (use-modules (srfi srfi-11) (srfi srfi-64) + (gnu build marionette) + (web uri) + (web client) + (web response)) + + (define marionette + (make-marionette (list #$vm))) + + (mkdir #$output) + (chdir #$output) + + (test-begin "guix-data-service") + + (test-assert "service running" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (match (start-service 'guix-data-service) + (#f #f) + (('service response-parts ...) + (match (assq-ref response-parts 'running) + ((pid) (number? pid)))))) + marionette)) + + (test-assert "process jobs service running" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (match (start-service 'guix-data-service-process-jobs) + (#f #f) + (('service response-parts ...) + (match (assq-ref response-parts 'running) + ((pid) (number? pid)))))) + marionette)) + + (test-equal "http-get" + 200 + (let-values + (((response text) + (http-get #$(simple-format + #f "http://localhost:~A/healthcheck" forwarded-port) + #:decode-body? #t))) + (response-code response))) + + (test-end) + (exit (= (test-runner-fail-count (test-runner-current)) 0))))) + + (gexp->derivation "guix-data-service-test" test)) + +(define %test-guix-data-service + (system-test + (name "guix-data-service") + (description "Connect to a running Guix Data Service.") + (value (run-guix-data-service-test)))) -- 2.23.0 ^ permalink raw reply related [flat|nested] 17+ messages in thread
* [bug#37412] [PATCH 2/2] services: Add the Guix Data Service. 2019-09-15 18:21 ` [bug#37412] [PATCH 2/2] services: Add the Guix Data Service Christopher Baines @ 2019-09-16 7:53 ` Ludovic Courtès 2019-09-22 12:15 ` Christopher Baines 0 siblings, 1 reply; 17+ messages in thread From: Ludovic Courtès @ 2019-09-16 7:53 UTC (permalink / raw) To: Christopher Baines; +Cc: 37412 Christopher Baines <mail@cbaines.net> skribis: > * gnu/services/guix.scm: New file. > * gnu/tests/guix.scm: New file. > * gnu/local.mk (GNU_SYSTEM_MODULES): Add both new files. Yay, nice! > + (commits-getmail-retriever-configuration > + guix-data-service-commits-getmail-retriever-configuration New world record: longer than ‘call-with-current-continuation’. :-) > +(define (guix-data-service-shepherd-services config) > + (match config > + (($ <guix-data-service-configuration> package user group > + port host) Perhaps use ‘match-record’ here and elsewhere, to avoid mistakes when the record type changes. > +(define %test-guix-data-service > + (system-test > + (name "guix-data-service") > + (description "Connect to a running Guix Data Service.") > + (value (run-guix-data-service-test)))) Nice. Could you add a bit to guix.texi, maybe under “Continuous Integration”? You don’t have to go into too much detail if you think some of the configuration elements are subject to change. Thank you! Ludo’. ^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#37412] [PATCH 2/2] services: Add the Guix Data Service. 2019-09-16 7:53 ` Ludovic Courtès @ 2019-09-22 12:15 ` Christopher Baines 0 siblings, 0 replies; 17+ messages in thread From: Christopher Baines @ 2019-09-22 12:15 UTC (permalink / raw) To: Ludovic Courtès; +Cc: 37412 [-- Attachment #1: Type: text/plain, Size: 1167 bytes --] Ludovic Courtès <ludo@gnu.org> writes: > Christopher Baines <mail@cbaines.net> skribis: > >> +(define (guix-data-service-shepherd-services config) >> + (match config >> + (($ <guix-data-service-configuration> package user group >> + port host) > > Perhaps use ‘match-record’ here and elsewhere, to avoid mistakes when > the record type changes. Yeah, I've switched a couple of the match statements across to match-record, but I've left one, as it has a couple of clauses. >> +(define %test-guix-data-service >> + (system-test >> + (name "guix-data-service") >> + (description "Connect to a running Guix Data Service.") >> + (value (run-guix-data-service-test)))) > > Nice. > > Could you add a bit to guix.texi, maybe under “Continuous Integration”? > You don’t have to go into too much detail if you think some of the > configuration elements are subject to change. I forgot you mentioned the "Continuous Integration" section, so I actually added something under a new "Guix" section, but regardless, I've now sent some new patches, which includes some documentation. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 962 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. 2019-09-15 18:21 ` [bug#37412] [PATCH 1/2] gnu: Add guix-data-service Christopher Baines 2019-09-15 18:21 ` [bug#37412] [PATCH 2/2] services: Add the Guix Data Service Christopher Baines @ 2019-09-15 20:45 ` Alex ter Weele 2019-09-15 21:29 ` Christopher Baines 2019-09-16 7:48 ` Ludovic Courtès 2019-09-16 7:55 ` [bug#37412] " Ludovic Courtès 3 siblings, 1 reply; 17+ messages in thread From: Alex ter Weele @ 2019-09-15 20:45 UTC (permalink / raw) To: Christopher Baines; +Cc: 37412 Christopher Baines <mail@cbaines.net> writes: > * gnu/packages/web.scm (guix-data-service): New variable. > --- > gnu/packages/web.scm | 83 ++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 83 insertions(+) > > diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm > index 39475bd6a8..30adc75aa1 100644 > --- a/gnu/packages/web.scm > +++ b/gnu/packages/web.scm > @@ -3973,6 +3973,89 @@ CDF, Atom 0.3, and Atom 1.0 feeds.") > (define-public python2-feedparser > (package-with-python2 python-feedparser)) > > +(define-public guix-data-service > + (let ((commit "6a9f11fb94fbfd969396dd4d9f768a0c9ec67734") > + (revision "1")) > + (package > + (name "guix-data-service") > + (version (string-append "0.0.1-" revision "." (string-take commit 7))) > + (source (origin > + (method git-fetch) > + (uri (git-reference > + (url "https://git.savannah.gnu.org/git/guix/data-service.git") > + (commit commit))) > + (file-name (git-file-name name version)) > + (sha256 > + (base32 > + "08kfi74dp8h1i03gfmjqfv07ff7m93psnh6xmrgwzr0ja5cdqjri")))) > + (build-system gnu-build-system) > + (arguments > + '(#:tests? #f ; TODO Tests require PostgreSQL > + #:modules ((guix build utils) > + (guix build gnu-build-system) > + (ice-9 rdelim) > + (ice-9 popen)) > + #:phases > + (modify-phases %standard-phases > + (add-after 'install 'wrap-executable > + (lambda* (#:key inputs outputs #:allow-other-keys) > + (let* ((out (assoc-ref outputs "out")) > + (bin (string-append out "/bin")) > + (guile (assoc-ref inputs "guile")) > + (guile-effective-version > + (read-line > + (open-pipe* OPEN_READ > + (string-append guile "/bin/guile") > + "-c" "(display (effective-version))"))) > + (scm (string-append out "/share/guile/site/" > + guile-effective-version)) > + (go (string-append out "/lib/guile/" > + guile-effective-version > + "/site-ccache"))) > + (for-each > + (lambda (file) > + (wrap-program (string-append bin "/" file) > + `("PATH" ":" prefix > + (,bin)) > + `("GUILE_LOAD_PATH" ":" prefix > + (,scm ,(getenv "GUILE_LOAD_PATH"))) > + `("GUILE_LOAD_COMPILED_PATH" ":" prefix > + (,go ,(getenv "GUILE_LOAD_COMPILED_PATH"))))) > + '("guix-data-service" > + "guix-data-service-process-branch-updated-email" > + "guix-data-service-process-job" > + "guix-data-service-process-jobs" > + "guix-data-service-query-build-servers")) > + #t)))))) > + (inputs > + `(("guile" ,guile-2.2) > + ("guix" ,guix) > + ("guile-fibers" ,guile-fibers) > + ("guile-json" ,guile-json-3) > + ("guile-email" ,guile-email) > + ("guile-squee" ,guile-squee) > + ("postgresql" ,postgresql) > + ("sqitch" ,sqitch))) > + (native-inputs > + `(("autoconf" ,autoconf) > + ("automake" ,automake) > + ("pkg-config" ,pkg-config))) > + (native-search-paths > + ;; guile-git requires this to be set > + (list (search-path-specification > + (variable "GIT_SSL_CAINFO") > + (file-type 'regular) > + (separator #f) > + (files '("etc/ssl/certs/ca-certificates.crt"))))) > + (synopsis "Store and provide data about GNU Guix") > + (description > + "The Guix Data Service stores data about GNU Guix, and provides this > +through a web interface. It supports listening to the guix-commits mailing > +list to find out about new revisions, then loads the data from these in to a > +PostgreSQL database.") > + (home-page "https://data.guix.gnu.org/") I found that only http://data.guix.gnu.org/ works. > > + (license license:agpl3+)))) + (define-public gumbo-parser (package > (name "gumbo-parser") ^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. 2019-09-15 20:45 ` [bug#37412] [PATCH 1/2] gnu: Add guix-data-service Alex ter Weele @ 2019-09-15 21:29 ` Christopher Baines 0 siblings, 0 replies; 17+ messages in thread From: Christopher Baines @ 2019-09-15 21:29 UTC (permalink / raw) To: Alex ter Weele; +Cc: 37412 [-- Attachment #1: Type: text/plain, Size: 397 bytes --] Alex ter Weele <alex.ter.weele@gmail.com> writes: >> + (home-page "https://data.guix.gnu.org/") > > I found that only http://data.guix.gnu.org/ works. Good spot. When I wrote this, I was planning on setting up HTTPS for data.guix.gnu.org, but when I came to do it, I didn't quite know how to configure it. It's still on my list of things to do, but I've now changed the patch to use http. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 962 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. 2019-09-15 18:21 ` [bug#37412] [PATCH 1/2] gnu: Add guix-data-service Christopher Baines 2019-09-15 18:21 ` [bug#37412] [PATCH 2/2] services: Add the Guix Data Service Christopher Baines 2019-09-15 20:45 ` [bug#37412] [PATCH 1/2] gnu: Add guix-data-service Alex ter Weele @ 2019-09-16 7:48 ` Ludovic Courtès 2019-09-22 10:18 ` Christopher Baines 2019-09-16 7:55 ` [bug#37412] " Ludovic Courtès 3 siblings, 1 reply; 17+ messages in thread From: Ludovic Courtès @ 2019-09-16 7:48 UTC (permalink / raw) To: Christopher Baines; +Cc: 37412 Good morning, Chris! Christopher Baines <mail@cbaines.net> skribis: > * gnu/packages/web.scm (guix-data-service): New variable. Woohoo! \o/ > + (inputs > + `(("guile" ,guile-2.2) Guile should be ‘native-inputs’ because we use it for its compiler (which can act as a “cross-compiler” with ‘--target’). > + (native-search-paths > + ;; guile-git requires this to be set > + (list (search-path-specification > + (variable "GIT_SSL_CAINFO") > + (file-type 'regular) > + (separator #f) > + (files '("etc/ssl/certs/ca-certificates.crt"))))) It’s a bit of a workaround for <https://issues.guix.gnu.org/issue/22138>, isn’t it? :-) What about either removing it, or adding an “XXX” to the comment? Otherwise LGTM! Ludo’. ^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. 2019-09-16 7:48 ` Ludovic Courtès @ 2019-09-22 10:18 ` Christopher Baines 2019-09-22 21:04 ` Ludovic Courtès 0 siblings, 1 reply; 17+ messages in thread From: Christopher Baines @ 2019-09-22 10:18 UTC (permalink / raw) To: Ludovic Courtès; +Cc: 37412 [-- Attachment #1: Type: text/plain, Size: 1342 bytes --] Ludovic Courtès <ludo@gnu.org> writes: > Good morning, Chris! > > Christopher Baines <mail@cbaines.net> skribis: > >> * gnu/packages/web.scm (guix-data-service): New variable. > > Woohoo! \o/ > >> + (inputs >> + `(("guile" ,guile-2.2) > > Guile should be ‘native-inputs’ because we use it for its compiler > (which can act as a “cross-compiler” with ‘--target’). This makes sense, but the scripts will still refer to the guile used to build the package, so building for a different architecture won't work right? Does the package somehow need to take as input two Guiles, one for build time, and one for runtime? >> + (native-search-paths >> + ;; guile-git requires this to be set >> + (list (search-path-specification >> + (variable "GIT_SSL_CAINFO") >> + (file-type 'regular) >> + (separator #f) >> + (files '("etc/ssl/certs/ca-certificates.crt"))))) > > It’s a bit of a workaround for > <https://issues.guix.gnu.org/issue/22138>, isn’t it? :-) > > What about either removing it, or adding an “XXX” to the comment? I can't quite remember, I think I added that a long while ago when trying to get the Guix service working. I'm not sure it's a great approach. I'll add XXX to the comment though. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 962 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. 2019-09-22 10:18 ` Christopher Baines @ 2019-09-22 21:04 ` Ludovic Courtès 2019-09-25 7:46 ` bug#37412: " Christopher Baines 0 siblings, 1 reply; 17+ messages in thread From: Ludovic Courtès @ 2019-09-22 21:04 UTC (permalink / raw) To: Christopher Baines; +Cc: 37412 Hi Chris, Christopher Baines <mail@cbaines.net> skribis: > Ludovic Courtès <ludo@gnu.org> writes: > >> Good morning, Chris! >> >> Christopher Baines <mail@cbaines.net> skribis: >> >>> * gnu/packages/web.scm (guix-data-service): New variable. >> >> Woohoo! \o/ >> >>> + (inputs >>> + `(("guile" ,guile-2.2) >> >> Guile should be ‘native-inputs’ because we use it for its compiler >> (which can act as a “cross-compiler” with ‘--target’). > > This makes sense, but the scripts will still refer to the guile used to > build the package, so building for a different architecture won't work > right? Does the package somehow need to take as input two Guiles, one > for build time, and one for runtime? Yes, we probably need to have Guile both as input and native input. Well don’t lose your hair on it anyway. >>> + (native-search-paths >>> + ;; guile-git requires this to be set >>> + (list (search-path-specification >>> + (variable "GIT_SSL_CAINFO") >>> + (file-type 'regular) >>> + (separator #f) >>> + (files '("etc/ssl/certs/ca-certificates.crt"))))) >> >> It’s a bit of a workaround for >> <https://issues.guix.gnu.org/issue/22138>, isn’t it? :-) >> >> What about either removing it, or adding an “XXX” to the comment? > > I can't quite remember, I think I added that a long while ago when > trying to get the Guix service working. I'm not sure it's a great > approach. I'll add XXX to the comment though. What about removing it altogether and instead setting GIT_SSL_CAINFO in the service itself? Thanks, Ludo’. ^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#37412: [PATCH 1/2] gnu: Add guix-data-service. 2019-09-22 21:04 ` Ludovic Courtès @ 2019-09-25 7:46 ` Christopher Baines 0 siblings, 0 replies; 17+ messages in thread From: Christopher Baines @ 2019-09-25 7:46 UTC (permalink / raw) To: Ludovic Courtès; +Cc: 37412-done [-- Attachment #1: Type: text/plain, Size: 2015 bytes --] Ludovic Courtès <ludo@gnu.org> writes: > Hi Chris, > > Christopher Baines <mail@cbaines.net> skribis: > >> Ludovic Courtès <ludo@gnu.org> writes: >> >>> Good morning, Chris! >>> >>> Christopher Baines <mail@cbaines.net> skribis: >>> >>>> * gnu/packages/web.scm (guix-data-service): New variable. >>> >>> Woohoo! \o/ >>> >>>> + (inputs >>>> + `(("guile" ,guile-2.2) >>> >>> Guile should be ‘native-inputs’ because we use it for its compiler >>> (which can act as a “cross-compiler” with ‘--target’). >> >> This makes sense, but the scripts will still refer to the guile used to >> build the package, so building for a different architecture won't work >> right? Does the package somehow need to take as input two Guiles, one >> for build time, and one for runtime? > > Yes, we probably need to have Guile both as input and native input. > Well don’t lose your hair on it anyway. I've just gone with making it a native input for now. >>>> + (native-search-paths >>>> + ;; guile-git requires this to be set >>>> + (list (search-path-specification >>>> + (variable "GIT_SSL_CAINFO") >>>> + (file-type 'regular) >>>> + (separator #f) >>>> + (files '("etc/ssl/certs/ca-certificates.crt"))))) >>> >>> It’s a bit of a workaround for >>> <https://issues.guix.gnu.org/issue/22138>, isn’t it? :-) >>> >>> What about either removing it, or adding an “XXX” to the comment? >> >> I can't quite remember, I think I added that a long while ago when >> trying to get the Guix service working. I'm not sure it's a great >> approach. I'll add XXX to the comment though. > > What about removing it altogether and instead setting GIT_SSL_CAINFO in > the service itself? Turns out I was already setting GIT_SSL_CAINFO in the service, so I've removed the native-search-paths from the package definition. I've now pushed both patches, thanks for taking a look! Chris [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 962 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. 2019-09-15 18:21 ` [bug#37412] [PATCH 1/2] gnu: Add guix-data-service Christopher Baines ` (2 preceding siblings ...) 2019-09-16 7:48 ` Ludovic Courtès @ 2019-09-16 7:55 ` Ludovic Courtès 2019-09-22 10:10 ` Christopher Baines 3 siblings, 1 reply; 17+ messages in thread From: Ludovic Courtès @ 2019-09-16 7:55 UTC (permalink / raw) To: Christopher Baines; +Cc: 37412 Christopher Baines <mail@cbaines.net> skribis: > + (home-page "https://data.guix.gnu.org/") It would be nice to add a reference to data.guix.gnu.org at <https://guix.gnu.org/contribute/> or in the manual. WDYT? Ludo’. ^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. 2019-09-16 7:55 ` [bug#37412] " Ludovic Courtès @ 2019-09-22 10:10 ` Christopher Baines 0 siblings, 0 replies; 17+ messages in thread From: Christopher Baines @ 2019-09-22 10:10 UTC (permalink / raw) To: Ludovic Courtès; +Cc: 37412 [-- Attachment #1: Type: text/plain, Size: 397 bytes --] Ludovic Courtès <ludo@gnu.org> writes: > Christopher Baines <mail@cbaines.net> skribis: > >> + (home-page "https://data.guix.gnu.org/") > > It would be nice to add a reference to data.guix.gnu.org at > <https://guix.gnu.org/contribute/> or in the manual. WDYT? Yeah, maybe there could be a "Services" section, and that could mention Cuirass, Mumi, and the Guix Data Service. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 962 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. 2019-09-15 18:18 [bug#37412] [PATCH 0/2] Add package and service for the Guix Data Service Christopher Baines 2019-09-15 18:21 ` [bug#37412] [PATCH 1/2] gnu: Add guix-data-service Christopher Baines @ 2019-09-22 12:03 ` Christopher Baines 2019-09-22 12:03 ` [bug#37412] [PATCH 2/2] services: Add the Guix Data Service Christopher Baines 2019-09-22 21:11 ` [bug#37412] [PATCH 1/2] gnu: Add guix-data-service Ludovic Courtès 1 sibling, 2 replies; 17+ messages in thread From: Christopher Baines @ 2019-09-22 12:03 UTC (permalink / raw) To: 37412 * gnu/packages/web.scm (guix-data-service): New variable. --- gnu/packages/web.scm | 102 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 102 insertions(+) diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm index 39475bd6a8..065979f959 100644 --- a/gnu/packages/web.scm +++ b/gnu/packages/web.scm @@ -56,6 +56,7 @@ #:use-module ((guix licenses) #:prefix license:) #:use-module (guix packages) #:use-module (guix download) + #:use-module (guix gexp) #:use-module (guix git-download) #:use-module (guix cvs-download) #:use-module (guix hg-download) @@ -77,6 +78,8 @@ #:use-module (gnu packages check) #:use-module (gnu packages documentation) #:use-module (gnu packages docbook) + #:use-module (gnu packages emacs) + #:use-module (gnu packages emacs-xyz) #:use-module (gnu packages autotools) #:use-module (gnu packages compression) #:use-module (gnu packages curl) @@ -3973,6 +3976,105 @@ CDF, Atom 0.3, and Atom 1.0 feeds.") (define-public python2-feedparser (package-with-python2 python-feedparser)) +(define-public guix-data-service + (let ((commit "970bcb31cf994b0e3b67e7f912734e5b53355d8d") + (revision "1")) + (package + (name "guix-data-service") + (version (string-append "0.0.1-" revision "." (string-take commit 7))) + (source (origin + (method git-fetch) + (uri (git-reference + (url "https://git.savannah.gnu.org/git/guix/data-service.git") + (commit commit))) + (file-name (git-file-name name version)) + (sha256 + (base32 + "0wiw1m4g30sx155ldjrg4060ydqwgxv93day89pf6w515n5fv08j")))) + (build-system gnu-build-system) + (arguments + '(#:tests? #f ; TODO Tests require PostgreSQL + #:modules ((guix build utils) + (guix build gnu-build-system) + (ice-9 rdelim) + (ice-9 popen)) + #:phases + (modify-phases %standard-phases + (add-after 'set-paths 'set-GUIX_ENVIRONMENT + (lambda* (#:key inputs #:allow-other-keys) + ;; This means guix.el finds the Emacs modules + (setenv "GUIX_ENVIRONMENT" + (assoc-ref inputs "emacs-with-modules")) + #t)) + (add-before 'build 'set-GUILE_AUTO_COMPILE + (lambda _ + ;; To avoid errors relating to guild + (setenv "GUILE_AUTO_COMPILE" "0") + #t)) + (add-after 'install 'wrap-executable + (lambda* (#:key inputs outputs #:allow-other-keys) + (let* ((out (assoc-ref outputs "out")) + (bin (string-append out "/bin")) + (guile (assoc-ref inputs "guile")) + (guile-effective-version + (read-line + (open-pipe* OPEN_READ + (string-append guile "/bin/guile") + "-c" "(display (effective-version))"))) + (scm (string-append out "/share/guile/site/" + guile-effective-version)) + (go (string-append out "/lib/guile/" + guile-effective-version + "/site-ccache"))) + (for-each + (lambda (file) + (wrap-program (string-append bin "/" file) + `("PATH" ":" prefix + (,bin)) + `("GUILE_LOAD_PATH" ":" prefix + (,scm ,(getenv "GUILE_LOAD_PATH"))) + `("GUILE_LOAD_COMPILED_PATH" ":" prefix + (,go ,(getenv "GUILE_LOAD_COMPILED_PATH"))))) + '("guix-data-service" + "guix-data-service-process-branch-updated-email" + "guix-data-service-process-job" + "guix-data-service-process-jobs" + "guix-data-service-query-build-servers")) + #t))) + (delete 'strip)))) ; As the .go files aren't compatible + (inputs + `(("guile" ,guile-2.2) + ("guix" ,guix) + ("guile-fibers" ,guile-fibers) + ("guile-json" ,guile-json-3) + ("guile-email" ,guile-email) + ("guile-squee" ,guile-squee) + ("postgresql" ,postgresql) + ("sqitch" ,sqitch))) + (native-inputs + `(("autoconf" ,autoconf) + ("automake" ,automake) + ("emacs-with-modules" ,(directory-union + "emacs-union" + (list emacs-no-x + emacs-htmlize))) + ("pkg-config" ,pkg-config))) + (native-search-paths + ;; XXX guile-git requires this to be set, maybe there's a better way + (list (search-path-specification + (variable "GIT_SSL_CAINFO") + (file-type 'regular) + (separator #f) + (files '("etc/ssl/certs/ca-certificates.crt"))))) + (synopsis "Store and provide data about GNU Guix") + (description + "The Guix Data Service stores data about GNU Guix, and provides this +through a web interface. It supports listening to the guix-commits mailing +list to find out about new revisions, then loads the data from these in to a +PostgreSQL database.") + (home-page "http://data.guix.gnu.org/") + (license license:agpl3+)))) + (define-public gumbo-parser (package (name "gumbo-parser") -- 2.23.0 ^ permalink raw reply related [flat|nested] 17+ messages in thread
* [bug#37412] [PATCH 2/2] services: Add the Guix Data Service. 2019-09-22 12:03 ` Christopher Baines @ 2019-09-22 12:03 ` Christopher Baines 2019-09-22 21:11 ` [bug#37412] [PATCH 1/2] gnu: Add guix-data-service Ludovic Courtès 1 sibling, 0 replies; 17+ messages in thread From: Christopher Baines @ 2019-09-22 12:03 UTC (permalink / raw) To: 37412 * gnu/services/guix.scm: New file. * gnu/tests/guix.scm: New file. * gnu/local.mk (GNU_SYSTEM_MODULES): Add both new files. * doc/guix.texi (Guix Services): New section documenting the Guix Data Service. --- doc/guix.texi | 52 +++++++++++ gnu/local.mk | 2 + gnu/services/guix.scm | 212 ++++++++++++++++++++++++++++++++++++++++++ gnu/tests/guix.scm | 173 ++++++++++++++++++++++++++++++++++ 4 files changed, 439 insertions(+) create mode 100644 gnu/services/guix.scm create mode 100644 gnu/tests/guix.scm diff --git a/doc/guix.texi b/doc/guix.texi index 39d4b865f6..24da18ea13 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -11657,6 +11657,7 @@ declaration. * Virtualization Services:: Virtualization services. * Version Control Services:: Providing remote access to Git repositories. * Game Services:: Game servers. +* Guix Services:: Services relating specifically to Guix. * Miscellaneous Services:: Other services. @end menu @@ -24167,6 +24168,57 @@ The port to bind the server to. @end table @end deftp + +@node Guix Services +@subsection Guix Services + +@subsubheading Guix Data Service +The @uref{http://data.guix.gnu.org,Guix Data Service} processes, stores +and provides data about GNU Guix. This includes information about +packages, derivations and lint warnings. + +The data is stored in a PostgreSQL database, and available through a web +interface. + +@defvar {Scheme Variable} guix-data-service-type +Service type for the Guix Data Service. Its value must be a +@code{guix-data-service-configuration} object. The service optionally +extends the getmail service, as the guix-commits mailing list is used to +find out about changes in the Guix git repository. +@end defvar + +@deftp {Data Type} guix-data-service-configuration +Data type representing the configuration of the Guix Data Service. + +@table @asis +@item @code{package} (default: @code{guix-data-service}) +The Guix Data Service package to use. + +@item @code{user} (default: @code{"guix-data-service"}) +The system user to run the service as. + +@item @code{group} (default: @code{"guix-data-service"}) +The system group to run the service as. + +@item @code{port} (default: @code{8765}) +The port to bind the web service to. + +@item @code{host} (default: @code{"127.0.0.1"}) +The host to bind the web service to. + +@item @code{getmail-idle-mailboxes} (default: @code{#f}) +If set, this is the list of mailboxes that the getmail service will be +configured to listen to. + +@item @code{commits-getmail-retriever-configuration} (default: @code{#f}) +If set, this is the @code{getmail-retriever-configuration} object with +which to configure getmail to fetch mail from the guix-commits mailing +list. + +@end table +@end deftp + + @node Miscellaneous Services @subsection Miscellaneous Services diff --git a/gnu/local.mk b/gnu/local.mk index 3f32b9cbf2..b30afdd585 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -532,6 +532,7 @@ GNU_SYSTEM_MODULES = \ %D%/services/authentication.scm \ %D%/services/games.scm \ %D%/services/getmail.scm \ + %D%/services/guix.scm \ %D%/services/kerberos.scm \ %D%/services/lirc.scm \ %D%/services/virtualization.scm \ @@ -596,6 +597,7 @@ GNU_SYSTEM_MODULES = \ %D%/tests/desktop.scm \ %D%/tests/dict.scm \ %D%/tests/docker.scm \ + %D%/tests/guix.scm \ %D%/tests/monitoring.scm \ %D%/tests/nfs.scm \ %D%/tests/install.scm \ diff --git a/gnu/services/guix.scm b/gnu/services/guix.scm new file mode 100644 index 0000000000..0f0fad39b0 --- /dev/null +++ b/gnu/services/guix.scm @@ -0,0 +1,212 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2019 Christopher Baines <mail@cbaines.net> +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify +;;; it under the terms of the GNU General Public License as published by +;;; the Free Software Foundation, either version 3 of the License, or +;;; (at your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, +;;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. + +(define-module (gnu services guix) + #:use-module (ice-9 match) + #:use-module (guix gexp) + #:use-module (guix records) + #:use-module ((gnu packages base) + #:select (glibc-utf8-locales)) + #:use-module (gnu packages admin) + #:use-module (gnu packages web) + #:use-module (gnu services) + #:use-module (gnu services base) + #:use-module (gnu services admin) + #:use-module (gnu services shepherd) + #:use-module (gnu services getmail) + #:use-module (gnu system shadow) + #:export (<guix-data-service-configuration> + guix-data-service-configuration + guix-data-service-configuration? + guix-data-service-package + guix-data-service-user + guix-data-service-group + guix-data-service-port + guix-data-service-host + guix-data-service-getmail-idle-mailboxes + guix-data-service-commits-getmail-retriever-configuration + + guix-data-service-type)) + +;;;; Commentary: +;;; +;;; This module implements a service that to run instances of the Guix Data +;;; Service, which provides data about Guix over time. +;;; +;;;; Code: + +(define-record-type* <guix-data-service-configuration> + guix-data-service-configuration make-guix-data-service-configuration + guix-data-service-configuration? + (package guix-data-service-package + (default guix-data-service)) + (user guix-data-service-configuration-user + (default "guix-data-service")) + (group guix-data-service-configuration-group + (default "guix-data-service")) + (port guix-data-service-port + (default 8765)) + (host guix-data-service-host + (default "127.0.0.1")) + (getmail-idle-mailboxes + guix-data-service-getmail-idle-mailboxes + (default #f)) + (commits-getmail-retriever-configuration + guix-data-service-commits-getmail-retriever-configuration + (default #f))) + +(define (guix-data-service-profile-packages config) + "Return the guix-data-service package, this will populate the +ca-certificates.crt file in the system profile." + (list + (guix-data-service-package config))) + +(define (guix-data-service-shepherd-services config) + (match-record config <guix-data-service-configuration> + (package user group port host) + (list + (shepherd-service + (documentation "Guix Data Service web server") + (provision '(guix-data-service)) + (requirement '(postgres networking)) + (start #~(make-forkexec-constructor + (list #$(file-append package + "/bin/guix-data-service") + "--pid-file=/var/run/guix-data-service/pid" + #$(string-append "--port=" (number->string port)) + #$(string-append "--host=" host) + ;; Perform any database migrations when the + ;; service is started + "--update-database") + + #:user #$user + #:group #$group + #:pid-file "/var/run/guix-data-service/pid" + ;; Allow time for migrations to run + #:pid-file-timeout 60 + #:environment-variables + `(,(string-append + "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale") + "LC_ALL=en_US.utf8") + #:log-file "/var/log/guix-data-service/web.log")) + (stop #~(make-kill-destructor))) + + (shepherd-service + (documentation "Guix Data Service process jobs") + (provision '(guix-data-service-process-jobs)) + (requirement '(postgres + networking + ;; Require guix-data-service, as that the database + ;; migrations are handled through this service + guix-data-service)) + (start #~(make-forkexec-constructor + (list + #$(file-append package + "/bin/guix-data-service-process-jobs")) + #:user #$user + #:group #$group + #:environment-variables + `("HOME=/var/lib/guix-data-service" + "GIT_SSL_CAINFO=/etc/ssl/certs/ca-certificates.crt" + ,(string-append + "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale") + "LC_ALL=en_US.utf8") + #:log-file "/var/log/guix-data-service/process-jobs.log")) + (stop #~(make-kill-destructor)))))) + +(define (guix-data-service-activation config) + #~(begin + (use-modules (guix build utils)) + + (define %user (getpw "guix-data-service")) + + (chmod "/var/lib/guix-data-service" #o755) + + (mkdir-p "/var/log/guix-data-service") + + ;; Allow writing the PID file + (mkdir-p "/var/run/guix-data-service") + (chown "/var/run/guix-data-service" + (passwd:uid %user) + (passwd:gid %user)))) + +(define (guix-data-service-account config) + (match-record config <guix-data-service-configuration> + (user group) + (list (user-group + (name group) + (system? #t)) + (user-account + (name user) + (group group) + (system? #t) + (comment "Guix Data Service user") + (home-directory "/var/lib/guix-data-service") + (shell (file-append shadow "/sbin/nologin")))))) + +(define (guix-data-service-getmail-configuration config) + (match config + (($ <guix-data-service-configuration> package user group + port host + #f #f) + '()) + (($ <guix-data-service-configuration> package user group + port host + getmail-idle-mailboxes + commits-getmail-retriever-configuration) + (list + (getmail-configuration + (name 'guix-data-service) + (user user) + (group group) + (directory "/var/lib/getmail/guix-data-service") + (rcfile + (getmail-configuration-file + (retriever commits-getmail-retriever-configuration) + (destination + (getmail-destination-configuration + (type "MDA_external") + (path (file-append + package + "/bin/guix-data-service-process-branch-updated-email")))) + (options + (getmail-options-configuration + (read-all #f) + (delivered-to #f) + (received #f))))) + (idle getmail-idle-mailboxes)))))) + +(define guix-data-service-type + (service-type + (name 'guix-data-service) + (extensions + (list + (service-extension profile-service-type + guix-data-service-profile-packages) + (service-extension shepherd-root-service-type + guix-data-service-shepherd-services) + (service-extension activation-service-type + guix-data-service-activation) + (service-extension account-service-type + guix-data-service-account) + (service-extension getmail-service-type + guix-data-service-getmail-configuration))) + (default-value + (guix-data-service-configuration)) + (description + "Run an instance of the Guix Data Service."))) diff --git a/gnu/tests/guix.scm b/gnu/tests/guix.scm new file mode 100644 index 0000000000..6139e31cf0 --- /dev/null +++ b/gnu/tests/guix.scm @@ -0,0 +1,173 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2019 Christopher Baines <mail@cbaines.net> +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. + +(define-module (gnu tests guix) + #:use-module (gnu tests) + #:use-module (gnu system) + #:use-module (gnu system file-systems) + #:use-module (gnu system shadow) + #:use-module (gnu system vm) + #:use-module (gnu services) + #:use-module (gnu services guix) + #:use-module (gnu services databases) + #:use-module (gnu services shepherd) + #:use-module (gnu services networking) + #:use-module (gnu packages databases) + #:use-module (guix packages) + #:use-module (guix modules) + #:use-module (guix records) + #:use-module (guix gexp) + #:use-module (guix store) + #:use-module (guix utils) + #:use-module (ice-9 match) + #:export (%test-guix-data-service)) + +\f +;;; +;;; Guix Data Service +;;; + +(define guix-data-service-initial-database-setup-service + (let ((user "guix_data_service") + (name "guix_data_service")) + (define start-gexp + #~(lambda () + (let ((pid (primitive-fork)) + (postgres (getpwnam "postgres"))) + (if (eq? pid 0) + (dynamic-wind + (const #t) + (lambda () + (setgid (passwd:gid postgres)) + (setuid (passwd:uid postgres)) + (primitive-exit + (if (and + (zero? + (system* #$(file-append postgresql "/bin/createuser") + #$user)) + (zero? + (system* #$(file-append postgresql "/bin/createdb") + "-O" #$user #$name))) + 0 + 1))) + (lambda () + (primitive-exit 1))) + (zero? (cdr (waitpid pid))))))) + + (shepherd-service + (requirement '(postgres)) + (provision '(guix-data-service-initial-database-setup)) + (start start-gexp) + (stop #~(const #f)) + (respawn? #f) + (one-shot? #t) + (documentation "Setup Guix Data Service database.")))) + +(define %guix-data-service-os + (simple-operating-system + (service dhcp-client-service-type) + (service postgresql-service-type + (postgresql-configuration + (config-file + (postgresql-config-file + (hba-file + (plain-file "pg_hba.conf" + " +local all all trust +host all all 127.0.0.1/32 trust +host all all ::1/128 trust")))))) + (service guix-data-service-type + (guix-data-service-configuration + (host "0.0.0.0"))) + (simple-service 'guix-data-service-database-setup + shepherd-root-service-type + (list guix-data-service-initial-database-setup-service)))) + +(define (run-guix-data-service-test) + (define os + (marionette-operating-system + %guix-data-service-os + #:imported-modules '((gnu services herd) + (guix combinators)))) + + (define forwarded-port 8080) + + (define vm + (virtual-machine + (operating-system os) + (memory-size 1024) + (port-forwardings `((,forwarded-port . 8765))))) + + (define test + (with-imported-modules '((gnu build marionette)) + #~(begin + (use-modules (srfi srfi-11) (srfi srfi-64) + (gnu build marionette) + (web uri) + (web client) + (web response)) + + (define marionette + (make-marionette (list #$vm))) + + (mkdir #$output) + (chdir #$output) + + (test-begin "guix-data-service") + + (test-assert "service running" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (match (start-service 'guix-data-service) + (#f #f) + (('service response-parts ...) + (match (assq-ref response-parts 'running) + ((pid) (number? pid)))))) + marionette)) + + (test-assert "process jobs service running" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (match (start-service 'guix-data-service-process-jobs) + (#f #f) + (('service response-parts ...) + (match (assq-ref response-parts 'running) + ((pid) (number? pid)))))) + marionette)) + + (test-equal "http-get" + 200 + (let-values + (((response text) + (http-get #$(simple-format + #f "http://localhost:~A/healthcheck" forwarded-port) + #:decode-body? #t))) + (response-code response))) + + (test-end) + (exit (= (test-runner-fail-count (test-runner-current)) 0))))) + + (gexp->derivation "guix-data-service-test" test)) + +(define %test-guix-data-service + (system-test + (name "guix-data-service") + (description "Connect to a running Guix Data Service.") + (value (run-guix-data-service-test)))) -- 2.23.0 ^ permalink raw reply related [flat|nested] 17+ messages in thread
* [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. 2019-09-22 12:03 ` Christopher Baines 2019-09-22 12:03 ` [bug#37412] [PATCH 2/2] services: Add the Guix Data Service Christopher Baines @ 2019-09-22 21:11 ` Ludovic Courtès 2019-09-24 19:34 ` Christopher Baines 1 sibling, 1 reply; 17+ messages in thread From: Ludovic Courtès @ 2019-09-22 21:11 UTC (permalink / raw) To: Christopher Baines; +Cc: 37412 Christopher Baines <mail@cbaines.net> skribis: > * gnu/packages/web.scm (guix-data-service): New variable. [...] > + (native-inputs > + `(("autoconf" ,autoconf) > + ("automake" ,automake) > + ("emacs-with-modules" ,(directory-union > + "emacs-union" > + (list emacs-no-x > + emacs-htmlize))) Should it be ‘emacs-minimal’? > + (native-search-paths > + ;; XXX guile-git requires this to be set, maybe there's a better way > + (list (search-path-specification > + (variable "GIT_SSL_CAINFO") > + (file-type 'regular) > + (separator #f) > + (files '("etc/ssl/certs/ca-certificates.crt"))))) Like I wrote, I’d rather remove the search path and add the environment variable… > + (start #~(make-forkexec-constructor > + (list #$(file-append package > + "/bin/guix-data-service") > + "--pid-file=/var/run/guix-data-service/pid" > + #$(string-append "--port=" (number->string port)) > + #$(string-append "--host=" host) > + ;; Perform any database migrations when the > + ;; service is started > + "--update-database") > + > + #:user #$user > + #:group #$group > + #:pid-file "/var/run/guix-data-service/pid" > + ;; Allow time for migrations to run > + #:pid-file-timeout 60 > + #:environment-variables > + `(,(string-append > + "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale") > + "LC_ALL=en_US.utf8") > + #:log-file "/var/log/guix-data-service/web.log")) … here. ↑ But anyway, these are details so I think you can go ahead and push as you see fit. Thanks! Ludo’. ^ permalink raw reply [flat|nested] 17+ messages in thread
* [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. 2019-09-22 21:11 ` [bug#37412] [PATCH 1/2] gnu: Add guix-data-service Ludovic Courtès @ 2019-09-24 19:34 ` Christopher Baines 0 siblings, 0 replies; 17+ messages in thread From: Christopher Baines @ 2019-09-24 19:34 UTC (permalink / raw) To: Ludovic Courtès; +Cc: 37412 [-- Attachment #1: Type: text/plain, Size: 3026 bytes --] Ludovic Courtès <ludo@gnu.org> writes: > Christopher Baines <mail@cbaines.net> skribis: > >> * gnu/packages/web.scm (guix-data-service): New variable. > > [...] > >> + (native-inputs >> + `(("autoconf" ,autoconf) >> + ("automake" ,automake) >> + ("emacs-with-modules" ,(directory-union >> + "emacs-union" >> + (list emacs-no-x >> + emacs-htmlize))) > > Should it be ‘emacs-minimal’? It probably could be, but at the moment this doesn't work with the Emacs Lisp code that's run at package build time to convert the README to HTML [1]. 1: https://git.savannah.gnu.org/cgit/guix/data-service.git/tree/Makefile.am#n35 Do you know if there's a way to remove the need for the autoloading behaviour from the guix-emacs.el file? The error I get when using emacs-minimal suggests htmlize isn't being found. >> + (native-search-paths >> + ;; XXX guile-git requires this to be set, maybe there's a better way >> + (list (search-path-specification >> + (variable "GIT_SSL_CAINFO") >> + (file-type 'regular) >> + (separator #f) >> + (files '("etc/ssl/certs/ca-certificates.crt"))))) > > Like I wrote, I’d rather remove the search path and add the environment > variable… Yeah, I'm actually unsure if this is doing anything. I'll remove it. >> + (start #~(make-forkexec-constructor >> + (list #$(file-append package >> + "/bin/guix-data-service") >> + "--pid-file=/var/run/guix-data-service/pid" >> + #$(string-append "--port=" (number->string port)) >> + #$(string-append "--host=" host) >> + ;; Perform any database migrations when the >> + ;; service is started >> + "--update-database") >> + >> + #:user #$user >> + #:group #$group >> + #:pid-file "/var/run/guix-data-service/pid" >> + ;; Allow time for migrations to run >> + #:pid-file-timeout 60 >> + #:environment-variables >> + `(,(string-append >> + "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale") >> + "LC_ALL=en_US.utf8") >> + #:log-file "/var/log/guix-data-service/web.log")) > > … here. ↑ So, this shepherd service which serves requests doesn't do any Git access, and the process jobs shepherd service defined below actually already sets it: "GIT_SSL_CAINFO=/etc/ssl/certs/ca-certificates.crt" So yeah, I think I can just remove the native-search-paths bit from the package definition. > But anyway, these are details so I think you can go ahead and push as > you see fit. Great, thanks for taking a look Ludo! Chris [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 962 bytes --] ^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2019-09-25 7:47 UTC | newest] Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-09-15 18:18 [bug#37412] [PATCH 0/2] Add package and service for the Guix Data Service Christopher Baines 2019-09-15 18:21 ` [bug#37412] [PATCH 1/2] gnu: Add guix-data-service Christopher Baines 2019-09-15 18:21 ` [bug#37412] [PATCH 2/2] services: Add the Guix Data Service Christopher Baines 2019-09-16 7:53 ` Ludovic Courtès 2019-09-22 12:15 ` Christopher Baines 2019-09-15 20:45 ` [bug#37412] [PATCH 1/2] gnu: Add guix-data-service Alex ter Weele 2019-09-15 21:29 ` Christopher Baines 2019-09-16 7:48 ` Ludovic Courtès 2019-09-22 10:18 ` Christopher Baines 2019-09-22 21:04 ` Ludovic Courtès 2019-09-25 7:46 ` bug#37412: " Christopher Baines 2019-09-16 7:55 ` [bug#37412] " Ludovic Courtès 2019-09-22 10:10 ` Christopher Baines 2019-09-22 12:03 ` Christopher Baines 2019-09-22 12:03 ` [bug#37412] [PATCH 2/2] services: Add the Guix Data Service Christopher Baines 2019-09-22 21:11 ` [bug#37412] [PATCH 1/2] gnu: Add guix-data-service Ludovic Courtès 2019-09-24 19:34 ` Christopher Baines
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).