From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33757) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dl0oD-0003l3-QF for guix-patches@gnu.org; Thu, 24 Aug 2017 18:41:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dl0oA-0003UX-KM for guix-patches@gnu.org; Thu, 24 Aug 2017 18:41:05 -0400 Received: from debbugs.gnu.org ([208.118.235.43]:44641) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dl0oA-0003UP-CV for guix-patches@gnu.org; Thu, 24 Aug 2017 18:41:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1dl0oA-0001pC-4s for guix-patches@gnu.org; Thu, 24 Aug 2017 18:41:02 -0400 Subject: [bug#27855] [PATCH] gnu: Add rsync service. Resent-Message-ID: From: Oleg Pykhalov References: <20170727220151.2116-1-go.wigust@gmail.com> <20170728231747.5eae3af9@cbaines.net> <874ltvh5d6.fsf@gmail.com> <20170729125554.29836b28@cbaines.net> <87r2wszni8.fsf@gmail.com> <20170803163322.3e87b004@cbaines.net> <87fud8zkqv.fsf@gmail.com> <20170810081820.71b29b0a@cbaines.net> <20170810192139.2c3f04da@cbaines.net> <87mv778d66.fsf@gmail.com> <87y3qprjjd.fsf@gmail.com> <20170812075903.0dc36c0c@cbaines.net> <87valtylnc.fsf@gmail.com> <20170812184649.2003677a@cbaines.net> <87wp68wnbz.fsf@gmail.com> <20170812221817.3f04b061@cbaines.net> <87o9rb8fg3.fsf@gmail.com> <20170819231949.0ce64135@cbaines.net> Date: Fri, 25 Aug 2017 01:40:43 +0300 In-Reply-To: <20170819231949.0ce64135@cbaines.net> (Christopher Baines's message of "Sat, 19 Aug 2017 23:19:49 +0100") Message-ID: <87d17kh9n8.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: Christopher Baines Cc: 27855@debbugs.gnu.org --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello Christopher, Christopher Baines writes: >> >> > Yep, I think I just stopped writing the test after finding the >> >> > issue with the PID file. >> >> > >> >> > I haven't looked in to how to fix this in the test, so if you >> >> > could, that would be great. Otherwise, I'll probably have time >> >> > to look at this again within a week or so. >> >> > >> >> > You'll probably need to refactor the test a bit, as at the >> >> > moment, the information regarding the port isn't available where >> >> > you run the commands.=20=20=20=20 >> >>=20 >> >> Of course I'll try. By the way, how to run a =E2=80=9Cvm=E2=80=9D? = Previous >> >> method =E2=80=9C./pre-inst-env guix system vm gnu/tests/rsync.scm=E2= =80=9D doesn't >> >> work for me.=20=20 >> > >> > I'm guessing that you'll need to make the file evaluate (I'm not >> > sure if that is the right word here) to an operating-system, e.g. >> > put %rsync-os-with-port-2000 right at the bottom of the file, and >> > then guix system vm should give you a start script that will start >> > a VM for that OS.=20=20 >>=20 >> I did some work on rsync service: >>=20 >> - Fixed PID and synchronization to specific port. >> - Merged two rsync oses in one with optional port. >> - Added ports to rsync synchronization tests and change protocol from >> ssh to rsync. >> - Added some logic to config: chroot (can use only root), user and >> group. >>=20 >> All tests passed successfully for me. > > Great :) Now that the tests pass at least, I don't see any reason to > not merge this soonish. > > I've still done some thingking about how the configuration works > though, and I've been considering a few ways of tweaking this so that > its harder to break, and clear in how it works. > > One way I've managed to break the service so far is setting the user > and group to root in the configuration. This causes the tests to fail, > and in a odd way, as I think the problem is that the creation of > the /var/run/rsync directory relies on the account service, but I'm > guessing that the account service does nothing in this case, as root is > already an account. > > One way of making this harder to break would be to explicitly create > the necessary directories when this service is activated, e.g.: > > (mkdir-p (dirname #$(rsync-configuration-pid-file config))) > (mkdir-p (dirname #$(rsync-configuration-lock-file config))) > > I think there is also the opportunity to make the service configuration > clearer here, as considering the default port test, the default > configuration says it will run as the rsync user, but the service will > actually run as the root user. > > This could be improved by making the configuration more uncertain by > default, e.g. user defaults to #f, which means the correct user is > decided based on the port. > > Also on the subject of clarity, the use-chroot? option is something > that can be specified, but the value might not be used. My preference > would be to change the "logic" in the configuration file generation, to > validation, e.g.: > > (if (and use-chroot? (not (eq? user "root"))) > (error "rsync-service: to run rsync in a chroot, the user must be r= oot")) > > The use-chroot? option might also benefit from making the user default > to #f, as then the service could decide the user based on the port and > use-chroot? settings, without contradicting the configuration. I tried to apply all your suggestion, but I really don't know why 'rsync-configuration-user' is ignored (it's always set to "rsyncd"). I use 'pk' to debug this, but still confused. --8<---------------cut here---------------start------------->8--- ;;; (rsync-config-file-config #< package: # port-number: 581 pid-file: "/v= ar/run/rsyncd/rsyncd.pid" lock-file: "/var/run/rsyncd/rsyncd.lock" log-file= : "/var/log/rsyncd.log" use-chroot?: #f share-path: "/srv/rsyncd" share-com= ment: "Rsync share" read-only?: #f timeout: 300 user: "rsyncd" group: "rsyn= cd" uid: #f gid: #f>) --8<---------------cut here---------------end--------------->8--- --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=0001-gnu-Add-rsync-service.patch Content-Transfer-Encoding: quoted-printable >From 4013506b89ecfd9cc5c59f3311dfa9b07bc140d3 Mon Sep 17 00:00:00 2001 From: Oleg Pykhalov Date: Thu, 27 Jul 2017 04:01:01 +0300 Subject: [PATCH] gnu: Add rsync service. * doc/guix.texi (Incremental file transfer): Add documentation. * gnu/services/rsync.scm (): New record type. (rsync-accounts, rsync-shepherd-service): New service extensions. (rsync-service-type): New service type. * gnu/tests/rsync.scm: New file. * gnu/local.mk (GNU_SYSTEM_MODULES): Add it. --- doc/guix.texi | 67 +++++++++++++++++++ gnu/local.mk | 2 + gnu/services/rsync.scm | 170 +++++++++++++++++++++++++++++++++++++++++++++= ++++ gnu/tests/rsync.scm | 136 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 375 insertions(+) create mode 100644 gnu/services/rsync.scm create mode 100644 gnu/tests/rsync.scm diff --git a/doc/guix.texi b/doc/guix.texi index 6b4b19d0c..35833ea68 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -15820,6 +15820,73 @@ Extra options will be passed to @code{git daemon},= please run @end table @end deftp =20 +@subsubsection Incremental file transfer + +The @code{(gnu services rsync)} module provides the following services: + +@subsubheading Rsync service + +You might want an rsync daemon if you have files that you want available +so anyone (or just yourself) can download existing files or upload new +files. + +@deffn {Scheme Variable} rsync-service-type +This is the type for the @uref{https://rsync.samba.org} rsync daemon, +@command{rsync-configuration} record as in this example: + +@example +(service rsync-service-type + (rsync-configuration)) +@end example + +See below for details about @code{rsync-configuration}. +@end deffn + +@deftp {Data Type} rsync-configuration +Data type representing the configuration for @code{rsync-service}. + +@table @asis +@item @code{package} (default: @var{rsync}) +@code{rsync} package to use. + +@item @code{port-number} (default: @code{873}) +TCP port on which @command{rsync} listens for incoming connections. If +port is less than @code{1024} @command{rsync} will be started as the +@code{root} user and group. + +@item @code{pid-file} (default: @code{"/var/run/rsyncd/rsyncd.pid"}) +Name of the file where @command{rsync} writes its PID. + +@item @code{lock-file} (default: @code{"/var/run/rsyncd/rsyncd.lock"}) +Name of the file where @command{rsync} writes its lock file. + +@item @code{log-file} (default: @code{"/var/log/rsyncd.log"}) +Name of the file where @command{rsync} writes its log file. + +@item @code{use-chroot?} (default: @var{#f}) +Whether to use chroot for @command{rsync} shared directory. + +@item @code{share-path} (default: @file{/srv/rsync}) +Location of the @command{rsync} shared directory. + +@item @code{share-comment} (default: @code{"Rsync share"}) +Comment of the @command{rsync} shared directory. + +@item @code{read-only?} (default: @var{#f}) +Read-write permissions to shared directory. + +@item @code{timeout} (default: @code{300}) +I/O timeout in seconds. + +@item @code{user} (default: @var{"rsyncd"}) +Owner of the @code{rsync} process. + +@item @code{group} (default: @var{"rsyncd"}) +Group of the @code{rsync} process. + +@end table +@end deftp + @node Setuid Programs @subsection Setuid Programs =20 diff --git a/gnu/local.mk b/gnu/local.mk index ff4777765..a1cace5ec 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -444,6 +444,7 @@ GNU_SYSTEM_MODULES =3D \ %D%/services/shepherd.scm \ %D%/services/herd.scm \ %D%/services/pm.scm \ + %D%/services/rsync.scm \ %D%/services/sddm.scm \ %D%/services/spice.scm \ %D%/services/ssh.scm \ @@ -487,6 +488,7 @@ GNU_SYSTEM_MODULES =3D \ %D%/tests/mail.scm \ %D%/tests/messaging.scm \ %D%/tests/networking.scm \ + %D%/tests/rsync.scm \ %D%/tests/ssh.scm \ %D%/tests/web.scm =20 diff --git a/gnu/services/rsync.scm b/gnu/services/rsync.scm new file mode 100644 index 000000000..769ad9c0c --- /dev/null +++ b/gnu/services/rsync.scm @@ -0,0 +1,170 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2017 Oleg Pykhalov +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu services rsync) + #:use-module (gnu services) + #:use-module (gnu services base) + #:use-module (gnu services shepherd) + #:use-module (gnu system shadow) + #:use-module (gnu packages rsync) + #:use-module (gnu packages admin) + #:use-module (guix records) + #:use-module (guix gexp) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:use-module (ice-9 match) + #:export (rsync-configuration + rsync-configuration? + rsync-service-type)) + +;;;; Commentary: +;;; +;;; This module implements a service that to run instance of Rsync, +;;; files synchronization tool. +;;; +;;;; Code: + +(define-record-type* + rsync-configuration make-rsync-configuration + rsync-configuration? + (package rsync-configuration-package ; package + (default rsync)) + (port-number rsync-configuration-port-number ; integer + (default 873)) + (pid-file rsync-configuration-pid-file ; string + (default "/var/run/rsyncd/rsyncd.pid")) + (lock-file rsync-configuration-lock-file ; string + (default "/var/run/rsyncd/rsyncd.lock")) + (log-file rsync-configuration-log-file ; string + (default "/var/log/rsyncd.log")) + (use-chroot? rsync-configuration-use-chroot? ; boolean + (default #f)) + (share-path rsync-configuration-share-path ; string + (default "/srv/rsyncd")) + (share-comment rsync-configuration-share-comment ; string + (default "Rsync share")) + (read-only? rsync-configuration-read-only? ; boolean + (default #f)) + (timeout rsync-configuration-timeout ; integer + (default 300)) + (user rsync-configuration-user ; string + (default "root")) + (group rsync-configuration-group ; string + (default "root")) + (uid rsync-configuration-uid + (default #f)) + (gid rsync-configuration-gid + (default #f))) + +(define (rsync-account config) + "Return the user accounts and user groups for CONFIG." + (let ((rsync-user (rsync-configuration-user config)) + (rsync-group (rsync-configuration-group config))) + (list (user-group (name rsync-group) (system? #t)) + (user-account + (name rsync-user) + (system? #t) + (group rsync-group) + (comment "rsyncd privilege separation user") + (home-directory "/var/run/rsyncd") + (shell #~(string-append #$shadow "/sbin/nologin")))))) + +(define (rsync-activation config) + "Return the activation GEXP for CONFIG." + #~(begin + (use-modules (guix build utils)) + (let ((share-directory #$(rsync-configuration-share-path config)) + (user #$(rsync-configuration-user config)) + (group #$(rsync-configuration-group config))) + (mkdir-p (dirname #$(rsync-configuration-pid-file config))) + (and=3D> share-directory mkdir-p) + (if (not (eq? user "root")) + (chown share-directory + (passwd:uid (getpw user)) + (group:gid (getpw group))))))) + +(define (rsync-config-file config) + "Return the rsync configuration file corresponding to CONFIG." + (let ((port-number (rsync-configuration-port-number (pk 'rsync-confi= g-file-config config))) + (pid-file (rsync-configuration-pid-file config)) + (lock-file (rsync-configuration-lock-file config)) + (log-file (rsync-configuration-log-file config)) + (use-chroot? (rsync-configuration-use-chroot? config)) + (share-path (rsync-configuration-share-path config)) + (share-comment (rsync-configuration-share-comment config)) + (read-only? (rsync-configuration-read-only? config)) + (timeout (rsync-configuration-timeout config)) + (user (rsync-configuration-user config)) + (group (rsync-configuration-group config)) + (uid (rsync-configuration-uid config)) + (gid (rsync-configuration-gid config))) + (if (not (eq? user "root")) + (begin + (if (<=3D port-number 1024) + (error (string-append "rsync-service: to run on port " + (number->string port-number) + ", user must be root."))) + (if use-chroot? (error (string-append + "rsync-service: to run in a chroot" + ", user must be root."))) + (if uid (error "rsync-service: to use uid, user must be root.")) + (if gid (error "rsync-service: to use gid, user must be root."))= )) + (mixed-text-file + "rsync.conf" + "# Generated by 'rsync-service'.\n" + "pid file =3D " pid-file "\n" + "lock file =3D " lock-file "\n" + "log file =3D " log-file "\n" + "port =3D " (number->string port-number) "\n" + "use chroot =3D " (if use-chroot? "true" "false") "\n" + (if uid (string-append "uid =3D " uid "\n") "") + (if uid (string-append "gid =3D " gid "\n") "") + "\n" + "[files]\n" + "path =3D " share-path "\n" + "comment =3D " share-comment "\n" + "read only =3D " (if read-only? "true" "false") "\n" + "timeout =3D " (number->string timeout) "\n"))) + +(define (rsync-shepherd-service config) + "Return a for rsync with CONFIG." + (let* ((rsync (rsync-configuration-package config)) + (pid-file (rsync-configuration-pid-file config)) + (port-number (rsync-configuration-port-number config)) + (user (rsync-configuration-user config)) + (group (rsync-configuration-group config))) + (list (shepherd-service + (provision '(rsync)) + (documentation "Run rsync daemon.") + (start #~(make-forkexec-constructor + (list (string-append #$rsync "/bin/rsync") + "--config" (pk 'shepherd-rsync-config-file #$(r= sync-config-file config)) + "--daemon") + #:pid-file (pk 'shepherd-pid-file #$pid-file) + #:user (pk 'shepherd-user #$user) + #:group (pk 'shepherd-group #$group))) + (stop #~(make-kill-destructor)))))) + +(define rsync-service-type + (service-type + (name 'rsync) + (extensions + (list (service-extension shepherd-root-service-type rsync-shepherd-ser= vice) + (service-extension account-service-type rsync-account) + (service-extension activation-service-type rsync-activation))) + (default-value (rsync-configuration)))) diff --git a/gnu/tests/rsync.scm b/gnu/tests/rsync.scm new file mode 100644 index 000000000..782c2355c --- /dev/null +++ b/gnu/tests/rsync.scm @@ -0,0 +1,136 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2017 Christopher Baines +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu tests rsync) + #:use-module (gnu packages rsync) + #:use-module (gnu tests) + #:use-module (gnu system) + #:use-module (gnu system file-systems) + #:use-module (gnu system shadow) + #:use-module (gnu system vm) + #:use-module (gnu services) + #:use-module (gnu services rsync) + #:use-module (gnu services networking) + #:use-module (guix gexp) + #:use-module (guix store) + #:export (%test-rsync-with-default-port + %test-rsync-with-port-2000)) + +(define* (run-rsync-test rsync-os #:optional (rsync-port 581)) + "Run tests in %RSYNC-OS, which has rsync running and listening on +PORT." + (define os + (marionette-operating-system + rsync-os + #:imported-modules '((gnu services herd) + (guix combinators)))) + + (define vm + (virtual-machine + (operating-system os) + (port-forwardings '()))) + + (define test + (with-imported-modules '((gnu build marionette)) + #~(begin + (use-modules (srfi srfi-11) (srfi srfi-64) + (gnu build marionette)) + + (define marionette + (make-marionette (list #$vm))) + + (mkdir #$output) + (chdir #$output) + + (test-begin "rsync") + + ;; Wait for rsync to be up and running. + (test-eq "service running" + 'running! + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (start-service 'rsync) + 'running!) + marionette)) + + ;; Make sure the PID file is created. + (test-assert "PID file" + (marionette-eval + '(file-exists? "/var/run/rsyncd/rsyncd.pid") + marionette)) + + (test-assert "Test file copied to share" + (marionette-eval + '(begin + (call-with-output-file "/tmp/input" + (lambda (port) + (display "test-file-contents\n" port))) + (zero? + (system* "rsync" "/tmp/input" + (string-append "rsync://localhost:" + (number->string #$rsync-port) + "/files/input")))) + marionette)) + + (test-equal "Test file correctly received from share" + "test-file-contents" + (marionette-eval + '(begin + (use-modules (ice-9 rdelim)) + (zero? + (system* "rsync" + (string-append "rsync://localhost:" + (number->string #$rsync-port) + "/files/input") + "/tmp/output")) + (call-with-input-file "/tmp/output" + (lambda (port) + (read-line port)))) + marionette)) + + (test-end) + (exit (=3D (test-runner-fail-count (test-runner-current)) 0))))) + + (gexp->derivation "rsync-test" test)) + +(define* (%rsync-os #:optional (rsync-port 581)) + ;; Operating system under test. + (let ((base-os + (simple-operating-system + (dhcp-client-service) + (service rsync-service-type + (rsync-configuration + (port-number rsync-port)))))) + (operating-system + (inherit base-os) + (packages (cons* rsync + (operating-system-packages base-os)))))) + +(define %test-rsync-with-default-port + (system-test + (name "rsync-with-default-port") + (description "Connect to a running RSYNC server.") + (value (run-rsync-test (%rsync-os))))) + +(define %test-rsync-with-port-2000 + (let ((port 2000)) + (system-test + (name "rsync-with-port-2000") + (description "Connect to a running RSYNC server.") + (value (run-rsync-test (%rsync-os port) port))))) --=20 2.14.1 --=-=-=--