From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:33581) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1djASX-0000CW-R0 for guix-patches@gnu.org; Sat, 19 Aug 2017 16:35:08 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1djASU-0005rd-Jw for guix-patches@gnu.org; Sat, 19 Aug 2017 16:35:05 -0400 Received: from debbugs.gnu.org ([208.118.235.43]:37184) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1djASU-0005rD-CY for guix-patches@gnu.org; Sat, 19 Aug 2017 16:35:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1djASU-0002uL-38 for guix-patches@gnu.org; Sat, 19 Aug 2017 16:35:02 -0400 Subject: [bug#27855] [PATCH] gnu: Add rsync service. Resent-Message-ID: From: Oleg Pykhalov References: <20170727220151.2116-1-go.wigust@gmail.com> <20170728231747.5eae3af9@cbaines.net> <874ltvh5d6.fsf@gmail.com> <20170729125554.29836b28@cbaines.net> <87r2wszni8.fsf@gmail.com> <20170803163322.3e87b004@cbaines.net> <87fud8zkqv.fsf@gmail.com> <20170810081820.71b29b0a@cbaines.net> <20170810192139.2c3f04da@cbaines.net> <87mv778d66.fsf@gmail.com> <87y3qprjjd.fsf@gmail.com> <20170812075903.0dc36c0c@cbaines.net> <87valtylnc.fsf@gmail.com> <20170812184649.2003677a@cbaines.net> <87wp68wnbz.fsf@gmail.com> <20170812221817.3f04b061@cbaines.net> Date: Sat, 19 Aug 2017 23:34:20 +0300 In-Reply-To: <20170812221817.3f04b061@cbaines.net> (Christopher Baines's message of "Sat, 12 Aug 2017 22:19:00 +0100") Message-ID: <87o9rb8fg3.fsf@gmail.com> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: Christopher Baines Cc: 27855@debbugs.gnu.org --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello Christopher, Christopher Baines writes: >> > Yep, I think I just stopped writing the test after finding the issue >> > with the PID file. >> > >> > I haven't looked in to how to fix this in the test, so if you could, >> > that would be great. Otherwise, I'll probably have time to look at >> > this again within a week or so. >> > >> > You'll probably need to refactor the test a bit, as at the moment, >> > the information regarding the port isn't available where you run the >> > commands.=20=20 >>=20 >> Of course I'll try. By the way, how to run a =E2=80=9Cvm=E2=80=9D? Pre= vious method >> =E2=80=9C./pre-inst-env guix system vm gnu/tests/rsync.scm=E2=80=9D does= n't work for >> me. > > I'm guessing that you'll need to make the file evaluate (I'm not sure if > that is the right word here) to an operating-system, e.g. put > %rsync-os-with-port-2000 right at the bottom of the file, and then guix > system vm should give you a start script that will start a VM for that > OS. I did some work on rsync service: - Fixed PID and synchronization to specific port. - Merged two rsync oses in one with optional port. - Added ports to rsync synchronization tests and change protocol from ssh to rsync. - Added some logic to config: chroot (can use only root), user and group. All tests passed successfully for me. --=-=-= Content-Type: text/x-patch; charset=utf-8 Content-Disposition: inline; filename=0001-gnu-Add-rsync-service.patch Content-Transfer-Encoding: quoted-printable Content-Description: [PATCH] gnu: Add rsync service. >From 21ccef0c4fa23949d6a59e317d2e258f18ec8f5a Mon Sep 17 00:00:00 2001 From: Oleg Pykhalov Date: Thu, 27 Jul 2017 04:01:01 +0300 Subject: [PATCH] gnu: Add rsync service. * doc/guix.texi (Incremental file transfer): Add documentation. * gnu/services/rsync.scm (): New record type. (rsync-accounts, rsync-shepherd-service): New service extensions. (rsync-service-type): New service type. * gnu/tests/rsync.scm: New file. * gnu/local.mk (GNU_SYSTEM_MODULES): Add it. --- doc/guix.texi | 67 ++++++++++++++++++++++ gnu/local.mk | 2 + gnu/services/rsync.scm | 153 +++++++++++++++++++++++++++++++++++++++++++++= ++++ gnu/tests/rsync.scm | 136 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 358 insertions(+) create mode 100644 gnu/services/rsync.scm create mode 100644 gnu/tests/rsync.scm diff --git a/doc/guix.texi b/doc/guix.texi index 6b4b19d0c..35833ea68 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -15820,6 +15820,73 @@ Extra options will be passed to @code{git daemon},= please run @end table @end deftp =20 +@subsubsection Incremental file transfer + +The @code{(gnu services rsync)} module provides the following services: + +@subsubheading Rsync service + +You might want an rsync daemon if you have files that you want available +so anyone (or just yourself) can download existing files or upload new +files. + +@deffn {Scheme Variable} rsync-service-type +This is the type for the @uref{https://rsync.samba.org} rsync daemon, +@command{rsync-configuration} record as in this example: + +@example +(service rsync-service-type + (rsync-configuration)) +@end example + +See below for details about @code{rsync-configuration}. +@end deffn + +@deftp {Data Type} rsync-configuration +Data type representing the configuration for @code{rsync-service}. + +@table @asis +@item @code{package} (default: @var{rsync}) +@code{rsync} package to use. + +@item @code{port-number} (default: @code{873}) +TCP port on which @command{rsync} listens for incoming connections. If +port is less than @code{1024} @command{rsync} will be started as the +@code{root} user and group. + +@item @code{pid-file} (default: @code{"/var/run/rsyncd/rsyncd.pid"}) +Name of the file where @command{rsync} writes its PID. + +@item @code{lock-file} (default: @code{"/var/run/rsyncd/rsyncd.lock"}) +Name of the file where @command{rsync} writes its lock file. + +@item @code{log-file} (default: @code{"/var/log/rsyncd.log"}) +Name of the file where @command{rsync} writes its log file. + +@item @code{use-chroot?} (default: @var{#f}) +Whether to use chroot for @command{rsync} shared directory. + +@item @code{share-path} (default: @file{/srv/rsync}) +Location of the @command{rsync} shared directory. + +@item @code{share-comment} (default: @code{"Rsync share"}) +Comment of the @command{rsync} shared directory. + +@item @code{read-only?} (default: @var{#f}) +Read-write permissions to shared directory. + +@item @code{timeout} (default: @code{300}) +I/O timeout in seconds. + +@item @code{user} (default: @var{"rsyncd"}) +Owner of the @code{rsync} process. + +@item @code{group} (default: @var{"rsyncd"}) +Group of the @code{rsync} process. + +@end table +@end deftp + @node Setuid Programs @subsection Setuid Programs =20 diff --git a/gnu/local.mk b/gnu/local.mk index ff4777765..a1cace5ec 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -444,6 +444,7 @@ GNU_SYSTEM_MODULES =3D \ %D%/services/shepherd.scm \ %D%/services/herd.scm \ %D%/services/pm.scm \ + %D%/services/rsync.scm \ %D%/services/sddm.scm \ %D%/services/spice.scm \ %D%/services/ssh.scm \ @@ -487,6 +488,7 @@ GNU_SYSTEM_MODULES =3D \ %D%/tests/mail.scm \ %D%/tests/messaging.scm \ %D%/tests/networking.scm \ + %D%/tests/rsync.scm \ %D%/tests/ssh.scm \ %D%/tests/web.scm =20 diff --git a/gnu/services/rsync.scm b/gnu/services/rsync.scm new file mode 100644 index 000000000..695ea9bd5 --- /dev/null +++ b/gnu/services/rsync.scm @@ -0,0 +1,153 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2017 Oleg Pykhalov +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu services rsync) + #:use-module (gnu services) + #:use-module (gnu services base) + #:use-module (gnu services shepherd) + #:use-module (gnu system shadow) + #:use-module (gnu packages rsync) + #:use-module (gnu packages admin) + #:use-module (guix records) + #:use-module (guix gexp) + #:use-module (srfi srfi-1) + #:use-module (srfi srfi-26) + #:use-module (ice-9 match) + #:export (rsync-configuration + rsync-configuration? + rsync-service-type)) + +;;;; Commentary: +;;; +;;; This module implements a service that to run instance of Rsync, +;;; files synchronization tool. +;;; +;;;; Code: + +(define-record-type* + rsync-configuration make-rsync-configuration + rsync-configuration? + (package rsync-configuration-package ;package + (default rsync)) + (port-number rsync-configuration-port-number ;integer + (default 873)) + (pid-file rsync-configuration-pid-file ;string + (default "/var/run/rsyncd/rsyncd.pid")) + (lock-file rsync-configuration-lock-file ;string + (default "/var/run/rsyncd/rsyncd.lock")) + (log-file rsync-configuration-log-file ;string + (default "/var/log/rsyncd.log")) + (use-chroot? rsync-configuration-use-chroot? ;boolean + (default #f)) + (share-path rsync-configuration-share-path ;string + (default "/srv/rsyncd")) + (share-comment rsync-configuration-share-comment ;string + (default "Rsync share")) + (read-only? rsync-configuration-read-only? ;boolean + (default #f)) + (timeout rsync-configuration-timeout ;integer + (default 300)) + (user rsync-configuration-user ;string + (default "rsyncd")) + (group rsync-configuration-group ;string + (default "rsyncd"))) + +(define (rsync-account config) + "Return the user accounts and user groups for CONFIG." + (let ((rsync-user (rsync-configuration-user config)) + (rsync-group (rsync-configuration-group config))) + (list (user-group (name rsync-group) (system? #t)) + (user-account + (name rsync-user) + (system? #t) + (group rsync-group) + (comment "rsyncd privilege separation user") + (home-directory "/var/run/rsyncd") + (shell #~(string-append #$shadow "/sbin/nologin")))))) + +(define (rsync-activation config) + "Return the activation GEXP for CONFIG." + #~(begin + (use-modules (guix build utils)) + (let ((share-directory #$(rsync-configuration-share-path config)) + (user (getpw #$(rsync-configuration-user config))) + (group (getpw #$(rsync-configuration-group config)))) + (and=3D> share-directory mkdir-p) + (chown share-directory + (passwd:uid user) + (group:gid group))))) + +(define (rsync-config-file config) + "Return the rsync configuration file corresponding to CONFIG." + (match config + (($ package port-number pid-file + lock-file log-file use-chroot? share-path + share-comment read-only? timeout user group) + (mixed-text-file + "rsync.conf" + "# Generated by 'rsync-service'.\n" + "pid file =3D " pid-file "\n" + "lock file =3D " lock-file "\n" + "log file =3D " log-file "\n" + "port =3D " (number->string port-number) "\n" + "use chroot =3D " (if (and use-chroot? + (eq? user "root")) + "true" "false") + "\n" + (if (or (eq? user "root") (<=3D port-number 1024)) + (string-append "uid =3D " user "\n") "") + (if (or (eq? user "root") (<=3D port-number 1024)) + (string-append "gid =3D " group "\n") "") + "\n" + "[files]\n" + "path =3D " share-path "\n" + "comment =3D " share-comment "\n" + "read only =3D " (if read-only? "true" "false") "\n" + "timeout =3D " (number->string timeout) "\n")))) + +(define (rsync-shepherd-service config) + "Return a for rsync with CONFIG." + (let* ((rsync (rsync-configuration-package config)) + (pid-file (rsync-configuration-pid-file config)) + (port-number (rsync-configuration-port-number config)) + (user (if (> port-number 1024) + (rsync-configuration-user config) + "root")) + (group (if (> port-number 1024) + (rsync-configuration-group config) + "root"))) + (list (shepherd-service + (provision '(rsync)) + (documentation "Run rsync daemon.") + (start #~(make-forkexec-constructor + (list (string-append #$rsync "/bin/rsync") + "--config" #$(rsync-config-file config) + "--daemon") + #:pid-file #$pid-file + #:user #$user + #:group #$group)) + (stop #~(make-kill-destructor)))))) + +(define rsync-service-type + (service-type + (name 'rsync) + (extensions + (list (service-extension shepherd-root-service-type rsync-shepherd-ser= vice) + (service-extension account-service-type rsync-account) + (service-extension activation-service-type rsync-activation))) + (default-value (rsync-configuration)))) diff --git a/gnu/tests/rsync.scm b/gnu/tests/rsync.scm new file mode 100644 index 000000000..782c2355c --- /dev/null +++ b/gnu/tests/rsync.scm @@ -0,0 +1,136 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright =C2=A9 2017 Christopher Baines +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu tests rsync) + #:use-module (gnu packages rsync) + #:use-module (gnu tests) + #:use-module (gnu system) + #:use-module (gnu system file-systems) + #:use-module (gnu system shadow) + #:use-module (gnu system vm) + #:use-module (gnu services) + #:use-module (gnu services rsync) + #:use-module (gnu services networking) + #:use-module (guix gexp) + #:use-module (guix store) + #:export (%test-rsync-with-default-port + %test-rsync-with-port-2000)) + +(define* (run-rsync-test rsync-os #:optional (rsync-port 581)) + "Run tests in %RSYNC-OS, which has rsync running and listening on +PORT." + (define os + (marionette-operating-system + rsync-os + #:imported-modules '((gnu services herd) + (guix combinators)))) + + (define vm + (virtual-machine + (operating-system os) + (port-forwardings '()))) + + (define test + (with-imported-modules '((gnu build marionette)) + #~(begin + (use-modules (srfi srfi-11) (srfi srfi-64) + (gnu build marionette)) + + (define marionette + (make-marionette (list #$vm))) + + (mkdir #$output) + (chdir #$output) + + (test-begin "rsync") + + ;; Wait for rsync to be up and running. + (test-eq "service running" + 'running! + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (start-service 'rsync) + 'running!) + marionette)) + + ;; Make sure the PID file is created. + (test-assert "PID file" + (marionette-eval + '(file-exists? "/var/run/rsyncd/rsyncd.pid") + marionette)) + + (test-assert "Test file copied to share" + (marionette-eval + '(begin + (call-with-output-file "/tmp/input" + (lambda (port) + (display "test-file-contents\n" port))) + (zero? + (system* "rsync" "/tmp/input" + (string-append "rsync://localhost:" + (number->string #$rsync-port) + "/files/input")))) + marionette)) + + (test-equal "Test file correctly received from share" + "test-file-contents" + (marionette-eval + '(begin + (use-modules (ice-9 rdelim)) + (zero? + (system* "rsync" + (string-append "rsync://localhost:" + (number->string #$rsync-port) + "/files/input") + "/tmp/output")) + (call-with-input-file "/tmp/output" + (lambda (port) + (read-line port)))) + marionette)) + + (test-end) + (exit (=3D (test-runner-fail-count (test-runner-current)) 0))))) + + (gexp->derivation "rsync-test" test)) + +(define* (%rsync-os #:optional (rsync-port 581)) + ;; Operating system under test. + (let ((base-os + (simple-operating-system + (dhcp-client-service) + (service rsync-service-type + (rsync-configuration + (port-number rsync-port)))))) + (operating-system + (inherit base-os) + (packages (cons* rsync + (operating-system-packages base-os)))))) + +(define %test-rsync-with-default-port + (system-test + (name "rsync-with-default-port") + (description "Connect to a running RSYNC server.") + (value (run-rsync-test (%rsync-os))))) + +(define %test-rsync-with-port-2000 + (let ((port 2000)) + (system-test + (name "rsync-with-port-2000") + (description "Connect to a running RSYNC server.") + (value (run-rsync-test (%rsync-os port) port))))) --=20 2.14.1 --=-=-=--