[bug#27855] [PATCH] gnu: Add rsync service.

[Oleg Pykhalov <go.wigust@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1353 bytes --]

Hello Christopher,

Christopher Baines <mail@cbaines.net> writes:

>> > Yep, I think I just stopped writing the test after finding the issue
>> > with the PID file.
>> >
>> > I haven't looked in to how to fix this in the test, so if you could,
>> > that would be great. Otherwise, I'll probably have time to look at
>> > this again within a week or so.
>> >
>> > You'll probably need to refactor the test a bit, as at the moment,
>> > the information regarding the port isn't available where you run the
>> > commands.  
>> 
>> Of course I'll try.  By the way, how to run a “vm”?  Previous method
>> “./pre-inst-env guix system vm gnu/tests/rsync.scm” doesn't work for
>> me.
>
> I'm guessing that you'll need to make the file evaluate (I'm not sure if
> that is the right word here) to an operating-system, e.g. put
> %rsync-os-with-port-2000 right at the bottom of the file, and then guix
> system vm should give you a start script that will start a VM for that
> OS.

I did some work on rsync service:

- Fixed PID and synchronization to specific port.
- Merged two rsync oses in one with optional port.
- Added ports to rsync synchronization tests and change protocol from
  ssh to rsync.
- Added some logic to config: chroot (can use only root), user and
  group.

All tests passed successfully for me.


[-- Attachment #2: [PATCH] gnu: Add rsync service. --]
[-- Type: text/x-patch, Size: 15822 bytes --]

From 21ccef0c4fa23949d6a59e317d2e258f18ec8f5a Mon Sep 17 00:00:00 2001
From: Oleg Pykhalov <go.wigust@gmail.com>
Date: Thu, 27 Jul 2017 04:01:01 +0300
Subject: [PATCH] gnu: Add rsync service.

* doc/guix.texi (Incremental file transfer): Add documentation.
* gnu/services/rsync.scm (<rsync-configuration>): New record type.
(rsync-accounts, rsync-shepherd-service): New service extensions.
(rsync-service-type): New service type.
* gnu/tests/rsync.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
---
 doc/guix.texi          |  67 ++++++++++++++++++++++
 gnu/local.mk           |   2 +
 gnu/services/rsync.scm | 153 +++++++++++++++++++++++++++++++++++++++++++++++++
 gnu/tests/rsync.scm    | 136 +++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 358 insertions(+)
 create mode 100644 gnu/services/rsync.scm
 create mode 100644 gnu/tests/rsync.scm

diff --git a/doc/guix.texi b/doc/guix.texi
index 6b4b19d0c..35833ea68 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -15820,6 +15820,73 @@ Extra options will be passed to @code{git daemon}, please run
 @end table
 @end deftp
 
+@subsubsection Incremental file transfer
+
+The @code{(gnu services rsync)} module provides the following services:
+
+@subsubheading Rsync service
+
+You might want an rsync daemon if you have files that you want available
+so anyone (or just yourself) can download existing files or upload new
+files.
+
+@deffn {Scheme Variable} rsync-service-type
+This is the type for the @uref{https://rsync.samba.org} rsync daemon,
+@command{rsync-configuration} record as in this example:
+
+@example
+(service rsync-service-type
+         (rsync-configuration))
+@end example
+
+See below for details about @code{rsync-configuration}.
+@end deffn
+
+@deftp {Data Type} rsync-configuration
+Data type representing the configuration for @code{rsync-service}.
+
+@table @asis
+@item @code{package} (default: @var{rsync})
+@code{rsync} package to use.
+
+@item @code{port-number} (default: @code{873})
+TCP port on which @command{rsync} listens for incoming connections.  If
+port is less than @code{1024} @command{rsync} will be started as the
+@code{root} user and group.
+
+@item @code{pid-file} (default: @code{"/var/run/rsyncd/rsyncd.pid"})
+Name of the file where @command{rsync} writes its PID.
+
+@item @code{lock-file} (default: @code{"/var/run/rsyncd/rsyncd.lock"})
+Name of the file where @command{rsync} writes its lock file.
+
+@item @code{log-file} (default: @code{"/var/log/rsyncd.log"})
+Name of the file where @command{rsync} writes its log file.
+
+@item @code{use-chroot?} (default: @var{#f})
+Whether to use chroot for @command{rsync} shared directory.
+
+@item @code{share-path} (default: @file{/srv/rsync})
+Location of the @command{rsync} shared directory.
+
+@item @code{share-comment} (default: @code{"Rsync share"})
+Comment of the @command{rsync} shared directory.
+
+@item @code{read-only?} (default: @var{#f})
+Read-write permissions to shared directory.
+
+@item @code{timeout} (default: @code{300})
+I/O timeout in seconds.
+
+@item @code{user} (default: @var{"rsyncd"})
+Owner of the @code{rsync} process.
+
+@item @code{group} (default: @var{"rsyncd"})
+Group of the @code{rsync} process.
+
+@end table
+@end deftp
+
 @node Setuid Programs
 @subsection Setuid Programs
 
diff --git a/gnu/local.mk b/gnu/local.mk
index ff4777765..a1cace5ec 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -444,6 +444,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/services/shepherd.scm			\
   %D%/services/herd.scm				\
   %D%/services/pm.scm				\
+  %D%/services/rsync.scm			\
   %D%/services/sddm.scm				\
   %D%/services/spice.scm				\
   %D%/services/ssh.scm				\
@@ -487,6 +488,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/tests/mail.scm				\
   %D%/tests/messaging.scm			\
   %D%/tests/networking.scm			\
+  %D%/tests/rsync.scm				\
   %D%/tests/ssh.scm				\
   %D%/tests/web.scm
 
diff --git a/gnu/services/rsync.scm b/gnu/services/rsync.scm
new file mode 100644
index 000000000..695ea9bd5
--- /dev/null
+++ b/gnu/services/rsync.scm
@@ -0,0 +1,153 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2017 Oleg Pykhalov <go.wigust@gmail.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services rsync)
+  #:use-module (gnu services)
+  #:use-module (gnu services base)
+  #:use-module (gnu services shepherd)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu packages rsync)
+  #:use-module (gnu packages admin)
+  #:use-module (guix records)
+  #:use-module (guix gexp)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-26)
+  #:use-module (ice-9 match)
+  #:export (rsync-configuration
+            rsync-configuration?
+            rsync-service-type))
+
+;;;; Commentary:
+;;;
+;;; This module implements a service that to run instance of Rsync,
+;;; files synchronization tool.
+;;;
+;;;; Code:
+
+(define-record-type* <rsync-configuration>
+  rsync-configuration make-rsync-configuration
+  rsync-configuration?
+  (package       rsync-configuration-package ;package
+                 (default rsync))
+  (port-number   rsync-configuration-port-number ;integer
+                 (default 873))
+  (pid-file      rsync-configuration-pid-file ;string
+                 (default "/var/run/rsyncd/rsyncd.pid"))
+  (lock-file     rsync-configuration-lock-file ;string
+                 (default "/var/run/rsyncd/rsyncd.lock"))
+  (log-file      rsync-configuration-log-file ;string
+                 (default "/var/log/rsyncd.log"))
+  (use-chroot?   rsync-configuration-use-chroot? ;boolean
+                 (default #f))
+  (share-path    rsync-configuration-share-path ;string
+                 (default "/srv/rsyncd"))
+  (share-comment rsync-configuration-share-comment ;string
+                 (default "Rsync share"))
+  (read-only?    rsync-configuration-read-only? ;boolean
+                 (default #f))
+  (timeout       rsync-configuration-timeout ;integer
+                 (default 300))
+  (user          rsync-configuration-user ;string
+                 (default "rsyncd"))
+  (group         rsync-configuration-group ;string
+                 (default "rsyncd")))
+
+(define (rsync-account config)
+  "Return the user accounts and user groups for CONFIG."
+  (let ((rsync-user  (rsync-configuration-user config))
+        (rsync-group (rsync-configuration-group config)))
+    (list (user-group (name rsync-group) (system? #t))
+          (user-account
+           (name rsync-user)
+           (system? #t)
+           (group rsync-group)
+           (comment "rsyncd privilege separation user")
+           (home-directory "/var/run/rsyncd")
+           (shell #~(string-append #$shadow "/sbin/nologin"))))))
+
+(define (rsync-activation config)
+  "Return the activation GEXP for CONFIG."
+  #~(begin
+      (use-modules (guix build utils))
+      (let ((share-directory #$(rsync-configuration-share-path config))
+            (user            (getpw #$(rsync-configuration-user config)))
+            (group           (getpw #$(rsync-configuration-group config))))
+        (and=> share-directory mkdir-p)
+        (chown share-directory
+               (passwd:uid user)
+               (group:gid group)))))
+
+(define (rsync-config-file config)
+  "Return the rsync configuration file corresponding to CONFIG."
+  (match config
+    (($ <rsync-configuration> package port-number pid-file
+                              lock-file log-file use-chroot? share-path
+                              share-comment read-only? timeout user group)
+     (mixed-text-file
+      "rsync.conf"
+      "# Generated by 'rsync-service'.\n"
+      "pid file = " pid-file "\n"
+      "lock file = " lock-file "\n"
+      "log file = " log-file "\n"
+      "port = " (number->string port-number) "\n"
+      "use chroot = " (if (and use-chroot?
+                               (eq? user "root"))
+                          "true" "false")
+      "\n"
+      (if (or (eq? user "root") (<= port-number 1024))
+          (string-append "uid = " user "\n") "")
+      (if (or (eq? user "root") (<= port-number 1024))
+          (string-append "gid = " group "\n") "")
+      "\n"
+      "[files]\n"
+      "path = " share-path "\n"
+      "comment = " share-comment "\n"
+      "read only = " (if read-only? "true" "false") "\n"
+      "timeout = " (number->string timeout) "\n"))))
+
+(define (rsync-shepherd-service config)
+  "Return a <shepherd-service> for rsync with CONFIG."
+  (let* ((rsync        (rsync-configuration-package config))
+         (pid-file     (rsync-configuration-pid-file config))
+         (port-number  (rsync-configuration-port-number config))
+         (user         (if (> port-number 1024)
+                           (rsync-configuration-user config)
+                           "root"))
+         (group        (if (> port-number 1024)
+                           (rsync-configuration-group config)
+                           "root")))
+    (list (shepherd-service
+           (provision '(rsync))
+           (documentation "Run rsync daemon.")
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$rsync "/bin/rsync")
+                           "--config" #$(rsync-config-file config)
+                           "--daemon")
+                     #:pid-file #$pid-file
+                     #:user #$user
+                     #:group #$group))
+           (stop #~(make-kill-destructor))))))
+
+(define rsync-service-type
+  (service-type
+   (name 'rsync)
+   (extensions
+    (list (service-extension shepherd-root-service-type rsync-shepherd-service)
+          (service-extension account-service-type rsync-account)
+          (service-extension activation-service-type rsync-activation)))
+   (default-value (rsync-configuration))))
diff --git a/gnu/tests/rsync.scm b/gnu/tests/rsync.scm
new file mode 100644
index 000000000..782c2355c
--- /dev/null
+++ b/gnu/tests/rsync.scm
@@ -0,0 +1,136 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2017 Christopher Baines <mail@cbaines.net>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu tests rsync)
+  #:use-module (gnu packages rsync)
+  #:use-module (gnu tests)
+  #:use-module (gnu system)
+  #:use-module (gnu system file-systems)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu system vm)
+  #:use-module (gnu services)
+  #:use-module (gnu services rsync)
+  #:use-module (gnu services networking)
+  #:use-module (guix gexp)
+  #:use-module (guix store)
+  #:export (%test-rsync-with-default-port
+            %test-rsync-with-port-2000))
+
+(define* (run-rsync-test rsync-os #:optional (rsync-port 581))
+  "Run tests in %RSYNC-OS, which has rsync running and listening on
+PORT."
+  (define os
+    (marionette-operating-system
+     rsync-os
+     #:imported-modules '((gnu services herd)
+                          (guix combinators))))
+
+  (define vm
+    (virtual-machine
+     (operating-system os)
+     (port-forwardings '())))
+
+  (define test
+    (with-imported-modules '((gnu build marionette))
+      #~(begin
+          (use-modules (srfi srfi-11) (srfi srfi-64)
+                       (gnu build marionette))
+
+          (define marionette
+            (make-marionette (list #$vm)))
+
+          (mkdir #$output)
+          (chdir #$output)
+
+          (test-begin "rsync")
+
+          ;; Wait for rsync to be up and running.
+          (test-eq "service running"
+            'running!
+            (marionette-eval
+             '(begin
+                (use-modules (gnu services herd))
+                (start-service 'rsync)
+                'running!)
+             marionette))
+
+          ;; Make sure the PID file is created.
+          (test-assert "PID file"
+            (marionette-eval
+             '(file-exists? "/var/run/rsyncd/rsyncd.pid")
+             marionette))
+
+          (test-assert "Test file copied to share"
+            (marionette-eval
+             '(begin
+                (call-with-output-file "/tmp/input"
+                  (lambda (port)
+                    (display "test-file-contents\n" port)))
+                (zero?
+                 (system* "rsync" "/tmp/input"
+                          (string-append "rsync://localhost:"
+                                         (number->string #$rsync-port)
+                                         "/files/input"))))
+             marionette))
+
+          (test-equal "Test file correctly received from share"
+            "test-file-contents"
+            (marionette-eval
+             '(begin
+                (use-modules (ice-9 rdelim))
+                (zero?
+                 (system* "rsync"
+                          (string-append "rsync://localhost:"
+                                         (number->string #$rsync-port)
+                                         "/files/input")
+                          "/tmp/output"))
+                (call-with-input-file "/tmp/output"
+                  (lambda (port)
+                    (read-line port))))
+             marionette))
+
+          (test-end)
+          (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+  (gexp->derivation "rsync-test" test))
+
+(define* (%rsync-os #:optional (rsync-port 581))
+  ;; Operating system under test.
+  (let ((base-os
+         (simple-operating-system
+          (dhcp-client-service)
+          (service rsync-service-type
+                   (rsync-configuration
+                    (port-number rsync-port))))))
+    (operating-system
+      (inherit base-os)
+      (packages (cons* rsync
+                       (operating-system-packages base-os))))))
+
+(define %test-rsync-with-default-port
+  (system-test
+   (name "rsync-with-default-port")
+   (description "Connect to a running RSYNC server.")
+   (value (run-rsync-test (%rsync-os)))))
+
+(define %test-rsync-with-port-2000
+  (let ((port 2000))
+    (system-test
+     (name "rsync-with-port-2000")
+     (description "Connect to a running RSYNC server.")
+     (value (run-rsync-test (%rsync-os port) port)))))
-- 
2.14.1