From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id qNKoDqGKXmStMwEASxT56A (envelope-from ) for ; Fri, 12 May 2023 20:51:13 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id 8JXFDqGKXmSiVwAA9RJhRA (envelope-from ) for ; Fri, 12 May 2023 20:51:13 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id BEFFA3F143 for ; Fri, 12 May 2023 20:51:12 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pxXr5-0005xS-K8; Fri, 12 May 2023 14:51:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pxXr4-0005xK-Ov for guix-patches@gnu.org; Fri, 12 May 2023 14:51:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pxXr4-0007Pe-GB for guix-patches@gnu.org; Fri, 12 May 2023 14:51:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pxXr4-0003U5-Bq for guix-patches@gnu.org; Fri, 12 May 2023 14:51:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#63182] [PATCH v2] gnu: services: Add cachefilesd service. (Closes: #63182) References: In-Reply-To: Resent-From: Felix Lechner Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 12 May 2023 18:51:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 63182 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 63182@debbugs.gnu.org Cc: Bruno Victal , Felix Lechner Received: via spool by 63182-submit@debbugs.gnu.org id=B63182.168391743513216 (code B ref 63182); Fri, 12 May 2023 18:51:02 +0000 Received: (at 63182) by debbugs.gnu.org; 12 May 2023 18:50:35 +0000 Received: from localhost ([127.0.0.1]:32958 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pxXqc-0003Qz-ES for submit@debbugs.gnu.org; Fri, 12 May 2023 14:50:35 -0400 Received: from sail-ipv4.us-core.com ([208.82.101.137]:56624) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pxXqa-0003Qb-BN for 63182@debbugs.gnu.org; Fri, 12 May 2023 14:50:33 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; s=2017; bh=sxp2VbR/avxnzr9 c6ADzuFAyDDhP/AZrsw3n8YnsI7M=; h=date:subject:cc:to:from; d=lease-up.com; b=Ns7aWErxkEhrXvu8oWQHpdc7u8D0d4msqFxwqvVQo9CVJm5SUivE LT85M7bLa74Q+Ven8LZR88vVHcePskhRBO/HVXdCra4++pqg4Vq/7CLjquFDI1Yg1R4gYP D/lIX0U5OpF+m0z/Y30fmV5WYuWEyUSHfX6w4rkdMjfzxf6XM= Received: by sail-ipv4.us-core.com (OpenSMTPD) with ESMTPSA id eef35609 (TLSv1.3:TLS_CHACHA20_POLY1305_SHA256:256:NO); Fri, 12 May 2023 18:50:31 +0000 (UTC) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 15a5570b; Fri, 12 May 2023 18:50:31 +0000 (UTC) Date: Fri, 12 May 2023 11:50:22 -0700 Message-Id: X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: Felix Lechner X-ACL-Warn: , Felix Lechner via Guix-patches From: Felix Lechner via Guix-patches via Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Seal: i=1; s=key1; d=yhetil.org; t=1683917473; a=rsa-sha256; cv=none; b=ovPH29AdxMgTrR4cdUh4aJpSl93ywHTQqrNql5HUv6i0FaJ70O5p203VFEWCYjmEqNcZxN iYk6zn2i/GTVbKRV2LEk0JXRbUeIRZ3mlgIZIkVfAE84P1B1h1ZNdOkBkvQup8k43BeU7X Ni71dJfzAqMKH1gQYQtfCOzNR5DgXJj2IXc6ciO8sLVKiBuCp+l1uDA0lDCeaUENVZKwR7 Z/VBClczEs+6MOLIKZKHX+rlf1Vs27IeyXaLiaX8awu8qbPsW+vLk8yzljx8O71sFgwr1v GSn5DCjt8qo/HjL91Ps69ZjKttBioAbgxboxy9EPbubUdXxcRPlE5bViYiZhAQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=lease-up.com header.s=2017 header.b=Ns7aWErx; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1683917473; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=N0RQYfHrlf9gZwG/ZG6iZj6bN5RKCU5pP+OZVZu1nn0=; b=PyAIlycMihtB9bgtdcmhngDcYx5oxTG4hCeXCrswcFkKNpD2q8o8+NLTYJ/HPDjFN+6wWV c6LBgXaZURSEcyFwHR47yF/VjcY0I1PfLHm8cCLEmdrk0BCNFuij0GPU3ikNmpGSyFncLm aMscmE23CbvNAu5neH1MjneqMFscLvTRf514vzdNloRp80t8m6zq0+/NZG4Qkg7J0N+FOS 5ak/L4I7cUr6/vv5W9SXH03jNKKI2OLwwfb6Ozzdu20tqNrLMUUDuAnNnxHnypeGAu5rxx JKEEbUy+L6yaJ3UZhJzUH537/Sx/1D4HMuVZ0lYxRjFk4ORpjvCffYau+KR3Ew== X-Migadu-Scanner: scn1.migadu.com Authentication-Results: aspmx1.migadu.com; dkim=fail ("headers rsa verify failed") header.d=lease-up.com header.s=2017 header.b=Ns7aWErx; dmarc=pass (policy=none) header.from=gnu.org; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -3.68 X-Spam-Score: -3.68 X-Migadu-Queue-Id: BEFFA3F143 X-TUID: LSW9EmQq+TQ6 Thanks to Bruno Victal "mirai" for cooperating on this patch and for generously sharing a wealth of insights about Guix services. Thanks to Jean-Baptiste Note for an early version of this service! Co-authored-by: Bruno Victal --- doc/guix.texi | 90 +++++++++++++++++ gnu/local.mk | 1 + gnu/services/linux.scm | 199 +++++++++++++++++++++++++++++++++++++- gnu/tests/cachefilesd.scm | 71 ++++++++++++++ 4 files changed, 360 insertions(+), 1 deletion(-) create mode 100644 gnu/tests/cachefilesd.scm diff --git a/doc/guix.texi b/doc/guix.texi index 09763d86ab..270b3f5958 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -115,6 +115,7 @@ Copyright @copyright{} 2023 Giacomo Leidi@* Copyright @copyright{} 2022 Antero Mejr@* Copyright @copyright{} 2023 Karl Hallsby Copyright @copyright{} 2023 Nathaniel Nicandro +Copyright @copyright{} 2023 Felix Lechner@* Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or @@ -37990,6 +37991,95 @@ parameters, can be done as follow: @end lisp @end defvar +@cindex cachefilesd +@cindex cachefiles +@cindex fscache +@subsubheading Cachefilesd Service + +The Cachefilesd service starts a daemon that caches network filesystem +data locally. It is especially useful for NFS and AFS shares, where it +reduces latencies for repeated access when reading files. + +The daemon can be started as follows: + +@lisp +(service cachefilesd-service-type + (cachefilesd-configuration + (cache-directory "/var/cache/fscache"))) +@end lisp + +@defvar cachefilesd-service-type +The service type for starting @command{cachefilesd}. The value for this +service type is a @code{cachefilesd-configuration}, whose only required +field is @var{cache-directory}. + +@end defvar + +@c %start of fragment +@deftp {Data Type} cachefilesd-configuration +Available @code{cachefilesd-configuration} fields are: + +@table @asis +@item @code{cachefilesd} (default: @code{cachefilesd}) (type: file-like) +The cachefilesd package to use. + +@item @code{debug-output?} (default: @code{#f}) (type: boolean) +Print debugging output to stderr. + +@item @code{use-syslog?} (default: @code{#t}) (type: boolean) +Log to syslog facility instead of stdout. + +@item @code{scan?} (default: @code{#t}) (type: boolean) +Scan for cachable objects. + +@item @code{cache-directory} (type: maybe-string) +Location of the cache directory. + +@item @code{cache-name} (default: @code{"CacheFiles"}) (type: maybe-string) +Name of cache (keep unique). + +@item @code{security-context} (type: maybe-string) +SELinux security context. + +@item @code{pause-culling-for-block-percentage} (default: @code{7}) (type: maybe-non-negative-integer) +Pause culling when available blocks exceed this percentage. + +@item @code{pause-culling-for-file-percentage} (default: @code{7}) (type: maybe-non-negative-integer) +Pause culling when available files exceed this percentage. + +@item @code{resume-culling-for-block-percentage} (default: @code{5}) (type: maybe-non-negative-integer) +Start culling when available blocks drop below this percentage. + +@item @code{resume-culling-for-file-percentage} (default: @code{5}) (type: maybe-non-negative-integer) +Start culling when available files drop below this percentage. + +@item @code{pause-caching-for-block-percentage} (default: @code{1}) (type: maybe-non-negative-integer) +Pause further allocations when available blocks drop below this +percentage. + +@item @code{pause-caching-for-file-percentage} (default: @code{1}) (type: maybe-non-negative-integer) +Pause further allocations when available files drop below this +percentage. + +@item @code{log2-table-size} (default: @code{12}) (type: maybe-non-negative-integer) +Size of tables holding cullable objects in logarithm of base 2. + +@item @code{cull?} (default: @code{#t}) (type: boolean) +Create free space by culling (consumes system load). + +@item @code{trace-function-entry-in-kernel-module?} (default: @code{#f}) (type: boolean) +Trace function entry in the kernel module (for debugging). + +@item @code{trace-function-exit-in-kernel-module?} (default: @code{#f}) (type: boolean) +Trace function exit in the kernel module (for debugging). + +@item @code{trace-internal-checkpoints-in-kernel-module?} (default: @code{#f}) (type: boolean) +Trace internal checkpoints in the kernel module (for debugging). + +@end table +@end deftp +@c %end of fragment + @cindex rasdaemon @cindex Platform Reliability, Availability and Serviceability daemon @subsubheading Rasdaemon Service diff --git a/gnu/local.mk b/gnu/local.mk index d31b40f3bf..a9e6f54088 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -765,6 +765,7 @@ GNU_SYSTEM_MODULES = \ %D%/tests.scm \ %D%/tests/audio.scm \ %D%/tests/base.scm \ + %D%/tests/cachefilesd.scm \ %D%/tests/ci.scm \ %D%/tests/cups.scm \ %D%/tests/databases.scm \ diff --git a/gnu/services/linux.scm b/gnu/services/linux.scm index d105c42850..e42ef1ae97 100644 --- a/gnu/services/linux.scm +++ b/gnu/services/linux.scm @@ -6,6 +6,7 @@ ;;; Copyright © 2021 B. Wilson ;;; Copyright © 2022 Josselin Poiret ;;; Copyright © 2023 Bruno Victal +;;; Copyright © 2023 Felix Lechner ;;; ;;; This file is part of GNU Guix. ;;; @@ -67,6 +68,28 @@ (define-module (gnu services linux) kernel-module-loader-service-type + cachefilesd-configuration + cachefilesd-configuration? + cachefilesd-configuration-cachefilesd + cachefilesd-configuration-debug-output? + cachefilesd-configuration-use-syslog? + cachefilesd-configuration-scan? + cachefilesd-configuration-cache-directory + cachefilesd-configuration-cache-name + cachefilesd-configuration-security-context + cachefilesd-configuration-pause-culling-for-block-percentage + cachefilesd-configuration-pause-culling-for-file-percentage + cachefilesd-configuration-resume-culling-for-block-percentage + cachefilesd-configuration-resume-culling-for-file-percentage + cachefilesd-configuration-pause-caching-for-block-percentage + cachefilesd-configuration-pause-caching-for-file-percentage + cachefilesd-configuration-log2-table-size + cachefilesd-configuration-cull? + cachefilesd-configuration-trace-function-entry-in-kernel-module + cachefilesd-configuration-trace-function-exit-in-kernel-module + cachefilesd-configuration-trace-internal-checkpoints-in-kernel-module + cachefilesd-service-type + rasdaemon-configuration rasdaemon-configuration? rasdaemon-configuration-record? @@ -306,6 +329,180 @@ (define kernel-module-loader-service-type (extend append) (default-value '()))) + +;;; +;;; Cachefilesd, an FS-Cache daemon +;;; + +(define (serialize-string variable-symbol value) + #~(format #f "~a ~a~%" #$(symbol->string variable-symbol) #$value)) + +(define-maybe string) + +(define (non-negative-integer? val) + (and (exact-integer? val) (not (negative? val)))) + +(define (serialize-non-negative-integer variable-symbol value) + #~(format #f "~a ~d~%" #$(symbol->string variable-symbol) #$value)) + +(define-maybe non-negative-integer) + +(define (make-option-serializer option-symbol) + (lambda (variable-symbol text) + (if (maybe-value-set? text) + #~(format #f "~a ~a~%" #$(symbol->string option-symbol) #$text) + ""))) + +(define (make-percentage-threshold-serializer threshold-symbol) + (lambda (variable-symbol percentage) + (if (maybe-value-set? percentage) + #~(format #f "~a ~a%~%" #$(symbol->string threshold-symbol) #$percentage) + ""))) + +(define-configuration cachefilesd-configuration + (cachefilesd + (file-like cachefilesd) + "The cachefilesd package to use." + (serializer empty-serializer)) + + ;; command-line options + (debug-output? + (boolean #f) + "Print debugging output to stderr." + (serializer empty-serializer)) + + (use-syslog? + (boolean #t) + "Log to syslog facility instead of stdout." + (serializer empty-serializer)) + + ;; culling is part of the configuration file + ;; despite the name of the command-line option + (scan? + (boolean #t) + "Scan for cachable objects." + (serializer empty-serializer)) + + ;; sole required field in the configuration file + (cache-directory + maybe-string + "Location of the cache directory." + (serializer (make-option-serializer 'dir))) + + (cache-name + (maybe-string "CacheFiles") + "Name of cache (keep unique)." + (serializer (make-option-serializer 'tag))) + + (security-context + maybe-string + "SELinux security context." + (serializer (make-option-serializer 'secctx))) + + ;; percentage thresholds in the configuration file + (pause-culling-for-block-percentage + (maybe-non-negative-integer 7) + "Pause culling when available blocks exceed this percentage." + (serializer (make-percentage-threshold-serializer 'brun))) + + (pause-culling-for-file-percentage + (maybe-non-negative-integer 7) + "Pause culling when available files exceed this percentage." + (serializer (make-percentage-threshold-serializer 'frun))) + + (resume-culling-for-block-percentage + (maybe-non-negative-integer 5) + "Start culling when available blocks drop below this percentage." + (serializer (make-percentage-threshold-serializer 'bcull))) + + (resume-culling-for-file-percentage + (maybe-non-negative-integer 5) + "Start culling when available files drop below this percentage." + (serializer (make-percentage-threshold-serializer 'fcull))) + + (pause-caching-for-block-percentage + (maybe-non-negative-integer 1) + "Pause further allocations when available blocks drop below this percentage." + (serializer (make-percentage-threshold-serializer 'bstop))) + + (pause-caching-for-file-percentage + (maybe-non-negative-integer 1) + "Pause further allocations when available files drop below this percentage." + (serializer (make-percentage-threshold-serializer 'fstop))) + + ;; run time optimizations in the configuration file + (log2-table-size + (maybe-non-negative-integer 12) + "Size of tables holding cullable objects in logarithm of base 2." + (serializer (make-option-serializer 'culltable))) + + (cull? + (boolean #t) + "Create free space by culling (consumes system load)." + (serializer + (lambda (variable-symbol value) + (if value "" "nocull\n")))) + + ;; kernel module debugging in the configuration file + (trace-function-entry-in-kernel-module? + (boolean #f) + "Trace function entry in the kernel module (for debugging)." + (serializer empty-serializer)) + + (trace-function-exit-in-kernel-module? + (boolean #f) + "Trace function exit in the kernel module (for debugging)." + (serializer empty-serializer)) + + (trace-internal-checkpoints-in-kernel-module? + (boolean #f) + "Trace internal checkpoints in the kernel module (for debugging)." + (serializer empty-serializer))) + +(define (serialize-cachefilesd-configuration configuration) + (mixed-text-file + "cachefilesd.conf" + (serialize-configuration configuration cachefilesd-configuration-fields))) + +(define (cachefilesd-shepherd-service config) + "Return a list of for cachefilesd for CONFIG." + (match-record + config (cachefilesd + debug-output? + use-syslog? + scan? + cache-directory) + (let ((configuration-file (serialize-cachefilesd-configuration config))) + (shepherd-service + (documentation "Run the cachefilesd daemon for FS-Cache.") + (provision '(cachefilesd)) + (requirement (append '(file-systems) + (if use-syslog? '(syslogd) '()))) + (start #~(begin + (and=> #$(maybe-value cache-directory) mkdir-p) + (make-forkexec-constructor + `(#$(file-append cachefilesd "/sbin/cachefilesd") + ;; do not detach + "-n" + #$@(if debug-output? '("-d") '()) + #$@(if use-syslog? '() '("-s")) + #$@(if scan? '() '("-N")) + "-f" #$configuration-file)))) + (stop #~(make-kill-destructor)))))) + +(define cachefilesd-service-type + (service-type + (name 'cachefilesd) + (description + "Run the FS-Cache backend daemon @command{cachefilesd}.") + (extensions + (list + (service-extension kernel-module-loader-service-type + (const '("cachefiles"))) + (service-extension shepherd-root-service-type + (compose list cachefilesd-shepherd-service)))) + (default-value (cachefilesd-configuration)))) + ;;; ;;; Reliability, Availability, and Serviceability (RAS) daemon @@ -351,7 +548,7 @@ (define rasdaemon-service-type ;;; -;;; Kernel module loader. +;;; Zram device ;;; (define-record-type* diff --git a/gnu/tests/cachefilesd.scm b/gnu/tests/cachefilesd.scm new file mode 100644 index 0000000000..7f5d513067 --- /dev/null +++ b/gnu/tests/cachefilesd.scm @@ -0,0 +1,71 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2017 Peter Mikkelsen +;;; Copyright © 2022 Bruno Victal +;;; Copyright © 2023 Felix Lechner +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu tests cachefilesd) + #:use-module (gnu tests) + #:use-module (gnu system) + #:use-module (gnu system vm) + #:use-module (gnu services) + #:use-module (gnu services linux) + #:use-module (guix gexp) + #:export (%test-cachefilesd)) + +(define %cachefilesd-os + (simple-operating-system + (service cachefilesd-service-type + (cachefilesd-configuration + (cache-directory "/var/cache/fscache"))))) + +(define (run-cachefilesd-test) + "Run tests in %cachefilesd-os, which has cachefilesd running." + (define os + (marionette-operating-system + %cachefilesd-os + #:imported-modules '((gnu services herd)))) + + (define vm + (virtual-machine os)) + + (define test + (with-imported-modules '((gnu build marionette)) + #~(begin + (use-modules (srfi srfi-64) + (gnu build marionette)) + (define marionette + (make-marionette (list #$vm))) + + (test-runner-current (system-test-runner #$output)) + (test-begin "cachefilesd") + + (test-assert "service is running" + (marionette-eval + '(begin + (use-modules (gnu services herd)) + (start-service 'cachefilesd)) + marionette)) + + (test-end)))) + (gexp->derivation "cachefilesd-test" test)) + +(define %test-cachefilesd + (system-test + (name "cachefilesd") + (description "Test that the cachefilesd runs when started.") + (value (run-cachefilesd-test)))) -- 2.40.1