From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id cE40HO5gMWRycQEASxT56A (envelope-from ) for ; Sat, 08 Apr 2023 14:41:18 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id SEUGHO5gMWSJXwAAauVa8A (envelope-from ) for ; Sat, 08 Apr 2023 14:41:18 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 15F87D10D for ; Sat, 8 Apr 2023 14:41:17 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pl7sO-0001je-8F; Sat, 08 Apr 2023 08:41:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pl7sM-0001jB-Q4 for guix-patches@gnu.org; Sat, 08 Apr 2023 08:41:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pl7sM-00071T-IQ for guix-patches@gnu.org; Sat, 08 Apr 2023 08:41:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pl7sM-00081i-08 for guix-patches@gnu.org; Sat, 08 Apr 2023 08:41:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#60788] [PATCH] services: Add vnstat-service-type. Resent-From: Bruno Victal Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 08 Apr 2023 12:41:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 60788 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: maxim.cournoyer@gmail.com, 60788@debbugs.gnu.org Received: via spool by 60788-submit@debbugs.gnu.org id=B60788.168095765630835 (code B ref 60788); Sat, 08 Apr 2023 12:41:01 +0000 Received: (at 60788) by debbugs.gnu.org; 8 Apr 2023 12:40:56 +0000 Received: from localhost ([127.0.0.1]:57977 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pl7sF-00081H-NW for submit@debbugs.gnu.org; Sat, 08 Apr 2023 08:40:56 -0400 Received: from smtpm3.myservices.hosting ([185.26.105.234]:36422) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pl7sD-000818-IY for 60788@debbugs.gnu.org; Sat, 08 Apr 2023 08:40:54 -0400 Received: from mail1.netim.hosting (unknown [185.26.106.173]) by smtpm3.myservices.hosting (Postfix) with ESMTP id 47FC120F4F; Sat, 8 Apr 2023 14:40:52 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mail1.netim.hosting (Postfix) with ESMTP id D30F68009F; Sat, 8 Apr 2023 14:40:48 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mail1.netim.hosting Received: from mail1.netim.hosting ([127.0.0.1]) by localhost (mail1-2.netim.hosting [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Elt0UqAbUdtr; Sat, 8 Apr 2023 14:40:48 +0200 (CEST) Received: from [192.168.1.239] (unknown [10.192.1.83]) (Authenticated sender: lumen@makinata.eu) by mail1.netim.hosting (Postfix) with ESMTPSA id 1AE728009D; Sat, 8 Apr 2023 14:40:48 +0200 (CEST) Message-ID: <07feadac-18b2-7e82-9798-3e3872efcd4a@makinata.eu> Date: Sat, 8 Apr 2023 13:40:39 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.9.1 Content-Language: en-US References: <95b646eb6b23dec213cba43b6e4e7ddc4a601d0f.1673640404.git.mirai@makinata.eu> <87edovd8em.fsf_-_@gnu.org> From: Bruno Victal In-Reply-To: <87edovd8em.fsf_-_@gnu.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Seal: i=1; s=key1; d=yhetil.org; t=1680957678; a=rsa-sha256; cv=none; b=UsNKqOYVIG/kKg1aby59BvewYUsM6l4VfnilSyCf4CB1tgcYhw5TYkjXEoWGQxSOmLE1cG 1jz5TzgAfuhbF4v6FkkY5UxAsnF7Bg3nfHNNCl8hMn/WHpoRSKgBtaIcBT0JLRrDfTtEB/ mk239Xz73PoazUQOhnCaTlhRPuAvBhXXkKJwOuvdrmSDhNZrdYQLCUglzeMRfiIwG9lXrO +v2rxWOT3PZP4odiIYePU7fP6avU/EvaMmOkehdIkQjFdFSNkuAvS5bvM7qk/97drmE5Rz JA6E6CWk2zpKE6AzfR4HDhcGRR+W5JhZSD3+DFU7JXcKG4OXkq1+CLJOMpgowQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1680957678; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post; bh=WMf7Lih38SOVbIzmTEWYVcaNp/7ezPtCs5zoB3Akwzw=; b=tXtVRnFc6Szi2B2z/l2jajt/uQoSRpMrUb0qNLvIH2cBJHmDPn7Wz4dSf1l/VPTM93pQlb k7kEzejfdyyW5wkJzkzZ12T7cqSBtEpTXUQYqtjqxo9MbFbSx5jOQL/D0LdAyaZDdIJHdi ZGoTfSI8YWxxBWWjBXsXdGE6FMHt9ESy75I1WRzVgPvEQL/RYGNghbLudkXYyGCeRR1Fx4 YjKxadqjsq3co6H5vIG/tGB9rzyTrkBL8VDaEuPeiLr52jU+ps3RZvKW4pwdyCpaMv+tks 188OClIGNeE1P7rpFJH2NFJOadFZX7sqxG8T6yjO78SA5T2+vM52WNhwIDkmyA== Authentication-Results: aspmx1.migadu.com; dkim=none; dmarc=none; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org" X-Migadu-Scanner: scn0.migadu.com X-Migadu-Spam-Score: -4.52 X-Spam-Score: -4.52 X-Migadu-Queue-Id: 15F87D10D X-TUID: I4nAGrsEXDGI Hi Ludo’, On 2023-04-07 16:22, Ludovic Courtès wrote: > I think a system test would be nice, we generally require it upfront, > but since Maxim wrote it can come later, let’s not let it block this > patch any longer. Originally I thought it was unfeasible to write a test for this service since it required network activity within the VM and it seemed to take more than 10 minutes for it to pick it up. I did some manual experiments on it much later and I managed to get it down to approx. between 2 and 3 minutes for a partially automated test. What's blocking the test from being implemented in Guix is that my (incomplete) test-suite for this depends on guile 'spawn', which isn't available yet, For reference, I've attached the test-suite here [1], the plan is to finish it up and add it to Guix after the 'spawn' issue is resolved. (or perhaps refactor this to use another approach?) > > One comment: > >> +@item @code{database-dir} (default: @code{"/var/lib/vnstat"}) (type: string) > > [...] > >> +@item @code{create-dirs?} (default: @code{#t}) (type: maybe-boolean) > > For consistency, both within this record and with the rest of Guix, I > suggest avoiding abbreviations. Since this will be part of the API, > better fix it now than later. I should mention that almost all of the field names here are near verbatim vnstat config-file directives, i.e. a near 1-1 Scheme translation of vnstat config. This has the benefit that it makes serialization pretty much straightforward. It's possible to override their names by the use of the custom serializer parameter but would it be acceptable to leave them as-is? >> +@item @code{max-bandwidth} (type: maybe-integer) >> +Maximum bandwidth for all interfaces. If the interface specific traffic >> +exceeds the given value then the data is assumed to be invalid and >> +rejected. Set to 0 in order to disable the feature. Value range: >> +@samp{0}..@samp{50000} >> + >> +@item @code{max-bw} (type: maybe-alist) >> +Same as @var{max-bandwidth} but can be used for setting individual >> +limits for selected interfaces. This is an association list of >> +interfaces as symbols/strings to integer values. For example, >> +@lisp >> +(max-bw `((eth0 . 15000) >> + (ppp0 . 10000))) >> +@end lisp > > Both the naming and semantics are a bit confusing to me. > > How about s/max-bw/per-interface-max-bandwidth/ ? I found it a bit confusing as well but I'm not too familiar with this part of the config to comment about it. I lifted most of the field-names and documentations straight from the manpage. > Side note: I’d represent interfaces as strings because there’s no > guarantee they “fit” in a symbol. Thanks! I'll have this amended in the next revision. [1]: Listing of vnstat-test.scm --8<---------------cut here---------------start------------->8--- (define-module (gnu tests vnstat) #:use-module (gnu tests) #:use-module ((gnu packages networking) #:select (socat vnstat)) #:use-module (gnu services) #:use-module (gnu services networking) #:use-module (gnu services monitoring) #:use-module (gnu system) #:use-module (gnu system vm) #:use-module (guix gexp) #:use-module (ice-9 format) #:export (%test-vnstat)) (define (run-vnstat-test) "Run tests in a vm which has vnstat running." (define vnstat-config (vnstat-configuration (max-bandwidth 0) (time-sync-wait 0) (bandwidth-detection-interval 0))) (define os (marionette-operating-system (simple-operating-system (service dhcp-client-service-type) (service vnstat-service-type vnstat-config) (service inetd-service-type (inetd-configuration (entries (list (inetd-entry (name "discard") (socket-type 'stream) (protocol "tcp") ;; FIXME: originally this was UDP but port-forwardings hardcodes TCP (wait? #t) (user "nobody"))))))) #:imported-modules '((gnu services herd)))) (define forwarded-port 9999) (define vm (virtual-machine (operating-system os) ;; Note: port 9 corresponds to "discard" service. (port-forwardings `((,forwarded-port . 9))))) ;; FIXME: Allow UDP forward. (define test-timeout (* 60 2)) ; wait for 2 minutes tops. (define test (with-imported-modules '((gnu build marionette)) #~(begin (use-modules (gnu build marionette) (srfi srfi-64)) (let ((marionette (make-marionette (list #$vm))) (pid-file #$(vnstat-configuration-pid-file vnstat-config))) (test-runner-current (system-test-runner #$output)) (test-begin "vnstat") (test-assert "service is running" (marionette-eval '(begin (use-modules (gnu services herd)) (start-service 'vnstatd)) marionette)) (test-assert "vnstatd ready" (wait-for-file pid-file marionette)) (test-assert "vnstatd is logging" ;; pump garbage into the "discard" service within the vm ;; TODO: guile socket client instead? Is it feasible? (let* ((socat #$(file-append socat "/bin/socat")) (dest-addr #$(format #f "TCP4:localhost:~d" forwarded-port)) (args `("socat" "-u" "/dev/zero" ,dest-addr)) ;; XXX: Guile bug (22/03/2023, Guile 3.0.9) ;; Fixed in main: #;(output-port (%make-void-port "w")) (garbage-pump-pid (spawn socat args))) (let ((retval (marionette-eval '(begin (use-modules (ice-9 popen) ;(ice-9 rdelim) (ice-9 match) (sxml simple) (sxml xpath)) (define selector (let ((xpath '(vnstat interface traffic total))) (compose (node-pos 1) (sxpath xpath)))) (let loop ((i 0)) (let* ((vnstat #$(file-append vnstat "/bin/vnstat")) (query-cmd (format #f "~a --xml") vnstat) (result #;(call-with-port (open-input-pipe query-cmd) read-line) (call-with-port (open-input-pipe query-cmd) xml->sxml)) (iface-stats (selector result))) (match iface-stats ((('total ('rx "0") ('tx "0"))) (sleep 1) (if (< i #$test-timeout) (loop (+ i 1)) #f)) ((('total ('rx rx) ('tx tx))) #t) (_ #f))))) marionette))) ;; shutdown garbage pump (kill garbage-pump-pid SIGTERM) retval))) (test-end))))) (gexp->derivation "vnstat-test" test)) (define %test-vnstat (system-test (name "vnstat") (description "Basic tests for vnstat service.") (value (run-vnstat-test)))) --8<---------------cut here---------------end--------------->8--- Cheers, Bruno