[bug#31911] [PATCH] services: Add prometheus-node-exporter-service-type.

[bug#31911] [PATCH] services: Add prometheus-node-exporter-service-type.

[Gábor Boskovits]

* gnu/services/monitoring.scm (prometheus-node-exporter-service-type):
New variable.
(<prometheus-node-exporter-configuration>): New record type.
(prometheus-node-exporter-shepherd-service): New procedure.
* gnu/doc/guix.texi (Monitoring Services): Document it.
---
 doc/guix.texi               | 32 ++++++++++++++++++++++++++++++++
 gnu/services/monitoring.scm | 37 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 68 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 1ecb11002..6a649c549 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -15570,6 +15570,38 @@ Specify the path of the base URL.  This can be useful if
 @end table
 @end deftp
 
+@subsubheading Prometheus Node Exporter Service
+@cindex prometheus-node-exporter
+Prometheus node exporter is a Prometheus exporter. It makes hardware and
+operating system statistics provided by *NIX kernels available for the
+Prometheus monitoring system. This service should be deployed on all
+physical nodes and virtual machines, where monitoring these statistics is
+desirable.
+
+@defvar {Scheme variable} prometheus-node-exporter-service-type
+This is the service type for the
+@uref{https://github.com/prometheus/node_exporter/, prometheus-node-exporter}
+service, its value must be a @code{prometheus-node-exporter-configuration}
+record as in this example:
+
+@example
+(service prometheus-node-exporter-service-type
+         (prometheus-node-exporter-configuration
+           (web-listen-address ":9100")))
+@end example
+@end defvar
+
+@deftp {Data Type} prometheus-node-exporter-configuration
+Data type representing the configuration of @command{node_exporter}.
+
+@table @asis
+@item @code{package} (default: @code{go-github-com-prometheus-node-exporter})
+The prometheus-node-exporter package to use.
+@item @code{web-listen-address} (default: @code{":9100"})
+Bind the web interface to the specified address.
+
+@end table
+@end deftp
 
 @node Kerberos Services
 @subsubsection Kerberos Services
diff --git a/gnu/services/monitoring.scm b/gnu/services/monitoring.scm
index 49a65db4b..2fc90c867 100644
--- a/gnu/services/monitoring.scm
+++ b/gnu/services/monitoring.scm
@@ -26,7 +26,9 @@
   #:use-module (guix records)
   #:use-module (ice-9 match)
   #:export (darkstat-configuration
-            darkstat-service-type))
+            prometheus-node-exporter-configuration
+            darkstat-service-type
+            prometheus-node-exporter-service-type))
 
 \f
 ;;;
@@ -89,3 +91,36 @@ HTTP.")
                              (const %darkstat-accounts))
           (service-extension shepherd-root-service-type
                              (compose list darkstat-shepherd-service))))))
+
+(define-record-type* <prometheus-node-exporter-configuration>
+  prometheus-node-exporter-configuration
+  make-prometheus-node-exporter-configuration
+  prometheus-node-exporter-configuration?
+  (package prometheus-node-exporter-configuration-package
+           (default go-github-com-prometheus-node-exporter))
+  (web-listen-address prometheus-node-exporter-web-listen-address
+                      (default ":9100")))
+
+(define prometheus-node-exporter-shepherd-service
+  (match-lambda
+    (( $ <prometheus-node-exporter-configuration>
+         package web-listen-address)
+     (shepherd-service
+      (documentation "Prometheus node exporter.")
+      (provision '(prometheus-node-exporter))
+      (requirement '(networking))
+      (start #~(make-forkexec-constructor
+                (list #$(file-append package "/bin/node_exporter")
+                      "--web.listen-address" #$web-listen-address)))
+      (stop #~(make-kill-destructor))))))
+
+(define prometheus-node-exporter-service-type
+  (service-type
+   (name 'prometheus-node-exporter)
+   (description
+    "Run @command{node_exporter} to serve hardware and OS metrics to
+prometheus.")
+   (extensions
+    (list (service-extension
+           shepherd-root-service-type
+           (compose list prometheus-node-exporter-shepherd-service))))))
-- 
2.17.1

[bug#31911] [PATCH] services: Add prometheus-node-exporter-service-type.

[Ludovic Courtès]

Hello Gábor!

Gábor Boskovits <boskovits@gmail.com> skribis:

> * gnu/services/monitoring.scm (prometheus-node-exporter-service-type):
> New variable.
> (<prometheus-node-exporter-configuration>): New record type.
> (prometheus-node-exporter-shepherd-service): New procedure.
> * gnu/doc/guix.texi (Monitoring Services): Document it.

That’s very useful!  Hopefully we can start using it on our build farm,
though we’ll need Grafena (?) or something to visualize those stats,
right?  OTOH apparently it already provides a web interface, so…?

It would be nice to add a system test for this service.  It could ensure
that representative URLs return 200 or 404, for instance (see the
hpcguix-web test in (gnu tests web) as an example.)

WDYT?

Minor comments:

> +@subsubheading Prometheus Node Exporter Service

Leave a newline here.

> +@cindex prometheus-node-exporter
> +Prometheus node exporter is a Prometheus exporter. It makes hardware and
> +operating system statistics provided by *NIX kernels available for the

The first sentence looks like a tautology.  :-)
I’m also unconvinced by the “*NIX” notation.

What about:

  The Prometheus ``node exporter'' makes hardware and operating system
  statistics available for the…

Please leave two spaces after end-of-sentence periods.

> +@defvar {Scheme variable} prometheus-node-exporter-service-type

Should be @defvr instead of @defvar.

> +@deftp {Data Type} prometheus-node-exporter-configuration
> +Data type representing the configuration of @command{node_exporter}.
> +
> +@table @asis
> +@item @code{package} (default: @code{go-github-com-prometheus-node-exporter})
> +The prometheus-node-exporter package to use.
> +@item @code{web-listen-address} (default: @code{":9100"})
> +Bind the web interface to the specified address.

Please add a newline before the second @item.

With these changes and ideally a simple test, this is ready to go IMO!

Thanks,
Ludo’.

[bug#31911] [PATCH] services: Add prometheus-node-exporter-service-type.

[Gábor Boskovits]

[-- Attachment #1: Type: text/plain, Size: 2671 bytes --]

Ludovic Courtès <ludo@gnu.org> ezt írta (időpont: 2018. jún. 23., Szo,
23:51):

> Hello Gábor!
>
> Gábor Boskovits <boskovits@gmail.com> skribis:
>
> > * gnu/services/monitoring.scm (prometheus-node-exporter-service-type):
> > New variable.
> > (<prometheus-node-exporter-configuration>): New record type.
> > (prometheus-node-exporter-shepherd-service): New procedure.
> > * gnu/doc/guix.texi (Monitoring Services): Document it.
>
> That’s very useful!  Hopefully we can start using it on our build farm,
> though we’ll need Grafena (?) or something to visualize those stats,
> right?  OTOH apparently it already provides a web interface, so…?
>
>
This is actually only one half of the solution, this provides an endpoint
for
the prometheus server to scrape. I've not yet packaged the server part.
The server has the needed visualization capabilities.


> It would be nice to add a system test for this service.  It could ensure
> that representative URLs return 200 or 404, for instance (see the
> hpcguix-web test in (gnu tests web) as an example.)
>
> WDYT?
>
>
Ok, will do.


> Minor comments:
>
> > +@subsubheading Prometheus Node Exporter Service
>
> Leave a newline here.
>
> > +@cindex prometheus-node-exporter
> > +Prometheus node exporter is a Prometheus exporter. It makes hardware and
> > +operating system statistics provided by *NIX kernels available for the
>
> The first sentence looks like a tautology.  :-)
> I’m also unconvinced by the “*NIX” notation.
>
> What about:
>
>   The Prometheus ``node exporter'' makes hardware and operating system
>   statistics available for the…
>
> Please leave two spaces after end-of-sentence periods.
>
> > +@defvar {Scheme variable} prometheus-node-exporter-service-type
>
> Should be @defvr instead of @defvar.

> +@deftp {Data Type} prometheus-node-exporter-configuration
> > +Data type representing the configuration of @command{node_exporter}.
> > +
> > +@table @asis
> > +@item @code{package} (default:
> @code{go-github-com-prometheus-node-exporter})
> > +The prometheus-node-exporter package to use.
> > +@item @code{web-listen-address} (default: @code{":9100"})
> > +Bind the web interface to the specified address.
>
> Please add a newline before the second @item.
>
>
Most of these were copied almost verbatim form darkstat service. Maybe
these modification should be also applied there.
Will send an updated patch. Should I contact Sou Bunnbu regadring the
darkstat documentation, or should I fix it?


> With these changes and ideally a simple test, this is ready to go IMO!
>
> Thanks,
> Ludo’.
>

[-- Attachment #2: Type: text/html, Size: 3749 bytes --]

[bug#31911] [PATCH] services: Add prometheus-node-exporter-service-type.

[Ludovic Courtès]

Hello,

Gábor Boskovits <boskovits@gmail.com> skribis:

> Ludovic Courtès <ludo@gnu.org> ezt írta (időpont: 2018. jún. 23., Szo,
> 23:51):

[...]

>> That’s very useful!  Hopefully we can start using it on our build farm,
>> though we’ll need Grafena (?) or something to visualize those stats,
>> right?  OTOH apparently it already provides a web interface, so…?
>>
>>
> This is actually only one half of the solution, this provides an endpoint
> for
> the prometheus server to scrape. I've not yet packaged the server part.
> The server has the needed visualization capabilities.

OK, I see.

>> +@deftp {Data Type} prometheus-node-exporter-configuration
>> > +Data type representing the configuration of @command{node_exporter}.
>> > +
>> > +@table @asis
>> > +@item @code{package} (default:
>> @code{go-github-com-prometheus-node-exporter})
>> > +The prometheus-node-exporter package to use.
>> > +@item @code{web-listen-address} (default: @code{":9100"})
>> > +Bind the web interface to the specified address.
>>
>> Please add a newline before the second @item.
>>
>>
> Most of these were copied almost verbatim form darkstat service. Maybe
> these modification should be also applied there.
> Will send an updated patch. Should I contact Sou Bunnbu regadring the
> darkstat documentation, or should I fix it?

You can go ahead and fix them as a separate commit.

Thanks,
Ludo’.

[bug#31911] [PATCH] services: Add prometheus-node-exporter-service-type.

[Ludovic Courtès]

Ping!  :-)

ludo@gnu.org (Ludovic Courtès) skribis:

> Hello,
>
> Gábor Boskovits <boskovits@gmail.com> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> ezt írta (időpont: 2018. jún. 23., Szo,
>> 23:51):
>
> [...]
>
>>> That’s very useful!  Hopefully we can start using it on our build farm,
>>> though we’ll need Grafena (?) or something to visualize those stats,
>>> right?  OTOH apparently it already provides a web interface, so…?
>>>
>>>
>> This is actually only one half of the solution, this provides an endpoint
>> for
>> the prometheus server to scrape. I've not yet packaged the server part.
>> The server has the needed visualization capabilities.
>
> OK, I see.
>
>>> +@deftp {Data Type} prometheus-node-exporter-configuration
>>> > +Data type representing the configuration of @command{node_exporter}.
>>> > +
>>> > +@table @asis
>>> > +@item @code{package} (default:
>>> @code{go-github-com-prometheus-node-exporter})
>>> > +The prometheus-node-exporter package to use.
>>> > +@item @code{web-listen-address} (default: @code{":9100"})
>>> > +Bind the web interface to the specified address.
>>>
>>> Please add a newline before the second @item.
>>>
>>>
>> Most of these were copied almost verbatim form darkstat service. Maybe
>> these modification should be also applied there.
>> Will send an updated patch. Should I contact Sou Bunnbu regadring the
>> darkstat documentation, or should I fix it?
>
> You can go ahead and fix them as a separate commit.
>
> Thanks,
> Ludo’.

[bug#31911] [PATCH] services: Add prometheus-node-exporter-service-type.

[Gábor Boskovits]

[-- Attachment #1: Type: text/plain, Size: 1797 bytes --]

Ludovic Courtès <ludo@gnu.org> ezt írta (időpont: 2018. júl. 7., Szo,
17:54):

> Ping!  :-)
>
>
Will find some time tomorrow to have a look at this :-)



> ludo@gnu.org (Ludovic Courtès) skribis:
>
> > Hello,
> >
> > Gábor Boskovits <boskovits@gmail.com> skribis:
> >
> >> Ludovic Courtès <ludo@gnu.org> ezt írta (időpont: 2018. jún. 23., Szo,
> >> 23:51):
> >
> > [...]
> >
> >>> That’s very useful!  Hopefully we can start using it on our build farm,
> >>> though we’ll need Grafena (?) or something to visualize those stats,
> >>> right?  OTOH apparently it already provides a web interface, so…?
> >>>
> >>>
> >> This is actually only one half of the solution, this provides an
> endpoint
> >> for
> >> the prometheus server to scrape. I've not yet packaged the server part.
> >> The server has the needed visualization capabilities.
> >
> > OK, I see.
> >
> >>> +@deftp {Data Type} prometheus-node-exporter-configuration
> >>> > +Data type representing the configuration of @command{node_exporter}.
> >>> > +
> >>> > +@table @asis
> >>> > +@item @code{package} (default:
> >>> @code{go-github-com-prometheus-node-exporter})
> >>> > +The prometheus-node-exporter package to use.
> >>> > +@item @code{web-listen-address} (default: @code{":9100"})
> >>> > +Bind the web interface to the specified address.
> >>>
> >>> Please add a newline before the second @item.
> >>>
> >>>
> >> Most of these were copied almost verbatim form darkstat service. Maybe
> >> these modification should be also applied there.
> >> Will send an updated patch. Should I contact Sou Bunnbu regadring the
> >> darkstat documentation, or should I fix it?
> >
> > You can go ahead and fix them as a separate commit.
> >
> > Thanks,
> > Ludo’.
>

[-- Attachment #2: Type: text/html, Size: 2748 bytes --]

bug#31911: services: Add prometheus-node-exporter-service-type.

[Gábor Boskovits]

[-- Attachment #1: Type: text/plain, Size: 62 bytes --]

Pushed to master as a33652ee336ae9a5d2ab5fd54bf2397caec42a0e.

[-- Attachment #2: Type: text/html, Size: 299 bytes --]

[bug#31911] services: Add prometheus-node-exporter-service-type.

[Clément Lassieur]

Hi Gábor,

Gábor Boskovits <boskovits@gmail.com> writes:

> Pushed to master as a33652ee336ae9a5d2ab5fd54bf2397caec42a0e.

> +          (test-assert "prometheus-node-exporter running"
> +            (marionette-eval
> +             '(begin
> +                (use-modules (gnu services herd))
> +                (match (start-service 'prometheus-node-exporter)
> +                  (#f #f)
> +                  (('service response-parts ...)
> +                   (match (assq-ref response-parts 'running)
> +                     ((pid) (number? pid))))))
> +             marionette))

The PID check is useless because START-SERVICE will return #f if the
service fails to start.  Instead, I'd use:

          (test-assert "prometheus-node-exporter running"
            (marionette-eval
             '(begin
                (use-modules (gnu services herd))
                (start-service 'prometheus-node-exporter))
             marionette))

This would also make the test more robust to service changes (e.g. it
would still work if MAKE-FORKEXEC-CONSTRUCTOR is removed).

Clément

[bug#31911] services: Add prometheus-node-exporter-service-type.

[Gábor Boskovits]

[-- Attachment #1: Type: text/plain, Size: 1403 bytes --]

Clément Lassieur <clement@lassieur.org> ezt írta (időpont: 2018. júl. 9.,
H, 11:17):

> Hi Gábor,
>
> Gábor Boskovits <boskovits@gmail.com> writes:
>
> > Pushed to master as a33652ee336ae9a5d2ab5fd54bf2397caec42a0e.
>
> > +          (test-assert "prometheus-node-exporter running"
> > +            (marionette-eval
> > +             '(begin
> > +                (use-modules (gnu services herd))
> > +                (match (start-service 'prometheus-node-exporter)
> > +                  (#f #f)
> > +                  (('service response-parts ...)
> > +                   (match (assq-ref response-parts 'running)
> > +                     ((pid) (number? pid))))))
> > +             marionette))
>
> The PID check is useless because START-SERVICE will return #f if the
> service fails to start.  Instead, I'd use:
>
>           (test-assert "prometheus-node-exporter running"
>             (marionette-eval
>              '(begin
>                 (use-modules (gnu services herd))
>                 (start-service 'prometheus-node-exporter))
>              marionette))
>
> This would also make the test more robust to service changes (e.g. it
> would still work if MAKE-FORKEXEC-CONSTRUCTOR is removed).
>
>
Thanks, I will adjust accordingly. Incidentally the code is almost the same
as in hpcguix-web test. Should we also adjust that?


> Clément
>

[-- Attachment #2: Type: text/html, Size: 2116 bytes --]

[bug#31911] services: Add prometheus-node-exporter-service-type.

[Clément Lassieur]

Gábor Boskovits <boskovits@gmail.com> writes:

> Clément Lassieur <clement@lassieur.org> ezt írta (időpont: 2018. júl. 9.,
> H, 11:17):
>
>> Hi Gábor,
>>
>> Gábor Boskovits <boskovits@gmail.com> writes:
>>
>> > Pushed to master as a33652ee336ae9a5d2ab5fd54bf2397caec42a0e.
>>
>> > +          (test-assert "prometheus-node-exporter running"
>> > +            (marionette-eval
>> > +             '(begin
>> > +                (use-modules (gnu services herd))
>> > +                (match (start-service 'prometheus-node-exporter)
>> > +                  (#f #f)
>> > +                  (('service response-parts ...)
>> > +                   (match (assq-ref response-parts 'running)
>> > +                     ((pid) (number? pid))))))
>> > +             marionette))
>>
>> The PID check is useless because START-SERVICE will return #f if the
>> service fails to start.  Instead, I'd use:
>>
>>           (test-assert "prometheus-node-exporter running"
>>             (marionette-eval
>>              '(begin
>>                 (use-modules (gnu services herd))
>>                 (start-service 'prometheus-node-exporter))
>>              marionette))
>>
>> This would also make the test more robust to service changes (e.g. it
>> would still work if MAKE-FORKEXEC-CONSTRUCTOR is removed).
>>
>>
> Thanks, I will adjust accordingly. Incidentally the code is almost the same
> as in hpcguix-web test. Should we also adjust that?

You're welcome :-)  It's not urgent, but in the long term, it would be
good that all similar code is ajusted accordingly.  It's in lots of
other places as well if I remember well.