[bug#54659] [PATCH 0/2] Periodically delete build logs

[bug#54659] [PATCH 0/2] Periodically delete build logs

[Ludovic Courtès]

Hi!

We’ve been accumulating cruft in /var/guix/log/drvs for too long,
time to do something about it!

Thoughts?

Ludo’.

Ludovic Courtès (2):
  services: Add 'log-cleanup-service-type'.
  services: Add 'log-cleanup' service to '%base-services' for build
    logs.

 doc/guix.texi          | 39 +++++++++++++++++++++++++++++++
 gnu/services/admin.scm | 53 +++++++++++++++++++++++++++++++++++++++++-
 gnu/services/base.scm  |  6 +++++
 3 files changed, 97 insertions(+), 1 deletion(-)


base-commit: 53b04339fe521f486d3017930a419d5ca8a6cffd
-- 
2.34.0

[bug#54659] [PATCH 1/2] services: Add 'log-cleanup-service-type'.

[Ludovic Courtès]

* gnu/services/admin.scm (<log-cleanup-configuration>): New record
type.
(log-cleanup-program, log-cleanup-mcron-jobs): New procedures.
(log-cleanup-service-type): New variable.
* doc/guix.texi (Log Rotation): Document it.
---
 doc/guix.texi          | 28 ++++++++++++++++++++++
 gnu/services/admin.scm | 53 +++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 80 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index e8ef4286be..ad2763ec8a 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -17641,6 +17641,34 @@ The list of syslog-controlled files to be rotated.  By default it is:
 "/var/log/maillog")}.
 @end defvr
 
+Some log files just need to be deleted periodically once they are old,
+without any other criterion and without any archival step.  This is the
+case of build logs stored by @command{guix-daemon} under
+@file{/var/log/guix/drvs} (@pxref{Invoking guix-daemon}).  The
+@code{log-cleanup} service addresses this use case.
+
+@defvr {Scheme Variable} log-cleanup-service-type
+This is the type of the service to delete old logs.  Its value must be a
+@code{log-cleanup-configuration} record as described below.
+@end defvr
+
+@deftp {Data Type} log-cleanup-configuration
+Data type representing the log cleanup configuration
+
+@table @asis
+@item @code{directory}
+Name of the directory containing log files.
+
+@item @code{expiry} (default: @code{(* 6 30 24 3600)})
+Age in seconds after which a file is subject to deletion (six months by
+default).
+
+@item @code{schedule} (default: @code{"30 12 01,08,15,22 * *"})
+String or gexp denoting the corresponding mcron job schedule
+(@pxref{Scheduled Job Execution}).
+@end table
+@end deftp
+
 @node Networking Setup
 @subsection Networking Setup
 
diff --git a/gnu/services/admin.scm b/gnu/services/admin.scm
index 043517262f..3096acdf5a 100644
--- a/gnu/services/admin.scm
+++ b/gnu/services/admin.scm
@@ -1,6 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2016 Jan Nieuwenhuizen <janneke@gnu.org>
-;;; Copyright © 2016, 2017, 2018, 2019, 2020, 2021 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2016-2022 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2020 Brice Waegeneire <brice@waegenei.re>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -46,6 +46,13 @@ (define-module (gnu services admin)
             rottlog-service
             rottlog-service-type
 
+            log-cleanup-service-type
+            log-cleanup-configuration
+            log-cleanup-configuration?
+            log-cleanup-configuration-directory
+            log-cleanup-configuration-expiry
+            log-cleanup-configuration-schedule
+
             unattended-upgrade-service-type
             unattended-upgrade-configuration
             unattended-upgrade-configuration?
@@ -191,6 +198,50 @@ (define rottlog-service-type
                                  rotations)))))
    (default-value (rottlog-configuration))))
 
+\f
+;;;
+;;; Build log removal.
+;;;
+
+(define-record-type* <log-cleanup-configuration>
+  log-cleanup-configuration make-log-cleanup-configuration
+  log-cleanup-configuration?
+  (directory log-cleanup-configuration-directory) ;string
+  (expiry    log-cleanup-configuration-expiry     ;integer (seconds)
+             (default (* 6 30 24 3600)))
+  (schedule  log-cleanup-configuration-schedule   ;string or gexp
+             (default "30 12 01,08,15,22 * *")))
+
+(define (log-cleanup-program directory expiry)
+  (program-file "delete-old-logs"
+                (with-imported-modules '((guix build utils))
+                  #~(begin
+                      (use-modules (guix build utils))
+
+                      (let* ((now  (car (gettimeofday)))
+                             (logs (find-files #$directory
+					       (lambda (file stat)
+					         (> (- now (stat:mtime stat))
+						    #$expiry)))))
+                        (format #t "deleting ~a log files from '~a'...~%"
+                                (length logs) #$directory)
+                        (for-each delete-file logs))))))
+
+(define (log-cleanup-mcron-jobs configuration)
+  (match-record configuration <log-cleanup-configuration>
+    (directory expiry schedule)
+    (list #~(job #$schedule
+                 #$(log-cleanup-program directory expiry)))))
+
+(define log-cleanup-service-type
+  (service-type
+   (name 'log-cleanup)
+   (extensions
+    (list (service-extension mcron-service-type
+                             log-cleanup-mcron-jobs)))
+   (description
+    "Periodically delete old log files.")))
+
 \f
 ;;;
 ;;; Unattended upgrade.
-- 
2.34.0

[bug#54659] [PATCH 2/2] services: Add 'log-cleanup' service to '%base-services' for build logs.

[Ludovic Courtès]

* gnu/services/base.scm (%base-services): Add 'log-cleanup-service-type'
instance.
* doc/guix.texi (Log Rotation): Add example and mention '%base-services'.
---
 doc/guix.texi         | 13 ++++++++++++-
 gnu/services/base.scm |  6 ++++++
 2 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index ad2763ec8a..eaaf829aa2 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -17645,7 +17645,18 @@ Some log files just need to be deleted periodically once they are old,
 without any other criterion and without any archival step.  This is the
 case of build logs stored by @command{guix-daemon} under
 @file{/var/log/guix/drvs} (@pxref{Invoking guix-daemon}).  The
-@code{log-cleanup} service addresses this use case.
+@code{log-cleanup} service addresses this use case.  For example,
+@code{%base-services} (@pxref{Base Services}) includes the following:
+
+@lisp
+;; Periodically delete build logs more than 4 months old.
+(service log-cleanup-service-type
+         (log-cleanup-configuration
+          (directory "/var/log/guix/drvs")
+          (expiry (* 4 30 24 3600))))
+@end lisp
+
+That ensures build logs do not accumulate endlessly.
 
 @defvr {Scheme Variable} log-cleanup-service-type
 This is the type of the service to delete old logs.  Its value must be a
diff --git a/gnu/services/base.scm b/gnu/services/base.scm
index f278cb76de..ebaba524bc 100644
--- a/gnu/services/base.scm
+++ b/gnu/services/base.scm
@@ -2817,6 +2817,12 @@ (define %base-services
 
         (service rottlog-service-type)
 
+        ;; Periodically delete build logs more than 4 months old.
+        (service log-cleanup-service-type
+                 (log-cleanup-configuration
+                  (directory "/var/log/guix/drvs")
+                  (expiry (* 4 30 24 3600))))
+
         ;; The LVM2 rules are needed as soon as LVM2 or the device-mapper is
         ;; used, so enable them by default.  The FUSE and ALSA rules are
         ;; less critical, but handy.
-- 
2.34.0

[bug#54659] [PATCH 2/2] services: Add 'log-cleanup' service to '%base-services' for build logs.

[Liliana Marie Prikler]

Am Donnerstag, dem 31.03.2022 um 23:22 +0200 schrieb Ludovic Courtès:
> * gnu/services/base.scm (%base-services): Add 'log-cleanup-service-
> type'
> instance.
> * doc/guix.texi (Log Rotation): Add example and mention '%base-
> services'.
> ---
>  doc/guix.texi         | 13 ++++++++++++-
>  gnu/services/base.scm |  6 ++++++
>  2 files changed, 18 insertions(+), 1 deletion(-)
> 
> diff --git a/doc/guix.texi b/doc/guix.texi
> index ad2763ec8a..eaaf829aa2 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -17645,7 +17645,18 @@ Some log files just need to be deleted
> periodically once they are old,
>  without any other criterion and without any archival step.  This is
> the
>  case of build logs stored by @command{guix-daemon} under
>  @file{/var/log/guix/drvs} (@pxref{Invoking guix-daemon}).  The
> -@code{log-cleanup} service addresses this use case.
> +@code{log-cleanup} service addresses this use case.  For example,
> +@code{%base-services} (@pxref{Base Services}) includes the
> following:
> +
> +@lisp
> +;; Periodically delete build logs more than 4 months old.
> +(service log-cleanup-service-type
> +         (log-cleanup-configuration
> +          (directory "/var/log/guix/drvs")
> +          (expiry (* 4 30 24 3600))))
> +@end lisp
> +
> +That ensures build logs do not accumulate endlessly.
>  
>  @defvr {Scheme Variable} log-cleanup-service-type
>  This is the type of the service to delete old logs.  Its value must
> be a
> diff --git a/gnu/services/base.scm b/gnu/services/base.scm
> index f278cb76de..ebaba524bc 100644
> --- a/gnu/services/base.scm
> +++ b/gnu/services/base.scm
> @@ -2817,6 +2817,12 @@ (define %base-services
>  
>          (service rottlog-service-type)
>  
> +        ;; Periodically delete build logs more than 4 months old.
> +        (service log-cleanup-service-type
> +                 (log-cleanup-configuration
> +                  (directory "/var/log/guix/drvs")
> +                  (expiry (* 4 30 24 3600))))
> +
This might be very nitpicky, but I think we should leave expiry
undefined so that the default of 6 months is applied.  This way, if
there's a future decision that all logs should probably kept e.g. for
only 3 months, there is just one place to edit rather than two.

Cheers

[bug#54659] [PATCH 2/2] services: Add 'log-cleanup' service to '%base-services' for build logs.

[Ludovic Courtès]

Hi!

Liliana Marie Prikler <liliana.prikler@ist.tugraz.at> skribis:

>> +        ;; Periodically delete build logs more than 4 months old.
>> +        (service log-cleanup-service-type
>> +                 (log-cleanup-configuration
>> +                  (directory "/var/log/guix/drvs")
>> +                  (expiry (* 4 30 24 3600))))
>> +
> This might be very nitpicky, but I think we should leave expiry
> undefined so that the default of 6 months is applied.  This way, if
> there's a future decision that all logs should probably kept e.g. for
> only 3 months, there is just one place to edit rather than two.

Yeah, dunno; I thought we’d rather be explicit, but maybe you’re right.

Ludo’.

bug#54659: [PATCH 0/2] Periodically delete build logs

[Ludovic Courtès]

Hi,

Liliana Marie Prikler <liliana.prikler@ist.tugraz.at> skribis:

> This might be very nitpicky, but I think we should leave expiry
> undefined so that the default of 6 months is applied.  This way, if
> there's a future decision that all logs should probably kept e.g. for
> only 3 months, there is just one place to edit rather than two.

In the end I followed your suggestion and pushed as
e692dc632cbb0e6d21ed6f09f4c7f52391802cfb.

Thanks!

Ludo’.