unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* [PATCH v2 0/4] Make certbot play more nicely with nginx
       [not found] <cover.1706098718.git.carlo@zancanaro.id.au>
@ 2024-01-30 13:26 ` Carlo Zancanaro
  2024-01-30 13:26   ` bug#46961: " Carlo Zancanaro
                     ` (2 more replies)
  2024-01-30 13:26 ` [PATCH v2 1/4] services: certbot: Symlink certificates to /etc/certs Carlo Zancanaro
                   ` (3 subsequent siblings)
  4 siblings, 3 replies; 11+ messages in thread
From: Carlo Zancanaro @ 2024-01-30 13:26 UTC (permalink / raw)
  To: 46961; +Cc: clement, brice, guix-devel

Hi Guix,

This patch series is a few changes to make certbot default to doing
"the right thing" in the common case of wanting certificates for an
nginx web server.

The initial change (in v1 of these patches) was to solve the certbot
bootstrapping problem. Nginx won't start without valid certificates,
but certbot can't produce certificates without a functional
nginx. This is solved by generating self-signed certificates to start
with, and then replacing them once certbot has run. Doing this
requires storing certificates in a different location (because certbot
is very particular). I've chosen /etc/certs/.

The other two changes (new to v2 of this series) make things a bit
easier to use: a one-shot shepherd service to renew certificates when
the machine starts up, and a default deploy-hook to reload the nginx
configuration (which picks up the new certificates). I think these
changes make certbot "do the right thing", at the expense of being
slightly more magical.

On IRC podiki suggested I should copy guix-devel and Brice (the
original bug reporter), so I've done that, too.

Carlo Zancanaro (4):
  services: certbot: Symlink certificates to /etc/certs.
  services: certbot: Create self-signed certificates before certbot
    runs.
  services: certbot: Add a default deploy hook to reload nginx.
  services: certbot: Add one-shot service to renew certificates.

 doc/guix.texi            |  38 ++++++---
 gnu/services/certbot.scm | 178 ++++++++++++++++++++++++++++++++++++---
 2 files changed, 188 insertions(+), 28 deletions(-)


base-commit: 144c95032e517bb8ce466b930fe91506bcc92b2b
-- 
2.41.0



^ permalink raw reply	[flat|nested] 11+ messages in thread

* bug#46961: [PATCH v2 0/4] Make certbot play more nicely with nginx
  2024-01-30 13:26 ` [PATCH v2 0/4] Make certbot play more nicely with nginx Carlo Zancanaro
@ 2024-01-30 13:26   ` Carlo Zancanaro
  2024-01-30 14:49   ` Felix Lechner via Development of GNU Guix and the GNU System distribution.
       [not found]   ` <875xzanaer.fsf__22488.5524179385$1706626282$gmane$org@lease-up.com>
  2 siblings, 0 replies; 11+ messages in thread
From: Carlo Zancanaro @ 2024-01-30 13:26 UTC (permalink / raw)
  To: 46961; +Cc: guix-devel, brice, clement

Hi Guix,

This patch series is a few changes to make certbot default to doing
"the right thing" in the common case of wanting certificates for an
nginx web server.

The initial change (in v1 of these patches) was to solve the certbot
bootstrapping problem. Nginx won't start without valid certificates,
but certbot can't produce certificates without a functional
nginx. This is solved by generating self-signed certificates to start
with, and then replacing them once certbot has run. Doing this
requires storing certificates in a different location (because certbot
is very particular). I've chosen /etc/certs/.

The other two changes (new to v2 of this series) make things a bit
easier to use: a one-shot shepherd service to renew certificates when
the machine starts up, and a default deploy-hook to reload the nginx
configuration (which picks up the new certificates). I think these
changes make certbot "do the right thing", at the expense of being
slightly more magical.

On IRC podiki suggested I should copy guix-devel and Brice (the
original bug reporter), so I've done that, too.

Carlo Zancanaro (4):
  services: certbot: Symlink certificates to /etc/certs.
  services: certbot: Create self-signed certificates before certbot
    runs.
  services: certbot: Add a default deploy hook to reload nginx.
  services: certbot: Add one-shot service to renew certificates.

 doc/guix.texi            |  38 ++++++---
 gnu/services/certbot.scm | 178 ++++++++++++++++++++++++++++++++++++---
 2 files changed, 188 insertions(+), 28 deletions(-)


base-commit: 144c95032e517bb8ce466b930fe91506bcc92b2b
-- 
2.41.0





^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH v2 1/4] services: certbot: Symlink certificates to /etc/certs.
       [not found] <cover.1706098718.git.carlo@zancanaro.id.au>
  2024-01-30 13:26 ` [PATCH v2 0/4] Make certbot play more nicely with nginx Carlo Zancanaro
@ 2024-01-30 13:26 ` Carlo Zancanaro
  2024-01-30 13:26 ` [PATCH v2 2/4] services: certbot: Create self-signed certificates before certbot runs Carlo Zancanaro
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 11+ messages in thread
From: Carlo Zancanaro @ 2024-01-30 13:26 UTC (permalink / raw)
  To: 46961; +Cc: clement, brice, guix-devel

* gnu/services/certbot.scm (certbot-deploy-hook): New procedure.
(certbot-command): Pass new deploy hook to certbot.
* doc/guix.texi: Replace "letsencrypt/live" with "certs" throughout.

Change-Id: I2ba5e4903d1e293e566b732a84b07d5a134b697d
---
 doc/guix.texi            | 26 +++++++++++++-------------
 gnu/services/certbot.scm | 36 ++++++++++++++++++++++++++++++++++--
 2 files changed, 47 insertions(+), 15 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index bb0af26d93..b134d45a16 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -43,7 +43,7 @@
 Copyright @copyright{} 2017, 2018, 2019 Clément Lassieur@*
 Copyright @copyright{} 2017, 2018, 2020, 2021, 2022 Mathieu Othacehe@*
 Copyright @copyright{} 2017 Federico Beffa@*
-Copyright @copyright{} 2017, 2018 Carlo Zancanaro@*
+Copyright @copyright{} 2017, 2018, 2024 Carlo Zancanaro@*
 Copyright @copyright{} 2017 Thomas Danckaert@*
 Copyright @copyright{} 2017 humanitiesNerd@*
 Copyright @copyright{} 2017, 2021 Christine Lemmer-Webber@*
@@ -28135,7 +28135,7 @@ Messaging Services
 them.  See @url{https://prosody.im/doc/letsencrypt}.
 
 @example
-prosodyctl --root cert import /etc/letsencrypt/live
+prosodyctl --root cert import /etc/certs
 @end example
 
 The available configuration parameters follow.  Each parameter
@@ -28846,8 +28846,8 @@ Telephony Services
           (welcome-text
             "Welcome to this Mumble server running on Guix!")
           (cert-required? #t) ;disallow text password logins
-          (ssl-cert "/etc/letsencrypt/live/mumble.example.com/fullchain.pem")
-          (ssl-key "/etc/letsencrypt/live/mumble.example.com/privkey.pem")))
+          (ssl-cert "/etc/certs/mumble.example.com/fullchain.pem")
+          (ssl-key "/etc/certs/mumble.example.com/privkey.pem")))
 @end lisp
 
 After reconfiguring your system, you can manually set the mumble-server
@@ -28965,12 +28965,12 @@ Telephony Services
 File name of the SSL/TLS certificate used for encrypted connections.
 
 @lisp
-(ssl-cert "/etc/letsencrypt/live/example.com/fullchain.pem")
+(ssl-cert "/etc/certs/example.com/fullchain.pem")
 @end lisp
 @item @code{ssl-key} (default: @code{#f})
 Filepath to the ssl private key used for encrypted connections.
 @lisp
-(ssl-key "/etc/letsencrypt/live/example.com/privkey.pem")
+(ssl-key "/etc/certs/example.com/privkey.pem")
 @end lisp
 
 @item @code{ssl-dh-params} (default: @code{#f})
@@ -32685,7 +32685,7 @@ Certificate Services
 Command to be run in a shell once for each successfully issued
 certificate.  For this command, the shell variable
 @code{$RENEWED_LINEAGE} will point to the config live subdirectory (for
-example, @samp{"/etc/letsencrypt/live/example.com"}) containing the new
+example, @samp{"/etc/certs/example.com"}) containing the new
 certificates and keys; the shell variable @code{$RENEWED_DOMAINS} will
 contain a space-delimited list of renewed certificate domains (for
 example, @samp{"example.com www.example.com"}.
@@ -32694,8 +32694,8 @@ Certificate Services
 @end deftp
 
 For each @code{certificate-configuration}, the certificate is saved to
-@code{/etc/letsencrypt/live/@var{name}/fullchain.pem} and the key is
-saved to @code{/etc/letsencrypt/live/@var{name}/privkey.pem}.
+@code{/etc/certs/@var{name}/fullchain.pem} and the key is
+saved to @code{/etc/certs/@var{name}/privkey.pem}.
 @node DNS Services
 @subsection DNS Services
 @cindex DNS (domain name system)
@@ -37381,9 +37381,9 @@ Version Control Services
              (listen '("443 ssl"))
              (server-name "git.my-host.org")
              (ssl-certificate
-              "/etc/letsencrypt/live/git.my-host.org/fullchain.pem")
+              "/etc/certs/git.my-host.org/fullchain.pem")
              (ssl-certificate-key
-              "/etc/letsencrypt/live/git.my-host.org/privkey.pem")
+              "/etc/certs/git.my-host.org/privkey.pem")
              (locations
               (list
                (git-http-nginx-location-configuration
@@ -38508,9 +38508,9 @@ Version Control Services
            (nginx-server-block
              (nginx-server-configuration
                (ssl-certificate
-                 "/etc/letsencrypt/live/myweb.site/fullchain.pem")
+                 "/etc/certs/myweb.site/fullchain.pem")
                (ssl-certificate-key
-                 "/etc/letsencrypt/live/myweb.site/privkey.pem")
+                 "/etc/certs/myweb.site/privkey.pem")
                (listen '("443 ssl http2" "[::]:443 ssl http2"))
                (locations
                  (list
diff --git a/gnu/services/certbot.scm b/gnu/services/certbot.scm
index 0c45471659..3926d0551a 100644
--- a/gnu/services/certbot.scm
+++ b/gnu/services/certbot.scm
@@ -6,6 +6,7 @@
 ;;; Copyright © 2020 Jack Hill <jackhill@jackhill.us>
 ;;; Copyright © 2020 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2021 Raghav Gururajan <rg@raghavgururajan.name>
+;;; Copyright © 2024 Carlo Zancanaro <carlo@zancanaro.id.au>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -87,6 +88,35 @@ (define-record-type* <certbot-configuration>
                           (body
                            (list "return 301 https://$host$request_uri;"))))))
 
+(define (certbot-deploy-hook name deploy-hook-script)
+  "Returns a gexp which creates symlinks for privkey.pem and fullchain.pem
+from /etc/certs/NAME to /etc/letsenctypt/live/NAME.  If DEPLOY-HOOK-SCRIPT is
+not #f then it is run after the symlinks have been created."
+  (program-file
+   (string-append name "-deploy-hook")
+   (with-imported-modules '((guix build utils))
+     #~(begin
+         (use-modules (guix build utils))
+         (mkdir-p #$(string-append "/etc/certs/" name))
+         (chmod #$(string-append "/etc/certs/" name) #o755)
+
+         ;; Create new symlinks
+         (symlink #$(string-append
+                     "/etc/letsencrypt/live/" name "/privkey.pem")
+                  #$(string-append "/etc/certs/" name "/privkey.pem.new"))
+         (symlink #$(string-append
+                     "/etc/letsencrypt/live/" name "/fullchain.pem")
+                  #$(string-append "/etc/certs/" name "/fullchain.pem.new"))
+
+         ;; Rename over the top of the old ones, if there are any.
+         (rename-file #$(string-append "/etc/certs/" name "/privkey.pem.new")
+                      #$(string-append "/etc/certs/" name "/privkey.pem"))
+         (rename-file #$(string-append "/etc/certs/" name "/fullchain.pem.new")
+                      #$(string-append "/etc/certs/" name "/fullchain.pem"))
+         #$@(if deploy-hook-script
+                (list #~(invoke #$deploy-hook-script))
+                '())))))
+
 (define certbot-command
   (match-lambda
     (($ <certbot-configuration> package webroot certificates email
@@ -118,7 +148,8 @@ (define certbot-command
                           `("--manual-auth-hook" ,authentication-hook)
                           '())
                       (if cleanup-hook `("--manual-cleanup-hook" ,cleanup-hook) '())
-                      (if deploy-hook `("--deploy-hook" ,deploy-hook) '()))
+                      (list "--deploy-hook"
+                            (certbot-deploy-hook name deploy-hook)))
                      (append
                       (list name certbot "certonly" "-n" "--agree-tos"
                             "--webroot" "-w" webroot
@@ -130,7 +161,8 @@ (define certbot-command
                           '("--register-unsafely-without-email"))
                       (if server `("--server" ,server) '())
                       (if rsa-key-size `("--rsa-key-size" ,rsa-key-size) '())
-                      (if deploy-hook `("--deploy-hook" ,deploy-hook) '()))))))
+                      (list "--deploy-hook"
+                            (certbot-deploy-hook name deploy-hook)))))))
               certificates)))
        (program-file
         "certbot-command"
-- 
2.41.0



^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH v2 2/4] services: certbot: Create self-signed certificates before certbot runs.
       [not found] <cover.1706098718.git.carlo@zancanaro.id.au>
  2024-01-30 13:26 ` [PATCH v2 0/4] Make certbot play more nicely with nginx Carlo Zancanaro
  2024-01-30 13:26 ` [PATCH v2 1/4] services: certbot: Symlink certificates to /etc/certs Carlo Zancanaro
@ 2024-01-30 13:26 ` Carlo Zancanaro
  2024-01-30 13:26 ` [PATCH v2 3/4] services: certbot: Add a default deploy hook to reload nginx Carlo Zancanaro
  2024-01-30 13:26 ` [PATCH v2 4/4] services: certbot: Add one-shot service to renew certificates Carlo Zancanaro
  4 siblings, 0 replies; 11+ messages in thread
From: Carlo Zancanaro @ 2024-01-30 13:26 UTC (permalink / raw)
  To: 46961; +Cc: clement, brice, guix-devel

* gnu/services/certbot.scm (<certificate-configuration>): Add
start-self-signed? field.
(generate-certificate-gexp): New procedure.
(certbot-activation): Generate self-signed certificates when
start-self-signed? is #t.
* doc/guix.texi (Certificate services): Document start-self-signed?.

Change-Id: Icfd85ae0c3e29324acbcde6ba283546cf0e27a1d
---
 doc/guix.texi            |  6 ++++
 gnu/services/certbot.scm | 62 ++++++++++++++++++++++++++++++++++++++--
 2 files changed, 65 insertions(+), 3 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index b134d45a16..58a65fe0b7 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -32690,6 +32690,12 @@ Certificate Services
 contain a space-delimited list of renewed certificate domains (for
 example, @samp{"example.com www.example.com"}.
 
+@item @code{start-self-signed?} (default: @code{#t})
+Whether to generate an initial self-signed certificate during system
+activation.  This option is particularly useful to allow @code{nginx} to
+start before @code{certbot} has run, because @code{certbot} relies on
+@code{nginx} running to perform HTTP challenges.
+
 @end table
 @end deftp
 
diff --git a/gnu/services/certbot.scm b/gnu/services/certbot.scm
index 3926d0551a..10b99f5630 100644
--- a/gnu/services/certbot.scm
+++ b/gnu/services/certbot.scm
@@ -35,6 +35,7 @@ (define-module (gnu services certbot)
   #:use-module (guix records)
   #:use-module (guix gexp)
   #:use-module (srfi srfi-1)
+  #:use-module (ice-9 format)
   #:use-module (ice-9 match)
   #:export (certbot-service-type
             certbot-configuration
@@ -64,7 +65,9 @@ (define-record-type* <certificate-configuration>
   (cleanup-hook        certificate-cleanup-hook
                        (default #f))
   (deploy-hook         certificate-configuration-deploy-hook
-                       (default #f)))
+                       (default #f))
+  (start-self-signed?  certificate-configuration-start-self-signed?
+                       (default #t)))
 
 (define-record-type* <certbot-configuration>
   certbot-configuration make-certbot-configuration
@@ -91,7 +94,10 @@ (define-record-type* <certbot-configuration>
 (define (certbot-deploy-hook name deploy-hook-script)
   "Returns a gexp which creates symlinks for privkey.pem and fullchain.pem
 from /etc/certs/NAME to /etc/letsenctypt/live/NAME.  If DEPLOY-HOOK-SCRIPT is
-not #f then it is run after the symlinks have been created."
+not #f then it is run after the symlinks have been created.  This wrapping is
+necessary for certificates with start-self-signed? set to #t, as it will
+overwrite the initial self-signed certificates upon the first successful
+deploy."
   (program-file
    (string-append name "-deploy-hook")
    (with-imported-modules '((guix build utils))
@@ -108,7 +114,8 @@ (define (certbot-deploy-hook name deploy-hook-script)
                      "/etc/letsencrypt/live/" name "/fullchain.pem")
                   #$(string-append "/etc/certs/" name "/fullchain.pem.new"))
 
-         ;; Rename over the top of the old ones, if there are any.
+         ;; Rename over the top of the old ones, just in case they were the
+         ;; original self-signed certificates.
          (rename-file #$(string-append "/etc/certs/" name "/privkey.pem.new")
                       #$(string-append "/etc/certs/" name "/privkey.pem"))
          (rename-file #$(string-append "/etc/certs/" name "/fullchain.pem.new")
@@ -184,6 +191,47 @@ (define (certbot-renewal-jobs config)
    #~(job '(next-minute-from (next-hour '(0 12)) (list (random 60)))
           #$(certbot-command config))))
 
+(define (generate-certificate-gexp certbot-cert-directory rsa-key-size)
+  (match-lambda
+    (($ <certificate-configuration> name (primary-domain other-domains ...)
+                                    challenge
+                                    csr authentication-hook
+                                    cleanup-hook deploy-hook)
+     (let (;; Arbitrary default subject, with just the
+           ;; right domain filled in. These values don't
+           ;; have any real significance.
+           (subject (string-append
+                     "/C=US/ST=Oregon/L=Portland/O=Company Name/OU=Org/CN="
+                     primary-domain))
+           (alt-names (if (null? other-domains)
+                          #f
+                          (format #f "subjectAltName=~{DNS:~a~^,~}"
+                                  other-domains)))
+           (directory (string-append "/etc/certs/" (or name primary-domain))))
+       #~(when (not (file-exists? #$directory))
+           ;; We generate self-signed certificates in /etc/certs/{domain},
+           ;; because certbot is very sensitive to its directory
+           ;; structure. It refuses to write over the top of existing files,
+           ;; so we need to use a directory outside of its control.
+           ;;
+           ;; These certificates are overwritten by the certbot deploy hook
+           ;; the first time it successfully obtains a letsencrypt-signed
+           ;; certificate.
+           (mkdir-p #$directory)
+           (chmod #$directory #o755)
+           (invoke #$(file-append openssl "/bin/openssl")
+                   "req" "-x509"
+                   "-newkey" #$(string-append "rsa:" (or rsa-key-size "4096"))
+                   "-keyout" #$(string-append directory "/privkey.pem")
+                   "-out" #$(string-append directory "/fullchain.pem")
+                   "-sha256"
+                   "-days" "1" ; Only one day, because we expect certbot to run
+                   "-nodes"
+                   "-subj" #$subject
+                   #$@(if alt-names
+                          (list "-addext" alt-names)
+                          (list))))))))
+
 (define (certbot-activation config)
   (let* ((certbot-directory "/var/lib/certbot")
          (certbot-cert-directory "/etc/letsencrypt/live")
@@ -198,6 +246,14 @@ (define (certbot-activation config)
              (mkdir-p #$webroot)
              (mkdir-p #$certbot-directory)
              (mkdir-p #$certbot-cert-directory)
+
+             #$@(let ((rsa-key-size (and rsa-key-size
+                                         (number->string rsa-key-size))))
+                  (map (generate-certificate-gexp certbot-cert-directory
+                                                  rsa-key-size)
+                       (filter certificate-configuration-start-self-signed?
+                               certificates)))
+
              (copy-file #$(certbot-command config) #$script)
              (display #$message)))))))
 
-- 
2.41.0



^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH v2 3/4] services: certbot: Add a default deploy hook to reload nginx.
       [not found] <cover.1706098718.git.carlo@zancanaro.id.au>
                   ` (2 preceding siblings ...)
  2024-01-30 13:26 ` [PATCH v2 2/4] services: certbot: Create self-signed certificates before certbot runs Carlo Zancanaro
@ 2024-01-30 13:26 ` Carlo Zancanaro
  2024-01-30 13:26 ` [PATCH v2 4/4] services: certbot: Add one-shot service to renew certificates Carlo Zancanaro
  4 siblings, 0 replies; 11+ messages in thread
From: Carlo Zancanaro @ 2024-01-30 13:26 UTC (permalink / raw)
  To: 46961; +Cc: clement, brice, guix-devel

* gnu/services/certbot.scm (%default-deploy-hook): New variable.
(<certificate-configuration>)[deploy-hook]: Use it as default deploy hook.
* doc/guix.texi (Certificate services): Document new default deploy hook.

Change-Id: Ibb10481170a6fda7df72492072b939dd6a6ad176
---
 doc/guix.texi            |  6 +++++-
 gnu/services/certbot.scm | 13 +++++++++++--
 2 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 58a65fe0b7..0f372a460f 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -32681,7 +32681,7 @@ Certificate Services
 additionally @code{$CERTBOT_AUTH_OUTPUT} will contain the standard output
 of the @code{auth-hook} script.
 
-@item @code{deploy-hook} (default: @code{#f})
+@item @code{deploy-hook} (default: @code{%default-deploy-hook})
 Command to be run in a shell once for each successfully issued
 certificate.  For this command, the shell variable
 @code{$RENEWED_LINEAGE} will point to the config live subdirectory (for
@@ -32690,6 +32690,10 @@ Certificate Services
 contain a space-delimited list of renewed certificate domains (for
 example, @samp{"example.com www.example.com"}.
 
+The default deploy hook calls the @code{reload} action of the
+@code{nginx} Shepherd service, to reload the newly generated
+certificates.
+
 @item @code{start-self-signed?} (default: @code{#t})
 Whether to generate an initial self-signed certificate during system
 activation.  This option is particularly useful to allow @code{nginx} to
diff --git a/gnu/services/certbot.scm b/gnu/services/certbot.scm
index 10b99f5630..490b9e8d6d 100644
--- a/gnu/services/certbot.scm
+++ b/gnu/services/certbot.scm
@@ -37,7 +37,8 @@ (define-module (gnu services certbot)
   #:use-module (srfi srfi-1)
   #:use-module (ice-9 format)
   #:use-module (ice-9 match)
-  #:export (certbot-service-type
+  #:export (%default-deploy-hook
+            certbot-service-type
             certbot-configuration
             certbot-configuration?
             certificate-configuration))
@@ -49,6 +50,14 @@ (define-module (gnu services certbot)
 ;;; Code:
 
 \f
+(define %default-deploy-hook
+  (program-file
+   "reload-nginx.scm"
+   (with-imported-modules '((gnu services herd))
+     #~(begin
+         (use-modules (gnu services herd))
+         (with-shepherd-action 'nginx ('reload) result result)))))
+
 (define-record-type* <certificate-configuration>
   certificate-configuration make-certificate-configuration
   certificate-configuration?
@@ -65,7 +74,7 @@ (define-record-type* <certificate-configuration>
   (cleanup-hook        certificate-cleanup-hook
                        (default #f))
   (deploy-hook         certificate-configuration-deploy-hook
-                       (default #f))
+                       (default %default-deploy-hook))
   (start-self-signed?  certificate-configuration-start-self-signed?
                        (default #t)))
 
-- 
2.41.0



^ permalink raw reply related	[flat|nested] 11+ messages in thread

* [PATCH v2 4/4] services: certbot: Add one-shot service to renew certificates.
       [not found] <cover.1706098718.git.carlo@zancanaro.id.au>
                   ` (3 preceding siblings ...)
  2024-01-30 13:26 ` [PATCH v2 3/4] services: certbot: Add a default deploy hook to reload nginx Carlo Zancanaro
@ 2024-01-30 13:26 ` Carlo Zancanaro
  2024-01-30 13:26   ` bug#46961: " Carlo Zancanaro
  4 siblings, 1 reply; 11+ messages in thread
From: Carlo Zancanaro @ 2024-01-30 13:26 UTC (permalink / raw)
  To: 46961; +Cc: clement, brice, guix-devel

* gnu/services/certbot.scm (certbot-renewal-one-shot): New procedure.
(certbot-service-type)[extensions]: Add it to shepherd-root extension.
(certbot-command): Make connection errors return a different exit code.
(certbot-activation): Remove message with certificate renewal instructions.

Change-Id: I614ac6214a753dba0396e2385a75926c8355caa1
---
 gnu/services/certbot.scm | 77 +++++++++++++++++++++++++++++++++-------
 1 file changed, 65 insertions(+), 12 deletions(-)

diff --git a/gnu/services/certbot.scm b/gnu/services/certbot.scm
index 490b9e8d6d..d6354c86d3 100644
--- a/gnu/services/certbot.scm
+++ b/gnu/services/certbot.scm
@@ -183,15 +183,37 @@ (define certbot-command
        (program-file
         "certbot-command"
         #~(begin
-            (use-modules (ice-9 match))
-            (let ((code 0))
+            (use-modules (ice-9 match)
+                         (ice-9 textual-ports))
+
+            (define (file-contains? file string)
+              (string-contains (call-with-input-file file
+                                 get-string-all)
+                               string))
+
+            (define (connection-error?)
+              (file-contains? "/var/log/letsencrypt/letsencrypt.log"
+                              "Failed to establish a new connection"))
+
+            (let ((script-code 0))
               (for-each
                (match-lambda
                  ((name . command)
                   (begin
                     (format #t "Acquiring or renewing certificate: ~a~%" name)
-                    (set! code (or (apply system* command) code)))))
-               '#$commands) code)))))))
+                    (unless (zero? (status:exit-val (apply system* command)))
+                      ;; Certbot errors are always exit code 1, but we'd like
+                      ;; to separate connection errors from other error types.
+                      (if (connection-error?)
+                          ;; If we have a connection error, then bail early
+                          ;; with exit code 2. We don't expect this to
+                          ;; resolve within the timespan of this script.
+                          (exit 2)
+                          ;; If we have any other type of error, then continue
+                          ;; but exit with a failing status code in the end.
+                          (set! script-code 1))))))
+               '#$commands)
+              (exit script-code))))))))
 
 (define (certbot-renewal-jobs config)
   (list
@@ -200,6 +222,40 @@ (define (certbot-renewal-jobs config)
    #~(job '(next-minute-from (next-hour '(0 12)) (list (random 60)))
           #$(certbot-command config))))
 
+(define (certbot-renewal-one-shot config)
+  (list
+   ;; Renew certificates when the system first starts. This is a one-shot
+   ;; service, because the mcron configuration will take care of running this
+   ;; periodically. This is most useful the very first time the system starts,
+   ;; to overwrite our self-signed certificates as soon as possible without
+   ;; user intervention.
+   (shepherd-service
+    (provision '(renew-certbot-certificates))
+    (requirement '(nginx))
+    (one-shot? #t)
+    (start #~(lambda _
+               ;; This needs the network, but there's no reliable way to know
+               ;; if the network is up other than trying. If we fail due to a
+               ;; connection error we retry a number of times in the hope that
+               ;; the network comes up soon.
+               (let loop ((attempt 0))
+                 (let ((code (status:exit-val
+                              (system* #$(certbot-command config)))))
+                   (cond
+                    ((and (= code 2)      ; Exit code 2 means connection error
+                          (< attempt 12)) ; 12 * 10 seconds = 2 minutes
+                     (sleep 10)
+                     (loop (1+ attempt)))
+                    ((zero? code)
+                     ;; Success!
+                     #t)
+                    (else
+                     ;; Failure.
+                     #f))))))
+    (auto-start? #t)
+    (documentation "Call certbot to renew certificates.")
+    (actions (list (shepherd-configuration-action (certbot-command config)))))))
+
 (define (generate-certificate-gexp certbot-cert-directory rsa-key-size)
   (match-lambda
     (($ <certificate-configuration> name (primary-domain other-domains ...)
@@ -243,9 +299,7 @@ (define (generate-certificate-gexp certbot-cert-directory rsa-key-size)
 
 (define (certbot-activation config)
   (let* ((certbot-directory "/var/lib/certbot")
-         (certbot-cert-directory "/etc/letsencrypt/live")
-         (script (in-vicinity certbot-directory "renew-certificates"))
-         (message (format #f (G_ "~a may need to be run~%") script)))
+         (certbot-cert-directory "/etc/letsencrypt/live"))
     (match config
       (($ <certbot-configuration> package webroot certificates email
                                   server rsa-key-size default-location)
@@ -261,10 +315,7 @@ (define (certbot-activation config)
                   (map (generate-certificate-gexp certbot-cert-directory
                                                   rsa-key-size)
                        (filter certificate-configuration-start-self-signed?
-                               certificates)))
-
-             (copy-file #$(certbot-command config) #$script)
-             (display #$message)))))))
+                               certificates)))))))))
 
 (define certbot-nginx-server-configurations
   (match-lambda
@@ -297,7 +348,9 @@ (define certbot-service-type
                        (service-extension activation-service-type
                                           certbot-activation)
                        (service-extension mcron-service-type
-                                          certbot-renewal-jobs)))
+                                          certbot-renewal-jobs)
+                       (service-extension shepherd-root-service-type
+                                          certbot-renewal-one-shot)))
                 (compose concatenate)
                 (extend (lambda (config additional-certificates)
                           (certbot-configuration
-- 
2.41.0



^ permalink raw reply related	[flat|nested] 11+ messages in thread

* bug#46961: [PATCH v2 4/4] services: certbot: Add one-shot service to renew certificates.
  2024-01-30 13:26 ` [PATCH v2 4/4] services: certbot: Add one-shot service to renew certificates Carlo Zancanaro
@ 2024-01-30 13:26   ` Carlo Zancanaro
  0 siblings, 0 replies; 11+ messages in thread
From: Carlo Zancanaro @ 2024-01-30 13:26 UTC (permalink / raw)
  To: 46961; +Cc: guix-devel, brice, clement

* gnu/services/certbot.scm (certbot-renewal-one-shot): New procedure.
(certbot-service-type)[extensions]: Add it to shepherd-root extension.
(certbot-command): Make connection errors return a different exit code.
(certbot-activation): Remove message with certificate renewal instructions.

Change-Id: I614ac6214a753dba0396e2385a75926c8355caa1
---
 gnu/services/certbot.scm | 77 +++++++++++++++++++++++++++++++++-------
 1 file changed, 65 insertions(+), 12 deletions(-)

diff --git a/gnu/services/certbot.scm b/gnu/services/certbot.scm
index 490b9e8d6d..d6354c86d3 100644
--- a/gnu/services/certbot.scm
+++ b/gnu/services/certbot.scm
@@ -183,15 +183,37 @@ (define certbot-command
        (program-file
         "certbot-command"
         #~(begin
-            (use-modules (ice-9 match))
-            (let ((code 0))
+            (use-modules (ice-9 match)
+                         (ice-9 textual-ports))
+
+            (define (file-contains? file string)
+              (string-contains (call-with-input-file file
+                                 get-string-all)
+                               string))
+
+            (define (connection-error?)
+              (file-contains? "/var/log/letsencrypt/letsencrypt.log"
+                              "Failed to establish a new connection"))
+
+            (let ((script-code 0))
               (for-each
                (match-lambda
                  ((name . command)
                   (begin
                     (format #t "Acquiring or renewing certificate: ~a~%" name)
-                    (set! code (or (apply system* command) code)))))
-               '#$commands) code)))))))
+                    (unless (zero? (status:exit-val (apply system* command)))
+                      ;; Certbot errors are always exit code 1, but we'd like
+                      ;; to separate connection errors from other error types.
+                      (if (connection-error?)
+                          ;; If we have a connection error, then bail early
+                          ;; with exit code 2. We don't expect this to
+                          ;; resolve within the timespan of this script.
+                          (exit 2)
+                          ;; If we have any other type of error, then continue
+                          ;; but exit with a failing status code in the end.
+                          (set! script-code 1))))))
+               '#$commands)
+              (exit script-code))))))))
 
 (define (certbot-renewal-jobs config)
   (list
@@ -200,6 +222,40 @@ (define (certbot-renewal-jobs config)
    #~(job '(next-minute-from (next-hour '(0 12)) (list (random 60)))
           #$(certbot-command config))))
 
+(define (certbot-renewal-one-shot config)
+  (list
+   ;; Renew certificates when the system first starts. This is a one-shot
+   ;; service, because the mcron configuration will take care of running this
+   ;; periodically. This is most useful the very first time the system starts,
+   ;; to overwrite our self-signed certificates as soon as possible without
+   ;; user intervention.
+   (shepherd-service
+    (provision '(renew-certbot-certificates))
+    (requirement '(nginx))
+    (one-shot? #t)
+    (start #~(lambda _
+               ;; This needs the network, but there's no reliable way to know
+               ;; if the network is up other than trying. If we fail due to a
+               ;; connection error we retry a number of times in the hope that
+               ;; the network comes up soon.
+               (let loop ((attempt 0))
+                 (let ((code (status:exit-val
+                              (system* #$(certbot-command config)))))
+                   (cond
+                    ((and (= code 2)      ; Exit code 2 means connection error
+                          (< attempt 12)) ; 12 * 10 seconds = 2 minutes
+                     (sleep 10)
+                     (loop (1+ attempt)))
+                    ((zero? code)
+                     ;; Success!
+                     #t)
+                    (else
+                     ;; Failure.
+                     #f))))))
+    (auto-start? #t)
+    (documentation "Call certbot to renew certificates.")
+    (actions (list (shepherd-configuration-action (certbot-command config)))))))
+
 (define (generate-certificate-gexp certbot-cert-directory rsa-key-size)
   (match-lambda
     (($ <certificate-configuration> name (primary-domain other-domains ...)
@@ -243,9 +299,7 @@ (define (generate-certificate-gexp certbot-cert-directory rsa-key-size)
 
 (define (certbot-activation config)
   (let* ((certbot-directory "/var/lib/certbot")
-         (certbot-cert-directory "/etc/letsencrypt/live")
-         (script (in-vicinity certbot-directory "renew-certificates"))
-         (message (format #f (G_ "~a may need to be run~%") script)))
+         (certbot-cert-directory "/etc/letsencrypt/live"))
     (match config
       (($ <certbot-configuration> package webroot certificates email
                                   server rsa-key-size default-location)
@@ -261,10 +315,7 @@ (define (certbot-activation config)
                   (map (generate-certificate-gexp certbot-cert-directory
                                                   rsa-key-size)
                        (filter certificate-configuration-start-self-signed?
-                               certificates)))
-
-             (copy-file #$(certbot-command config) #$script)
-             (display #$message)))))))
+                               certificates)))))))))
 
 (define certbot-nginx-server-configurations
   (match-lambda
@@ -297,7 +348,9 @@ (define certbot-service-type
                        (service-extension activation-service-type
                                           certbot-activation)
                        (service-extension mcron-service-type
-                                          certbot-renewal-jobs)))
+                                          certbot-renewal-jobs)
+                       (service-extension shepherd-root-service-type
+                                          certbot-renewal-one-shot)))
                 (compose concatenate)
                 (extend (lambda (config additional-certificates)
                           (certbot-configuration
-- 
2.41.0





^ permalink raw reply related	[flat|nested] 11+ messages in thread

* Re: [PATCH v2 0/4] Make certbot play more nicely with nginx
  2024-01-30 13:26 ` [PATCH v2 0/4] Make certbot play more nicely with nginx Carlo Zancanaro
  2024-01-30 13:26   ` bug#46961: " Carlo Zancanaro
@ 2024-01-30 14:49   ` Felix Lechner via Development of GNU Guix and the GNU System distribution.
  2024-01-30 21:48     ` Carlo Zancanaro
       [not found]   ` <875xzanaer.fsf__22488.5524179385$1706626282$gmane$org@lease-up.com>
  2 siblings, 1 reply; 11+ messages in thread
From: Felix Lechner via Development of GNU Guix and the GNU System distribution. @ 2024-01-30 14:49 UTC (permalink / raw)
  To: Carlo Zancanaro, 46961; +Cc: clement, brice, guix-devel

Hi Carlo,

On Tue, Jan 30 2024, Carlo Zancanaro wrote:

> certbot can't produce certificates without a functional nginx

Yes, it can. The option is called --standalone. [1]

Maybe another way to bootstrap the certificates would be to hold off on
starting Nginx or Apache until all certificates are obtained?

Anyway, that's what I do manually.

Kind regards
Felix

[1] https://eff-certbot.readthedocs.io/en/latest/using.html#standalone


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: bug#46961: [PATCH v2 0/4] Make certbot play more nicely with nginx
       [not found]   ` <875xzanaer.fsf__22488.5524179385$1706626282$gmane$org@lease-up.com>
@ 2024-01-30 19:39     ` Clément Lassieur
  0 siblings, 0 replies; 11+ messages in thread
From: Clément Lassieur @ 2024-01-30 19:39 UTC (permalink / raw)
  To: Felix Lechner via Bug reports for GNU Guix
  Cc: Carlo Zancanaro, 46961, Felix Lechner, guix-devel, brice

On Tue, Jan 30 2024, Felix Lechner via Bug reports for GNU Guix wrote:

> Hi Carlo,
>
> On Tue, Jan 30 2024, Carlo Zancanaro wrote:
>
>> certbot can't produce certificates without a functional nginx
>
> Yes, it can. The option is called --standalone. [1]
>
> Maybe another way to bootstrap the certificates would be to hold off on
> starting Nginx or Apache until all certificates are obtained?

Yes but if we do this and there is no internet, nginx won't start right?

Carlo's solution allows to have a working nginx even when certbot fails.

(If I understand well)

> Anyway, that's what I do manually.
>
> Kind regards
> Felix
>
> [1] https://eff-certbot.readthedocs.io/en/latest/using.html#standalone


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v2 0/4] Make certbot play more nicely with nginx
  2024-01-30 14:49   ` Felix Lechner via Development of GNU Guix and the GNU System distribution.
@ 2024-01-30 21:48     ` Carlo Zancanaro
  2024-01-31  0:04       ` Wojtek Kosior via Development of GNU Guix and the GNU System distribution.
  0 siblings, 1 reply; 11+ messages in thread
From: Carlo Zancanaro @ 2024-01-30 21:48 UTC (permalink / raw)
  To: Felix Lechner; +Cc: 46961, clement, brice, guix-devel

Hi Felix,

On Tue, Jan 30 2024, Felix Lechner wrote:
> On Tue, Jan 30 2024, Carlo Zancanaro wrote:
>> certbot can't produce certificates without a functional nginx
>
> Yes, it can. The option is called --standalone. [1]

You are correct, of course. If I had been more precise I would 
have said "with our current configuration, certbot can't produce 
certificates without a functional nginx".

> Maybe another way to bootstrap the certificates would be to hold 
> off on starting Nginx or Apache until all certificates are 
> obtained?

This could work, but I see a few downsides.

As Clément has already mentioned, this would make nginx dependent 
on certbot. This causes problems for servers disconnected from the 
general internet, but it also shifts complexity into the nginx 
service without much benefit over the patch series I'm proposing. 
We'd need to add more configuration on the nginx side to control 
whether to delay startup based on whether we actually want 
certificates. This would delay the startup of the whole nginx 
process, even if some server configurations don't require new 
certificates.

For renewal, we would also have two options: (1) use --standalone, 
and require a period of downtime for our web server; or (2) use 
--webroot, and maintain two code paths for the two cases. I think 
it's a bad idea for Guix to make a decision that requires downtime 
of user systems if there's an alternative, so I don't like (1). 
Maintaining two "similar but different" code paths for (2) doesn't 
seem like a clear advantage over the patch series I'm proposing.

> Anyway, that's what I do manually.

I use the DNS challenge type, with hooks which automatically 
create/remove DNS records. This solves all the problems I'm 
bringing up (i.e. doesn't require nginx, doesn't involve downtime, 
has a single code path), but I don't think Guix can assume that 
all users have the ability to do this. My aim with this patch 
series is to make the default certbot configuration work for the 
common case of a simple web server, without manual intervention.

Carlo


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v2 0/4] Make certbot play more nicely with nginx
  2024-01-30 21:48     ` Carlo Zancanaro
@ 2024-01-31  0:04       ` Wojtek Kosior via Development of GNU Guix and the GNU System distribution.
  0 siblings, 0 replies; 11+ messages in thread
From: Wojtek Kosior via Development of GNU Guix and the GNU System distribution. @ 2024-01-31  0:04 UTC (permalink / raw)
  To: Carlo Zancanaro; +Cc: Felix Lechner, 46961, clement, brice, guix-devel

[-- Attachment #1: Type: text/plain, Size: 3985 bytes --]

I sympathize with your approach (I, too, have been supplementing
Certbot with self-signed certs for some time).

What would also be cool is not to have `certbot-service-type` depend on
`nginx-service-type` in the first place.  So that one can more easily
use another HTTP server.  It can of course be achieved with
`remove-service-extensions` from `(gnu services)`, it's just less
elegant than having it supported directly.

Perhaps some variant of the "dependency inversion principle" would fit
here?  How about the following set of service types?

- certbot-tool-service-type — does what `certbot-service-type` used to
  do until now except it doesn't extend `nginx-service-type` and
  can itself be extended with not just `<certificate-configuration>`s
  but also `<certbot-configuration>`
- certbot/nginx-service-type — takes in `<certbot-configuration>`,
  extends both `certbot-tool-service-type` and `nginx-service-type`
- certbot/httpd-service-type — takes in `<certbot-configuration>`,
  extends both `certbot-tool-service-type` and `httpd-service-type`
- certbot-service-type — deprecated, functions as an alias for
  `certbot/nginx-service-type`

Your proposals are of course useful as well, regardless of this being
done

Best :)
Wojtek


-- (sig_start)
website: https://koszko.org/koszko.html
fingerprint: E972 7060 E3C5 637C 8A4F  4B42 4BC5 221C 5A79 FD1A
follow me on Fediverse: https://friendica.me/profile/koszko/profile

♥ R29kIGlzIHRoZXJlIGFuZCBsb3ZlcyBtZQ== | ÷ c2luIHNlcGFyYXRlZCBtZSBmcm9tIEhpbQ==
✝ YnV0IEplc3VzIGRpZWQgdG8gc2F2ZSBtZQ== | ? U2hhbGwgSSBiZWNvbWUgSGlzIGZyaWVuZD8=
-- (sig_end)


On Wed, 31 Jan 2024 08:48:54 +1100 Carlo Zancanaro <carlo@zancanaro.id.au> wrote:

> Hi Felix,
> 
> On Tue, Jan 30 2024, Felix Lechner wrote:
> > On Tue, Jan 30 2024, Carlo Zancanaro wrote:  
> >> certbot can't produce certificates without a functional nginx  
> >
> > Yes, it can. The option is called --standalone. [1]  
> 
> You are correct, of course. If I had been more precise I would 
> have said "with our current configuration, certbot can't produce 
> certificates without a functional nginx".
> 
> > Maybe another way to bootstrap the certificates would be to hold 
> > off on starting Nginx or Apache until all certificates are 
> > obtained?  
> 
> This could work, but I see a few downsides.
> 
> As Clément has already mentioned, this would make nginx dependent 
> on certbot. This causes problems for servers disconnected from the 
> general internet, but it also shifts complexity into the nginx 
> service without much benefit over the patch series I'm proposing. 
> We'd need to add more configuration on the nginx side to control 
> whether to delay startup based on whether we actually want 
> certificates. This would delay the startup of the whole nginx 
> process, even if some server configurations don't require new 
> certificates.
> 
> For renewal, we would also have two options: (1) use --standalone, 
> and require a period of downtime for our web server; or (2) use 
> --webroot, and maintain two code paths for the two cases. I think 
> it's a bad idea for Guix to make a decision that requires downtime 
> of user systems if there's an alternative, so I don't like (1). 
> Maintaining two "similar but different" code paths for (2) doesn't 
> seem like a clear advantage over the patch series I'm proposing.
> 
> > Anyway, that's what I do manually.  
> 
> I use the DNS challenge type, with hooks which automatically 
> create/remove DNS records. This solves all the problems I'm 
> bringing up (i.e. doesn't require nginx, doesn't involve downtime, 
> has a single code path), but I don't think Guix can assume that 
> all users have the ability to do this. My aim with this patch 
> series is to make the default certbot configuration work for the 
> common case of a simple web server, without manual intervention.
> 
> Carlo
> 

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2024-01-31  0:05 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
     [not found] <cover.1706098718.git.carlo@zancanaro.id.au>
2024-01-30 13:26 ` [PATCH v2 0/4] Make certbot play more nicely with nginx Carlo Zancanaro
2024-01-30 13:26   ` bug#46961: " Carlo Zancanaro
2024-01-30 14:49   ` Felix Lechner via Development of GNU Guix and the GNU System distribution.
2024-01-30 21:48     ` Carlo Zancanaro
2024-01-31  0:04       ` Wojtek Kosior via Development of GNU Guix and the GNU System distribution.
     [not found]   ` <875xzanaer.fsf__22488.5524179385$1706626282$gmane$org@lease-up.com>
2024-01-30 19:39     ` bug#46961: " Clément Lassieur
2024-01-30 13:26 ` [PATCH v2 1/4] services: certbot: Symlink certificates to /etc/certs Carlo Zancanaro
2024-01-30 13:26 ` [PATCH v2 2/4] services: certbot: Create self-signed certificates before certbot runs Carlo Zancanaro
2024-01-30 13:26 ` [PATCH v2 3/4] services: certbot: Add a default deploy hook to reload nginx Carlo Zancanaro
2024-01-30 13:26 ` [PATCH v2 4/4] services: certbot: Add one-shot service to renew certificates Carlo Zancanaro
2024-01-30 13:26   ` bug#46961: " Carlo Zancanaro

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).