bug#26685: certbot service

bug#26685: certbot service

[Andy Wingo]

[-- Attachment #1: Type: text/plain, Size: 274 bytes --]

Attached patch adds a certbot service extending nginx.  Only question
is, how to ensure that nginx is runing when the certbot activation runs?
In practice I bet this races so it's not a big issue but if there's a
way to require nginx before activation, that would be nice.


[-- Attachment #2: 0002-gnu-Add-certbot-service.patch --]
[-- Type: text/x-patch, Size: 10989 bytes --]

From 641607bb3ea2b46e80f43216783833c1fc8d70d6 Mon Sep 17 00:00:00 2001
From: Andy Wingo <wingo@igalia.com>
Date: Thu, 27 Apr 2017 14:09:16 +0200
Subject: [PATCH 2/5] gnu: Add certbot service.

* gnu/services/certbot.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add new file.
* doc/guix.texi (Certificate Services): New section.
---
 doc/guix.texi            |  80 +++++++++++++++++++++++++++++
 gnu/local.mk             |   1 +
 gnu/services/certbot.scm | 128 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 209 insertions(+)
 create mode 100644 gnu/services/certbot.scm

diff --git a/doc/guix.texi b/doc/guix.texi
index 0d334e302..21b6d7d88 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -215,6 +215,7 @@ Services
 * Messaging Services::          Messaging services.
 * Kerberos Services::           Kerberos services.
 * Web Services::                Web servers.
+* Certificate Services::        TLS certificates via Let's Encrypt.
 * VPN Services::                VPN daemons.
 * Network File System::         NFS related services.
 * Continuous Integration::      The Cuirass service.
@@ -8605,6 +8606,7 @@ declaration.
 * Messaging Services::          Messaging services.
 * Kerberos Services::           Kerberos services.
 * Web Services::                Web servers.
+* Certificate Services::        TLS certificates via Let's Encrypt.
 * VPN Services::                VPN daemons.
 * Network File System::         NFS related services.
 * Continuous Integration::      The Cuirass service.
@@ -13379,6 +13381,84 @@ Whether the server should add its configuration to response.
 @end table
 @end deftp
 
+
+@node Certificate Services
+@subsubsection Certificate Services
+
+@cindex web
+@cindex www
+@cindex HTTP
+The @code{(gnu services certbot)} module allows Guix systems to
+automatically obtain a valid TLS certificate from the Let's Encrypt
+certificate authority.  These certificates can then be used to serve
+content securely over HTTPS or other TLS-based protocols, with the
+knowledge that the client will be able to verify the server's
+authenticity.
+
+Let's Encrypt (@url{https://letsencrypt.org/}) provides the
+@code{certbot} tool to automate the certification process.  This tool
+first securely generates a key on the server.  It then makes a request
+to the Let's Encrypt certificate authority (CA) to sign the key.  The CA
+checks that the request originates from the host in question by using a
+challenge-response protocol, requiring the server to provide its
+response over HTTP.  If that protocol completes successfully, the CA
+signs the key, resulting in a certificate.  That certificate is valid
+for a limited period of time, and therefore to continue to provide TLS
+services, the server needs to periodically ask the CA to renew its
+signature.
+
+The Guix certbot service automates this process: the initial key
+generation, the initial certification request to the Let's Encrypt
+service, the web server challenge/response integration, writing the
+certificate to disk, and the automated periodic renewals.
+
+@defvr {Scheme Variable} certbot-service-type
+A service type for the @code{certbot} Let's Encrypt client.
+@end defvr
+
+@deftp {Data Type} certbot-configuration
+Data type representing the configuration of the @code{certbot} serice.
+This type has the following parameters:
+@table @asis
+@item @code{package} (default: @code{certbot})
+The certbot package to use.
+
+@item @code{webroot} (default: @code{/var/www})
+The directory from which to serve the Let's Encrypt challenge/response
+files.
+
+@item @code{hosts} (default: @code{()})
+A list of hosts for which to generate certificates and request
+signatures.
+
+@item @code{default-location} (default: @i{see below})
+The default @code{nginx-location-configuration}.  Because @code{certbot}
+needs to be able to serve challenges and responses, it needs to be able
+to run a web server.  It does so by extending the @code{nginx} web
+service with an @code{nginx-server-configuration} listening on the
+@var{hosts} on port 80, and which has a
+@code{nginx-location-configuration} for the @code{/.well-known/} URI
+path subspace used by Let's Encrypt.  @xref{Web Services}, for more on
+these nginx configuration data types.
+
+Requests to other URL paths will be matched by the
+@code{default-location}, which if present is added to all
+@code{nginx-server-configuration}s.
+
+By default, the @code{default-location} will issue a redirect from
+@code{http://@var{host}/...} to @code{https://@var{host}/...}, leaving
+you to define what to serve on your site via @code{https}.
+
+Pass @code{#f} to not issue a default location.
+@end table
+@end deftp
+
+The public key and its signatures will be written to
+@code{/etc/letsencrypt/live/@var{host}/fullchain.pem}, for each
+@var{host} in the configuration.  The private key is written to
+@code{/etc/letsencrypt/live/@var{host}/privkey.pem}.
+
+
 @node VPN Services
 @subsubsection VPN Services
 @cindex VPN (virtual private network)
diff --git a/gnu/local.mk b/gnu/local.mk
index 31239933c..ac0c9d510 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -414,6 +414,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/services/admin.scm			\
   %D%/services/avahi.scm			\
   %D%/services/base.scm				\
+  %D%/services/certbot.scm			\
   %D%/services/configuration.scm		\
   %D%/services/cuirass.scm			\
   %D%/services/cups.scm				\
diff --git a/gnu/services/certbot.scm b/gnu/services/certbot.scm
new file mode 100644
index 000000000..c11c9a66b
--- /dev/null
+++ b/gnu/services/certbot.scm
@@ -0,0 +1,128 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2016 ng0 <ng0@we.make.ritual.n0.is>
+;;; Copyright © 2016 Sou Bunnbu <iyzsong@member.fsf.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services certbot)
+  #:use-module (gnu services)
+  #:use-module (gnu services base)
+  #:use-module (gnu services shepherd)
+  #:use-module (gnu services mcron)
+  #:use-module (gnu services web)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu packages tls)
+  #:use-module (guix records)
+  #:use-module (guix gexp)
+  #:use-module (srfi srfi-1)
+  #:use-module (ice-9 match)
+  #:export (certbot-service-type
+            certbot-configuration
+            certbot-configuration?))
+
+;;; Commentary:
+;;;
+;;; Automatically obtaining TLS certificates from Let's Encrypt.
+;;;
+;;; Code:
+
+\f
+(define-record-type* <certbot-configuration>
+  certbot-configuration make-certbot-configuration
+  certbot-configuration?
+  (package             certbot-configuration-package
+                       (default certbot))
+  (webroot             certbot-configuration-webroot
+                       (default "/var/www"))
+  (hosts               certbot-configuration-hosts
+                       (default '()))
+  (default-location    certbot-configuration-default-location
+                       (default
+                         (nginx-location-configuration
+                          (uri "/")
+                          (body
+                           (list "return 301 https://$host$request_uri;"))))))
+
+(define certbot-renewal-jobs
+  (match-lambda
+    (($ <certbot-configuration> package webroot hosts default-location)
+     (match hosts
+       ;; Avoid pinging certbot if we have no hosts.
+       (() '())
+       (_
+        (list
+         ;; Attempt to renew the certificates twice a week.
+         #~(job (lambda (now)
+                  (next-day-from (next-hour-from now '(3))
+                                 '(2 5)))
+                (string-append #$package "/bin/certbot renew"
+                               (string-concatenate
+                                (map (lambda (host)
+                                       (string-append " -d " host))
+                                     #$hosts))))))))))
+
+(define certbot-activation
+  (match-lambda
+    (($ <certbot-configuration> package webroot hosts default-location)
+     (with-imported-modules '((guix build utils))
+       #~(begin
+	   (use-modules (guix build utils))
+	   (mkdir-p #$webroot)
+           (for-each
+            (lambda (host)
+              (unless (file-exists? (in-vicinity "/etc/letsencrypt/live" host))
+                (unless (zero? (system*
+                                (string-append #$certbot "/bin/certbot")
+                                "certonly" "--webroot" "-w" #$webroot
+                                "-d" host))
+                  (error "failed to acquire cert for host" host))))
+            '#$hosts))))))
+
+(define certbot-nginx-server-configurations
+  (match-lambda
+    (($ <certbot-configuration> package webroot hosts default-location)
+     (map
+      (lambda (host)
+        (nginx-server-configuration
+         (http-port 80)
+         (https-port #f)
+         (ssl-certificate #f)
+         (ssl-certificate-key #f)
+         (server-name (list host))
+         (locations
+          (filter identity
+                  (list
+                   (nginx-location-configuration
+                    (uri "/.well-known")
+                    (body (list (list "root " webroot ";"))))
+                   default-location)))))
+      hosts))))
+
+(define certbot-service-type
+  (service-type (name 'certbot)
+                (extensions
+                 (list (service-extension nginx-service-type
+                                          certbot-nginx-server-configurations)
+                       (service-extension activation-service-type
+                                          certbot-activation)
+                       (service-extension mcron-service-type
+                                          certbot-renewal-jobs)))
+                (compose concatenate)
+                (extend (lambda (config additional-hosts)
+                          (certbot-configuration
+                           (inherit config)
+                           (hosts (append (certbot-configuration-hosts config)
+                                          additional-hosts)))))))
-- 
2.12.2

bug#26685: certbot service

[Clément Lassieur]

Hi Andy,

Thanks for working on this!

> +;;; GNU Guix --- Functional package management for GNU
> +;;; Copyright © 2016 ng0 <ng0@we.make.ritual.n0.is>
> +;;; Copyright © 2016 Sou Bunnbu <iyzsong@member.fsf.org>

Or maybe you didn't work on this?..

> +(define certbot-renewal-jobs
> +  (match-lambda
> +    (($ <certbot-configuration> package webroot hosts default-location)
> +     (match hosts
> +       ;; Avoid pinging certbot if we have no hosts.
> +       (() '())
> +       (_
> +        (list
> +         ;; Attempt to renew the certificates twice a week.
> +         #~(job (lambda (now)
> +                  (next-day-from (next-hour-from now '(3))
> +                                 '(2 5)))

This is not twice a week, but twice a month at days 2 and 5, because
'next-day-from' will look after the next day (in month) that has number
2 and 5.  'next-hour-from' is not taken into account because next day
from any hour still runs at 0 o'clock.

But anyway I think it should be twice a day, and at a random minute
within the hour, as advised by certbot:

--8<---------------cut here---------------start------------->8---
from https://certbot.eff.org/all-instructions/

if you're setting up a cron or systemd job, we recommend running it
twice per day (it won't do anything until your certificates are due for
renewal or revoked, but running it regularly would give your site a
chance of staying online in case a Let's Encrypt-initiated revocation
happened for some reason). Please select a random minute within the hour
for your renewal tasks.
--8<---------------cut here---------------end--------------->8---

What do you think of:

    '(next-minute-from (next-hour '(0 12)) (list (random 60)))

instead?  Schedules can be debbuged with '--schedule=count' option.

Also I think some services have to be reloaded/restarted after their
certificates are upgraded.  That could be done via a mcron post-hook,
but I'm not sure how to pass the list of services that have to be
restarted.  WDYT?

Clément

bug#26685: certbot service

[Andy Wingo]

On Fri 28 Apr 2017 11:24, Clément Lassieur <clement@lassieur.org> writes:

> Hi Andy,
>
> Thanks for working on this!

Thanks for the review :-)

>> +;;; GNU Guix --- Functional package management for GNU
>> +;;; Copyright © 2016 ng0 <ng0@we.make.ritual.n0.is>
>> +;;; Copyright © 2016 Sou Bunnbu <iyzsong@member.fsf.org>
>
> Or maybe you didn't work on this?..

I did but also parts of it came from:

  https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/bayfront.scm

I should fix the attribution I guess!

>> +(define certbot-renewal-jobs
>> +  (match-lambda
>> +    (($ <certbot-configuration> package webroot hosts default-location)
>> +     (match hosts
>> +       ;; Avoid pinging certbot if we have no hosts.
>> +       (() '())
>> +       (_
>> +        (list
>> +         ;; Attempt to renew the certificates twice a week.
>> +         #~(job (lambda (now)
>> +                  (next-day-from (next-hour-from now '(3))
>> +                                 '(2 5)))
>
> This is not twice a week, but twice a month at days 2 and 5, because
> 'next-day-from' will look after the next day (in month) that has number
> 2 and 5.  'next-hour-from' is not taken into account because next day
> from any hour still runs at 0 o'clock.
>
> But anyway I think it should be twice a day, and at a random minute
> within the hour, as advised by certbot:
>
> from https://certbot.eff.org/all-instructions/
>
> if you're setting up a cron or systemd job, we recommend running it
> twice per day (it won't do anything until your certificates are due for
> renewal or revoked, but running it regularly would give your site a
> chance of staying online in case a Let's Encrypt-initiated revocation
> happened for some reason). Please select a random minute within the hour
> for your renewal tasks.
>
> What do you think of:
>
>     '(next-minute-from (next-hour '(0 12)) (list (random 60)))
>
> instead?  Schedules can be debbuged with '--schedule=count' option.

Sounds fine to me!

> Also I think some services have to be reloaded/restarted after their
> certificates are upgraded.  That could be done via a mcron post-hook,
> but I'm not sure how to pass the list of services that have to be
> restarted.  WDYT?

Good question.  I don't even know how to know when running certbot
results in a rotation and when it leaves things as-is.  It's a great
question though!  BTW if you are interested in using/hacking on this, I
think we could just get it into master and patch it from there.  WDYT?

Andy

bug#26685: certbot service

[Clément Lassieur]

Andy Wingo <wingo@igalia.com> writes:

> BTW if you are interested in using/hacking on this, I think we could
> just get it into master and patch it from there.  WDYT?

Ok!

bug#26685: certbot service

[Andy Wingo]

On Sat 29 Apr 2017 11:14, Clément Lassieur <clement@lassieur.org> writes:

> Andy Wingo <wingo@igalia.com> writes:
>
>> BTW if you are interested in using/hacking on this, I think we could
>> just get it into master and patch it from there.  WDYT?
>
> Ok!

Cool :)  I guess to do this, we need #26684 first.  Would you mind
reviewing it?

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26684

Maybe there is a nicer way to do that, but it seems like a useful,
not-incompatible change to make, so for me it seems safe to go in.  YMMV
though :)

Andy

bug#26685: certbot service

[Clément Lassieur]

Andy Wingo <wingo@igalia.com> writes:

> On Sat 29 Apr 2017 11:14, Clément Lassieur <clement@lassieur.org> writes:
>
>> Andy Wingo <wingo@igalia.com> writes:
>>
>>> BTW if you are interested in using/hacking on this, I think we could
>>> just get it into master and patch it from there.  WDYT?
>>
>> Ok!
>
> Cool :)  I guess to do this, we need #26684 first.  Would you mind
> reviewing it?

Sure!  I'm doing it.

>   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26684
>
> Maybe there is a nicer way to do that, but it seems like a useful,
> not-incompatible change to make, so for me it seems safe to go in.  YMMV
> though :)
>
> Andy

bug#26685: certbot service

[Clément Lassieur]

Andy Wingo <wingo@igalia.com> writes:
> On Fri 28 Apr 2017 11:24, Clément Lassieur <clement@lassieur.org> writes:

>> Also I think some services have to be reloaded/restarted after their
>> certificates are upgraded.  That could be done via a mcron post-hook,
I meant certbot --post-hook, not mcron.

>> but I'm not sure how to pass the list of services that have to be
>> restarted.  WDYT?

> Good question.  I don't even know how to know when running certbot
> results in a rotation and when it leaves things as-is.  It's a great
> question though!

Actually I was wrong about certbot --post-hook: there is a certbot
--renew-hook.  This one gets called after every successful renewal.
Thus in this hook we should restart/reload services.

bug#26685: certbot service

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 502 bytes --]

On Fri, Apr 28, 2017 at 11:24:47AM +0200, Clément Lassieur wrote:
> Also I think some services have to be reloaded/restarted after their
> certificates are upgraded.  That could be done via a mcron post-hook,
> but I'm not sure how to pass the list of services that have to be
> restarted.  WDYT?

I don't have the answer either, but this is a prime use case for
implementing `nginx reload` in the nginx-service. Otherwise, nginx will
have to be killed in order to deploy the new certificate.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#26685: certbot service

[Clément Lassieur]

Leo Famulari <leo@famulari.name> writes:

> On Fri, Apr 28, 2017 at 11:24:47AM +0200, Clément Lassieur wrote:
>> Also I think some services have to be reloaded/restarted after their
>> certificates are upgraded.  That could be done via a mcron post-hook,
>> but I'm not sure how to pass the list of services that have to be
>> restarted.  WDYT?
>
> I don't have the answer either, but this is a prime use case for
> implementing `nginx reload` in the nginx-service. Otherwise, nginx will
> have to be killed in order to deploy the new certificate.

But this is not just about nginx right?  (Otherwise 'nginx -s reload'
would do the job, I think.)  It is about every service that has its
certificates renewed by certbot.  Don't we need a generic way to reload
services, like 'herd reload service'?  I will have a look at how we
could implement this.

certbot service experience

[Christopher Allan Webber]

I'm crossposting this to guix-devel, even though it's in reply to
guix-patches bug #26685 adding the cerbot service, because I think it's
more about my experiences with workflow and less about what might affect
that specific bug.  (If you reply on guix-devel, maybe remove the
debbugs address.)

Andy Wingo writes:

> Attached patch adds a certbot service extending nginx.  Only question
> is, how to ensure that nginx is runing when the certbot activation runs?
> In practice I bet this races so it's not a big issue but if there's a
> way to require nginx before activation, that would be nice.

So I got a server up and running using certbot and this.  Yay!  It went
pretty well, and I'm happy about it now that it's running, though I did
run into some bumps (which is okay, I ran into bumps doing this on
Debian too, I just think now's a good time to reflect on the experience
and see if we can do anything better... plus maybe it's useful to
explore for someone else too).  I figured I'd document what the process
was like for me.  Here's my observations:

 - I was surprised that I was prompted for an email while doing guix
   system reconfigure (and in being asked if I was willing to supply
   that info to the EFF so they can put me on their mailing lists, which
   I'm not against but was surprising to encounter in a reconfigure).
   IIRC this is the first time anything like that has happened.  I
   generally anticipate such a reconfigure to be "prompt-less".
   I wonder if there should be an email field we can provide, so that it
   doesn't do this prompt?  Granted, it's for the first time run only.
   Maybe it's okay to do things as they are, but we should document it?
   I dunno?

 - Not specifically something to do with this package, but nginx wasn't
   coming up at a few points (btw, default nginx configuration won't
   work, because it points at certificate filepaths that we don't
   "automatically" put in place).  When nginx doesn't come up, there's
   pretty much no information as to why; nothing in the shepherd logs,
   nothing useful from shepherd itself ("shepherd: Service nginx could
   not be started."), and nothing in the nginx logs either.

 - Getting the certbot stuff to work basically involved three stages:

   1) Enable nginx without cert certbot, and point the root somewhere
      which will be the same place you'll set certbot-configuration's
      webroot at in the next step.  This looks something like the
      following:

        (service nginx-service-type
          (nginx-configuration
           (server-blocks
            (list
             (nginx-server-configuration
              ;; Replace these with your own domain and web root
              (server-name '("test.activitypub.rocks"))
              (root "/srv/activitypub.rocks/site/")
              ;; Note that you have to disable https and any certificates
              ;; esp because our nginx config defaults to pointing at keys
              ;; that probably aren't there.  Without doing this nginx
              ;; will die mysteriously.
              (https-port #f)
              (ssl-certificate #f)
              (ssl-certificate-key #f))))))

      (You can maybe do this in the same phase as the next step but
      there's a risk of a race condition I think.)

      Anyway now do "guix system reconfigure" and nginx should come up
      and serve your stuff *without* https.

  2) Enable the certbot-service-type (and mcron-service-type if you
     haven't already):

       (service certbot-service-type
                (certbot-configuration
                 ;; Replace these with your own domain and web root
                 (hosts '("test.activitypub.rocks"))
                 (webroot "/srv/activitypub.rocks/site/")))
       ;; if you don't have an mcron service already
       (service mcron-service-type)

     Okay, with those added, run "guix system reconfigure" again.
     You'll be prompted for your email address and about whether or not
     you want to join the EFF mailing lists, so watch for that.

  3) Okay hopefully that went successfully!  It should say.  Assuming it
     did, *now* we can add the keys appropriately to the nginx config.

       (service nginx-service-type
                (nginx-configuration
                 (server-blocks
                  (list
                   (nginx-server-configuration
                    ;; Again, adjust to your site
                    (server-name '("test.activitypub.rocks"))
                    (root "/srv/activitypub.rocks/site/")
                    (ssl-certificate
                     "/etc/letsencrypt/live/test.activitypub.rocks/fullchain.pem")
                    (ssl-certificate-key
                     "/etc/letsencrypt/live/test.activitypub.rocks/privkey.pem"))))))

     Reconfigure and cross your fingers!

  4) At this point I was surprised that it seemed like nginx should have
     been working with https since everything was in place, but I
     couldn't access it from my browser over https.  Frustrated, I
     restarted the server.

     And then it worked! :)

So, this involved reconfiguring, reconfiguring, reconfiguring, and then
a restart, then it worked for me.  (Well, plus a few reconfigures where
nothing worked at all because I broke things of course. ;))  I wonder if
that can be improved?

That said, it's still really exciting to be able to describe these
things declaratively, and to have Guix take care of keeping things
renewed for me. :)  Excited to have this landing, and to be that much
closer to doing server deployment with GuixSD!

Re: certbot service experience

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 2981 bytes --]

Christopher Allan Webber <cwebber@dustycloud.org> writes:

>  - I was surprised that I was prompted for an email while doing guix
>    system reconfigure

That does seem odd.  Why were you prompted for an email address?  Can
that be fixed somehow?

>   2) Enable the certbot-service-type (and mcron-service-type if you
>      haven't already):
>
>        (service certbot-service-type
>                 (certbot-configuration
>                  ;; Replace these with your own domain and web root
>                  (hosts '("test.activitypub.rocks"))
>                  (webroot "/srv/activitypub.rocks/site/")))
>        ;; if you don't have an mcron service already
>        (service mcron-service-type)

Where is the certbot-service-type defined?  I couldn't find it in the
master branch.  Also, why is mcron required?  I don't know much about
LetsEncrypt, but I thought certbot was a one-time thing that you do
manually...  Why is it a "service" here?

>   3) Okay hopefully that went successfully!  It should say.  Assuming it
>      did, *now* we can add the keys appropriately to the nginx config.
>
>        (service nginx-service-type
>                 (nginx-configuration
>                  (server-blocks
>                   (list
>                    (nginx-server-configuration
>                     ;; Again, adjust to your site
>                     (server-name '("test.activitypub.rocks"))
>                     (root "/srv/activitypub.rocks/site/")
>                     (ssl-certificate
>                      "/etc/letsencrypt/live/test.activitypub.rocks/fullchain.pem")
>                     (ssl-certificate-key
>                      "/etc/letsencrypt/live/test.activitypub.rocks/privkey.pem"))))))
>
>      Reconfigure and cross your fingers!
>
>   4) At this point I was surprised that it seemed like nginx should have
>      been working with https since everything was in place, but I
>      couldn't access it from my browser over https.  Frustrated, I
>      restarted the server.
>
>      And then it worked! :)
>
> So, this involved reconfiguring, reconfiguring, reconfiguring, and then
> a restart, then it worked for me.  (Well, plus a few reconfigures where
> nothing worked at all because I broke things of course. ;))  I wonder if
> that can be improved?

I wonder if it is possible to define a custom service which orchestrates
the execution of nginx and certbot in the way you require, so that you
can define it all in one place, at once, without needing to reconfigure
multiple times?

> That said, it's still really exciting to be able to describe these
> things declaratively, and to have Guix take care of keeping things
> renewed for me. :)  Excited to have this landing, and to be that much
> closer to doing server deployment with GuixSD!

Pretty cool!  Thanks for sharing your experience.  It's always neat to
read about how people are using the system.

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: certbot service experience

[Christopher Allan Webber]

[-bug]

Chris Marusich writes:

> Christopher Allan Webber <cwebber@dustycloud.org> writes:
>
>>  - I was surprised that I was prompted for an email while doing guix
>>    system reconfigure
>
> That does seem odd.  Why were you prompted for an email address?  Can
> that be fixed somehow?

It's the certbot initial-setup script firing off here.  The email
address is given so you can be notified for security updates, etc.
Maybe recovery?  Don't remember about that last bit.

>>   2) Enable the certbot-service-type (and mcron-service-type if you
>>      haven't already):
>>
>>        (service certbot-service-type
>>                 (certbot-configuration
>>                  ;; Replace these with your own domain and web root
>>                  (hosts '("test.activitypub.rocks"))
>>                  (webroot "/srv/activitypub.rocks/site/")))
>>        ;; if you don't have an mcron service already
>>        (service mcron-service-type)
>
> Where is the certbot-service-type defined?  I couldn't find it in the
> master branch.  Also, why is mcron required?  I don't know much about
> LetsEncrypt, but I thought certbot was a one-time thing that you do
> manually...  Why is it a "service" here?

It's not in the master branch is why.  I was looking at the
wip-git-https branch. :)  I think it's also in the wip-potluck branch.

It's extending the mcron service so it can install an auto-update rule
for you, which is pretty cool!

>>   3) Okay hopefully that went successfully!  It should say.  Assuming it
>>      did, *now* we can add the keys appropriately to the nginx config.
>>
>>        (service nginx-service-type
>>                 (nginx-configuration
>>                  (server-blocks
>>                   (list
>>                    (nginx-server-configuration
>>                     ;; Again, adjust to your site
>>                     (server-name '("test.activitypub.rocks"))
>>                     (root "/srv/activitypub.rocks/site/")
>>                     (ssl-certificate
>>                      "/etc/letsencrypt/live/test.activitypub.rocks/fullchain.pem")
>>                     (ssl-certificate-key
>>                      "/etc/letsencrypt/live/test.activitypub.rocks/privkey.pem"))))))
>>
>>      Reconfigure and cross your fingers!
>>
>>   4) At this point I was surprised that it seemed like nginx should have
>>      been working with https since everything was in place, but I
>>      couldn't access it from my browser over https.  Frustrated, I
>>      restarted the server.
>>
>>      And then it worked! :)
>>
>> So, this involved reconfiguring, reconfiguring, reconfiguring, and then
>> a restart, then it worked for me.  (Well, plus a few reconfigures where
>> nothing worked at all because I broke things of course. ;))  I wonder if
>> that can be improved?
>
> I wonder if it is possible to define a custom service which orchestrates
> the execution of nginx and certbot in the way you require, so that you
> can define it all in one place, at once, without needing to reconfigure
> multiple times?

It might be, I dunno!  Maybe in starting the service, if it sees that
the keys have not been generated yet, it pulls up nginx temporarily just
to do the registration with the nginx that doesn't have the keys in it,
does the generation of the keys and verifies them with letsencrypt, then
pulls up the proper nginx at last.

I don't know how easy/feasible this is.

>> That said, it's still really exciting to be able to describe these
>> things declaratively, and to have Guix take care of keeping things
>> renewed for me. :)  Excited to have this landing, and to be that much
>> closer to doing server deployment with GuixSD!
>
> Pretty cool!  Thanks for sharing your experience.  It's always neat to
> read about how people are using the system.

Yeah!  I'm excited to finally move to hosting my servers with GuixSD at
last, after ages of talking about it. ;)

Re: certbot service experience

[Chris Marusich]

[-- Attachment #1: Type: text/plain, Size: 4460 bytes --]

Christopher Allan Webber <cwebber@dustycloud.org> writes:

> [-bug]
>
> Chris Marusich writes:
>
>> Christopher Allan Webber <cwebber@dustycloud.org> writes:
>>
>>>  - I was surprised that I was prompted for an email while doing guix
>>>    system reconfigure
>>
>> That does seem odd.  Why were you prompted for an email address?  Can
>> that be fixed somehow?
>
> It's the certbot initial-setup script firing off here.  The email
> address is given so you can be notified for security updates, etc.
> Maybe recovery?  Don't remember about that last bit.

I see.  Having never used certbot before myself, I don't think I can
really give useful advice here.  It just seems a little strange to
require user interaction during the reconfigure. One would think that
one could provide any necessary information declaratively in the
operating system configuration file.

>
>>>   2) Enable the certbot-service-type (and mcron-service-type if you
>>>      haven't already):
>>>
>>>        (service certbot-service-type
>>>                 (certbot-configuration
>>>                  ;; Replace these with your own domain and web root
>>>                  (hosts '("test.activitypub.rocks"))
>>>                  (webroot "/srv/activitypub.rocks/site/")))
>>>        ;; if you don't have an mcron service already
>>>        (service mcron-service-type)
>>
>> Where is the certbot-service-type defined?  I couldn't find it in the
>> master branch.  Also, why is mcron required?  I don't know much about
>> LetsEncrypt, but I thought certbot was a one-time thing that you do
>> manually...  Why is it a "service" here?
>
> It's not in the master branch is why.  I was looking at the
> wip-git-https branch. :)  I think it's also in the wip-potluck branch.
>

That explains it!  :)

>
> It's extending the mcron service so it can install an auto-update rule
> for you, which is pretty cool!


I see.  I would have thought that if service A extends service B, and
one defines service A in the operating system configuration file,
service B gets pulled in automatically.  Is that not true?

>>>   3) Okay hopefully that went successfully!  It should say.  Assuming it
>>>      did, *now* we can add the keys appropriately to the nginx config.
>>>
>>>        (service nginx-service-type
>>>                 (nginx-configuration
>>>                  (server-blocks
>>>                   (list
>>>                    (nginx-server-configuration
>>>                     ;; Again, adjust to your site
>>>                     (server-name '("test.activitypub.rocks"))
>>>                     (root "/srv/activitypub.rocks/site/")
>>>                     (ssl-certificate
>>>                      "/etc/letsencrypt/live/test.activitypub.rocks/fullchain.pem")
>>>                     (ssl-certificate-key
>>>                      "/etc/letsencrypt/live/test.activitypub.rocks/privkey.pem"))))))
>>>
>>>      Reconfigure and cross your fingers!
>>>
>>>   4) At this point I was surprised that it seemed like nginx should have
>>>      been working with https since everything was in place, but I
>>>      couldn't access it from my browser over https.  Frustrated, I
>>>      restarted the server.
>>>
>>>      And then it worked! :)
>>>
>>> So, this involved reconfiguring, reconfiguring, reconfiguring, and then
>>> a restart, then it worked for me.  (Well, plus a few reconfigures where
>>> nothing worked at all because I broke things of course. ;))  I wonder if
>>> that can be improved?
>>
>> I wonder if it is possible to define a custom service which orchestrates
>> the execution of nginx and certbot in the way you require, so that you
>> can define it all in one place, at once, without needing to reconfigure
>> multiple times?
>
> It might be, I dunno!  Maybe in starting the service, if it sees that
> the keys have not been generated yet, it pulls up nginx temporarily just
> to do the registration with the nginx that doesn't have the keys in it,
> does the generation of the keys and verifies them with letsencrypt, then
> pulls up the proper nginx at last.
>
> I don't know how easy/feasible this is.

It would be nice if we could just declare what we want in the operating
system configuration file, and get it without having to run multiple
reconfigures or interact with the user.  But hey, anything that works is
a step in the right direction!

-- 
Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: certbot service experience

[Andy Wingo]

On Sat 29 Apr 2017 23:33, Christopher Allan Webber <cwebber@dustycloud.org> writes:

> I'm crossposting this to guix-devel, even though it's in reply to
> guix-patches bug #26685 adding the cerbot service, because I think it's
> more about my experiences with workflow and less about what might affect
> that specific bug.  (If you reply on guix-devel, maybe remove the
> debbugs address.)

Oh, I think this does bear on the service.  Thanks for the detailed
report!

>  - I was surprised that I was prompted for an email while doing guix
>    system reconfigure (and in being asked if I was willing to supply
>    that info to the EFF so they can put me on their mailing lists, which
>    I'm not against but was surprising to encounter in a reconfigure).
>    IIRC this is the first time anything like that has happened.  I
>    generally anticipate such a reconfigure to be "prompt-less".
>    I wonder if there should be an email field we can provide, so that it
>    doesn't do this prompt?  Granted, it's for the first time run only.
>    Maybe it's okay to do things as they are, but we should document it?
>    I dunno?

Yeah I don't know :)  I mean, good that it worked.  I hadn't tested this
bit entirely -- I didn't have a spare source of domain names handy (!).
And I forgot about it asking me for my email.

However apparently what happens is that certbot will prompt you for
things if you run it with an interactive terminal.  We can pass
--no-interactive, but AFAIU that will require you to go later and
"register" your account with an after-step -- and if you miss that you
have to do that, perhaps your cert stops working soon.  Not sure what
the best practice is here.  In any case we need to document this!

>  - Not specifically something to do with this package, but nginx wasn't
>    coming up at a few points (btw, default nginx configuration won't
>    work, because it points at certificate filepaths that we don't
>    "automatically" put in place).  When nginx doesn't come up, there's
>    pretty much no information as to why; nothing in the shepherd logs,
>    nothing useful from shepherd itself ("shepherd: Service nginx could
>    not be started."), and nothing in the nginx logs either.

I think the default nginx configuration works, because it defines no
servers.  If you run just (nginx-configuration), then certbot extends it
correctly to allow it to request a certificate.  You are right that the
default server config does want to listen on HTTPS and it will balk
if/when there is no cert.  (Does that prevent the whole nginx server
from running?  That would be a bummer.)

>  - Getting the certbot stuff to work basically involved three stages:
>
>    1) Enable nginx without cert certbot, and point the root somewhere
>       which will be the same place you'll set certbot-configuration's
>       webroot at in the next step.  This looks something like the
>       following:
>
>         (service nginx-service-type
>           (nginx-configuration
>            (server-blocks
>             (list
>              (nginx-server-configuration
>               ;; Replace these with your own domain and web root
>               (server-name '("test.activitypub.rocks"))
>               (root "/srv/activitypub.rocks/site/")
>               ;; Note that you have to disable https and any certificates
>               ;; esp because our nginx config defaults to pointing at keys
>               ;; that probably aren't there.  Without doing this nginx
>               ;; will die mysteriously.
>               (https-port #f)
>               (ssl-certificate #f)
>               (ssl-certificate-key #f))))))

Pretty sure you could do this with just the default nginx-configuration
and no sites configured and having the certbot service.

>   3) Okay hopefully that went successfully!  It should say.  Assuming it
>      did, *now* we can add the keys appropriately to the nginx config.
>
>        (service nginx-service-type
>                 (nginx-configuration
>                  (server-blocks
>                   (list
>                    (nginx-server-configuration
>                     ;; Again, adjust to your site
>                     (server-name '("test.activitypub.rocks"))
>                     (root "/srv/activitypub.rocks/site/")
>                     (ssl-certificate
>                      "/etc/letsencrypt/live/test.activitypub.rocks/fullchain.pem")
>                     (ssl-certificate-key
>                      "/etc/letsencrypt/live/test.activitypub.rocks/privkey.pem"))))))

Now it works yes :)

>   4) At this point I was surprised that it seemed like nginx should have
>      been working with https since everything was in place, but I
>      couldn't access it from my browser over https.  Frustrated, I
>      restarted the server.
>
>      And then it worked! :)

Incidentally I found this a few times too.  Like you change the
nginx config but the nginx server never restarts because somehow
shepherd wasn't updated with the new config.  Why would that be?  I
didn't understand it either and just restarted the server.  Oh well.

Thanks again for the feedback!  Definitely needs to be better documented
I guess.  We need to make sure nginx can partially start, and if that's
not the case, then introduce a dependency between nginx start and
certbot run, and arrange for the initial certbot run to make its own web
server for that initial phase.

Andy

Re: certbot service experience

[Christopher Allan Webber]

Andy Wingo writes:

> On Sat 29 Apr 2017 23:33, Christopher Allan Webber <cwebber@dustycloud.org> writes:
>
>> I'm crossposting this to guix-devel, even though it's in reply to
>> guix-patches bug #26685 adding the cerbot service, because I think it's
>> more about my experiences with workflow and less about what might affect
>> that specific bug.  (If you reply on guix-devel, maybe remove the
>> debbugs address.)
>
> Oh, I think this does bear on the service.  Thanks for the detailed
> report!

Glad it helps!

>>  - I was surprised that I was prompted for an email while doing guix
>>    system reconfigure (and in being asked if I was willing to supply
>>    that info to the EFF so they can put me on their mailing lists, which
>>    I'm not against but was surprising to encounter in a reconfigure).
>>    IIRC this is the first time anything like that has happened.  I
>>    generally anticipate such a reconfigure to be "prompt-less".
>>    I wonder if there should be an email field we can provide, so that it
>>    doesn't do this prompt?  Granted, it's for the first time run only.
>>    Maybe it's okay to do things as they are, but we should document it?
>>    I dunno?
>
> Yeah I don't know :)  I mean, good that it worked.  I hadn't tested this
> bit entirely -- I didn't have a spare source of domain names handy (!).
> And I forgot about it asking me for my email.
>
> However apparently what happens is that certbot will prompt you for
> things if you run it with an interactive terminal.  We can pass
> --no-interactive, but AFAIU that will require you to go later and
> "register" your account with an after-step -- and if you miss that you
> have to do that, perhaps your cert stops working soon.  Not sure what
> the best practice is here.  In any case we need to document this!

Not sure what the best practice is either, but documenting it would be a
step in the right direction!

>>  - Not specifically something to do with this package, but nginx wasn't
>>    coming up at a few points (btw, default nginx configuration won't
>>    work, because it points at certificate filepaths that we don't
>>    "automatically" put in place).  When nginx doesn't come up, there's
>>    pretty much no information as to why; nothing in the shepherd logs,
>>    nothing useful from shepherd itself ("shepherd: Service nginx could
>>    not be started."), and nothing in the nginx logs either.
>
> I think the default nginx configuration works, because it defines no
> servers.  If you run just (nginx-configuration), then certbot extends it
> correctly to allow it to request a certificate.  You are right that the
> default server config does want to listen on HTTPS and it will balk
> if/when there is no cert.  (Does that prevent the whole nginx server
> from running?  That would be a bummer.)
> [...]
> Pretty sure you could do this with just the default nginx-configuration
> and no sites configured and having the certbot service.
>

Oh, I didn't realize certbot was extending the nginx config...!  Clever.

>>   4) At this point I was surprised that it seemed like nginx should have
>>      been working with https since everything was in place, but I
>>      couldn't access it from my browser over https.  Frustrated, I
>>      restarted the server.
>>
>>      And then it worked! :)
>
> Incidentally I found this a few times too.  Like you change the
> nginx config but the nginx server never restarts because somehow
> shepherd wasn't updated with the new config.  Why would that be?  I
> didn't understand it either and just restarted the server.  Oh well.
>
> Thanks again for the feedback!  Definitely needs to be better documented
> I guess.  We need to make sure nginx can partially start, and if that's
> not the case, then introduce a dependency between nginx start and
> certbot run, and arrange for the initial certbot run to make its own web
> server for that initial phase.

Cool.  I have no idea how to do that with current shepherd setup!  But
maybe it doesn't need to be complex; maybe these tests can just be part
of the service start-up recipe or something.

Thanks for the thoughtful reply :)

[bug#26685] certbot service experience

[Ludovic Courtès]

Hello Andy, Chris, and Clément!  :-)

Could we decide on what really needs to be done before we push this
patch, and what can be done later?  It’s a pity that this patch has been
sitting there for so long despite being very useful!

  https://bugs.gnu.org/26685

TIA.  :-)

Ludo’.

[bug#26685] certbot service experience

[Christopher Allan Webber]

Ludovic Courtès writes:

> Hello Andy, Chris, and Clément!  :-)
>
> Could we decide on what really needs to be done before we push this
> patch, and what can be done later?  It’s a pity that this patch has been
> sitting there for so long despite being very useful!
>
>   https://bugs.gnu.org/26685
>
> TIA.  :-)
>
> Ludo’.

I'll reply to the issue; the one thing I think that we should fix for
sure is that it prompts for the email during guix system reconfigure.
There should be a way to pass this in as an option... IMO having prompts
there is unexpected!  But maybe even if that can't be done soon we
should push it and open the email thing as another bug.

And yes I was thinking the same thing; I'm still running a server using
this, so upgrading it is a bit awkward with the stale branch.

[bug#26685] certbot service experience

[ng0]

[-- Attachment #1: Type: text/plain, Size: 1449 bytes --]

Christopher Allan Webber transcribed 0.8K bytes:
> Ludovic Courtès writes:
> 
> > Hello Andy, Chris, and Clément!  :-)
> >
> > Could we decide on what really needs to be done before we push this
> > patch, and what can be done later?  It’s a pity that this patch has been
> > sitting there for so long despite being very useful!
> >
> >   https://bugs.gnu.org/26685
> >
> > TIA.  :-)
> >
> > Ludo’.
> 
> I'll reply to the issue; the one thing I think that we should fix for
> sure is that it prompts for the email during guix system reconfigure.
> There should be a way to pass this in as an option... IMO having prompts

I think this can be configured in a file for letsencrypt, at least it
keeps a config file where common options can be defined, and this
config file can even be passed as argument, so no fixed location for
it if I understood it right after quick reading of the documents.

reference documentation:
https://letsencrypt.readthedocs.io/en/latest/using.html

> there is unexpected!  But maybe even if that can't be done soon we
> should push it and open the email thing as another bug.
> 
> And yes I was thinking the same thing; I'm still running a server using
> this, so upgrading it is a bit awkward with the stale branch.
> 
> 
> 
> 

-- 
ng0
GnuPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588
GnuPG: https://n0is.noblogs.org/my-keys
https://www.infotropique.org https://krosos.org

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#26685] certbot service experience

[ng0]

[-- Attachment #1: Type: text/plain, Size: 1981 bytes --]

ng0 transcribed 2.5K bytes:
> Christopher Allan Webber transcribed 0.8K bytes:
> > Ludovic Courtès writes:
> > 
> > > Hello Andy, Chris, and Clément!  :-)
> > >
> > > Could we decide on what really needs to be done before we push this
> > > patch, and what can be done later?  It’s a pity that this patch has been
> > > sitting there for so long despite being very useful!
> > >
> > >   https://bugs.gnu.org/26685
> > >
> > > TIA.  :-)
> > >
> > > Ludo’.
> > 
> > I'll reply to the issue; the one thing I think that we should fix for
> > sure is that it prompts for the email during guix system reconfigure.
> > There should be a way to pass this in as an option... IMO having prompts
> 
> I think this can be configured in a file for letsencrypt, at least it
> keeps a config file where common options can be defined, and this
> config file can even be passed as argument, so no fixed location for
> it if I understood it right after quick reading of the documents.

Sorry, there are default locations:

  By default, the following locations are searched:

  /etc/letsencrypt/cli.ini
  $XDG_CONFIG_HOME/letsencrypt/cli.ini (or ~/.config/letsencrypt/cli.ini if $XDG_CONFIG_HOME is not set).

But I think this already helps.

> reference documentation:
> https://letsencrypt.readthedocs.io/en/latest/using.html
> 
> > there is unexpected!  But maybe even if that can't be done soon we
> > should push it and open the email thing as another bug.
> > 
> > And yes I was thinking the same thing; I'm still running a server using
> > this, so upgrading it is a bit awkward with the stale branch.
> > 
> > 
> > 
> > 
> 
> -- 
> ng0
> GnuPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588
> GnuPG: https://n0is.noblogs.org/my-keys
> https://www.infotropique.org https://krosos.org



-- 
ng0
GnuPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588
GnuPG: https://n0is.noblogs.org/my-keys
https://www.infotropique.org https://krosos.org

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#26685] certbot service experience

[Julien Lepiller]

Le Sun, 30 Jul 2017 09:22:12 +0000,
ng0 <ng0@infotropique.org> a écrit :

> Sorry, there are default locations:
> 
>   By default, the following locations are searched:
> 
>   /etc/letsencrypt/cli.ini
>   $XDG_CONFIG_HOME/letsencrypt/cli.ini (or
> ~/.config/letsencrypt/cli.ini if $XDG_CONFIG_HOME is not set).
> 
> But I think this already helps.

We could use the --config flag and pass the location of a generated
config file as the argument.

> 
> > reference documentation:
> > https://letsencrypt.readthedocs.io/en/latest/using.html
> >   

[bug#26685] certbot service experience

[Tobias Geerinckx-Rice]

[-- Attachment #1.1: Type: text/plain, Size: 699 bytes --]

Ludovic Courtès wrote on 26/07/17 at 10:59:
> Hello Andy, Chris, and Clément!  :-)

...and Ludo',

> Could we decide on what really needs to be done before we push this
> patch, and what can be done later?  It’s a pity that this patch has been
> sitting there for so long despite being very useful!

Zut alors. I missed that service! Nice. Necessary.

>   https://bugs.gnu.org/26685

If nobody objects, I'd like a few days to play with this before it gets
merged. It's a fine service, but I think it privileges the ‘--webroot’
plugin too much (‘-w’ is a plugin-specific option, not global). I'd
rather not have my mail box spin up nginx...

Kind regards,

T G-R



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 248 bytes --]

[bug#26685] certbot service experience

[Ludovic Courtès]

Hi there!

Tobias Geerinckx-Rice <me@tobias.gr> skribis:

> Ludovic Courtès wrote on 26/07/17 at 10:59:
>> Hello Andy, Chris, and Clément!  :-)
>
> ...and Ludo',
>
>> Could we decide on what really needs to be done before we push this
>> patch, and what can be done later?  It’s a pity that this patch has been
>> sitting there for so long despite being very useful!
>
> Zut alors. I missed that service! Nice. Necessary.
>
>>   https://bugs.gnu.org/26685
>
> If nobody objects, I'd like a few days to play with this before it gets
> merged. It's a fine service, but I think it privileges the ‘--webroot’
> plugin too much (‘-w’ is a plugin-specific option, not global). I'd
> rather not have my mail box spin up nginx...

Any update on this?  :-)

Ludo’.

[bug#26685] certbot service experience

[Christopher Allan Webber]

Ludovic Courtès writes:

> Hi there!
>
> Tobias Geerinckx-Rice <me@tobias.gr> skribis:
>
>> Ludovic Courtès wrote on 26/07/17 at 10:59:
>>> Hello Andy, Chris, and Clément!  :-)
>>
>> ...and Ludo',
>>
>>> Could we decide on what really needs to be done before we push this
>>> patch, and what can be done later?  It’s a pity that this patch has been
>>> sitting there for so long despite being very useful!
>>
>> Zut alors. I missed that service! Nice. Necessary.
>>
>>>   https://bugs.gnu.org/26685
>>
>> If nobody objects, I'd like a few days to play with this before it gets
>> merged. It's a fine service, but I think it privileges the ‘--webroot’
>> plugin too much (‘-w’ is a plugin-specific option, not global). I'd
>> rather not have my mail box spin up nginx...
>
> Any update on this?  :-)
>
> Ludo’.

My feeling is that it's a good idea to allow for non-webroot
configuration, but I think it shouldn't hold us back from moving forward
with getting the certbot service in.

[bug#26685] certbot service experience

[Christopher Allan Webber]

Christopher Allan Webber writes:

> Ludovic Courtès writes:
>
>> Hi there!
>>
>> Tobias Geerinckx-Rice <me@tobias.gr> skribis:
>>
>>> Ludovic Courtès wrote on 26/07/17 at 10:59:
>>>> Hello Andy, Chris, and Clément!  :-)
>>>
>>> ...and Ludo',
>>>
>>>> Could we decide on what really needs to be done before we push this
>>>> patch, and what can be done later?  It’s a pity that this patch has been
>>>> sitting there for so long despite being very useful!
>>>
>>> Zut alors. I missed that service! Nice. Necessary.
>>>
>>>>   https://bugs.gnu.org/26685
>>>
>>> If nobody objects, I'd like a few days to play with this before it gets
>>> merged. It's a fine service, but I think it privileges the ‘--webroot’
>>> plugin too much (‘-w’ is a plugin-specific option, not global). I'd
>>> rather not have my mail box spin up nginx...
>>
>> Any update on this?  :-)
>>
>> Ludo’.
>
> My feeling is that it's a good idea to allow for non-webroot
> configuration, but I think it shouldn't hold us back from moving forward
> with getting the certbot service in.

I especially feel this way this morning, when I had to push a server
upgrade out and I suddenly realized I'm about to have to rebase this
branch!

Ludo’, are you okay with me working on getting this into master with
just the webroot plugin for now and marking that the service
configuration interface for this is semi-unstable for now in the manual?
That way we can get this in and get other people to test it, and not
let it languish.

I think the service needs some serious work but we're likely to have
more success getting around the rough edges by having more people using
it anyway.

WDYT?

[bug#26685] certbot service experience

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 293 bytes --]

On Tue, Oct 24, 2017 at 09:26:13AM -0500, Christopher Allan Webber wrote:
> I think the service needs some serious work but we're likely to have
> more success getting around the rough edges by having more people using
> it anyway.

+1

People will show up to help after it has been pushed :)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#26685] certbot service experience

[Ludovic Courtès]

Heya!

Christopher Allan Webber <cwebber@dustycloud.org> skribis:

> Ludo’, are you okay with me working on getting this into master with
> just the webroot plugin for now and marking that the service
> configuration interface for this is semi-unstable for now in the manual?
> That way we can get this in and get other people to test it, and not
> let it languish.

I agree, I’m definitely okay with you pushing this!

Thank you for taking care of it,
Ludo’.

bug#26685: certbot service experience

[Ludovic Courtès]

Hello!

Since everyone was so hesitant I was so blissful ;-), I went ahead and
pushed this patch as commit c2e5d74997bdb57dc35e0189188c6917cd5f0f19!
\o/

I think it’s better than the status quo, because even if it’s not
perfect, it *is* useful as it is.  We can start from here if there are
ideas on how to improve it.

Ludo’.

[bug#26685] certbot service experience

[Christopher Allan Webber]

Ludovic Courtès writes:

> Hello!
>
> Since everyone was so hesitant I was so blissful ;-), I went ahead and
> pushed this patch as commit c2e5d74997bdb57dc35e0189188c6917cd5f0f19!
> \o/
>
> I think it’s better than the status quo, because even if it’s not
> perfect, it *is* useful as it is.  We can start from here if there are
> ideas on how to improve it.
>
> Ludo’.

Yay!  But oh nooooo!  I really should have replied to this!  I had
stopped working on it because I got overloaded with final-push situation
in ActivityPub.

Maybe this is worth moving to the mailing list instead, but one reason I
didn't push it was I had discovered a significant and serious
issue... building a system based off of its config is no longer
determistic based off of the system config alone... because this
checks for the rpesence of nginx-server-configuration-ssl-certificate
at *build* time of the system config.  The following code is the
culprit:

  (define (emit-nginx-server-config server)
    ;; [...]
      (for-each
       (match-lambda
        ((record-key . file)
         (if (and file (not (file-exists? file)))
             (error
              (simple-format
               #f
               "~A in the nginx configuration for the server with name \"~A\" does not exist" record-key server-name)))))
       `(("ssl-certificate"     . ,ssl-certificate)
         ("ssl-certificate-key" . ,ssl-certificate-key)))

Relatedly, the current setup has these defaults:

  (ssl-certificate     nginx-server-configuration-ssl-certificate
                       (default "/etc/nginx/cert.pem"))
  (ssl-certificate-key nginx-server-configuration-ssl-certificate-key
                       (default "/etc/nginx/key.pem"))

IMO they should probably default to #f so that looking for the files
doesn't fail to boot nginx.

[bug#26685] certbot service experience

[Ludovic Courtès]

Hello Chris!

Christopher Allan Webber <cwebber@dustycloud.org> skribis:

> Yay!  But oh nooooo!  I really should have replied to this!  I had
> stopped working on it because I got overloaded with final-push situation
> in ActivityPub.

Understood, no problem!  We’ve had seven months and I thought we’d
rather make progress on this than let everyone re-implement more-or-less
the same service.

> Maybe this is worth moving to the mailing list instead, but one reason I
> didn't push it was I had discovered a significant and serious
> issue... building a system based off of its config is no longer
> determistic based off of the system config alone... because this
> checks for the rpesence of nginx-server-configuration-ssl-certificate
> at *build* time of the system config.  The following code is the
> culprit:

OK, I agree it’s a problem.

That said, the ‘file-exists?’ check comes from an earlier and IIUC
unrelated commit, f2d7a492df759ab7416ff4b244e37896835f04fa.

> Relatedly, the current setup has these defaults:
> 
>   (ssl-certificate     nginx-server-configuration-ssl-certificate
>                        (default "/etc/nginx/cert.pem"))
>   (ssl-certificate-key nginx-server-configuration-ssl-certificate-key
>                        (default "/etc/nginx/key.pem"))
> 
> IMO they should probably default to #f so that looking for the files
> doesn't fail to boot nginx.

I agree (apparently it comes from a Sept. 2016 commit,
8c00b83818acbbc3dfd406465e50439c3418afc5.)

Could you send a patch?  :-)

Thanks,
Ludo’.

[bug#26685] certbot service experience

[Christopher Allan Webber]

Ludovic Courtès writes:

> I agree (apparently it comes from a Sept. 2016 commit,
> 8c00b83818acbbc3dfd406465e50439c3418afc5.)
>
> Could you send a patch?  :-)

Have some things to wrap up this week, but can work on it next week.
Thanks!

[bug#26685] certbot service experience

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 1105 bytes --]

On Thu, Jul 27, 2017 at 07:30:48PM +0200, Tobias Geerinckx-Rice wrote:
> If nobody objects, I'd like a few days to play with this before it gets
> merged. It's a fine service, but I think it privileges the ‘--webroot’
> plugin too much (‘-w’ is a plugin-specific option, not global). I'd
> rather not have my mail box spin up nginx...

I agree that we should, in the long run, offer a more generalized ACME
client service.

However, the --webroot method is not specific to any of the other
plugins. Instead, it is a general purpose method of obtaining and
renewing signed x509 certificates with a running webserver. Certbot
requires no server-specific configuration with this method, and the
server only needs to be configured to serve a particular directory which
will contain the temporary cryptographic "challenge" file. It's not a
very tight coupling.

Since serving HTTPS is, in practice, one of the primary use cases for
the x509 CA system (as opposed to self-signed certs), I think we should
add the service as-is and let people generalize it as they see fit later
on.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#26685] certbot service experience

[Christopher Allan Webber]

Leo Famulari writes:

> On Thu, Jul 27, 2017 at 07:30:48PM +0200, Tobias Geerinckx-Rice wrote:
>> If nobody objects, I'd like a few days to play with this before it gets
>> merged. It's a fine service, but I think it privileges the ‘--webroot’
>> plugin too much (‘-w’ is a plugin-specific option, not global). I'd
>> rather not have my mail box spin up nginx...
>
> I agree that we should, in the long run, offer a more generalized ACME
> client service.
>
> However, the --webroot method is not specific to any of the other
> plugins. Instead, it is a general purpose method of obtaining and
> renewing signed x509 certificates with a running webserver. Certbot
> requires no server-specific configuration with this method, and the
> server only needs to be configured to serve a particular directory which
> will contain the temporary cryptographic "challenge" file. It's not a
> very tight coupling.
>
> Since serving HTTPS is, in practice, one of the primary use cases for
> the x509 CA system (as opposed to self-signed certs), I think we should
> add the service as-is and let people generalize it as they see fit later
> on.

Sounds like the right approach to me.

I'll add a note about the service configuration possibly being unstable
to the docs and push this today.  I just did a rebase on my end.