bug#26685: certbot service

bug#26685: certbot service

[Andy Wingo]

[-- Attachment #1: Type: text/plain, Size: 274 bytes --]

Attached patch adds a certbot service extending nginx.  Only question
is, how to ensure that nginx is runing when the certbot activation runs?
In practice I bet this races so it's not a big issue but if there's a
way to require nginx before activation, that would be nice.


[-- Attachment #2: 0002-gnu-Add-certbot-service.patch --]
[-- Type: text/x-patch, Size: 10989 bytes --]

From 641607bb3ea2b46e80f43216783833c1fc8d70d6 Mon Sep 17 00:00:00 2001
From: Andy Wingo <wingo@igalia.com>
Date: Thu, 27 Apr 2017 14:09:16 +0200
Subject: [PATCH 2/5] gnu: Add certbot service.

* gnu/services/certbot.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add new file.
* doc/guix.texi (Certificate Services): New section.
---
 doc/guix.texi            |  80 +++++++++++++++++++++++++++++
 gnu/local.mk             |   1 +
 gnu/services/certbot.scm | 128 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 209 insertions(+)
 create mode 100644 gnu/services/certbot.scm

diff --git a/doc/guix.texi b/doc/guix.texi
index 0d334e302..21b6d7d88 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -215,6 +215,7 @@ Services
 * Messaging Services::          Messaging services.
 * Kerberos Services::           Kerberos services.
 * Web Services::                Web servers.
+* Certificate Services::        TLS certificates via Let's Encrypt.
 * VPN Services::                VPN daemons.
 * Network File System::         NFS related services.
 * Continuous Integration::      The Cuirass service.
@@ -8605,6 +8606,7 @@ declaration.
 * Messaging Services::          Messaging services.
 * Kerberos Services::           Kerberos services.
 * Web Services::                Web servers.
+* Certificate Services::        TLS certificates via Let's Encrypt.
 * VPN Services::                VPN daemons.
 * Network File System::         NFS related services.
 * Continuous Integration::      The Cuirass service.
@@ -13379,6 +13381,84 @@ Whether the server should add its configuration to response.
 @end table
 @end deftp
 
+
+@node Certificate Services
+@subsubsection Certificate Services
+
+@cindex web
+@cindex www
+@cindex HTTP
+The @code{(gnu services certbot)} module allows Guix systems to
+automatically obtain a valid TLS certificate from the Let's Encrypt
+certificate authority.  These certificates can then be used to serve
+content securely over HTTPS or other TLS-based protocols, with the
+knowledge that the client will be able to verify the server's
+authenticity.
+
+Let's Encrypt (@url{https://letsencrypt.org/}) provides the
+@code{certbot} tool to automate the certification process.  This tool
+first securely generates a key on the server.  It then makes a request
+to the Let's Encrypt certificate authority (CA) to sign the key.  The CA
+checks that the request originates from the host in question by using a
+challenge-response protocol, requiring the server to provide its
+response over HTTP.  If that protocol completes successfully, the CA
+signs the key, resulting in a certificate.  That certificate is valid
+for a limited period of time, and therefore to continue to provide TLS
+services, the server needs to periodically ask the CA to renew its
+signature.
+
+The Guix certbot service automates this process: the initial key
+generation, the initial certification request to the Let's Encrypt
+service, the web server challenge/response integration, writing the
+certificate to disk, and the automated periodic renewals.
+
+@defvr {Scheme Variable} certbot-service-type
+A service type for the @code{certbot} Let's Encrypt client.
+@end defvr
+
+@deftp {Data Type} certbot-configuration
+Data type representing the configuration of the @code{certbot} serice.
+This type has the following parameters:
+@table @asis
+@item @code{package} (default: @code{certbot})
+The certbot package to use.
+
+@item @code{webroot} (default: @code{/var/www})
+The directory from which to serve the Let's Encrypt challenge/response
+files.
+
+@item @code{hosts} (default: @code{()})
+A list of hosts for which to generate certificates and request
+signatures.
+
+@item @code{default-location} (default: @i{see below})
+The default @code{nginx-location-configuration}.  Because @code{certbot}
+needs to be able to serve challenges and responses, it needs to be able
+to run a web server.  It does so by extending the @code{nginx} web
+service with an @code{nginx-server-configuration} listening on the
+@var{hosts} on port 80, and which has a
+@code{nginx-location-configuration} for the @code{/.well-known/} URI
+path subspace used by Let's Encrypt.  @xref{Web Services}, for more on
+these nginx configuration data types.
+
+Requests to other URL paths will be matched by the
+@code{default-location}, which if present is added to all
+@code{nginx-server-configuration}s.
+
+By default, the @code{default-location} will issue a redirect from
+@code{http://@var{host}/...} to @code{https://@var{host}/...}, leaving
+you to define what to serve on your site via @code{https}.
+
+Pass @code{#f} to not issue a default location.
+@end table
+@end deftp
+
+The public key and its signatures will be written to
+@code{/etc/letsencrypt/live/@var{host}/fullchain.pem}, for each
+@var{host} in the configuration.  The private key is written to
+@code{/etc/letsencrypt/live/@var{host}/privkey.pem}.
+
+
 @node VPN Services
 @subsubsection VPN Services
 @cindex VPN (virtual private network)
diff --git a/gnu/local.mk b/gnu/local.mk
index 31239933c..ac0c9d510 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -414,6 +414,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/services/admin.scm			\
   %D%/services/avahi.scm			\
   %D%/services/base.scm				\
+  %D%/services/certbot.scm			\
   %D%/services/configuration.scm		\
   %D%/services/cuirass.scm			\
   %D%/services/cups.scm				\
diff --git a/gnu/services/certbot.scm b/gnu/services/certbot.scm
new file mode 100644
index 000000000..c11c9a66b
--- /dev/null
+++ b/gnu/services/certbot.scm
@@ -0,0 +1,128 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2016 ng0 <ng0@we.make.ritual.n0.is>
+;;; Copyright © 2016 Sou Bunnbu <iyzsong@member.fsf.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services certbot)
+  #:use-module (gnu services)
+  #:use-module (gnu services base)
+  #:use-module (gnu services shepherd)
+  #:use-module (gnu services mcron)
+  #:use-module (gnu services web)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu packages tls)
+  #:use-module (guix records)
+  #:use-module (guix gexp)
+  #:use-module (srfi srfi-1)
+  #:use-module (ice-9 match)
+  #:export (certbot-service-type
+            certbot-configuration
+            certbot-configuration?))
+
+;;; Commentary:
+;;;
+;;; Automatically obtaining TLS certificates from Let's Encrypt.
+;;;
+;;; Code:
+
+\f
+(define-record-type* <certbot-configuration>
+  certbot-configuration make-certbot-configuration
+  certbot-configuration?
+  (package             certbot-configuration-package
+                       (default certbot))
+  (webroot             certbot-configuration-webroot
+                       (default "/var/www"))
+  (hosts               certbot-configuration-hosts
+                       (default '()))
+  (default-location    certbot-configuration-default-location
+                       (default
+                         (nginx-location-configuration
+                          (uri "/")
+                          (body
+                           (list "return 301 https://$host$request_uri;"))))))
+
+(define certbot-renewal-jobs
+  (match-lambda
+    (($ <certbot-configuration> package webroot hosts default-location)
+     (match hosts
+       ;; Avoid pinging certbot if we have no hosts.
+       (() '())
+       (_
+        (list
+         ;; Attempt to renew the certificates twice a week.
+         #~(job (lambda (now)
+                  (next-day-from (next-hour-from now '(3))
+                                 '(2 5)))
+                (string-append #$package "/bin/certbot renew"
+                               (string-concatenate
+                                (map (lambda (host)
+                                       (string-append " -d " host))
+                                     #$hosts))))))))))
+
+(define certbot-activation
+  (match-lambda
+    (($ <certbot-configuration> package webroot hosts default-location)
+     (with-imported-modules '((guix build utils))
+       #~(begin
+	   (use-modules (guix build utils))
+	   (mkdir-p #$webroot)
+           (for-each
+            (lambda (host)
+              (unless (file-exists? (in-vicinity "/etc/letsencrypt/live" host))
+                (unless (zero? (system*
+                                (string-append #$certbot "/bin/certbot")
+                                "certonly" "--webroot" "-w" #$webroot
+                                "-d" host))
+                  (error "failed to acquire cert for host" host))))
+            '#$hosts))))))
+
+(define certbot-nginx-server-configurations
+  (match-lambda
+    (($ <certbot-configuration> package webroot hosts default-location)
+     (map
+      (lambda (host)
+        (nginx-server-configuration
+         (http-port 80)
+         (https-port #f)
+         (ssl-certificate #f)
+         (ssl-certificate-key #f)
+         (server-name (list host))
+         (locations
+          (filter identity
+                  (list
+                   (nginx-location-configuration
+                    (uri "/.well-known")
+                    (body (list (list "root " webroot ";"))))
+                   default-location)))))
+      hosts))))
+
+(define certbot-service-type
+  (service-type (name 'certbot)
+                (extensions
+                 (list (service-extension nginx-service-type
+                                          certbot-nginx-server-configurations)
+                       (service-extension activation-service-type
+                                          certbot-activation)
+                       (service-extension mcron-service-type
+                                          certbot-renewal-jobs)))
+                (compose concatenate)
+                (extend (lambda (config additional-hosts)
+                          (certbot-configuration
+                           (inherit config)
+                           (hosts (append (certbot-configuration-hosts config)
+                                          additional-hosts)))))))
-- 
2.12.2

bug#26685: certbot service

[Clément Lassieur]

Hi Andy,

Thanks for working on this!

> +;;; GNU Guix --- Functional package management for GNU
> +;;; Copyright © 2016 ng0 <ng0@we.make.ritual.n0.is>
> +;;; Copyright © 2016 Sou Bunnbu <iyzsong@member.fsf.org>

Or maybe you didn't work on this?..

> +(define certbot-renewal-jobs
> +  (match-lambda
> +    (($ <certbot-configuration> package webroot hosts default-location)
> +     (match hosts
> +       ;; Avoid pinging certbot if we have no hosts.
> +       (() '())
> +       (_
> +        (list
> +         ;; Attempt to renew the certificates twice a week.
> +         #~(job (lambda (now)
> +                  (next-day-from (next-hour-from now '(3))
> +                                 '(2 5)))

This is not twice a week, but twice a month at days 2 and 5, because
'next-day-from' will look after the next day (in month) that has number
2 and 5.  'next-hour-from' is not taken into account because next day
from any hour still runs at 0 o'clock.

But anyway I think it should be twice a day, and at a random minute
within the hour, as advised by certbot:

--8<---------------cut here---------------start------------->8---
from https://certbot.eff.org/all-instructions/

if you're setting up a cron or systemd job, we recommend running it
twice per day (it won't do anything until your certificates are due for
renewal or revoked, but running it regularly would give your site a
chance of staying online in case a Let's Encrypt-initiated revocation
happened for some reason). Please select a random minute within the hour
for your renewal tasks.
--8<---------------cut here---------------end--------------->8---

What do you think of:

    '(next-minute-from (next-hour '(0 12)) (list (random 60)))

instead?  Schedules can be debbuged with '--schedule=count' option.

Also I think some services have to be reloaded/restarted after their
certificates are upgraded.  That could be done via a mcron post-hook,
but I'm not sure how to pass the list of services that have to be
restarted.  WDYT?

Clément

bug#26685: certbot service

[Andy Wingo]

On Fri 28 Apr 2017 11:24, Clément Lassieur <clement@lassieur.org> writes:

> Hi Andy,
>
> Thanks for working on this!

Thanks for the review :-)

>> +;;; GNU Guix --- Functional package management for GNU
>> +;;; Copyright © 2016 ng0 <ng0@we.make.ritual.n0.is>
>> +;;; Copyright © 2016 Sou Bunnbu <iyzsong@member.fsf.org>
>
> Or maybe you didn't work on this?..

I did but also parts of it came from:

  https://git.savannah.gnu.org/cgit/guix/maintenance.git/tree/hydra/bayfront.scm

I should fix the attribution I guess!

>> +(define certbot-renewal-jobs
>> +  (match-lambda
>> +    (($ <certbot-configuration> package webroot hosts default-location)
>> +     (match hosts
>> +       ;; Avoid pinging certbot if we have no hosts.
>> +       (() '())
>> +       (_
>> +        (list
>> +         ;; Attempt to renew the certificates twice a week.
>> +         #~(job (lambda (now)
>> +                  (next-day-from (next-hour-from now '(3))
>> +                                 '(2 5)))
>
> This is not twice a week, but twice a month at days 2 and 5, because
> 'next-day-from' will look after the next day (in month) that has number
> 2 and 5.  'next-hour-from' is not taken into account because next day
> from any hour still runs at 0 o'clock.
>
> But anyway I think it should be twice a day, and at a random minute
> within the hour, as advised by certbot:
>
> from https://certbot.eff.org/all-instructions/
>
> if you're setting up a cron or systemd job, we recommend running it
> twice per day (it won't do anything until your certificates are due for
> renewal or revoked, but running it regularly would give your site a
> chance of staying online in case a Let's Encrypt-initiated revocation
> happened for some reason). Please select a random minute within the hour
> for your renewal tasks.
>
> What do you think of:
>
>     '(next-minute-from (next-hour '(0 12)) (list (random 60)))
>
> instead?  Schedules can be debbuged with '--schedule=count' option.

Sounds fine to me!

> Also I think some services have to be reloaded/restarted after their
> certificates are upgraded.  That could be done via a mcron post-hook,
> but I'm not sure how to pass the list of services that have to be
> restarted.  WDYT?

Good question.  I don't even know how to know when running certbot
results in a rotation and when it leaves things as-is.  It's a great
question though!  BTW if you are interested in using/hacking on this, I
think we could just get it into master and patch it from there.  WDYT?

Andy

bug#26685: certbot service

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 502 bytes --]

On Fri, Apr 28, 2017 at 11:24:47AM +0200, Clément Lassieur wrote:
> Also I think some services have to be reloaded/restarted after their
> certificates are upgraded.  That could be done via a mcron post-hook,
> but I'm not sure how to pass the list of services that have to be
> restarted.  WDYT?

I don't have the answer either, but this is a prime use case for
implementing `nginx reload` in the nginx-service. Otherwise, nginx will
have to be killed in order to deploy the new certificate.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

bug#26685: certbot service

[Clément Lassieur]

Andy Wingo <wingo@igalia.com> writes:

> BTW if you are interested in using/hacking on this, I think we could
> just get it into master and patch it from there.  WDYT?

Ok!

bug#26685: certbot service

[Clément Lassieur]

Andy Wingo <wingo@igalia.com> writes:
> On Fri 28 Apr 2017 11:24, Clément Lassieur <clement@lassieur.org> writes:

>> Also I think some services have to be reloaded/restarted after their
>> certificates are upgraded.  That could be done via a mcron post-hook,
I meant certbot --post-hook, not mcron.

>> but I'm not sure how to pass the list of services that have to be
>> restarted.  WDYT?

> Good question.  I don't even know how to know when running certbot
> results in a rotation and when it leaves things as-is.  It's a great
> question though!

Actually I was wrong about certbot --post-hook: there is a certbot
--renew-hook.  This one gets called after every successful renewal.
Thus in this hook we should restart/reload services.

bug#26685: certbot service

[Clément Lassieur]

Leo Famulari <leo@famulari.name> writes:

> On Fri, Apr 28, 2017 at 11:24:47AM +0200, Clément Lassieur wrote:
>> Also I think some services have to be reloaded/restarted after their
>> certificates are upgraded.  That could be done via a mcron post-hook,
>> but I'm not sure how to pass the list of services that have to be
>> restarted.  WDYT?
>
> I don't have the answer either, but this is a prime use case for
> implementing `nginx reload` in the nginx-service. Otherwise, nginx will
> have to be killed in order to deploy the new certificate.

But this is not just about nginx right?  (Otherwise 'nginx -s reload'
would do the job, I think.)  It is about every service that has its
certificates renewed by certbot.  Don't we need a generic way to reload
services, like 'herd reload service'?  I will have a look at how we
could implement this.

bug#26685: certbot service

[Andy Wingo]

On Sat 29 Apr 2017 11:14, Clément Lassieur <clement@lassieur.org> writes:

> Andy Wingo <wingo@igalia.com> writes:
>
>> BTW if you are interested in using/hacking on this, I think we could
>> just get it into master and patch it from there.  WDYT?
>
> Ok!

Cool :)  I guess to do this, we need #26684 first.  Would you mind
reviewing it?

  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26684

Maybe there is a nicer way to do that, but it seems like a useful,
not-incompatible change to make, so for me it seems safe to go in.  YMMV
though :)

Andy

bug#26685: certbot service

[Clément Lassieur]

Andy Wingo <wingo@igalia.com> writes:

> On Sat 29 Apr 2017 11:14, Clément Lassieur <clement@lassieur.org> writes:
>
>> Andy Wingo <wingo@igalia.com> writes:
>>
>>> BTW if you are interested in using/hacking on this, I think we could
>>> just get it into master and patch it from there.  WDYT?
>>
>> Ok!
>
> Cool :)  I guess to do this, we need #26684 first.  Would you mind
> reviewing it?

Sure!  I'm doing it.

>   https://debbugs.gnu.org/cgi/bugreport.cgi?bug=26684
>
> Maybe there is a nicer way to do that, but it seems like a useful,
> not-incompatible change to make, so for me it seems safe to go in.  YMMV
> though :)
>
> Andy

[bug#26685] certbot service experience

[Ludovic Courtès]

Hello Andy, Chris, and Clément!  :-)

Could we decide on what really needs to be done before we push this
patch, and what can be done later?  It’s a pity that this patch has been
sitting there for so long despite being very useful!

  https://bugs.gnu.org/26685

TIA.  :-)

Ludo’.

[bug#26685] certbot service experience

[Christopher Allan Webber]

Ludovic Courtès writes:

> Hello Andy, Chris, and Clément!  :-)
>
> Could we decide on what really needs to be done before we push this
> patch, and what can be done later?  It’s a pity that this patch has been
> sitting there for so long despite being very useful!
>
>   https://bugs.gnu.org/26685
>
> TIA.  :-)
>
> Ludo’.

I'll reply to the issue; the one thing I think that we should fix for
sure is that it prompts for the email during guix system reconfigure.
There should be a way to pass this in as an option... IMO having prompts
there is unexpected!  But maybe even if that can't be done soon we
should push it and open the email thing as another bug.

And yes I was thinking the same thing; I'm still running a server using
this, so upgrading it is a bit awkward with the stale branch.

[bug#26685] certbot service experience

[Tobias Geerinckx-Rice]

[-- Attachment #1.1: Type: text/plain, Size: 699 bytes --]

Ludovic Courtès wrote on 26/07/17 at 10:59:
> Hello Andy, Chris, and Clément!  :-)

...and Ludo',

> Could we decide on what really needs to be done before we push this
> patch, and what can be done later?  It’s a pity that this patch has been
> sitting there for so long despite being very useful!

Zut alors. I missed that service! Nice. Necessary.

>   https://bugs.gnu.org/26685

If nobody objects, I'd like a few days to play with this before it gets
merged. It's a fine service, but I think it privileges the ‘--webroot’
plugin too much (‘-w’ is a plugin-specific option, not global). I'd
rather not have my mail box spin up nginx...

Kind regards,

T G-R



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 248 bytes --]

[bug#26685] certbot service experience

[ng0]

[-- Attachment #1: Type: text/plain, Size: 1449 bytes --]

Christopher Allan Webber transcribed 0.8K bytes:
> Ludovic Courtès writes:
> 
> > Hello Andy, Chris, and Clément!  :-)
> >
> > Could we decide on what really needs to be done before we push this
> > patch, and what can be done later?  It’s a pity that this patch has been
> > sitting there for so long despite being very useful!
> >
> >   https://bugs.gnu.org/26685
> >
> > TIA.  :-)
> >
> > Ludo’.
> 
> I'll reply to the issue; the one thing I think that we should fix for
> sure is that it prompts for the email during guix system reconfigure.
> There should be a way to pass this in as an option... IMO having prompts

I think this can be configured in a file for letsencrypt, at least it
keeps a config file where common options can be defined, and this
config file can even be passed as argument, so no fixed location for
it if I understood it right after quick reading of the documents.

reference documentation:
https://letsencrypt.readthedocs.io/en/latest/using.html

> there is unexpected!  But maybe even if that can't be done soon we
> should push it and open the email thing as another bug.
> 
> And yes I was thinking the same thing; I'm still running a server using
> this, so upgrading it is a bit awkward with the stale branch.
> 
> 
> 
> 

-- 
ng0
GnuPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588
GnuPG: https://n0is.noblogs.org/my-keys
https://www.infotropique.org https://krosos.org

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#26685] certbot service experience

[ng0]

[-- Attachment #1: Type: text/plain, Size: 1981 bytes --]

ng0 transcribed 2.5K bytes:
> Christopher Allan Webber transcribed 0.8K bytes:
> > Ludovic Courtès writes:
> > 
> > > Hello Andy, Chris, and Clément!  :-)
> > >
> > > Could we decide on what really needs to be done before we push this
> > > patch, and what can be done later?  It’s a pity that this patch has been
> > > sitting there for so long despite being very useful!
> > >
> > >   https://bugs.gnu.org/26685
> > >
> > > TIA.  :-)
> > >
> > > Ludo’.
> > 
> > I'll reply to the issue; the one thing I think that we should fix for
> > sure is that it prompts for the email during guix system reconfigure.
> > There should be a way to pass this in as an option... IMO having prompts
> 
> I think this can be configured in a file for letsencrypt, at least it
> keeps a config file where common options can be defined, and this
> config file can even be passed as argument, so no fixed location for
> it if I understood it right after quick reading of the documents.

Sorry, there are default locations:

  By default, the following locations are searched:

  /etc/letsencrypt/cli.ini
  $XDG_CONFIG_HOME/letsencrypt/cli.ini (or ~/.config/letsencrypt/cli.ini if $XDG_CONFIG_HOME is not set).

But I think this already helps.

> reference documentation:
> https://letsencrypt.readthedocs.io/en/latest/using.html
> 
> > there is unexpected!  But maybe even if that can't be done soon we
> > should push it and open the email thing as another bug.
> > 
> > And yes I was thinking the same thing; I'm still running a server using
> > this, so upgrading it is a bit awkward with the stale branch.
> > 
> > 
> > 
> > 
> 
> -- 
> ng0
> GnuPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588
> GnuPG: https://n0is.noblogs.org/my-keys
> https://www.infotropique.org https://krosos.org



-- 
ng0
GnuPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588
GnuPG: https://n0is.noblogs.org/my-keys
https://www.infotropique.org https://krosos.org

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#26685] certbot service experience

[Julien Lepiller]

Le Sun, 30 Jul 2017 09:22:12 +0000,
ng0 <ng0@infotropique.org> a écrit :

> Sorry, there are default locations:
> 
>   By default, the following locations are searched:
> 
>   /etc/letsencrypt/cli.ini
>   $XDG_CONFIG_HOME/letsencrypt/cli.ini (or
> ~/.config/letsencrypt/cli.ini if $XDG_CONFIG_HOME is not set).
> 
> But I think this already helps.

We could use the --config flag and pass the location of a generated
config file as the argument.

> 
> > reference documentation:
> > https://letsencrypt.readthedocs.io/en/latest/using.html
> >   

[bug#26685] certbot service experience

[Ludovic Courtès]

Hi there!

Tobias Geerinckx-Rice <me@tobias.gr> skribis:

> Ludovic Courtès wrote on 26/07/17 at 10:59:
>> Hello Andy, Chris, and Clément!  :-)
>
> ...and Ludo',
>
>> Could we decide on what really needs to be done before we push this
>> patch, and what can be done later?  It’s a pity that this patch has been
>> sitting there for so long despite being very useful!
>
> Zut alors. I missed that service! Nice. Necessary.
>
>>   https://bugs.gnu.org/26685
>
> If nobody objects, I'd like a few days to play with this before it gets
> merged. It's a fine service, but I think it privileges the ‘--webroot’
> plugin too much (‘-w’ is a plugin-specific option, not global). I'd
> rather not have my mail box spin up nginx...

Any update on this?  :-)

Ludo’.

[bug#26685] certbot service experience

[Christopher Allan Webber]

Ludovic Courtès writes:

> Hi there!
>
> Tobias Geerinckx-Rice <me@tobias.gr> skribis:
>
>> Ludovic Courtès wrote on 26/07/17 at 10:59:
>>> Hello Andy, Chris, and Clément!  :-)
>>
>> ...and Ludo',
>>
>>> Could we decide on what really needs to be done before we push this
>>> patch, and what can be done later?  It’s a pity that this patch has been
>>> sitting there for so long despite being very useful!
>>
>> Zut alors. I missed that service! Nice. Necessary.
>>
>>>   https://bugs.gnu.org/26685
>>
>> If nobody objects, I'd like a few days to play with this before it gets
>> merged. It's a fine service, but I think it privileges the ‘--webroot’
>> plugin too much (‘-w’ is a plugin-specific option, not global). I'd
>> rather not have my mail box spin up nginx...
>
> Any update on this?  :-)
>
> Ludo’.

My feeling is that it's a good idea to allow for non-webroot
configuration, but I think it shouldn't hold us back from moving forward
with getting the certbot service in.

[bug#26685] certbot service experience

[Christopher Allan Webber]

Christopher Allan Webber writes:

> Ludovic Courtès writes:
>
>> Hi there!
>>
>> Tobias Geerinckx-Rice <me@tobias.gr> skribis:
>>
>>> Ludovic Courtès wrote on 26/07/17 at 10:59:
>>>> Hello Andy, Chris, and Clément!  :-)
>>>
>>> ...and Ludo',
>>>
>>>> Could we decide on what really needs to be done before we push this
>>>> patch, and what can be done later?  It’s a pity that this patch has been
>>>> sitting there for so long despite being very useful!
>>>
>>> Zut alors. I missed that service! Nice. Necessary.
>>>
>>>>   https://bugs.gnu.org/26685
>>>
>>> If nobody objects, I'd like a few days to play with this before it gets
>>> merged. It's a fine service, but I think it privileges the ‘--webroot’
>>> plugin too much (‘-w’ is a plugin-specific option, not global). I'd
>>> rather not have my mail box spin up nginx...
>>
>> Any update on this?  :-)
>>
>> Ludo’.
>
> My feeling is that it's a good idea to allow for non-webroot
> configuration, but I think it shouldn't hold us back from moving forward
> with getting the certbot service in.

I especially feel this way this morning, when I had to push a server
upgrade out and I suddenly realized I'm about to have to rebase this
branch!

Ludo’, are you okay with me working on getting this into master with
just the webroot plugin for now and marking that the service
configuration interface for this is semi-unstable for now in the manual?
That way we can get this in and get other people to test it, and not
let it languish.

I think the service needs some serious work but we're likely to have
more success getting around the rough edges by having more people using
it anyway.

WDYT?

[bug#26685] certbot service experience

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 1105 bytes --]

On Thu, Jul 27, 2017 at 07:30:48PM +0200, Tobias Geerinckx-Rice wrote:
> If nobody objects, I'd like a few days to play with this before it gets
> merged. It's a fine service, but I think it privileges the ‘--webroot’
> plugin too much (‘-w’ is a plugin-specific option, not global). I'd
> rather not have my mail box spin up nginx...

I agree that we should, in the long run, offer a more generalized ACME
client service.

However, the --webroot method is not specific to any of the other
plugins. Instead, it is a general purpose method of obtaining and
renewing signed x509 certificates with a running webserver. Certbot
requires no server-specific configuration with this method, and the
server only needs to be configured to serve a particular directory which
will contain the temporary cryptographic "challenge" file. It's not a
very tight coupling.

Since serving HTTPS is, in practice, one of the primary use cases for
the x509 CA system (as opposed to self-signed certs), I think we should
add the service as-is and let people generalize it as they see fit later
on.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#26685] certbot service experience

[Christopher Allan Webber]

Leo Famulari writes:

> On Thu, Jul 27, 2017 at 07:30:48PM +0200, Tobias Geerinckx-Rice wrote:
>> If nobody objects, I'd like a few days to play with this before it gets
>> merged. It's a fine service, but I think it privileges the ‘--webroot’
>> plugin too much (‘-w’ is a plugin-specific option, not global). I'd
>> rather not have my mail box spin up nginx...
>
> I agree that we should, in the long run, offer a more generalized ACME
> client service.
>
> However, the --webroot method is not specific to any of the other
> plugins. Instead, it is a general purpose method of obtaining and
> renewing signed x509 certificates with a running webserver. Certbot
> requires no server-specific configuration with this method, and the
> server only needs to be configured to serve a particular directory which
> will contain the temporary cryptographic "challenge" file. It's not a
> very tight coupling.
>
> Since serving HTTPS is, in practice, one of the primary use cases for
> the x509 CA system (as opposed to self-signed certs), I think we should
> add the service as-is and let people generalize it as they see fit later
> on.

Sounds like the right approach to me.

I'll add a note about the service configuration possibly being unstable
to the docs and push this today.  I just did a rebase on my end.

[bug#26685] certbot service experience

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 293 bytes --]

On Tue, Oct 24, 2017 at 09:26:13AM -0500, Christopher Allan Webber wrote:
> I think the service needs some serious work but we're likely to have
> more success getting around the rough edges by having more people using
> it anyway.

+1

People will show up to help after it has been pushed :)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#26685] certbot service experience

[Ludovic Courtès]

Heya!

Christopher Allan Webber <cwebber@dustycloud.org> skribis:

> Ludo’, are you okay with me working on getting this into master with
> just the webroot plugin for now and marking that the service
> configuration interface for this is semi-unstable for now in the manual?
> That way we can get this in and get other people to test it, and not
> let it languish.

I agree, I’m definitely okay with you pushing this!

Thank you for taking care of it,
Ludo’.

bug#26685: certbot service experience

[Ludovic Courtès]

Hello!

Since everyone was so hesitant I was so blissful ;-), I went ahead and
pushed this patch as commit c2e5d74997bdb57dc35e0189188c6917cd5f0f19!
\o/

I think it’s better than the status quo, because even if it’s not
perfect, it *is* useful as it is.  We can start from here if there are
ideas on how to improve it.

Ludo’.

[bug#26685] certbot service experience

[Christopher Allan Webber]

Ludovic Courtès writes:

> Hello!
>
> Since everyone was so hesitant I was so blissful ;-), I went ahead and
> pushed this patch as commit c2e5d74997bdb57dc35e0189188c6917cd5f0f19!
> \o/
>
> I think it’s better than the status quo, because even if it’s not
> perfect, it *is* useful as it is.  We can start from here if there are
> ideas on how to improve it.
>
> Ludo’.

Yay!  But oh nooooo!  I really should have replied to this!  I had
stopped working on it because I got overloaded with final-push situation
in ActivityPub.

Maybe this is worth moving to the mailing list instead, but one reason I
didn't push it was I had discovered a significant and serious
issue... building a system based off of its config is no longer
determistic based off of the system config alone... because this
checks for the rpesence of nginx-server-configuration-ssl-certificate
at *build* time of the system config.  The following code is the
culprit:

  (define (emit-nginx-server-config server)
    ;; [...]
      (for-each
       (match-lambda
        ((record-key . file)
         (if (and file (not (file-exists? file)))
             (error
              (simple-format
               #f
               "~A in the nginx configuration for the server with name \"~A\" does not exist" record-key server-name)))))
       `(("ssl-certificate"     . ,ssl-certificate)
         ("ssl-certificate-key" . ,ssl-certificate-key)))

Relatedly, the current setup has these defaults:

  (ssl-certificate     nginx-server-configuration-ssl-certificate
                       (default "/etc/nginx/cert.pem"))
  (ssl-certificate-key nginx-server-configuration-ssl-certificate-key
                       (default "/etc/nginx/key.pem"))

IMO they should probably default to #f so that looking for the files
doesn't fail to boot nginx.

[bug#26685] certbot service experience

[Ludovic Courtès]

Hello Chris!

Christopher Allan Webber <cwebber@dustycloud.org> skribis:

> Yay!  But oh nooooo!  I really should have replied to this!  I had
> stopped working on it because I got overloaded with final-push situation
> in ActivityPub.

Understood, no problem!  We’ve had seven months and I thought we’d
rather make progress on this than let everyone re-implement more-or-less
the same service.

> Maybe this is worth moving to the mailing list instead, but one reason I
> didn't push it was I had discovered a significant and serious
> issue... building a system based off of its config is no longer
> determistic based off of the system config alone... because this
> checks for the rpesence of nginx-server-configuration-ssl-certificate
> at *build* time of the system config.  The following code is the
> culprit:

OK, I agree it’s a problem.

That said, the ‘file-exists?’ check comes from an earlier and IIUC
unrelated commit, f2d7a492df759ab7416ff4b244e37896835f04fa.

> Relatedly, the current setup has these defaults:
> 
>   (ssl-certificate     nginx-server-configuration-ssl-certificate
>                        (default "/etc/nginx/cert.pem"))
>   (ssl-certificate-key nginx-server-configuration-ssl-certificate-key
>                        (default "/etc/nginx/key.pem"))
> 
> IMO they should probably default to #f so that looking for the files
> doesn't fail to boot nginx.

I agree (apparently it comes from a Sept. 2016 commit,
8c00b83818acbbc3dfd406465e50439c3418afc5.)

Could you send a patch?  :-)

Thanks,
Ludo’.

[bug#26685] certbot service experience

[Christopher Allan Webber]

Ludovic Courtès writes:

> I agree (apparently it comes from a Sept. 2016 commit,
> 8c00b83818acbbc3dfd406465e50439c3418afc5.)
>
> Could you send a patch?  :-)

Have some things to wrap up this week, but can work on it next week.
Thanks!