[bug#63877] [PATCH] gnu: services: web: Set SSL_CERT_DIR in php-fpm environment.

[bug#63877] [PATCH] gnu: services: web: Set SSL_CERT_DIR in php-fpm environment.

[Timo Wilken]

Some PHP programs, like Nextcloud, make HTTPS requests to other servers. For
this, they need to know where the system CA certificates are.

* gnu/services/web.scm (php-fpm-shepherd-service): Set SSL_CERT_DIR
  environment variable.
---

This solution adds a dependency from the resulting Shepherd service to the
nss-certs package, which weighs 0.3 MiB. An alternative solution might be to
set SSL_CERT_DIR=/etc/ssl/certs instead and rely on nss-certs being installed
system-wide.

 gnu/services/web.scm | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/gnu/services/web.scm b/gnu/services/web.scm
index 45897d7d6f..e46710a040 100644
--- a/gnu/services/web.scm
+++ b/gnu/services/web.scm
@@ -16,6 +16,7 @@
 ;;; Copyright © 2020, 2021 Alexandru-Sergiu Marton <brown121407@posteo.ro>
 ;;; Copyright © 2022 Simen Endsjø <simendsjo@gmail.com>
 ;;; Copyright © 2023 Bruno Victal <mirai@makinata.eu>
+;;; Copyright © 2023 Timo Wilken <guix@twilken.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -1096,6 +1097,12 @@ (define php-fpm-shepherd-service
                         #$@(if php-ini-file
                                `("-c" ,php-ini-file)
                                '()))
+                      #:environment-variables
+                      (cons*
+                       ;; Needed by e.g. Nextcloud to make HTTPS requests.
+                       (string-append
+                        "SSL_CERT_DIR=" #$(file-append nss-certs "/etc/ssl/certs"))
+                       (default-environment-variables))
                       #:pid-file #$pid-file))
             (stop #~(make-kill-destructor)))))))
 

base-commit: 66c9b82fed3c59ee07187898592c688c82fed273
-- 
2.40.1

[bug#63877] [PATCH] gnu: services: web: Set SSL_CERT_DIR in php-fpm environment.

[Bruno Victal]

Hi Timo,

On 2023-06-03 19:25, Timo Wilken wrote:
> Some PHP programs, like Nextcloud, make HTTPS requests to other servers. For
> this, they need to know where the system CA certificates are.
> 
> * gnu/services/web.scm (php-fpm-shepherd-service): Set SSL_CERT_DIR
>   environment variable.
> ---
> 
> This solution adds a dependency from the resulting Shepherd service to the
> nss-certs package, which weighs 0.3 MiB. An alternative solution might be to
> set SSL_CERT_DIR=/etc/ssl/certs instead and rely on nss-certs being installed
> system-wide.

How about exposing this as a new environment-variable record field à
la mpd-configuration (gnu services audio)?
Forcing the service to use a specific package seems overly rigid since
it would make it impossible to specify alternate/custom certificates or
nss-certs package variants.


-- 
Furthermore, I consider that nonfree software must be eradicated.

Cheers,
Bruno.

[bug#63877] [PATCH v2] gnu: services: web: Allow specifying extra php-fpm environment variables.

[Timo Wilken]

Some PHP programs, like Nextcloud, make HTTPS requests to other servers. For
this, they need to know where the system CA certificates are, so SSL_CERT_DIR
needs to be set.

This can be accomplished by the user using the new environment-variables field
of <php-fpm-configuration>.

This field is empty by default to preserve the existing behaviour of php-fpm.

* gnu/services/web.scm (<php-fpm-configuration>): Add environment-variables field.
  (php-fpm-shepherd-service): Use the new field.
* doc/guix.texi (Web Services): Document the new field.
---

> How about exposing this as a new environment-variable record field à la
> mpd-configuration (gnu services audio)?
Hi Bruno, that's a good point!

I've added a new field instead where the user can specify arbitrary
environment variables. I've left it empty by default so there's no added
dependency on any package, and documented my intended use case in the info
manual instead.

Caveat: I haven't tested this "live" yet.

 doc/guix.texi        | 12 ++++++++++++
 gnu/services/web.scm | 11 +++++++++--
 2 files changed, 21 insertions(+), 2 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 7f8d8d66e9..441867afee 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -30994,6 +30994,18 @@ Web Services
 An optional override of the default php settings.
 It may be any ``file-like'' object (@pxref{G-Expressions, file-like objects}).
 You can use the @code{mixed-text-file} function or an absolute filepath for it.
+@item @code{environment-variables} (default @code{#~(list)})
+A gexp (@pxref{G-Expressions}) which produces a list of strings
+representing environment variable assignments.
+These environment variables are set for the php-fpm process.
+This can be used to, for example, point php-fpm at the CA certificates
+in the @code{nss-certs} package from @code{(gnu packages certs)}:
+@lisp
+(php-fpm-configuration
+ ;; @dots{}
+ (environment-variables
+  #~(list (string-append "SSL_CERT_DIR=" #$nss-certs "/etc/ssl/certs"))))
+@end lisp
 
 For local development it is useful to set a higher timeout and memory
 limit for spawned php processes.  This be accomplished with the
diff --git a/gnu/services/web.scm b/gnu/services/web.scm
index 45897d7d6f..1c496d5946 100644
--- a/gnu/services/web.scm
+++ b/gnu/services/web.scm
@@ -16,6 +16,7 @@
 ;;; Copyright © 2020, 2021 Alexandru-Sergiu Marton <brown121407@posteo.ro>
 ;;; Copyright © 2022 Simen Endsjø <simendsjo@gmail.com>
 ;;; Copyright © 2023 Bruno Victal <mirai@makinata.eu>
+;;; Copyright © 2023 Timo Wilken <guix@twilken.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -974,7 +975,9 @@ (define-record-type* <php-fpm-configuration> php-fpm-configuration
   (file             php-fpm-configuration-file ;#f | file-like
                     (default #f))
   (php-ini-file     php-fpm-configuration-php-ini-file ;#f | file-like
-                    (default #f)))
+                    (default #f))
+  (environment-variables php-fpm-configuration-environment-variables ;gexp producing list-of-strings
+                         (default #~(list))))
 
 (define-record-type* <php-fpm-dynamic-process-manager-configuration>
   php-fpm-dynamic-process-manager-configuration
@@ -1081,7 +1084,8 @@ (define php-fpm-shepherd-service
   (match-lambda
     (($ <php-fpm-configuration> php socket user group socket-user socket-group
                                 pid-file log-file pm display-errors
-                                timezone workers-log-file file php-ini-file)
+                                timezone workers-log-file file php-ini-file
+                                environment-variables)
      (list (shepherd-service
             (provision '(php-fpm))
             (documentation "Run the php-fpm daemon.")
@@ -1096,6 +1100,9 @@ (define php-fpm-shepherd-service
                         #$@(if php-ini-file
                                `("-c" ,php-ini-file)
                                '()))
+                      #:environment-variables
+                      (append #$environment-variables
+                              (default-environment-variables))
                       #:pid-file #$pid-file))
             (stop #~(make-kill-destructor)))))))
 

base-commit: 66c9b82fed3c59ee07187898592c688c82fed273
-- 
2.40.1

[bug#63877] [PATCH v2] gnu: services: web: Allow specifying extra php-fpm environment variables.

[Bruno Victal]

On 2023-06-04 14:59, Timo Wilken wrote:
> @@ -1096,6 +1100,9 @@ (define php-fpm-shepherd-service
>                          #$@(if php-ini-file
>                                 `("-c" ,php-ini-file)
>                                 '()))
> +                      #:environment-variables
> +                      (append #$environment-variables
> +                              (default-environment-variables))

Ungexp-ing lists can be rather tricky since your snippet will expand to:

--8<---------------cut here---------------start------------->8---
...
#:environment-variables (append ("FOO=bar" ...)
                                (default-environment-variables))
...
--8<---------------cut here---------------end--------------->8---

Which is interpreted as a procedure call. (and results in a hanged shepherd)

You need to quote the list here:

--8<---------------cut here---------------start------------->8---
#:environment-variables (append '#$environment-variables
                                (default-environment-variables))
--8<---------------cut here---------------end--------------->8---

Bonus points if you can write a small system test for this. (see
gnu/tests/web.scm for inspiration)
For our purposes, a pair of HTTP servers where one of them uses a
self-signed certificate will suffice.


-- 
Furthermore, I consider that nonfree software must be eradicated.

Cheers,
Bruno.

[bug#63877] [PATCH] gnu: services: web: Set SSL_CERT_DIR in php-fpm environment.

[Ludovic Courtès]

Hi Timo,

Did you have a chance to look into implementing Bruno’s suggestions?

  https://issues.guix.gnu.org/63877

Ludo’.

Bruno Victal <mirai@makinata.eu> skribis:

> On 2023-06-04 14:59, Timo Wilken wrote:
>> @@ -1096,6 +1100,9 @@ (define php-fpm-shepherd-service
>>                          #$@(if php-ini-file
>>                                 `("-c" ,php-ini-file)
>>                                 '()))
>> +                      #:environment-variables
>> +                      (append #$environment-variables
>> +                              (default-environment-variables))
>
> Ungexp-ing lists can be rather tricky since your snippet will expand to:
>
> ...
> #:environment-variables (append ("FOO=bar" ...)
>                                 (default-environment-variables))
> ...
>
>
> Which is interpreted as a procedure call. (and results in a hanged shepherd)
>
> You need to quote the list here:
>
> #:environment-variables (append '#$environment-variables
>                                 (default-environment-variables))
>
> Bonus points if you can write a small system test for this. (see
> gnu/tests/web.scm for inspiration)
> For our purposes, a pair of HTTP servers where one of them uses a
> self-signed certificate will suffice.

[bug#63877] [PATCH v2] gnu: services: web: Allow specifying extra php-fpm environment variables.

[Timo Wilken]

Hi Bruno, (hi Ludo'), thank you for your detailed feedback and sorry for not
responding earlier!

On Mon Jun 5, 2023 at 5:44 AM CEST, Bruno Victal wrote:
> Ungexp-ing lists can be rather tricky [...]
>
> You need to quote the list [...]

I was thinking of something closer to the example I added to doc/guix.texi in
my patch. The gexp would not be a list directly, but instead be some code that
would produce a list when evaluated, e.g.:

--8<---------------cut here---------------start------------->8---
#~(list (string-append "SSL_CERT_DIR=" #$nss-certs "/etc/ssl/certs"))))
--8<---------------cut here---------------end--------------->8---

That would let you refer to store paths in variable values, instead of being
limited to literal strings.

As far as I know, the following throws an error, and `file-append' instead of
`string-append' wouldn't work because of the `"SSL_CERT_DIR="' prefix, right?

--8<---------------cut here---------------start------------->8---
#~(#$(string-append "SSL_CERT_DIR=" nss-certs "/etc/ssl/certs"))))
--8<---------------cut here---------------end--------------->8---

If you have any ideas on a better way to do this, let me know!

> Bonus points if you can write a small system test for this. (see
> gnu/tests/web.scm for inspiration)
> For our purposes, a pair of HTTP servers where one of them uses a
> self-signed certificate will suffice.

Thanks for the pointer! I'll try to get something basic working along the
lines of the php-fpm tests already there, and send a PATCH v3 soon. I was
thinking of only verifying that an arbitrary sentinel variable is set, and not
bother to test SSL_*-related behaviour, but I can try to get the latter
working if you think that would be better.

[bug#63877] [PATCH v2] gnu: services: web: Allow specifying extra php-fpm environment variables.

[Bruno Victal]

Hi Timo,

On 2023-10-15 21:54, Timo Wilken wrote:
> Hi Bruno, (hi Ludo'), thank you for your detailed feedback and sorry for not
> responding earlier!
> 
> On Mon Jun 5, 2023 at 5:44 AM CEST, Bruno Victal wrote:
>> Ungexp-ing lists can be rather tricky [...]
>>
>> You need to quote the list [...]
> 
> I was thinking of something closer to the example I added to doc/guix.texi in
> my patch. The gexp would not be a list directly, but instead be some code that
> would produce a list when evaluated, e.g.:
> 
> --8<---------------cut here---------------start------------->8---
> #~(list (string-append "SSL_CERT_DIR=" #$nss-certs "/etc/ssl/certs"))))
> --8<---------------cut here---------------end--------------->8---
> 
> That would let you refer to store paths in variable values, instead of being
> limited to literal strings.

Right, I can see that it is indeed useful to accept a G-Exp instead.

> As far as I know, the following throws an error, and `file-append' instead of
> `string-append' wouldn't work because of the `"SSL_CERT_DIR="' prefix, right?
> 
> --8<---------------cut here---------------start------------->8---
> #~(#$(string-append "SSL_CERT_DIR=" nss-certs "/etc/ssl/certs"))))
> --8<---------------cut here---------------end--------------->8---

This ungexp doesn't work because it's “too wide”, in fact the bug
in [1] was caused by a very similar snippet.

Furthermore this would still run into the ungexp pitfall of being
interpreted as a procedure call since you now have:

--8<---------------cut here---------------start------------->8---
…
#:environment-variables (append ("SSL_CERT_DIR=<garbage-here>…" …)
                                (default-environment-variables))
…
--8<---------------cut here---------------end--------------->8---

You could try using a list gexps/strings like this:

--8<---------------cut here---------------start------------->8---
(list #~(string-append "SSL_CERT_DIR=" #$nss-certs "/etc/ssl/certs")
      "FOO=bar"
      (string-append "BAR=" 999))
--8<---------------cut here---------------end--------------->8---

Although your G-Exp idea might be better as it obviates the
need to do things like '#$ (by using #~(list …) or #~'("foo" …)).


[1]: <https://issues.guix.gnu.org/65383>

-- 
Furthermore, I consider that nonfree software must be eradicated.

Cheers,
Bruno.

[bug#63877] Reworked patch for setting php-fpm environment variables

[guix]

Hi Bruno, sorry for taking a while to get back to this.

Writing a test for curl's behaviour with the SSL_CERT_DIR variable proved too
fiddly for me, so I gave up and wrote a simpler test that just checks for a
sentinel variable in the phpinfo output instead.

I also found out that php-fpm clears environment variables when it starts,
except for those listed in its configuration. However, libcurl isn't affected
by this as far as I can tell -- it needs the SSL_CERT_DIR variable to be set
in the process environment, not only in the php-fpm config file!

I decided to set environment variables in the process environment and list
them in the generated configuration file, so they're passed through to any PHP
programs run through PHP-FPM. This should minimise surprise, I hope.

(That's also be useful for setting e.g. PATH -- Nextcloud has started
complaining that that variable is unset, and it needs the variable to be
listed in the php-fpm configuration.)

The reworked patch also removes some of the gexp-related hairyness -- the
`environment-variables' property just takes a list of (variable-name . value)
pairs now, no gexp'ing required, though file-like objects like what
`file-append' returns are accepted.

Please let me know what you think, and thank you for your considerable
patience with this patch series! :)

[bug#63877] [PATCH 1/2] gnu: services: web: Allow specifying extra php-fpm environment variables.

[guix]

From: Timo Wilken <guix@twilken.net>

Some PHP programs, like Nextcloud, make HTTPS requests to other servers. For
this, they need to know where the system CA certificates are, so SSL_CERT_DIR
needs to be set.

This can be accomplished by the user using the new environment-variables field
of <php-fpm-configuration>.

This field is empty by default to preserve the existing behaviour of php-fpm.

* gnu/services/web.scm (<php-fpm-configuration>): Add environment-variables field.
  (php-fpm-shepherd-service): Use the new field.
* doc/guix.texi (Web Services): Document the new field.
---
 doc/guix.texi        | 14 ++++++++++++++
 gnu/services/web.scm | 32 ++++++++++++++++++++++++++++----
 2 files changed, 42 insertions(+), 4 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 04119a5955..2bb076a8fa 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -124,6 +124,7 @@ Copyright @copyright{} 2023 Thomas Ieong@*
 Copyright @copyright{} 2023 Saku Laesvuori@*
 Copyright @copyright{} 2023 Graham James Addis@*
 Copyright @copyright{} 2023 Tomas Volf@*
+Copyright @copyright{} 2024 Timo Wilken@*
 
 Permission is granted to copy, distribute and/or modify this document
 under the terms of the GNU Free Documentation License, Version 1.3 or
@@ -32227,6 +32228,19 @@ max_execution_time = 1800"))
 Consult the @url{https://www.php.net/manual/en/ini.core.php,core php.ini
 directives} for comprehensive documentation on the acceptable
 @file{php.ini} directives.
+@item @code{environment-variables} (default @code{(list)})
+A list of @code{(variable-name . value)} pairs, representing environment
+variable assignments.  @code{value} may be a string or a store object,
+for example returned by @code{file-append}.  These environment variables
+are set for the php-fpm process.  This can be used to, for example,
+point PHP at the CA certificates in the @code{nss-certs} package from
+@code{(gnu packages certs)}:
+@lisp
+(php-fpm-configuration
+ ;; @dots{}
+ (environment-variables
+  `(("SSL_CERT_DIR" . ,(file-append nss-certs "/etc/ssl/certs")))))
+@end lisp
 @end table
 @end deftp
 
diff --git a/gnu/services/web.scm b/gnu/services/web.scm
index 05fd71f994..5fd09c8945 100644
--- a/gnu/services/web.scm
+++ b/gnu/services/web.scm
@@ -16,6 +16,7 @@
 ;;; Copyright © 2020, 2021 Alexandru-Sergiu Marton <brown121407@posteo.ro>
 ;;; Copyright © 2022 Simen Endsjø <simendsjo@gmail.com>
 ;;; Copyright © 2023 Bruno Victal <mirai@makinata.eu>
+;;; Copyright © 2024 Timo Wilken <guix@twilken.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -974,7 +975,9 @@ (define-record-type* <php-fpm-configuration> php-fpm-configuration
   (file             php-fpm-configuration-file ;#f | file-like
                     (default #f))
   (php-ini-file     php-fpm-configuration-php-ini-file ;#f | file-like
-                    (default #f)))
+                    (default #f))
+  (environment-variables php-fpm-configuration-environment-variables ;list of pairs of file-like
+                         (default '())))
 
 (define-record-type* <php-fpm-dynamic-process-manager-configuration>
   php-fpm-dynamic-process-manager-configuration
@@ -1024,7 +1027,8 @@ (define php-fpm-accounts
          (shell (file-append shadow "/sbin/nologin")))))))
 
 (define (default-php-fpm-config socket user group socket-user socket-group
-          pid-file log-file pm display-errors timezone workers-log-file)
+          pid-file log-file pm display-errors timezone workers-log-file
+          environment-variables)
   (apply mixed-text-file "php-fpm.conf"
          (flatten
           "[global]\n"
@@ -1068,6 +1072,10 @@ (define (default-php-fpm-config socket user group socket-user socket-group
               "pm.max_children =" (number->string pm.max-children) "\n"
               "pm.process_idle_timeout =" (number->string pm.process-idle-timeout) "s\n")))
 
+          (map (lambda (variable)
+                 ;; PHP-FPM will interpolate $VARIABLES from the outside environment.
+                 (list "env[" variable "] = $" variable "\n"))
+               (map car environment-variables))
 
           "php_flag[display_errors] = " (if display-errors "on" "off") "\n"
 
@@ -1081,7 +1089,8 @@ (define php-fpm-shepherd-service
   (match-lambda
     (($ <php-fpm-configuration> php socket user group socket-user socket-group
                                 pid-file log-file pm display-errors
-                                timezone workers-log-file file php-ini-file)
+                                timezone workers-log-file file php-ini-file
+                                environment-variables)
      (list (shepherd-service
             (provision '(php-fpm))
             (documentation "Run the php-fpm daemon.")
@@ -1092,10 +1101,25 @@ (define php-fpm-shepherd-service
                         #$(or file
                               (default-php-fpm-config socket user group
                                 socket-user socket-group pid-file log-file
-                                pm display-errors timezone workers-log-file))
+                                pm display-errors timezone workers-log-file
+                                environment-variables))
                         #$@(if php-ini-file
                                `("-c" ,php-ini-file)
                                '()))
+                      ;; Environment variables must be explicitly passed
+                      ;; through in PHP-FPM's configuration.  However, we
+                      ;; can't just set them there, since libraries loaded by
+                      ;; PHP (e.g. libcurl) will not see them if they are only
+                      ;; set there.  For those libraries, the variables also
+                      ;; need to be present in the "outer" environment, so set
+                      ;; them here as well.
+                      #:environment-variables
+                      (cons*
+                       #$@(map (match-lambda
+                                 ((variable . value)
+                                  #~(string-append #$variable "=" #$value)))
+                               environment-variables)
+                       (default-environment-variables))
                       #:pid-file #$pid-file))
             (stop #~(make-kill-destructor)))))))
 
-- 
2.41.0

[bug#63877] [PATCH 2/2] tests: web: Test environment variables are set for php-fpm.

[guix]

From: Timo Wilken <guix@twilken.net>

Test the new `environment-variables' field of <php-fpm-configuration> by
looking for a sentinel variable and value in the output of `phpinfo()'.

* gnu/tests/web.scm (run-php-fpm-test): Add test case.
---
 gnu/tests/web.scm | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/gnu/tests/web.scm b/gnu/tests/web.scm
index 16dc6bea49..f1688bfd3a 100644
--- a/gnu/tests/web.scm
+++ b/gnu/tests/web.scm
@@ -272,7 +272,10 @@ (define %php-fpm-os
   ;; Operating system under test.
   (simple-operating-system
    (service dhcp-client-service-type)
-   (service php-fpm-service-type)
+   (service php-fpm-service-type
+            (php-fpm-configuration
+             (environment-variables
+              '(("GUIX_TEST_PHPFPM_ENV" . "sentinel")))))
    (service nginx-service-type
             (nginx-configuration
              (server-blocks %php-fpm-nginx-server-blocks)))
@@ -345,6 +348,13 @@ (define marionette
                   (and matches
                        (match:substring matches 0))))))
 
+          (test-assert "php environment variable is applied"
+            (let-values (((response text)
+                          (http-get "http://localhost:8080/index.php"
+                                    #:decode-body? #t)))
+              (and (string-contains text "GUIX_TEST_PHPFPM_ENV")
+                   (string-contains text "sentinel"))))
+
           (test-end))))
 
   (gexp->derivation "php-fpm-test" test))
-- 
2.41.0