bug#26302: [website] translations

["pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de>]

[-- Attachment #1: Type: text/plain, Size: 1905 bytes --]

On Sun, Sep 15, 2019 at 10:18:20PM +0200, pelzflorian (Florian Pelz) wrote:
> What’s missing is code for adding the nginx plugin
> 
> https://www.nginx.com/resources/wiki/modules/accept_language/
> 
> to Guix so it can be used on berlin.scm.  It seems to offer all that
> is required for redirecting existing URLs.
> 

Please review the attached patch to Guix that adds
nginx-mod-accept-language.  (This is not the upstream name; various
nginx modules use no consistent naming scheme for nginx modules.)  I
have put it in a separate module from (gnu packages web) due to an
import cycle with tar/(gnu packages base).  The second patch allows
using dynamic modules with the nginx service.

Using these patches, the attached slightly wrong patch to
guix/maintenance.git is meant to make the web server default to
serving the language configured in the browser when no language code
is specified in the URL, so old URLs remain functional.  (Redirects of
non-html URLs are wrong and nginx does not run with this config,
perhaps I should use rewrite instead of try_files.  I can try later
today.)

Nonetheless, href links on the website point to the localized pages on
purpose so if someone changes the language the change remains in
effect when following a link.

Some people on the internet (I forgot who) preferred having a cookie
store the preferred language.  That would make it possible to continue
using the old URLs in href links on the website.  However, I am not
sure how it would affect SEO.

On Thu, Sep 19, 2019 at 01:50:10PM +0200, Ludovic Courtès wrote:
> We can then maybe set up a ‘static-web-site’ service on berlin, with
> appropriate nginx rules, to build and publish the manual at a separate
> URL so we can all test it.  See hydra/berlin.scm in maintenance.git.
> 

I see you @Ludo added such a service, but I cannot reach it.  Perhaps
I am using the wrong URL.


Regards,
Florian

[-- Attachment #2: 0001-wip-gnu-Add-ngx_http_accept_language_module.patch --]
[-- Type: text/plain, Size: 9990 bytes --]

From 9ec69c888b978cb870a5873af8e327541fe4ef7a Mon Sep 17 00:00:00 2001
From: Florian Pelz <pelzflorian@pelzflorian.de>
Date: Sun, 6 Oct 2019 20:45:34 +0200
Subject: [PATCH 1/2] [wip] gnu: Add ngx_http_accept_language_module.

* gnu/packages/web-xyz.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add package.
---
 gnu/local.mk             |   1 +
 gnu/packages/web-xyz.scm | 175 +++++++++++++++++++++++++++++++++++++++
 2 files changed, 176 insertions(+)
 create mode 100644 gnu/packages/web-xyz.scm

diff --git a/gnu/local.mk b/gnu/local.mk
index 54ae09f619..96a6c0087a 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -497,6 +497,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/packages/wdiff.scm			\
   %D%/packages/web.scm				\
   %D%/packages/web-browsers.scm			\
+  %D%/packages/web-xyz.scm			\
   %D%/packages/webkit.scm			\
   %D%/packages/wget.scm				\
   %D%/packages/wicd.scm				\
diff --git a/gnu/packages/web-xyz.scm b/gnu/packages/web-xyz.scm
new file mode 100644
index 0000000000..61e7f847a7
--- /dev/null
+++ b/gnu/packages/web-xyz.scm
@@ -0,0 +1,175 @@
+;;; GNU Guix --- Functional package management for GNU
+;;;; TODO should I really add copyright lines for people I copied from??
+;;; Copyright © 2014, 2015 Mark H Weaver <mhw@netris.org>
+;;; Copyright © 2016 Tobias Geerinckx-Rice <me@tobias.gr>
+;;; Copyright © 2017, 2018 Marius Bakke <mbakke@fastmail.com>
+
+;;; Copyright © 2019 Florian Pelz <pelzflorian@pelzflorian.de>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages web-xyz)
+  #:use-module (guix build-system gnu)
+  #:use-module (guix gexp)
+  #:use-module (guix git-download)
+  #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix packages)
+  #:use-module (gnu packages base)
+  #:use-module (gnu packages compression)
+  #:use-module (gnu packages pcre)
+  #:use-module (gnu packages tls)
+  #:use-module (gnu packages web)
+  #:use-module (ice-9 match)
+  #:use-module ((srfi srfi-1) #:select (delete-duplicates)))
+
+(define-public nginx-mod-accept-language
+  (let ((commit "2f69842f83dac77f7d98b41a2b31b13b87aeaba7")
+        (revision "1"))
+    (package
+      (name "nginx-mod-accept-language")
+      (version (git-version "0.0.0" ;upstream has no version number
+                            revision commit))
+      (source
+       (origin
+         (method git-fetch)
+         (uri (git-reference
+               (url "https://github.com/giom/nginx_accept_language_module.git")
+               (commit commit)))
+         (file-name (git-file-name name version))
+         (sha256
+          (base32 "1hjysrl15kh5233w7apq298cc2bp4q1z5mvaqcka9pdl90m0vhbw"))
+         (modules '((guix build utils)
+                    (ice-9 popen)))
+         (snippet
+          #~(begin
+              ;; the nginx source code is part of the module’s source
+              (format #t "decompressing nginx source code~%")
+              (call-with-output-file "nginx.tar"
+                (lambda (out)
+                  (let ((pipe (open-pipe* OPEN_READ
+                                          #+(file-append gzip "/bin/gzip") "-cd"
+                                          #$(package-source nginx))))
+                    (dump-port pipe out)
+                    (unless (= (status:exit-val (close-pipe pipe)) 0)
+                      (error "gzip decompress failed")))))
+              (invoke #+(file-append tar "/bin/tar") "xvf" "nginx.tar"
+                      "--strip-components=1")
+              (delete-file "nginx.tar")
+              #t))))
+      (build-system gnu-build-system)
+      (inputs `(("openssl" ,openssl)
+                ("pcre" ,pcre)
+                ("zlib" ,zlib)))
+      (arguments
+       `(#:tests? #f                      ; no test target
+         #:make-flags (list "modules")
+         #:modules ((guix build utils)
+                    (guix build gnu-build-system)
+                    (ice-9 regex)
+                    (ice-9 textual-ports))
+         #:phases
+         (modify-phases %standard-phases
+           (add-after 'unpack 'convert-to-dynamic-module
+             (lambda _
+               (begin
+                 (with-atomic-file-replacement "config"
+                   (lambda (in out)
+                     ;; cf. https://www.nginx.com/resources/wiki/extending/new_config/
+                     (format out "ngx_module_type=HTTP~%")
+                     (format out "ngx_module_name=\
+ngx_http_accept_language_module~%")
+                     (let* ((str (get-string-all in))
+                            (rx (make-regexp
+                                 "NGX_ADDON_SRCS=\"\\$NGX_ADDON_SRCS (.*)\""))
+                            (m (regexp-exec rx str))
+                            (srcs (match:substring m 1)))
+                       (format out (string-append "ngx_module_srcs=\""
+                                                  srcs "\"~%")))
+                     (format out ". auto/module~%")
+                     (format out "ngx_addon_name=$ngx_module_name~%"))))))
+           (add-before 'configure 'patch-/bin/sh
+             (lambda _
+               (substitute* "auto/feature"
+                 (("/bin/sh") (which "sh")))
+               #t))
+           (replace 'configure
+             (lambda* (#:key outputs #:allow-other-keys)
+               (let ((flags
+                      (list ;; A copy of nginx’ flags follows, otherwise we
+                            ;; get a binary compatibility error.  FIXME: Code
+                            ;; duplication is bad.
+                       (string-append "--prefix=" (assoc-ref outputs "out"))
+                       "--with-http_ssl_module"
+                       "--with-http_v2_module"
+                       "--with-pcre-jit"
+                       "--with-debug"
+                       ;; Even when not cross-building, we pass the
+                       ;; --crossbuild option to avoid customizing for the
+                       ;; kernel version on the build machine.
+                       ,(let ((system "Linux")    ; uname -s
+                              (release "3.2.0")   ; uname -r
+                              ;; uname -m
+                              (machine (match (or (%current-target-system)
+                                                  (%current-system))
+                                         ("x86_64-linux"   "x86_64")
+                                         ("i686-linux"     "i686")
+                                         ("mips64el-linux" "mips64")
+                                         ;; Prevent errors when querying
+                                         ;; this package on unsupported
+                                         ;; platforms, e.g. when running
+                                         ;; "guix package --search="
+                                         (_                "UNSUPPORTED"))))
+                          (string-append "--crossbuild="
+                                         system ":" release ":" machine))
+                       ;; The following are the args decribed on
+                       ;; <https://www.nginx.com/blog/compiling-dynamic-modules-nginx-plus>.
+                       ;; Enabling --with-compat here and in the nginx package
+                       ;; would ensure binary compatibility even when using
+                       ;; different configure options from the main nginx
+                       ;; package.  This is not needed for Guix.
+                       ;; "--with-compat"
+                       "--add-dynamic-module=.")))
+                 (setenv "CC" "gcc")
+                 (format #t "environment variable `CC' set to `gcc'~%")
+                 (format #t "configure flags: ~s~%" flags)
+                 (apply invoke "./configure" flags)
+                 #t)))
+           (replace 'install
+             (lambda* (#:key outputs #:allow-other-keys)
+               (let* ((out (assoc-ref outputs "out"))
+                      (modules-dir (string-append out "/etc/nginx/modules"))
+                      (doc-dir (string-append
+                                out "/share/doc/nginx-mod-accept-language")))
+                 (mkdir-p modules-dir)
+                 (copy-file "objs/ngx_http_accept_language_module.so"
+                            (string-append
+                             modules-dir "/ngx_http_accept_language_module.so"))
+                 (mkdir-p doc-dir)
+                 (copy-file "README.textile"
+                            (string-append doc-dir "/README.textile"))
+                 #t))))))
+      (home-page
+       "https://www.nginx.com/resources/wiki/modules/accept_language/")
+      (synopsis "Nginx module for parsing the Accept-Language HTTP header")
+      (description
+       "Nginx module that parses the Accept-Language field in HTTP headers and
+chooses the most suitable locale for the user from the list of locales
+supported at your website.")
+      (license (delete-duplicates
+                (cons license:bsd-2 ;license of nginx-mod-accept-language
+                      (package-license nginx))))))) ;the module’s code is linked
+                                        ;statically with nginx, therefore
+                                        ;nginx’ licenses apply to its binary
-- 
2.23.0


[-- Attachment #3: 0002-services-Make-it-possible-to-include-dynamic-modules.patch --]
[-- Type: text/plain, Size: 3418 bytes --]

From 7747a416feae7139c2b2661ffeb37f2eb03eb2f9 Mon Sep 17 00:00:00 2001
From: Florian Pelz <pelzflorian@pelzflorian.de>
Date: Sun, 6 Oct 2019 15:31:23 +0200
Subject: [PATCH 2/2] services: Make it possible to include dynamic modules in
 nginx.

* gnu/services/web.scm (<nginx-configuration>): Add load-modules field.
(nginx-configuration-load-modules): New field accessor.
(emit-load-module): New procedure.
(default-nginx-config): Add support for the load-modules field.
* doc/guix.texi (NGINX): Document it.
---
 doc/guix.texi        | 4 ++++
 gnu/services/web.scm | 8 ++++++++
 2 files changed, 12 insertions(+)

diff --git a/doc/guix.texi b/doc/guix.texi
index 42d2d77ae6..5c39e26155 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -19718,6 +19718,10 @@ use the size of the processors cache line.
 @item @code{server-names-hash-bucket-max-size} (default: @code{#f})
 Maximum bucket size for the server names hash tables.
 
+@item @code{load-modules} (default: @code{'()})
+List of nginx dynamic modules to load.  Should be a list of strings or
+string valued G-expressions.
+
 @item @code{extra-content} (default: @code{""})
 Extra content for the @code{http} block.  Should be string or a string
 valued G-expression.
diff --git a/gnu/services/web.scm b/gnu/services/web.scm
index 899be1c168..a68f0235b7 100644
--- a/gnu/services/web.scm
+++ b/gnu/services/web.scm
@@ -9,6 +9,7 @@
 ;;; Copyright © 2018 Pierre-Antoine Rouby <pierre-antoine.rouby@inria.fr>
 ;;; Copyright © 2017, 2018, 2019 Christopher Baines <mail@cbaines.net>
 ;;; Copyright © 2018 Marius Bakke <mbakke@fastmail.com>
+;;; Copyright © 2019 Florian Pelz <pelzflorian@pelzflorian.de>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -95,6 +96,7 @@
             nginx-configuration-upstream-blocks
             nginx-configuration-server-names-hash-bucket-size
             nginx-configuration-server-names-hash-bucket-max-size
+            nginx-configuration-load-modules
             nginx-configuration-extra-content
             nginx-configuration-file
 
@@ -522,6 +524,7 @@
                                  (default #f))
   (server-names-hash-bucket-max-size nginx-configuration-server-names-hash-bucket-max-size
                                      (default #f))
+  (load-modules nginx-configuration-load-modules (default '()))
   (extra-content nginx-configuration-extra-content
                  (default ""))
   (file          nginx-configuration-file         ;#f | string | file-like
@@ -542,6 +545,9 @@ of index files."
         ((? string? str) (list str " ")))
       names))
 
+(define (emit-load-module module)
+  (list "load_module " module ";\n"))
+
 (define emit-nginx-location-config
   (match-lambda
     (($ <nginx-location-configuration> uri body)
@@ -615,12 +621,14 @@ of index files."
                  server-blocks upstream-blocks
                  server-names-hash-bucket-size
                  server-names-hash-bucket-max-size
+                 load-modules
                  extra-content)
    (apply mixed-text-file "nginx.conf"
           (flatten
            "user nginx nginx;\n"
            "pid " run-directory "/pid;\n"
            "error_log " log-directory "/error.log info;\n"
+           (map emit-load-module load-modules)
            "http {\n"
            "    client_body_temp_path " run-directory "/client_body_temp;\n"
            "    proxy_temp_path " run-directory "/proxy_temp;\n"
-- 
2.23.0


[-- Attachment #4: 0001-berlin-Redirect-to-localized-website-depending-on-Ac.patch --]
[-- Type: text/plain, Size: 1922 bytes --]

From ea5edd15586722b3557912e81171e69f7be339fa Mon Sep 17 00:00:00 2001
From: Florian Pelz <pelzflorian@pelzflorian.de>
Date: Mon, 7 Oct 2019 07:58:30 +0200
Subject: [PATCH] berlin: Redirect to localized website depending on
 Accept-Language header.

* hydra/nginx/berlin.scm (guix.gnu.org-locations): Redirect html URLs.
(%nginx-configuration): Load required nginx dynamic module.
---
 hydra/nginx/berlin.scm | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)

diff --git a/hydra/nginx/berlin.scm b/hydra/nginx/berlin.scm
index df90873..9085183 100644
--- a/hydra/nginx/berlin.scm
+++ b/hydra/nginx/berlin.scm
@@ -1,6 +1,7 @@
 ;; Nginx configuration for ci.guix.info
 
-(use-modules (gnu services web))
+(use-modules (gnu packages web-xyz)
+             (gnu services web))
 
 ;; TODO: these settings cannot currently expressed with Guix:
 
@@ -462,6 +463,16 @@ PUBLISH-URL."
    (nginx-location-configuration
     (uri "~ \\.pdf$")                             ;*.pdf at the top level
     (body (list "root /srv/guix-pdfs;")))
+   (nginx-location-configuration
+    (uri "~ (.*)/$")
+    (body (list
+           "try_files $1/index.html;")))
+   (nginx-location-configuration
+    (uri "~ (.html|.htm)$")
+    (body (list
+           ;; put en first so it is the default:
+           "set_from_accept_language $lang en de;"
+           "try_files /$lang/$uri $uri =404;")))
 
    (nginx-location-configuration                  ;certbot
     (uri "/.well-known")
@@ -758,5 +769,11 @@ PUBLISH-URL."
 (define %nginx-configuration
   (nginx-configuration
    (server-blocks %berlin-servers)
+   (load-modules
+    (list
+     ;; We need this module for redirecting users to the localized
+     ;; website of their choice.
+     (file-append nginx-mod-accept-language "\
+/etc/nginx/modules/ngx_http_accept_language_module.so")))
    (extra-content
     (string-join %extra-content "\n"))))
-- 
2.23.0