From mboxrd@z Thu Jan 1 00:00:00 1970 From: "pelzflorian (Florian Pelz)" Subject: bug#26302: [website] translations Date: Mon, 7 Oct 2019 10:15:03 +0200 Message-ID: <20191007081502.wog4q4wjptvhmejf@pelzflorian.localdomain> References: <20170329154040.ddscahwp2agknihb@abyayala> <16d07cb5a1d.ca69255821627.8144164908931126811@zoho.com> <20190908171638.cna67eearj4rbn2k@pelzflorian.localdomain> <16d18a5a9e8.12ab66c8254154.2756956535677606704@zoho.com> <20190915201819.3yxm25fayvbxwdpl@pelzflorian.localdomain> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="sdu57zujk3qaqzhp" Content-Transfer-Encoding: 8bit Return-path: Received: from eggs.gnu.org ([2001:470:142:3::10]:46579) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iHOBa-0005Te-9X for bug-guix@gnu.org; Mon, 07 Oct 2019 04:16:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iHOBW-0002W8-R9 for bug-guix@gnu.org; Mon, 07 Oct 2019 04:16:06 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:38523) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iHOBW-0002Vs-MB for bug-guix@gnu.org; Mon, 07 Oct 2019 04:16:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1iHOBW-0007qU-GJ for bug-guix@gnu.org; Mon, 07 Oct 2019 04:16:02 -0400 Sender: "Debbugs-submit" Resent-Message-ID: Content-Disposition: inline In-Reply-To: <20190915201819.3yxm25fayvbxwdpl@pelzflorian.localdomain> List-Id: Bug reports for GNU Guix List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guix-bounces+gcggb-bug-guix=m.gmane.org@gnu.org Sender: "bug-Guix" To: sirgazil , Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 26302 <26302@debbugs.gnu.org> --sdu57zujk3qaqzhp Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit On Sun, Sep 15, 2019 at 10:18:20PM +0200, pelzflorian (Florian Pelz) wrote: > What’s missing is code for adding the nginx plugin > > https://www.nginx.com/resources/wiki/modules/accept_language/ > > to Guix so it can be used on berlin.scm. It seems to offer all that > is required for redirecting existing URLs. > Please review the attached patch to Guix that adds nginx-mod-accept-language. (This is not the upstream name; various nginx modules use no consistent naming scheme for nginx modules.) I have put it in a separate module from (gnu packages web) due to an import cycle with tar/(gnu packages base). The second patch allows using dynamic modules with the nginx service. Using these patches, the attached slightly wrong patch to guix/maintenance.git is meant to make the web server default to serving the language configured in the browser when no language code is specified in the URL, so old URLs remain functional. (Redirects of non-html URLs are wrong and nginx does not run with this config, perhaps I should use rewrite instead of try_files. I can try later today.) Nonetheless, href links on the website point to the localized pages on purpose so if someone changes the language the change remains in effect when following a link. Some people on the internet (I forgot who) preferred having a cookie store the preferred language. That would make it possible to continue using the old URLs in href links on the website. However, I am not sure how it would affect SEO. On Thu, Sep 19, 2019 at 01:50:10PM +0200, Ludovic Courtès wrote: > We can then maybe set up a ‘static-web-site’ service on berlin, with > appropriate nginx rules, to build and publish the manual at a separate > URL so we can all test it. See hydra/berlin.scm in maintenance.git. > I see you @Ludo added such a service, but I cannot reach it. Perhaps I am using the wrong URL. Regards, Florian --sdu57zujk3qaqzhp Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="0001-wip-gnu-Add-ngx_http_accept_language_module.patch" Content-Transfer-Encoding: 8bit >From 9ec69c888b978cb870a5873af8e327541fe4ef7a Mon Sep 17 00:00:00 2001 From: Florian Pelz Date: Sun, 6 Oct 2019 20:45:34 +0200 Subject: [PATCH 1/2] [wip] gnu: Add ngx_http_accept_language_module. * gnu/packages/web-xyz.scm: New file. * gnu/local.mk (GNU_SYSTEM_MODULES): Add package. --- gnu/local.mk | 1 + gnu/packages/web-xyz.scm | 175 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 176 insertions(+) create mode 100644 gnu/packages/web-xyz.scm diff --git a/gnu/local.mk b/gnu/local.mk index 54ae09f619..96a6c0087a 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -497,6 +497,7 @@ GNU_SYSTEM_MODULES = \ %D%/packages/wdiff.scm \ %D%/packages/web.scm \ %D%/packages/web-browsers.scm \ + %D%/packages/web-xyz.scm \ %D%/packages/webkit.scm \ %D%/packages/wget.scm \ %D%/packages/wicd.scm \ diff --git a/gnu/packages/web-xyz.scm b/gnu/packages/web-xyz.scm new file mode 100644 index 0000000000..61e7f847a7 --- /dev/null +++ b/gnu/packages/web-xyz.scm @@ -0,0 +1,175 @@ +;;; GNU Guix --- Functional package management for GNU +;;;; TODO should I really add copyright lines for people I copied from?? +;;; Copyright © 2014, 2015 Mark H Weaver +;;; Copyright © 2016 Tobias Geerinckx-Rice +;;; Copyright © 2017, 2018 Marius Bakke + +;;; Copyright © 2019 Florian Pelz +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see . + +(define-module (gnu packages web-xyz) + #:use-module (guix build-system gnu) + #:use-module (guix gexp) + #:use-module (guix git-download) + #:use-module ((guix licenses) #:prefix license:) + #:use-module (guix packages) + #:use-module (gnu packages base) + #:use-module (gnu packages compression) + #:use-module (gnu packages pcre) + #:use-module (gnu packages tls) + #:use-module (gnu packages web) + #:use-module (ice-9 match) + #:use-module ((srfi srfi-1) #:select (delete-duplicates))) + +(define-public nginx-mod-accept-language + (let ((commit "2f69842f83dac77f7d98b41a2b31b13b87aeaba7") + (revision "1")) + (package + (name "nginx-mod-accept-language") + (version (git-version "0.0.0" ;upstream has no version number + revision commit)) + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/giom/nginx_accept_language_module.git") + (commit commit))) + (file-name (git-file-name name version)) + (sha256 + (base32 "1hjysrl15kh5233w7apq298cc2bp4q1z5mvaqcka9pdl90m0vhbw")) + (modules '((guix build utils) + (ice-9 popen))) + (snippet + #~(begin + ;; the nginx source code is part of the module’s source + (format #t "decompressing nginx source code~%") + (call-with-output-file "nginx.tar" + (lambda (out) + (let ((pipe (open-pipe* OPEN_READ + #+(file-append gzip "/bin/gzip") "-cd" + #$(package-source nginx)))) + (dump-port pipe out) + (unless (= (status:exit-val (close-pipe pipe)) 0) + (error "gzip decompress failed"))))) + (invoke #+(file-append tar "/bin/tar") "xvf" "nginx.tar" + "--strip-components=1") + (delete-file "nginx.tar") + #t)))) + (build-system gnu-build-system) + (inputs `(("openssl" ,openssl) + ("pcre" ,pcre) + ("zlib" ,zlib))) + (arguments + `(#:tests? #f ; no test target + #:make-flags (list "modules") + #:modules ((guix build utils) + (guix build gnu-build-system) + (ice-9 regex) + (ice-9 textual-ports)) + #:phases + (modify-phases %standard-phases + (add-after 'unpack 'convert-to-dynamic-module + (lambda _ + (begin + (with-atomic-file-replacement "config" + (lambda (in out) + ;; cf. https://www.nginx.com/resources/wiki/extending/new_config/ + (format out "ngx_module_type=HTTP~%") + (format out "ngx_module_name=\ +ngx_http_accept_language_module~%") + (let* ((str (get-string-all in)) + (rx (make-regexp + "NGX_ADDON_SRCS=\"\\$NGX_ADDON_SRCS (.*)\"")) + (m (regexp-exec rx str)) + (srcs (match:substring m 1))) + (format out (string-append "ngx_module_srcs=\"" + srcs "\"~%"))) + (format out ". auto/module~%") + (format out "ngx_addon_name=$ngx_module_name~%")))))) + (add-before 'configure 'patch-/bin/sh + (lambda _ + (substitute* "auto/feature" + (("/bin/sh") (which "sh"))) + #t)) + (replace 'configure + (lambda* (#:key outputs #:allow-other-keys) + (let ((flags + (list ;; A copy of nginx’ flags follows, otherwise we + ;; get a binary compatibility error. FIXME: Code + ;; duplication is bad. + (string-append "--prefix=" (assoc-ref outputs "out")) + "--with-http_ssl_module" + "--with-http_v2_module" + "--with-pcre-jit" + "--with-debug" + ;; Even when not cross-building, we pass the + ;; --crossbuild option to avoid customizing for the + ;; kernel version on the build machine. + ,(let ((system "Linux") ; uname -s + (release "3.2.0") ; uname -r + ;; uname -m + (machine (match (or (%current-target-system) + (%current-system)) + ("x86_64-linux" "x86_64") + ("i686-linux" "i686") + ("mips64el-linux" "mips64") + ;; Prevent errors when querying + ;; this package on unsupported + ;; platforms, e.g. when running + ;; "guix package --search=" + (_ "UNSUPPORTED")))) + (string-append "--crossbuild=" + system ":" release ":" machine)) + ;; The following are the args decribed on + ;; . + ;; Enabling --with-compat here and in the nginx package + ;; would ensure binary compatibility even when using + ;; different configure options from the main nginx + ;; package. This is not needed for Guix. + ;; "--with-compat" + "--add-dynamic-module=."))) + (setenv "CC" "gcc") + (format #t "environment variable `CC' set to `gcc'~%") + (format #t "configure flags: ~s~%" flags) + (apply invoke "./configure" flags) + #t))) + (replace 'install + (lambda* (#:key outputs #:allow-other-keys) + (let* ((out (assoc-ref outputs "out")) + (modules-dir (string-append out "/etc/nginx/modules")) + (doc-dir (string-append + out "/share/doc/nginx-mod-accept-language"))) + (mkdir-p modules-dir) + (copy-file "objs/ngx_http_accept_language_module.so" + (string-append + modules-dir "/ngx_http_accept_language_module.so")) + (mkdir-p doc-dir) + (copy-file "README.textile" + (string-append doc-dir "/README.textile")) + #t)))))) + (home-page + "https://www.nginx.com/resources/wiki/modules/accept_language/") + (synopsis "Nginx module for parsing the Accept-Language HTTP header") + (description + "Nginx module that parses the Accept-Language field in HTTP headers and +chooses the most suitable locale for the user from the list of locales +supported at your website.") + (license (delete-duplicates + (cons license:bsd-2 ;license of nginx-mod-accept-language + (package-license nginx))))))) ;the module’s code is linked + ;statically with nginx, therefore + ;nginx’ licenses apply to its binary -- 2.23.0 --sdu57zujk3qaqzhp Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: attachment; filename="0002-services-Make-it-possible-to-include-dynamic-modules.patch" Content-Transfer-Encoding: 8bit >From 7747a416feae7139c2b2661ffeb37f2eb03eb2f9 Mon Sep 17 00:00:00 2001 From: Florian Pelz Date: Sun, 6 Oct 2019 15:31:23 +0200 Subject: [PATCH 2/2] services: Make it possible to include dynamic modules in nginx. * gnu/services/web.scm (): Add load-modules field. (nginx-configuration-load-modules): New field accessor. (emit-load-module): New procedure. (default-nginx-config): Add support for the load-modules field. * doc/guix.texi (NGINX): Document it. --- doc/guix.texi | 4 ++++ gnu/services/web.scm | 8 ++++++++ 2 files changed, 12 insertions(+) diff --git a/doc/guix.texi b/doc/guix.texi index 42d2d77ae6..5c39e26155 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -19718,6 +19718,10 @@ use the size of the processors cache line. @item @code{server-names-hash-bucket-max-size} (default: @code{#f}) Maximum bucket size for the server names hash tables. +@item @code{load-modules} (default: @code{'()}) +List of nginx dynamic modules to load. Should be a list of strings or +string valued G-expressions. + @item @code{extra-content} (default: @code{""}) Extra content for the @code{http} block. Should be string or a string valued G-expression. diff --git a/gnu/services/web.scm b/gnu/services/web.scm index 899be1c168..a68f0235b7 100644 --- a/gnu/services/web.scm +++ b/gnu/services/web.scm @@ -9,6 +9,7 @@ ;;; Copyright © 2018 Pierre-Antoine Rouby ;;; Copyright © 2017, 2018, 2019 Christopher Baines ;;; Copyright © 2018 Marius Bakke +;;; Copyright © 2019 Florian Pelz ;;; ;;; This file is part of GNU Guix. ;;; @@ -95,6 +96,7 @@ nginx-configuration-upstream-blocks nginx-configuration-server-names-hash-bucket-size nginx-configuration-server-names-hash-bucket-max-size + nginx-configuration-load-modules nginx-configuration-extra-content nginx-configuration-file @@ -522,6 +524,7 @@ (default #f)) (server-names-hash-bucket-max-size nginx-configuration-server-names-hash-bucket-max-size (default #f)) + (load-modules nginx-configuration-load-modules (default '())) (extra-content nginx-configuration-extra-content (default "")) (file nginx-configuration-file ;#f | string | file-like @@ -542,6 +545,9 @@ of index files." ((? string? str) (list str " "))) names)) +(define (emit-load-module module) + (list "load_module " module ";\n")) + (define emit-nginx-location-config (match-lambda (($ uri body) @@ -615,12 +621,14 @@ of index files." server-blocks upstream-blocks server-names-hash-bucket-size server-names-hash-bucket-max-size + load-modules extra-content) (apply mixed-text-file "nginx.conf" (flatten "user nginx nginx;\n" "pid " run-directory "/pid;\n" "error_log " log-directory "/error.log info;\n" + (map emit-load-module load-modules) "http {\n" " client_body_temp_path " run-directory "/client_body_temp;\n" " proxy_temp_path " run-directory "/proxy_temp;\n" -- 2.23.0 --sdu57zujk3qaqzhp Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="0001-berlin-Redirect-to-localized-website-depending-on-Ac.patch" >From ea5edd15586722b3557912e81171e69f7be339fa Mon Sep 17 00:00:00 2001 From: Florian Pelz Date: Mon, 7 Oct 2019 07:58:30 +0200 Subject: [PATCH] berlin: Redirect to localized website depending on Accept-Language header. * hydra/nginx/berlin.scm (guix.gnu.org-locations): Redirect html URLs. (%nginx-configuration): Load required nginx dynamic module. --- hydra/nginx/berlin.scm | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/hydra/nginx/berlin.scm b/hydra/nginx/berlin.scm index df90873..9085183 100644 --- a/hydra/nginx/berlin.scm +++ b/hydra/nginx/berlin.scm @@ -1,6 +1,7 @@ ;; Nginx configuration for ci.guix.info -(use-modules (gnu services web)) +(use-modules (gnu packages web-xyz) + (gnu services web)) ;; TODO: these settings cannot currently expressed with Guix: @@ -462,6 +463,16 @@ PUBLISH-URL." (nginx-location-configuration (uri "~ \\.pdf$") ;*.pdf at the top level (body (list "root /srv/guix-pdfs;"))) + (nginx-location-configuration + (uri "~ (.*)/$") + (body (list + "try_files $1/index.html;"))) + (nginx-location-configuration + (uri "~ (.html|.htm)$") + (body (list + ;; put en first so it is the default: + "set_from_accept_language $lang en de;" + "try_files /$lang/$uri $uri =404;"))) (nginx-location-configuration ;certbot (uri "/.well-known") @@ -758,5 +769,11 @@ PUBLISH-URL." (define %nginx-configuration (nginx-configuration (server-blocks %berlin-servers) + (load-modules + (list + ;; We need this module for redirecting users to the localized + ;; website of their choice. + (file-append nginx-mod-accept-language "\ +/etc/nginx/modules/ngx_http_accept_language_module.so"))) (extra-content (string-join %extra-content "\n")))) -- 2.23.0 --sdu57zujk3qaqzhp--