bug#72257: Non-English manuals for 1.4.0 are unavailable

bug#72257: Non-English manuals for 1.4.0 are unavailable

[Ludovic Courtès]

Hello,

Right now, non-English manuals for 1.4.0 are 404:

  https://guix.gnu.org/manual/de/html_node/
  https://guix.gnu.org/manual/es/html_node/
  https://guix.gnu.org/manual/fr/html_node/

The development version is available though:

  https://guix.gnu.org/manual/devel/de/html_node/
  https://guix.gnu.org/manual/devel/es/html_node/
  https://guix.gnu.org/manual/devel/fr/html_node/

Ludo’.

bug#72257: Non-English manuals for 1.4.0 are unavailable

[Ludovic Courtès]

Hi,

(+Cc translations team.)

Ludovic Courtès <ludo@gnu.org> skribis:

> Right now, non-English manuals for 1.4.0 are 404:
>
>   https://guix.gnu.org/manual/de/html_node/
>   https://guix.gnu.org/manual/es/html_node/
>   https://guix.gnu.org/manual/fr/html_node/

Looks like current po4a as invoked by ‘doc/build.scm’ doesn’t work:

--8<---------------cut here---------------start------------->8---
$ guix build /gnu/store/ffb4pkrfn9s8nxdmf65nziqhs4qlla2m-texinfo-manual-source.drv --check
The following derivations will be built:
  /gnu/store/ffb4pkrfn9s8nxdmf65nziqhs4qlla2m-texinfo-manual-source.drv
  /gnu/store/j87wq7m117c3d3ja8jla2jn2wkv1lg4k-guix-translated-texinfo.drv
substitute: updating substitutes from 'http://ci.guix.gnu.org'... 100.0%
building /gnu/store/j87wq7m117c3d3ja8jla2jn2wkv1lg4k-guix-translated-texinfo.drv...
File /gnu/store/gh2qlibnp6zy3ish7l5fxgq2v7k8n7f8-doc/po4a.cfg does not exist.
File /gnu/store/gh2qlibnp6zy3ish7l5fxgq2v7k8n7f8-doc/po4a.cfg does not exist.
File /gnu/store/gh2qlibnp6zy3ish7l5fxgq2v7k8n7f8-doc/po4a.cfg does not exist.
File /gnu/store/gh2qlibnp6zy3ish7l5fxgq2v7k8n7f8-doc/po4a.cfg does not exist.
File /gnu/store/gh2qlibnp6zy3ish7l5fxgq2v7k8n7f8-doc/po4a.cfg does not exist.
File /gnu/store/gh2qlibnp6zy3ish7l5fxgq2v7k8n7f8-doc/po4a.cfg does not exist.
File /gnu/store/gh2qlibnp6zy3ish7l5fxgq2v7k8n7f8-doc/po4a.cfg does not exist.
File /gnu/store/gh2qlibnp6zy3ish7l5fxgq2v7k8n7f8-doc/po4a.cfg does not exist.
In thread:
uncaught throw to %exception: (#<&invoke-error program: "/gnu/store/wf4gkjz3pqax3ids9xlgzg9pj2il7dni-po4a-0.69/bin/po4a" arguments: ("--no-update" "--variable" "localized=guix.fr.texi.tmp" "--variable" "master=guix.texi" "--variable" "po=./guix-manual.fr.po" "--variable" "pot=/tmp/fileNBHtSJ.pot" "--srcdir=/gnu/store/3fk7j3g14q1h6prb93xhy44f3f55rxbr-guix" "--destdir=." "/gnu/store/gh2qlibnp6zy3ish7l5fxgq2v7k8n7f8-doc/po4a.cfg") exit-status: 2 term-signal: #f stop-signal: #f>)
In thread:
uncaught throw to %exception: (#<&invoke-error program: "/gnu/store/wf4gkjz3pqax3ids9xlgzg9pj2il7dni-po4a-0.69/bin/po4a" arguments: ("--no-update" "--variable" "localized=guix.pt_BR.texi.tmp" "--variable" "master=guix.texi" "--variable" "po=./guix-manual.pt_BR.po" "--variable" "pot=/tmp/fileQkL7Re.pot" "--srcdir=/gnu/store/3fk7j3g14q1h6prb93xhy44f3f55rxbr-guix" "--destdir=." "/gnu/store/gh2qlibnp6zy3ish7l5fxgq2v7k8n7f8-doc/po4a.cfg") exit-status: 2 term-signal: #f stop-signal: #f>)
In thread:
uncaught throw to %exception: (#<&invoke-error program: "/gnu/store/wf4gkjz3pqax3ids9xlgzg9pj2il7dni-po4a-0.69/bin/po4a" arguments: ("--no-update" "--variable" "localized=guix.es.texi.tmp" "--variable" "master=guix.texi" "--variable" "po=./guix-manual.es.po" "--variable" "pot=/tmp/fileJ2khxq.pot" "--srcdir=/gnu/store/3fk7j3g14q1h6prb93xhy44f3f55rxbr-guix" "--destdir=." "/gnu/store/gh2qlibnp6zy3ish7l5fxgq2v7k8n7f8-doc/po4a.cfg") exit-status: 2 term-signal: #f stop-signal: #f>)
In thread:
uncaught throw to %exception: (#<&invoke-error program: "/gnu/store/wf4gkjz3pqax3ids9xlgzg9pj2il7dni-po4a-0.69/bin/po4a" arguments: ("--no-update" "--variable" "localized=guix.de.texi.tmp" "--variable" "master=guix.texi" "--variable" "po=./guix-manual.de.po" "--variable" "pot=/tmp/fileI2FBVS.pot" "--srcdir=/gnu/store/3fk7j3g14q1h6prb93xhy44f3f55rxbr-guix" "--destdir=." "/gnu/store/gh2qlibnp6zy3ish7l5fxgq2v7k8n7f8-doc/po4a.cfg") exit-status: 2 term-signal: #f stop-signal: #f>)
In thread:
uncaught throw to %exception: (#<&invoke-error program: "/gnu/store/wf4gkjz3pqax3ids9xlgzg9pj2il7dni-po4a-0.69/bin/po4a" arguments: ("--no-update" "--variable" "localized=guix-cookbook.sk.texi.tmp" "--variable" "master=guix-cookbook.texi" "--variable" "po=./guix-cookbook.sk.po" "--variable" "pot=/tmp/file8Dg90X.pot" "--srcdir=/gnu/store/3fk7j3g14q1h6prb93xhy44f3f55rxbr-guix" "--destdir=." "/gnu/store/gh2qlibnp6zy3ish7l5fxgq2v7k8n7f8-doc/po4a.cfg") exit-status: 2 term-signal: #f stop-signal: #f>)
In thread:
uncaught throw to %exception: (#<&invoke-error program: "/gnu/store/wf4gkjz3pqax3ids9xlgzg9pj2il7dni-po4a-0.69/bin/po4a" arguments: ("--no-update" "--variable" "localized=guix-cookbook.fr.texi.tmp" "--variable" "master=guix-cookbook.texi" "--variable" "po=./guix-cookbook.fr.po" "--variable" "pot=/tmp/file6io6rG.pot" "--srcdir=/gnu/store/3fk7j3g14q1h6prb93xhy44f3f55rxbr-guix" "--destdir=." "/gnu/store/gh2qlibnp6zy3ish7l5fxgq2v7k8n7f8-doc/po4a.cfg") exit-status: 2 term-signal: #f stop-signal: #f>)
In thread:
uncaught throw to %exception: (#<&invoke-error program: "/gnu/store/wf4gkjz3pqax3ids9xlgzg9pj2il7dni-po4a-0.69/bin/po4a" arguments: ("--no-update" "--variable" "localized=guix-cookbook.de.texi.tmp" "--variable" "master=guix-cookbook.texi" "--variable" "po=./guix-cookbook.de.po" "--variable" "pot=/tmp/filer0TSAC.pot" "--srcdir=/gnu/store/3fk7j3g14q1h6prb93xhy44f3f55rxbr-guix" "--destdir=." "/gnu/store/gh2qlibnp6zy3ish7l5fxgq2v7k8n7f8-doc/po4a.cfg") exit-status: 2 term-signal: #f stop-signal: #f>)
In thread:
uncaught throw to %exception: (#<&invoke-error program: "/gnu/store/wf4gkjz3pqax3ids9xlgzg9pj2il7dni-po4a-0.69/bin/po4a" arguments: ("--no-update" "--variable" "localized=guix-cookbook.ko.texi.tmp" "--variable" "master=guix-cookbook.texi" "--variable" "po=./guix-cookbook.ko.po" "--variable" "pot=/tmp/filebmkQ3a.pot" "--srcdir=/gnu/store/3fk7j3g14q1h6prb93xhy44f3f55rxbr-guix" "--destdir=." "/gnu/store/gh2qlibnp6zy3ish7l5fxgq2v7k8n7f8-doc/po4a.cfg") exit-status: 2 term-signal: #f stop-signal: #f>)

Some deprecated features have been used.  Set the environment
variable GUILE_WARN_DEPRECATED to "detailed" and rerun the
program to get more information.  Set it to "no" to suppress
this message.
successfully built /gnu/store/j87wq7m117c3d3ja8jla2jn2wkv1lg4k-guix-translated-texinfo.drv
--8<---------------cut here---------------end--------------->8---

Thoughts?

Ludo’.

bug#72257: Non-English manuals for 1.4.0 are unavailable

[Maxim Cournoyer]

[-- Attachment #1: Type: text/plain, Size: 107 bytes --]

Hi Ludo,

I think there is a patch on our tracker to use the latest po4a; perhaps it could help.

Maxim

[-- Attachment #2: Type: text/html, Size: 178 bytes --]

bug#72257: Non-English manuals for 1.4.0 are unavailable

[pelzflorian (Florian Pelz)]

Hello all, I do not understand why these old manuals are built with
po4a.cfg.  Why does it not use old guix 1.4.0 to build old manual 1.4.0?

po4a.cfg was added to guix.git in early June 2024 in commits starting from

commit 2d4af5247057505956e841d396ec78af9f769837
Author: gemmaro <gemmaro.dev@gmail.com>
Date:   Sun Apr 14 16:18:51 2024 +0900

    build: Update the POT creation task with the po4a command.
    
    * configure.ac (Documentation translation): Add the po4a program and remove
    the po4a-updatepo program.  The PO4A defined here can be used in
    Makefiles (e.g. po/doc/local.mk below).  The po4a command integrates the
    functionality of po4a-updatepo and is currently recommended for use [1].  The
    po4a-updatepo command has been replaced by the po4a command and are not used
    from anywhere.
    * po/doc/local.mk (%D%/%.pot): Use a po4a command instead of the po4a-updatepo
    for the POT generation.  This eliminates the warning of po4a-updatepo;
    "po4a-updatepo is deprecated.  The unified po4a(1) program is more convenient
    and less error prone."  Here, the file paths are passed using variables.  This
    is because the po4a command needs to be executed for each language and each
    file.  Dummy values are passed for those that are not needed to update the POT
    file.
    * po/doc/po4a.cfg: Add a po4a config file for the POT generation.  The po4a
    command requires a configulation file.
    
    [1] https://po4a.org/man/man7/po4a.7.php.en#lbAS
    
    Change-Id: Ica39a0d1f77b6a64d37c592ac2e693319443d3c5
    Signed-off-by: Florian Pelz <pelzflorian@pelzflorian.de>


The old manual as referred to in maintenance.git in
hydra/modules/sysadmin/web.scm was expected to build with the
guix/self.scm of old guix.git; it should not try to use po4a.cfg.

Current po4a also still includes the old po4a tools (po4a-updatepo and
po4a-translate) from before; they just are little maintained.

Regards,
Florian

bug#72257: Non-English manuals for 1.4.0 are unavailable

[pelzflorian (Florian Pelz)]

"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> writes:
> The old manual as referred to in maintenance.git in
> hydra/modules/sysadmin/web.scm was expected to build with the
> guix/self.scm of old guix.git; it should not try to use po4a.cfg.

The issue seems to be, maintenance.git in hydra/modules/sysadmin/web.scm
uses old guix 1.4.0’s doc/build.scm, but it imports the (guix self)
module by @@ from the new guix, which expects po4a.cfg.

Can we make it @@ the modules from old guix 1.4.0?

Regards,
Florian

bug#72257: Non-English manuals for 1.4.0 are unavailable

[pelzflorian (Florian Pelz)]

The easiest resolution would be to generally use an old 1.4.0 guix
package in with-extensions, because the modules from a guix@1.4.0 still
should work fine even for the latest devel manual (I have not tested,
but autonomous guix/self.scm ignores the changed Makefile).

Regards,
Florian

bug#72257: Non-English manuals for 1.4.0 are unavailable

[pelzflorian (Florian Pelz)]

Could someone knowledgeable in maintenance.git in
hydra/modules/sysadmin/web.scm change the guix-extensions to use an
older guix package earlier than June 2024, such as guix version 1.4.0?

I try

((options->transformation '((with-commit . "guix=v1.4.0"))) guix)

but could it be made to get a guix@1.4.0 with substitutes instead?

Or should it define a new guix@1.4.0 package with (package (inherit
guix))?

Should it look more like how guix.git’s guix/scripts/time-machine.scm
uses old guix?

Then the doc/build.scm would again @@ out to a (guix self) module that
does not require po4a.cfg, which is not yet present when building old
guix’s manual.

Regards,
Florian

bug#72257: Non-English manuals for 1.4.0 are unavailable

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 1083 bytes --]

Hello,

"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:

> "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> writes:
>> The old manual as referred to in maintenance.git in
>> hydra/modules/sysadmin/web.scm was expected to build with the
>> guix/self.scm of old guix.git; it should not try to use po4a.cfg.
>
> The issue seems to be, maintenance.git in hydra/modules/sysadmin/web.scm
> uses old guix 1.4.0’s doc/build.scm, but it imports the (guix self)
> module by @@ from the new guix, which expects po4a.cfg.
>
> Can we make it @@ the modules from old guix 1.4.0?

Alternatively, could we use (@ (guix self) translate-texi-manuals) as it
existed in Guix commit 5f86eebd240958001ab4f178005f355d24d9b7f1 by
simply copying it to ‘doc/build.scm’ on the ‘version-1.4.0’ branch?

That seems simpler and more orthogonal to me than special-casing
(sysadmin web).

Patch below.

WDYT?

Ludo’.

PS: I’ll be away from keyboard in the coming days so feel free to push
    to ‘version-1.4.0’ if it looks good to you.


[-- Attachment #2: Type: text/x-patch, Size: 5922 bytes --]

diff --git a/doc/build.scm b/doc/build.scm
index 0a5bddbcb6..345909bd57 100644
--- a/doc/build.scm
+++ b/doc/build.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2019-2023 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2019-2024 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2020 Björn Höfling <bjoern.hoefling@bjoernhoefling.de>
 ;;; Copyright © 2022 Maxim Cournoyer <maxim.cournoyer@gmail.com>
 ;;;
@@ -34,6 +34,7 @@
              (guix profiles)
              (guix utils)
              (git)
+             (gnu packages)
              (gnu packages base)
              (gnu packages compression)
              (gnu packages gawk)
@@ -52,8 +53,116 @@
 (define file-append*
   (@@ (guix self) file-append*))
 
-(define translated-texi-manuals
-  (@@ (guix self) translate-texi-manuals))
+(define (translated-texi-manuals source)
+  "Return the translated texinfo manuals built from SOURCE."
+  (define po4a
+    (specification->package "po4a"))
+
+  (define gettext-minimal
+    (specification->package "gettext-minimal"))
+
+  (define documentation
+    (file-append* source "doc"))
+
+  (define documentation-po
+    (file-append* source "po/doc"))
+
+  (define build
+    (with-imported-modules '((guix build utils) (guix build po))
+      #~(begin
+          (use-modules (guix build utils) (guix build po)
+                       (ice-9 match) (ice-9 regex) (ice-9 textual-ports)
+                       (ice-9 vlist) (ice-9 threads)
+                       (srfi srfi-1))
+
+          (define (translate-tmp-texi po source output)
+            "Translate Texinfo file SOURCE using messages from PO, and write
+the result to OUTPUT."
+            (invoke #+(file-append po4a "/bin/po4a-translate")
+              "-M" "UTF-8" "-L" "UTF-8" "-k" "0" "-f" "texinfo"
+              "-m" source "-p" po "-l" output))
+
+          (define (canonicalize-whitespace str)
+            ;; Change whitespace (newlines, etc.) in STR to #\space.
+            (string-map (lambda (chr)
+                          (if (char-set-contains? char-set:whitespace chr)
+                              #\space
+                              chr))
+                        str))
+
+          (define* (translate-texi prefix po lang
+                                   #:key (extras '()))
+            "Translate the manual for one language LANG using the PO file.
+PREFIX must be the prefix of the manual, 'guix' or 'guix-cookbook'.  EXTRAS is
+a list of extra files, such as '(\"contributing\")."
+            (for-each (lambda (file)
+                        (translate-tmp-texi po (string-append file ".texi")
+                                            (string-append file "." lang
+                                                           ".texi.tmp")))
+                      (cons prefix extras))
+
+            (for-each (lambda (file)
+                        (let* ((texi (string-append file "." lang ".texi"))
+                               (tmp  (string-append texi ".tmp")))
+                          (copy-file tmp texi)
+                          (translate-cross-references texi po)))
+                      (cons prefix extras)))
+
+          (define (available-translations directory domain)
+            ;; Return the list of available translations under DIRECTORY for
+            ;; DOMAIN, a gettext domain such as "guix-manual".  The result is
+            ;; a list of language/PO file pairs.
+            (filter-map (lambda (po)
+                          (let ((base (basename po)))
+                            (and (string-prefix? (string-append domain ".")
+                                                 base)
+                                 (match (string-split base #\.)
+                                   ((_ ... lang "po")
+                                    (cons lang po))))))
+                        (find-files directory
+                                    "\\.[a-z]{2}(_[A-Z]{2})?\\.po$")))
+
+          (define parallel-jobs
+            ;; Limit thread creation by 'n-par-for-each', mostly to put an
+            ;; upper bound on memory usage.
+            (min (parallel-job-count) 4))
+
+          (mkdir #$output)
+          (copy-recursively #$documentation "."
+                            #:log (%make-void-port "w"))
+
+          (for-each
+            (lambda (file)
+              (copy-file file (basename file)))
+            (find-files #$documentation-po ".*.po$"))
+
+          (setenv "GUIX_LOCPATH"
+                  #+(file-append glibc-utf8-locales "/lib/locale"))
+          (setenv "PATH" #+(file-append gettext-minimal "/bin"))
+          (setenv "LC_ALL" "en_US.UTF-8")
+          (setlocale LC_ALL "en_US.UTF-8")
+
+          (n-par-for-each parallel-jobs
+                          (match-lambda
+                            ((language . po)
+                             (translate-texi "guix" po language
+                                             #:extras '("contributing"))))
+                          (available-translations "." "guix-manual"))
+
+          (n-par-for-each parallel-jobs
+                          (match-lambda
+                            ((language . po)
+                             (translate-texi "guix-cookbook" po language)))
+                          (available-translations "." "guix-cookbook"))
+
+          (for-each (lambda (file)
+                      (install-file file #$output))
+                    (append
+                     (find-files "." "contributing\\..*\\.texi$")
+                     (find-files "." "guix\\..*\\.texi$")
+                     (find-files "." "guix-cookbook\\..*\\.texi$"))))))
+
+  (computed-file "guix-translated-texinfo" build))
 
 (define info-manual
   (@@ (guix self) info-manual))

bug#72257: Non-English manuals for 1.4.0 are unavailable

[pelzflorian (Florian Pelz)]

Thank you Ludo, with your diff, building doc/build.scm failed previously
and works for me now.  I tested and pushed it to branch `version-1.4.0'
yesterday evening.

But mcron should have picked it up by now, but has not.  Maybe someone
could look at the logs?  Or wait some more.

Regards,
Florian

bug#72257: Non-English manuals for 1.4.0 are unavailable

[pelzflorian (Florian Pelz)]

The manual builds now.  Closing.

(I had miscalculated how long it takes, but thanks if someone intervened
and initiated the build.)

Regards,
Florian