[bug#54302] [PATCH] nls: Implement translation thresholds.

["Ludovic Courtès" <ludo@gnu.org>]

Hi!

Julien Lepiller <julien@lepiller.eu> skribis:

> manual and cookbook: only include new languages when they reach 10%
> completion. Remove languages when they fall below 5%.

SGTM (or even 15%/10%).

> website (unrelated to this repo, but still important): only include new
> languages when they reach 80% completion. Remove languages when they
> fall below 60%. The reason for the higher threshold is that the website
> acts as some sort of advertisement, so we want a higher quality than
> half English half translated.

SGTM.

>>From 37071410629a7d70c9b4e4f673f2c625d3ed4b47 Mon Sep 17 00:00:00 2001
> Message-Id: <37071410629a7d70c9b4e4f673f2c625d3ed4b47.1646763369.git.julien@lepiller.eu>
> From: Julien Lepiller <julien@lepiller.eu>
> Date: Tue, 8 Mar 2022 13:14:58 +0100
> Subject: [PATCH 1/3] doc: Document inclusion requirements for new
>  translations.
>
> * doc/contributing.texi (Translating Guix)[Conditions for Inclusion]:
> New section.

[...]

> +There are no conditions for adding new translations of the guix and
> +guix-packages components, other than they need at least one translated

@code{guix} and @code{guix-packages}

> +Given that the website is dedicated to new users, we want its translation

“web site” (two words).

> +target.  Everytime we synchronise translations, developpers need to

“developers” and (if you feel overseas-inclined) “synchronize”.

> +When a language is included, it may be removed in the future, if it stays

Remove comma.

>>From 5cbb70ebcbf141cd05fa60bf0bfa806125a56381 Mon Sep 17 00:00:00 2001
> Message-Id: <5cbb70ebcbf141cd05fa60bf0bfa806125a56381.1646763369.git.julien@lepiller.eu>
> In-Reply-To: <37071410629a7d70c9b4e4f673f2c625d3ed4b47.1646763369.git.julien@lepiller.eu>
> References: <37071410629a7d70c9b4e4f673f2c625d3ed4b47.1646763369.git.julien@lepiller.eu>
> From: Julien Lepiller <julien@lepiller.eu>
> Date: Tue, 8 Mar 2022 19:11:38 +0100
> Subject: [PATCH 2/3] maint: Implement translation thresholds.
>
> Do not download new translations for the cookbook and the manual when
> they are below 10% completion, and remove existing translations when
> they fall below 5%.
>
> * Makefile.am (download-po): Implement translation thresholds.
> ---
>  Makefile.am | 20 +++++++++++++++++---
>  1 file changed, 17 insertions(+), 3 deletions(-)
>
> diff --git a/Makefile.am b/Makefile.am
> index 8850c4562c..164804d96a 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -1066,21 +1066,35 @@ WEBLATE_REPO = https://framagit.org/tyreunom/guix-translations
>  # form.
>  download-po:
>  	dir=$$(mktemp -d); \
> -	git clone --depth 1 "$(WEBLATE_REPO)" "$$dir/translations"; \
> +	git clone --depth 1 "$(WEBLATE_REPO)" "$$dir/translations" && \
>  	for domain in po/doc po/guix po/packages; do \
>  	    for po in "$$dir/translations/$$domain"/*.po; do \
>  	        translated=$$(LANG=en_US.UTF-8 msgfmt --statistics "$$po" 2>&1 | cut -f1 -d' '); \
> +	        untranslated=$$(LANG=en_US.UTF-8 msgfmt --statistics "$$po" 2>&1 | cut -f4 -d' '); \
> +			untranslated=$${untranslated:-0}; \
> +			total=$$(($$translated+$$untranslated)); \
>  	        target=$$(basename "$$po"); \
>  	        target="$$domain/$$target"; \
> -	        if msgfmt -c "$$po" && [ "$$translated" != "0" ]; then \
> +	        msgfmt -c "$$po"; \
> +	        if msgfmt -c "$$po" && [ "$$translated" != "0" ] && ([ "$$domain" != "po/doc" ] || [ "$$translated" -gt $$(($$total/10)) ] || [ -f $$target ]); then \
>  	            msgfilter --no-wrap -i "$$po" cat > "$$po".tmp; \
>  	            mv "$$po".tmp "$$target"; \
>  	            echo "copied $$target."; \
>  	        else \
> -	            echo "WARN: $$target ($$translated translated messages) was not added/updated."; \
> +	            echo "WARN: $$target ($$translated translated messages ($$((translated/total*100))%)) was not added/updated."; \
>  	        fi; \
>  	    done; \
>  	done; \
> +	for po in po/doc/*.po; do \
> +	    translated=$$(LANG=en_US.UTF-8 msgfmt --statistics "$$po" 2>&1 | cut -f1 -d' '); \
> +	    untranslated=$$(LANG=en_US.UTF-8 msgfmt --statistics "$$po" 2>&1 | cut -f4 -d' '); \
> +		untranslated=$${untranslated:-0}; \
> +		total=$$(($$translated + $$untranslated)); \
> +		if [ "$$translated" -lt "$$(($$total/20))" ]; then \
> +		    echo "WARN: $$po was removed because it is below the 5% threshold: $$((translated/total*100))%"; \
> +			rm $$po; \
> +		fi; \
> +	done; \

LGTM, but this is getting a bit hairy.  :-)

No concrete suggestions, but it would be great if we could somehow split
it and/or move it to a script in build-aux/ (that’d avoid double dollar
signs) and/or write it in Scheme.  Future work…

> From 726ef94f91d5dab25c3ccfb2986dcba6d39a4ab8 Mon Sep 17 00:00:00 2001
> Message-Id: <726ef94f91d5dab25c3ccfb2986dcba6d39a4ab8.1646763369.git.julien@lepiller.eu>
> In-Reply-To: <37071410629a7d70c9b4e4f673f2c625d3ed4b47.1646763369.git.julien@lepiller.eu>
> References: <37071410629a7d70c9b4e4f673f2c625d3ed4b47.1646763369.git.julien@lepiller.eu>
> From: Julien Lepiller <julien@lepiller.eu>
> Date: Tue, 8 Mar 2022 19:14:47 +0100
> Subject: [PATCH 3/3] nls: Enforce translation thresholds.
>
> * po/doc/guix-cookbook.es.po: Remove file.
> * po/doc/guix-cookbook.fa.po: Remove file.
> * po/doc/guix-cookbook.fi.po: Remove file.
> * po/doc/guix-cookbook.uk.po: Remove file.
> * po/doc/local.mk: Remove them.
> * doc/local.mk: Remove them.

[...]

>  # If adding a language, update the following variables, and info_TEXINFOS.
>  MANUAL_LANGUAGES = de es fa fi fr it ko pt_BR ru sk zh_CN
> -COOKBOOK_LANGUAGES = de es fa fi fr ko pt_BR ru sk uk zh_Hans
> +COOKBOOK_LANGUAGES = de fr ko pt_BR ru sk zh_Hans

Should we also remove fa, fi, it, ko, and sk from MANUAL_LANGUAGES and
info_TEXINFOS?

  https://translate.fedoraproject.org/projects/guix/documentation-manual/

Thank you!

Ludo’.