From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id oBWXOhC32WKcTAEAbAwnHQ (envelope-from ) for ; Thu, 21 Jul 2022 22:29:05 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id UDqaOhC32WILQgEA9RJhRA (envelope-from ) for ; Thu, 21 Jul 2022 22:29:04 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id A0DB615129 for ; Thu, 21 Jul 2022 22:29:04 +0200 (CEST) Received: from localhost ([::1]:42586 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oEcn9-0003ED-Oe for larch@yhetil.org; Thu, 21 Jul 2022 16:29:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45186) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oEcmy-0003E4-Ik for guix-devel@gnu.org; Thu, 21 Jul 2022 16:28:52 -0400 Received: from lepiller.eu ([2a00:5884:8208::1]:34048) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oEcmu-000314-RO for guix-devel@gnu.org; Thu, 21 Jul 2022 16:28:52 -0400 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id f7c75d9c for ; Thu, 21 Jul 2022 20:27:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date:from :to:subject:message-id:mime-version:content-type; s=dkim; bh=uzw oXdquSb4LjNsSs0JJgv9GKe5Sxvyz7yG7PNs/RLk=; b=ALWWw/d0qAU67bK5gFY MXNgCQhOFOO1RFJQppdaEKbPTCELsRpjub0wXXrGIzOgzx4xmKaHKH3W0Xoa1EaI x6dM4F+nKfp7ntqTL0Bfi4DPk3SybFg0MbsiqDozZr6eFXRXrm4Wl+xHrq9w5fvq DYKh1msdXFLnbDuso68g//qyk7PxlSRVBPkjSKgsmuPwrUnYutjNe0APlAPyZlPx 2RSqHwyrGRlHOfuTS5IRRLMtrclMBxXxG4DEGlU9NMQxJ/iYoGAz2lKgJWMM/lwm dXOe92qQLpQrJQd7YDMyCgLEwTi1jIFkmdWudVJrgccNOUD/7gUILcs/Oe25SuAr smw== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id b89ca0bc (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO) for ; Thu, 21 Jul 2022 20:27:40 +0000 (UTC) Date: Thu, 21 Jul 2022 22:27:34 +0200 From: Julien Lepiller To: guix-devel@gnu.org Subject: Translating news on weblate? Message-ID: <20220721222734.0f0ac881@sybil.lepiller.eu> X-Mailer: Claws Mail 4.1.0 (GTK 3.24.30; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="MP_/zeaOd/Tdehz7sv4JfQa993w" Received-SPF: pass client-ip=2a00:5884:8208::1; envelope-from=julien@lepiller.eu; helo=lepiller.eu X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN X-Migadu-To: larch@yhetil.org X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1658435344; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=odAvIIlCEdyNd0sJFca0TwYRuI3KfCXbD0w6LwSbZYo=; b=mYVQn3vk7pKmhRPtoiVQbvuVOdgd5qNzuGh1lQSKEy4na7tgmsQlKk2cUamwLtz9z8/az8 J80prC9AjKKIL09xvBfrVzo3+MqrMPPh6+HiCC6P03pen/5nOIrYXBVaCdUx8QyEsHvmIw DQsHsJm2/2LpY6fm4IP+glsCppBh5fovfikFVyFq1gtr38wAQW741hGKsKQDjC8ZVDMGrc U7xNWweBVUgtGHF1MJYwa4/HVFphOgxlIwlEru3AlokBlaUTrJxJiZ7J26YE6Z97bn+reM Yvtzj60pMAK1SC6x2jSEPOdy4ubrG1lZdZdDgwWOaqUXysLPv1jqsGITZ5kCjg== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1658435344; a=rsa-sha256; cv=none; b=GeOY6/zoxGMjSnBjbHfPlCjOdDtG80CJapGz3xN6/3174Dz2ahZYT9bGTUmL6mKlT5qgL7 dKwlLhTlyK8+NayMVbTNR6KAvXoSGWa5mdCfxqmIXL/2YzNC2EBkqoiyXVA/AVk1TyeG/o eutUC/ENJmY7gQdnscpd7HZHSgJXyBG3ZUmsoiQIfV93J3F+F1Vh05B1xi3n3u1Qyqw3EX /PAg/JIzJjkxGBWXdRymGe7Dwl/fB3iFPVcR4xPsERAgQ4IJR2fsJIypLWqo/BOHf22V8P pokPsS1cZdvB3J+uq6T64L4DgQ8gGbIfekW7AamC7fCZH74kIjbcr7UpB5B7ow== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=lepiller.eu header.s=dkim header.b="ALWWw/d0"; dmarc=pass (policy=none) header.from=lepiller.eu; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -4.44 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=lepiller.eu header.s=dkim header.b="ALWWw/d0"; dmarc=pass (policy=none) header.from=lepiller.eu; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: A0DB615129 X-Spam-Score: -4.44 X-Migadu-Scanner: scn0.migadu.com X-TUID: nKorLUbTzhZy --MP_/zeaOd/Tdehz7sv4JfQa993w Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Content-Disposition: inline Hi Guix! I was thinking we could have our news file translated at weblate, which would help having more people translate it. Attached is a script that is able to generate a pot file from the news.scm file and a translated news.scm file from the existing news.scm and a directory that contains the po files and a LINGUAS file. The goal is to regularly fetch the news.scm file, generate a new pot that would automatically be updated on weblate. Translators can do their work, and the result is regularly merged back (after being checked that it doesn't break) in our repo. The script tries to reformat the news in a standard way and to merge existing copyright lines with the authorship lines in the po files. It is also able to generate the initial po files that I would give weblate, so translators don't have to redo translations that have already been done. I also attached the LINGUAS file with all languages for which we have at least one entry in news.scm. WDYT? --MP_/zeaOd/Tdehz7sv4JfQa993w Content-Type: application/octet-stream; name=LINGUAS Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=LINGUAS ZGUKZXMKZnIKbmwKcnUKemgK --MP_/zeaOd/Tdehz7sv4JfQa993w Content-Type: text/x-scheme Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename=news-po.scm #!/gnu/store/5rbr7gi8q7gpmb4gmqrpnpk55a4gjpkz-profile/bin/guile \ --no-auto-compile -s !# ;;; GNU Guix --- Functional package management for GNU ;;; Copyright =C2=A9 2022 Julien Lepiller ;;; ;;; This file is part of GNU Guix. ;;; ;;; GNU Guix is free software; you can redistribute it and/or modify it ;;; under the terms of the GNU General Public License as published by ;;; the Free Software Foundation; either version 3 of the License, or (at ;;; your option) any later version. ;;; ;;; GNU Guix is distributed in the hope that it will be useful, but ;;; WITHOUT ANY WARRANTY; without even the implied warranty of ;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;;; GNU General Public License for more details. ;;; ;;; You should have received a copy of the GNU General Public License ;;; along with GNU Guix. If not, see . ;;; Commentary: ;; This code is a helper for news file translation using Gettext. ;;; Code: (use-modules (srfi srfi-1) (srfi srfi-9) (srfi srfi-19) (ice-9 match) (ice-9 pretty-print) (ice-9 rdelim) (ice-9 regex) (guix build po) (guix channels)) (define read-channel-news (@@ (guix channels) read-channel-news)) (define channel-news-entries (@@ (guix channels) channel-news-entries)) (define (@@ (guix channels) )) (define channel-news-entry (@@ (guix channels) channel-news-entry)) (define (print-msg msg) (if msg (let* ((msg (string-join (string-split msg #\\) "\\\\")) (msg (string-join (string-split msg #\") "\\\"")) (msg (string-join (string-split msg #\newline) "\\n\"\n\""))) (if (string-contains msg "\n") (string-append "\"\n\"" msg) msg)) "")) (define* (generate-po news-file pot-file #:optional lang) "Generate a pot file from a news file." (let ((news (call-with-input-file news-file read-channel-news))) (call-with-output-file pot-file (lambda (port) (format port "# SOME DESCRIPTIVE TITLE~%") (format port "# Copyright (C) YEAR the authors of Guix (msgids) and= the following authors (msgstr)~%") (format port "# This file is distributed under the same license as = the guix news file.~%") (format port "# ~%") (format port "msgid \"\"~%") (format port "msgstr \"\" \"Project-Id-Version: guix news checkout\\n\" \"Report-Msgid-Bugs-To: bug-guix@gnu.org\\n\" \"POT-Creation-Date: ~a\\n\" \"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\\n\" \"Last-Translator: FULL NAME \\n\" \"Language-Team: LANGUAGE \\n\" \"Language: \\n\" \"MIME-Version: 1.0\\n\" \"Content-Type: text/plain; charset=3DUTF-8\\n\" \"Content-Transfer-Encoding: 8bit\\n\"~%~%" (date->string (current-date))) (for-each (match-lambda (($ commit tag title body) (let ((msgid1 (assoc-ref title "en")) (msgid2 (assoc-ref body "en")) (msgstr1 (assoc-ref title lang)) (msgstr2 (assoc-ref body lang))) (when commit (format port "#. commit: ~a~%" commit)) (when tag (format port "#. tag: ~a~%" tag)) (format port "msgid \"~a\"~%" (print-msg msgid1)) (format port "msgstr \"~a\"~%~%" (print-msg msgstr1)) (when commit (format port "#. commit: ~a~%" commit)) (when tag (format port "#. tag: ~a~%" tag)) (format port "msgid \"~a\"~%" (print-msg msgid2)) (format port "msgstr \"~a\"~%~%" (print-msg msgstr2))))) (channel-news-entries news)))))) (define (languages lingua) (with-input-from-file lingua (lambda _ (let loop ((line (read-line))) (if (eof-object? line) '() ;; else read linguas before comment (let ((before-comment (car (string-split line #\#)))) (append (map match:substring (list-matches "[^ \t]+" before-comment)) (loop (read-line))))))))) (define (clean-string entry) (string-join (string-split entry #\newline) "\n")) (define (complete-msg entries pos) "Complete an entry's message list using po files that were parsed in pos." (let ((entry (assoc-ref entries "en"))) (let loop ((pos pos) (entries entries)) (match pos (() entries) (((lang . po) pos ...) (if (assoc-ref po entry) (loop pos (cons (cons lang (assoc-ref po entry)) entries)) (loop pos entries))))))) (define (merge-entries news pos languages) (let ((entries (channel-news-entries news))) (map (lambda (entry) (let* ((title (list (cons "en" (clean-string (assoc-ref (channel-news-entry-title en= try) "en"))))) (title (complete-msg title pos)) (body (list (cons "en" (clean-string (assoc-ref (channel-news-entry-body entr= y) "en"))))) (body (complete-msg body pos))) (channel-news-entry (channel-news-entry-commit entry) (channel-news-entry-tag entry) title body))) entries))) (define-record-type (make-copyright name years) copyright? (name copyright-name) (years copyright-years)) (define (parse-years years) (let ((years (map string-trim (string-split years #\,)))) (let loop ((years years) (res '())) (match years (() (sort res <)) ((year years ...) (match (string-split year #\-) ((year) (loop years (cons (string->number year) res))) ((start end) (let ((start (string->number start)) (end (string->number end))) (loop years (append (iota (- end start -1) start) res))))))))= )) (define (parse-scm-copyrights file) (with-input-from-file file (lambda _ (let loop ((line (read-line))) (if (eof-object? line) '() (let ((matches (string-match "^;; Copyright =C2=A9 ([0-9, -]*) (.= *)$" line))) (if matches (cons (make-copyright (match:substring matches 2) (parse-years (match:substring matches 1))) (loop (read-line))) (loop (read-line))))))))) (define (parse-po-copyrights file) (with-input-from-file file (lambda _ (let loop ((line (read-line))) (if (eof-object? line) '() (let ((matches (string-match "^# (.* <.*@.*>), ([0-9, -]*)\\.$" l= ine))) (if matches (cons (make-copyright (match:substring matches 1) (parse-years (match:substring matches 2))) (loop (read-line))) (loop (read-line))))))))) (define (merge-copyrights copyrights) (define (uniq lst) (match lst (() '()) ((e lst ...) (let ((rest (uniq lst))) (if (member e rest) rest (cons e rest)))))) (define (merge-copyright-of name copyrights) (make-copyright name (uniq (sort (apply append (map copyright-years cop= yrights)) <)))) (fold (lambda (copyright res) (if (find (lambda (r) (equal? (copyright-name r) (copyright-name copy= right))) res) (sort res (lambda (c1 c2) (string<=3D? (copyright-name c1) (copyr= ight-name c2)))) (cons (merge-copyright-of (copyright-name copyright) (filter (lambda (c) (equal? (copyright-name c) (copyright-name copyright))) copyrights)) res))) '() copyrights)) (define (parse-copyrights news-file po-dir languages) (let ((scm-copyrights (parse-scm-copyrights news-file)) (po-copyrights (apply append (map (lambda (lang) (parse-po-copyrights (string-append po-dir "/" lang ".= po"))) languages)))) (merge-copyrights (append scm-copyrights po-copyrights)))) (define (print-copyrights copyrights) (define (format-years years) (let loop ((years years) (first-consecutive #f) (last-consecutive #f) (= res '())) (match years ((year years ...) (cond ((and first-consecutive (equal? (+ last-consecutive 1) year)) (loop years first-consecutive year res)) ((and first-consecutive=20 (equal? first-consecutive last-consecutive)) (loop years year year (cons (number->string first-consecutive) = res))) ((and first-consecutive=20 (equal? (+ first-consecutive 1) last-consecutive)) (loop years year year (cons* (number->string last-consecutive) = (number->string first-consecutive) res))) (first-consecutive (loop years year year (cons (string-append number->string first= -consecutive) "-" (number->string last-consecutive)))) (else (loop years year year res)))) (() (cond ((and first-consecutive (equal? first-consecutive last-consecutive)) (cons (number->string first-consecutive) res)) ((and first-consecutive (equal? (+ first-consecutive 1) last-consecutive)) (cons* (number->string last-consecutive) (number->string first-consecutive) res)) (first-consecutive (cons (string-append (number->string first-consecutive) "-" (number->string last-consecutive)) res))))))) (for-each (match-lambda (($ name years) (format #t ";; Copyright =C2=A9 ~a ~a~%" (string-join (reverse (form= at-years years)) ", ") name))) copyrights)) (define (print-entry entry) (define (print-content content) (for-each (match-lambda ((lang . content) (let* ((content (string-join (string-split content #\\) "\\\\")) (content (string-join (string-split content #\") "\\\""))) (format #t "~% (~a \"~a\")" lang content)))) content)) (define (content<=3D? c1 c2) (or (equal? (car c1) "en") (and (not (equal? (car c2) "en")) (string<=3D? (car c1) (car c2))))) (format #t "~%~% (entry ") (when (channel-news-entry-commit entry) (format #t "(commit \"~a\")~%" (channel-news-entry-commit entry))) (when (channel-news-entry-tag entry) (format #t " (tag \"~a\")~%" (channel-news-entry-tag entry))) (format #t " (title") (print-content (sort (channel-news-entry-title entry) content<=3D?)) (display ")") (newline) (format #t " (body") (print-content (sort (channel-news-entry-body entry) content<=3D?)) (display "))")) (define (merge-news news-file po-dir) (let* ((lingua (string-append po-dir "/LINGUAS")) (languages (languages lingua)) (pos (map (lambda (lang) (cons lang (call-with-input-file (string-append po-dir "/" lang ".po") read-po-file))) languages)) (news (call-with-input-file news-file read-channel-news)) (entries (merge-entries news pos languages)) (copyrights (parse-copyrights news-file po-dir languages))) (with-output-to-file (string-append news-file ".tmp") (lambda _ (display ";; GNU Guix news, for use by 'guix pull'. ;; ") (print-copyrights copyrights) (display ";; ;; Copying and distribution of this file, with or without modification, are ;; permitted in any medium without royalty provided the copyright notice and ;; this notice are preserved. (channel-news (version 0)") (for-each print-entry entries) (display ")") (newline))))) (define (main . args) (match args (("pot" news-file pot-file) (generate-po news-file pot-file)) (("po" news-file lang po-file) (generate-po news-file po-file lang)) (("merge" news-file po-dir) (merge-news news-file po-dir)) (_ (format (current-error-port) "Usage: etc/teams.scm []~%")))) (apply main (cdr (command-line))) --MP_/zeaOd/Tdehz7sv4JfQa993w--