From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: ng0 Newsgroups: gmane.lisp.guile.user,gmane.lisp.guile.devel Subject: Re: Website translations with Haunt Date: Sat, 9 Dec 2017 18:15:29 +0000 Message-ID: <20171209181529.xzpfpjl4y4ugrqhv@abyayala> References: <20171209180619.GA10254@floriannotebook.localdomain> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="hhsycmrid3dzub3x" X-Trace: blaine.gmane.org 1512843381 14247 195.159.176.226 (9 Dec 2017 18:16:21 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 9 Dec 2017 18:16:21 +0000 (UTC) Cc: Guile User , guile-devel To: "pelzflorian (Florian Pelz)" Original-X-From: guile-user-bounces+guile-user=m.gmane.org@gnu.org Sat Dec 09 19:16:08 2017 Return-path: Envelope-to: guile-user@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eNjfU-0003OO-0O for guile-user@m.gmane.org; Sat, 09 Dec 2017 19:16:08 +0100 Original-Received: from localhost ([::1]:42190 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eNjfb-0001fI-7R for guile-user@m.gmane.org; Sat, 09 Dec 2017 13:16:15 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:51175) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eNjfB-0001ex-GD for guile-user@gnu.org; Sat, 09 Dec 2017 13:15:51 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eNjf8-00061o-4z for guile-user@gnu.org; Sat, 09 Dec 2017 13:15:49 -0500 Original-Received: from aibo.runbox.com ([91.220.196.211]:49280) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eNjf7-00061E-P4; Sat, 09 Dec 2017 13:15:46 -0500 Original-Received: from [10.9.9.211] (helo=mailfront11.runbox.com) by mailtransmit03.runbox with esmtp (Exim 4.86_2) (envelope-from ) id 1eNjf6-00009p-CI; Sat, 09 Dec 2017 19:15:44 +0100 Original-Received: from dslb-092-073-183-021.092.073.pools.vodafone-ip.de ([92.73.183.21] helo=localhost) by mailfront11.runbox.com with esmtpsa (uid:892961 ) (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) id 1eNjes-0003Ju-Ax; Sat, 09 Dec 2017 19:15:30 +0100 Mail-Followup-To: "pelzflorian (Florian Pelz)" , Guile User , guile-devel Content-Disposition: inline In-Reply-To: <20171209180619.GA10254@floriannotebook.localdomain> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 91.220.196.211 X-BeenThere: guile-user@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: General Guile related discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-user-bounces+guile-user=m.gmane.org@gnu.org Original-Sender: "guile-user" Xref: news.gmane.org gmane.lisp.guile.user:14330 gmane.lisp.guile.devel:19401 Archived-At: --hhsycmrid3dzub3x Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hey Florian, interesting work. I opened a bug a very long time ago for Guix, and the content applied to Guile (as well as to the project I initiated (infotropique) and very likely many other Haunt using websites like Haunt itself, 8sync, etc). I've seen the first versions of this a while back in the code for your website, I'll check out the changes soon, exciting work! My idea was a reimplementation of prep (a text format and html document generator that is capable to include multiple languages in its source files, written by lynX back in the early 90s (or earlier) iirc (and still used today). pelzflorian (Florian Pelz) transcribed 7.6K bytes: > Hello, >=20 > First of all, I want to say thank you to the Guile, Haunt, ffi-helper > and related projects=E2=80=99 developers. >=20 > I built my personal website [1] using David Thompson=E2=80=99s Haunt [2] = and > recently talked my university=E2=80=99s Islamic Students=E2=80=99 Associa= tion > (Islamische Hochschulvereinigung) into using Haunt for their > not-yet-finished website as well, because I think the concept of Haunt > and SHTML is superior to alternatives. However in order to make the > website multilingual (the user can choose to view it in German or > English) so far I used an association list with assoc-ref which is not > very comfortable since all strings have to be added in two places, > i.e. in the SHTML code and in the association list where the code > looks for translations. >=20 > I want to ask for your thoughts on my new solution since translations > are probably important to many Haunt users. In particular, I believe > there was some discussion on Website translation on the Guile or Guix > lists as well. >=20 > I did not want to use the ordinary gettext functions in order to not > call setlocale very often to switch languages. It seems the Gettext > system is not designed for rapidly changing locales, but maybe I am > wrong about this and very many setlocale calls would not be that bad. >=20 > Using Matt Wette=E2=80=99s ffi-helper [3] and the libgettextpo from GNU > Gettext, [4] I now wrote code to convert a po file into an association > list where I can now look up translations. (Note that I had to make > some patches to the c99dev branch of nyacc before building nyacc and > ffi-helper because the Makefile had spaces instead of tabs at one > place and it did not find guile and guild. Maybe these patches were > just necessary because of a broken setup on my part though.) >=20 > I used this dot.ffi file to create libgettextpo bindings: >=20 > (define-ffi-module (gettext-po) > #:include '("gettext-po.h") > #:library '("libgettextpo")) >=20 > Then I wrote a procedure to convert a po file to an association list > to look up the msgstr for a msgid. Note that by lingua I basically > mean a locale. >=20 > (use-modules > (gettext-po) > (system ffi-help-rt) > ((system foreign) > #:prefix ffi:) > [=E2=80=A6]) >=20 > (define xerror-handler-struct > (make-struct-po_xerror_handler)) ; TODO SET HANDLERS: > ;; [=E2=80=A6] >=20 > (define (translations-for-lingua lingua) > "Returns po/.po converted to an association list of msgid=E2=80= =93msgstr pairs." > ;; TODO: STILL DISREGARDING PLURALS AND OTHER INFORMATION > (let* ((po-file > (po_file_read_v3 > (string-append "po/" lingua ".po") > (pointer-to xerror-handler-struct))) > (translations > (if (ffi:null-pointer? (unwrap~pointer po-file)) > '() > ;; otherwise: > (let ((iter (po_message_iterator po-file ffi:%null-pointer)= )) > (let loop ((message (po_next_message iter))) > (if (ffi:null-pointer? (unwrap~pointer message)) > (begin > (po_message_iterator_free iter) > '()) > ;; otherwise: > (cons > (cons > (ffi:pointer->string (po_message_msgid message)) > (ffi:pointer->string (po_message_msgstr message))) > (loop (po_next_message iter))))))))) > (if (not (ffi:null-pointer? (unwrap~pointer po-file))) > (po_file_free po-file)) > translations)) >=20 >=20 >=20 > I did this for every locale and made a second association list mapping > the locales to the msgid=E2=80=93msgstr association lists. Then I wrote > translated-msg to do the lookup. >=20 >=20 > (define (translations-entry-for-lingua lingua) > "Returns a pair of LINGUA and an association list of its translations." > (cons > lingua > (translations-for-lingua lingua))) >=20 > (define translated-msg > ;; gettext is not used directly because it would require repeated > ;; setlocale calls, which should not be necessary. > ;; See: https://stackoverflow.com/questions/3398113/php-gettext-problems > (let ((translation-lists > (map translations-entry-for-lingua linguas))) > (define (with-default value default) > (if value value > default)) > (lambda (msgid lingua) > "Returns the msgstr for MSGID from the po file for LINGUA." > (let ((translations (assoc-ref translation-lists lingua))) > (with-default > (assoc-ref translations msgid) > msgid))))) >=20 >=20 > As a Gettext-like shorthand I wrote a macro called _ which calls the > above translated-msg function. It takes the locale from the > current-lingua variable, so the macro deliberately breaks hygiene. >=20 > (define-syntax _ > (lambda (x) > "Gettext-like shorthand for translated-msg with what currently is cur= rent-lingua." > (syntax-case x () > ((_ msg) > (with-syntax ((current-lingua (datum->syntax x 'current-lingua))) > #'(translated-msg msg current-lingua)))))) >=20 >=20 > I use it like in this excerpt: >=20 > (define (back-button-for-lingua lingua) > "SXML for a link back to the home page." > (let ((current-lingua lingua)) > `(a (@ (href ,(string-append "/index" "-" lingua ".html")) > (class "full-width-link")) > ,(_ "=E2=86=90 Back to home page")))) >=20 >=20 > Then I ran the xgettext program from the terminal to create a pot file > from all strings marked with _. >=20 > xgettext -f po/POTFILES -o po/pelzfloriande-website.pot --from-code=3DUTF= -8 --copyright-holder=3D"" --package-name=3D"pelzfloriande-website" --msgid= -bugs-address=3D"pelzflorian@pelzflorian.de" --keyword=3D_ >=20 > Xgettext autodetected all the strings that were marked for > translation. This is much better than my previous approach where I > had to list all of them manually in a manually written association > list. >=20 > To create a po file from a pot file, I do the usual: >=20 > cd po > msginit -l de --no-translator > msginit -l en --no-translator >=20 >=20 > Then I filled out the po files using gtranslator. I can now run > =E2=80=9Chaunt build=E2=80=9D with an appropriate GUILE_LOAD_PATH, for me= currently: >=20 > GUILE_LOAD_PATH=3D$HOME/keep/projects/pelzfloriande-website:$HOME/build/n= yacc/src/nyacc/examples:$GUILE_LOAD_PATH GUILE_LOAD_COMPILED_PATH=3D$GUILE_= LOAD_COMPILED_PATH:$HOME/.cache/guile/ccache/2.2-LE-8-3.A/home/florian/keep= /projects/pelzfloriande-website haunt build >=20 >=20 >=20 > Is this the right approach? >=20 > You can find my unfinished, not very clean code at [5]. >=20 > Regards, > Florian >=20 >=20 > [1] https://pelzflorian.de > [2] https://haunt.dthompson.us/ > [3] https://savannah.nongnu.org/projects/nyacc/ > [4] https://www.gnu.org/software/gettext/manual/html_node/libgettextpo.ht= ml > [5] https://pelzflorian.de/git/pelzfloriande-website/commit/?id=3D5f97bf1= 57eaddcfe722c97dcab349b7dcfbbcd9d --=20 GnuPG: A88C8ADD129828D7EAC02E52E22F9BBFEE348588 GnuPG: https://c.n0.is/ng0_pubkeys/tree/keys WWW: https://n0.is --hhsycmrid3dzub3x Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEqIyK3RKYKNfqwC5S4i+bv+40hYgFAlosKEEACgkQ4i+bv+40 hYhNWw/+NOEIs79wPcl4rtmfmmuqRSJoF0/lkzC5hCwgDimH23QcRH+VHb6MZQ1V a1495hp4OSK1tc4OsTgdgopp8fppgNY9+CesLL4hQlIDjPrQjYWhoKDhv9KuhAuD 3zZuJsthEDiqpwTwwmd4Pjs9u7c/FjLEBJDaGKKaGeKW6+yMGAbZtNWE5o4xpEMD LwhvNROcXmByOth1XzE3xeMc/HS9rBx5MF6o8UBBnV82gfq8E6+zAqARrJvxMZ6A muYJshDlrsuOrg7LgXd3L6UmL4qSd7v2Xpu4NUctun0kO1CmoFiBQqbGZFlioJay +GG/sKJYwfdBqnqQ7A9Vsc6SBHCYjPXPbqR3XmqLrL4QuGK4h64jY5HotVT0gP97 LZ5PR/lnB7kGHGKcZEUX/r+xX3/ZRXpBOaEXBMbg/xFSXHJ24adgrCWlDnKPR9Xn Qh5ZKncG0g+69uL+Ds0gea2ph/Ny2A73wzH0l+f/RxVQgIuXaK6czXQtJZSjdSHX SgjBWkBnYdjG5mbi4eVNrAhpSXBjugVny8Alr+a2dsMSrI/YXfwIirsFhres/odt BRheja4t7FfoWPWEIzvVGUmoP47OaUVo8VUwXJCK12BMSZWdNshoX0Tz345gqYRa ywS0qHM2/eLLKk0+S1TKrXy0GYAAAAFFBw/9nNr3PasgLHwyLE0= =EMzi -----END PGP SIGNATURE----- --hhsycmrid3dzub3x--