From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: "pelzflorian (Florian Pelz)" Newsgroups: gmane.lisp.guile.devel,gmane.lisp.guile.user Subject: Website translations with Haunt Date: Sat, 9 Dec 2017 19:06:19 +0100 Message-ID: <20171209180619.GA10254@floriannotebook.localdomain> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="AhhlLboLdkugWU4S" X-Trace: blaine.gmane.org 1512842803 13028 195.159.176.226 (9 Dec 2017 18:06:43 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 9 Dec 2017 18:06:43 +0000 (UTC) User-Agent: Mutt/1.9.1 (2017-09-22) To: Guile User , guile-devel Original-X-From: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Sat Dec 09 19:06:36 2017 Return-path: Envelope-to: guile-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eNjWF-00031Z-DV for guile-devel@m.gmane.org; Sat, 09 Dec 2017 19:06:35 +0100 Original-Received: from localhost ([::1]:42167 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eNjWJ-0005r5-9A for guile-devel@m.gmane.org; Sat, 09 Dec 2017 13:06:39 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:49930) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eNjW6-0005qu-7T for guile-devel@gnu.org; Sat, 09 Dec 2017 13:06:27 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eNjW3-0000TI-0c for guile-devel@gnu.org; Sat, 09 Dec 2017 13:06:26 -0500 Original-Received: from pelzflorian.de ([5.45.111.108]:49404 helo=mail.pelzflorian.de) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1eNjW2-0000SS-IA; Sat, 09 Dec 2017 13:06:22 -0500 Original-Received: from floriannotebook.localdomain (ip5b431f77.dynamic.kabel-deutschland.de [91.67.31.119]) by mail.pelzflorian.de (Postfix) with ESMTPSA id 99715360007; Sat, 9 Dec 2017 19:06:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=pelzflorian.de; s=mail; t=1512842780; bh=28bcLSz8fnIf2C7y84QzizKsbMt/IYUtR3lGdolvsG0=; h=Date:From:To:Subject; b=JtPpAWMmUv8g//5uLarrrsSncDt15hMzlQOvMnqy5CwKPFvCh8iYyiB3DKXU1pRrx Jt60Q1YuY/G9i9bADO7IokFParoyEJ0EvX+H5BWQ5kJ7P8ItVYOoEdLNXlwbh4av0/ T9oetPuw/7RA4ah/MHwzIvnElo7RybhR0m/E5XHQ= Content-Disposition: inline X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 5.45.111.108 X-BeenThere: guile-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Developers list for Guile, the GNU extensibility library" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guile-devel-bounces+guile-devel=m.gmane.org@gnu.org Original-Sender: "guile-devel" Xref: news.gmane.org gmane.lisp.guile.devel:19400 gmane.lisp.guile.user:14329 Archived-At: --AhhlLboLdkugWU4S Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, First of all, I want to say thank you to the Guile, Haunt, ffi-helper and related projects=E2=80=99 developers. I built my personal website [1] using David Thompson=E2=80=99s Haunt [2] and recently talked my university=E2=80=99s Islamic Students=E2=80=99 Associati= on (Islamische Hochschulvereinigung) into using Haunt for their not-yet-finished website as well, because I think the concept of Haunt and SHTML is superior to alternatives. However in order to make the website multilingual (the user can choose to view it in German or English) so far I used an association list with assoc-ref which is not very comfortable since all strings have to be added in two places, i.e. in the SHTML code and in the association list where the code looks for translations. I want to ask for your thoughts on my new solution since translations are probably important to many Haunt users. In particular, I believe there was some discussion on Website translation on the Guile or Guix lists as well. I did not want to use the ordinary gettext functions in order to not call setlocale very often to switch languages. It seems the Gettext system is not designed for rapidly changing locales, but maybe I am wrong about this and very many setlocale calls would not be that bad. Using Matt Wette=E2=80=99s ffi-helper [3] and the libgettextpo from GNU Gettext, [4] I now wrote code to convert a po file into an association list where I can now look up translations. (Note that I had to make some patches to the c99dev branch of nyacc before building nyacc and ffi-helper because the Makefile had spaces instead of tabs at one place and it did not find guile and guild. Maybe these patches were just necessary because of a broken setup on my part though.) I used this dot.ffi file to create libgettextpo bindings: (define-ffi-module (gettext-po) #:include '("gettext-po.h") #:library '("libgettextpo")) Then I wrote a procedure to convert a po file to an association list to look up the msgstr for a msgid. Note that by lingua I basically mean a locale. (use-modules (gettext-po) (system ffi-help-rt) ((system foreign) #:prefix ffi:) [=E2=80=A6]) (define xerror-handler-struct (make-struct-po_xerror_handler)) ; TODO SET HANDLERS: ;; [=E2=80=A6] (define (translations-for-lingua lingua) "Returns po/.po converted to an association list of msgid=E2=80= =93msgstr pairs." ;; TODO: STILL DISREGARDING PLURALS AND OTHER INFORMATION (let* ((po-file (po_file_read_v3 (string-append "po/" lingua ".po") (pointer-to xerror-handler-struct))) (translations (if (ffi:null-pointer? (unwrap~pointer po-file)) '() ;; otherwise: (let ((iter (po_message_iterator po-file ffi:%null-pointer))) (let loop ((message (po_next_message iter))) (if (ffi:null-pointer? (unwrap~pointer message)) (begin (po_message_iterator_free iter) '()) ;; otherwise: (cons (cons (ffi:pointer->string (po_message_msgid message)) (ffi:pointer->string (po_message_msgstr message))) (loop (po_next_message iter))))))))) (if (not (ffi:null-pointer? (unwrap~pointer po-file))) (po_file_free po-file)) translations)) I did this for every locale and made a second association list mapping the locales to the msgid=E2=80=93msgstr association lists. Then I wrote translated-msg to do the lookup. (define (translations-entry-for-lingua lingua) "Returns a pair of LINGUA and an association list of its translations." (cons lingua (translations-for-lingua lingua))) (define translated-msg ;; gettext is not used directly because it would require repeated ;; setlocale calls, which should not be necessary. ;; See: https://stackoverflow.com/questions/3398113/php-gettext-problems (let ((translation-lists (map translations-entry-for-lingua linguas))) (define (with-default value default) (if value value default)) (lambda (msgid lingua) "Returns the msgstr for MSGID from the po file for LINGUA." (let ((translations (assoc-ref translation-lists lingua))) (with-default (assoc-ref translations msgid) msgid))))) As a Gettext-like shorthand I wrote a macro called _ which calls the above translated-msg function. It takes the locale from the current-lingua variable, so the macro deliberately breaks hygiene. (define-syntax _ (lambda (x) "Gettext-like shorthand for translated-msg with what currently is curre= nt-lingua." (syntax-case x () ((_ msg) (with-syntax ((current-lingua (datum->syntax x 'current-lingua))) #'(translated-msg msg current-lingua)))))) I use it like in this excerpt: (define (back-button-for-lingua lingua) "SXML for a link back to the home page." (let ((current-lingua lingua)) `(a (@ (href ,(string-append "/index" "-" lingua ".html")) (class "full-width-link")) ,(_ "=E2=86=90 Back to home page")))) Then I ran the xgettext program from the terminal to create a pot file =66rom all strings marked with _. xgettext -f po/POTFILES -o po/pelzfloriande-website.pot --from-code=3DUTF-8= --copyright-holder=3D"" --package-name=3D"pelzfloriande-website" --msgid-b= ugs-address=3D"pelzflorian@pelzflorian.de" --keyword=3D_ Xgettext autodetected all the strings that were marked for translation. This is much better than my previous approach where I had to list all of them manually in a manually written association list. To create a po file from a pot file, I do the usual: cd po msginit -l de --no-translator msginit -l en --no-translator Then I filled out the po files using gtranslator. I can now run =E2=80=9Chaunt build=E2=80=9D with an appropriate GUILE_LOAD_PATH, for me c= urrently: GUILE_LOAD_PATH=3D$HOME/keep/projects/pelzfloriande-website:$HOME/build/nya= cc/src/nyacc/examples:$GUILE_LOAD_PATH GUILE_LOAD_COMPILED_PATH=3D$GUILE_LO= AD_COMPILED_PATH:$HOME/.cache/guile/ccache/2.2-LE-8-3.A/home/florian/keep/p= rojects/pelzfloriande-website haunt build Is this the right approach? You can find my unfinished, not very clean code at [5]. Regards, Florian [1] https://pelzflorian.de [2] https://haunt.dthompson.us/ [3] https://savannah.nongnu.org/projects/nyacc/ [4] https://www.gnu.org/software/gettext/manual/html_node/libgettextpo.html [5] https://pelzflorian.de/git/pelzfloriande-website/commit/?id=3D5f97bf157= eaddcfe722c97dcab349b7dcfbbcd9d --AhhlLboLdkugWU4S Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEwRjGsqAMqXB4uw3y3T6EbElHBVsFAlosJhoACgkQ3T6EbElH BVu5Kw//aOcja3XUYRcpqGlQyDMX1TtiHFbu6i6v0OawZ+h6DK8TsFm7+9UZPzdY kF/kehB9itVZDmONloggaguDWlqXfPLJNgXPQZBgbNV0RqTA5CHSYlL3t5CSRFcU KxO+x3crNS8Z+ng0oxLTV298OoMG5z5mLnanApDyVXk1re+dJptM+ueAOgPbI2tU sHhvpMbf/M8p87+4ZN7AKGFv8u4xun11Dced/gAjQKmWwLcZfi3JRzYWVrDgDQzS QOyWNOqM59IbXEO7hVw2dMYCbq7wKN4CP/1NfPwGv3tqP6WlKBYlexe7wa2FcRxZ RZ9s8Vrju3+h72RrZU1uaLNkZZF0/aqh+JGZ3vnr7U7IaVt8Zvo5ILTh1GR18YHH 1hN6Ik1vyXaaDhwuIzAf/FwbNgojQO8Vj/9ZcpVTw6MC6NGV4CI33zstEQQQzrEc 99/gtReDOFA55GcetSoXsx5kwklvt12t4O3bYx2EQFxUo2RK5H8+xTbPOpmMSw8y rTIq65WFye+tBFR/t0ZEtMeNDFW86bnr9RPCvK8gFO3qQowLuhpUksmApPJXZhNx PAl+W8+e6rQ8mEgT7p1JepcMbJOAwQzubN5hn/fShtW/RPAcU8w9LbWVgK0Hjo9T vC3bGfCojMFonJ0UhNnSAhrWxuJpXnVfzT2dJs95cZhMTW13Mco= =bIZp -----END PGP SIGNATURE----- --AhhlLboLdkugWU4S--