Website translations with Haunt

["pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de>]

[-- Attachment #1: Type: text/plain, Size: 6765 bytes --]

Hello,

First of all, I want to say thank you to the Guile, Haunt, ffi-helper
and related projects’ developers.

I built my personal website [1] using David Thompson’s Haunt [2] and
recently talked my university’s Islamic Students’ Association
(Islamische Hochschulvereinigung) into using Haunt for their
not-yet-finished website as well, because I think the concept of Haunt
and SHTML is superior to alternatives.  However in order to make the
website multilingual (the user can choose to view it in German or
English) so far I used an association list with assoc-ref which is not
very comfortable since all strings have to be added in two places,
i.e. in the SHTML code and in the association list where the code
looks for translations.

I want to ask for your thoughts on my new solution since translations
are probably important to many Haunt users.  In particular, I believe
there was some discussion on Website translation on the Guile or Guix
lists as well.

I did not want to use the ordinary gettext functions in order to not
call setlocale very often to switch languages.  It seems the Gettext
system is not designed for rapidly changing locales, but maybe I am
wrong about this and very many setlocale calls would not be that bad.

Using Matt Wette’s ffi-helper [3] and the libgettextpo from GNU
Gettext, [4] I now wrote code to convert a po file into an association
list where I can now look up translations.  (Note that I had to make
some patches to the c99dev branch of nyacc before building nyacc and
ffi-helper because the Makefile had spaces instead of tabs at one
place and it did not find guile and guild.  Maybe these patches were
just necessary because of a broken setup on my part though.)

I used this dot.ffi file to create libgettextpo bindings:

(define-ffi-module (gettext-po)
  #:include '("gettext-po.h")
  #:library '("libgettextpo"))

Then I wrote a procedure to convert a po file to an association list
to look up the msgstr for a msgid.  Note that by lingua I basically
mean a locale.

(use-modules
 (gettext-po)
 (system ffi-help-rt)
 ((system foreign)
         #:prefix ffi:)
 […])

(define xerror-handler-struct
  (make-struct-po_xerror_handler)) ; TODO SET HANDLERS:
;; […]

(define (translations-for-lingua lingua)
  "Returns po/<lingua>.po converted to an association list of msgid–msgstr pairs."
  ;; TODO: STILL DISREGARDING PLURALS AND OTHER INFORMATION
  (let* ((po-file
          (po_file_read_v3
           (string-append "po/" lingua ".po")
           (pointer-to xerror-handler-struct)))
         (translations
          (if (ffi:null-pointer? (unwrap~pointer po-file))
              '()
              ;; otherwise:
              (let ((iter (po_message_iterator po-file ffi:%null-pointer)))
                (let loop ((message (po_next_message iter)))
                  (if (ffi:null-pointer? (unwrap~pointer message))
                      (begin
                        (po_message_iterator_free iter)
                        '())
                      ;; otherwise:
                      (cons
                       (cons
                        (ffi:pointer->string (po_message_msgid message))
                        (ffi:pointer->string (po_message_msgstr message)))
                       (loop (po_next_message iter)))))))))
    (if (not (ffi:null-pointer? (unwrap~pointer po-file)))
        (po_file_free po-file))
    translations))



I did this for every locale and made a second association list mapping
the locales to the msgid–msgstr association lists.  Then I wrote
translated-msg to do the lookup.


(define (translations-entry-for-lingua lingua)
  "Returns a pair of LINGUA and an association list of its translations."
  (cons
   lingua
   (translations-for-lingua lingua)))

(define translated-msg
  ;; gettext is not used directly because it would require repeated
  ;; setlocale calls, which should not be necessary.
  ;; See: https://stackoverflow.com/questions/3398113/php-gettext-problems
  (let ((translation-lists
         (map translations-entry-for-lingua linguas)))
    (define (with-default value default)
      (if value value
          default))
    (lambda (msgid lingua)
      "Returns the msgstr for MSGID from the po file for LINGUA."
      (let ((translations (assoc-ref translation-lists lingua)))
        (with-default
         (assoc-ref translations msgid)
         msgid)))))


As a Gettext-like shorthand I wrote a macro called _ which calls the
above translated-msg function.  It takes the locale from the
current-lingua variable, so the macro deliberately breaks hygiene.

(define-syntax _
  (lambda (x)
    "Gettext-like shorthand for translated-msg with what currently is current-lingua."
    (syntax-case x ()
      ((_ msg)
       (with-syntax ((current-lingua (datum->syntax x 'current-lingua)))
                    #'(translated-msg msg current-lingua))))))


I use it like in this excerpt:

(define (back-button-for-lingua lingua)
  "SXML for a link back to the home page."
  (let ((current-lingua lingua))
    `(a (@ (href ,(string-append "/index" "-" lingua ".html"))
           (class "full-width-link"))
        ,(_ "← Back to home page"))))


Then I ran the xgettext program from the terminal to create a pot file
from all strings marked with _.

xgettext -f po/POTFILES -o po/pelzfloriande-website.pot --from-code=UTF-8 --copyright-holder="" --package-name="pelzfloriande-website" --msgid-bugs-address="pelzflorian@pelzflorian.de" --keyword=_

Xgettext autodetected all the strings that were marked for
translation.  This is much better than my previous approach where I
had to list all of them manually in a manually written association
list.

To create a po file from a pot file, I do the usual:

cd po
msginit -l de --no-translator
msginit -l en --no-translator


Then I filled out the po files using gtranslator.  I can now run
“haunt build” with an appropriate GUILE_LOAD_PATH, for me currently:

GUILE_LOAD_PATH=$HOME/keep/projects/pelzfloriande-website:$HOME/build/nyacc/src/nyacc/examples:$GUILE_LOAD_PATH GUILE_LOAD_COMPILED_PATH=$GUILE_LOAD_COMPILED_PATH:$HOME/.cache/guile/ccache/2.2-LE-8-3.A/home/florian/keep/projects/pelzfloriande-website haunt build



Is this the right approach?

You can find my unfinished, not very clean code at [5].

Regards,
Florian


[1] https://pelzflorian.de
[2] https://haunt.dthompson.us/
[3] https://savannah.nongnu.org/projects/nyacc/
[4] https://www.gnu.org/software/gettext/manual/html_node/libgettextpo.html
[5] https://pelzflorian.de/git/pelzfloriande-website/commit/?id=5f97bf157eaddcfe722c97dcab349b7dcfbbcd9d

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]