Re: Haunt patches

Re: Haunt patches

[pelzflorian (Florian Pelz)]

[-- Attachment #1: Type: text/plain, Size: 2233 bytes --]

Hello,

(Ccing guix-devel@gnu.org.)

I wrote patches that introduce page variants for translations.  I will
send them right after this mail.  If there are interested guix-devel
readers, they can find them at

https://pelzflorian.de/git/haunt/commit/?id=ac3e93fe35363fd0066cf93b969c95c0fde7a25a
https://pelzflorian.de/git/haunt/commit/?id=ca32925a58c8ec526653aff9825be2359bf358c3
https://pelzflorian.de/git/haunt/commit/?id=34f6b56bfe3a3059ced4be5e9b768a6a3a93c671

A site can have a list of variants such as various languages,
e.g. '("de" "en").

Each page body can then optionally be a procedure that given a variant
returns a body.  If a page body is a procedure, the page is written
multiple times, once for each variant, unless the procedure returns #f
for the variant.

I previously wrote about variants in this discussion on the Guix
mailing list.

https://lists.gnu.org/archive/html/guix-devel/2018-02/msg00118.html

While changing things I also added a #:subtitle key to
Atom-feeds-by-tag.

I already restructured my website’s code to make use of the patches.

https://pelzflorian.de/git/pelzfloriande-website/commit/?id=02e80b47fe0a9929a25d4a5f7ce11a15e1042d6f

I would be grateful if you could comment further on style etc. so this
can eventually be merged, unless you think the approach is wrong.
Everything that can be done with variants is possible already by
adding one site builder for each language instead of just one site
builder, e.g. adding the same blog site builder once for each variant.
I think variants are the cleaner solution even though it means adding
many lambdas for each body.

If the approach is right, I should probably write documentation for
it.

Alternatively backwards compatibility could be disregarded and
procedures returning bodies could take one more argument, the variant.
Another alternative would be doing nothing (except for adding
subtitles to atom-feeds-by-tag) since variants are just prettier but
not strictly needed.

One problem is that the page file name is not an argument to blog
layouts, so one cannot easily put links to the other variants of a
page on a page.  This does not yet address this issue.

Regards,
Florian

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Haunt patches

[Ludovic Courtès]

Heya,

"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:

> I wrote patches that introduce page variants for translations.  I will
> send them right after this mail.  If there are interested guix-devel
> readers, they can find them at
>
> https://pelzflorian.de/git/haunt/commit/?id=ac3e93fe35363fd0066cf93b969c95c0fde7a25a
> https://pelzflorian.de/git/haunt/commit/?id=ca32925a58c8ec526653aff9825be2359bf358c3
> https://pelzflorian.de/git/haunt/commit/?id=34f6b56bfe3a3059ced4be5e9b768a6a3a93c671
>
> A site can have a list of variants such as various languages,
> e.g. '("de" "en").

Nice, looks like a great starting point!

This does not address i18n in itself, right?  Do you have examples of an
i18n’d web site using this Haunt branch + gettext?

Thank you!

Ludo’.

Re: Haunt patches

[pelzflorian (Florian Pelz)]

[-- Attachment #1: Type: text/plain, Size: 1258 bytes --]

On Fri, Feb 09, 2018 at 02:18:13PM +0100, Ludovic Courtès wrote:
> Heya,
> 
> "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:
> 
> > I wrote patches that introduce page variants for translations.  I will
> > send them right after this mail.  If there are interested guix-devel
> > readers, they can find them at
> >
> > https://pelzflorian.de/git/haunt/commit/?id=ac3e93fe35363fd0066cf93b969c95c0fde7a25a
> > https://pelzflorian.de/git/haunt/commit/?id=ca32925a58c8ec526653aff9825be2359bf358c3
> > https://pelzflorian.de/git/haunt/commit/?id=34f6b56bfe3a3059ced4be5e9b768a6a3a93c671
> >
> > A site can have a list of variants such as various languages,
> > e.g. '("de" "en").
> 
> Nice, looks like a great starting point!
> 

:)

> This does not address i18n in itself, right?  Do you have examples of an
> i18n’d web site using this Haunt branch + gettext?
> 
> Thank you!
> 
> Ludo’.

My Website uses it:

https://pelzflorian.de/git/pelzfloriande-website/

I use xgettext, msginit, msgmerge and msgfmt tools from Gettext to
create the PO files.  I now added a NOTES file on how to do so.

I use the po files with my haunt.scm’s translate-msg function and _
and __ macros.

Regards,
Florian

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Web site i18n with Haunt

[Ludovic Courtès]

"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis:

> On Fri, Feb 09, 2018 at 02:18:13PM +0100, Ludovic Courtès wrote:

[...]

>> This does not address i18n in itself, right?  Do you have examples of an
>> i18n’d web site using this Haunt branch + gettext?
>> 
>> Thank you!
>> 
>> Ludo’.
>
> My Website uses it:
>
> https://pelzflorian.de/git/pelzfloriande-website/
>
> I use xgettext, msginit, msgmerge and msgfmt tools from Gettext to
> create the PO files.  I now added a NOTES file on how to do so.
>
> I use the po files with my haunt.scm’s translate-msg function and _
> and __ macros.

Awesome.  There were a couple of people interested in internationalizing
our web site during the Guix workshop, so hopefully they can follow your
lead and ping you if they need help!

Ludo’.

Re: Web site i18n with Haunt

[pelzflorian (Florian Pelz)]

[-- Attachment #1: Type: text/plain, Size: 2337 bytes --]

On Fri, Feb 09, 2018 at 06:02:22PM +0100, Ludovic Courtès wrote:
> […]
> Awesome.  There were a couple of people interested in internationalizing
> our web site during the Guix workshop, so hopefully they can follow your
> lead and ping you if they need help!
> 
> Ludo’.

Gladly.  But perhaps others also have better ideas for how to do
it. ;) In particular, independent from Haunt, using XML for
translation strings as Ricardo proposed would look more familiar to
translators (and my own __ macro’s implementation is buggy for corner
cases and could certainly be simplified and cleaned up).  This must be
decided and written before the Guix website can be properly
internationalized.

How for example would this excerpt from my website better be rendered
as XML or is this or another non-XML approach better?


(p ,@(__ "Thank you for your interest in my workshop \
“GUI Programming with GTK+”. ||register_|To register please go |here|. ||For \
more information see ||link_|here||."
         `(("register_" .
            ,(lambda (before-link link-text after-link)
               (if enable-registration
                   `(span
                     ,before-link
                     ,(a-href
                       "/gui-prog-anmelden/"
                       link-text)
                     ,after-link)
                   "")))
           ("link_" .
            ,(lambda (text)
               (a-href
                (poster-url-for-lingua current-lingua)
                text))))))

Some people also had preferred automatic extraction of strings instead
of marking each one.  If this were to be done, it would need support
in Haunt itself.  However, false positives could not be avoided when
automatically looking for what strings to extract.

Also Haunt’s design is not complete.  I’m not entirely sure about my
patches’ approach regarding Atom feeds, and blog post layouts are
definitely a problem – layouts should take the file name as an
argument somehow.

Other tooling like what sirgazil proposed here
https://lists.gnu.org/archive/html/guile-devel/2017-12/msg00051.html
might also be desirable but is not urgent.  It probably should not be
part of Haunt but an external library just like __.

Then this all needs documentation.

Regards,
Florian

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Web site i18n with Haunt

[Ricardo Wurmus]

pelzflorian (Florian Pelz) <pelzflorian@pelzflorian.de> writes:

> How for example would this excerpt from my website better be rendered
> as XML or is this or another non-XML approach better?

I’d much prefer XML here.  One reason is that I find it very hard to
understand the syntax and how it is processed in your example.  Even
after looking at the example for some time, it’s still difficult to
understand.  You have non-translatable tags that end on an underscore,
but the number of pipe characters is not clear to me.  How is the
sentence with “register_” parsed?  Which parts are bound to
“before-link”, “link-text”, and “after-link”?

> (p ,@(__ "Thank you for your interest in my workshop \
> “GUI Programming with GTK+”. ||register_|To register please go |here|. ||For \
> more information see ||link_|here||."
>          `(("register_" .
>             ,(lambda (before-link link-text after-link)
>                (if enable-registration
>                    `(span
>                      ,before-link
>                      ,(a-href
>                        "/gui-prog-anmelden/"
>                        link-text)
>                      ,after-link)
>                    "")))
>            ("link_" .
>             ,(lambda (text)
>                (a-href
>                 (poster-url-for-lingua current-lingua)
>                 text))))))

The translatable text could be written as two or more strings (it
doesn’t matter that they end up in the same paragraph).  The first one
is easy:

  "Thank you for your interest in my workshop “GUI Programming with GTK+”."

The second one might be:

  "To register please go <register-link>here</register-link>."

The third:

  "For more information see <link>here</link>."

(Whatever tag name is used is arbitrary and only has to be unique within
the context of a single translatable string.)  Or they could all be one
big XML snippet.

With sxpath we can easily pick tagged substrings by specifying their
path.  Or we could fold over the parse tree and transform it by applying
an arbitrary transformation.

Here’s a trivial example of such a transform:

--8<---------------cut here---------------start------------->8---
(use-modules (sxml simple) (sxml transform))
;; This is the translated string, wrapped in some tag to make it a valid
;; XML fragment.
(define tr "<translation>Click <link>here</link></translation>")

;; Convert to SXML, then transform it.
(pre-post-order (xml->sxml tr)
 ;; When we find the tag “link” wrap the contents in a URL anchor.
 `((link . ,(lambda (tag . kids)
              `(a (@ (href "http://gnu.org")) ,kids)))
   ;; Just wrap the contents in a tag for everything else
   (*default*  . ,(lambda (tag . kids) `(,tag ,@kids)))
   ;; Unwrap all text
   (*text*     . ,(lambda (_ txt) txt))))

=> (*TOP* (translation "Click " (a (@ (href "http://gnu.org")) "here")))
--8<---------------cut here---------------end--------------->8---

Using “pre-post-order” directly is verbose as you need to specify the
*default* and *text* handlers, but this can easily be hidden by a
friendlier procedure that would present a user interface that wouldn’t
look too different from what your macro presents to users.

--8<---------------cut here---------------start------------->8---
(define (foo str . handlers)
 (pre-post-order (xml->sxml (string-append "<translation>" str "</translation>"))
  `(,@handlers
    (*default*  . ,(lambda (tag . kids) `(,tag ,@kids)))
    (*text*     . ,(lambda (_ txt) txt)))))

(foo "Click <link>here</link>"
 `(link . ,(lambda (tag . contents)
           `(a (@ (href "/help")) ,@contents))))
--8<---------------cut here---------------end--------------->8---

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Web site i18n with Haunt

[pelzflorian (Florian Pelz)]

[-- Attachment #1: Type: text/plain, Size: 3408 bytes --]

On Sat, Feb 10, 2018 at 12:14:26AM +0100, Ricardo Wurmus wrote:
> pelzflorian (Florian Pelz) <pelzflorian@pelzflorian.de> writes:
> 
> > How for example would this excerpt from my website better be rendered
> > as XML or is this or another non-XML approach better?
> 
> I’d much prefer XML here.  One reason is that I find it very hard to
> understand the syntax and how it is processed in your example.  […]

I agree.

> The translatable text could be written as two or more strings (it
> doesn’t matter that they end up in the same paragraph).  […]

Yes, you are right, the context will still be clear for translators
because the consecutive line numbers are next to each other in the PO
file and they do not appear at two unrelated places in the code.

> With sxpath we can easily pick tagged substrings by specifying their
> path.  Or we could fold over the parse tree and transform it by applying
> an arbitrary transformation.
> 
> Here’s a trivial example of such a transform:
> 
> --8<---------------cut here---------------start------------->8---
> (use-modules (sxml simple) (sxml transform))
> ;; This is the translated string, wrapped in some tag to make it a valid
> ;; XML fragment.
> (define tr "<translation>Click <link>here</link></translation>")
> 
> ;; Convert to SXML, then transform it.
> (pre-post-order (xml->sxml tr)
>  ;; When we find the tag “link” wrap the contents in a URL anchor.
>  `((link . ,(lambda (tag . kids)
>               `(a (@ (href "http://gnu.org")) ,kids)))
>    ;; Just wrap the contents in a tag for everything else
>    (*default*  . ,(lambda (tag . kids) `(,tag ,@kids)))
>    ;; Unwrap all text
>    (*text*     . ,(lambda (_ txt) txt))))
> 
> => (*TOP* (translation "Click " (a (@ (href "http://gnu.org")) "here")))
> --8<---------------cut here---------------end--------------->8---
> 
> Using “pre-post-order” directly is verbose as you need to specify the
> *default* and *text* handlers, but this can easily be hidden by a
> friendlier procedure that would present a user interface that wouldn’t
> look too different from what your macro presents to users.
> 
> --8<---------------cut here---------------start------------->8---
> (define (foo str . handlers)
>  (pre-post-order (xml->sxml (string-append "<translation>" str "</translation>"))
>   `(,@handlers
>     (*default*  . ,(lambda (tag . kids) `(,tag ,@kids)))
>     (*text*     . ,(lambda (_ txt) txt)))))
> 
> (foo "Click <link>here</link>"
>  `(link . ,(lambda (tag . contents)
>            `(a (@ (href "/help")) ,@contents))))
> --8<---------------cut here---------------end--------------->8---
>

So I took your approach and ran with it:

https://pelzflorian.de/git/pelzfloriande-website/commit/?id=77e1c30532f0bf834ddc6709c7883c17bb46740f

I did not use SXPath.  This one still looks ugly:

(div (@ (id "powered-by"))
     ,@(__ "Powered by <link \
url=\"https://www.gnu.org/software/guile/\">GNU Guile</link> and \
<link url=\"https://haunt.dthompson.us/\">Haunt</link>."
           `(link
             . ,(lambda (tag attr text)
                  (a-href
                   (cadadr attr)
                   text)))))

I suppose I should have used SXPath for it to get the URL out of the
(@ (url "…")) attribute?

Thank you all so much for your code and your help.

Regards,
Florian

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Web site i18n with Haunt

[Ricardo Wurmus]

pelzflorian (Florian Pelz) <pelzflorian@pelzflorian.de> writes:

> I did not use SXPath.  This one still looks ugly:
>
> (div (@ (id "powered-by"))
>      ,@(__ "Powered by <link \
> url=\"https://www.gnu.org/software/guile/\">GNU Guile</link> and \
> <link url=\"https://haunt.dthompson.us/\">Haunt</link>."
>            `(link
>              . ,(lambda (tag attr text)
>                   (a-href
>                    (cadadr attr)
>                    text)))))
>
> I suppose I should have used SXPath for it to get the URL out of the
> (@ (url "…")) attribute?

Since the URL isn’t supposed to be translated I wouldn’t present it as
part of the translatable text, i.e. the string would be:

    Powered by <guile>GNU Guile</guile> and <haunt>Haunt</haunt>.

And then I’d apply the transformations:

    (foo s
     `(guile . ,(lambda (_ . contents)
                 (a-href "https://www.gnu.org/software/guile/" ,@contents)))
     `(haunt . ,(lambda (_ . contents)
                 (a-href "https://haunt.dthompson.us/" ,@contents))))

You could use sxpath to avoid cadadr, but it wouldn’t be less verbose:

    (foo s
     `(link . ,(lambda (_ attr contents)
                (a-href (car ((sxpath '(url *text*)) attr)) ,contents))))

(Correctly but annoyingly, “sxpath” always returns a list, so we still
need to unpack the URL from the result with “car”.)

sxpath takes a path as a list of symbols and returns a selector function
that takes a nodeset.  So the selector function applied to “attr” gets
us a list of all “*text*” nodes inside of a “url” node in the argument
“attr”.

--
Ricardo

GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
https://elephly.net

Re: Web site i18n with Haunt

[pelzflorian (Florian Pelz)]

[-- Attachment #1: Type: text/plain, Size: 2177 bytes --]

On Sun, Feb 11, 2018 at 03:45:12PM +0100, Ricardo Wurmus wrote:
> 
> pelzflorian (Florian Pelz) <pelzflorian@pelzflorian.de> writes:
> 
> > I did not use SXPath.  This one still looks ugly:
> >
> > (div (@ (id "powered-by"))
> >      ,@(__ "Powered by <link \
> > url=\"https://www.gnu.org/software/guile/\">GNU Guile</link> and \
> > <link url=\"https://haunt.dthompson.us/\">Haunt</link>."
> >            `(link
> >              . ,(lambda (tag attr text)
> >                   (a-href
> >                    (cadadr attr)
> >                    text)))))
> >
> > I suppose I should have used SXPath for it to get the URL out of the
> > (@ (url "…")) attribute?
> 
> Since the URL isn’t supposed to be translated I wouldn’t present it as
> part of the translatable text, i.e. the string would be:
> 
>     Powered by <guile>GNU Guile</guile> and <haunt>Haunt</haunt>.
> […]

Well, if eventually the websites of Guile and/or Haunt were to be
translated, then I would like to localize the URL. ;)



> You could use sxpath to avoid cadadr, but it wouldn’t be less verbose:
> 
>     (foo s
>      `(link . ,(lambda (_ attr contents)
>                 (a-href (car ((sxpath '(url *text*)) attr)) ,contents))))
> 
> (Correctly but annoyingly, “sxpath” always returns a list, so we still
> need to unpack the URL from the result with “car”.)
> 
> sxpath takes a path as a list of symbols and returns a selector function
> that takes a nodeset.  So the selector function applied to “attr” gets
> us a list of all “*text*” nodes inside of a “url” node in the argument
> “attr”.
> 

I switched my website to use your sxpath code.  Thank you.  It is more
maintainable and I do not really care about verbosity.  The SXPath
info page was too long for me to read thoroughly without examples.

I chose not to use _ as its the same as what I use for my Gettext
macro.  sirgazil said here
https://lists.gnu.org/archive/html/guile-devel/2017-12/msg00051.html
that they prefer l10n as the gettext alias instead of _.  I guess I’ll
stick with _ albeit a somewhat unfortunate choice.

Regardsm
Florian

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]