Before l10n, better practices for (message) ?

Before l10n, better practices for (message) ?

[Jean-Christophe Helary]

I just bumped into an English/code bug this morning. In package.el, when 1 package is not needed anymore, the message is:

"Package menu: Operation finished.  1 packages are no longer needed, type ‘M-x package-autoremove’ to remove them"


The error comes from the following (message) on line 3273

(message "Package menu: Operation finished.  %d packages %s"
	(length removable)
	(substitute-command-keys
		"are no longer needed, type `\\[package-autoremove]' to remove them"))

So I'm asking whether do we have "best practices" for using messages...

Jean-Christophe 

Re: Before l10n, better practices for (message) ?

[Tino Calancha]

[-- Attachment #1: Type: text/plain, Size: 1976 bytes --]



On Tue, 23 May 2017, Jean-Christophe Helary wrote:

> I just bumped into an English/code bug this morning. In package.el, when 1 package is not needed anymore, the message is:
>
> "Package menu: Operation finished.  1 packages are no longer needed, type ‘M-x package-autoremove’ to remove them"
>
>
> The error comes from the following (message) on line 3273
>
> (message "Package menu: Operation finished.  %d packages %s"
> 	(length removable)
> 	(substitute-command-keys
> 		"are no longer needed, type `\\[package-autoremove]' to remove them"))
>
> So I'm asking whether do we have "best practices" for using messages...
This was discussed in
https://lists.gnu.org/archive/html/emacs-devel/2016-09/msg00502.html

I proposed to add a simple function `string-plural-s' to standarize those
plurarizations.


Using such `string-plural-s' in your package example:

(let* ((options '("foo"))
        (num (length options)))
   (message
    "Package menu: Operation finished.  %d %s %s no longer needed, \
type ‘M-x package-autoremove’ to remove %s"
    num (string-plural-s num "package" "packages")
    (string-plural-s options "is" "are")
    (string-plural-s num "it" "them")))
=> "Package menu: Operation finished.  1 package is no longer needed, type ‘M-x package-autoremove’ to remove it"


(let* ((options '("foo" "bar" "baz"))
        (num (length options)))
   (message
    "Package menu: Operation finished.  %d %s %s no longer needed, \
type ‘M-x package-autoremove’ to remove %s"
    num (string-plural-s num "package" "packages")
    (string-plural-s num "is" "are")
    (string-plural-s options "it" "them")))
=> "Package menu: Operation finished.  3 packages are no longer needed, type ‘M-x package-autoremove’ to remove them"


Then, the thread somehow evolved with more sophisticated
multilingual suggestions, which caused my head spin like
the girl in 'The exorcist' movie.
It would be good to have an standard way to handle these issues.

Re: Before l10n, better practices for (message) ?

[Jean-Christophe Helary]

> On May 23, 2017, at 9:59, Tino Calancha <tino.calancha@gmail.com> wrote:
> On Tue, 23 May 2017, Jean-Christophe Helary wrote:
> 
>> I just bumped into an English/code bug this morning. In package.el, when 1 package is not needed anymore, the message is:
>> 
>> "Package menu: Operation finished.  1 packages are no longer needed, type ‘M-x package-autoremove’ to remove them"
...
>> So I'm asking whether do we have "best practices" for using messages...
> This was discussed in
> https://lists.gnu.org/archive/html/emacs-devel/2016-09/msg00502.html
> 
> I proposed to add a simple function `string-plural-s' to standarize those
> plurarizations.

Yes, but no.

As a person who pays his bills doing translation and l10n, I would *never* recommend to create UI strings programatically based on *grammatical* assumptions. In fact I would forbid that if I could :) And I think that should be an Emacs policy for all UI strings...

I'm currently going through package.el and the easiest solution I have for now is replace things like "%d package%s to install" (where %s is "s" depending on whether %d is greater than 1...) into "Number of packages to install: %d.".

The packages.el authors have created incredible messages where "ing" is replaced by "ed" depending on whether the process is running or completed, just so as to save a few lines of static strings. Of course, the code is very clever, but if we want to create a base for l10n one day, all these clever grammatical hacks must permanently go away...

So, as I just wrote, I'm working on package.el right now and I'll submit a patch, and when I'm done, I'd like to see what kind of mechanisms we have (or we need to create) to extract strings and use localized ressources.

Jean-Christophe

Re: Before l10n, better practices for (message) ?

[Eli Zaretskii]

> From: Jean-Christophe Helary <jean.christophe.helary@gmail.com>
> Date: Tue, 23 May 2017 10:18:46 +0900
> 
> > I proposed to add a simple function `string-plural-s' to standarize those
> > plurarizations.
> 
> Yes, but no.
> 
> As a person who pays his bills doing translation and l10n, I would *never* recommend to create UI strings programatically based on *grammatical* assumptions. In fact I would forbid that if I could :) And I think that should be an Emacs policy for all UI strings...

Right.

> I'm currently going through package.el and the easiest solution I have for now is replace things like "%d package%s to install" (where %s is "s" depending on whether %d is greater than 1...) into "Number of packages to install: %d.".

Just replace it with 2 different spellings dispatched by the number.
Using the likes of package%s is frowned upon from the translations
POV, exactly as replacing "ing" with "ed".

> So, as I just wrote, I'm working on package.el right now and I'll submit a patch, and when I'm done, I'd like to see what kind of mechanisms we have (or we need to create) to extract strings and use localized ressources.

We don't have any mechanisms that I know of, and won't have until the
large part of the i10n issues is resolved by motivated individuals.
So for now let's fix this bug in the most direct and "unclever" way we
know of.

Thanks.

Re: Before l10n, better practices for (message) ?

[Jean-Christophe Helary]

> On May 23, 2017, at 11:55, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> I'm currently going through package.el and the easiest solution I have for now is replace things like "%d package%s to install" (where %s is "s" depending on whether %d is greater than 1...) into "Number of packages to install: %d.".
> 
> Just replace it with 2 different spellings dispatched by the number.

I know we don't have l10n now, but what you propose is also based on the grammatical assumption that we have only 2 possible forms so I would try to avoid that too.

>> So, as I just wrote, I'm working on package.el right now and I'll submit a patch, and when I'm done, I'd like to see what kind of mechanisms we have (or we need to create) to extract strings and use localized ressources.
> 
> So for now let's fix this bug in the most direct and "unclever" way we
> know of.

Ok, I'll do it.

> We don't have any mechanisms that I know of, and won't have until the
> large part of the i10n issues is resolved by motivated individuals.

I guess I just found something to do in the coming months... :)

What I'll do is propose a patch for package.el that makes UI strings the most linguistically neutral possible (within my understanding of English), and after discussing the changes, if we can use that as a base to create a list of "best practices", I'll use that to check the other packages included in the distribution.

Once we have the strings right, we'll worry about the other mechanisms that lead to the internationalization of the packages.

Jean-Christophe 

Re: Before l10n, better practices for (message) ?

[Eli Zaretskii]

> From: Jean-Christophe Helary <jean.christophe.helary@gmail.com>
> Date: Tue, 23 May 2017 12:38:29 +0900
> 
> 
> > On May 23, 2017, at 11:55, Eli Zaretskii <eliz@gnu.org> wrote:
> > 
> >> I'm currently going through package.el and the easiest solution I have for now is replace things like "%d package%s to install" (where %s is "s" depending on whether %d is greater than 1...) into "Number of packages to install: %d.".
> > 
> > Just replace it with 2 different spellings dispatched by the number.
> 
> I know we don't have l10n now, but what you propose is also based on the grammatical assumption that we have only 2 possible forms so I would try to avoid that too.

Having 2 forms is as close to the ideal as we can have, given that we
don't have infrastructure for more complicated cases.  Introducing
functions that will meanwhile do nothing non-trivial doesn't sound a
good idea to me; such functions will only make sense when they
actually support non-trivial plurals (and for that, we need first to
figure out how to select the language into which any given message
needs to be translated).

> > We don't have any mechanisms that I know of, and won't have until the
> > large part of the i10n issues is resolved by motivated individuals.
> 
> I guess I just found something to do in the coming months... :)
> 
> What I'll do is propose a patch for package.el that makes UI strings the most linguistically neutral possible (within my understanding of English), and after discussing the changes, if we can use that as a base to create a list of "best practices", I'll use that to check the other packages included in the distribution.

I'm not sure package.el is a good starting point.  I think a much
better starting point is to design the i10n infrastructure for Emacs.

Re: Before l10n, better practices for (message) ?

[Jean-Christophe Helary]

> On May 24, 2017, at 3:36, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> Having 2 forms is as close to the ideal as we can have, given that we
> don't have infrastructure for more complicated cases.

I'm just saying that for the case we are discussing, 1 form is enough:

>>>> "%d package%s to install"

->

>>>> "Number of packages to install: %d."

That's 1 form and that's way enough to convey the intended meaning.
It's just a way to write the original string. No need to use fancy grammar there.

>> What I'll do is propose a patch for package.el that makes UI strings the most linguistically neutral possible (within my understanding of English), and after discussing the changes, if we can use that as a base to create a list of "best practices", I'll use that to check the other packages included in the distribution.
> 
> I'm not sure package.el is a good starting point.  I think a much
> better starting point is to design the i10n infrastructure for Emacs.

I'm not considering l10n right now. Only best practices for writing UI strings. Considering the amount of convoluted strings in package.el, I think it is as good a start as it can be, at least for *that* purpose.

I'll have a few questions regarding the last 3 pieces of string generating code I found sometimes next week.

Jean-Christophe 

Re: Before l10n, better practices for (message) ?

[Eli Zaretskii]

> From: Jean-Christophe Helary <jean.christophe.helary@gmail.com>
> Date: Wed, 24 May 2017 07:00:46 +0900
> 
> > I'm not sure package.el is a good starting point.  I think a much
> > better starting point is to design the i10n infrastructure for Emacs.
> 
> I'm not considering l10n right now. Only best practices for writing UI strings.

IMO, such best practices will only make sense as part of some general
l10n effort.

But if you want to scratch that particular itch, no one can prevent
you from doping that.

Re: Before l10n, better practices for (message) ?

[Jean-Christophe Helary]

> On May 24, 2017, at 11:32, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> I'm not considering l10n right now. Only best practices for writing UI strings.
> 
> IMO, such best practices will only make sense as part of some general
> l10n effort.
> 
> But if you want to scratch that particular itch, no one can prevent
> you from doping that.

Ok, I'll drop the best practices. But there won't be a "general l10n effort" if we don't start somewhere. Where do you think that "somewhere" is?

Jean-Christophe 

Re: Before l10n, better practices for (message) ?

[Paul Eggert]

Jean-Christophe Helary wrote:
> there won't be a "general l10n effort" if we don't start somewhere. Where do you think that "somewhere" is?

It's here:

https://www.gnu.org/software/gettext/

Re: Before l10n, better practices for (message) ?

[Jean-Christophe Helary]

> On May 24, 2017, at 13:22, Paul Eggert <eggert@cs.ucla.edu> wrote:
> 
> Jean-Christophe Helary wrote:
>> there won't be a "general l10n effort" if we don't start somewhere. Where do you think that "somewhere" is?
> 
> It's here:
> 
> https://www.gnu.org/software/gettext/

Paul,

Thank you. I know what gettext is. But between gettexting emacs and now there are plenty of steps, that include straightening up the strings because as seen in package.el, code writers don't seem to have that in mind when they write their packages, but Eli seems to be talking about something else.

Jean-Christophe

Re: Before l10n, better practices for (message) ?

[Eli Zaretskii]

> From: Jean-Christophe Helary <jean.christophe.helary@gmail.com>
> Date: Wed, 24 May 2017 17:08:51 +0900
> 
> > https://www.gnu.org/software/gettext/
> 
> Paul,
> 
> Thank you. I know what gettext is.

I'm quite sure telling you about gettext was not what Paul had in
mind.  He most probably wanted to point out that any infrastructure of
this kind should at least consider using gettext, if not actually use
it.  Which is something I could agree with.

> But between gettexting emacs and now there are plenty of steps, that include straightening up the strings because as seen in package.el, code writers don't seem to have that in mind when they write their packages, but Eli seems to be talking about something else.

I thought about taking first steps towards providing the missing
infrastructure in Emacs.  For example, providing a way to display a
translation of a given string to a given language.  Is that something
you'd like to work on?

Re: Before l10n, better practices for (message) ?

[Jean-Christophe Helary]

> On May 25, 2017, at 4:12, Eli Zaretskii <eliz@gnu.org> wrote:

> I thought about taking first steps towards providing the missing
> infrastructure in Emacs.  For example, providing a way to display a
> translation of a given string to a given language.  Is that something
> you'd like to work on?

Yes, very much. Considering my current understanding of Emacs and of Elisp, it's going to take some time. But yes.

Jean-Christophe 

Re: Before l10n, better practices for (message) ?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  >   For example, providing a way to display a
  > > translation of a given string to a given language.  Is that something
  > > you'd like to work on?

  > Yes, very much. Considering my current understanding of Emacs and of Elisp, it's going to take some time. But yes.

Thank you for working on this.  Most GNU packages have translations for
their messages -- Emacs is a sad exception.  We really should set up
a scheme for Emacs to handle translations of is messages.
-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.

Re: Before l10n, better practices for (message) ?

[Paul Eggert]

On 05/24/2017 12:12 PM, Eli Zaretskii wrote:
> He most probably wanted to point out that any infrastructure of
> this kind should at least consider using gettext, if not actually use
> it.

Yes, that was the point. Emacs should be able to internationalize at 
least as well as gettext does. And basing Emacs i18n on gettext should 
let us easily share infrastructure with other GNU applications and with 
the GNU Translation project. There should be no need to reinvent this 
particular wheel.

On 05/24/2017 01:08 AM, Jean-Christophe Helary wrote:
 > between gettexting emacs and now there are plenty of steps, that 
include straightening up the strings

There is no point to straightening up strings if we don't have a 
translation infrastructure. In contrast, it would be helpful to have the 
infrastructure even if some strings still need straightening up, because 
the other strings will be translated. This suggests that we should focus 
our initial efforts on getting the infrastructure up, and worry about 
straightening up strings later. That is why I suggested gettext in 
response to your question about where we should start.

Re: Before l10n, better practices for (message) ?

[Jean-Christophe Helary]

> On May 25, 2017, at 7:09, Paul Eggert <eggert@cs.ucla.edu> wrote:
> 
> On 05/24/2017 12:12 PM, Eli Zaretskii wrote:
>> He most probably wanted to point out that any infrastructure of
>> this kind should at least consider using gettext, if not actually use
>> it.
> 
> There is no point to straightening up strings if we don't have a translation infrastructure.

As the title suggests I was talking about better practices for UI messages regardless of whether we do l10n or not.
i18n and l10n are one thing, but having manageable strings that don't generate grammatical errors because developers consider elisp as a macro language for Natural Languages is a different thing.

It is not even a chicken and egg situation. Fixing strings and setting rules for developers will benefit all users right now. And we can also work on i18n, which is a totally different thing and requires a skill set that I'm pretty sure I don't have right now (while figuring out what a complex concat does is reasonably within what I can do right now).

> In contrast, it would be helpful to have the infrastructure even if some strings still need straightening up, because the other strings will be translated.

It would be helpful to have a lot of things. But l10n has been discussed for a while here and nobody is working on it as far as I can tell.

> This suggests that we should focus our initial efforts on getting the infrastructure up, and worry about straightening up strings later. That is why I suggested gettext in response to your question about where we should start.

No, this suggests that if you're interested in leading the i18n efforts, you can go ahead.

And as I just wrote to Eli, I'm fine with helping in that area but there are people who are much more qualified in terms of experience with Emacs and Elisp. My priority right now is to learn about Emacs and Elisp while helping where I can, and that does not include creating a full i18n infrastructure on my own.

Jean-Christophe 

Re: Before l10n, better practices for (message) ?

[Eli Zaretskii]

> From: Jean-Christophe Helary <jean.christophe.helary@gmail.com>
> Date: Thu, 25 May 2017 07:35:07 +0900
> Cc: Eli Zaretskii <eliz@gnu.org>,
>  Emacs-devel@gnu.org
> 
> > There is no point to straightening up strings if we don't have a translation infrastructure.
> 
> As the title suggests I was talking about better practices for UI messages regardless of whether we do l10n or not.
> i18n and l10n are one thing, but having manageable strings that don't generate grammatical errors because developers consider elisp as a macro language for Natural Languages is a different thing.
> 
> It is not even a chicken and egg situation. Fixing strings and setting rules for developers will benefit all users right now. And we can also work on i18n, which is a totally different thing and requires a skill set that I'm pretty sure I don't have right now (while figuring out what a complex concat does is reasonably within what I can do right now).

I see your point.  However, it seems to me that maintaining our doc
strings in good translatable order without actually having the ability
to translate them will impose a burden on the contributors which we
will be unable to justify in good faith.  So, although the initial
work, if done by you or some other motivated individual, should be
well within our reach, it is its maintenance henceforth that bothers
me.

Once we do have some minimal infrastructure for translations, we then
can ask the contributors to comply with its requirements without
having any moral dilemmas.

> And as I just wrote to Eli, I'm fine with helping in that area but there are people who are much more qualified in terms of experience with Emacs and Elisp. My priority right now is to learn about Emacs and Elisp while helping where I can, and that does not include creating a full i18n infrastructure on my own.

Thank you.  I do encourage you (and anyone else who is interested) to
start thinking and working on such an infrastructure.  My advice would
be to have this high-level goal guide you in your study of Emacs and
ELisp, i.e. I suggest to focus on those parts in Emacs that are most
relevant to that goal.  Then 2 good things will happen: (1) you will
have a specific practical goal and context for using your new
knowledge, and (2) you move closer to your goal by learning the
relevant stuff and coding relevant features.

Re: Before l10n, better practices for (message) ?

[Jean-Christophe Helary]

> On May 25, 2017, at 7:09, Paul Eggert <eggert@cs.ucla.edu> wrote:
> 
> That is why I suggested gettext in response to your question about where we should start.

Gettext is an l10n tool that requires an i18n infrastructure. As such it is at the *end* of the process. It is not where we start.

Eli's suggestion:

> For example, providing a way to display a translation of a given string to a given language.

*is* a place to start.

When we have that, we can worry about how to use a gettext-like process to get the l10n done.

Jean-Christophe 

Re: Before l10n, better practices for (message) ?

[Jean-Christophe Helary]

> On May 23, 2017, at 11:55, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> We don't have any mechanisms that I know of, and won't have until the
> large part of the i10n issues is resolved by motivated individuals.
> So for now let's fix this bug in the most direct and "unclever" way we
> know of.

If we were to externalize strings in a given file, would a (setq) at the beginning of the file do the trick or are there better ways to store "key=value" data?

Jean-Christophe 

Re: Before l10n, better practices for (message) ?

[Eli Zaretskii]

> From: Jean-Christophe Helary <jean.christophe.helary@gmail.com>
> Date: Tue, 23 May 2017 16:52:51 +0900
> 
> If we were to externalize strings in a given file, would a (setq) at the beginning of the file do the trick or are there better ways to store "key=value" data?

I don't think I understand the proposal.  Please elaborate.

Re: Before l10n, better practices for (message) ?

[Jean-Christophe Helary]

> On May 24, 2017, at 3:42, Eli Zaretskii <eliz@gnu.org> wrote:
> 
>> If we were to externalize strings in a given file, would a (setq) at the beginning of the file do the trick or are there better ways to store "key=value" data?
> 
> I don't think I understand the proposal.  Please elaborate.

(setq
(key1 "string1")
(key2 "string2"))

(message key1)
(message key2)

Jean-Christophe 

Re: Before l10n, better practices for (message) ?

[Eli Zaretskii]

> From: Jean-Christophe Helary <jean.christophe.helary@gmail.com>
> Date: Wed, 24 May 2017 07:03:36 +0900
> 
> 
> > On May 24, 2017, at 3:42, Eli Zaretskii <eliz@gnu.org> wrote:
> > 
> >> If we were to externalize strings in a given file, would a (setq) at the beginning of the file do the trick or are there better ways to store "key=value" data?
> > 
> > I don't think I understand the proposal.  Please elaborate.
> 
> (setq
> (key1 "string1")
> (key2 "string2"))

I believe you meant something like

  (setq key1 "string1"
        key2 "string2")

> (message key1)
> (message key2)

One problem with this approach is that the programmer who writes the
original code and provides the messages in English will have to
manually create the English message catalog.

Another problem is how to combine such catalogs from different source
files, and/or make sure the "keyN" names from different files don't
clash.

IOW, the question of suitable infrastructure is still there, with any
approach.  That's why it is better to start by using whatever relevant
infrastructure is provided by gettext, because at least some of these
issues are already solved there, and because the package itself is
widely available.

Re: Before l10n, better practices for (message) ?

[Jean-Christophe Helary]

> I believe you meant something like
> 
>  (setq key1 "string1"
>        key2 "string2")
> 
>> (message key1)
>> (message key2)
> 
> One problem with this approach is that the programmer who writes the
> original code and provides the messages in English will have to
> manually create the English message catalog.

It just occurred to me that in gettext, "key" *is* the English string... So all gettext really does is create a POT with the key, the PO is created with the localized string, and the binary MO provides the localized string if present and if not the program displays the original which already is *in* the code...

The key=string format I'm showing above is actually the format Java programs use (and until now I was mostly involved with i18n/l10n of FOSS java programs). So the programer is actually creating the catalogue leaving only arbitrary key names in the source...

> Another problem is how to combine such catalogs from different source
> files, and/or make sure the "keyN" names from different files don't
> clash.

Is it very different from global variables clashing or not between packages?

> IOW, the question of suitable infrastructure is still there, with any
> approach.  That's why it is better to start by using whatever relevant
> infrastructure is provided by gettext, because at least some of these
> issues are already solved there, and because the package itself is
> widely available.

Yes, but to me it looks like gettext works like it does this because C is not an interpreted language and forcing Lisp code to use the full gettext process does look a bit unnatural.

I can see that we use gettext to extract strings and create POTs to get POs. But once the POs are there, do we need to create binary MO blobs ? I'm not sure at all. So in the end gettext would be used only for string extraction and a few checks. But I may be missing something important here...

Jean-Christophe

Re: Before l10n, better practices for (message) ?

[Eli Zaretskii]

> From: Jean-Christophe Helary <jean.christophe.helary@gmail.com>
> Date: Fri, 26 May 2017 23:21:59 +0900
> 
> >  (setq key1 "string1"
> >        key2 "string2")
> > 
> >> (message key1)
> >> (message key2)
> > 
> > One problem with this approach is that the programmer who writes the
> > original code and provides the messages in English will have to
> > manually create the English message catalog.
> 
> It just occurred to me that in gettext, "key" *is* the English string...

Yes.  But then you cannot use setq on literal strings.

> > Another problem is how to combine such catalogs from different source
> > files, and/or make sure the "keyN" names from different files don't
> > clash.
> 
> Is it very different from global variables clashing or not between packages?

No.  But the need to name messages with package qualifiers is a
significant nuisance, and will fill the "namespace" of a file with
many symbols.

> > IOW, the question of suitable infrastructure is still there, with any
> > approach.  That's why it is better to start by using whatever relevant
> > infrastructure is provided by gettext, because at least some of these
> > issues are already solved there, and because the package itself is
> > widely available.
> 
> Yes, but to me it looks like gettext works like it does this because C is not an interpreted language and forcing Lisp code to use the full gettext process does look a bit unnatural.

I didn't say we should use all of the capabilities of gettext.  We
just need to use what suits us.

> I can see that we use gettext to extract strings and create POTs to get POs. But once the POs are there, do we need to create binary MO blobs ? I'm not sure at all. So in the end gettext would be used only for string extraction and a few checks. But I may be missing something important here...

Figuring out which parts of gettext should be used is part of the
job.  Whatever we find useful will be net win, since we will not need
to develop and maintain that.

Also, even if we don't use some part, it could provide ideas for
similar infrastructure tailored to our needs.

Re: Before l10n, better practices for (message) ?

[Yuri Khan]

On Fri, May 26, 2017 at 9:44 PM, Eli Zaretskii <eliz@gnu.org> wrote:

>> Is it very different from global variables clashing or not between packages?
>
> No.  But the need to name messages with package qualifiers is a
> significant nuisance, and will fill the "namespace" of a file with
> many symbols.

Messages do not have to be global variables. They could be retrieved
by a helper function accepting a package namespace and a message key.
In gettext, this function is dgettext.

    (require 'gettext)

    (message (dgettext "hello" "Hello World!"))

Gettext also has a helper function named gettext and aliased _, which
use the domain set by an earlier invocation of textdomain(). In Elisp,
it seems most natural to express that like this:

    (defun hello-_ (key)
      (dgettext "hello" key))
    (message (hello-_ "Hello World!"))

The function hello-_ would be defined once in the hello package and
then used throughout it. Another package would define its own wrapper
around dgettext, with a different domain.


Inside, dgettext could consult a two-level hashtable, first keyed by
domain, second by key. Alternatively, a domain could be a hashtable
variable; in the latter case:

    ;; in hello.el
    (defvar hello-domain nil)
    (defun hello-_ (key)
      (dgettext hello-domain key))
    (message (hello-_ "Hello World!"))

    ;; in gettext.el
    (defun dgettext (domain key)
      (gethash key domain key))

Re: Before l10n, better practices for (message) ?

[Eli Zaretskii]

> From: Yuri Khan <yuri.v.khan@gmail.com>
> Date: Sat, 27 May 2017 01:08:20 +0700
> Cc: Jean-Christophe Helary <jean.christophe.helary@gmail.com>, 
> 	Emacs developers <emacs-devel@gnu.org>
> 
>     (require 'gettext)
> 
>     (message (dgettext "hello" "Hello World!"))
> 
> Gettext also has a helper function named gettext and aliased _, which
> use the domain set by an earlier invocation of textdomain(). In Elisp,
> it seems most natural to express that like this:
> 
>     (defun hello-_ (key)
>       (dgettext "hello" key))
>     (message (hello-_ "Hello World!"))
> 
> The function hello-_ would be defined once in the hello package and
> then used throughout it. Another package would define its own wrapper
> around dgettext, with a different domain.
> 
> 
> Inside, dgettext could consult a two-level hashtable, first keyed by
> domain, second by key. Alternatively, a domain could be a hashtable
> variable; in the latter case:
> 
>     ;; in hello.el
>     (defvar hello-domain nil)
>     (defun hello-_ (key)
>       (dgettext hello-domain key))
>     (message (hello-_ "Hello World!"))
> 
>     ;; in gettext.el
>     (defun dgettext (domain key)
>       (gethash key domain key))

Why not hide all this ugliness inside 'message'?

Re: Before l10n, better practices for (message) ?

[Jean-Christophe Helary]

> On May 27, 2017, at 4:00, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> Why not hide all this ugliness inside 'message'?

I agree. We need to hide the l10n selection/display mechanism inside the string display functions in a transparent way so that developers *just* have to be reasonably careful with the strings they build (and we can provide guidance for the strings, or even correct them if necessary).

My very uninformed idea is that we need an independent function that handles the preferred language check and the catalog parsing based on a key, and all the string displaying functions (message etc) would be redefined to call that function when a non default preferred langage (currently English) is detected.

The only "extra burden" for the developers would be to specify the file default language somewhere in their code. Currently it would be set to English. But we can imagine developers working only "locally" and using native strings, and when l10n versions are provided (for ex from Japanese to English) the display would also provide the English for English readers, for ex.

Jean-Christophe

Re: Before l10n, better practices for (message) ?

[Etienne Prud’homme]

Eli Zaretskii <eliz@gnu.org> writes:

> One problem with this approach is that the programmer who writes the
> original code and provides the messages in English will have to
> manually create the English message catalog.

Reading the ‘message’ documentation, I realized that the function
already has the ability to use a custom face by providing the ‘face’
text property.

With that in mind, could we simply use a text property alike for an l18n
registery (whatever we choose).

One of the clear benefit would be that we won’t have to add function
calls (or flow control) and could let ‘message’ manage that for us.

Maybe the registery could cooperate with gettext, maybe it could be a
special type like face is.

What do you think?

--
Etienne Prud’homme

Re: Before l10n, better practices for (message) ?

[Eli Zaretskii]

> From: Etienne Prud’homme <e.e.f.prudhomme@gmail.com>
> Cc: Jean-Christophe Helary <jean.christophe.helary@gmail.com>,  emacs-devel@gnu.org
> Date: Fri, 26 May 2017 14:54:31 -0400
> 
> Reading the ‘message’ documentation, I realized that the function
> already has the ability to use a custom face by providing the ‘face’
> text property.
> 
> With that in mind, could we simply use a text property alike for an l18n
> registery (whatever we choose).

What kind of property would you like to put on a message string?  What
would its value convey?

Re: Before l10n, better practices for (message) ?

[Etienne Prud’homme]

Eli Zaretskii <eliz@gnu.org> writes:

> What kind of property would you like to put on a message string?  What
> would its value convey?

As I said, that’s just an idea, but it could be a symbol that got an
entry either in a global registry or the local one (if we allow a per
file registry).

If the language is different than English and ‘message’ got the
specified text property, it will try to fetch the registry.

--
Etienne Prud’homme

Re: Before l10n, better practices for (message) ?

[Eli Zaretskii]

> From: Etienne Prud’homme <e.e.f.prudhomme@gmail.com>
> Cc: jean.christophe.helary@gmail.com,  emacs-devel@gnu.org
> Date: Fri, 26 May 2017 15:15:15 -0400
> 
> If the language is different than English and ‘message’ got the
> specified text property, it will try to fetch the registry.

Why not do this for every string that 'message' needs to display?

Re: Before l10n, better practices for (message) ?

[Etienne Prud’homme]

Eli Zaretskii <eliz@gnu.org> writes:

> Why not do this for every string that 'message' needs to display?

I’m not sure to understand what you refer to every string?

What I mean is something like this:

> (let ((str "Hello %s World!")
>       (arg "you"))
>   (put-text-property 0 (length str) 'l10n 'hello-world str)
>   (message str arg))

Where message would try to fetch the ‘hello-world’ entry from the
translations registry if the configured language is not English.

I might have been wrong that we’re not adding a function call, but it’s
still much simpler (I think) than using a library for that.

I’m thinking of a text-property like ‘face’, but we could call it ‘i10n’
(or ‘l18n’?) or anything more meaningful.

--
Etienne Prud’homme

Re: Before l10n, better practices for (message) ?

[Eli Zaretskii]

> From: Etienne Prud’homme <e.e.f.prudhomme@gmail.com>
> Cc: jean.christophe.helary@gmail.com,  emacs-devel@gnu.org
> Date: Fri, 26 May 2017 17:57:06 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Why not do this for every string that 'message' needs to display?
> 
> I’m not sure to understand what you refer to every string?

I mean that 'message' should try to look up in the translation
registry every string it gets as its first argument, regardless of
whether it has text properties or not.

> > (let ((str "Hello %s World!")
> >       (arg "you"))
> >   (put-text-property 0 (length str) 'l10n 'hello-world str)
> >   (message str arg))
> 
> Where message would try to fetch the ‘hello-world’ entry from the
> translations registry if the configured language is not English.

I'm asking why do we need to put the text properties at all.

Text properties on strings are a nuisance for long-living strings, as
some APIs remove them.

Re: Before l10n, better practices for (message) ?

[Jean-Christophe Helary]

[-- Attachment #1: Type: text/plain, Size: 428 bytes --]


> On May 27, 2017, at 3:54, Etienne Prud’homme <e.e.f.prudhomme@gmail.com> wrote:
> 
> Reading the ‘message’ documentation, I realized that the function
> already has the ability to use a custom face by providing the ‘face’
> text property.

Yes but from what I've seen in package/el, a lot of translatable texts are not displayed with "message". Some use "error", some use other mechanisms.

Jean-Christophe

[-- Attachment #2: Type: text/html, Size: 2790 bytes --]

Re: Before l10n, better practices for (message) ?

[Eli Zaretskii]

> From: Jean-Christophe Helary <jean.christophe.helary@gmail.com>
> Date: Sat, 27 May 2017 10:16:14 +0900
> 
> Yes but from what I've seen in package/el, a lot of translatable texts are not displayed with "message". Some
> use "error", some use other mechanisms.

Internally, they all boil down to a small set of C functions, which is
where we should make these changes.

Re: Before l10n, better practices for (message) ?

[Marcin Borkowski]

On 2017-05-23, at 04:55, Eli Zaretskii <eliz@gnu.org> wrote:

> Just replace it with 2 different spellings dispatched by the number.

I guess you are aware that 2 different spellings are not enough for
certain languages...?

In Polish:

0 pakietów
1 pakiet
2, 3, 4 pakiety
5, 6, ..., 21 pakietów
22, 23, 24 pakiety
etc.

Good luck with l10n.

(I do not want to discourage work on it.  But I'm afraid it's more
difficult than many people think...  OTOH, if a reasonable
infrastructure is introduced in Emacs, I might be tempted to provide
some translations for Polish.)

Best,

-- 
Marcin Borkowski

Re: Before l10n, better practices for (message) ?

[Eli Zaretskii]

> From: Marcin Borkowski <mbork@mbork.pl>
> Cc: Jean-Christophe Helary <jean.christophe.helary@gmail.com>, emacs-devel@gnu.org
> Date: Sat, 27 May 2017 06:08:24 +0200
> 
> On 2017-05-23, at 04:55, Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > Just replace it with 2 different spellings dispatched by the number.
> 
> I guess you are aware that 2 different spellings are not enough for
> certain languages...?

I guess you are unaware that I'm the team leader for one of the TR
translation teams? ;-)

Yes, I know very well that 2 spellings are not enough for some
languages.  However, the above was written with only 1 language in
sight -- English -- which we currently support, so it would be
ridiculous to provide a more complex dispatch as long as other
languages aren't supported.

> Good luck with l10n.
> 
> (I do not want to discourage work on it.  But I'm afraid it's more
> difficult than many people think...

This complexity is already accounted for in gettext, AFAIK.

Re: Before l10n, better practices for (message) ?

[Marcin Borkowski]

On 2017-05-27, at 09:49, Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Marcin Borkowski <mbork@mbork.pl>
>> Cc: Jean-Christophe Helary <jean.christophe.helary@gmail.com>, emacs-devel@gnu.org
>> Date: Sat, 27 May 2017 06:08:24 +0200
>>
>> On 2017-05-23, at 04:55, Eli Zaretskii <eliz@gnu.org> wrote:
>>
>> > Just replace it with 2 different spellings dispatched by the number.
>>
>> I guess you are aware that 2 different spellings are not enough for
>> certain languages...?
>
> I guess you are unaware that I'm the team leader for one of the TR
> translation teams? ;-)

I was.  Now I'm not. ;-)

But I wasn't ironical.  I really thought you were aware of that.

>> (I do not want to discourage work on it.  But I'm afraid it's more
>> difficult than many people think...
>
> This complexity is already accounted for in gettext, AFAIK.

OK, I'll take a look (out of sheer curiosity).

Best,

--
Marcin Borkowski

Re: Before l10n, better practices for (message) ?

[Göktuğ Kayaalp]

On 2017-05-27 06:08 +02, Marcin Borkowski <mbork@mbork.pl> wrote:
> On 2017-05-23, at 04:55, Eli Zaretskii <eliz@gnu.org> wrote:
>
>> Just replace it with 2 different spellings dispatched by the number.
>
> I guess you are aware that 2 different spellings are not enough for
> certain languages...?
>
> In Polish:
>
> 0 pakietów
> 1 pakiet
> 2, 3, 4 pakiety
> 5, 6, ..., 21 pakietów
> 22, 23, 24 pakiety
> etc.
>
> Good luck with l10n.
>
> (I do not want to discourage work on it.  But I'm afraid it's more
> difficult than many people think...  OTOH, if a reasonable
> infrastructure is introduced in Emacs, I might be tempted to provide
> some translations for Polish.)

Hi,

I don't know whether or not I'm supposed to post here, but just thought
that I'd add my two cents as a mostly-user:  I'd rather have all of
Emacs in English.  These are my reasons:

- Some packages will have translations, some won't, and thus some
  messages will be in my language, and some English, which will be
  confusing (and annoying).

- The manuals will have to be translated too.

- It will undermine online discussion, as people will start posting
  (error) messages in their locales, and others will have to decode
  them.  Also, using online documentation will become harder as now one
  will need to translate.

- Emacs probably has way more strings to be translated than most
  applications out there, which will hinder the translations and make it
  more error prone.  Also, the community is not large enough that the
  messages will always be up-to-date and correct.

I believe most users would just use Emacs in English even if the strings
were translated, as many have their LC_MESSAGES or entire OS in English:
It's just more convenient for a power user.  The Emacs community is
mostly anglophone already anyways, and I doubt having translations will
change that at all.  Also, given the size of our community and the
amount of work (translating Emacs is translating a programming language
runtime + a collection of applications), it may result to be too much of
a burden on it.

Nevertheless, if translations happen, I'll try to help with Turkish (my
mother tongue) translations.  And please excuse me if I weren't supposed
to post here as I'm not the most experienced user and have only a bunch
of patches in Emacs.

Best,
        gk.

Re: Before l10n, better practices for (message) ?

[Jean-Christophe Helary]

> On May 28, 2017, at 8:27, Göktuğ Kayaalp <self@gkayaalp.com> wrote:

Göktuğ,

Thank you for your mail. You raise very important questions, that are not really technical in nature (ie they are not solvable with technology only). But we need to solve legitimate technical issues before worrying.

What you describe is common to all l10n endeavor and I think very big projects (Debian for ex) have already shown that we can find solutions. As a translator who's participated to FOSS translations for the last 20 years (besides for paying my bills with non-FOSS/non-fun translations for about the same time), I am positive that a l10n effort will only bring good things to Emacs in general, even if it puts a slight burden on the development community. But if we find the right solutions, it won't be more effort than other projects have to handle.

> - Some packages will have translations, some won't, and thus some
>  messages will be in my language, and some English, which will be
>  confusing (and annoying).
> 
> - The manuals will have to be translated too.

Once the infrastructure exists, I believe Emacs has such a name recognition that creating (or getting support from already existing) l10n groups will be relatively easy, especially since existing groups don't work on Emacs simply because Emacs does not provide the proper mechanism for l10n.

> - It will undermine online discussion, as people will start posting
>  (error) messages in their locales, and others will have to decode
>  them.  Also, using online documentation will become harder as now one
>  will need to translate.

Just as with all the big l10n projects, we will have native communities that will handle the issues in native language. This works already. There are already Emacs experts in all languages so there is little to worry about. I think when Emacs gets i18n/l10n there will be a huge amount of enthusiasm in the *whole* FOSS world.

> - Emacs probably has way more strings to be translated than most
>  applications out there, which will hinder the translations and make it
>  more error prone.  Also, the community is not large enough that the
>  messages will always be up-to-date and correct.

Here again, the existing native language groups have the infrastructure to cope with that. I am not worried.

I have a po4a based estimate for the existing documentation: the Emacs/Elisp/Introduction manuals are about 900,000 words (180 man/day). The various modes that are included in the distribution are about 700,000 words (140 man/day). I estimate that the UI strings are at most 15% of the total written mode manuals (no way to get the strings from the code right now), counting all the redundancy between the manuals and the package inline documentation, so the strings themselves would be at about 100,000 (20 man/day).

With 10 translation volunteers, that represents 34 days of work. With only 1 day per week, that's about 10 months, let's say 1 year to account for coordination, checking etc. It really is not that much of a task. And once that is done, the amount of new strings is really low. Such volunteers do not need to be Emacs experts. They need to be Emacs users and be involved in their native language l10n effort. It will be easy for them to translate (I can tell since I am not an Emacs/Elisp expert but have little difficulty working on the French translation). Plus the translation effort will find a lot of issues in the documentation, so even the English community will benefit from it.

> I believe most users would just use Emacs in English even if the strings
> were translated, as many have their LC_MESSAGES or entire OS in English:
> It's just more convenient for a power user.

People who *easily* work in English are not the target of the l10n effort. The l10n effort will allow for the creation of native communities about Emacs/Elisp which do not exist at the moment (or are limited to power users).

> The Emacs community is
> mostly anglophone already anyways, and I doubt having translations will
> change that at all.

The Emacs development community will not be affected by the changes.

> Also, given the size of our community and the amount of work (translating Emacs is translating a programming language runtime + a collection of applications), it may result to be too much of a burden on it.

See above.

> Nevertheless, if translations happen, I'll try to help with Turkish (my
> mother tongue) translations.  And please excuse me if I weren't supposed
> to post here as I'm not the most experienced user and have only a bunch
> of patches in Emacs.

Thank you very much for your concern. I am not even a power user and I've contributed only a few lines to Emacs. If you are interested in the internationalization effort, please join the discussion and help. Other projects have shown that l10n efforts add a little overhead (considering the whole development effort) but also brings huge benefits related to FOSS adoption. Let's work on how to make that a reality!


Jean-Christophe 

Re: Before l10n, better practices for (message) ?

[Göktuğ Kayaalp]

Hi Jean-Christophe,

thanks a lot for the detailed response, I'm glad that I wasn't
unintentionally spamming the list.  Given your experience and your
confidence about the success of the effort, I'm convinced.  I'll follow
the discussion on the topic, and although I doubt I can be helpful with
the technical part, I'll definitely try to take part in translation once
that becomes the next task to do.  If anybody is making a directory of
potential translators: I'm native turkish speaker, and have near-native
english, B2-C1 in italian (studying italian philology).

All the best,

	gk.