Libgettextpo wrapper for Guile

Libgettextpo wrapper for Guile

[Miguel]

Hi to all!

I'm crossposting to these lists because I wasn't unable to find a
library to work with PO files in Guile after a script written in
Python (using polib) was posted in another mailing list.  It was really
useful, and I cannot thank enough its author.  However, I felt what
people from Galicia call "morriña" while I was downloading Python for
the first time in this machine, missing the parens of my preferred
language.  To help with the mild pain, I implemented a wrapper library
for Guile and a couple of higher level functions, tests and I'm
documenting everything.

I'd like to contribute it to GNU---a wrapper of a GNU library in the
official GNU extension language, I think it's sensible---, but I would
like to ask you where it would fit better:

	- In GNU gettext as part of gettext-tools?  It could cause
	  problems in the Guile bootstrap, as several tools from there
	  are needed when NLS is enabled.  Nevertheless, it seems to be
	  the better fit in terms of code locality and cross
	  maintenance.

	- In Guile as an ice-9 module?  As readline, it could be a
	  GPLv3+ library, but also available from scratch.  From my
	  point of view, it would be a great option in terms of
	  (zero) increment of dependencies and tight integration with
	  the language.

	- As an external library?

I'll choose the last one if the general response doesn't point clearly
in any other direction, but I wanted to ask first for advice and ideas,
as wise folks like you probably have some I haven't thought of.

Happy hacking!
Miguel

Re: Libgettextpo wrapper for Guile

[Bruno Haible]

Hi Miguel,

> I implemented a wrapper library
> for Guile and a couple of higher level functions, tests and I'm
> documenting everything.

This is great. Many thanks for this contribution!!

> I'd like to contribute it to GNU---a wrapper of a GNU library in the
> official GNU extension language, I think it's sensible---, but I would
> like to ask you where it would fit better:
> 
> 	- In GNU gettext as part of gettext-tools?  It could cause
> 	  problems in the Guile bootstrap, as several tools from there
> 	  are needed when NLS is enabled.  Nevertheless, it seems to be
> 	  the better fit in terms of code locality and cross
> 	  maintenance.

Very good question. I think there are two aspects:

  - Development aspects. Does your guile wrapper need more know-how
    from the guile world or from the gettext world? (*) Will it need
    regular changes as guile evolves? Surely it will need changes
    when gettext-po.h evolves, but gettext-po.h has been stable
    since 2010.
    (*) This is relevant, because when at some point in time you
    will not want to maintain it any more, will it be more easy to
    find a new maintainer for it among the camp of guile developers
    or among the gettext developers?

  - Distributions aspects. Distros will likely package it in a
    package different from gettext and different from guile, since
    both gettext and guile can be used without the wrapper library.
    Right?
    But distros are already used to create multiple binary packages
    from a single source package.

> 	- In Guile as an ice-9 module?  As readline, it could be a
> 	  GPLv3+ library, but also available from scratch.  From my
> 	  point of view, it would be a great option in terms of
> 	  (zero) increment of dependencies and tight integration with
> 	  the language.

I don't know what an ice-9 module is. I also don't know whether you
can define an ice-9 module outside of the guile package.

> 	- As an external library?

Would you want to have a project infrastructure with mailing list,
bug trackers, etc. for this wrapper library? I would think this is
overkill.

Bruno

Re: Libgettextpo wrapper for Guile

[Miguel]

Hi Bruno,

> Hi Miguel,
> 
> > I implemented a wrapper library
> > for Guile and a couple of higher level functions, tests and I'm
> > documenting everything.  
> 
> This is great. Many thanks for this contribution!!

You're welcome!  I hope it will be useful.  And it would not be
possible without your work, so actually thank you all.

> > I'd like to contribute it to GNU---a wrapper of a GNU library in the
> > official GNU extension language, I think it's sensible---, but I
> > would like to ask you where it would fit better:
> > 
> > 	- In GNU gettext as part of gettext-tools?  It could cause
> > 	  problems in the Guile bootstrap, as several tools from
> > there are needed when NLS is enabled.  Nevertheless, it seems to be
> > 	  the better fit in terms of code locality and cross
> > 	  maintenance.  
> 
> Very good question. I think there are two aspects:
> 
>   - Development aspects. Does your guile wrapper need more know-how
>     from the guile world or from the gettext world? (*) Will it need
>     regular changes as guile evolves? Surely it will need changes
>     when gettext-po.h evolves, but gettext-po.h has been stable
>     since 2010.
>     (*) This is relevant, because when at some point in time you
>     will not want to maintain it any more, will it be more easy to
>     find a new maintainer for it among the camp of guile developers
>     or among the gettext developers?

On one hand, this Guile library is tightly coupled with libgettextpo and
its integration in the gettext world could drive some of improvements
into the C library itself, such as an extension of the iteration
interface to a bidirectional or random access one.

On the other hand, it only has one source file written in C, it is
mainly intended to the Lisp world.  Gettext's source code is mainly
written in C, with the main exception of the example code and the java
runtime.

I think the choice should have more to do with Gettext's desire as a
project to extend its code base in Guile in the future, as the Guile
library could be the foundation for new tools, or to keep C as the main
code base, a reasonable position for such a key software and its amount
of codified knowledge.

It is also coupled to Guile as it uses (system ffi), even though
that interface is quite stable too.  The other sparsely used modules
are (ice-9 format) and (ice-9 regex).  Anything else is standard Scheme
or available as an SRFI, at least that's what I've tried.

I'm sure there are plenty of people in both worlds that are able to
hack any bug in my code in an hour, and much better I'd do it.

>   - Distributions aspects. Distros will likely package it in a
>     package different from gettext and different from guile, since
>     both gettext and guile can be used without the wrapper library.
>     Right?

Yes, for sure, breaking any of them would be a huge bug. 0:-)

>     But distros are already used to create multiple binary packages
>     from a single source package.

That shouldn't be a problem, I agree.

> > 	- In Guile as an ice-9 module?  As readline, it could be a
> > 	  GPLv3+ library, but also available from scratch.  From my
> > 	  point of view, it would be a great option in terms of
> > 	  (zero) increment of dependencies and tight integration
> > with the language.  
> 
> I don't know what an ice-9 module is.

Ice-nine is a fictional water crystal stable at room temperature from
Kurt Vonnegut's novel Cat's Cradle, that makes solid all water it comes
into contact with.  AFAIK, this reference is used in the Guile world for
the official modules that are pervasively useful, including the
bootstrapping of the language itself and libraries like readline.  I
don't remember where I've read it but I think I'm not making it up.

> I also don't know whether you can define an ice-9 module outside of
> the guile package.

You probably can, although Guile has several modules in other
namespaces too.  My wish is that it is so useful that it deserves that
name, although the module I'm using in the implementation right now is
(gettext po).  The key point for its inclusion in Guile would be its
availability right out-of-the-box, not really the name.

> > 	- As an external library?  
> 
> Would you want to have a project infrastructure with mailing list,
> bug trackers, etc. for this wrapper library? I would think this is
> overkill.

I was planning to use Savannah, I've prepared the form twice but didn't
hit send yet.  It might be an overkill, that's true, but I don't want
to use other easier and faster (and with extra lack of freedom)
repository server as GH, neither keep it to myself.  Also, with a small
code base there should not be so many maintenance tasks neither, and I
think I could face that up.

Even more, it would not be available on older versions than the first
Guile release with it, so an external project would be useful anyway as
does not require a version update of neither Gettext nor Guile to start
using it, at least as some kind of "backports"---I'm not aware of any
project of this kind.

Happy hacking,
Miguel

Re: Libgettextpo wrapper for Guile

[Bruno Haible]

Hi Miguel,

> an external project would be useful anyway as
> does not require a version update of neither Gettext nor Guile to start
> using it

A separate project also means an independent release cycle.

How are other Guile bindings organized?
- guile-cairo         separate project   https://www.nongnu.org/guile-cairo/
- guile-gnome         separate project   https://www.gnu.org/software/guile-gnome/
- guile-git           separate project   https://gitlab.com/guile-git/guile-git
- libgccjit           separate project   https://savannah.nongnu.org/projects/gccjit-guile/
- guile-gnutls        at gnutls          https://www.gnutls.org/manual/gnutls-guile.html
- zeromq              at zeromq          http://zeromq.org/bindings:guile-binding
- gmp                 part of guile      https://gmplib.org/manual/Language-Bindings.html

The majority seems to have chosen to be available as separate project.

> I think the choice should have more to do with Gettext's desire as a
> project to extend its code base in Guile in the future, as the Guile
> library could be the foundation for new tools, or to keep C as the main
> code base

There are no plans to use a different implementation language for
PO file manipulations. With the existing code base as a start, it
is not much harder to code new functionality in C than it would be
to code it in Python, guile, Java, or other languages.

Gettext needs a different implementation language in other areas -
such as extracting relevant information from HTML pages - and is
using POSIX sh for this purpose. It's a balance between language
features, portability, and ease of installation / minimization of
dependencies.

Bruno

Re: Libgettextpo wrapper for Guile

[amirouche]

Hello all!

On 2019-05-05 20:45, Bruno Haible wrote:
> Hi Miguel,
> 
>> an external project would be useful anyway as
>> does not require a version update of neither Gettext nor Guile to 
>> start
>> using it
> 
> A separate project also means an independent release cycle.
> 
> How are other Guile bindings organized?
> - guile-cairo         separate project   
> https://www.nongnu.org/guile-cairo/
> - guile-gnome         separate project
> https://www.gnu.org/software/guile-gnome/
> - guile-git           separate project   
> https://gitlab.com/guile-git/guile-git
> - libgccjit           separate project
> https://savannah.nongnu.org/projects/gccjit-guile/
> - guile-gnutls        at gnutls
> https://www.gnutls.org/manual/gnutls-guile.html
> - zeromq              at zeromq
> http://zeromq.org/bindings:guile-binding
> - gmp                 part of guile
> https://gmplib.org/manual/Language-Bindings.html
> 
> The majority seems to have chosen to be available as separate project.
> 

Miguel you decide to host, I can create a repository at source hut
so that you don't have to pay (the less evil, most free except you
must to host project yourself).

Also, I can review you code if you send it to me.


Best regards,


Amirouche ~ amz3

Re: Libgettextpo wrapper for Guile

[Miguel]

amirouche <amirouche@hypermove.net>:
> Hello all!
 
Hello, amirouche!

> Miguel you decide to host, I can create a repository at source hut
> so that you don't have to pay (the less evil, most free except you
> must to host project yourself).

After looking other options I decided that Savannah was the best option
available, mainly as a ethical/moral point of view without having to
provide the service myself.  Thank you for your offer anyway.

> Also, I can review you code if you send it to me.

You can find the latest code here:
https://savannah.nongnu.org/task/download.php?file_id=46883

Thank you very much,
Miguel

Re: Libgettextpo wrapper for Guile

[Miguel]

(Links at the end for readability.)

Bruno Haible <bruno-nWNVUoHt2MvYtjvyW6yDsg@public.gmane.org>:
> Hi Miguel,
 
Hi Bruno,

After some suggestions made by private email and being suggested in
the savannah admission process to integrate it in gettext[1], I gave a
second look into the topic, and check the pros and cons more
thoroughly.

> > an external project would be useful anyway as
> > does not require a version update of neither Gettext nor Guile to
> > start using it  
> 
> A separate project also means an independent release cycle.

This can be a problem until the library reaches an stable status, as
GNU gettext is a mature project and the library may need better advice
in some areas.  Nonetheless, the pace of evolution will be reduced
very much when some design issues are clarified[2] and a set of clear
Scheme idioms is found[3], and it will follow surely libgettextpo
evolution and not Guile.  I've tested it with Guile 2.0.14, Guile 2.2.4
and Guile 2.9.1, with all tests are working as expected.  It does not
work with Guile 1.8, as the ffi module at least was added with 2.0
release (scm_to_pointer, scm_t_pointer_finalizer, etc.) and I don't
know if it worths the effort porting it over that version, but it
could be a secondary effect of some changes in the implementation[4].

> How are other Guile bindings organized?
> - guile-cairo         separate project[5]

It seems that all language bindings are separate projects[6].

> - guile-gnome         separate project[7]

Gtk+ bindings are separate projects too, even though guile-gnome needs
some updates as it was Gtk+2 last time I played with it.

> - guile-git           separate project[8]

Git seems to have all its bindings as separate projects too.

> - libgccjit           separate project[9]

This one is quite new and I didn't know about it.  GNU Lightning is
there too and it is used for the JIT implementation in the future Guile
3.0[10].

> - zeromq              at zeromq[11]

This actually is a separate project if you look into the page.

> - guile-gnutls        at gnutls[12]
> - gmp                 part of guile[13]

AFAIK guile does not export bindings from any GMP library, but it uses
GMP internally to implement the number tower required by Scheme. 

I would like to add these examples too:

- gettext		part of guile[14]
- texinfo manipulation	part of guile[15]
- gdb			part of gdb[16]
- mailutils		part of mailutils[17]
- make			part of make[18]
- guile-opengl		separate project[19]
- guile-parted		separate-project[20]
- guile-gcrypt		separate project[21]

> The majority seems to have chosen to be available as separate project.

Most of them are available as separate projects, I agree.  Although I
think the integration in any of them (Guile or Gettext) benefits the
GNU Project itself[22].

> > I think the choice should have more to do with Gettext's desire as a
> > project to extend its code base in Guile in the future, as the Guile
> > library could be the foundation for new tools, or to keep C as the
> > main code base  
> 
> There are no plans to use a different implementation language for
> PO file manipulations. With the existing code base as a start, it
> is not much harder to code new functionality in C than it would be
> to code it in Python, guile, Java, or other languages.
>
> Gettext needs a different implementation language in other areas -
> such as extracting relevant information from HTML pages - and is
> using POSIX sh for this purpose. It's a balance between language
> features, portability, and ease of installation / minimization of
> dependencies.

My ideas were not clear at that point, so I must retract of what I
said; your point is more than valid.

In this case I could think the extension as a user of gettext's tools,
not only as the main code of the project.  I'm probably the only
user of the library at this moment, but I'm using it for translation
tasks such as translating the same fragment of text again and again.
There is a similar library for Python[23], and gettext already has the
code to do the same task.

What do you think?

Best regards,
Miguel

[1] https://savannah.nongnu.org/task/index.php?15273
[2] The main issue in my TODO list is clarifying the non-local exits.
I'm currently checking the code, after I have enough information I'll
open a separate thread probably about this.
[3] I'm not completely confortable with the /with-po-files/ approach,
but I think it is a good idea to perform file management through
declaration than imperatively, and I'd like to evolve a bit at least
this interface, as most of the functions are mostly calls to the C
library.
[4] The (gettext po internal-api) module could be removed and the C
source could export all the needed functions instead of relying on
(system ffi).  Would this be enough?  I thought about that regarding
speed/efficiency, but that can be another reason.
[5] https://www.nongnu.org/guile-cairo/
[6] https://cairographics.org/bindings/
[7] https://www.gnu.org/software/guile-gnome/
[8] https://gitlab.com/guile-git/guile-git
[9] https://savannah.nongnu.org/projects/gccjit-guile/
[10] http://git.savannah.gnu.org/cgit/guile.git/tree/NEWS
[11] http://zeromq.org/bindings:guile-binding
[12] https://www.gnutls.org/manual/gnutls-guile.html
[13] https://gmplib.org/manual/Language-Bindings.html
[14] https://www.gnu.org/software/guile/manual/html_node/Gettext-Support.html
[15] https://www.gnu.org/software/guile/manual/html_node/texinfo.html
[16] https://sourceware.org/gdb/current/onlinedocs/gdb/Guile.html
[17] https://mailutils.org/manual/html_node/guimb.html
[18] https://www.gnu.org/software/make/manual/html_node/Guile-Integration.html
[19] https://www.gnu.org/software/guile-opengl/
[20] https://gitlab.com/mothacehe/guile-parted
[21] https://notabug.org/cwebber/guile-gcrypt
[22] https://www.gnu.org/prep/standards/html_node/Source-Language.html
[23] https://pypi.org/project/polib/