Synchronizing descriptions with GSRC?

Synchronizing descriptions with GSRC?

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 221 bytes --]

Hello!

After some refactoring, commit c4ca941 extends our
<gnu-package-description> structures to incorporate a ‘description’
field whose value is taken from GSRC’s manifest [0].

If you apply this patch:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 1057 bytes --]

diff --git a/build-aux/sync-synopses.scm b/build-aux/sync-synopses.scm
index c1049d3..7d64533 100644
--- a/build-aux/sync-synopses.scm
+++ b/build-aux/sync-synopses.scm
@@ -58,5 +58,15 @@
                 (format (guix-warning-port)
                         "~a: ~a: proposed synopsis: ~s~%"
                         (location->string loc) (package-name package)
-                        upstream)))))
+                        upstream)))
+
+            (let ((upstream   (gnu-package-doc-description descriptor))
+                  (downstream (package-description package))
+                  (loc        (or (package-field-location package 'description)
+                                  (package-location package))))
+              (when (and upstream (not (string=? upstream downstream)))
+                (format (guix-warning-port)
+                        "~a: ~a: proposed description:~%    ~a~%"
+                        (location->string loc) (package-name package)
+                        (fill-paragraph upstream 79 5))))))
           gnus)


[-- Attachment #3: Type: text/plain, Size: 1081 bytes --]


then ‘make sync-synopses’ reports suggested descriptions (and if you do
it from Emacs with M-x compile, you can use C-x ` to jump to the
location in Guix, and thus easily compare the descriptions.)

After a quick glance over the differences, it’s not clear to me what
should be done.  Often, descriptions in GSRC are shorter that what we
have.  Sometimes they’re the same, or stripped version of those we have.
Sometimes they paraphrase those we have.

Descriptions in Guix are typically taken from the upstream web page or
README file, so I would intuitively feel like the upstream description
better characterizes the package.

I find some of the stripped descriptions better (Bison, for instance),
while others look too terse to me (GMP, MPFR, and MPC, for instance).

Perhaps we should have clearer criteria as for what to put in
descriptions, to start with.

Anyway, what do people think?

Would someone like to compare all these and pick-and-choose?

Thanks,
Ludo’.

[0] http://lists.gnu.org/archive/html/bug-guix/2013-04/msg00117.html

Re: Synchronizing descriptions with GSRC?

[Brandon Invergo]

[-- Attachment #1: Type: text/plain, Size: 1370 bytes --]

Hi Ludovic,

ludo@gnu.org (Ludovic Courtès) writes:

> After a quick glance over the differences, it’s not clear to me what
> should be done.  Often, descriptions in GSRC are shorter that what we
> have.  Sometimes they’re the same, or stripped version of those we have.
> Sometimes they paraphrase those we have.
>
> Descriptions in Guix are typically taken from the upstream web page or
> README file, so I would intuitively feel like the upstream description
> better characterizes the package.

Just a quick response to this.  When I was writing the descriptions for
GSRC, I wasn't sure whether I could copy descriptions wholesale from the
project websites due to copyright claims.  I doubt that anyone would get
upset but I was just playing it safe.  For example, they take a similar
precaution with the free software directory on fsf.org.  So, that's the
reason most of them are paraphrased.

Anyway, I'm happy to sync in the other direction and pull some
descriptions from Guix into GSRC. 

Perhaps it would be best to keep all canonical package descriptions,
short and long, in a single file under revision control somewhere, such
as in womb.  They would then be available for anyone who needs them,
should any need arise in the future and it would be easier for all
involved to stay in sync.  What do you think?

-brandon



[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: Synchronizing descriptions with GSRC?

[Ludovic Courtès]

Hi Brandon,

Brandon Invergo <brandon@invergo.net> skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> After a quick glance over the differences, it’s not clear to me what
>> should be done.  Often, descriptions in GSRC are shorter that what we
>> have.  Sometimes they’re the same, or stripped version of those we have.
>> Sometimes they paraphrase those we have.
>>
>> Descriptions in Guix are typically taken from the upstream web page or
>> README file, so I would intuitively feel like the upstream description
>> better characterizes the package.
>
> Just a quick response to this.  When I was writing the descriptions for
> GSRC, I wasn't sure whether I could copy descriptions wholesale from the
> project websites due to copyright claims.  I doubt that anyone would get
> upset but I was just playing it safe.  For example, they take a similar
> precaution with the free software directory on fsf.org.  So, that's the
> reason most of them are paraphrased.

Yes, but it’s unclear whether these are copyrightable, and since these
are GNU packages anyway, it’s safe to directly copy their descriptions,
I think.

> Anyway, I'm happy to sync in the other direction and pull some
> descriptions from Guix into GSRC. 
>
> Perhaps it would be best to keep all canonical package descriptions,
> short and long, in a single file under revision control somewhere, such
> as in womb.  They would then be available for anyone who needs them,
> should any need arise in the future and it would be easier for all
> involved to stay in sync.  What do you think?

Sounds good to me.

A related question is i18n: Guix uses gettext, and the plan is to use
the Translation Project for the translation of synopses/descriptions
too.  Should that be handled externally too?  If it is, we’d still need
to have a gettext catalog for our purposes.  How could that work?

Thanks,
Ludo’.

Re: Synchronizing descriptions with GSRC?

[Brandon Invergo]

[-- Attachment #1: Type: text/plain, Size: 2063 bytes --]

Ack, sorry about the late reply.  I've been overly busy these days.

ludo@gnu.org (Ludovic Courtès) writes:

> Yes, but it’s unclear whether these are copyrightable, and since these
> are GNU packages anyway, it’s safe to directly copy their descriptions,
> I think.

That sounds fine then.  I was probably being overly cautious.

>> Perhaps it would be best to keep all canonical package descriptions,
>> short and long, in a single file under revision control somewhere, such
>> as in womb.  They would then be available for anyone who needs them,
>> should any need arise in the future and it would be easier for all
>> involved to stay in sync.  What do you think?
>
> Sounds good to me.

Ok, sometime in the next week I'll put together a file.  It shouldn't be
difficult; I just need to do a query against the recfile package
manifest that I have.  Nevertheless, my PhD research is melting my
brain, so there might be a small delay in getting it done...

> A related question is i18n: Guix uses gettext, and the plan is to use
> the Translation Project for the translation of synopses/descriptions
> too.  Should that be handled externally too?  If it is, we’d still need
> to have a gettext catalog for our purposes.  How could that work?

Good question.  If we keep all the translation in the separate
repository, then the Guix translators will have to retrieve the package
description translations from that repository, though then the
translation would be done.  On the other hand, if only the English
descriptions were kept in the external repository, the translators could
just do their work directly on Guix.  I don't have plans at the moment
to have GSRC translated; as I understand it, gettext can't be used to
translate text in Makefiles (though I'd be glad to find out that I'm
wrong).  Therefore, since Guix is the only one using the translations,
perhaps it's ok for translators to just work within Guix and not on the
external repo..

-brandon

-- 
Brandon Invergo
http://brandon.invergo.net

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: Synchronizing descriptions with GSRC?

[Nikita Karetnikov]

[-- Attachment #1: Type: text/plain, Size: 212 bytes --]

> That sounds fine then.  I was probably being overly cautious.

I share your concerns.  Could you ask the FSF?  I understand that they
are very busy, but even a single mistake might spoil several years of
work.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Synchronizing descriptions with GSRC?

[Ludovic Courtès]

Nikita Karetnikov <nikita@karetnikov.org> skribis:

>> That sounds fine then.  I was probably being overly cautious.
>
> I share your concerns.  Could you ask the FSF?

Could you be more specific about what your concerns are?  AFAIK most GNU
web pages where we/I took summaries from at copyright FSF, to start with.

Of course I’m all in favor of clarifying any issues that could exist,
and I’m all ears for advice on that.

Now, saying that this might “spoil years of work” is clearly an
overstatement.  When I asked Karl about this some time ago, he wasn’t
concerned, even wondering whether a descriptive paragraph like these
could be subject to copyright.

Thanks,
Ludo’.

Re: Synchronizing descriptions with GSRC?

[Nikita Karetnikov]

[-- Attachment #1: Type: text/plain, Size: 565 bytes --]

> Could you be more specific about what your concerns are?

My main concern is that people say things like "I'm not sure, but let's
do it anyway."  Why don't you want to ask beforehand?

> AFAIK most GNU web pages where we/I took summaries from at copyright
> FSF, to start with.

Yep, I forgot that GSRC doesn't include non-GNU packages.

> Now, saying that this might “spoil years of work” is clearly an
> overstatement.

Of course, I understand.  But one shouldn't forget about the worst case
scenario.

If you are confident, then go ahead.

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Synchronizing descriptions with GSRC?

[Brandon Invergo]

[-- Attachment #1: Type: text/plain, Size: 1800 bytes --]

Hi Ludovic et al,

>> Perhaps it would be best to keep all canonical package descriptions,
>> short and long, in a single file under revision control somewhere, such
>> as in womb.  They would then be available for anyone who needs them,
>> should any need arise in the future and it would be easier for all
>> involved to stay in sync.  What do you think?
>
> Sounds good to me.

OK, I've finally gotten around to moving package descriptions (long and
short) to a file in Womb (gnumaint/pkgdescr.txt).  The file is
recutils-compatible.  

Currently, all of the long descriptions ("blurbs") are taken from GSRC.
Between Karl and I, we're currently in agreement that writing/adapting
new descriptions for all of the packages is better than straight copying
From websites/READMEs, mainly for keeping a certain level of consistency
(length, style, level of detail, etc.) between the descriptions.  It
also avoids any potential copyright problem, no matter how unlikely it
may be.  In addition to Guix and GSRC, these descriptions will also be
used to provide descriptions for featured packages on gnu.org, so they
can't be too long.

The file is fresh out of the oven, so there may be some changes after
Karl reviews it.  But anyway, it's there as a starting point.

> A related question is i18n: Guix uses gettext, and the plan is to use
> the Translation Project for the translation of synopses/descriptions
> too.  Should that be handled externally too?  If it is, we’d still need
> to have a gettext catalog for our purposes.  How could that work?

This is still something to work out.  Should the transltion be done in
Womb?  Can those be extracted easily for use in Guix?  I have no idea.
But again, at least we have a starting point now.

Cheers,
-brandon

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: Synchronizing descriptions with GSRC?

[Brandon Invergo]

[-- Attachment #1: Type: text/plain, Size: 1800 bytes --]

Hi Ludovic et al,

>> Perhaps it would be best to keep all canonical package descriptions,
>> short and long, in a single file under revision control somewhere, such
>> as in womb.  They would then be available for anyone who needs them,
>> should any need arise in the future and it would be easier for all
>> involved to stay in sync.  What do you think?
>
> Sounds good to me.

OK, I've finally gotten around to moving package descriptions (long and
short) to a file in Womb (gnumaint/pkgdescr.txt).  The file is
recutils-compatible.  

Currently, all of the long descriptions ("blurbs") are taken from GSRC.
Between Karl and I, we're currently in agreement that writing/adapting
new descriptions for all of the packages is better than straight copying
From websites/READMEs, mainly for keeping a certain level of consistency
(length, style, level of detail, etc.) between the descriptions.  It
also avoids any potential copyright problem, no matter how unlikely it
may be.  In addition to Guix and GSRC, these descriptions will also be
used to provide descriptions for featured packages on gnu.org, so they
can't be too long.

The file is fresh out of the oven, so there may be some changes after
Karl reviews it.  But anyway, it's there as a starting point.

> A related question is i18n: Guix uses gettext, and the plan is to use
> the Translation Project for the translation of synopses/descriptions
> too.  Should that be handled externally too?  If it is, we’d still need
> to have a gettext catalog for our purposes.  How could that work?

This is still something to work out.  Should the transltion be done in
Womb?  Can those be extracted easily for use in Guix?  I have no idea.
But again, at least we have a starting point now.

Cheers,
-brandon

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: Synchronizing descriptions with GSRC?

[Karl Berry]

    Should the transltion be done in Womb?  

I don't think that would be practical.

These descriptions (in HTML form) will end up on www.gnu.org.  Therefore
the web translators (not the TP translators) will be translating them.
I believe po format is involved and thus could conceivably be used
without too much trouble, but I've never actually delved into those
details.  Ineiev could tell us if it matters.

I expect that the web translators will cover one set of languages, the
TP translators will cover a second set, and the overlap will be a
classic Venn diagram :).  So perhaps it would be submitting to TP as
well, one way or another.  (I.e., from gsrc, or from guix if you decide
to use these blurbs too.)

karl

Translation of package descriptions and synopses

[Ludovic Courtès]

Karl Berry <karl@freefriends.org> skribis:

>     Should the transltion be done in Womb?  
>
> I don't think that would be practical.
>
> These descriptions (in HTML form) will end up on www.gnu.org.  Therefore
> the web translators (not the TP translators) will be translating them.
> I believe po format is involved and thus could conceivably be used
> without too much trouble, but I've never actually delved into those
> details.  Ineiev could tell us if it matters.

Currently Guix uses the Translation Project for l10n, and that includes
package descriptions (although I’ve purposefully restricted the set of
translatable files until a solution is agreed upon.)

The requirement for Guix is to be able to have native language support
when displaying/searching for package descriptions and synopses.  For
instance, ‘guix package --search’ must use and show descriptions in the
current locale’s language.

If translations were to be handled outside of Guix, we’d have to make
sure Guix can have .po files in the end and can just call ‘gettext’ on
descriptions and synopses.

WDYT?

Thanks,
Ludo’.

Re: Synchronizing descriptions with GSRC?

[Ludovic Courtès]

Hi, Brandon!

Brandon Invergo <brandon@gnu.org> skribis:

>>> Perhaps it would be best to keep all canonical package descriptions,
>>> short and long, in a single file under revision control somewhere, such
>>> as in womb.  They would then be available for anyone who needs them,
>>> should any need arise in the future and it would be easier for all
>>> involved to stay in sync.  What do you think?
>>
>> Sounds good to me.
>
> OK, I've finally gotten around to moving package descriptions (long and
> short) to a file in Womb (gnumaint/pkgdescr.txt).  The file is
> recutils-compatible.  
>
> Currently, all of the long descriptions ("blurbs") are taken from GSRC.
> Between Karl and I, we're currently in agreement that writing/adapting
> new descriptions for all of the packages is better than straight copying
> From websites/READMEs, mainly for keeping a certain level of consistency
> (length, style, level of detail, etc.) between the descriptions.  It
> also avoids any potential copyright problem, no matter how unlikely it
> may be.  In addition to Guix and GSRC, these descriptions will also be
> used to provide descriptions for featured packages on gnu.org, so they
> can't be too long.
>
> The file is fresh out of the oven, so there may be some changes after
> Karl reviews it.  But anyway, it's there as a starting point.

Excellent, thanks!  I’ve updated the synchronization tool in Guix to use
this file, and then I’ll update the descriptions (a bit of work for
today, but I’m glad that means less work in the longer term ;-)).

How complete is this list currently?  I see Guix is missing, for
instance.  :-)

Ludo’.

Re: Synchronizing descriptions with GSRC?

[Brandon Invergo]

[-- Attachment #1: Type: text/plain, Size: 2739 bytes --]

ludo@gnu.org (Ludovic Courtès) writes:

> Excellent, thanks!  I’ve updated the synchronization tool in Guix to use
> this file, and then I’ll update the descriptions (a bit of work for
> today, but I’m glad that means less work in the longer term ;-)).
>
> How complete is this list currently?  I see Guix is missing, for
> instance.  :-)

It's not 100%.  It is missing some that I haven't yet implemented in
GSRC as well as some that I (probably) won't implement in GSRC since it
wouldn't make sense (such as GSRC itself).  Finally, it doesn't include
any package that hasn't yet had a release (not including alpha
releases), which I think is reasonable enough.

Here's the list of released software that's missing from the file, with
comments about their status regarding writing descriptions:

TODO (high priority):
---
classpathx
gnufm
gnumed
gsrc
guile-gstreamer (from guile-gnome, though it doesn't build)
guix
health
hurd
social


TODO (separate files for sanity?):
---
GNOME packages (at least the main ones)
GNUstep packages (I actually have a lot of these in GSRC)


TODO (low priority):
---
xhippo (todo, but low priority since it requires GTK 1 to build)
dr-geo (gnu version is long out of date, though it just got a new
        maintainer, so wait and see)
phantom_home (way out of date, low priority)
+ any unreleased/alpha-only packages upon release


Undecided:
---
config (not sure...maybe not worth including)
gnujdoc (just documentation)
gnulib (not sure...probably worth including)
lispintro (just documentation)
pgccfd (just documentation)
womb (probably not appropriate to include)


Ignore:
---
bfd (packaged with binutils)
dc (packaged with bc)
gnat (packaged with gcc)
gnu-crypto (mostly merged with classpath)
gnucomm (prefer individual gnu telephony packages)
gnukart (only one ancient release, currently unmaintained, low priority)
gnustandards (not a package; redirects to gnu.org/prep)
gnuzilla (see icecat)
libiberty (packaged with binutils and gcc)
network (see gnunet, social, gnufm)
qexo (included in kawa)
trans-coord (see gnun)
vc-changelog (see vc-dwim)



BTW, you might find the GSRC TODO file to be handy in the development of
GSRC.  It contains an exhaustive list of all GNU packages past and
present, along with (very brief) notes on ones that fail to build or
haven't been released yet (it's how I was able to put together the above
lists pretty quickly).  It could save you a bit of info hunting or build
frustration.  It seems the Savannah bzr repo browser isn't working at
the moment, so you'll have to check out the repository to get it...

-brandon

-- 
Brandon Invergo
http://brandon.invergo.net

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: Synchronizing descriptions with GSRC?

[Ludovic Courtès]

ludo@gnu.org (Ludovic Courtès) skribis:

> Brandon Invergo <brandon@gnu.org> skribis:
>
>>>> Perhaps it would be best to keep all canonical package descriptions,
>>>> short and long, in a single file under revision control somewhere, such
>>>> as in womb.  They would then be available for anyone who needs them,
>>>> should any need arise in the future and it would be easier for all
>>>> involved to stay in sync.  What do you think?
>>>
>>> Sounds good to me.
>>
>> OK, I've finally gotten around to moving package descriptions (long and
>> short) to a file in Womb (gnumaint/pkgdescr.txt).  The file is
>> recutils-compatible.  
>>
>> Currently, all of the long descriptions ("blurbs") are taken from GSRC.
>> Between Karl and I, we're currently in agreement that writing/adapting
>> new descriptions for all of the packages is better than straight copying
>> From websites/READMEs, mainly for keeping a certain level of consistency
>> (length, style, level of detail, etc.) between the descriptions.  It
>> also avoids any potential copyright problem, no matter how unlikely it
>> may be.  In addition to Guix and GSRC, these descriptions will also be
>> used to provide descriptions for featured packages on gnu.org, so they
>> can't be too long.
>>
>> The file is fresh out of the oven, so there may be some changes after
>> Karl reviews it.  But anyway, it's there as a starting point.
>
> Excellent, thanks!  I’ve updated the synchronization tool in Guix to use
> this file, and then I’ll update the descriptions

Done:
<http://git.savannah.gnu.org/cgit/guix.git/commit/?id=a22dc0c49aed0babe16ef92ae24847b343b7eb02>.

Ludo’.

Re: Translation of package descriptions and synopses

[Karl Berry]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 634 bytes --]

    œôòùd have to make
    sure Guix can have .po files in the end 

I believe that is what you'll get with GNUN (the web translators).  What
I imagine is that you would be getting the .po from www instead of tp,
with no other difference in the process.

We'll have to talk with Ineiev about the details.  Before that I have to
put the pkgdescrs (which, BTW, I'm tempted to rename to "blurbs", which
seems somehow more descriptive than "descriptions") into www so there is
something tangible to talk about.  And before that I want to review the
text.  And before that I want to finish getting through today's email.

More later,
Karl