gnumaint changes

gnumaint changes

[Nils Gillmann]

Hi,

could someone verify (I have an old checkout) that guix/gnu-maintenance.scm still works?
gnupackages.txt is dead in gnu.org womb CVS.

Thanks

Re: gnumaint changes

[Ludovic Courtès]

Hello,

Nils Gillmann <ng0@n0.is> skribis:

> could someone verify (I have an old checkout) that guix/gnu-maintenance.scm still works?
> gnupackages.txt is dead in gnu.org womb CVS.

‘guix lint -c gnu-description’ works fine and these URLs are 200 for me:

  http://cvs.savannah.gnu.org/viewvc/*checkout*/womb/gnumaint/gnupackages.txt
  http://cvs.savannah.gnu.org/viewvc/*checkout*/womb/gnumaint/pkgblurbs.txt

Ludo’.

Re: gnumaint changes

[Nils Gillmann]

Ludovic Courtès transcribed 450 bytes:
> Hello,
> 
> Nils Gillmann <ng0@n0.is> skribis:
> 
> > could someone verify (I have an old checkout) that guix/gnu-maintenance.scm still works?
> > gnupackages.txt is dead in gnu.org womb CVS.
> 
> ‘guix lint -c gnu-description’ works fine and these URLs are 200 for me:
> 
>   http://cvs.savannah.gnu.org/viewvc/*checkout*/womb/gnumaint/gnupackages.txt
>   http://cvs.savannah.gnu.org/viewvc/*checkout*/womb/gnumaint/pkgblurbs.txt

Oh, this is literally '*checkout*' in the URL! without it, they are not present.

> 
> Ludo’.
> 

Re: gnumaint changes

[Mike Gerwitz]

[-- Attachment #1: Type: text/plain, Size: 1084 bytes --]

On Wed, Jun 27, 2018 at 08:23:45 +0000, Nils Gillmann wrote:
> could someone verify (I have an old checkout) that guix/gnu-maintenance.scm still works?
> gnupackages.txt is dead in gnu.org womb CVS.

I'll have to look at what guix/gnu-maintenance.scm does, but:

A couple years back, Brandon Invergo started migrating from
gnupackages.txt to a new recutils format (rec/gnupackages.rec).  This
unfortunately required us to maintain two separate files.

Rather than get rid of gnupackages.txt completely, I wrote a script last
week to generate it from rec/gnupackages.rec.  The formats are largely
the same---it's possible that you can use the recfile directly.
However, if you still need the old format, just run
`make gnupackages.txt`.

Lmk if that works for you.  In the future, if you have problems with
gnumaint in womb, please email maintainers@gnu.org; this message just
happened to catch my eye.

-- 
Mike Gerwitz
Free Software Hacker+Activist | GNU Maintainer & Volunteer
GPG: D6E9 B930 028A 6C38 F43B  2388 FEF6 3574 5E6F 6D05
https://mikegerwitz.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: gnumaint changes

[Mike Gerwitz]

[-- Attachment #1: Type: text/plain, Size: 1255 bytes --]

On Wed, Jun 27, 2018 at 21:40:19 -0400, Mike Gerwitz wrote:
> I'll have to look at what guix/gnu-maintenance.scm does, but:

[...]

> Rather than get rid of gnupackages.txt completely, I wrote a script last
> week to generate it from rec/gnupackages.rec.  The formats are largely
> the same---it's possible that you can use the recfile directly.
> However, if you still need the old format, just run
> `make gnupackages.txt`.

Ah, I see, it fetches that single file over HTTP.  Certainly running
`make` (or the underlying gawk script) is undesirable in that situation.

Can someone with more knowledge of what this is used for run a couple of
tests to see what is broken if you use rec/gnupackages.rec instead?
Worst case, I can commit gnupackages.txt until the script can be
updated, but I'd prefer to keep generated files out of the repository.

pkgblurbs.txt has also been replaced by rec/pkgblurbs.rec.

If anyone here also knows of any other external systems pulling from
womb, please lmk.  I wasn't aware that anyone outside of maintainers@
used that repo, tbh.

-- 
Mike Gerwitz
Free Software Hacker+Activist | GNU Maintainer & Volunteer
GPG: D6E9 B930 028A 6C38 F43B  2388 FEF6 3574 5E6F 6D05
https://mikegerwitz.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: gnumaint changes

[Ludovic Courtès]

Hello Mike,

Mike Gerwitz <mtg@gnu.org> skribis:

> On Wed, Jun 27, 2018 at 21:40:19 -0400, Mike Gerwitz wrote:
>> I'll have to look at what guix/gnu-maintenance.scm does, but:
>
> [...]
>
>> Rather than get rid of gnupackages.txt completely, I wrote a script last
>> week to generate it from rec/gnupackages.rec.  The formats are largely
>> the same---it's possible that you can use the recfile directly.
>> However, if you still need the old format, just run
>> `make gnupackages.txt`.
>
> Ah, I see, it fetches that single file over HTTP.  Certainly running
> `make` (or the underlying gawk script) is undesirable in that situation.
>
> Can someone with more knowledge of what this is used for run a couple of
> tests to see what is broken if you use rec/gnupackages.rec instead?
> Worst case, I can commit gnupackages.txt until the script can be
> updated, but I'd prefer to keep generated files out of the repository.
>
> pkgblurbs.txt has also been replaced by rec/pkgblurbs.rec.

Commit daf76c7cd54df428abc28d490747c7f83a844df0 changes
gnu-maintenance.scm to use the .rec files.  Thanks for the heads-up.

> If anyone here also knows of any other external systems pulling from
> womb, please lmk.  I wasn't aware that anyone outside of maintainers@
> used that repo, tbh.

It’s used by ‘guix import gnu’ and ‘guix lint -c gnu-descriptions’
currently.

It’d be nice if synopses and descriptions in the Womb could contain
Texinfo markup.

In fact, perhaps it’d make sense to reverse the roles, i.e., have the
Womb take (some of its) descriptions from Guix?

Ludo’.

Re: gnumaint changes

[Mike Gerwitz]

[-- Attachment #1: Type: text/plain, Size: 2300 bytes --]

On Wed, Jul 11, 2018 at 16:11:37 +0200, Ludovic Courtès wrote:
> Hello Mike,
>
> Mike Gerwitz <mtg@gnu.org> skribis:

[...]

>> pkgblurbs.txt has also been replaced by rec/pkgblurbs.rec.
>
> Commit daf76c7cd54df428abc28d490747c7f83a844df0 changes
> gnu-maintenance.scm to use the .rec files.  Thanks for the heads-up.

Thanks.  I was planning to submit a patch this weekend, but you beat me
to it.  Sorry for the trouble!  I'm glad to see that the change was
pretty simple.

>> If anyone here also knows of any other external systems pulling from
>> womb, please lmk.  I wasn't aware that anyone outside of maintainers@
>> used that repo, tbh.
>
> It’s used by ‘guix import gnu’ and ‘guix lint -c gnu-descriptions’
> currently.
>
> It’d be nice if synopses and descriptions in the Womb could contain
> Texinfo markup.
>
> In fact, perhaps it’d make sense to reverse the roles, i.e., have the
> Womb take (some of its) descriptions from Guix?

`blub' in pkgblurbs (which is what `official-description' uses) is
provided by package authors after they've been dubbed by rms.  That is
in turn used on gnu.org.  Consequently, I think it's best to have such
blurbs maintained independently of guix.

What sort of Texinfo markup are you looking for, and are we talking
about the same field?  What field does guix use for the synopsis?
Everything in rec/gnupackages.rec is handled by us at maintainers@, so
we can do whatever we want there.

Do you have a couple examples of what you think would be beneficial to
pull form Guix?  I'm certainly open to the idea where it makes sense;
there's no sense in us duplicating effort within GNU unnecessarily.

I'm also working on automating parts of our recordkeeping: in the next
few weeks, Womb will have up-to-date version information automatically
pulled from info-gnu release announcements; the FTP server; and a couple
websites where necessary, though I'll be manually committing it for the
first few months to verify that it is all working properly.  So Guix
might also be able to depend on rec/gnupackages.rec for checking for new
releases as well, since unfortunately GNU doesn't mandate the use of the
FTP server, or even info-gnu (so releases are all over the place).

-- 
Mike Gerwitz

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: gnumaint changes

[Ludovic Courtès]

Hello,

Mike Gerwitz <mtg@gnu.org> skribis:

>> It’d be nice if synopses and descriptions in the Womb could contain
>> Texinfo markup.
>>
>> In fact, perhaps it’d make sense to reverse the roles, i.e., have the
>> Womb take (some of its) descriptions from Guix?
>
> `blub' in pkgblurbs (which is what `official-description' uses) is
> provided by package authors after they've been dubbed by rms.  That is
> in turn used on gnu.org.  Consequently, I think it's best to have such
> blurbs maintained independently of guix.

I see, that makes sense.

> What sort of Texinfo markup are you looking for, and are we talking
> about the same field?  What field does guix use for the synopsis?
> Everything in rec/gnupackages.rec is handled by us at maintainers@, so
> we can do whatever we want there.

For packages we occasionally use Texinfo markup, typically ornaments
like @code or @itemize bullet lists.  Not every synopsis/description
needs it, but it’s nice to be able to use it.

> Do you have a couple examples of what you think would be beneficial to
> pull form Guix?  I'm certainly open to the idea where it makes sense;
> there's no sense in us duplicating effort within GNU unnecessarily.

I realize that Guix doesn’t have all GNU packages yet so in fact there’s
not so much to pull from at this point.  I was suspecting blurbs are
likely to be more up-to-date in Guix, but that’s very subjective, I
don’t know if this is the case.

> I'm also working on automating parts of our recordkeeping: in the next
> few weeks, Womb will have up-to-date version information automatically
> pulled from info-gnu release announcements; the FTP server; and a couple
> websites where necessary, though I'll be manually committing it for the
> first few months to verify that it is all working properly.  So Guix
> might also be able to depend on rec/gnupackages.rec for checking for new
> releases as well, since unfortunately GNU doesn't mandate the use of the
> FTP server, or even info-gnu (so releases are all over the place).

The (guix gnu-maintenance) modules are tools to retrieve the latest
version of a GNU package by traversing its ftp.gnu.org (or similar)
directory.  That’s something you might find useful.  Here’s an example:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(gnu packages gcc)
scheme@(guile-user)> ,use(guix upstream)
scheme@(guile-user)> (package-latest-release gcc (force %updaters))
$4 = #<<upstream-source> package: "gcc" version: "8.1.0" urls: ("mirror://gnu/gcc/gcc-8.1.0/gcc-8.1.0.tar.xz" "mirror://gnu/gcc/gcc-8.1.0/gcc-8.1.0.tar.gz") signature-urls: ("mirror://gnu/gcc/gcc-8.1.0/gcc-8.1.0.tar.xz.sig" "mirror://gnu/gcc/gcc-8.1.0/gcc-8.1.0.tar.gz.sig")>
--8<---------------cut here---------------end--------------->8---

Or simply:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(guix gnu-maintenance)
scheme@(guile-user)> (latest-release "emacs")
$5 = #<<upstream-source> package: "emacs" version: "26.1" urls: ("ftp://ftp.gnu.org/gnu/emacs/emacs-26.1.tar.xz" "ftp://ftp.gnu.org/gnu/emacs/emacs-26.1.tar.gz") signature-urls: ("ftp://ftp.gnu.org/gnu/emacs/emacs-26.1.tar.xz.sig" "ftp://ftp.gnu.org/gnu/emacs/emacs-26.1.tar.gz.sig")>
--8<---------------cut here---------------end--------------->8---

This relies primarily on <https://ftp.gnu.org/find.txt.gz>.

Packages not hosted on gnu.org are typically annotated with the download
URL such that the update-checking code does the right thing.

Thanks,
Ludo’.

Re: gnumaint changes

[Mike Gerwitz]

[-- Attachment #1: Type: text/plain, Size: 2844 bytes --]

On Thu, Jul 12, 2018 at 17:57:01 +0200, Ludovic Courtès wrote:
> Hello,
>
> Mike Gerwitz <mtg@gnu.org> skribis:

[...]

>> Do you have a couple examples of what you think would be beneficial to
>> pull form Guix?  I'm certainly open to the idea where it makes sense;
>> there's no sense in us duplicating effort within GNU unnecessarily.
>
> I realize that Guix doesn’t have all GNU packages yet so in fact there’s
> not so much to pull from at this point.  I was suspecting blurbs are
> likely to be more up-to-date in Guix, but that’s very subjective, I
> don’t know if this is the case.

It seems like the blurbs in Guix may be slightly different: in Womb they
are provided by the package author for use here:

  https://www.gnu.org/manual/blurbs.html

In Guix they may be augmented with additional information that the
Guix package author finds useful, and may deviate from what the GNU
package author provided.  Is that true?

It makes sense to me, though, that Guix and that page would be in
sync.  But if the intent is to have the blurbs be written by the package
authors, syncing them would mean that Guix would forefit the ability to
manage its own package descriptions.  I'm not sure if that's something
Guix would want to do.

I'm also unaware of how many GNU package maintainers even remember that
the blurbs page even exists.  So it's possible that such descriptions
could be updated.  It'd be worth maintainers@ occasionally asking
package maintainers to review our records.

>> I'm also working on automating parts of our recordkeeping: in the next
>> few weeks, Womb will have up-to-date version information automatically
>> pulled from info-gnu release announcements; the FTP server; and a couple
>> websites where necessary, though I'll be manually committing it for the
>> first few months to verify that it is all working properly.  So Guix
>> might also be able to depend on rec/gnupackages.rec for checking for new
>> releases as well, since unfortunately GNU doesn't mandate the use of the
>> FTP server, or even info-gnu (so releases are all over the place).
>
> The (guix gnu-maintenance) modules are tools to retrieve the latest
> version of a GNU package by traversing its ftp.gnu.org (or similar)
> directory.  That’s something you might find useful.  Here’s an example:

Thanks---I was going to reference Guix's implementation.

But do note that many GNU packages don't make use of GNU's FTP server,
so this doesn't work on its own as a comprehensive version check
tool for GNU software.  But if this hasn't been a practical problem for
Guix yet, then there's no need to change that.

-- 
Mike Gerwitz
Free Software Hacker+Activist | GNU Maintainer & Volunteer
GPG: D6E9 B930 028A 6C38 F43B  2388 FEF6 3574 5E6F 6D05
https://mikegerwitz.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: gnumaint changes

[Nils Gillmann]

Mike Gerwitz transcribed 3.7K bytes:
> On Thu, Jul 12, 2018 at 17:57:01 +0200, Ludovic Courtès wrote:
> > Hello,
> >
> > Mike Gerwitz <mtg@gnu.org> skribis:
> 
> [...]
> 
> >> Do you have a couple examples of what you think would be beneficial to
> >> pull form Guix?  I'm certainly open to the idea where it makes sense;
> >> there's no sense in us duplicating effort within GNU unnecessarily.
> >
> > I realize that Guix doesn’t have all GNU packages yet so in fact there’s
> > not so much to pull from at this point.  I was suspecting blurbs are
> > likely to be more up-to-date in Guix, but that’s very subjective, I
> > don’t know if this is the case.
> 
> It seems like the blurbs in Guix may be slightly different: in Womb they
> are provided by the package author for use here:
> 
>   https://www.gnu.org/manual/blurbs.html
> 
> In Guix they may be augmented with additional information that the
> Guix package author finds useful, and may deviate from what the GNU
> package author provided.  Is that true?

Yes. And for that reason I would not like that they are picked from
Guix. The package authors should keep the autonomy to decide what's
right as a description.
In Guix we change the descriptions (blurbs) according to our needs.

> It makes sense to me, though, that Guix and that page would be in
> sync.  But if the intent is to have the blurbs be written by the package
> authors, syncing them would mean that Guix would forefit the ability to
> manage its own package descriptions.  I'm not sure if that's something
> Guix would want to do.
> 
> I'm also unaware of how many GNU package maintainers even remember that
> the blurbs page even exists.  So it's possible that such descriptions
> could be updated.  It'd be worth maintainers@ occasionally asking
> package maintainers to review our records.
> 
> >> I'm also working on automating parts of our recordkeeping: in the next
> >> few weeks, Womb will have up-to-date version information automatically
> >> pulled from info-gnu release announcements; the FTP server; and a couple
> >> websites where necessary, though I'll be manually committing it for the
> >> first few months to verify that it is all working properly.  So Guix
> >> might also be able to depend on rec/gnupackages.rec for checking for new
> >> releases as well, since unfortunately GNU doesn't mandate the use of the
> >> FTP server, or even info-gnu (so releases are all over the place).
> >
> > The (guix gnu-maintenance) modules are tools to retrieve the latest
> > version of a GNU package by traversing its ftp.gnu.org (or similar)
> > directory.  That’s something you might find useful.  Here’s an example:
> 
> Thanks---I was going to reference Guix's implementation.
> 
> But do note that many GNU packages don't make use of GNU's FTP server,
> so this doesn't work on its own as a comprehensive version check
> tool for GNU software.  But if this hasn't been a practical problem for
> Guix yet, then there's no need to change that.
> 
> -- 
> Mike Gerwitz
> Free Software Hacker+Activist | GNU Maintainer & Volunteer
> GPG: D6E9 B930 028A 6C38 F43B  2388 FEF6 3574 5E6F 6D05
> https://mikegerwitz.com

Re: gnumaint changes

[Ludovic Courtès]

Mike Gerwitz <mtg@gnu.org> skribis:

> On Thu, Jul 12, 2018 at 17:57:01 +0200, Ludovic Courtès wrote:
>> Hello,
>>
>> Mike Gerwitz <mtg@gnu.org> skribis:
>
> [...]
>
>>> Do you have a couple examples of what you think would be beneficial to
>>> pull form Guix?  I'm certainly open to the idea where it makes sense;
>>> there's no sense in us duplicating effort within GNU unnecessarily.
>>
>> I realize that Guix doesn’t have all GNU packages yet so in fact there’s
>> not so much to pull from at this point.  I was suspecting blurbs are
>> likely to be more up-to-date in Guix, but that’s very subjective, I
>> don’t know if this is the case.
>
> It seems like the blurbs in Guix may be slightly different: in Womb they
> are provided by the package author for use here:
>
>   https://www.gnu.org/manual/blurbs.html
>
> In Guix they may be augmented with additional information that the
> Guix package author finds useful, and may deviate from what the GNU
> package author provided.  Is that true?

In theory, yes, but ‘guix lint’ warns when this happens.

(IME package authors provide the initial blurb, but very few GNU people
actually know about the Womb and the fact there’s a blurb for their
package their.  Back when I looked more closely at this, it was often
Brandon or John who’d take care of providing blurbs.)

> It makes sense to me, though, that Guix and that page would be in
> sync.  But if the intent is to have the blurbs be written by the package
> authors, syncing them would mean that Guix would forefit the ability to
> manage its own package descriptions.  I'm not sure if that's something
> Guix would want to do.

Yes, we’ve been doing that from the start, but occasionally, people
would like to amend blurbs, and I point them to the Womb, but it’s a bit
cumbersome; no big deal, but not as fluid as could be.

> I'm also unaware of how many GNU package maintainers even remember that
> the blurbs page even exists.  So it's possible that such descriptions
> could be updated.  It'd be worth maintainers@ occasionally asking
> package maintainers to review our records.

Yeah.

>>> I'm also working on automating parts of our recordkeeping: in the next
>>> few weeks, Womb will have up-to-date version information automatically
>>> pulled from info-gnu release announcements; the FTP server; and a couple
>>> websites where necessary, though I'll be manually committing it for the
>>> first few months to verify that it is all working properly.  So Guix
>>> might also be able to depend on rec/gnupackages.rec for checking for new
>>> releases as well, since unfortunately GNU doesn't mandate the use of the
>>> FTP server, or even info-gnu (so releases are all over the place).
>>
>> The (guix gnu-maintenance) modules are tools to retrieve the latest
>> version of a GNU package by traversing its ftp.gnu.org (or similar)
>> directory.  That’s something you might find useful.  Here’s an example:
>
> Thanks---I was going to reference Guix's implementation.
>
> But do note that many GNU packages don't make use of GNU's FTP server,
> so this doesn't work on its own as a comprehensive version check
> tool for GNU software.  But if this hasn't been a practical problem for
> Guix yet, then there's no need to change that.

Like I said it uses a different method and URL for packages not on
gnu.org.

Thanks for your feedback,
Ludo’.