Builds of https://guix.gnu.org/{packages,sources}.json

Builds of https://guix.gnu.org/{packages,sources}.json

[Ludovic Courtès]

Hello Guix!

Until now, the files https://guix.gnu.org/{packages,sources}.json were
built as part of the web site’s build process, via Haunt.  Since these
files provide metadata about packages available in the tip of the Guix
repo, this required getting an inferior for that, and fiddling with it.

This was expensive and made the build process rather complicated (on
bayfront, which currently hosts the web site, missing substitutes would
often cause the web site to not build within the allocate 1h time slot;
this is why recent blog posts took hours to get there).

Thus I moved the former (apps packages builder) module of the web site
to a script in maintenance.git, and had it run as a periodic mcron job
populating /srv/package-metadata, with nginx serving these two files
from that directory:

  5664984 * hydra: web: Add mcron job to build /packages.json and /sources.json.
  318db3e * hydra: Add 'build-package-metadata.scm' script.

I then removed the corresponding bits in guix-artwork.git:

  fe11359 website: Remove references to the latest Guix in '.guix.scm'.
  f71ca12 website: Remove (apps packages builder).

I reconfigured both berlin and bayfront, so they’re now using this
method to build and publish these files.

Feedback welcome!

Ludo’.

Re: Builds of https://guix.gnu.org/{packages,sources}.json

[zimoun]

Hi,

On Wed, 04 Jan 2023 at 23:30, Ludovic Courtès <ludo@gnu.org> wrote:

> Thus I moved the former (apps packages builder) module of the web site
> to a script in maintenance.git, and had it run as a periodic mcron job
> populating /srv/package-metadata, with nginx serving these two files
> from that directory:
>
>   5664984 * hydra: web: Add mcron job to build /packages.json and /sources.json.
>   318db3e * hydra: Add 'build-package-metadata.scm' script.

Cool!  Thank you.

I still have a WIP about turning this into manifest and then build
sources.json via CI (Cuirass).  Well, I do not know if it would be
useful or better compared to this new Guix REPL script.


Cheers,
simon

Re: Builds of https://guix.gnu.org/{packages,sources}.json

[Ludovic Courtès]

Hi!

zimoun <zimon.toutoune@gmail.com> skribis:

> On Wed, 04 Jan 2023 at 23:30, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> Thus I moved the former (apps packages builder) module of the web site
>> to a script in maintenance.git, and had it run as a periodic mcron job
>> populating /srv/package-metadata, with nginx serving these two files
>> from that directory:
>>
>>   5664984 * hydra: web: Add mcron job to build /packages.json and /sources.json.
>>   318db3e * hydra: Add 'build-package-metadata.scm' script.
>
> Cool!  Thank you.
>
> I still have a WIP about turning this into manifest and then build
> sources.json via CI (Cuirass).  Well, I do not know if it would be
> useful or better compared to this new Guix REPL script.

I initially considered making it a manifest, as you had proposed a while
back, but all the “computation” (turning metadata into JSON) happens on
the “host side”, so I figured there’s no need for derivations and all.

Thanks,
Ludo’.