[bug#39547] [PATCH] website: Provide JSON sources list used by Software Heritage.

["Ludovic Courtès" <ludovic.courtes@inria.fr>]

Hi!

zimoun <zimon.toutoune@gmail.com> skribis:

> On Fri, 14 Feb 2020 at 11:20, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:
>> zimoun <zimon.toutoune@gmail.com> skribis:
>
>> >> Could you, in a first patch, move ‘origin->json’ out of
>> >> ‘packages-json-builder’, and in a second patch, add mirror-expansion
>> >> feature?
>> >
>> > Yes, I will try but I am a bit lost. There is:
>> >  - packages-json-builder that I did not modified
>> >  - sources-json-builder which the "adaptation" of the former to output
>> > to the sources.json format.
>> >
>> > Well, do you want a refactor of 'origin->json' shared by the 2
>> > "{sources,packages}-json-builder"?
>>
>> Yes, exactly.  No reason to have two copies of that code.
>
> The minor "issue" is the 'match' piece of 'origin->json':
>
>                `(("url" . ,(match uri
>                              ((? string? url) (vector url))
>                              ((urls ...) (list->vector urls))))))
>
>
> Because in the 'sources.json' it is always one unique string.
> However, in 'packages.json' it is always a vector of one or several
> elements. And in the previous patch, the job of 'resolve' was to
> uniquify these elements for 'sources.json'.

Ah, good point.  What about adding a keyword parameter to ‘origin->json’
that would tell whether to return a single URL or a list thereof?

> For example, see the package 'adns' in 'packages.json'
>
> {
>     "name": "adns",
>     "version": "1.5.1",
>     "source": {
>       "type": "url",
>       "url": [
>         "mirror://gnu/adns/adns-1.5.1.tar.gz",
>         "http://www.chiark.greenend.org.uk/~ian/adns/ftp/adns-1.5.1.tar.gz"
>       ]
>     },
>     "synopsis": "Asynchronous DNS client library and utilities",
>     "homepage": "https://www.gnu.org/software/adns/",
>     "location": "gnu/packages/adns.scm:32"
>   },
>
>
>
> Just to be on the same wavelength, does 'packages.json' stay as it is
> (vector for the source url field)?

I think ‘packages.json’ should still return a list of URLs.

I’ve also added a comment asking whether ‘sources.json’ could be changed
to contain a list of URLs:

  https://forge.softwareheritage.org/D2025#64037

> Or is 'packages.json' modified to record only one url for the source
> url field with mirror:// expanded?

Yeah, I think we can expand mirror URLs.

Thanks,
Ludo’.