Guix Wikidata module - next steps

Guix Wikidata module - next steps

[swedebugia]

[-- Attachment #1.1: Type: text/html, Size: 3427 bytes --]

[-- Attachment #1.2: Type: text/plain, Size: 2758 bytes --]

Hi

I started today writing a new module and would like to share my ideas with you.
Feel free to add your ideas  too!

I imagine it as an utils collection for a wd importer (correlate the two) and a wd exporter that can push e.g. version updates, synopsis and descriptions and whole new entries to wd.

Steps:

First thing to add is the procedures from guile-wikidata that enables us to get a label for a given Qid and the output of queries like the one below.

The second thing to add is fold procedures to guess the right qid according to the upstream names of guix packages.

The third thing to add is an importer that helps us add qid to our packages. 

Here we have a decision to make. Should we add them as properties or extend our record with a field? 
I.e.
...
(properties
 `((wikidata . "qid"))
Or
...
(wikidata "qid")

I prefer the latter because I find it more easily handled.

Upstreaming and FSD:

Personally I would like to see new properties on wd making it possible for them to hold all our synopses and descriptions.
Guix is quickly growing towards becoming the largest general repository of free software with these details.

Writing this I have not forgotten the FSD, but it is in my view not a good approach to the "problem" of providing and curating a free software only repository. For one thing it seems to lack proper APIs. Also it does not have the traction, manpower and programming features of wd or guix. It is obsolete.

I beleive we could generate a much better read-only FSD rather easily with guile from guix and wd and tell users some place to report errors and contribute corrections (preferably directly on wd).

This said, One thing we currently miss that FSD recently got is an importer based on the Mozilla API to import add-ons. 

I believe we will soon have 1000+ npm packages imported as well. I did a little comparison of FSD vs WD earlier and the result was that only a few of the most prominent npm packages are in either of the databases.

--------

Here comes a little statistics about wd:

The query below lists all software on wikidata.

Of the millions of software out there WD currently have only 14686 entries. 

Of these not all are correctly tagged correctly e.g. free software tagged instance of "software".

https://query.wikidata.org/#SELECT%20%3Fsoftware%20%3FsoftwareLabel%20WHERE%20%7B%0A%20%20SERVICE%20wikibase%3Alabel%20%7B%20bd%3AserviceParam%20wikibase%3Alanguage%20%22%5BAUTO_LANGUAGE%5D%2Cen%22.%20%7D%0A%20%20VALUES%20%28%3Fv%29%20%7B%0A%20%20%20%20%28wd%3AQ506883%29%0A%20%20%20%20%28wd%3AQ341%29%0A%20%20%20%20%28wd%3AQ7397%29%0A%20%20%7D%0A%20%20%3Fsoftware%20wdt%3AP31%20%3Fv.%0A%7D%0ALIMIT%2050000
-- 
Sent from my p≡p for Android.

[-- Attachment #2: pEpkey.asc --]
[-- Type: application/pgp-keys, Size: 3825 bytes --]

Re: Guix Wikidata module - next steps

[Ludovic Courtès]

Hello!

Interesting work!

swedebugia <swedebugia@riseup.net> skribis:

> First thing to add is the procedures from guile-wikidata that enables us to get a label for a given Qid and the output of queries like the one below.
>
> The second thing to add is fold procedures to guess the right qid according to the upstream names of guix packages.
>
> The third thing to add is an importer that helps us add qid to our packages. 
>
> Here we have a decision to make. Should we add them as properties or extend our record with a field? 
> I.e.
> ...
> (properties
>  `((wikidata . "qid"))
> Or
> ...
> (wikidata "qid")

Does Wikidata store Common Platform Enumeration (CPE) names for
software?  CPE is a attempts to provide standard names for software
packages, primarily so that one can unambiguously refer to a software
package in a CVE.

If Wikidata supported CPE, that’d be nice because in Guix we already
store (sometimes…) ‘cpe-name’ properties so that things like ‘guix lint
-c cve’ know which package to look for in the CVE database.

Ludo’.

Re: Guix Wikidata module - next steps

[swedebugia]

[-- Attachment #1.1: Type: text/html, Size: 1830 bytes --]

[-- Attachment #1.2: Type: text/plain, Size: 1402 bytes --]

"Ludovic Courtès" <ludo@gnu.org> skrev: (8 januari 2019 18:25:05 CET)
>Hello!
>
>Interesting work!
>
>swedebugia <swedebugia@riseup.net> skribis:
>
>> First thing to add is the procedures from guile-wikidata that enables
>us to get a label for a given Qid and the output of queries like the
>one below.
>>
>> The second thing to add is fold procedures to guess the right qid
>according to the upstream names of guix packages.
>>
>> The third thing to add is an importer that helps us add qid to our
>packages. 
>>
>> Here we have a decision to make. Should we add them as properties or
>extend our record with a field? 
>> I.e.
>> ...
>> (properties
>>  `((wikidata . "qid"))
>> Or
>> ...
>> (wikidata "qid")
>
>Does Wikidata store Common Platform Enumeration (CPE) names for
>software?  CPE is a attempts to provide standard names for software
>packages, primarily so that one can unambiguously refer to a software
>package in a CVE.
>
>If Wikidata supported CPE, that’d be nice because in Guix we already
>store (sometimes…) ‘cpe-name’ properties so that things like ‘guix lint
>-c cve’ know which package to look for in the CVE database.
>
>Ludo’.

No, not yet for the packages I looked at.
I will try proposing it as this would be really nice as a package-id which is really important when synchronizing.
-- 
Sent from my p≡p for Android.

[-- Attachment #2: pEpkey.asc --]
[-- Type: application/pgp-keys, Size: 3825 bytes --]