From mboxrd@z Thu Jan 1 00:00:00 1970 From: swedebugia Subject: Guix Wikidata module - next steps Date: Tue, 08 Jan 2019 12:29:09 +0100 Message-ID: <0BD0CA1F-AF8F-4584-969E-5C9A2D78E768@pretty.Easy.privacy> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="----8U7W6TILFT2R83QZRWP2CA53FJJJZE" Content-Transfer-Encoding: 7bit Return-path: Received: from eggs.gnu.org ([209.51.188.92]:59857) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ggpZL-0007Mk-0s for guix-devel@gnu.org; Tue, 08 Jan 2019 06:29:16 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ggpZJ-0003D4-1N for guix-devel@gnu.org; Tue, 08 Jan 2019 06:29:14 -0500 Received: from mx1.riseup.net ([198.252.153.129]:40400) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ggpZI-00039O-L2 for guix-devel@gnu.org; Tue, 08 Jan 2019 06:29:12 -0500 Received: from piha.riseup.net (piha-pn.riseup.net [10.0.1.163]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "*.riseup.net", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.riseup.net (Postfix) with ESMTPS id 799C71A0A04 for ; Tue, 8 Jan 2019 03:29:11 -0800 (PST) Received: from [127.0.0.1] (localhost [127.0.0.1]) by piha.riseup.net with ESMTPSA id 8F0E71C2CD8 for ; Tue, 8 Jan 2019 03:29:10 -0800 (PST) In-Reply-To: References: List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Guix-devel ------8U7W6TILFT2R83QZRWP2CA53FJJJZE Content-Type: multipart/alternative; boundary="----1BT1RXGWP82INQBHGZLTZ7FW703MK6" Content-Transfer-Encoding: 7bit ------1BT1RXGWP82INQBHGZLTZ7FW703MK6 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi

I started today writing a new module and would like to share my ideas with= you=2E
Feel free to add your ideas too!

I imagine it as an utils collection for a wd importer (correlate the two) = and a wd exporter that can push e=2Eg=2E version updates, synopsis and desc= riptions and whole new entries to wd=2E

Steps:

First thing to add is the procedures from guile-wikidata that enables us t= o get a label for a given Qid and the output of queries like the one below= =2E

The second thing to add is fold procedures to guess the right qid accordin= g to the upstream names of guix packages=2E

The third thing to add is an importer that helps us add qid to our package= s=2E

Here we have a decision to make=2E Should we add them as properties or ext= end our record with a field?
I=2Ee=2E
=2E=2E=2E
(properties
`((wikidata =2E "qid"))
Or
=2E=2E=2E
(wikidata "qid")

I prefer the latter because I find it more easily handled=2E

Upstreaming and FSD:

Personally I would like to see new properties on wd making it possible for= them to hold all our synopses and descriptions=2E
Guix is quickly growing towards becoming the largest general repository of= free software with these details=2E

Writing this I have not forgotten the FSD, but it is in my view not a good= approach to the "problem" of providing and curating a free softw= are only repository=2E For one thing it seems to lack proper APIs=2E Also i= t does not have the traction, manpower and programming features of wd or gu= ix=2E It is obsolete=2E

I beleive we could generate a much better read-only FSD rather easily with= guile from guix and wd and tell users some place to report errors and cont= ribute corrections (preferably directly on wd)=2E

This said, One thing we currently miss that FSD recently got is an importe= r based on the Mozilla API to import add-ons=2E

I believe we will soon have 1000+ npm packages imported as well=2E I did a= little comparison of FSD vs WD earlier and the result was that only a few = of the most prominent npm packages are in either of the databases=2E

--------

Here comes a little statistics about wd:

The query below lists all software on wikidata=2E

Of the millions of software out there WD currently have only 14686 entries= =2E

Of these not all are correctly tagged correctly e=2Eg=2E free software tag= ged instance of "software"=2E

https://query=2Ewikida= ta=2Eorg/#SELECT%20%3Fsoftware%20%3FsoftwareLabel%20WHERE%20%7B%0A%20%20SER= VICE%20wikibase%3Alabel%20%7B%20bd%3AserviceParam%20wikibase%3Alanguage%20%= 22%5BAUTO_LANGUAGE%5D%2Cen%22=2E%20%7D%0A%20%20VALUES%20%28%3Fv%29%20%7B%0A= %20%20%20%20%28wd%3AQ506883%29%0A%20%20%20%20%28wd%3AQ341%29%0A%20%20%20%20= %28wd%3AQ7397%29%0A%20%20%7D%0A%20%20%3Fsoftware%20wdt%3AP31%20%3Fv=2E%0A%7= D%0ALIMIT%2050000
--
Sent from my p=E2=89=A1p for Android=2E
------1BT1RXGWP82INQBHGZLTZ7FW703MK6 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi I started today writing a new module and would like to share my ideas with= you=2E Feel free to add your ideas too! I imagine it as an utils collection for a wd importer (correlate the two) = and a wd exporter that can push e=2Eg=2E version updates, synopsis and desc= riptions and whole new entries to wd=2E Steps: First thing to add is the procedures from guile-wikidata that enables us t= o get a label for a given Qid and the output of queries like the one below= =2E The second thing to add is fold procedures to guess the right qid accordin= g to the upstream names of guix packages=2E The third thing to add is an importer that helps us add qid to our package= s=2E=20 Here we have a decision to make=2E Should we add them as properties or ext= end our record with a field?=20 I=2Ee=2E =2E=2E=2E (properties `((wikidata =2E "qid")) Or =2E=2E=2E (wikidata "qid") I prefer the latter because I find it more easily handled=2E Upstreaming and FSD: Personally I would like to see new properties on wd making it possible for= them to hold all our synopses and descriptions=2E Guix is quickly growing towards becoming the largest general repository of= free software with these details=2E Writing this I have not forgotten the FSD, but it is in my view not a good= approach to the "problem" of providing and curating a free software only r= epository=2E For one thing it seems to lack proper APIs=2E Also it does not= have the traction, manpower and programming features of wd or guix=2E It i= s obsolete=2E I beleive we could generate a much better read-only FSD rather easily with= guile from guix and wd and tell users some place to report errors and cont= ribute corrections (preferably directly on wd)=2E This said, One thing we currently miss that FSD recently got is an importe= r based on the Mozilla API to import add-ons=2E=20 I believe we will soon have 1000+ npm packages imported as well=2E I did a= little comparison of FSD vs WD earlier and the result was that only a few = of the most prominent npm packages are in either of the databases=2E -------- Here comes a little statistics about wd: The query below lists all software on wikidata=2E Of the millions of software out there WD currently have only 14686 entries= =2E=20 Of these not all are correctly tagged correctly e=2Eg=2E free software tag= ged instance of "software"=2E https://query=2Ewikidata=2Eorg/#SELECT%20%3Fsoftware%20%3FsoftwareLabel%20= WHERE%20%7B%0A%20%20SERVICE%20wikibase%3Alabel%20%7B%20bd%3AserviceParam%20= wikibase%3Alanguage%20%22%5BAUTO_LANGUAGE%5D%2Cen%22=2E%20%7D%0A%20%20VALUE= S%20%28%3Fv%29%20%7B%0A%20%20%20%20%28wd%3AQ506883%29%0A%20%20%20%20%28wd%3= AQ341%29%0A%20%20%20%20%28wd%3AQ7397%29%0A%20%20%7D%0A%20%20%3Fsoftware%20w= dt%3AP31%20%3Fv=2E%0A%7D%0ALIMIT%2050000 --=20 Sent from my p=E2=89=A1p for Android=2E ------1BT1RXGWP82INQBHGZLTZ7FW703MK6-- ------8U7W6TILFT2R83QZRWP2CA53FJJJZE Content-Type: application/pgp-keys; name="pEpkey.asc" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="pEpkey.asc"; size=3825 LS0tLS1CRUdJTiBQR1AgUFVCTElDIEtFWSBCTE9DSy0tLS0tCgptUUlOQkZ3Y25XMEJFQURhNWti U0t0ZEd6ZzQ1SjZkQnRFS21lZFM2bEtEQkx1UmlNZTRTNHppTFVMaVV5RTlWCmVNeTA0TUUzbFN6 WHg4TlpSNjVxSzdENDRhVlRvV29iUlcxMEhGWi9YOWh5bFlrcnFzUk45cndieW5mOTFrcU0KcVI2 TkRxQjJKRHZTek9LRzM0U2dnTzVkVGh1RE1GdzNmT3Z1cGhtSnN5b2s3OGVYQlFqTkxyaXNvQnhT SEFyVgpzZjdIMlJIYlNmTmZaclMxaEtUUzRtbFBpNnJZS2ZFMUFvbm8wdUs5SnVVZHBTTyswODdV ejhRam5GVTE5c09BClJWTWtUTnpuZk12MHVzeUsxSzc5U2pjYzJjTzhkNDRUalJlTGkxcFlHMjBV OHhtQXBpRlJhRXhaOHlyR3c5RTkKYm9CcVo2TUZoNjhQWFNQa0ZjMWhTQ3BUV0xVZW1ZdEFrMkpE RHo4R1EwYWNRNU1aMlRJSzZJR0RCaEw4b0xCNwo2anZOMlBVYUN3VmFDYUNwS1JncU9Fd2dvdDBr MlFZRnZkODZUM0kwYVVNMDkzYVZ6aWdmUnYzaDlaMFhINytUCmZickhxalMrUlZTcjVKYytVeGox KzlqWEFQeEllVmRqMWpIOXE2ejJ2Z3UzSHRpaXlsQUp4SlFhUFFEZkhHQzEKNm9HODBwd3JoWWFj Q3JPT2QxUWhwZzhSOUlsaUQ4bU85WERTcHZoNGo2UFpXY1F6YWRYOFl5b3Z0UDNLS1B1dQpUM0h0 UDRmcXJhVmIzV2p2Si9ST0E3aTd5UHhGMTk4TExyKy9kMC9uamEvZHlWTjNHam1BellaL2dnZTdW aHp1CmZ2SFNGNFdLWWRFWFZWZzBhRjFVZ2FyTW1BaTUyNlhzTm1QRDU4cHdwc2hjL1F5SkV5Znlq ZnJuVndBUkFRQUIKdENKemQyVmtaV0oxWjJsaElEeHpkMlZrWldKMVoybGhRSEpwYzJWMWNDNXVa WFEraVFJK0JCTUJDQUFvQlFKYwpISjF0QWhzdkJRa0I0VE9BQndzSkNBY0RBZ0VHRlFnQ0NRb0xB eFlDQVFJZUFRSVhnQUFLQ1JBWWdFY1UxcWYyCjIwOGhEL29EcktQOTJIRDdKbVdWd0FKTkF6Qzh5 aW4wLzllVFJKb05XaXNjd2IrMG9hOVhzdXFwTktZNjdiWHgKamxUZmwxZSsvSmM2ZEoxTmR3cDVm Vk5FTmxhYmMxcnU0L3BQVk5PaXJOVjMrZFRYTU5ISEZkeHhRRWxENFhLYwowTmxLN0ZrUWxCdmNR ay94cmlQR1JuZEx4MmtZeVRFSStLcTRVdWNiZnVkT1YzaFJmOExaWVY3SzZEcTlWM2l2CmY4OWJV Q3lQRjhoZGN1czJBMnJvUU5NWUdLK2FPTGFWRzJYVnM1QS9DZFA3UEh2Yzk5TjY4a2xYbHZ5ZU1Y NGYKdG1FUFhOcnBsVElxQm5PbHdHZkN3QUY2OHIvVlVxcjNzek5OV3hDTk1sTmdORnA4ZmpKdjVx S3hnMkpzbGU4awppb0RYcUtJaWtISDBQTXJ2Nm5xNHEwbU96M2N5VkEwclpCS2E0WVhlTEhINDVm T1N5Yit3aStqSUJDaTlsQms2ClM1c2Y4dGVmd0hIaS9GNGZEZVI2NXdlSmhZTVpWRHlZS1NwKytx b2NFMHQ4NVFjOGdKS0pqQmRWQ3IwTTcrVWwKeGtybjBUQzRSelc4ck5ZQ2FiYWZxenVuRmplcmxG TDRXWnVNS2ZNNXFkUUxRUTFLZWJleTN2TlB3M2xUZ21VcgpLb2hmVXpKcmhmMkFNdXdQWjl6N1Vl WVZVSHNOS2Q2aTk2eGpkZHZFTkprOExlUDZHM3QrR3RSNWExWTRDdnk1CnVPcXpVczlUSnlXM3dj ejFYSkI0TS9UUkt6aDJaZWNHSUNSM2lPb1AxRHlZUlpuZ0ZSQ0FiOEQ1ZFAvWTdSM0YKYmNIeDR4 NHQ2a2Z3K0VVZituS2tKNzhPTHJOUVhiNlR1a3RiZy8wSWRCN2NNeWcwc2JrQ0RRUmNISjF0QVJB QQp1K2c3K2Y0VnJub3pwRGJzejNsRXAzZ1NtS3BmUFl2WXdTK0h3VTZEQm4vblVaNmVIUGRvcXcv RUE4bFl1OFkyCmhHY2ZGKzRxZUxmWEptWEg2SC9RVXdTNU1QTjZLcnJzUnVKU083OFR3UWRNUmtB RzJVbktrT3YrM2hRbk9LTG8KMmFySTVzNDZRbk0ya0JMN3lIM0t3N3ArUEFyT0JUOSs1alN2V0FM dUFvMWNpNCsybTR2cmhha05wR0NFcFhZNQpjcG9MYnlrQlNOMGwwTVFrYnZ0UHE1RmJpb1pRclpF MUVQQ2ZLdU1Ld25vL3pWcDBtbWNpZmluSEdENVVodENoCmt5VUoxaUxnMTdLZlo0eXNOMmVWOHFx YklqbWRnN00wRlVWMHhodGY3S1FpWFo2K1k1YWpBSi9BVHVYQUs3T2YKak1JdG50TzltSnBsQ0JH Y3VRbERtd1JpaHVQTjkrczVrak0rcTdTUENzNDFxREdNaXp2ZnVuSk1LblJMQmRNUApsS0NMNWhE QWhDSHRyTzhqSnJNS1lWcnkyemRkT1VhK1FmOFNwNFRjTVhaMjN4d1hqU3B5UkpDUU9ibG5DaUkv ClI0NUpsZERsYm42V29UZWlzUThJSTE1cnlibEtPeDVwdlk2VC9QQ1hxWW9nSENxSmJwVXNiMitU b29uZHU5Z0kKSENDSFFUZUhET3J3UGFheVZCSWcyNjhOUEZFalJza2hnOFg1a0FpZTZWYTNOajR6 elpmdk93ektpQjVDZktnbgpJaTgvbTN5QnZwZ2ZDK1RSQWMrdmhDYWEvejBTQU5UTXJjMzhLd0Vu ZmNJSG9Mek9JZzEvS0dWNlpnUzRmYVFECjlWeGswT0NNL3JscHZCT3ptNjJ5bzcxZWtzOUUrQTd1 L3pOOWdWZ0JHaFVBRVFFQUFZa0VSQVFZQVFnQUR3VUMKWEJ5ZGJRSWJMZ1VKQWVFemdBSXBDUkFZ Z0VjVTFxZjIyOEZkSUFRWkFRZ0FCZ1VDWEJ5ZGJRQUtDUkFCUEtQZwpwcWRNTFdweEQvNG0zLy8z MVAvS0JkZlJtTkk3UVV4akN5WVhibDBOYUpiT1NrUWMvRjk4TDlabk9FaGR2dU0zCmpoM1hrWW16 WWZmT21RMWV1TFppV1N2bEk5WWhSU1N1dmI0dVhtR1JVVGJZbVdMRVNaQmF3bElPNW1rT3h6cTQK cU5oK0UrTW4wZDFBSkw3cHVUd2RETDFPVXBML3hZUUFYVzF3bS9HOTcrOHQ2WVhkdTd4SG5uaHpE UmpIMUVDTQpPOXZ3YXp3V2tEUjdTc1Y2MHI1blhNcUw5dDhxZDJpRTRqNVFISXljTkxnSGsrSmIw QmxMclAvNHphcDFNNFNOCk0xMk4vSlZ0LzZ3ancrNk5KS0ZldE1OcG9Ic3pxOC80K0w2cEEzOUdP VkxYY3dmZzQ0T0JxRHRsUTlFVFF4VGMKV0JCSTg3ZVlNQU9Db3QyQlF2Rms5aENZRFJlQ2NFckxF YmsxV0RtWTNHcyswZDIrQnRSNmNEYnZHSWlaSDEvVQp3Mnc3eVJaMWgvenNYYWpva2tJM2ljTXhh VGVlVVJCSUNjOFBESWNVS0EzemVsdThuS25pUmZscUE3dGNtMHV2CktGSjVjS0JUa3N6RWdNb2x2 cUR6VWlhUkdyeko2R2grMTV1bHc2NTd4SnRTUkZ4Y3RpN3hENnlkUmt1VkFqSmgKVHRqTCtOZEtm VXl6ZFJCQlhoaFppTklsLzgxSkZ1b2ZJLzhLN1NvODRKb0h6V1JDbVVsdVladXMvdWFkalBkSApQ TXRySnZqKys1a01vYjV4OEg0eHIwdHA0Q0Y4TWo1VkRyYitiU0NsMFBOU2xaQVdUVkJDTm92QlZp MUVod0daCnRUMXNNU3dsblgrRjlxbERhdVBzVEhvb1NYRkVvU21lZmwrQ3VCZDZ0WG44eGFpZWhE NHNQMzJBRC80eGxkWWQKczhSM0haTTBIOGo2akFmRUgwbmY2NnoySXZ0cEE3eDdyVGVNd21IVFNK NjIvVUxiVXdJeUZYWkxWSTVSSVJmYgp4cVNBb3NIbklhVktlanBUMXgyN0t2bEREQ3ZUTnVnMUxR OEozSU05U1pqOVNTV005RjZFMjh2SGt1ejNEMGpYCmpmL0ZHMEQvelhlTFFleVE3cWZ3KzUxUzR3 YUZHT0dwNitOOGRJL2FuQTdaR0ZpMmFjd2FVUENBZ1hvc2pienYKMkRzdVgvZGFMS0FLSWxNMXYx akVob0ttblRPYXk3SGtTbFZCT29TWWRYeG55cUN3SkI2bE42dm45ZzBCQ2M0NQo2U1pObXdkZW1K bXdLQlJjV2phWStZY2c0UUg5WUxGdHZjYlZwVDNsOU1rVkxYWGxiM2xXZE5vbDliaTJSY2kvCkJs dGZVY1RnVHhTS0lQWTBpd2J0SXVRSU4rTVBPbGtVZ1lKY1JIRlFWamhZTW9iMk16MHpjRjRTazdW elRTQmYKbzFvTlUzcGRDTDVTNXkyM2ZqWGc1dkk4QmtLTmt6OVV1ZUlJSzE2WnZ4RjBaaUorYWtm SjZGbGpVc044S1ZyNgo1T2w5akdraXQ1b01ZZUpmeXptZ2tudEExaS9MVWVDZG94d0l2cUprZSs4 VmxWMURzNnFUTVc2ZUh3NEwvSTltCmhsZG5nK2Q3eWJQbXFtL0FRZFhXZVR3aFpiRG81MU1XQW84 SFJPK0pIMEdXRmVLOFdpRWJTbFB2MFBrWG83cDIKUTVvcUpkc0p5cDlwNjlYOGpvQjFxQm9MczJ1 RWJRWFM5MnJJVHFEVDYybVFZdTNQQ09kTzdBOGdqWUpDSENndgpZQWpWUjcrRWlJM0gyVXVWWjBT aVVwS0ZuZXhzdEthcm9Id1dTZz09Cj1iV1RRCi0tLS0tRU5EIFBHUCBQVUJMSUMgS0VZIEJMT0NL LS0tLS0K ------8U7W6TILFT2R83QZRWP2CA53FJJJZE--