* More progress with the Guix Data Service @ 2019-05-17 7:56 Christopher Baines 2019-05-17 10:07 ` Amirouche 2019-05-20 15:20 ` Ludovic Courtès 0 siblings, 2 replies; 7+ messages in thread From: Christopher Baines @ 2019-05-17 7:56 UTC (permalink / raw) To: guix-devel [-- Attachment #1: Type: text/plain, Size: 3273 bytes --] Hey, In summary, I think the Guix Data Service might be getting useful enough that setting it up properly might be a good next step, and I'd be interested in what others think? A bit over a month ago, I sent out an update about one of the things I've been working on, something I've been calling the "Guix Data Service" [1]. 1: https://lists.gnu.org/archive/html/guix-devel/2019-04/msg00094.html Since then, I've made some more progress. There's a new statistics page [2]. I've got used to using Sqitch [3] to help manage the database schema, and I've added some basic tests. 2: https://prototype-guix-data-service.cbaines.net/statistic 3: http://guix.gnu.org/packages/sqitch-0.9999/ The error handling for loading new revisions is also more resilient now. As well as listening to the Guix Commits mailing list for emails about new revisions, more of the information in these emails is now stored, in particular, the time they were sent, and the branch the email applies to. This can be seen on the new Branches page [4]. 4: https://prototype-guix-data-service.cbaines.net/branches The content negotiation a little bit, at least in terms of the code, and JSON output support has been added to more pages. There's now a basic search function on the packages page [5], and the location, and the licenses for packages is now being stored (which can be seen on the page for a package, for example [6]). 5: https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages?search_query=Guile&field=version&field=synopsis&after_name=&limit_results=1000 6: https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/package/0ad/0.0.23b-alpha The location and license information is something I added specifically, as I noticed the Repology [7] service scraped [8] these from the Guix website. 7: https://repology.org/ 8: https://github.com/repology/repology/blob/master/repology/fetchers/fetchers/guix.py While the Guix Data Service started as something to enable better understanding patches in an automated way, I think there are more uses for it, and initially, it's probably better to focus on the simple ones. The Repology use case is pretty simple, I think ideally there would be some machine readable data about the current state of packages in Guix available over the internet, and Repology would be able to download that on a regular basis. The URL is a bit long, but I think that is now close to being possible with the Guix data service. I haven't got something working yet to easily access data for the latest revision, but for a particular revision, you can request a JSON file containing all the information I think Repology currently gets about all packages. For example: https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages.json?field=version&field=synopsis&field=description&field=home-page&field=location&field=licenses&limit_results=99999 This is just the software side of the problem though. If this was to be used by Repology, it would have to be a more permanent thing, similar to the Cuirass and Mumi services that are currently setup around Guix. Does anyone have any thoughts on this? Thanks, Chris [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 962 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: More progress with the Guix Data Service 2019-05-17 7:56 More progress with the Guix Data Service Christopher Baines @ 2019-05-17 10:07 ` Amirouche 2019-05-17 18:26 ` Christopher Baines 2019-05-20 15:20 ` Ludovic Courtès 1 sibling, 1 reply; 7+ messages in thread From: Amirouche @ 2019-05-17 10:07 UTC (permalink / raw) To: Christopher Baines; +Cc: guix-devel, Guix-devel On 2019-05-17 09:56, Christopher Baines wrote: > Hey, > > In summary, I think the Guix Data Service might be getting useful > enough > that setting it up properly might be a good next step, and I'd be > interested in what others think? Yeah. And it looks good :) > A bit over a month ago, I sent out an update about one of the things > I've been working on, something I've been calling the "Guix Data > Service" [1]. > > 1: https://lists.gnu.org/archive/html/guix-devel/2019-04/msg00094.html > > Since then, I've made some more progress. There's a new statistics page > [2]. I've got used to using Sqitch [3] to help manage the database > schema, and I've added some basic tests. statistic page is a 404. I am very interested to learn how you use sqitch. > > 2: https://prototype-guix-data-service.cbaines.net/statistic > 3: http://guix.gnu.org/packages/sqitch-0.9999/ > > The error handling for loading new revisions is also more resilient > now. > > As well as listening to the Guix Commits mailing list for emails about > new revisions, more of the information in these emails is now stored, > in > particular, the time they were sent, and the branch the email applies > to. This can be seen on the new Branches page [4]. > > 4: https://prototype-guix-data-service.cbaines.net/branches Here when I click on a commit, I would expect the message of the commit and the build status of the impacted softwares, e.g. https://prototype-guix-data-service.cbaines.net/revision/d59c90a5bfdd6b723bea939b8538c7c9b3c1b2a6 > > The content negotiation a little bit, at least in terms of the code, > and > JSON output support has been added to more pages. > > There's now a basic search function on the packages page [5], and the > location, and the licenses for packages is now being stored (which can > be seen on the page for a package, for example [6]). > > 5: > https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages?search_query=Guile&field=version&field=synopsis&after_name=&limit_results=1000 > 6: > https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/package/0ad/0.0.23b-alpha > > The location and license information is something I added specifically, > as I noticed the Repology [7] service scraped [8] these from the Guix > website. > > 7: https://repology.org/ > 8: > https://github.com/repology/repology/blob/master/repology/fetchers/fetchers/guix.py > > While the Guix Data Service started as something to enable better > understanding patches in an automated way, I think there are more uses > for it, and initially, it's probably better to focus on the simple > ones. It is not clear right now that it is related to patches, there is no patch anywhere to see. > The Repology use case is pretty simple, I think ideally there would be > some machine readable data about the current state of packages in Guix > available over the internet, and Repology would be able to download > that > on a regular basis. Repology is nice, but I would prefer wikidata support. > > The URL is a bit long, but I think that is now close to being possible > with the Guix data service. I haven't got something working yet to > easily access data for the latest revision, but for a particular > revision, you can request a JSON file containing all the information I > think Repology currently gets about all packages. For example: > > > https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages.json?field=version&field=synopsis&field=description&field=home-page&field=location&field=licenses&limit_results=99999 FWIW it is very slow. > Does anyone have any thoughts on this? It seems to me it would be useful to have that informations somewhere. Like you explain having the correct build information is a must have IMO along the logs. Where is the code? Thanks for sharing! ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: More progress with the Guix Data Service 2019-05-17 10:07 ` Amirouche @ 2019-05-17 18:26 ` Christopher Baines 2019-05-20 15:23 ` Ludovic Courtès 0 siblings, 1 reply; 7+ messages in thread From: Christopher Baines @ 2019-05-17 18:26 UTC (permalink / raw) To: Amirouche; +Cc: guix-devel, Guix-devel [-- Attachment #1: Type: text/plain, Size: 6478 bytes --] Amirouche <amirouche@hyper.dev> writes: > On 2019-05-17 09:56, Christopher Baines wrote: >> Hey, >> >> In summary, I think the Guix Data Service might be getting useful >> enough >> that setting it up properly might be a good next step, and I'd be >> interested in what others think? > > Yeah. And it looks good :) Thanks :) >> A bit over a month ago, I sent out an update about one of the things >> I've been working on, something I've been calling the "Guix Data >> Service" [1]. >> >> 1: https://lists.gnu.org/archive/html/guix-devel/2019-04/msg00094.html >> >> Since then, I've made some more progress. There's a new statistics page >> [2]. I've got used to using Sqitch [3] to help manage the database >> schema, and I've added some basic tests. > > statistic page is a 404. I am very interested to learn how you use > sqitch. Sorry, the link is wrong, it's actually: https://prototype-guix-data-service.cbaines.net/statistics As for Sqitch, all I've been doing so far is sqitch add ... to create a change, sqitch deploy ... to apply changes to a database and sqitch revert ... to revert changes to a database. It's just some structure and tooling that I don't have to write :) >> 2: https://prototype-guix-data-service.cbaines.net/statistic >> 3: http://guix.gnu.org/packages/sqitch-0.9999/ >> >> The error handling for loading new revisions is also more resilient >> now. >> >> As well as listening to the Guix Commits mailing list for emails about >> new revisions, more of the information in these emails is now >> stored, in >> particular, the time they were sent, and the branch the email applies >> to. This can be seen on the new Branches page [4]. >> >> 4: https://prototype-guix-data-service.cbaines.net/branches > > Here when I click on a commit, I would expect the message of the commit > and the build status of the impacted softwares, e.g. > > https://prototype-guix-data-service.cbaines.net/revision/d59c90a5bfdd6b723bea939b8538c7c9b3c1b2a6 I've just realised the table headers on the branches page are the wrong way around. Anyway, I've added some information about git repositories to the database so I could link to cgit for the location of a package, so I'll have a look at also putting a link to commits on that page. As for build status information, this is something I talked a little more about in my last email (link at the top), but in summary, I've done some initial work, but more needs doing to get the build status information in to the database. >> The content negotiation a little bit, at least in terms of the code, >> and >> JSON output support has been added to more pages. >> >> There's now a basic search function on the packages page [5], and the >> location, and the licenses for packages is now being stored (which can >> be seen on the page for a package, for example [6]). >> >> 5: >> https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages?search_query=Guile&field=version&field=synopsis&after_name=&limit_results=1000 >> 6: >> https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/package/0ad/0.0.23b-alpha >> >> The location and license information is something I added specifically, >> as I noticed the Repology [7] service scraped [8] these from the Guix >> website. >> >> 7: https://repology.org/ >> 8: >> https://github.com/repology/repology/blob/master/repology/fetchers/fetchers/guix.py >> >> While the Guix Data Service started as something to enable better >> understanding patches in an automated way, I think there are more uses >> for it, and initially, it's probably better to focus on the simple >> ones. > > It is not clear right now that it is related to patches, there is no > patch > anywhere to see. Indeed. What I've been using the Guix Data Service for is working out what a series of patches does. For example, on my test Patchwork instance, you can see this series of patches [1], and there's a corresponding page from the Guix Data Service showing what the effect of the patches is [2]. 1: https://patchwork.cbaines.net/project/guix-patches/list/?series=1048 2: https://prototype-guix-data-service.cbaines.net/compare?base_commit=4d46775b1d04f9edb6fd729cfcd4b745be74e03b&target_commit=b3cfde472e5b2e6a9489a1b6c437daa659a977a6 >> The Repology use case is pretty simple, I think ideally there would be >> some machine readable data about the current state of packages in Guix >> available over the internet, and Repology would be able to download >> that >> on a regular basis. > > Repology is nice, but I would prefer wikidata support. Interesting, I don't know much about wikidata, but taking Guile as an example, it looks like wikidata is tracking distributions that package Guile, along with the Free Software Directory [3] 3: https://www.wikidata.org/wiki/Q1486208#identifiers Linking to other data/representations of packages from the Guix Data Service has been something I've been thinking about. Especially things like the Free Software Directory. Maybe wikidata would be a good think to link to. I'd also be really interested what would be required to get wikidata linking to Guix. >> The URL is a bit long, but I think that is now close to being possible >> with the Guix data service. I haven't got something working yet to >> easily access data for the latest revision, but for a particular >> revision, you can request a JSON file containing all the information I >> think Repology currently gets about all packages. For example: >> >> >> https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages.json?field=version&field=synopsis&field=description&field=home-page&field=location&field=licenses&limit_results=99999 > > FWIW it is very slow. To be honest, I'm actually pretty happy with the speed! I was worried that NGinx was going to time out and that it wasn't even going to work. I think there's probably some optimisations I can do to the queries to make it faster, and I haven't added any caching anyware yet, and it should be possible to cache lots of things, which will help lots. >> Does anyone have any thoughts on this? > > It seems to me it would be useful to have that informations > somewhere. Like you > explain having the correct build information is a must have IMO along > the logs. > > Where is the code? In a git repository here: https://git.cbaines.net/guix/data-service/ Thanks for taking a look, Chris [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 962 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: More progress with the Guix Data Service 2019-05-17 18:26 ` Christopher Baines @ 2019-05-20 15:23 ` Ludovic Courtès 0 siblings, 0 replies; 7+ messages in thread From: Ludovic Courtès @ 2019-05-20 15:23 UTC (permalink / raw) To: Christopher Baines; +Cc: guix-devel, Guix-devel Christopher Baines <mail@cbaines.net> skribis: >>> >>> https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages.json?field=version&field=synopsis&field=description&field=home-page&field=location&field=licenses&limit_results=99999 >> >> FWIW it is very slow. This particular request is asking for a *lot* of data, and I guess IceCat takes a while to simply parse and pretty-print that data. > To be honest, I'm actually pretty happy with the speed! I was worried > that NGinx was going to time out and that it wasn't even going to work. I found everything to be snappy, kudos! Ludo’. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: More progress with the Guix Data Service 2019-05-17 7:56 More progress with the Guix Data Service Christopher Baines 2019-05-17 10:07 ` Amirouche @ 2019-05-20 15:20 ` Ludovic Courtès 2019-05-20 20:14 ` Christopher Baines 1 sibling, 1 reply; 7+ messages in thread From: Ludovic Courtès @ 2019-05-20 15:20 UTC (permalink / raw) To: Christopher Baines; +Cc: guix-devel Hi! Christopher Baines <mail@cbaines.net> skribis: > Since then, I've made some more progress. There's a new statistics page > [2]. I've got used to using Sqitch [3] to help manage the database > schema, and I've added some basic tests. The URL for [2] is actually: https://prototype-guix-data-service.cbaines.net/statistics (with an ‘s’.) See, I paid attention! ;-) > As well as listening to the Guix Commits mailing list for emails about > new revisions, more of the information in these emails is now stored, in > particular, the time they were sent, and the branch the email applies > to. This can be seen on the new Branches page [4]. > > 4: https://prototype-guix-data-service.cbaines.net/branches This is really nice. This information could also be gathered directly from the repo though, right? I would expect only patch submission info, and possibly commit notifications, to be grabbed from email, while the rest would be extracted from the repo, thereby hopefully limiting the risk of misinterpreting email. WDYT? > There's now a basic search function on the packages page [5], and the > location, and the licenses for packages is now being stored (which can > be seen on the page for a package, for example [6]). > > 5: https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages?search_query=Guile&field=version&field=synopsis&after_name=&limit_results=1000 > 6: https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/package/0ad/0.0.23b-alpha Nice! One thing that be great is a page similar to <https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/package/shepherd/0.6.1>, but keyed by package, where you get a list of the recent package versions (and/or derivations) and map them to specific commits. > The URL is a bit long, but I think that is now close to being possible > with the Guix data service. I haven't got something working yet to > easily access data for the latest revision, but for a particular > revision, you can request a JSON file containing all the information I > think Repology currently gets about all packages. For example: > > https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages.json?field=version&field=synopsis&field=description&field=home-page&field=location&field=licenses&limit_results=99999 Awesome. (I advise passing “limit_results=900” though, because the URL above gives a pretty big result. ;-)) > This is just the software side of the problem though. If this was to be > used by Repology, it would have to be a more permanent thing, similar to > the Cuirass and Mumi services that are currently setup around Guix. Does > anyone have any thoughts on this? I’d suggest having a Guix service for the whole thing, and making a branch in guix-maintenance.git such that bayfront (say) can run the service. Then we’ll have to reach consensus on guix-sysadmin as to which machine to use depending on the resources it needs, but if you have the config, I’d argue that we can happily run it on bayfront or perhaps berlin. And we can give you access to the machine so you can reconfigure once in a while. WDYT? Thanks, Ludo’. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: More progress with the Guix Data Service 2019-05-20 15:20 ` Ludovic Courtès @ 2019-05-20 20:14 ` Christopher Baines 2019-05-21 8:49 ` Ludovic Courtès 0 siblings, 1 reply; 7+ messages in thread From: Christopher Baines @ 2019-05-20 20:14 UTC (permalink / raw) To: Ludovic Courtès; +Cc: guix-devel [-- Attachment #1: Type: text/plain, Size: 5916 bytes --] Ludovic Courtès <ludo@gnu.org> writes: >> As well as listening to the Guix Commits mailing list for emails about >> new revisions, more of the information in these emails is now stored, in >> particular, the time they were sent, and the branch the email applies >> to. This can be seen on the new Branches page [4]. >> >> 4: https://prototype-guix-data-service.cbaines.net/branches > > This is really nice. > > This information could also be gathered directly from the repo though, > right? > > I would expect only patch submission info, and possibly commit > notifications, to be grabbed from email, while the rest would be > extracted from the repo, thereby hopefully limiting the risk of > misinterpreting email. WDYT? So, currently the branch name, commit hash and date are taken from the email. As far as I know, git branches are just pointers to commits, and don't have any date/time associated with them. The commit date, or author date in the commits could be stored and used, but I think these are less interesting, and often misleading. The author date is often quite different from the time a commit is pushed, and the commit date is often different by some amount as well. Currently, if you actually want to know what was the state of a particular branch in the Guix git repository on Savannah was, at a particular time, I think the most reliable way of checking would probably be to check the guix-commits mailing list. As the branch name, and commit hash both relate to the date, I don't see that much problem with storing them. One thing I've also been thinking about is loading in the guix-commits mailing list archives. That would backfill the branch information, which might be useful/interesting... I did consider trying to access the clone of the Git repository that's managed by the (guix inferiors) module, but I couldn't see an easy way to do it, and as above, I'm not sure the date/time information is as useful as what you can get from the mailing list. >> There's now a basic search function on the packages page [5], and the >> location, and the licenses for packages is now being stored (which can >> be seen on the page for a package, for example [6]). >> >> 5: https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages?search_query=Guile&field=version&field=synopsis&after_name=&limit_results=1000 >> 6: https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/package/0ad/0.0.23b-alpha > > Nice! > > One thing that be great is a page similar to > <https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/package/shepherd/0.6.1>, > but keyed by package, where you get a list of the recent package > versions (and/or derivations) and map them to specific commits. Interesting, yeah, were you thinking of filtering that data for a specific branch (like master or staging), or showing data for all branches? >> The URL is a bit long, but I think that is now close to being possible >> with the Guix data service. I haven't got something working yet to >> easily access data for the latest revision, but for a particular >> revision, you can request a JSON file containing all the information I >> think Repology currently gets about all packages. For example: >> >> https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages.json?field=version&field=synopsis&field=description&field=home-page&field=location&field=licenses&limit_results=99999 > > Awesome. (I advise passing “limit_results=900” though, because the URL > above gives a pretty big result. ;-)) Well, not that big? Icecat tells me it's 12MB. Also, I've recently added a "All results" checkbox/query parameter, so you no longer have to make up a large number. I wanted to make it possible to get all the data as a single file, as that could simplify processing it, but there's also some support for pagination. https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages.json?field=version&field=synopsis&field=description&field=home-page&field=location&field=licenses&all_results=on The all results option is especially important as I've now done some work on caching. That page should be served with a max-age of a day, it could probably be even longer as well, as the only thing that will change the contents is software changes. NGinx is also now caching responses, and you can see what it's doing by looking at the X-Cache-Status header in the response. >> This is just the software side of the problem though. If this was to be >> used by Repology, it would have to be a more permanent thing, similar to >> the Cuirass and Mumi services that are currently setup around Guix. Does >> anyone have any thoughts on this? > > I’d suggest having a Guix service for the whole thing, and making a > branch in guix-maintenance.git such that bayfront (say) can run the > service. > > Then we’ll have to reach consensus on guix-sysadmin as to which machine > to use depending on the resources it needs, but if you have the config, > I’d argue that we can happily run it on bayfront or perhaps berlin. And > we can give you access to the machine so you can reconfigure once in a > while. > > WDYT? That all sounds really good :D A package and service has been on my list of things to do, and I'll hopefully sort that out in the next few weeks. Currently I'm running it with Guix + support for isolated inferiors [1], but I think that's something that can be made optional in the Guix Data Service code, as initially I'd just be thinking about processing revisions in the Guix git repository. 1: https://issues.guix.info/issue/34638 Thanks, Chris [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 962 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: More progress with the Guix Data Service 2019-05-20 20:14 ` Christopher Baines @ 2019-05-21 8:49 ` Ludovic Courtès 0 siblings, 0 replies; 7+ messages in thread From: Ludovic Courtès @ 2019-05-21 8:49 UTC (permalink / raw) To: Christopher Baines; +Cc: guix-devel Hello! Christopher Baines <mail@cbaines.net> skribis: > Ludovic Courtès <ludo@gnu.org> writes: [...] >> I would expect only patch submission info, and possibly commit >> notifications, to be grabbed from email, while the rest would be >> extracted from the repo, thereby hopefully limiting the risk of >> misinterpreting email. WDYT? > > So, currently the branch name, commit hash and date are taken from the > email. As far as I know, git branches are just pointers to commits, and > don't have any date/time associated with them. The commit date, or > author date in the commits could be stored and used, but I think these > are less interesting, and often misleading. The author date is often > quite different from the time a commit is pushed, and the commit date is > often different by some amount as well. > > Currently, if you actually want to know what was the state of a > particular branch in the Guix git repository on Savannah was, at a > particular time, I think the most reliable way of checking would > probably be to check the guix-commits mailing list. Good point. Alternately, we could get that information first-hand if we were running the Git server ourselves (which I think we’ll have to do at some point to improve our workflow, for things like commit signature checks.) > As the branch name, and commit hash both relate to the date, I don't see > that much problem with storing them. > > One thing I've also been thinking about is loading in the guix-commits > mailing list archives. That would backfill the branch information, which > might be useful/interesting... > > I did consider trying to access the clone of the Git repository that's > managed by the (guix inferiors) module, but I couldn't see an easy way > to do it, and as above, I'm not sure the date/time information is as > useful as what you can get from the mailing list. OK, makes sense. >> One thing that be great is a page similar to >> <https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/package/shepherd/0.6.1>, >> but keyed by package, where you get a list of the recent package >> versions (and/or derivations) and map them to specific commits. > > Interesting, yeah, were you thinking of filtering that data for a > specific branch (like master or staging), or showing data for all > branches? As a user, I’d like to answer questions like “when was package X upgraded in Guix?”, “which commit should I pull to obtain that old version of the package?”, things like that. >>> https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages.json?field=version&field=synopsis&field=description&field=home-page&field=location&field=licenses&limit_results=99999 >> >> Awesome. (I advise passing “limit_results=900” though, because the URL >> above gives a pretty big result. ;-)) > > Well, not that big? Icecat tells me it's 12MB. Also, I've recently added > a "All results" checkbox/query parameter, so you no longer have to make > up a large number. I wanted to make it possible to get all the data as a > single file, as that could simplify processing it, but there's also some > support for pagination. > > https://prototype-guix-data-service.cbaines.net/revision/f52e83470b05b2473ea13feb2842a1330c316a00/packages.json?field=version&field=synopsis&field=description&field=home-page&field=location&field=licenses&all_results=on > > The all results option is especially important as I've now done some > work on caching. That page should be served with a max-age of a day, it > could probably be even longer as well, as the only thing that will > change the contents is software changes. NGinx is also now caching > responses, and you can see what it's doing by looking at the > X-Cache-Status header in the response. Sounds good! >> I’d suggest having a Guix service for the whole thing, and making a >> branch in guix-maintenance.git such that bayfront (say) can run the >> service. >> >> Then we’ll have to reach consensus on guix-sysadmin as to which machine >> to use depending on the resources it needs, but if you have the config, >> I’d argue that we can happily run it on bayfront or perhaps berlin. And >> we can give you access to the machine so you can reconfigure once in a >> while. >> >> WDYT? > > That all sounds really good :D > > A package and service has been on my list of things to do, and I'll > hopefully sort that out in the next few weeks. Awesome. > Currently I'm running it with Guix + support for isolated inferiors [1], > but I think that's something that can be made optional in the Guix Data > Service code, as initially I'd just be thinking about processing > revisions in the Guix git repository. OK. I need to take another look at the isolated inferior patches. Thanks, Ludo’. ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-05-21 8:49 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2019-05-17 7:56 More progress with the Guix Data Service Christopher Baines 2019-05-17 10:07 ` Amirouche 2019-05-17 18:26 ` Christopher Baines 2019-05-20 15:23 ` Ludovic Courtès 2019-05-20 15:20 ` Ludovic Courtès 2019-05-20 20:14 ` Christopher Baines 2019-05-21 8:49 ` Ludovic Courtès
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/guix.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.