From mboxrd@z Thu Jan 1 00:00:00 1970 From: Catonano Subject: jquery 3.1.1 Date: Thu, 19 Jan 2017 21:48:44 +0100 Message-ID: Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=001a11444496ca8962054678a82d Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:59102) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cUJda-000461-SZ for guix-devel@gnu.org; Thu, 19 Jan 2017 15:48:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cUJdZ-0006W1-C7 for guix-devel@gnu.org; Thu, 19 Jan 2017 15:48:50 -0500 Received: from mail-wm0-x22f.google.com ([2a00:1450:400c:c09::22f]:36534) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cUJdZ-0006Uh-2A for guix-devel@gnu.org; Thu, 19 Jan 2017 15:48:49 -0500 Received: by mail-wm0-x22f.google.com with SMTP id c85so10623308wmi.1 for ; Thu, 19 Jan 2017 12:48:47 -0800 (PST) List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: guix-devel --001a11444496ca8962054678a82d Content-Type: text/plain; charset=UTF-8 I made a crawler and I let it loose on the jquery 3.1.1 dependencies on registry.npmjs.com It recursevely fetched the dependencies of jquery 3.1.1, then the dependencies of the dependencies, then the dependencies of the dependencies of the dependencies... and so on Until there were no more dependencies to fetch. It stored all such dependencies in a graph database made by the amazing amz3 for their project, Culturia. They wrote about such project several times on the guile-users mailing list. It took days to download. And it took me months to produce a working version of this code. Because I don't know Guile very well and maybe I'm not that smart overall. Anyway, now I have a COMPLETE graph of the dependencies of jquery 3.1.1 It's made of 47311 vertices and 324569 edges I made a graph of a subset (graphviz chocked a bit on this ;-) ) It's here http://catonano.altervista.org/grafo.svg There are 448 "broken" packages. Probably I should explain how I classified packages as "broken". Maybe not in this email. Anyway, these broken packages pose a challenge to the mission of porting Jquery into Guix, in my opinion, and they should be considered with some attention. Here's the list http://catonano.altervista.org/broken_packages.txt There are 1314 packages with NO dependencies that could be used as starting points in porting Jquery into Guix. Here's the list http://catonano.altervista.org/broken_packages.txt If there's anyone interested, I can give you the data folder so you can try all the queries you want on these data without having to to run this thing for a bunch of hours In the future, I'd like to run this thing on some other package and merge the graphs so I will be able to investigate which are the common fundamental dependencies for SEVERAL important packages in Nodejs. So if someone wants to dedicate time to porting Nodejs stuff in Guix they will be able to select most urgent packages to start from. The same could be said of broken packages taht affect several important packages. The porting of Nodejs in Guix cannot be done with brute strength. A data oriented approach can help, in my opinion. The ideal would be to have something that, like bitcoin, coordinates a swarm in such a way that every node can contribute a tiniy bit of data to a common data structure, so all the nodes would have a complete copy of the database. Collecting a mantaining of datasets should be freed of the client server model too. Not only the social media. But that's more than I can handle, anyway. I'd like to talk about the stumbling blocks I run into to discuss Guile and my knowledge of it. For example, I can't use that thing in the autotools that processes configure.am files so I just forked amz3's project and added my files in there. As guests. Thanks amz3 ! I'd also like to describe te screw ups in the format I put the data into. I realized my mistakes when hours of crunching had already been done. They can be migrated to a better format though If you don't mind, I will discuss these issue in the future, not now. The code is here https://gitlab.com/humanitiesNerd/Culturia One last fun fact: while I was watching the output flowing in my terminal, I saw a package called "broccoli-funnel" No, really. It's here https://registry.npmjs.org/broccoli-funnel Ok, that's all for now. --001a11444496ca8962054678a82d Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
I made a crawler and I let it loose on the jquery 3.1.1 dependencies= on registry.npmjs.com

It recursevely fetched the dependencies of jquery 3.1.1, then the depend= encies of the dependencies, then the dependencies of the dependencies of th= e dependencies... and so on

Until there were no more depe= ndencies to fetch.

It stored all such dependencies = in a graph database made by the amazing amz3 for their project, Culturia. T= hey wrote about such project several times on the guile-users mailing list.=

It took days to download. And it took me months to produce a = working version of this code. Because I don't know Guile very well and = maybe I'm not that smart overall.

Anyway, now I have a COM= PLETE graph of the dependencies of jquery 3.1.1

It's made = of
47311 vertices and
324569 edges

I made a = graph of a subset (graphviz chocked a bit on this ;-) )
It's h= ere
http://catonan= o.altervista.org/grafo.svg

There are 448 "broken"= ; packages. Probably I should explain how I classified packages as "br= oken". Maybe not in this email.

Anyway, these broken= packages pose a challenge to the mission of porting Jquery into Guix, in m= y opinion, and they should be considered with some attention.
Here's the list
http://catonano.altervista.org/broken_packages.txt

There are 1314 packages with NO dependencies that could be = used as starting points in porting Jquery into Guix.
Here's th= e list
ht= tp://catonano.altervista.org/broken_packages.txt

If there&= #39;s anyone interested, I can give you the data folder so you can try all = the queries you want on these data without having to to run this thing for = a bunch of hours

In the future, I'd like to run = this thing on some other package and merge the graphs so I will be able to = investigate which are the common fundamental dependencies for SEVERAL impor= tant packages in Nodejs.

So if someone wants to dedicate = time to porting Nodejs stuff in Guix they will be able to select most urgen= t packages to start from.

The same could be said of broke= n packages taht affect several important packages.

The porting of No= dejs in Guix cannot be done with brute strength. A data oriented approach c= an help, in my opinion.

The ideal would be to have something that, l= ike bitcoin, coordinates a swarm in such a way that every node can contribu= te a tiniy bit of data to a common data structure, so all the nodes would h= ave a complete copy of the database.

Collecting a mantain= ing of datasets should be freed of the client server model too. Not only th= e social media.

But that's more than = I can handle, anyway.

I'd like to talk about the stum= bling blocks I run into to discuss Guile and my knowledge of it.

For example, I can't use that thing in the autotools that proce= sses configure.am files so I just forke= d amz3's project and added my files in there. As guests. Thanks amz3 !<= br>

I'd also like to describe te screw ups in = the format I put the data into. I realized my mistakes when hours of crunch= ing had already been done.

They can be migrated to a bett= er format though

If you don't mind, I will discuss th= ese issue in the future, not now.

One last fun fact: while I was wa= tching the output flowing in my terminal, I saw a package called

&q= uot;broccoli-funnel"

No, really. It's here
https://registry.npmjs= .org/broccoli-funnel

Ok, that's all for now= .
--001a11444496ca8962054678a82d--