From: Jelle Licht <jlicht@fsfe.org>
To: Catonano <catonano@gmail.com>
Cc: guix-devel <guix-devel@gnu.org>
Subject: Re: jquery 3.1.1
Date: Thu, 19 Jan 2017 22:07:57 +0100 [thread overview]
Message-ID: <CAPsKtfL_xaAkCsc5tngd-n-jV1v7s5PBK2x1T0FYyk7F5unC_w@mail.gmail.com> (raw)
In-Reply-To: <CAJ98PDxZXqtNANTXY1qa2uZQ8jFiW0R=HcnFJkaSnY7Q26Y5gQ@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 4851 bytes --]
Hello Catonano,
2017-01-19 21:48 GMT+01:00 Catonano <catonano@gmail.com>:
> I made a crawler and I let it loose on the jquery 3.1.1 dependencies on
> registry.npmjs.com
> It recursevely fetched the dependencies of jquery 3.1.1, then the
> dependencies of the dependencies, then the dependencies of the dependencies
> of the dependencies... and so on
>
> Until there were no more dependencies to fetch.
>
> It stored all such dependencies in a graph database made by the amazing
> amz3 for their project, Culturia. They wrote about such project several
> times on the guile-users mailing list.
>
> It took days to download. And it took me months to produce a working
> version of this code. Because I don't know Guile very well and maybe I'm
> not that smart overall.
>
> Anyway, now I have a COMPLETE graph of the dependencies of jquery 3.1.1
>
These pictures are very informative indeed. I will try to be brief, but I
quickly wanted to share
that I find your efforts (and results!) amazing.
>
> It's made of
> 47311 vertices and
> 324569 edges
>
> I made a graph of a subset (graphviz chocked a bit on this ;-) )
> It's here
> http://catonano.altervista.org/grafo.svg
>
> There are 448 "broken" packages. Probably I should explain how I
> classified packages as "broken". Maybe not in this email.
>
> Anyway, these broken packages pose a challenge to the mission of porting
> Jquery into Guix, in my opinion, and they should be considered with some
> attention.
> Here's the list
> http://catonano.altervista.org/broken_packages.txt
>
Something of note: a big chunk of the packages you classified as being
'broken' are
making me recall some (unpleasant) memories; From my own crawling
experiments,
which were not nearly as complete as this one, I also ran into a lot of the
same
show-stoppers. In a very real sense, resolving node dependencies quickly
devolves into
resolving dependencies for most of the popular build systems as well as
plugins like
broccoli-funnel. What I am trying to get at here is that fixing and
packaging these 448 packages
will likely contribute a lot towards making an npm-enriched guix possible.
>
>
> There are 1314 packages with NO dependencies that could be used as
> starting points in porting Jquery into Guix.
> Here's the list
> http://catonano.altervista.org/broken_packages.txt
>
These could probably make use of the npm importer I worked on earlier. Do
you make use of your own? Otherwase I'll get to
rebasing my version on the current guix master branch.
>
>
> If there's anyone interested, I can give you the data folder so you can
> try all the queries you want on these data without having to to run this
> thing for a bunch of hours
>
If possible, yes please. What would be the most convenient way you to share
this data?
>
> In the future, I'd like to run this thing on some other package and merge
> the graphs so I will be able to investigate which are the common
> fundamental dependencies for SEVERAL important packages in Nodejs.
>
> So if someone wants to dedicate time to porting Nodejs stuff in Guix they
> will be able to select most urgent packages to start from.
>
> The same could be said of broken packages taht affect several important
> packages.
>
> The porting of Nodejs in Guix cannot be done with brute strength. A data
> oriented approach can help, in my opinion.
>
> Indeed.
> The ideal would be to have something that, like bitcoin, coordinates a
> swarm in such a way that every node can contribute a tiniy bit of data to a
> common data structure, so all the nodes would have a complete copy of the
> database.
>
> Collecting a mantaining of datasets should be freed of the client server
> model too. Not only the social media.
>
> I have no idea what you are referring to. Could you please elaborate a bit
at a later point in time?
> But that's more than I can handle, anyway.
>
I am already thankful
>
> I'd like to talk about the stumbling blocks I run into to discuss Guile
> and my knowledge of it.
>
> For example, I can't use that thing in the autotools that processes
> configure.am files so I just forked amz3's project and added my files in
> there. As guests. Thanks amz3 !
>
> I'd also like to describe te screw ups in the format I put the data into.
> I realized my mistakes when hours of crunching had already been done.
>
> They can be migrated to a better format though
>
> If you don't mind, I will discuss these issue in the future, not now.
>
> The code is here
> https://gitlab.com/humanitiesNerd/Culturia
>
> One last fun fact: while I was watching the output flowing in my terminal,
> I saw a package called
>
> "broccoli-funnel"
>
> No, really. It's here
> https://registry.npmjs.org/broccoli-funnel
>
> Ok, that's all for now.
>
Thanks again for your efforts on this. I am looking forward to working with
your data.
Regards,
Jelle Licht
[-- Attachment #2: Type: text/html, Size: 7920 bytes --]
next prev parent reply other threads:[~2017-01-19 21:08 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-01-19 20:48 jquery 3.1.1 Catonano
2017-01-19 21:07 ` Jelle Licht [this message]
2017-01-19 22:44 ` Catonano
2017-01-19 22:47 ` Catonano
2017-01-19 22:51 ` Catonano
2017-01-21 19:12 ` Catonano
2017-01-21 19:22 ` Catonano
2017-01-21 19:25 ` Catonano
2017-01-21 21:41 ` Mike Gerwitz
2017-01-21 22:35 ` Catonano
2017-02-13 11:19 ` Catonano
2017-01-20 6:04 ` Mike Gerwitz
2017-01-20 21:14 ` Ludovic Courtès
2017-01-21 3:39 ` Mike Gerwitz
2017-01-20 21:23 ` Ludovic Courtès
2017-01-20 22:33 ` Catonano
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://guix.gnu.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAPsKtfL_xaAkCsc5tngd-n-jV1v7s5PBK2x1T0FYyk7F5unC_w@mail.gmail.com \
--to=jlicht@fsfe.org \
--cc=catonano@gmail.com \
--cc=guix-devel@gnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).