jquery 3.1.1

jquery 3.1.1

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 3339 bytes --]

I made a crawler and I let it loose on the jquery 3.1.1 dependencies on
registry.npmjs.com

It recursevely fetched the dependencies of jquery 3.1.1, then the
dependencies of the dependencies, then the dependencies of the dependencies
of the dependencies... and so on

Until there were no more dependencies to fetch.

It stored all such dependencies in a graph database made by the amazing
amz3 for their project, Culturia. They wrote about such project several
times on the guile-users mailing list.

It took days to download. And it took me months to produce a working
version of this code. Because I don't know Guile very well and maybe I'm
not that smart overall.

Anyway, now I have a COMPLETE graph of the dependencies of jquery 3.1.1

It's made of
47311 vertices and
324569 edges

I made a graph of a subset (graphviz chocked a bit on this ;-) )
It's here
http://catonano.altervista.org/grafo.svg

There are 448 "broken" packages. Probably I should explain how I classified
packages as "broken". Maybe not in this email.

Anyway, these broken packages pose a challenge to the mission of porting
Jquery into Guix, in my opinion, and they should be considered with some
attention.
Here's the list
http://catonano.altervista.org/broken_packages.txt

There are 1314 packages with NO dependencies that could be used as starting
points in porting Jquery into Guix.
Here's the list
http://catonano.altervista.org/broken_packages.txt

If there's anyone interested, I can give you the data folder so you can try
all the queries you want on these data without having to to run this thing
for a bunch of hours

In the future, I'd like to run this thing on some other package and merge
the graphs so I will be able to investigate which are the common
fundamental dependencies for SEVERAL important packages in Nodejs.

So if someone wants to dedicate time to porting Nodejs stuff in Guix they
will be able to select most urgent packages to start from.

The same could be said of broken packages taht affect several important
packages.

The porting of Nodejs in Guix cannot be done with brute strength. A data
oriented approach can help, in my opinion.

The ideal would be to have something that, like bitcoin, coordinates a
swarm in such a way that every node can contribute a tiniy bit of data to a
common data structure, so all the nodes would have a complete copy of the
database.

Collecting a mantaining of datasets should be freed of the client server
model too. Not only the social media.

But that's more than I can handle, anyway.

I'd like to talk about the stumbling blocks I run into to discuss Guile and
my knowledge of it.

For example, I can't use that thing in the autotools that processes
configure.am files so I just forked amz3's project and added my files in
there. As guests. Thanks amz3 !

I'd also like to describe te screw ups in the format I put the data into. I
realized my mistakes when hours of crunching had already been done.

They can be migrated to a better format though

If you don't mind, I will discuss these issue in the future, not now.

The code is here
https://gitlab.com/humanitiesNerd/Culturia

One last fun fact: while I was watching the output flowing in my terminal,
I saw a package called

"broccoli-funnel"

No, really. It's here
https://registry.npmjs.org/broccoli-funnel

Ok, that's all for now.

[-- Attachment #2: Type: text/html, Size: 4428 bytes --]

Re: jquery 3.1.1

[Jelle Licht]

[-- Attachment #1: Type: text/plain, Size: 4851 bytes --]

Hello Catonano,

2017-01-19 21:48 GMT+01:00 Catonano <catonano@gmail.com>:

> I made a crawler and I let it loose on the jquery 3.1.1 dependencies on
> registry.npmjs.com
> It recursevely fetched the dependencies of jquery 3.1.1, then the
> dependencies of the dependencies, then the dependencies of the dependencies
> of the dependencies... and so on
>
> Until there were no more dependencies to fetch.
>
> It stored all such dependencies in a graph database made by the amazing
> amz3 for their project, Culturia. They wrote about such project several
> times on the guile-users mailing list.
>
> It took days to download. And it took me months to produce a working
> version of this code. Because I don't know Guile very well and maybe I'm
> not that smart overall.
>
> Anyway, now I have a COMPLETE graph of the dependencies of jquery 3.1.1
>

These pictures are very informative indeed. I will try to be brief, but I
quickly wanted to share
that I find your efforts (and results!) amazing.


>
> It's made of
> 47311 vertices and
> 324569 edges
>
> I made a graph of a subset (graphviz chocked a bit on this ;-) )
> It's here
> http://catonano.altervista.org/grafo.svg
>
> There are 448 "broken" packages. Probably I should explain how I
> classified packages as "broken". Maybe not in this email.
>
> Anyway, these broken packages pose a challenge to the mission of porting
> Jquery into Guix, in my opinion, and they should be considered with some
> attention.
> Here's the list
> http://catonano.altervista.org/broken_packages.txt
>

Something of note: a big chunk of the packages you classified as being
'broken' are
making me recall some (unpleasant) memories; From my own crawling
experiments,
which were not nearly as complete as this one, I also ran into a lot of the
same
show-stoppers. In a very real sense, resolving node dependencies quickly
devolves into
resolving dependencies for most of the popular build systems as well as
plugins like
broccoli-funnel. What I am trying to get at here is that fixing and
packaging these 448 packages
will likely contribute a lot towards making an npm-enriched guix possible.


>
>
> There are 1314 packages with NO dependencies that could be used as
> starting points in porting Jquery into Guix.
> Here's the list
> http://catonano.altervista.org/broken_packages.txt
>

These could probably make use of the npm importer I worked on earlier. Do
you make use of your own? Otherwase I'll get to
rebasing my version on the current guix master branch.


>
>
> If there's anyone interested, I can give you the data folder so you can
> try all the queries you want on these data without having to to run this
> thing for a bunch of hours
>

If possible, yes please. What would be the most convenient way you to share
this data?


>
> In the future, I'd like to run this thing on some other package and merge
> the graphs so I will be able to investigate which are the common
> fundamental dependencies for SEVERAL important packages in Nodejs.
>
> So if someone wants to dedicate time to porting Nodejs stuff in Guix they
> will be able to select most urgent packages to start from.
>
> The same could be said of broken packages taht affect several important
> packages.
>
> The porting of Nodejs in Guix cannot be done with brute strength. A data
> oriented approach can help, in my opinion.
>
> Indeed.


> The ideal would be to have something that, like bitcoin, coordinates a
> swarm in such a way that every node can contribute a tiniy bit of data to a
> common data structure, so all the nodes would have a complete copy of the
> database.
>
> Collecting a mantaining of datasets should be freed of the client server
> model too. Not only the social media.
>
> I have no idea what you are referring to. Could you please elaborate a bit
at a later point in time?


> But that's more than I can handle, anyway.
>

I am already thankful

>
> I'd like to talk about the stumbling blocks I run into to discuss Guile
> and my knowledge of it.
>
> For example, I can't use that thing in the autotools that processes
> configure.am files so I just forked amz3's project and added my files in
> there. As guests. Thanks amz3 !
>
> I'd also like to describe te screw ups in the format I put the data into.
> I realized my mistakes when hours of crunching had already been done.
>
> They can be migrated to a better format though
>
> If you don't mind, I will discuss these issue in the future, not now.
>
> The code is here
> https://gitlab.com/humanitiesNerd/Culturia
>
> One last fun fact: while I was watching the output flowing in my terminal,
> I saw a package called
>
> "broccoli-funnel"
>
> No, really. It's here
> https://registry.npmjs.org/broccoli-funnel
>
> Ok, that's all for now.
>

Thanks again for your efforts on this. I am looking forward to working with
your data.

Regards,
Jelle Licht

[-- Attachment #2: Type: text/html, Size: 7920 bytes --]

Re: jquery 3.1.1

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 6013 bytes --]

2017-01-19 22:07 GMT+01:00 Jelle Licht <jlicht@fsfe.org>:

> Hello Catonano,
>
>
>




> These pictures are very informative indeed. I will try to be brief, but I
> quickly wanted to share
> that I find your efforts (and results!) amazing.
>

Thanks :-)

>

Something of note: a big chunk of the packages you classified as being
> 'broken' are
> making me recall some (unpleasant) memories; From my own crawling
> experiments,
> which were not nearly as complete as this one, I also ran into a lot of
> the same
> show-stoppers. In a very real sense, resolving node dependencies quickly
> devolves into
> resolving dependencies for most of the popular build systems as well as
> plugins like
> broccoli-funnel. What I am trying to get at here is that fixing and
> packaging these 448 packages
> will likely contribute a lot towards making an npm-enriched guix possible.
>

Yes. They're important. Some of them require you to be logged in in order
to be downloaded. Or anyway the download fails with an error message in the
response (the body is empty in those cases).

These are those with "nope" in the value

Others return error messages. I'm affraid I overlapped some error
conditions, though :-/
The errors I recorded are not completely accurate.

The errors. But the fact that they can't be downloaded stands.

They are still a bit too many but I'm afraid thhey have to be explored
manually


>
>>
>> There are 1314 packages with NO dependencies that could be used as
>> starting points in porting Jquery into Guix.
>> Here's the list
>> http://catonano.altervista.org/broken_packages.txt
>>
>
> These could probably make use of the npm importer I worked on earlier. Do
> you make use of your own? Otherwase I'll get to
> rebasing my version on the current guix master branch.
>

I used some code that I wrote on purpose.
This is the repo
https://gitlab.com/humanitiesNerd/Culturia

The file you want to consider is "npmjs.scm", the function is
"populate-store" on line 178

The other files were there when I forked amz3's repository. They're theirs



>
>
>>
>>
>> If there's anyone interested, I can give you the data folder so you can
>> try all the queries you want on these data without having to to run this
>> thing for a bunch of hours
>>
>
> If possible, yes please. What would be the most convenient way you to
> share this data?
>

Yes, here it is
http://catonano.altervista.org/npmjsdata.tar.gz

decompress it somewhere, then read amz3's instructions about their Culturia
http://hyperdev.fr/notes/a-graph-based-movie-recommender-engine-using-guile-scheme.html
http://hyperdev.fr/notes/somewhat-relational-database-library-using-wiredtiger.html
https://github.com/amirouche/Culturia


>
>
>>
>> In the future, I'd like to run this thing on some other package and merge
>> the graphs so I will be able to investigate which are the common
>> fundamental dependencies for SEVERAL important packages in Nodejs.
>>
>> So if someone wants to dedicate time to porting Nodejs stuff in Guix they
>> will be able to select most urgent packages to start from.
>>
>> The same could be said of broken packages taht affect several important
>> packages.
>>
>> The porting of Nodejs in Guix cannot be done with brute strength. A data
>> oriented approach can help, in my opinion.
>>
>> Indeed.
>
>
>> The ideal would be to have something that, like bitcoin, coordinates a
>> swarm in such a way that every node can contribute a tiniy bit of data to a
>> common data structure, so all the nodes would have a complete copy of the
>> database.
>>
>> Collecting a mantaining of datasets should be freed of the client server
>> model too. Not only the social media.
>>
>> I have no idea what you are referring to. Could you please elaborate a
> bit at a later point in time?
>

Briefly, bitcoin keeps a ledger among a swarm of peers. They cooperate to
keep a common datastructure, mantaining consense about the order of the
transactions, in a distributed way.

In my idea, some software (based on gnunet ?) could be made such that every
node would download a piece of the graph and contribute it to te common
data structure.

Because how will we deal with the fact that you could merge this graph with
the one off CoffeeScript and in the same time I could merge it with the one
of some other package ?

And what about other people tha would like to collaborate ?

Coordination will be needed.

The most immediate way would be with a central server

But I'm a nobody in the middle of nowhere, I don't want to set up a server
and keep it on line

Something similar to bitcoin would allow us to coordinate without the
hurdles of a pesky server

And this argument could be generalized. An alternative to the client server
model for collecting and mantaining datasets in general should be
envisioned.

I see proects for federating the social networks. But what about federating
the backend machinery ?


I will elaborate in the future, though


> But that's more than I can handle, anyway.
>>
>
> I am already thankful
>
>>
>> I'd like to talk about the stumbling blocks I run into to discuss Guile
>> and my knowledge of it.
>>
>> For example, I can't use that thing in the autotools that processes
>> configure.am files so I just forked amz3's project and added my files in
>> there. As guests. Thanks amz3 !
>>
>> I'd also like to describe te screw ups in the format I put the data into.
>> I realized my mistakes when hours of crunching had already been done.
>>
>> They can be migrated to a better format though
>>
>> If you don't mind, I will discuss these issue in the future, not now.
>>
>> The code is here
>> https://gitlab.com/humanitiesNerd/Culturia
>>
>> One last fun fact: while I was watching the output flowing in my
>> terminal, I saw a package called
>>
>> "broccoli-funnel"
>>
>> No, really. It's here
>> https://registry.npmjs.org/broccoli-funnel
>>
>> Ok, that's all for now.
>>
>
> Thanks again for your efforts on this. I am looking forward to working
> with your data.
>
> Regards,
> Jelle Licht
>

[-- Attachment #2: Type: text/html, Size: 11254 bytes --]

Re: jquery 3.1.1

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 5270 bytes --]

2017-01-19 23:44 GMT+01:00 Catonano <catonano@gmail.com>:

> 2017-01-19 22:07 GMT+01:00 Jelle Licht <jlicht@fsfe.org>:
>
>> Hello Catonano,
>>
>>
>>
>
>
>
>
>> These pictures are very informative indeed. I will try to be brief, but I
>> quickly wanted to share
>> that I find your efforts (and results!) amazing.
>>
>
> Thanks :-)
>
>>
>
> Something of note: a big chunk of the packages you classified as being
>> 'broken' are
>> making me recall some (unpleasant) memories; From my own crawling
>> experiments,
>> which were not nearly as complete as this one, I also ran into a lot of
>> the same
>> show-stoppers. In a very real sense, resolving node dependencies quickly
>> devolves into
>> resolving dependencies for most of the popular build systems as well as
>> plugins like
>> broccoli-funnel. What I am trying to get at here is that fixing and
>> packaging these 448 packages
>> will likely contribute a lot towards making an npm-enriched guix possible.
>>
>
> Yes. They're important. Some of them require you to be logged in in order
> to be downloaded. Or anyway the download fails with an error message in the
> response (the body is empty in those cases).
>
> These are those with "nope" in the value
>
> Others return error messages. I'm affraid I overlapped some error
> conditions, though :-/
> The errors I recorded are not completely accurate.
>
> The errors. But the fact that they can't be downloaded stands.
>
> They are still a bit too many but I'm afraid thhey have to be explored
> manually
>
>
>>
>>>
>>> There are 1314 packages with NO dependencies that could be used as
>>> starting points in porting Jquery into Guix.
>>> Here's the list
>>> http://catonano.altervista.org/broken_packages.txt
>>>
>>
>> These could probably make use of the npm importer I worked on earlier. Do
>> you make use of your own? Otherwase I'll get to
>> rebasing my version on the current guix master branch.
>>
>
> I used some code that I wrote on purpose.
> This is the repo
> https://gitlab.com/humanitiesNerd/Culturia
>
> The file you want to consider is "npmjs.scm", the function is
> "populate-store" on line 178
>
> The other files were there when I forked amz3's repository. They're theirs
>
>
>
>>
>>
>>>
>>>
>>> If there's anyone interested, I can give you the data folder so you can
>>> try all the queries you want on these data without having to to run this
>>> thing for a bunch of hours
>>>
>>
>> If possible, yes please. What would be the most convenient way you to
>> share this data?
>>
>
> Yes, here it is
> http://catonano.altervista.org/npmjsdata.tar.gz
>
> decompress it somewhere, then read amz3's instructions about their Culturia
> http://hyperdev.fr/notes/a-graph-based-movie-recommender-
> engine-using-guile-scheme.html
> http://hyperdev.fr/notes/somewhat-relational-database-
> library-using-wiredtiger.html
> https://github.com/amirouche/Culturia
>
>
>>
>>
>>>
>>> In the future, I'd like to run this thing on some other package and
>>> merge the graphs so I will be able to investigate which are the common
>>> fundamental dependencies for SEVERAL important packages in Nodejs.
>>>
>>> So if someone wants to dedicate time to porting Nodejs stuff in Guix
>>> they will be able to select most urgent packages to start from.
>>>
>>> The same could be said of broken packages taht affect several important
>>> packages.
>>>
>>> The porting of Nodejs in Guix cannot be done with brute strength. A data
>>> oriented approach can help, in my opinion.
>>>
>>> Indeed.
>>
>>
>>> The ideal would be to have something that, like bitcoin, coordinates a
>>> swarm in such a way that every node can contribute a tiniy bit of data to a
>>> common data structure, so all the nodes would have a complete copy of the
>>> database.
>>>
>>> Collecting a mantaining of datasets should be freed of the client server
>>> model too. Not only the social media.
>>>
>>> I have no idea what you are referring to. Could you please elaborate a
>> bit at a later point in time?
>>
>
> Briefly, bitcoin keeps a ledger among a swarm of peers. They cooperate to
> keep a common datastructure, mantaining consense about the order of the
> transactions, in a distributed way.
>
> In my idea, some software (based on gnunet ?) could be made such that
> every node would download a piece of the graph and contribute it to te
> common data structure.
>
> Because how will we deal with the fact that you could merge this graph
> with the one off CoffeeScript and in the same time I could merge it with
> the one of some other package ?
>
> And what about other people tha would like to collaborate ?
>
> Coordination will be needed.
>
> The most immediate way would be with a central server
>
> But I'm a nobody in the middle of nowhere, I don't want to set up a server
> and keep it on line
>
> Something similar to bitcoin would allow us to coordinate without the
> hurdles of a pesky server
>
> And this argument could be generalized. An alternative to the client
> server model for collecting and mantaining datasets in general should be
> envisioned.
>
> I see proects for federating the social networks. But what about
> federating the backend machinery ?
>
>
> I will elaborate in the future, though
>


I mistakenly hit send. Sorry. But I had finished anyway ;-)

[-- Attachment #2: Type: text/html, Size: 9933 bytes --]

Re: jquery 3.1.1

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 402 bytes --]

One last note

I pasted the wrong url for the packages with no dependencies

The right one is this
http://catonano.altervista.org/no-deps.txt

I noticed now that this list contains ALSO the broken ones. They have no
dependencies, but they shold be excluded from this list

Because I screwed up the data format, the queries are not exactly
immediate, they need honing :-/

I will fix this in the future

[-- Attachment #2: Type: text/html, Size: 689 bytes --]

Re: jquery 3.1.1

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 1250 bytes --]

2017-01-19 23:51 GMT+01:00 Catonano <catonano@gmail.com>:

> One last note
>

One more quick note

>
> I pasted the wrong url for the packages with no dependencies
>
> The right one is this
> http://catonano.altervista.org/no-deps.txt
>
> I noticed now that this list contains ALSO the broken ones. They have no
> dependencies, but they shold be excluded from this list
>
> Because I screwed up the data format, the queries are not exactly
> immediate, they need honing :-/
>
> I will fix this in the future
>

I produced a correct version of the "no-deps.txt" file

Also, a notion of "scoped" packages exists in npm

They can't be downloaded, as explained here
https://github.com/npm/registry/issues/34

These packages have the cons cell "(broken-package . nope)"

Yes, they are broken and they have the symbol "nope" nonetheless. It's
conterintuitive, I know. I messed this thing up. I made the download
function return "nope" when a download isn't allowed.

I marked them as broken and moved on. The subtree under these packages has
NOT been explored.

I could use some help, here, by someone used the the Nodejs intricacies

I don't know what they mean with "scoped" and I don't know what this could
mean exactly for the "Nodejs in Guix" mission

[-- Attachment #2: Type: text/html, Size: 2261 bytes --]

Re: jquery 3.1.1

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 600 bytes --]

2017-01-21 20:12 GMT+01:00 Catonano <catonano@gmail.com>:

I could use some help, here, by someone used the the Nodejs intricacies
>
>
I don't know what they mean with "scoped" and I don't know what this could
> mean exactly for the "Nodejs in Guix" mission
>

The same goes for packages like this

scheme@(guile-user)> ,m (npmjs)
scheme@(npmjs)> (downloaded-package "@types/minimatch" "^2.0.29")
$1 = (@ ("code" . "MethodNotAllowedError") ("message" . "GET is not
allowed"))
scheme@(npmjs)>

I don't know what this means exactly but the subtree of packages like this
one couldn't be explored either

[-- Attachment #2: Type: text/html, Size: 1437 bytes --]

Re: jquery 3.1.1

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 253 bytes --]

The 2 symbols

'the-package-does-not-exist
'the-version-does-not-exist

are clearer.

Some other package declares these as a dependency. But such dependency
doesn't exist

Will these dependants be buildable ? I don't know

There are many of these cases

[-- Attachment #2: Type: text/html, Size: 584 bytes --]

Re: jquery 3.1.1

[Mike Gerwitz]

[-- Attachment #1: Type: text/plain, Size: 593 bytes --]

On Sat, Jan 21, 2017 at 20:12:25 +0100, Catonano wrote:
> I could use some help, here, by someone used the the Nodejs intricacies

Scoped packages are described here:

  https://docs.npmjs.com/misc/scope

I've never actually used them, but they're installed normally:

  https://docs.npmjs.com/cli/install

What's a specific package you're having trouble with?

-- 
Mike Gerwitz
Free Software Hacker+Activist | GNU Maintainer & Volunteer
GPG: D6E9 B930 028A 6C38 F43B  2388 FEF6 3574 5E6F 6D05
Old: 2217 5B02 E626 BC98 D7C0  C2E5 F22B B815 8EE3 0EAB
https://mikegerwitz.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: jquery 3.1.1

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 763 bytes --]

2017-01-21 22:41 GMT+01:00 Mike Gerwitz <mtg@gnu.org>:

> On Sat, Jan 21, 2017 at 20:12:25 +0100, Catonano wrote:
> > I could use some help, here, by someone used the the Nodejs intricacies
>
> Scoped packages are described here:
>
>   https://docs.npmjs.com/misc/scope


thanks


>
>
> I've never actually used them, but they're installed normally:
>
>   https://docs.npmjs.com/cli/install
>
> What's a specific package you're having trouble with?
>

For example @types/minimatch ^2.0.29

I'm afraid it is set to restricted access, as it can't be downloaded.

It's in the dependencies tree of Jquery

Another one is @ljharb/eslint-config ^8.0.0

Another one is @remy/snyk-shrink-test ^1.0.0

Another one is @types/chalk ^0.4.28

How would Guix deal with these ?

[-- Attachment #2: Type: text/html, Size: 1658 bytes --]

Re: jquery 3.1.1

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 219 bytes --]

Just as a note, there was this talk at the Fosdem and I think it adds some
bits I had missed (mostly because I don' t know nodejs)

http://ftp.osuosl.org/pub/fosdem/2017/K.4.601/deploying_npm_packages_with_nix.vp8.webm

[-- Attachment #2: Type: text/html, Size: 364 bytes --]

Re: jquery 3.1.1

[Mike Gerwitz]

[-- Attachment #1: Type: text/plain, Size: 1761 bytes --]

On Thu, Jan 19, 2017 at 21:48:44 +0100, Catonano wrote:
> Anyway, now I have a COMPLETE graph of the dependencies of jquery 3.1.1
>
> It's made of
> 47311 vertices and
> 324569 edges

lol...

> Anyway, these broken packages pose a challenge to the mission of porting
> Jquery into Guix, in my opinion,

My greater concern is verifying licenses: that'd have to be considered
in the DAG (...I hope it's a DAG; who knows what those node packages
might be doing!) to flag potential problems.  The JS community is pretty
lax on licensing (in both the permissive sense and the I-don't-care
sense); the license might not be correct or might be missing
entirely.  Or might not match what's in the source files.

Verifying that many dependencies is going to be a challenge for an
automated system; we'd want humans to look at many of them too to make
sure things aren't fishy. :x  The problem is that one single dependency
that's mischaracterized as free---even if it's one of the
single-function packages---can destroy an entire project (e.g. jQuery).

For some packages, this task is feasible.

> The code is here
> https://gitlab.com/humanitiesNerd/Culturia

Thanks for all the hard work you've put into this.  I admit that I don't
have the time to read into it much right now, but I'll certainly be
following progress on this list.

> One last fun fact: while I was watching the output flowing in my terminal,
> I saw a package called
>
> "broccoli-funnel"

Ah, they missed a really good logo opportunity!

-- 
Mike Gerwitz
Free Software Hacker+Activist | GNU Maintainer & Volunteer
GPG: D6E9 B930 028A 6C38 F43B  2388 FEF6 3574 5E6F 6D05
Old: 2217 5B02 E626 BC98 D7C0  C2E5 F22B B815 8EE3 0EAB
https://mikegerwitz.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: jquery 3.1.1

[Ludovic Courtès]

Hey,

Mike Gerwitz <mtg@gnu.org> skribis:

> On Thu, Jan 19, 2017 at 21:48:44 +0100, Catonano wrote:
>> Anyway, now I have a COMPLETE graph of the dependencies of jquery 3.1.1
>>
>> It's made of
>> 47311 vertices and
>> 324569 edges
>
> lol...
>
>> Anyway, these broken packages pose a challenge to the mission of porting
>> Jquery into Guix, in my opinion,
>
> My greater concern is verifying licenses: that'd have to be considered
> in the DAG (...I hope it's a DAG; who knows what those node packages
> might be doing!) to flag potential problems.  The JS community is pretty
> lax on licensing (in both the permissive sense and the I-don't-care
> sense); the license might not be correct or might be missing
> entirely.  Or might not match what's in the source files.
>
> Verifying that many dependencies is going to be a challenge for an
> automated system; we'd want humans to look at many of them too to make
> sure things aren't fishy. :x  The problem is that one single dependency
> that's mischaracterized as free---even if it's one of the
> single-function packages---can destroy an entire project (e.g. jQuery).

Indeed, that’s terrible.

(One could argue that single-function packages are “trivial” from a
copyright standpoint.  Then the subset of the npm repo containing those
trivial packages could be viewed as a database of “facts” (which, in
some jurisdiction, is covered by a “sui generis” right disjoint from
copyright.))

>> One last fun fact: while I was watching the output flowing in my terminal,
>> I saw a package called
>>
>> "broccoli-funnel"
>
> Ah, they missed a really good logo opportunity!

:-)

Ludo’.

Re: jquery 3.1.1

[Mike Gerwitz]

[-- Attachment #1: Type: text/plain, Size: 1058 bytes --]

On Fri, Jan 20, 2017 at 22:14:42 +0100, Ludovic Courtès wrote:
> (One could argue that single-function packages are “trivial” from a
> copyright standpoint.  Then the subset of the npm repo containing those
> trivial packages could be viewed as a database of “facts” (which, in
> some jurisdiction, is covered by a “sui generis” right disjoint from
> copyright.))

That depends on the size/complexity of the function, but it's
interesting to consider the packaging situation in that manner.

It's also interesting to think about whether the recursive composition
of trivial (non-copyrightable) packages constitutes a copyrightable
work, and if so, at what point.  Because you'd otherwise imagine that
any program could be factored such that it is a sum of a large number of
trivial components.

I digress...

-- 
Mike Gerwitz
Free Software Hacker+Activist | GNU Maintainer & Volunteer
GPG: D6E9 B930 028A 6C38 F43B  2388 FEF6 3574 5E6F 6D05
Old: 2217 5B02 E626 BC98 D7C0  C2E5 F22B B815 8EE3 0EAB
https://mikegerwitz.com

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: jquery 3.1.1

[Ludovic Courtès]

Hi!

Catonano <catonano@gmail.com> skribis:

> I made a crawler and I let it loose on the jquery 3.1.1 dependencies on
> registry.npmjs.com
>
> It recursevely fetched the dependencies of jquery 3.1.1, then the
> dependencies of the dependencies, then the dependencies of the dependencies
> of the dependencies... and so on
>
> Until there were no more dependencies to fetch.
>
> It stored all such dependencies in a graph database made by the amazing
> amz3 for their project, Culturia. They wrote about such project several
> times on the guile-users mailing list.

Neat!

> It took days to download. And it took me months to produce a working
> version of this code. Because I don't know Guile very well and maybe I'm
> not that smart overall.
>
> Anyway, now I have a COMPLETE graph of the dependencies of jquery 3.1.1
>
> It's made of
> 47311 vertices and
> 324569 edges

Impressive.

I suppose the data you gathered about the graph could help guide usage
of Jelle’s importer?  (… which we should consider merging BTW!)

Thanks,
Ludo’.

Re: jquery 3.1.1

[Catonano]

[-- Attachment #1: Type: text/plain, Size: 1412 bytes --]

2017-01-20 22:23 GMT+01:00 Ludovic Courtès <ludo@gnu.org>:

> Hi!
>
> Catonano <catonano@gmail.com> skribis:
>
> > I made a crawler and I let it loose on the jquery 3.1.1 dependencies on
> > registry.npmjs.com
> >
> > It recursevely fetched the dependencies of jquery 3.1.1, then the
> > dependencies of the dependencies, then the dependencies of the
> dependencies
> > of the dependencies... and so on
> >
> > Until there were no more dependencies to fetch.
> >
> > It stored all such dependencies in a graph database made by the amazing
> > amz3 for their project, Culturia. They wrote about such project several
> > times on the guile-users mailing list.
>
> Neat!
>
> > It took days to download. And it took me months to produce a working
> > version of this code. Because I don't know Guile very well and maybe I'm
> > not that smart overall.
> >
> > Anyway, now I have a COMPLETE graph of the dependencies of jquery 3.1.1
> >
> > It's made of
> > 47311 vertices and
> > 324569 edges
>
> Impressive.
>
> I suppose the data you gathered about the graph could help guide usage
> of Jelle’s importer?  (… which we should consider merging BTW!)
>
>
I would be honored if my data could help guide usage of Jelle's importer.

That was exactly my idea when I started this. The Nodejs stuff can't be
treated with brute force.
A data oriented approach can help, here.

[-- Attachment #2: Type: text/html, Size: 2051 bytes --]