From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id OLUYO8DQk2PQ/QAAbAwnHQ (envelope-from ) for ; Sat, 10 Dec 2022 01:20:17 +0100 Received: from aspmx1.migadu.com ([2001:41d0:2:bcc0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id mAQeO8DQk2MhQgEA9RJhRA (envelope-from ) for ; Sat, 10 Dec 2022 01:20:16 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 6D45B3A90B for ; Sat, 10 Dec 2022 01:20:16 +0100 (CET) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1p3naO-0005UB-Mv; Fri, 09 Dec 2022 19:19:25 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p3naM-0005Tw-Ft for guix-devel@gnu.org; Fri, 09 Dec 2022 19:19:22 -0500 Received: from mx1.dismail.de ([78.46.223.134]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1p3naJ-0003Hq-7B for guix-devel@gnu.org; Fri, 09 Dec 2022 19:19:21 -0500 Received: from mx1.dismail.de (localhost [127.0.0.1]) by mx1.dismail.de (OpenSMTPD) with ESMTP id 3f68b5b5; Sat, 10 Dec 2022 01:19:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=dismail.de; h= mime-version:date:content-type:content-transfer-encoding:from :message-id:subject:to:in-reply-to:references; s=20190914; bh=B5 svRzZYp6orSN0OBzjpogbNfNQUeMkhphLY5qpO8sI=; b=YgkINLamwkumvBBC96 aG9eiP2CgsHwVcCix1N1tZRTxDmRqYs3Qn+2Z8/UDY/U6ZLpQ2cEWSPGEaccZkRg LdipkL7vM55dQSf/eb7vPHKsy4t7f4mvfckuKGuVy96jWVULtW+KE/1Bb3ZKDB89 0OMinl3hZ7JtSSHPV6Kg0KjN1vuEdy7oV+0zIg4LX2lBe78jneZjhkfX9NuI0ppz cVtL0URNj56LjLT+M5QQH85ZftaqCqiEkb+qKXaJrDjN4uQaTK+0bZux2ryZoScl 3BX5rVlzMM/muXjrPv8qlcPgAriye2xbCwfq5+uGKSCO/19aVlomp4kxP5SSXoAU zlHw== Received: from smtp1.dismail.de ( [10.240.26.11]) by mx1.dismail.de (OpenSMTPD) with ESMTP id 216b5225; Sat, 10 Dec 2022 01:19:02 +0100 (CET) Received: from smtp1.dismail.de (localhost [127.0.0.1]) by smtp1.dismail.de (OpenSMTPD) with ESMTP id e15f1dda; Sat, 10 Dec 2022 01:19:02 +0100 (CET) Received: by dismail.de (OpenSMTPD) with ESMTPSA id 745b2d19 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Sat, 10 Dec 2022 01:19:01 +0100 (CET) MIME-Version: 1.0 Date: Sat, 10 Dec 2022 00:19:01 +0000 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Mailer: RainLoop/1.16.0a From: jbranso@dismail.de Message-ID: <0790f770f69af2bffd6d6d9d4ba22bc7@dismail.de> Subject: Re: Some stats about the graph of dependencies To: "zimoun" , "Guix Devel" In-Reply-To: <874ju4qyd4.fsf@gmail.com> References: <874ju4qyd4.fsf@gmail.com> Received-SPF: pass client-ip=78.46.223.134; envelope-from=jbranso@dismail.de; helo=mx1.dismail.de X-Spam_score_int: -27 X-Spam_score: -2.8 X-Spam_bar: -- X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1670631616; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=B5svRzZYp6orSN0OBzjpogbNfNQUeMkhphLY5qpO8sI=; b=kEYP9DtO2wXEDyYHEqN81BhzQhmGQZIu+ecFpKvg1HHhAw0AM2GBbzHEbkr+D+ayb0hAtx uBjWvTQZEnsTudR0r5S8o39rMrOHW7YK4cSiO58GBc6uu1zfNC9g0Tlif6WhTOAMKfbFpv B4iHa23SyZN7zfoJPwRPG4bkbrGSpINEcm6urw+WAZhLhkNnWZVb2/HiJL4i3qc9OJklfI GqLjDBzVLu3u9tigcwlQH+sQswwGg4WDrYL/aciUxLv+pHz77wsGEW8jmyXhlo6+9ycqg+ xVG3Y6mebbzEwR+Q45BXuA9i4+7V5yF5HmE1b5QrdOG+BZRjYIgQLoc9pDeHog== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1670631616; a=rsa-sha256; cv=none; b=AhquwxAOhQgwRmzGykEnFX39QhQf6lVeVajmktIUjUZhzZGoreyr940b/v3Ylg9afDC3IU uAy3fQLKbVkELfA9DMtNJQfFE2FAMKtrx7fxeRiDAicPUjuN409RZR5MdXmqV+FYxfOl5a V1jg8Wq3xA9e889vx4qESgmfG7tRwdrxm4Xl+K+DXOdyd1gWWysD7xpGJ5U+TtXQd0ENln jxZtf0pLsCkqXHdgYWdHefalRo+oztcQKbOquLjytddG61mEVZLls6APV/zmf0Kow5xwZZ I+DQonH9BO4mAt/sQOzuZT6sKlTB71soxVRmTLp0c4ar/OtJRiJUDMgyTNIjlw== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=dismail.de header.s=20190914 header.b=YgkINLam; dmarc=pass (policy=reject) header.from=dismail.de; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Spam-Score: -5.05 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=dismail.de header.s=20190914 header.b=YgkINLam; dmarc=pass (policy=reject) header.from=dismail.de; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-Migadu-Queue-Id: 6D45B3A90B X-Spam-Score: -5.05 X-Migadu-Scanner: scn1.migadu.com X-TUID: Iu1oCXo7J5ba December 9, 2022 12:32 PM, "zimoun" wrote: > Hi, >=20 >=20Preparing some Python stuff, I was toying with the package > python-networkx. And Guix is awesome because it is easy to extract the > graph of dependencies. >=20 >=20Here dependencies are just inputs, native-inputs and propagated-input= s. > It could be interesting to also include build-system dependencies, I > have been lazy. :-) >=20 >=20My initial question is to know what are the =E2=80=9Cessentials=E2=80= =9D? By essential, > I mean the =E2=80=9Cimportant=E2=80=9C ones, the =E2=80=9Chot=E2=80=9D = ones, etc. The ones which are > =E2=80=9Cinfluencers=E2=80=9D =E2=80=93 yeah the world is a social netw= ork. :-) >=20 >=20First, let extract the graph with a tiny Scheme script: >=20 >=20$ guix repl -- packages-to-dict.scm > dod.py >=20 >=20Then, let import that into an IPython session: >=20 >=20$ guix shell python python-ipython \ > python-scipy python-matplotlib python-networkx -- ipython >=20 >=20and run another tiny Python script for plotting. See Figure attached. >=20 >=20We can compare a link analysis metrics [1] and a centrality measure > [2]; say PageRank [3] and Eigenvector [4]. More the value is large and > higher the package is =E2=80=9Cimportant=E2=80=9C (for this metrics). >=20 >=20And the Directed and Undirected graphs can be compared, using Network= x > [5,6]. Well, Eigenvector centrality (or Katz centrality [7]) is failing > because the power iteration does not converge but other metrics could b= e > also considered. Here is just a first rough toy. :-) >=20 >=20According to PageRank applied to the Directed Graph, the 10 most > =E2=80=9Cimportant=E2=80=9D packages are: >=20 >=20--8<---------------cut here---------------start------------->8--- > [('pkg-config-0.29.2', 0.02418335991713879), > ('perl-5.34.0', 0.015404032767249512), > ('coreutils-minimal-8.32', 0.013240458675517012), > ('zlib-1.2.11', 0.009107245584307803), > ('python-pytest-6.2.5', 0.008413060648307678), > ('ncurses-6.2.20210619', 0.007598925467605917), > ('r-knitr-1.41', 0.00554772892485958), > ('sbcl-rt-1990.12.19-1.a6a7503', 0.004884721933452539), > ('bzip2-1.0.8', 0.004800877844001881), > ('python-3.9.9', 0.00415536078558266)] > --8<---------------cut here---------------end--------------->8--- >=20 >=20And if we compare the 3 results (Undirected with PageRank and > Eigenvector, and Directed with PageRank only, then 10 most =E2=80=9Cimp= ortant=E2=80=9D > packages are: >=20 >=20--8<---------------cut here---------------start------------->8--- > ['pkg-config-0.29.2', > 'glib-2.70.2', > 'zlib-1.2.11', > 'gtk+-3.24.30', > 'perl-5.34.0', > 'gettext-minimal-0.21', > 'qtbase-5.15.5', > 'libxml2-2.9.12', > 'python-3.9.9', > 'autoconf-2.69'] > --8<---------------cut here---------------end--------------->8--- >=20 >=20Somehow, it means that these packages have an high influence on all t= he > others. Now, we can roughly compare with the release-manifest.scm [8], >=20 >=20--8<---------------cut here---------------start------------->8--- > '("bootstrap-tarballs" "gcc-toolchain" "nss-certs" > "openssh" "emacs" "vim" "python" "guile" "guix"))) > '("coreutils" "grep" "findutils" "gawk" "make" > #;"gcc-toolchain" "tar" "xz"))) > '("xorg-server" "xfce" "gnome" "mate" "enlightenment" > "openbox" "awesome" "i3-wm" "ratpoison" > "emacs" "emacs-exwm" "emacs-desktop-environment" > "xlockmore" "slock" "libreoffice" > "connman" "network-manager" "network-manager-applet" > "openssh" "ntp" "tor" > "linux-libre" "grub-hybrid" > '("coreutils" "grep" "sed" "findutils" "diffutils" "patch" > "gawk" "gettext" "gzip" "xz" > "hello" "zlib")))) > --8<---------------cut here---------------end--------------->8--- >=20 >=20Well, we could investigate more and play more with some graphs tools. > For instance, include all the build-system dependencies and so on. >=20 >=20Some list about =E2=80=9Cstatistically important=E2=80=9D packages co= uld help for > improving the list of =E2=80=9Cessential=E2=80=9D packages. >=20 >=20Although Python is great, I would like to run Guile. Any Guile librar= y > for manipulating graph is around? https://packages.guix.gnu.org/packages/guile2.2-charting/0.2.0-1.75f755b/ Thought it may be guile 2 only...? >=20 >=20All that to say, Guix is great! :-) And perhaps some of you have alre= ady > some Guile code for analysing graphs. Maybe. >=20 >=20Well, comment or idea is welcome. :-) >=20 >=201: > 2: > 3: > 4: > 5: > 6: > 7: > 8: >=20 >=20Cheers, > simon