From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp11.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms9.migadu.com with LMTPS id 6AhxL1+qb2RKIgAASxT56A (envelope-from ) for ; Thu, 25 May 2023 20:35:11 +0200 Received: from aspmx1.migadu.com ([2001:41d0:2:4a6f::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp11.migadu.com with LMTPS id wCFnL1+qb2R6wgAA9RJhRA (envelope-from ) for ; Thu, 25 May 2023 20:35:11 +0200 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 617293F0E1 for ; Thu, 25 May 2023 20:35:11 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q2Fmb-0000II-BF; Thu, 25 May 2023 14:33:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q2Fma-0000HS-43 for guix-devel@gnu.org; Thu, 25 May 2023 14:33:52 -0400 Received: from mail-wm1-x335.google.com ([2a00:1450:4864:20::335]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q2FmY-00035b-6M; Thu, 25 May 2023 14:33:51 -0400 Received: by mail-wm1-x335.google.com with SMTP id 5b1f17b1804b1-3f60a7e132aso4784165e9.0; Thu, 25 May 2023 11:33:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685039629; x=1687631629; h=content-transfer-encoding:mime-version:message-id:date:references :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=olX1+dYvZX8ndG+psxNeK74Ry2AaPeeGSJ4DDrHFWYM=; b=f/JQGJnZGvH3wAjRolPkaGNCLyYZei7UmtIoOTxMwrX/MJ7jSrUyAuFfEZkSFWg8jS mzCZ2px22JoVEfE6SQk90onqlaUq7yzTGPFXYBhMW8ky936H59Vb2uwqDgkcVY/fazZn lLbB5WvKBlxWGx6o5bE68aLqxaGJlidQxMA1IWP8N/f5P4pE7y1bpQ5jrR2mx9ItmxeM G7+BnqMkAKIs93ALZDX5oqxz6tqVxKHm+qnGyDuSZR8qqWY0kEm8OsSmpr1Rnj+D5Iaq CIQrTnrO1rP88CvXA+5GigS5GIomOIf32hxWJGc9k+GWZukJHaqksU74AAwonvVHhPY/ 8Utw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685039629; x=1687631629; h=content-transfer-encoding:mime-version:message-id:date:references :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=olX1+dYvZX8ndG+psxNeK74Ry2AaPeeGSJ4DDrHFWYM=; b=dI5RPvyUgPkjlNVJXcabPgETIm8B01vWIV3JkA0SrX4cwrzNTPW1fFelalX0HCzwc/ U0r3U21k34JtdDHOEH+kWKHlAfnjYRmtHRbBbiy1MZ8LwA9Us2C/czckBEoFsFylhals lHoB7XIF2J0aFlBzkvWfA77CXX4SnTh0REuzUoTX5lzImPsqPt8R5sww1/PGRuq1qZwM MParpcTYDSoKJ5PqEr0o45OyhxF86GMgMLaTXwDQy60KnTQD4p9s3cjNJzgh+QVFNKUT mkSVjPKMZT8Rf9R2YYILGAIsgBSPVT8fqXRERhPFmOGLQ4NLDIQwYDaa+PTlZbmadCuY Bi0A== X-Gm-Message-State: AC+VfDz/XKxiK+tUj3nXsImsOft9bp7+B2EeuZKLWFPZ0Kw8Gpyv9TrY dvk6b4EiVxoepIP1S5+WqRJOOpwRvXg= X-Google-Smtp-Source: ACHHUZ6xq9M3MhhGdd9KHB75MVASL+MhkKSiXhaLxSWXBrRVaJvXcZEXUfiLY9rFa/lHs/gXu83X/A== X-Received: by 2002:a05:600c:3b08:b0:3f6:87e:f89a with SMTP id m8-20020a05600c3b0800b003f6087ef89amr10202304wms.2.1685039629034; Thu, 25 May 2023 11:33:49 -0700 (PDT) Received: from pfiuh07 ([193.48.40.241]) by smtp.gmail.com with ESMTPSA id f16-20020a7bcc10000000b003f60fb2addbsm6630545wmh.44.2023.05.25.11.33.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 May 2023 11:33:48 -0700 (PDT) From: Simon Tournier To: Guix Devel Cc: Ludovic =?utf-8?Q?Court=C3=A8s?= , Andreas Enge Subject: How many bytes do we add (closure of guix) when adding one new package? References: <875y9jzl9m.fsf@gnu.org> <874jot19fd.fsf_-_@gnu.org> <87fs7rvv5s.fsf_-_@gnu.org> <878rddooy4.fsf@gnu.org> Date: Thu, 25 May 2023 20:24:30 +0200 Message-ID: <87r0r4uv4x.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Received-SPF: pass client-ip=2a00:1450:4864:20::335; envelope-from=zimon.toutoune@gmail.com; helo=mail-wm1-x335.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, PDS_BTC_ID=0.016, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: guix-devel-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN ARC-Seal: i=1; s=key1; d=yhetil.org; t=1685039711; a=rsa-sha256; cv=none; b=ewklHIyzZ2cSCUQcbVoyJ2HM6Hz2XKzPS7YXBWK/+cuHs4kgIUbiCujeRwia7fzJzxj1uh YzPohoIGGzXKI+xFbP0KXVPVkxFK56FIHAPY9c70Fg6JGGtFCxCh6qtOfX7HoEeWpTfRvk c95hPF9vcPxs/uU1Sl9TJstFSF+Y0F+4L7BIQbW+T8xo1s0PSW4BwjAlxmuf1eA7aOlCI8 H7h/oJyNSXeHoMoOog5NF0wB/IdbxRdQmssy2cqX6iihLAMcbHTZDOV43A2YEfNbYiu1Me C/D6UQ3ofFihesVWT1AKhq8cVq86qhaFIyJUbyr9sd965A3Eck73OVB4Uxlnvg== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20221208 header.b="f/JQGJnZ"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1685039711; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=olX1+dYvZX8ndG+psxNeK74Ry2AaPeeGSJ4DDrHFWYM=; b=KaXOhLX3NUIuZJCsF0KFusWX1m/nnQKRE0knxIqflBwkVap8IUYKuMBOgLJPCIaMyZB68a Peyk254gP5oiqwtb0joj6lWDbzr9k6ClYBj0Mj9K28MS2A+65Bd/f1UJWH1txoMjdh97eq EPXaUTStoLMmR+SRK0ScKUuXdChVKmB93rCn35oefGteNgYaoo8KbKQcKljzW6ZC/vGdi9 9DrDCaF66skjG3VRHk8XermaDh4d9r3IkHY9aETONUyV+45EMqjCPRg0UwsjVIKeVdR35n GS/pYfy//vwdNUl+loMXWMUOkTUiFXnwA14LgPTh6cLo4Rp4Ur79z8yo66tZag== X-Migadu-Spam-Score: -6.63 X-Spam-Score: -6.63 X-Migadu-Queue-Id: 617293F0E1 X-Migadu-Scanner: scn0.migadu.com Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=gmail.com header.s=20221208 header.b="f/JQGJnZ"; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (aspmx1.migadu.com: domain of "guix-devel-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-devel-bounces+larch=yhetil.org@gnu.org" X-TUID: QUHX2mpvYAen Hi, The initial discussion was about the closure of Guix and that =E2=80=9Cguix= pull=E2=80=9D brings graphical libraries. See #63050 [1]. Here, I would like to open a discussion about how Guix scales, i.e. about its size. I am trying to answer the question I am asking as the subject. ;-) It=E2=80=99s another angle to view Andreas and Ludo discussion: :-) >> Note that I do not care so much about the closure size, but about the >> number of packages that are needed to just build guix (although of course >> the two are related). Or otherwise said, the dependencies for "guix pull= ". > > Yes, understood. Graphviz is not in the closure anyway, it=E2=80=99s a > build-only dependency. Somehow, the closure is increasing: --8<---------------cut here---------------start------------->8--- $ for i in $(seq 1 4); do guix time-machine --commit=3Dv1.$i.0 -- size guix= | grep 'total:' ;done total: 410.9 MiB total: 496.0 MiB total: 564.8 MiB total: 637.2 MiB $ guix size guix | grep 'total:' total: 611.2 MiB --8<---------------cut here---------------end--------------->8--- (Yeah, the package guix is not exactly the same as guix itself, but it appears to me a good enough approximation. And my current revision is 14c0380.) Compare: --8<---------------cut here---------------start------------->8--- $ guix time-machine --commit=3Dv1.1.0 -- size guix --sort=3Dself | wc -l 44 $ guix time-machine --commit=3Dv1.4.0 -- size guix --sort=3Dself | wc -l 72 $ guix size guix --sort=3Dself | wc -l 70 --8<---------------cut here---------------end--------------->8--- which is the Andreas=E2=80=99s concern for =E2=80=9Cexotic=E2=80=9D archite= ctures. Moreover, the inflation (in size) is about some packages that are just becoming bigger. --8<---------------cut here---------------start------------->8--- $ guix time-machine --commit=3Dv1.1.0 -- size guix --sort=3Dself | head store item total s= elf /gnu/store/fp16m5hkzql7jwhvnkm1j1i5qch0arhx-guix-1.1.0rc2-1.9d0d27f 410.9= 221.6 53.9% /gnu/store/1mkkv2caiqbdbbd256c4dirfi4kwsacv-guile-2.2.6 123.9 = 44.4 10.8% /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29 37.4 = 35.8 8.7% /gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib 70.0 = 32.6 7.9% /gnu/store/n79cf8bvy3k96gjk1rf18d36w40lkwlr-glibc-utf8-locales-2.29 13.9= 13.9 3.4% /gnu/store/k2m4q2av9hw73hw2jx6qrxqdyh855398-openssl-1.1.1c 76.4 = 6.4 1.6% /gnu/store/gzp4ig4rdb1qf4i5dy1d9nl0zmj5q09y-ncurses-6.1-20190609 75.9 = 5.9 1.4% /gnu/store/hfvz18igm68p5yz7z4asn6ph363blp1z-gnutls-3.6.9 130.6 = 5.1 1.2% /gnu/store/b5vjmib411m74lbpf051fnwz3s9zcw79-guile-git-0.3.0 98.8 = 4.4 1.1% $ guix time-machine --commit=3Dv1.4.0 -- size guix --sort=3Dself | head store item total s= elf /gnu/store/9nvx97hr8kkr26gzwni2fblfn0yq0xjw-guix-1.4.0rc2 637.2 = 330.1 51.8% /gnu/store/qlmpcy5zi84m6dikq3fnx5dz38qpczlc-guile-3.0.8 130.0 = 53.0 8.3% /gnu/store/cnfsv9ywaacyafkqdqsv2ry8f01yr7a9-guile-3.0.7 129.1 = 52.0 8.2% /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33 38.3 = 36.6 5.7% /gnu/store/094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib 71.7 = 33.4 5.2% /gnu/store/96srhmpmxa20wmsck95g3iq4hb3lz4a0-glib-2.70.2 98.1 = 15.3 2.4% /gnu/store/mw3py6smb1pk8yx298hd9ivz9lzbksqi-glibc-utf8-locales-2.33 13.9= 13.9 2.2% /gnu/store/5583c2za2jsn9g6az79rnksgvigwnsk7-util-linux-2.37.2-lib 80.7 = 9.0 1.4% /gnu/store/9rrnm5hdjw7cy96a2a9rfgh6y08wsbmf-ncurses-6.2.20210619 77.6 = 5.9 0.9% $ guix size guix --sort=3Dself | head store item total s= elf /gnu/store/cgjddvw9zay626z8hyxl0zmn1354c24k-guix-1.4.0-6.dc5430c 611.2 = 350.2 57.3% /gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9 135.0 = 53.1 8.7% /gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35 40.6 = 38.8 6.3% /gnu/store/930nwsiysdvy2x5zv1sf6v7ym75z8ayk-gcc-11.3.0-lib 75.3 = 34.7 5.7% /gnu/store/nb40pwd37v6i1g4b1fq4l6q4h9px3asr-glib-2.72.3 101.3 = 14.9 2.4% /gnu/store/5fmqijrs5f7vx8mc2q2pmq26yvhb74sm-glibc-utf8-locales-2.35 13.9= 13.9 2.3% /gnu/store/gwx2sf5wl9bsl21lwv35g5la63bwyy95-util-linux-2.37.4-lib 84.3 = 9.0 1.5% /gnu/store/69wd3pd1hd3j84xr965jj2fk2qmxn0hl-openssl-3.0.8 83.4 = 8.1 1.3% /gnu/store/bcc053jvsbspdjr17gnnd9dg85b3a0gy-ncurses-6.2.20210619 81.2 = 5.9 1.0% --8<---------------cut here---------------end--------------->8--- Considering Guix itself, one explanation for the increase is the number of packages =E2=80=93 assuming the services and other are negligible; =E2= =80=9Cgit diff --shortstat=E2=80=9D is a good indicator at first sight. Well, we could be= more precise about the documentation. Hum, this ugly, --8<---------------cut here---------------start------------->8--- $ for doc in $(for ci in $(for t in $(git tag | grep v1 | grep -v rc ); do = git --no-pager show $t | grep commit ;done); do for d in $(find ~/.cache/gu= ix/inferiors/ -type l -print); do printf "$d "; $d/bin/guix --version 2>/de= v/null ;done | grep $ci ;done | cut -f1 -d' '); do du -sh $(readlink -f $do= c)/share/* ;done | grep info 172K /gnu/store/5pa1706ckwhn6x4mn5kl2b7h15k3in9x-profile/share/info 200K /gnu/store/z1icpkfbz59dr7k7rnb0jd8j1ii8mdph-profile/share/info 376K /gnu/store/hm0rwgcvrs85y3hgjsw8616cxy61h6si-profile/share/info 304K /gnu/store/zbrgzk7l0j7805i82sl3gmx6y2b0iz9q-profile/share/info --8<---------------cut here---------------end--------------->8--- is probably providing a clue about the assumption. Ok, let assume that the packages are the main source of size increasing. The question is: can we evaluate the size for one package? How many bytes do we add to the whole Guix when we add one package? On average and roughly. We have the number of packages and the whole size for successive versions. Therefore, we can do the difference between the two. We get [2078, 1848, 4704, 1532] which means 2078 packages had been added between v1.1.0 and v1.2.0, 1848 between v1.2.0 and v1.3.0, etc. We can do the same for the size, [22.9, 19.4, 66.2, 20.1] and then we can compute the ratio: the size per package. Something like: [0.011020211742059676, 0.010497835497835485, 0.01407312925170069, 0.0131201= 04438642276] Let get an average: 0.012177820232559531. Now, let take the number of packages for v1.1.0 and do the multiplication. We get: 159.6 MiB. Ok, it means that the difference is more or less the core of Guix =E2=80=93= what we are assuming that is slowly growing. It reads 62 MiB. Therefore, we can predict the size for the other versions using this linear model based on the previous evaluated average. size =3D mean * number_packages + core It reads: [246.9, 269.4, 326.7, 345.3] compared to [244.5, 263.9, 330.1, 350.2]. Hum, this quick back-to-the-envelope computation does not seem too bad. I guess. Conclusions: 1. the addition of one package leads to an increase of ~ 12 KiB 2. the core of Guix is about ~ 62 MiB 3. doubling the number of packages is doubling the size to download at =E2=80=9Cguix pull=E2=80=9D time. Maybe, we should re-think (guix self). Especially the *package-modules* part and re-discuss if we could split that part. From my understanding. Cheers, simon 1: https://issues.guix.gnu.org/issue/63050