* bug#63050: "guix pull" requires graphical libraries @ 2023-04-24 10:13 Andreas Enge 2023-04-25 21:48 ` Ludovic Courtès 0 siblings, 1 reply; 30+ messages in thread From: Andreas Enge @ 2023-04-24 10:13 UTC (permalink / raw) To: 63050 While trying out a "guix pull" on an aarch64 machine, for which many packages are currently not available as substitutes, I notice an extra- ordinary amount of dependencies, see below (and since I interrupted and restarted it, there are even more dependencies in reality; I remember X11 libraries such as libxi and libxt). Many of them are related to graphical environments, which should not happen for a command line program. Chances are they are pulled in for building documentation (not necessarily of Guix, but of packages that are needed for pulling); but this is still undesirable and should be sorted out. Actually there is a relatively high risk that on non-x86 machines "guix pull" will fail due to one of the packages not building. And then if the packages are fixed, they cannot be pulled... (but substitutes would still work then). Andreas The following derivations will be built: /gnu/store/vrzlz31xgsmz05m294maxvkwld98yvwp-profile.drv /gnu/store/bmk0f4b0ia9vvm9djkhjp5yiibgiwqkv-guix-827df9d1d.drv /gnu/store/75051kjkpqyk63jfcc21jxx6blh1v6xj-guix-command.drv /gnu/store/4mpya4wbmid6rxszs6qrlr56hgxpsmzx-guix-module-union.drv /gnu/store/81vk2gf0qfz325wk0rdfabpjxkwgnlcx-guile-git-0.5.2.drv /gnu/store/k8pg7wpmhzkip9h4x6vw958i81p9rrxd-libgit2-1.3.2.drv /gnu/store/w0irp6xn30nlmpizhcbjnvhqmsba41jn-cmake-minimal-3.24.2.drv /gnu/store/cqw34xafh837ikspy26787spipggx158-curl-7.85.0.drv /gnu/store/mss4yv015cil1vnjnglq506m83b7n3dy-cmake-bootstrap-3.24.2.drv /gnu/store/w8qxkrwpffd9qs5w1jggy1yi27ycm0xr-jsoncpp-1.9.5.drv /gnu/store/hlscqram59id51hxg0fj15041v52h1kw-meson-1.1.0.drv /gnu/store/97ghaxk1q09aqi92x9qg3nwb5vjm22hv-guile-gnutls-3.7.11.drv /gnu/store/n1rv809j9jwmax12057l0lcphz4bzi7s-gnulib-2022-12-06-1.440b528.drv /gnu/store/0nhbmnck41rl3i8hipkxfcvzyi36wgnr-git-minimal-2.33.1.drv /gnu/store/jhi11h8m8i86ivzrmvfhyj4h99rpqb4y-guix-827df9d1d-modules.drv /gnu/store/0sivy14wkr7g1wsn2jgyz6dh53883k0h-guix-config-modules.drv /gnu/store/1bwjqypxjn3671xmc9b9wh9bfg85nwra-guix-config-source.drv /gnu/store/0047yrla9lddhb9c1b4kl4bpd5v9d7ly-config.scm.drv /gnu/store/d11yp292a29g2m07xn12s6g2hs6w5rq2-guix-config.drv /gnu/store/1qwajh2vs8gmvm882zcw6np48fh7xva8-guix-packages-modules.drv /gnu/store/v2790dmh2savq6ddgq0ics8yz9y8ysvq-guix-packages.drv /gnu/store/034h41xz3m57gms9mx7yydssa66ns6xa-guix-extra.drv /gnu/store/lxanzzz28qk7ypdib5hz8xmibi73g6nv-guile-avahi-0.4.1.drv /gnu/store/s2vi6njxmxv4ng78rfdb9xkdiy40fngb-avahi-0.8.drv /gnu/store/1liwcy87b3cafm2wwfza5jl9c3xfh3k7-glib-2.72.3.drv /gnu/store/7yfb32ngiyx6gsky8ccmaq06qvg9qi8f-dbus-1.14.0.drv /gnu/store/0f25lrmw7yc1k9rxxq6af7bjr4kxpi7c-itstool-2.0.7.drv /gnu/store/sjj9z6kchsbmzskp5i23blpkj2b9v1na-python-libxml2-2.9.14.drv /gnu/store/4axvv1sli67w1jx0c5fxi4369pklby8p-yelp-tools-3.32.2.drv /gnu/store/a1y2z3nh0lmbyiddnc5qq92p991vc6fl-yelp-xsl-41.0.drv /gnu/store/mzial98cazz0wmigjd0by59ympr62zmv-xmlto-0.0.28.drv /gnu/store/r33mnrmy08757czq7x19lbawmngkswb8-docbook-xsl-1.79.2-0.fe16c90.drv /gnu/store/ywxvva73z0gmnjbdac9ml3fld4agy7ll-perl-xml-xpath-1.44.drv /gnu/store/744pph8mif8911dij1gw838slmgsplr4-perl-path-tiny-0.118.drv /gnu/store/nlrv7ll0gdf2k7g0v1g0d2c1sz2ff9pa-perl-unicode-utf8-0.62.drv /gnu/store/r84snp03sqlmvssfsa6f3wank4249dbm-perl-test-fatal-0.016.drv /gnu/store/rqk2rbnpjpcnqswz8hqari1rnw6r8v1m-doxygen-1.9.5.drv /gnu/store/zs50rf9mqd77lf7f7ycwj98mdvm3nwc1-guile-ssh-0.16.3.drv /gnu/store/cysfj5rhcnlwnd0skpb8x5cvh320qcpx-libssh-0.10.4.drv /gnu/store/kfqkj69hxgdl2yhn27l1cx3v4b83438k-guix-packages-base.drv /gnu/store/24bwa2hmiydpjjv58kbnszc67d0063w8-guix-extra-modules.drv /gnu/store/cl6757qxk66kpwhhwscwd3z3ir9ylfxy-guix-cli-core-modules.drv /gnu/store/fb71xmnmdi506sjjfqvvhk2n4q5nw9b5-guix-cli-core.drv /gnu/store/f5p7zbpi3f8bbc1bsdbr293r4p4qlr6k-guix-packages-base-modules.drv /gnu/store/gyncf7k5kvwcfbw1dx3jc73n9jmdw3ml-guix-home-modules.drv /gnu/store/9m9iiqd2yvlbjnpxzfwfk94bi9i2gj13-guix-home.drv /gnu/store/dqjb6lb1m8kaam2klgc44j1g040pr1h4-guix-system.drv /gnu/store/i2vpbmmyywk9sd11hl02zfg1nigq0k24-guix-system-tests-modules.drv /gnu/store/zm9yhv95zs1j68ypl9kr4gm4bbymgw3m-guix-system-tests.drv /gnu/store/ca9k2x3z22dn489g976p31fw8cr05p6s-guix-cli.drv /gnu/store/rk24641w60fqddyj0b4lizndcxvrpl45-guix-cli-modules.drv /gnu/store/wd94xqha92w7wj75704j48yh17pghv48-guix-system-modules.drv /gnu/store/sgyi2j6333mv08r8xxyxhaj47886q3hs-guix-daemon.drv /gnu/store/r80zify247zcsxdw1dm6aacr456zqxyf-guix-daemon-1.4.0-5.286cdf0.drv /gnu/store/w0ssgndl2aq7xzc3ibbkgg4dpgyf2mxb-guix-manual.drv /gnu/store/44l2hp82lmrhmsam320020pvf0wx79gb-guix-translated-texinfo.drv /gnu/store/hx8pv27k6r1q5gmdb0zmp9pqqadqp8gh-po4a-0.68.drv /gnu/store/04kfwmpg17hxxzq13b9s06zl63zcc706-texlive-fonts-latex-59745.drv /gnu/store/2lk5x3aw5vi59dkvf1qd0696fdmirgb7-texlive-bin-20210325.drv /gnu/store/1liwcy87b3cafm2wwfza5jl9c3xfh3k7-glib-2.72.3.drv /gnu/store/2l7j29ck29dcaaffi659pkpxr9bmp64f-gd-2.3.2.drv /gnu/store/dmsa2hy8mp0y8cgidp8mkmh0xlgbyjq8-libjpeg-turbo-2.1.4.drv /gnu/store/7lf76zp346d1qnc7i5laa5rwcrvvvyy1-potrace-1.16.drv /gnu/store/frc8zyviijzzaxkymjpq7dfz671y4hid-ghostscript-9.56.1.drv /gnu/store/qkcykrffj5hqyxvw9inf4ghah1zz29x0-libtiff-4.4.0.drv /gnu/store/91mcaq7l6sw788kivrz6577n6p00qm3h-fontforge-20220308.drv /gnu/store/68gzjwlzc6jpjylwdv44qh6rf85yivhs-libdatrie-0.2.13.drv /gnu/store/dcc68iwnbrgzc5n4h97kypkanp28m9f1-cairo-1.16.0.drv /gnu/store/01680manl8hnmqba20j0whsjjiwjvsc0-libdrm-2.4.114.drv /gnu/store/0xygchfhdlh9d56mwc4i0f1p6plylfjr-libspectre-0.2.10.drv /gnu/store/hy7skak01059cysyanc1pms88m69lp7z-gobject-introspection-1.72.0.drv /gnu/store/hz49a5gbwr82c57w4alij2605rf1214f-poppler-22.09.0.drv /gnu/store/6nzf6fvv04h147sahrqms7b84xrlmnd1-openjpeg-2.5.0.drv /gnu/store/g8i69smy5xn3ncsr9gvzira2wix94gja-lcms-2.13.1.drv /gnu/store/87crljzbhi7hnbqxdzq5wzaak7x8mrbx-nss-3.88.1.drv /gnu/store/j4z7wjq1l0j3j5mgccw9kacr9m4srpmb-cairo-1.16.0.drv /gnu/store/m3yxm0ib9cb0wf43rmi3fbjxfl7q5k9z-pango-1.50.10.drv /gnu/store/rfdpb3zb6v3azsr31zqfy12p5wzbfsg3-harfbuzz-5.3.1.drv /gnu/store/vjmv0713x8afrjz846fgaqfvm9bc9vzi-graphite2-1.3.13.drv /gnu/store/v1llsfvjnhsn31kn15apz8plqgjadrfa-libthai-0.1.29.drv /gnu/store/b8fj9n2xxf6lvhs0lihdvfmr72v8fgwv-zziplib-0.13.72.drv /gnu/store/7anj075n4l9f3y0ixmhr6x4sfa83dp70-texlive-cm-59745.drv /gnu/store/zgqyv9fml11jvzxrv8lcyzziqmf487dj-texlive-metafont-59745.drv /gnu/store/2ir50qnqphbrfags4bc8cw612rgi8nzg-texlive-tools-59745.drv /gnu/store/a30m07ydkq7j88ssiam7hc8srbdzps6h-texlive-latex-l3packages-59745.drv /gnu/store/hb0ln9zgh3g0knql99lh9kwlq4wz3szd-texlive-latex-l3kernel-59745.drv /gnu/store/lql7ys16bf3qck2p670y3wpbff0hqn2d-texlive-etex-59745.drv /gnu/store/m6k2nx744yn38dbhdqqblzqjcmhdm6ls-texlive-hyph-utf8-59745.drv /gnu/store/ihz0r7q4sfavmpnyd31kbngsd80qi6ic-texlive-knuth-lib-59745.drv /gnu/store/sabb3gyfgjlh6adhc8451f18197hjpzh-texlive-latex-base-59745.drv /gnu/store/srp22bx1xzrz45m9x8fr7aibbfs8mxxc-texlive-dehyph-exptl-59745.drv /gnu/store/yqaanhfsmq75znkb39cwpgggwmj019xy-texlive-latex-l3backend-59745.drv /gnu/store/5ryjqaiji6f2s6fzqzngdq5hwppfxxgd-texlive-psnfss-59745.drv /gnu/store/bxxjwnpp2z046q0w2a6m7sgfsx4mm7ri-texlive-amscls-59745.drv /gnu/store/d0hx90i02ra0ivfd6w9hm2zwaain74xr-texlive-babel-59745.drv /gnu/store/dclbvkad5iq6jpvhvvnqjmcs3qh7c01r-texlive-tiny-59745.drv /gnu/store/jwpqv502nwjbkrdn5av004ri4lbb6ch6-texlive-generic-babel-english-59745.drv /gnu/store/k11ia64br72qr69prh993l6hhxnx33hv-texlive-graphics-59745.drv /gnu/store/mid7kd1bp7cih7fhdpky17v3r7zpkr5i-texlive-amsmath-59745.drv /gnu/store/niylw7hb393w341r048ripagj84fh81a-texlive-latex-cyrillic-59745.drv /gnu/store/ixnyrs1sdi30kzr6s4ch8qh73qj34m32-graphviz-7.0.1.drv /gnu/store/4xyxa49yjz7ipcmmqwsyw25qgqy5p09p-gts-0.7.6.drv /gnu/store/c5sv8041g0kvz82xjvgvmz9f11qlnjmp-swig-4.0.2.drv /gnu/store/si6y8qs3lb01h2hmg1p3cgp5zpfr41ki-inferior-script.scm.drv /gnu/store/xiwclg882jbg0r7qx5mwivfhnbg0q6w1-profile.drv ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: "guix pull" requires graphical libraries 2023-04-24 10:13 bug#63050: "guix pull" requires graphical libraries Andreas Enge @ 2023-04-25 21:48 ` Ludovic Courtès 2023-04-26 7:28 ` Andreas Enge ` (2 more replies) 0 siblings, 3 replies; 30+ messages in thread From: Ludovic Courtès @ 2023-04-25 21:48 UTC (permalink / raw) To: Andreas Enge; +Cc: 63050 Hi, Andreas Enge <andreas@enge.fr> skribis: > While trying out a "guix pull" on an aarch64 machine, for which many > packages are currently not available as substitutes, I notice an extra- > ordinary amount of dependencies, see below (and since I interrupted and > restarted it, there are even more dependencies in reality; I remember > X11 libraries such as libxi and libxt). Many of them are related to > graphical environments, which should not happen for a command line > program. Chances are they are pulled in for building documentation > (not necessarily of Guix, but of packages that are needed for pulling); > but this is still undesirable and should be sorted out. This is apparently coming from Graphviz: --8<---------------cut here---------------start------------->8--- $ guix graph --path guix libx11 guix@1.4.0-5.286cdf0 graphviz@2.49.0 libx11@1.7.3.1 $ guix graph --path guix libxt guix@1.4.0-5.286cdf0 graphviz@2.49.0 libxaw@1.0.14 libxt@1.2.1 --8<---------------cut here---------------end--------------->8--- Surprising to me, but apparently it’s been this way from the start, commit b1b07d72c755ea314fb0c8333cd88293ee504ce4 (2013!). Maybe these are optional dependencies? Ludo’. ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: "guix pull" requires graphical libraries 2023-04-25 21:48 ` Ludovic Courtès @ 2023-04-26 7:28 ` Andreas Enge 2023-04-26 8:45 ` Josselin Poiret via Bug reports for GNU Guix 2023-04-28 15:18 ` Simon Tournier 2023-05-03 19:50 ` bug#63050: Reducing the closure size of Graphviz Ludovic Courtès 2 siblings, 1 reply; 30+ messages in thread From: Andreas Enge @ 2023-04-26 7:28 UTC (permalink / raw) To: Ludovic Courtès; +Cc: 63050 Am Tue, Apr 25, 2023 at 11:48:05PM +0200 schrieb Ludovic Courtès: > This is apparently coming from Graphviz > Surprising to me, but apparently it’s been this way from the start, > commit b1b07d72c755ea314fb0c8333cd88293ee504ce4 (2013!). > Maybe these are optional dependencies? So "guix pull" builds what is defined as the guix package, but with the current checkout as source? The package definition of guix has this among the native inputs: ;; XXX: Keep the development inputs here even though ;; they're unnecessary, just so that 'guix environment ;; guix' always contains them. ("autoconf" ,autoconf) ("automake" ,automake) ("gettext" ,gettext-minimal) ("texinfo" ,texinfo) ("graphviz" ,graphviz) ("help2man" ,help2man) ("po4a" ,po4a))) Maybe these could be dropped then, and we could have an expanded package guix-devel that would add these inputs for "guix shell -D guix-devel"? Or is it needed for "guix graph"? $ guix graph --list-backends - graphviz: Generate graph in DOT format for use with Graphviz. ... But for this we do not need any graphical output, it is just text file creation; could we have a graphviz-minimal in console mode? Andreas ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: "guix pull" requires graphical libraries 2023-04-26 7:28 ` Andreas Enge @ 2023-04-26 8:45 ` Josselin Poiret via Bug reports for GNU Guix 2023-04-26 16:59 ` Liliana Marie Prikler 0 siblings, 1 reply; 30+ messages in thread From: Josselin Poiret via Bug reports for GNU Guix @ 2023-04-26 8:45 UTC (permalink / raw) To: Andreas Enge, Ludovic Courtès; +Cc: 63050 [-- Attachment #1: Type: text/plain, Size: 3065 bytes --] Hi Andreas, Andreas Enge <andreas@enge.fr> writes: > So "guix pull" builds what is defined as the guix package, but with the > current checkout as source? No, guix pull uses (guix self), and the dependencies there are not used in a singular place like (inputs ...) or (native-inputs ...), but are peppered throughout the file. However, it uses a reduced dictionary for specification->package to speed it up, and so gives a pretty good idea of what's used: --8<---------------cut here---------------start------------->8--- (("guile" . ,(ref 'guile 'guile-3.0-latest)) ("guile-avahi" . ,(ref 'guile-xyz 'guile-avahi)) ("guile-json" . ,(ref 'guile 'guile-json-4)) ("guile-ssh" . ,(ref 'ssh 'guile-ssh)) ("guile-git" . ,(ref 'guile 'guile-git)) ("guile-semver" . ,(ref 'guile-xyz 'guile-semver)) ("guile-lib" . ,(ref 'guile-xyz 'guile-lib)) ("guile-sqlite3" . ,(ref 'guile 'guile-sqlite3)) ("guile-zlib" . ,(ref 'guile 'guile-zlib)) ("guile-lzlib" . ,(ref 'guile 'guile-lzlib)) ("guile-zstd" . ,(ref 'guile 'guile-zstd)) ("guile-gcrypt" . ,(ref 'gnupg 'guile-gcrypt)) ("guile-gnutls" . ,(ref 'tls 'guile-gnutls)) ("guix-daemon" . ,(ref 'package-management 'guix-daemon)) ("disarchive" . ,(ref 'backup 'disarchive)) ("guile-lzma" . ,(ref 'guile 'guile-lzma)) ("gzip" . ,(ref 'compression 'gzip)) ("bzip2" . ,(ref 'compression 'bzip2)) ("xz" . ,(ref 'compression 'xz)) ("po4a" . ,(ref 'gettext 'po4a)) ("gettext-minimal" . ,(ref 'gettext 'gettext-minimal)) ("gcc-toolchain" . ,(ref 'commencement 'gcc-toolchain)) ("glibc-utf8-locales" . ,(ref 'base 'glibc-utf8-locales)) ("graphviz" . ,(ref 'graphviz 'graphviz)) ("texinfo" . ,(ref 'texinfo 'texinfo))) --8<---------------cut here---------------end--------------->8--- > The package definition of guix has this among the native inputs: > ;; XXX: Keep the development inputs here even though > ;; they're unnecessary, just so that 'guix environment > ;; guix' always contains them. > ("autoconf" ,autoconf) > ("automake" ,automake) > ("gettext" ,gettext-minimal) > ("texinfo" ,texinfo) > ("graphviz" ,graphviz) > ("help2man" ,help2man) > ("po4a" ,po4a))) > > Maybe these could be dropped then, and we could have an expanded package > guix-devel that would add these inputs for "guix shell -D guix-devel"? > > Or is it needed for "guix graph"? No, guix graph uses its own graphviz implementation! It is used to generated png files from .dot files while building the documentation. I don't really know if we can skip graphical libraries for this reason. Best, -- Josselin Poiret [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 682 bytes --] ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: "guix pull" requires graphical libraries 2023-04-26 8:45 ` Josselin Poiret via Bug reports for GNU Guix @ 2023-04-26 16:59 ` Liliana Marie Prikler 2023-04-26 17:25 ` Andreas Enge 0 siblings, 1 reply; 30+ messages in thread From: Liliana Marie Prikler @ 2023-04-26 16:59 UTC (permalink / raw) To: Josselin Poiret, Andreas Enge, Ludovic Courtès; +Cc: 63050 Hi folks, just dropping by real quick Am Mittwoch, dem 26.04.2023 um 10:45 +0200 schrieb Josselin Poiret: > No, guix graph uses its own graphviz implementation! It is used to > generated png files from .dot files while building the documentation. > > I don't really know if we can skip graphical libraries for this > reason. Having built glib from scratch more often than is fun, I am quite certain that the package pulling in our graphics stack is texinfo with its reference to texlive. Cheers ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: "guix pull" requires graphical libraries 2023-04-26 16:59 ` Liliana Marie Prikler @ 2023-04-26 17:25 ` Andreas Enge 2023-04-26 18:39 ` Josselin Poiret via Bug reports for GNU Guix 0 siblings, 1 reply; 30+ messages in thread From: Andreas Enge @ 2023-04-26 17:25 UTC (permalink / raw) To: Liliana Marie Prikler; +Cc: Josselin Poiret, 63050, Ludovic Courtès Hello, Am Wed, Apr 26, 2023 at 06:59:44PM +0200 schrieb Liliana Marie Prikler: > Having built glib from scratch more often than is fun, I am quite > certain that the package pulling in our graphics stack is texinfo with > its reference to texlive. where do you see this? $ guix gc --references `guix build texinfo` /gnu/store/5j85qqflgx8nnzk86i43mxn0rjm8h2gv-perl-archive-zip-1.68 /gnu/store/930nwsiysdvy2x5zv1sf6v7ym75z8ayk-gcc-11.3.0-lib /gnu/store/a5i8avx826brw5grn3n4qv40g514505c-coreutils-9.1 /gnu/store/bcc053jvsbspdjr17gnnd9dg85b3a0gy-ncurses-6.2.20210619 /gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35 /gnu/store/hc05d76f1j3iz3v2bs5jz4fpljl1r4dj-gawk-5.2.1 /gnu/store/lj75fc25zx2y9pqvfp95la84rdhlj4f8-perl-5.36.0 /gnu/store/m8waimifhdjm8slb85jfihsm18jp1vc8-texinfo-7.0.3 /gnu/store/rib9g2ig1xf3kclyl076w28parmncg4k-bash-minimal-5.1.16 So texinfo should be fine. It is this explicit inclusion of graphviz that poses problems (glib is then pulled in also). Andreas ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: "guix pull" requires graphical libraries 2023-04-26 17:25 ` Andreas Enge @ 2023-04-26 18:39 ` Josselin Poiret via Bug reports for GNU Guix 2023-04-26 19:21 ` Andreas Enge 2023-04-26 19:34 ` Liliana Marie Prikler 0 siblings, 2 replies; 30+ messages in thread From: Josselin Poiret via Bug reports for GNU Guix @ 2023-04-26 18:39 UTC (permalink / raw) To: Andreas Enge, Liliana Marie Prikler; +Cc: Ludovic Courtès, 63050 [-- Attachment #1: Type: text/plain, Size: 591 bytes --] Hi Andreas, Andreas Enge <andreas@enge.fr> writes: > Hello, > > Am Wed, Apr 26, 2023 at 06:59:44PM +0200 schrieb Liliana Marie Prikler: >> Having built glib from scratch more often than is fun, I am quite >> certain that the package pulling in our graphics stack is texinfo with >> its reference to texlive. > > where do you see this? > $ guix gc --references `guix build texinfo` This would check the store path's references, but not necessarily all of its inputs! I would hope that no package with docs ever keeps references to texlive. Best, -- Josselin Poiret [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 682 bytes --] ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: "guix pull" requires graphical libraries 2023-04-26 18:39 ` Josselin Poiret via Bug reports for GNU Guix @ 2023-04-26 19:21 ` Andreas Enge 2023-04-26 19:34 ` Liliana Marie Prikler 1 sibling, 0 replies; 30+ messages in thread From: Andreas Enge @ 2023-04-26 19:21 UTC (permalink / raw) To: Josselin Poiret; +Cc: Ludovic Courtès, 63050, Liliana Marie Prikler Am Wed, Apr 26, 2023 at 08:39:59PM +0200 schrieb Josselin Poiret: > This would check the store path's references, but not necessarily all of > its inputs! I would hope that no package with docs ever keeps > references to texlive. Indeed! But here these are also the (native) inputs. Andreas ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: "guix pull" requires graphical libraries 2023-04-26 18:39 ` Josselin Poiret via Bug reports for GNU Guix 2023-04-26 19:21 ` Andreas Enge @ 2023-04-26 19:34 ` Liliana Marie Prikler 1 sibling, 0 replies; 30+ messages in thread From: Liliana Marie Prikler @ 2023-04-26 19:34 UTC (permalink / raw) To: Josselin Poiret, Andreas Enge; +Cc: Ludovic Courtès, 63050 Am Mittwoch, dem 26.04.2023 um 20:39 +0200 schrieb Josselin Poiret: > Hi Andreas, > > Andreas Enge <andreas@enge.fr> writes: > > > Hello, > > > > Am Wed, Apr 26, 2023 at 06:59:44PM +0200 schrieb Liliana Marie > > Prikler: > > > Having built glib from scratch more often than is fun, I am quite > > > certain that the package pulling in our graphics stack is texinfo > > > with > > > its reference to texlive. > > > > where do you see this? > > $ guix gc --references `guix build texinfo` > > This would check the store path's references, but not necessarily all > of its inputs! I would hope that no package with docs ever keeps > references to texlive. It does turn out there's no `guix graph texinfo --path-to texlive' either, though, so I was actually mistaken. Sorry for the noise ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: "guix pull" requires graphical libraries 2023-04-25 21:48 ` Ludovic Courtès 2023-04-26 7:28 ` Andreas Enge @ 2023-04-28 15:18 ` Simon Tournier 2023-05-03 19:33 ` Ludovic Courtès 2023-05-03 19:50 ` bug#63050: Reducing the closure size of Graphviz Ludovic Courtès 2 siblings, 1 reply; 30+ messages in thread From: Simon Tournier @ 2023-04-28 15:18 UTC (permalink / raw) To: Ludovic Courtès, Andreas Enge; +Cc: 63050 Hi, On mar., 25 avril 2023 at 23:48, Ludovic Courtès <ludo@gnu.org> wrote: > Maybe these are optional dependencies? Why does Guix require ’graphviz’ in the first place? Cheers, simon ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: "guix pull" requires graphical libraries 2023-04-28 15:18 ` Simon Tournier @ 2023-05-03 19:33 ` Ludovic Courtès 2023-05-04 8:56 ` Simon Tournier 0 siblings, 1 reply; 30+ messages in thread From: Ludovic Courtès @ 2023-05-03 19:33 UTC (permalink / raw) To: Simon Tournier; +Cc: Andreas Enge, 63050 Hi! Simon Tournier <zimon.toutoune@gmail.com> skribis: > On mar., 25 avril 2023 at 23:48, Ludovic Courtès <ludo@gnu.org> wrote: > >> Maybe these are optional dependencies? > > Why does Guix require ’graphviz’ in the first place? It uses it to build images in the manual. Ludo’. ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: "guix pull" requires graphical libraries 2023-05-03 19:33 ` Ludovic Courtès @ 2023-05-04 8:56 ` Simon Tournier 2023-05-05 15:21 ` Csepp 0 siblings, 1 reply; 30+ messages in thread From: Simon Tournier @ 2023-05-04 8:56 UTC (permalink / raw) To: Ludovic Courtès; +Cc: Andreas Enge, 63050 Hi, On Wed, 03 May 2023 at 21:33, Ludovic Courtès <ludo@gnu.org> wrote: >> Why does Guix require ’graphviz’ in the first place? > > It uses it to build images in the manual. Ah. So we are dragging X11 libraries as libx11 for one or two figures in the manual. :-) Although that’s not exactly the same as “guix pull”, guix graph guix -t bag-emerged gives an idea. Well, for example, there is a path from guix to ninja via graphviz. While I understand that the documentation is important, could we skip it for some architectures? Cheers, simon $ guix graph guix -t bag-emerged | grep label | cut -f2 -d'=' | cut -f1 -d',' | sort "acl@2.3.1" "attr@2.5.1" "autoconf@2.69" "autoconf-wrapper@2.69" "automake@1.16.3" "avahi@0.8" "bash@5.1.8" "bash-completion@2.8" "bash-minimal@5.1.8" "bash-minimal@5.1.8" "bash-static@5.1.8" "bdb@6.2.32" "binutils@2.37" "bison@3.7.6" "bison@3.7.6" "boost@1.77.0" "bzip2@1.0.8" "bzip2@1.0.8" "cairo@1.16.0" "cairo@1.16.0" "c-ares@1.17.2" "cmake-bootstrap@3.21.4" "cmake-minimal@3.21.4" "config@0.0.0-1.c8ddc84" "coreutils@8.32" "coreutils@8.32" "coreutils-minimal@8.32" "cunit@2.1-3" "curl@7.79.1" "datefudge@1.23" "dbus@1.12.20" "diffutils@3.8" "disarchive@0.4.0" "docbook-xml@4.1.2" "docbook-xml@4.4" "docbook-xsl@1.79.2" "doxygen@1.9.1" "expat@2.4.1" "file@5.39" "file@5.39" "findutils@4.8.0" "flex@2.6.4" "fontconfig-minimal@2.13.94" "font-dejavu@2.37" "fontforge@20201107" "font-ghostscript@8.11" "freetype@2.10.4" "fribidi@1.0.9" "gawk@5.1.0" "gawk@5.1.0" "gcc@10.3.0" "gd@2.3.2" "gdbm@1.20" "gettext-minimal@0.21" "ghostscript@9.54.0" "glib@2.70.2" "glibc@2.33" "glibc@2.33" "glibc-utf8-locales@2.33" "glibc-utf8-locales@2.33" "gmp@6.2.1" "gnutls@3.7.2" "gnutls@3.7.7" "gobject-introspection@1.66.1" "gperf@3.1" "graphite2@1.3.13" "graphviz@2.49.0" "grep@3.6" "grep@3.6" "gts@0.7.6" "guile@3.0.7" "guile@3.0.8" "guile-avahi@0.4.0-1.6d43caf" "guile-bytestructures@1.0.10" "guile-gcrypt@0.3.0" "guile-git@0.5.2" "guile-gnutls@3.7.9" "guile-json@4.7.1" "guile-lib@0.2.7" "guile-lzlib@0.0.2" "guile-lzma@0.1.1" "guile-quickcheck@0.1.0" "guile-sqlite3@0.1.3" "guile-ssh@0.15.1" "guile-zlib@0.1.0" "guile-zstd@0.1.1" "guix@1.3.0-31.3170843" "gzip@1.10" "gzip@1.10" "harfbuzz@2.8.2" "help2man@1.48.5" "http-parser@2.9.4-1.ec8b5ee" "icu4c@69.1" "intltool@0.51.0" "iproute2@5.15.0" "iptables@1.8.7" "itstool@2.0.6" "jansson@2.13.1" "jbig2dec@0.19" "jemalloc@5.2.1" "jsoncpp@1.9.4" "kmod@29" "lcms@2.12" "ld-wrapper@0" "libarchive@3.5.1" "libbsd@0.10.0" "libcap@2.62" "libdaemon@0.14" "libdatrie@0.2.13" "libdrm@2.4.107" "libelf@0.8.13" "libev@4.33" "libevent@2.1.12" "libffi@3.3" "libgc@8.0.4" "libgcrypt@1.8.8" "libgit2@1.3.0" "libgpg-error@1.42" "libice@1.0.10" "libidn@1.37" "libidn2@2.3.1" "libjpeg-turbo@2.0.5" "libltdl@2.4.6" "libmnl@1.0.4" "libnftnl@1.2.0" "libpaper@1.1.24" "libpciaccess@0.16" "libpng@1.6.37" "libpthread-stubs@0.4" "libsigsegv@2.13" "libsm@1.2.3" "libspectre@0.2.9" "libspiro@20200505" "libssh@0.9.6" "libssh2@1.9.0" "libtasn1@4.17.0" "libthai@0.1.28" "libtiff@4.3.0" "libtool@2.4.6" "libungif@4.1.4" "libuninameslist@20200313" "libunistring@0.9.10" "libuv@1.41.1" "libx11@1.7.3.1" "libxau@1.0.9" "libxaw@1.0.14" "libxcb@1.14" "libxdmcp@1.1.3" "libxext@1.3.4" "libxfixes@6.0.0" "libxft@2.3.3" "libxi@1.7.10" "libxml2@2.9.12" "libxmu@1.1.3" "libxpm@3.5.13" "libxrender@0.9.10" "libxslt@1.1.34" "libxt@1.2.1" "linux-libre-headers@5.10.35" "lzlib@1.13" "lzo@2.10" "m4@1.4.18" "make@4.3" "mallard-ducktype@1.0.2" "meson@0.60.3" "mit-krb5@1.19.2" "mpfr@4.1.0" "nasm@2.15.05" "ncurses@6.2.20210619" "net-base@5.3" "nettle@3.7.3" "net-tools@1.60-0.479bb4a" "nghttp2@1.44.0" "ninja@1.10.2" "openjpeg@2.4.0" "openjpeg-data@2020.11.30" "openssl@1.1.1l" "p11-kit@0.23.22" "pango@1.48.10" "patch@2.7.6" "pciutils@3.7.0" "pcre2@10.37" "pcre@8.45" "perl@5.34.0" "perl-common-sense@3.75" "perl-cpanel-json-xs@4.30" "perl-cpan-meta@2.150010" "perl-cpan-meta-requirements@2.140" "perl-cpan-meta-yaml@0.018" "perl-extutils-config@0.008" "perl-extutils-helpers@0.026" "perl-extutils-installpaths@0.012" "perl-gettext@1.07" "perl-json-maybexs@1.004003" "perl-module-build@0.4231" "perl-module-build-tiny@0.039" "perl-parse-cpan-meta@2.150010" "perl-pod-parser@1.65" "perl-test-harness@3.42" "perl-test-needs@0.002009" "perl-test-pod@1.52" "perl-xml-parser@2.46" "perl-yaml-tiny@1.73" "pixman@0.40.0" "pkg-config@0.29.2" "po4a@0.63" "poppler@21.07.0" "potrace@1.16" "python@3.9.9" "python-fonttools@4.28.5" "python-libxml2@2.9.12" "python-minimal@3.9.9" "python-minimal-wrapper@3.9.9" "python-wrapper@3.9.9" "readline@8.1.1" "rhash@1.4.2" "ruby@2.7.4" "ruby-hydra-minimal@0.0-0.5abfa37" "sed@4.8" "sed@4.8" "socat@1.7.4.1" "sqlite@3.36.0" "swig@4.0.2" "tar@1.34" "tar@1.34" "tcl@8.6.11" "tcsh@6.22.03" "teckit@2.5.10" "texinfo@6.7" "texlive-amscls@59745" "texlive-amsmath@59745" "texlive-babel@59745" "texlive-bin@20210325" "texlive-cm@59745" "texlive-cm-super@59745" "texlive-dehyph-exptl@59745" "texlive-docstrip@59745" "texlive-dvips@59745" "texlive-etex@59745" "texlive-fontname@59745" "texlive-fonts-latex@59745" "texlive-generic-babel-english@59745" "texlive-graphics-cfg@59745" "texlive-graphics-def@59745" "texlive-hyphen-afrikaans@59745" "texlive-hyphen-ancientgreek@59745" "texlive-hyphen-armenian@59745" "texlive-hyphen-base@59745" "texlive-hyphen-basque@59745" "texlive-hyphen-belarusian@59745" "texlive-hyphen-bulgarian@59745" "texlive-hyphen-catalan@59745" "texlive-hyphen-chinese@59745" "texlive-hyphen-churchslavonic@59745" "texlive-hyphen-coptic@59745" "texlive-hyphen-croatian@59745" "texlive-hyphen-czech@59745" "texlive-hyphen-danish@59745" "texlive-hyphen-dutch@59745" "texlive-hyphen-english@59745" "texlive-hyphen-esperanto@59745" "texlive-hyphen-estonian@59745" "texlive-hyphen-ethiopic@59745" "texlive-hyphen-finnish@59745" "texlive-hyphen-french@59745" "texlive-hyphen-friulan@59745" "texlive-hyphen-galician@59745" "texlive-hyphen-georgian@59745" "texlive-hyphen-german@59745" "texlive-hyphen-greek@59745" "texlive-hyphen-hungarian@59745" "texlive-hyphen-icelandic@59745" "texlive-hyphen-indic@59745" "texlive-hyphen-indonesian@59745" "texlive-hyphen-interlingua@59745" "texlive-hyphen-irish@59745" "texlive-hyphen-italian@59745" "texlive-hyphen-kurmanji@59745" "texlive-hyphen-latin@59745" "texlive-hyphen-latvian@59745" "texlive-hyphen-lithuanian@59745" "texlive-hyphen-macedonian@59745" "texlive-hyphen-mongolian@59745" "texlive-hyphen-norwegian@59745" "texlive-hyphen-occitan@59745" "texlive-hyphen-pali@59745" "texlive-hyphen-piedmontese@59745" "texlive-hyphen-polish@59745" "texlive-hyphen-portuguese@59745" "texlive-hyphen-romanian@59745" "texlive-hyphen-romansh@59745" "texlive-hyphen-russian@59745" "texlive-hyphen-sanskrit@59745" "texlive-hyphen-schoolfinnish@59745" "texlive-hyphen-serbian@59745" "texlive-hyphen-slovak@59745" "texlive-hyphen-slovenian@59745" "texlive-hyphen-spanish@59745" "texlive-hyphen-swedish@59745" "texlive-hyphen-thai@59745" "texlive-hyphen-turkish@59745" "texlive-hyphen-turkmen@59745" "texlive-hyphen-ukrainian@59745" "texlive-hyphen-uppersorbian@59745" "texlive-hyphen-welsh@59745" "texlive-hyph-utf8@59745" "texlive-knuth-lib@59745" "texlive-kpathsea@59745" "texlive-latex-base@59745" "texlive-latexconfig@59745" "texlive-latex-cyrillic@59745" "texlive-latex-epstopdf-pkg@59745" "texlive-latex-graphics@59745" "texlive-latex-l3backend@59745" "texlive-latex-l3kernel@59745" "texlive-latex-l3packages@59745" "texlive-latex-tools@59745" "texlive-metafont@59745" "texlive-psnfss@59745" "texlive-ruhyphen@59745" "texlive-tetex@59745" "texlive-tex-ini-files@59745" "texlive-tex-plain@59745" "texlive-tiny@59745" "texlive-ukrhyph@59745" "texlive-unicode-data@59745" "tk@8.6.11.1" "tzdata@2022a" "unzip@6.0" "util-linux@2.37.2" "util-macros@1.19.3" "which@2.21" "xcb-proto@1.14" "xmlto@0.0.28" "xorgproto@2021.5" "xtrans@1.4.0" "xz@5.2.5" "xz@5.2.5" "yelp-tools@3.32.2" "yelp-xsl@41.0" "zip@3.0" "zlib@1.2.11" "zstd@1.5.0" "zziplib@0.13.72" ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: "guix pull" requires graphical libraries 2023-05-04 8:56 ` Simon Tournier @ 2023-05-05 15:21 ` Csepp 2023-05-09 12:36 ` Simon Tournier 0 siblings, 1 reply; 30+ messages in thread From: Csepp @ 2023-05-05 15:21 UTC (permalink / raw) To: Simon Tournier; +Cc: ludo, 63050, andreas Simon Tournier <zimon.toutoune@gmail.com> writes: > Hi, > > On Wed, 03 May 2023 at 21:33, Ludovic Courtès <ludo@gnu.org> wrote: > >>> Why does Guix require ’graphviz’ in the first place? >> >> It uses it to build images in the manual. > > Ah. So we are dragging X11 libraries as libx11 for one or two figures > in the manual. :-) > > Although that’s not exactly the same as “guix pull”, > > guix graph guix -t bag-emerged > > gives an idea. Well, for example, there is a path from guix to ninja > via graphviz. > > While I understand that the documentation is important, could we skip it > for some architectures? Or just move it to a separate output or package? That should really be something done for all packages automatically tbh. Alpine gets this right. ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: "guix pull" requires graphical libraries 2023-05-05 15:21 ` Csepp @ 2023-05-09 12:36 ` Simon Tournier 2023-05-11 21:30 ` Csepp 0 siblings, 1 reply; 30+ messages in thread From: Simon Tournier @ 2023-05-09 12:36 UTC (permalink / raw) To: Csepp; +Cc: ludo, 63050, andreas Hi, On ven., 05 mai 2023 at 15:21, Csepp <raingloom@riseup.net> wrote: > Or just move it to a separate output or package? That should really be > something done for all packages automatically tbh. Alpine gets this right. Well, I do not think a separate output would be possible and we are not talking about the package named ’guix’ but about what is implemented by the module (guix self). Somehow, I agree that one direction would to make optional some features. The current proposal for tackling this issue is the reduction of the closure by removing lix11 and libxrender as discussed in [1]. 1: https://issues.guix.gnu.org/msgid/874jot19fd.fsf_-_@gnu.org Cheers, simon ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: "guix pull" requires graphical libraries 2023-05-09 12:36 ` Simon Tournier @ 2023-05-11 21:30 ` Csepp 0 siblings, 0 replies; 30+ messages in thread From: Csepp @ 2023-05-11 21:30 UTC (permalink / raw) To: Simon Tournier; +Cc: ludo, 63050, Csepp, andreas Simon Tournier <zimon.toutoune@gmail.com> writes: > Hi, > > On ven., 05 mai 2023 at 15:21, Csepp <raingloom@riseup.net> wrote: > >> Or just move it to a separate output or package? That should really be >> something done for all packages automatically tbh. Alpine gets this right. > > Well, I do not think a separate output would be possible and we are not > talking about the package named ’guix’ but about what is implemented by > the module (guix self). > > Somehow, I agree that one direction would to make optional some > features. The current proposal for tackling this issue is the reduction > of the closure by removing lix11 and libxrender as discussed in [1]. > > 1: https://issues.guix.gnu.org/msgid/874jot19fd.fsf_-_@gnu.org > > > Cheers, > simon It should be made possible IMHO. It's nice that our packages come with docs, including Guix, but they are often unnecessary. If an output won't work because guix-self is special, then maybe it could be moved to a separate package. ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: Reducing the closure size of Graphviz 2023-04-25 21:48 ` Ludovic Courtès 2023-04-26 7:28 ` Andreas Enge 2023-04-28 15:18 ` Simon Tournier @ 2023-05-03 19:50 ` Ludovic Courtès 2023-05-04 9:00 ` Simon Tournier 2023-05-20 16:12 ` bug#63050: "guix pull" requires graphical libraries Ludovic Courtès 2 siblings, 2 replies; 30+ messages in thread From: Ludovic Courtès @ 2023-05-03 19:50 UTC (permalink / raw) To: Andreas Enge; +Cc: 63050 [-- Attachment #1: Type: text/plain, Size: 731 bytes --] Hey! Ludovic Courtès <ludo@gnu.org> skribis: > This is apparently coming from Graphviz: > > $ guix graph --path guix libx11 > guix@1.4.0-5.286cdf0 > graphviz@2.49.0 > libx11@1.7.3.1 > $ guix graph --path guix libxt > guix@1.4.0-5.286cdf0 > graphviz@2.49.0 > libxaw@1.0.14 > libxt@1.2.1 > > Surprising to me, but apparently it’s been this way from the start, > commit b1b07d72c755ea314fb0c8333cd88293ee504ce4 (2013!). > > Maybe these are optional dependencies? All the X libraries can be seen in the output of: ldd $(guix build graphviz |grep -v 'doc$')/lib/graphviz/libgvplugin_xlib.so I haven’t checked but I suppose that’s used by ‘xdot’. We can get an X11-free Graphviz like so: [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: Type: text/x-patch, Size: 669 bytes --] diff --git a/gnu/packages/graphviz.scm b/gnu/packages/graphviz.scm index 26ee96afd4..3a5d33e662 100644 --- a/gnu/packages/graphviz.scm +++ b/gnu/packages/graphviz.scm @@ -94,16 +94,12 @@ (define-public graphviz (string-append extdir "/libgv_guile.so")))))))) (inputs - (list libxrender - libx11 - gts + (list gts gd guile-3.0 ;Guile bindings - pango fontconfig freetype libltdl - libxaw expat libjpeg-turbo libpng)) [-- Attachment #3: Type: text/plain, Size: 390 bytes --] The closure size reduction is substantial: --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix size graphviz | tail -1 total: 183.6 MiB $ guix size graphviz | tail -1 total: 242.3 MiB --8<---------------cut here---------------end--------------->8--- But I suspect we’d still need the full-blown variant for things like xdot. Ludo’. ^ permalink raw reply related [flat|nested] 30+ messages in thread
* bug#63050: Reducing the closure size of Graphviz 2023-05-03 19:50 ` bug#63050: Reducing the closure size of Graphviz Ludovic Courtès @ 2023-05-04 9:00 ` Simon Tournier 2023-05-20 16:12 ` bug#63050: "guix pull" requires graphical libraries Ludovic Courtès 1 sibling, 0 replies; 30+ messages in thread From: Simon Tournier @ 2023-05-04 9:00 UTC (permalink / raw) To: Ludovic Courtès, Andreas Enge; +Cc: 63050 Hi, On Wed, 03 May 2023 at 21:50, Ludovic Courtès <ludo@gnu.org> wrote: > - (list libxrender > - libx11 > - gts > + (list gts > gd > guile-3.0 ;Guile bindings > - pango > fontconfig > freetype > libltdl > - libxaw > expat > libjpeg-turbo > libpng)) Ah that’s better than my proposal elsewhere. ;-) > --8<---------------cut here---------------start------------->8--- > $ ./pre-inst-env guix size graphviz | tail -1 > total: 183.6 MiB > $ guix size graphviz | tail -1 > total: 242.3 MiB > --8<---------------cut here---------------end--------------->8--- > > But I suspect we’d still need the full-blown variant for things like > xdot. Yeah, we could have graphviz (with libx11) and graphviz-minimal (without libx11) and make Guix depends on graphviz-minimal. WDYT? Cheers, simon ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: "guix pull" requires graphical libraries 2023-05-03 19:50 ` bug#63050: Reducing the closure size of Graphviz Ludovic Courtès 2023-05-04 9:00 ` Simon Tournier @ 2023-05-20 16:12 ` Ludovic Courtès 2023-05-20 16:38 ` Andreas Enge 1 sibling, 1 reply; 30+ messages in thread From: Ludovic Courtès @ 2023-05-20 16:12 UTC (permalink / raw) To: Andreas Enge; +Cc: 63050, Simon Tournier Hi! Ludovic Courtès <ludo@gnu.org> skribis: > We can get an X11-free Graphviz like so: > > diff --git a/gnu/packages/graphviz.scm b/gnu/packages/graphviz.scm > index 26ee96afd4..3a5d33e662 100644 > --- a/gnu/packages/graphviz.scm > +++ b/gnu/packages/graphviz.scm > @@ -94,16 +94,12 @@ (define-public graphviz > (string-append extdir > "/libgv_guile.so")))))))) > (inputs > - (list libxrender > - libx11 > - gts > + (list gts > gd > guile-3.0 ;Guile bindings > - pango > fontconfig > freetype > libltdl > - libxaw > expat > libjpeg-turbo > libpng)) > > > The closure size reduction is substantial: > > $ ./pre-inst-env guix size graphviz | tail -1 > total: 183.6 MiB > $ guix size graphviz | tail -1 > total: 242.3 MiB > > But I suspect we’d still need the full-blown variant for things like > xdot. Here’s a proposal: https://issues.guix.gnu.org/63610 Ludo’. ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: "guix pull" requires graphical libraries 2023-05-20 16:12 ` bug#63050: "guix pull" requires graphical libraries Ludovic Courtès @ 2023-05-20 16:38 ` Andreas Enge 2023-05-24 13:10 ` Ludovic Courtès 0 siblings, 1 reply; 30+ messages in thread From: Andreas Enge @ 2023-05-20 16:38 UTC (permalink / raw) To: Ludovic Courtès; +Cc: 63050, Simon Tournier Am Sat, May 20, 2023 at 06:12:47PM +0200 schrieb Ludovic Courtès: > > The closure size reduction is substantial: > > $ ./pre-inst-env guix size graphviz | tail -1 > > total: 183.6 MiB > > $ guix size graphviz | tail -1 > > total: 242.3 MiB > > But I suspect we’d still need the full-blown variant for things like > > xdot. > Here’s a proposal: > https://issues.guix.gnu.org/63610 Typo? The issue is not found. Note that I do not care so much about the closure size, but about the number of packages that are needed to just build guix (although of course the two are related). Or otherwise said, the dependencies for "guix pull". On "exotic" architectures, each dependency is a potential cause of failure, and all in all it may take hours (days?) to run "guix pull" without substitutes, with a high chance of failure. Andreas ^ permalink raw reply [flat|nested] 30+ messages in thread
* bug#63050: "guix pull" requires graphical libraries 2023-05-20 16:38 ` Andreas Enge @ 2023-05-24 13:10 ` Ludovic Courtès 2023-05-25 18:24 ` How many bytes do we add (closure of guix) when adding one new package? Simon Tournier 0 siblings, 1 reply; 30+ messages in thread From: Ludovic Courtès @ 2023-05-24 13:10 UTC (permalink / raw) To: Andreas Enge; +Cc: 63050, Simon Tournier Hi, Andreas Enge <andreas@enge.fr> skribis: > Am Sat, May 20, 2023 at 06:12:47PM +0200 schrieb Ludovic Courtès: >> > The closure size reduction is substantial: >> > $ ./pre-inst-env guix size graphviz | tail -1 >> > total: 183.6 MiB >> > $ guix size graphviz | tail -1 >> > total: 242.3 MiB >> > But I suspect we’d still need the full-blown variant for things like >> > xdot. >> Here’s a proposal: >> https://issues.guix.gnu.org/63610 > > Typo? The issue is not found. Typo on your side then? :-) > Note that I do not care so much about the closure size, but about the > number of packages that are needed to just build guix (although of course > the two are related). Or otherwise said, the dependencies for "guix pull". Yes, understood. Graphviz is not in the closure anyway, it’s a build-only dependency. With commit 9fa92acbf0c4dbc734ac7d83b31bd6d12e09a401 this is mostly fixed. There’s still another path leading to libx11 though: --8<---------------cut here---------------start------------->8--- $ ./pre-inst-env guix graph --path guix libx11 guix@1.4.0-6.dc5430c guile-avahi@0.4.1 avahi@0.8 dbus@1.14.0 libx11@1.8.1 --8<---------------cut here---------------end--------------->8--- (The same applies to “guix pull”.) Not sure what can be done about it. Ludo’. ^ permalink raw reply [flat|nested] 30+ messages in thread
* How many bytes do we add (closure of guix) when adding one new package? 2023-05-24 13:10 ` Ludovic Courtès @ 2023-05-25 18:24 ` Simon Tournier 2023-05-26 16:21 ` Ludovic Courtès 0 siblings, 1 reply; 30+ messages in thread From: Simon Tournier @ 2023-05-25 18:24 UTC (permalink / raw) To: Guix Devel; +Cc: Ludovic Courtès, Andreas Enge Hi, The initial discussion was about the closure of Guix and that “guix pull” brings graphical libraries. See #63050 [1]. Here, I would like to open a discussion about how Guix scales, i.e. about its size. I am trying to answer the question I am asking as the subject. ;-) It’s another angle to view Andreas and Ludo discussion: :-) >> Note that I do not care so much about the closure size, but about the >> number of packages that are needed to just build guix (although of course >> the two are related). Or otherwise said, the dependencies for "guix pull". > > Yes, understood. Graphviz is not in the closure anyway, it’s a > build-only dependency. Somehow, the closure is increasing: --8<---------------cut here---------------start------------->8--- $ for i in $(seq 1 4); do guix time-machine --commit=v1.$i.0 -- size guix | grep 'total:' ;done total: 410.9 MiB total: 496.0 MiB total: 564.8 MiB total: 637.2 MiB $ guix size guix | grep 'total:' total: 611.2 MiB --8<---------------cut here---------------end--------------->8--- (Yeah, the package guix is not exactly the same as guix itself, but it appears to me a good enough approximation. And my current revision is 14c0380.) Compare: --8<---------------cut here---------------start------------->8--- $ guix time-machine --commit=v1.1.0 -- size guix --sort=self | wc -l 44 $ guix time-machine --commit=v1.4.0 -- size guix --sort=self | wc -l 72 $ guix size guix --sort=self | wc -l 70 --8<---------------cut here---------------end--------------->8--- which is the Andreas’s concern for “exotic” architectures. Moreover, the inflation (in size) is about some packages that are just becoming bigger. --8<---------------cut here---------------start------------->8--- $ guix time-machine --commit=v1.1.0 -- size guix --sort=self | head store item total self /gnu/store/fp16m5hkzql7jwhvnkm1j1i5qch0arhx-guix-1.1.0rc2-1.9d0d27f 410.9 221.6 53.9% /gnu/store/1mkkv2caiqbdbbd256c4dirfi4kwsacv-guile-2.2.6 123.9 44.4 10.8% /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29 37.4 35.8 8.7% /gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib 70.0 32.6 7.9% /gnu/store/n79cf8bvy3k96gjk1rf18d36w40lkwlr-glibc-utf8-locales-2.29 13.9 13.9 3.4% /gnu/store/k2m4q2av9hw73hw2jx6qrxqdyh855398-openssl-1.1.1c 76.4 6.4 1.6% /gnu/store/gzp4ig4rdb1qf4i5dy1d9nl0zmj5q09y-ncurses-6.1-20190609 75.9 5.9 1.4% /gnu/store/hfvz18igm68p5yz7z4asn6ph363blp1z-gnutls-3.6.9 130.6 5.1 1.2% /gnu/store/b5vjmib411m74lbpf051fnwz3s9zcw79-guile-git-0.3.0 98.8 4.4 1.1% $ guix time-machine --commit=v1.4.0 -- size guix --sort=self | head store item total self /gnu/store/9nvx97hr8kkr26gzwni2fblfn0yq0xjw-guix-1.4.0rc2 637.2 330.1 51.8% /gnu/store/qlmpcy5zi84m6dikq3fnx5dz38qpczlc-guile-3.0.8 130.0 53.0 8.3% /gnu/store/cnfsv9ywaacyafkqdqsv2ry8f01yr7a9-guile-3.0.7 129.1 52.0 8.2% /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33 38.3 36.6 5.7% /gnu/store/094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib 71.7 33.4 5.2% /gnu/store/96srhmpmxa20wmsck95g3iq4hb3lz4a0-glib-2.70.2 98.1 15.3 2.4% /gnu/store/mw3py6smb1pk8yx298hd9ivz9lzbksqi-glibc-utf8-locales-2.33 13.9 13.9 2.2% /gnu/store/5583c2za2jsn9g6az79rnksgvigwnsk7-util-linux-2.37.2-lib 80.7 9.0 1.4% /gnu/store/9rrnm5hdjw7cy96a2a9rfgh6y08wsbmf-ncurses-6.2.20210619 77.6 5.9 0.9% $ guix size guix --sort=self | head store item total self /gnu/store/cgjddvw9zay626z8hyxl0zmn1354c24k-guix-1.4.0-6.dc5430c 611.2 350.2 57.3% /gnu/store/4gvgcfdiz67wv04ihqfa8pqwzsb0qpv5-guile-3.0.9 135.0 53.1 8.7% /gnu/store/gsjczqir1wbz8p770zndrpw4rnppmxi3-glibc-2.35 40.6 38.8 6.3% /gnu/store/930nwsiysdvy2x5zv1sf6v7ym75z8ayk-gcc-11.3.0-lib 75.3 34.7 5.7% /gnu/store/nb40pwd37v6i1g4b1fq4l6q4h9px3asr-glib-2.72.3 101.3 14.9 2.4% /gnu/store/5fmqijrs5f7vx8mc2q2pmq26yvhb74sm-glibc-utf8-locales-2.35 13.9 13.9 2.3% /gnu/store/gwx2sf5wl9bsl21lwv35g5la63bwyy95-util-linux-2.37.4-lib 84.3 9.0 1.5% /gnu/store/69wd3pd1hd3j84xr965jj2fk2qmxn0hl-openssl-3.0.8 83.4 8.1 1.3% /gnu/store/bcc053jvsbspdjr17gnnd9dg85b3a0gy-ncurses-6.2.20210619 81.2 5.9 1.0% --8<---------------cut here---------------end--------------->8--- Considering Guix itself, one explanation for the increase is the number of packages – assuming the services and other are negligible; “git diff --shortstat” is a good indicator at first sight. Well, we could be more precise about the documentation. Hum, this ugly, --8<---------------cut here---------------start------------->8--- $ for doc in $(for ci in $(for t in $(git tag | grep v1 | grep -v rc ); do git --no-pager show $t | grep commit ;done); do for d in $(find ~/.cache/guix/inferiors/ -type l -print); do printf "$d "; $d/bin/guix --version 2>/dev/null ;done | grep $ci ;done | cut -f1 -d' '); do du -sh $(readlink -f $doc)/share/* ;done | grep info 172K /gnu/store/5pa1706ckwhn6x4mn5kl2b7h15k3in9x-profile/share/info 200K /gnu/store/z1icpkfbz59dr7k7rnb0jd8j1ii8mdph-profile/share/info 376K /gnu/store/hm0rwgcvrs85y3hgjsw8616cxy61h6si-profile/share/info 304K /gnu/store/zbrgzk7l0j7805i82sl3gmx6y2b0iz9q-profile/share/info --8<---------------cut here---------------end--------------->8--- is probably providing a clue about the assumption. Ok, let assume that the packages are the main source of size increasing. The question is: can we evaluate the size for one package? How many bytes do we add to the whole Guix when we add one package? On average and roughly. We have the number of packages and the whole size for successive versions. Therefore, we can do the difference between the two. We get [2078, 1848, 4704, 1532] which means 2078 packages had been added between v1.1.0 and v1.2.0, 1848 between v1.2.0 and v1.3.0, etc. We can do the same for the size, [22.9, 19.4, 66.2, 20.1] and then we can compute the ratio: the size per package. Something like: [0.011020211742059676, 0.010497835497835485, 0.01407312925170069, 0.013120104438642276] Let get an average: 0.012177820232559531. Now, let take the number of packages for v1.1.0 and do the multiplication. We get: 159.6 MiB. Ok, it means that the difference is more or less the core of Guix – what we are assuming that is slowly growing. It reads 62 MiB. Therefore, we can predict the size for the other versions using this linear model based on the previous evaluated average. size = mean * number_packages + core It reads: [246.9, 269.4, 326.7, 345.3] compared to [244.5, 263.9, 330.1, 350.2]. Hum, this quick back-to-the-envelope computation does not seem too bad. I guess. Conclusions: 1. the addition of one package leads to an increase of ~ 12 KiB 2. the core of Guix is about ~ 62 MiB 3. doubling the number of packages is doubling the size to download at “guix pull” time. Maybe, we should re-think (guix self). Especially the *package-modules* part and re-discuss if we could split that part. From my understanding. Cheers, simon 1: https://issues.guix.gnu.org/issue/63050 ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: How many bytes do we add (closure of guix) when adding one new package? 2023-05-25 18:24 ` How many bytes do we add (closure of guix) when adding one new package? Simon Tournier @ 2023-05-26 16:21 ` Ludovic Courtès 2023-05-30 12:10 ` Simon Tournier 0 siblings, 1 reply; 30+ messages in thread From: Ludovic Courtès @ 2023-05-26 16:21 UTC (permalink / raw) To: Simon Tournier; +Cc: Guix Devel, Andreas Enge Hello! Thanks for the detailed analysis! Simon Tournier <zimon.toutoune@gmail.com> skribis: > Conclusions: > > 1. the addition of one package leads to an increase of ~ 12 KiB > > 2. the core of Guix is about ~ 62 MiB > > 3. doubling the number of packages is doubling the size to download at > “guix pull” time. I agree that .go files are quite big (.scm files as well, but we’ve improved information density somewhat by removing input labels :-)). The size of .go files went down when we switch to the baseline compiler (aka. -O1): https://lists.gnu.org/archive/html/guix-devel/2020-06/msg00071.html That thread has ideas of things to do to further reduce .go size. Download size has to be treated separately though. For example, ‘git pull’ doesn’t redownload all of the repo or directory, and it uses compression heavily. Thus, a few hundred bytes of additional .scm text translate in less than that. As for the rest, download size can be reduced for example by choosing a content-address transport, like something based on ERIS. I think we must look precisely at what we want to optimize—on-disk size, or bandwidth requirement, in particular—and look at the whole solution space. My 2¢! Ludo’. ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: How many bytes do we add (closure of guix) when adding one new package? 2023-05-26 16:21 ` Ludovic Courtès @ 2023-05-30 12:10 ` Simon Tournier 2023-05-30 19:10 ` Csepp 2023-05-30 20:55 ` How many bytes do we add (closure of guix) when adding one new package? Jack Hill 0 siblings, 2 replies; 30+ messages in thread From: Simon Tournier @ 2023-05-30 12:10 UTC (permalink / raw) To: Ludovic Courtès; +Cc: Guix Devel, Andreas Enge Hi, On ven., 26 mai 2023 at 18:21, Ludovic Courtès <ludo@gnu.org> wrote: > I agree that .go files are quite big (.scm files as well, but we’ve > improved information density somewhat by removing input labels :-)). > > The size of .go files went down when we switch to the baseline compiler > (aka. -O1): > > https://lists.gnu.org/archive/html/guix-devel/2020-06/msg00071.html > > That thread has ideas of things to do to further reduce .go size. Just to put a figure on what means “big”: currently the .go files are 5 times bigger than their associated .scm. Somehow, it’s the trap of DSL. :-) Packages are declarative and the information they declare is not dense. However, because they are bytecompiled to a general programming language, their specificity is not exploited. In an ideal world, the compiled binary representation of the packages should be smaller than their human-readable text-file counterpart. The mentioned improvement is nice. And it’s visible: --8<---------------cut here---------------start------------->8--- 145M /gnu/store/nqrb3g4l59wd74w8mr9v0b992bj2sd1w-guix-d62c9b267-modules/lib/guile/3.0/site-ccache/gnu 117M /gnu/store/s6rqlhqr750k44ynkqqj5mwjj2cs2yln-guix-a09968565-modules/lib/guile/3.0/site-ccache/gnu 127M /gnu/store/ndii4bpyzh2rc05ya61s89rig9hdrl4k-guix-a0178d34f-modules/lib/guile/3.0/site-ccache/gnu 164M /gnu/store/ni63a203jf61dwxlv8kr9b8x3vb1pdsp-guix-8e2f32cee-modules/lib/guile/3.0/site-ccache/gnu --8<---------------cut here---------------end--------------->8--- However, it has almost no impact on the whole size; scaled by the number of packages. > Download size has to be treated separately though. For example, ‘git > pull’ doesn’t redownload all of the repo or directory, and it uses > compression heavily. Thus, a few hundred bytes of additional .scm text > translate in less than that. > > As for the rest, download size can be reduced for example by choosing a > content-address transport, like something based on ERIS. > > I think we must look precisely at what we want to optimize—on-disk size, > or bandwidth requirement, in particular—and look at the whole solution > space. I think one direction is to tackle the way *package-modules* is built. Because of that, Guix is building too much and the design is not optimal – whatever technical solutions we implement for improving after that. On my poor laptop, Guix is becoming unusable because many operations are becoming so slow – when it’s still acceptable with APT of Debian. For instance, it’s something like 20 minutes for running “guix pull” without substitutes. And when I am traveling without a fast Internet connection, it’s often too much for the network at hand. Currently, “guix pull” is either building too much and downloading too much; by design. Cheers, simon ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: How many bytes do we add (closure of guix) when adding one new package? 2023-05-30 12:10 ` Simon Tournier @ 2023-05-30 19:10 ` Csepp 2023-05-31 8:05 ` Faster “guix search” (was Re: How many bytes do we add (closure of guix) when adding one new package?) Simon Tournier 2023-05-30 20:55 ` How many bytes do we add (closure of guix) when adding one new package? Jack Hill 1 sibling, 1 reply; 30+ messages in thread From: Csepp @ 2023-05-30 19:10 UTC (permalink / raw) To: Simon Tournier; +Cc: Ludovic Courtès, Andreas Enge, guix-devel Simon Tournier <zimon.toutoune@gmail.com> writes: > Hi, > > On ven., 26 mai 2023 at 18:21, Ludovic Courtès <ludo@gnu.org> wrote: > >> I agree that .go files are quite big (.scm files as well, but we’ve >> improved information density somewhat by removing input labels :-)). >> >> The size of .go files went down when we switch to the baseline compiler >> (aka. -O1): >> >> https://lists.gnu.org/archive/html/guix-devel/2020-06/msg00071.html >> >> That thread has ideas of things to do to further reduce .go size. > > Just to put a figure on what means “big”: currently the .go files are 5 > times bigger than their associated .scm. > > Somehow, it’s the trap of DSL. :-) Packages are declarative and the > information they declare is not dense. However, because they are > bytecompiled to a general programming language, their specificity is not > exploited. In an ideal world, the compiled binary representation of the > packages should be smaller than their human-readable text-file > counterpart. > > The mentioned improvement is nice. And it’s visible: > > --8<---------------cut here---------------start------------->8--- > 145M /gnu/store/nqrb3g4l59wd74w8mr9v0b992bj2sd1w-guix-d62c9b267-modules/lib/guile/3.0/site-ccache/gnu > 117M /gnu/store/s6rqlhqr750k44ynkqqj5mwjj2cs2yln-guix-a09968565-modules/lib/guile/3.0/site-ccache/gnu > 127M /gnu/store/ndii4bpyzh2rc05ya61s89rig9hdrl4k-guix-a0178d34f-modules/lib/guile/3.0/site-ccache/gnu > 164M /gnu/store/ni63a203jf61dwxlv8kr9b8x3vb1pdsp-guix-8e2f32cee-modules/lib/guile/3.0/site-ccache/gnu > --8<---------------cut here---------------end--------------->8--- > > However, it has almost no impact on the whole size; scaled by the number > of packages. > >> Download size has to be treated separately though. For example, ‘git >> pull’ doesn’t redownload all of the repo or directory, and it uses >> compression heavily. Thus, a few hundred bytes of additional .scm text >> translate in less than that. >> >> As for the rest, download size can be reduced for example by choosing a >> content-address transport, like something based on ERIS. >> >> I think we must look precisely at what we want to optimize—on-disk size, >> or bandwidth requirement, in particular—and look at the whole solution >> space. > > I think one direction is to tackle the way *package-modules* is built. > Because of that, Guix is building too much and the design is not optimal > – whatever technical solutions we implement for improving after that. > > On my poor laptop, Guix is becoming unusable because many operations are > becoming so slow – when it’s still acceptable with APT of Debian. For > instance, it’s something like 20 minutes for running “guix pull” without > substitutes. And when I am traveling without a fast Internet > connection, it’s often too much for the network at hand. > > Currently, “guix pull” is either building too much and downloading too > much; by design. > > > Cheers, > simon Something I've been considering is if Guix could make use of database optimizations on its packages. Having access to Scheme for everything is nice, but using it as a storage solution is kind of silly when we are mostly just storing structs. Some kind of struct-of-arrays optimization could definitely reduce their size by a lot, might even speed up some operations. It makes zero sense to load full package definitions from disk for most queries, such as guix search, with an SoA representation we could load only the fields that we care about. ps.: Now I'm even more glad that I'm using a file system with transparent compression on all my Guix systems. ^ permalink raw reply [flat|nested] 30+ messages in thread
* Faster “guix search” (was Re: How many bytes do we add (closure of guix) when adding one new package?) 2023-05-30 19:10 ` Csepp @ 2023-05-31 8:05 ` Simon Tournier 2023-05-31 11:10 ` Csepp 0 siblings, 1 reply; 30+ messages in thread From: Simon Tournier @ 2023-05-31 8:05 UTC (permalink / raw) To: Csepp; +Cc: Ludovic Courtès, Andreas Enge, guix-devel Hi, On Tue, 30 May 2023 at 21:10, Csepp <raingloom@riseup.net> wrote: > It makes zero sense to load full package definitions from > disk for most queries, such as guix search, with an SoA representation > we could load only the fields that we care about. That’s already the case; see ~/.config/guix/current/lib/guix/package.cache. For instance, “guix package -A” exploits it and the performances are acceptable. Two past summers, wow already! I tried to augment it and exploit it for “guix search”. The implementation and benchmark is in #39258 [1]. Well, the whole thread of #39258 appears to me worth to consider because it spots various bottleneck specific to “guix search” and explains why the improvement is not straightforward. Well, I have started months ago to write a Guix extension using guile-xapian. My aim is to tackle two annoyances: 1. the speed and 2. the relevance. About the relevance #2, the issue is that the current scoring considers only the local information of one package without considering the global information of all the others. Well, see [2,3,4] for some details. :-) 1: https://issues.guix.gnu.org/39258#119 2: https://yhetil.org/guix/CAJ3okZ3E3bhZ5pROZS68wEKdKOcZ8SpXsvdi-bnB=9Jz3mPahA@mail.gmail.com 3: https://yhetil.org/guix/CAJ3okZ3+hn0nJP98OhnZYLWJvhLGpdTUK+jB0hoM5JArQxO=zw@mail.gmail.com 4: https://yhetil.org/guix/CAJ3okZ0LaJzWDBA7bjqZew_jAmtt1rj9PJhevwrtBiA_COXENg@mail.gmail.com > ps.: Now I'm even more glad that I'm using a file system with > transparent compression on all my Guix systems. Did you benchmarked the performances for some Guix operations on these compressed vs uncompressed file system? Cheers, simon ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Faster “guix search” (was Re: How many bytes do we add (closure of guix) when adding one new package?) 2023-05-31 8:05 ` Faster “guix search” (was Re: How many bytes do we add (closure of guix) when adding one new package?) Simon Tournier @ 2023-05-31 11:10 ` Csepp 2023-05-31 11:55 ` Attila Lendvai 0 siblings, 1 reply; 30+ messages in thread From: Csepp @ 2023-05-31 11:10 UTC (permalink / raw) To: Simon Tournier; +Cc: Csepp, Ludovic Courtès, Andreas Enge, guix-devel Simon Tournier <zimon.toutoune@gmail.com> writes: > Hi, > > On Tue, 30 May 2023 at 21:10, Csepp <raingloom@riseup.net> wrote: > >> It makes zero sense to load full package definitions from >> disk for most queries, such as guix search, with an SoA representation >> we could load only the fields that we care about. > > That’s already the case; see > ~/.config/guix/current/lib/guix/package.cache. > > For instance, “guix package -A” exploits it and the performances are > acceptable. Two past summers, wow already! I tried to augment it and > exploit it for “guix search”. The implementation and benchmark is in > #39258 [1]. Well, the whole thread of #39258 appears to me worth to > consider because it spots various bottleneck specific to “guix search” > and explains why the improvement is not straightforward. That's a good improvement, but it's in addition to the ELF files, so it doesn't save any space, and as far as I know it doesn't speed up non-textual queries, like searching for packages that use a specific build system. > Well, I have started months ago to write a Guix extension using > guile-xapian. My aim is to tackle two annoyances: 1. the speed and > 2. the relevance. > > About the relevance #2, the issue is that the current scoring considers > only the local information of one package without considering the global > information of all the others. Well, see [2,3,4] for some details. :-) > > 1: https://issues.guix.gnu.org/39258#119 > 2: https://yhetil.org/guix/CAJ3okZ3E3bhZ5pROZS68wEKdKOcZ8SpXsvdi-bnB=9Jz3mPahA@mail.gmail.com > 3: https://yhetil.org/guix/CAJ3okZ3+hn0nJP98OhnZYLWJvhLGpdTUK+jB0hoM5JArQxO=zw@mail.gmail.com > 4: https://yhetil.org/guix/CAJ3okZ0LaJzWDBA7bjqZew_jAmtt1rj9PJhevwrtBiA_COXENg@mail.gmail.com Thanks for the links, gonna read them in more detail later. >> ps.: Now I'm even more glad that I'm using a file system with >> transparent compression on all my Guix systems. > > Did you benchmarked the performances for some Guix operations on these > compressed vs uncompressed file system? I haven't, but I have recently tried to move to a larger drive and accidentally did a btrfs send without compression and the system didn't fit. ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Faster “guix search” (was Re: How many bytes do we add (closure of guix) when adding one new package?) 2023-05-31 11:10 ` Csepp @ 2023-05-31 11:55 ` Attila Lendvai 0 siblings, 0 replies; 30+ messages in thread From: Attila Lendvai @ 2023-05-31 11:55 UTC (permalink / raw) To: Csepp; +Cc: Simon Tournier, Ludovic Courtès, Andreas Enge, guix-devel > It makes zero sense to load full package definitions from disk for > most queries, such as guix search, with an SoA representation we > could load only the fields that we care about. i'd like to quickly point out something while we are discussing this: when i came to guix it was rather confusing that there two namespaces through which one can get hold of a package object: 1) module-global variables in the scheme module system 2) a reified repository of packages (i.e. a scheme hash table, and some lookup functions). these two namespaces are quite indepenedent from each other, and there are no formal rules that govern the relationship between the two. there are scheme variable reference in some places, some others issue a call to SPECIFICATION->PACKAGE, manifests do that implicitly (?), etc. my gut feeling is that there is potential here for unification and simplification, without limiting composability. -- • attila lendvai • PGP: 963F 5D5F 45C7 DFCD 0A39 -- “The greatest gift you can give someone is your own personal development. I used to say, 'If you’ll take care of me, I’ll take care of you'. Now I say, 'I will take care of me for you, if you will take care of you for me'.” — Jim Rohn ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: How many bytes do we add (closure of guix) when adding one new package? 2023-05-30 12:10 ` Simon Tournier 2023-05-30 19:10 ` Csepp @ 2023-05-30 20:55 ` Jack Hill 2023-05-31 8:27 ` Simon Tournier 1 sibling, 1 reply; 30+ messages in thread From: Jack Hill @ 2023-05-30 20:55 UTC (permalink / raw) To: Simon Tournier; +Cc: Ludovic Courtès, Guix Devel, Andreas Enge [-- Attachment #1: Type: text/plain, Size: 1656 bytes --] On Tue, 30 May 2023, Simon Tournier wrote: > Just to put a figure on what means “big”: currently the .go files are 5 > times bigger than their associated .scm. > > Somehow, it’s the trap of DSL. :-) Packages are declarative and the > information they declare is not dense. However, because they are > bytecompiled to a general programming language, their specificity is not > exploited. In an ideal world, the compiled binary representation of the > packages should be smaller than their human-readable text-file > counterpart. > > The mentioned improvement is nice. And it’s visible: > > --8<---------------cut here---------------start------------->8--- > 145M /gnu/store/nqrb3g4l59wd74w8mr9v0b992bj2sd1w-guix-d62c9b267-modules/lib/guile/3.0/site-ccache/gnu > 117M /gnu/store/s6rqlhqr750k44ynkqqj5mwjj2cs2yln-guix-a09968565-modules/lib/guile/3.0/site-ccache/gnu > 127M /gnu/store/ndii4bpyzh2rc05ya61s89rig9hdrl4k-guix-a0178d34f-modules/lib/guile/3.0/site-ccache/gnu > 164M /gnu/store/ni63a203jf61dwxlv8kr9b8x3vb1pdsp-guix-8e2f32cee-modules/lib/guile/3.0/site-ccache/gnu > --8<---------------cut here---------------end--------------->8--- This is probably a tagent, sorry, but I was curious how well the .go files compressed. It seems quite well, actually: jackhill@mimolette ~/.config/guix/current/lib/guile/3.0/site-ccache/gnu [env]$ sudo compsize . Processed 595 files, 1659 regular extents (1659 refs), 0 inline. Type Perc Disk Usage Uncompressed Referenced TOTAL 21% 36M 173M 173M none 100% 16K 16K 16K zstd 21% 36M 173M 173M Best, Jack ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: How many bytes do we add (closure of guix) when adding one new package? 2023-05-30 20:55 ` How many bytes do we add (closure of guix) when adding one new package? Jack Hill @ 2023-05-31 8:27 ` Simon Tournier 2023-05-31 12:47 ` Guillaume Le Vaillant 0 siblings, 1 reply; 30+ messages in thread From: Simon Tournier @ 2023-05-31 8:27 UTC (permalink / raw) To: Jack Hill; +Cc: Ludovic Courtès, Guix Devel, Andreas Enge Hi Jack, On Tue, 30 May 2023 at 16:55, Jack Hill <jackhill@jackhill.us> wrote: > $ ~/.config/guix/current/lib/guile/3.0/site-ccache/gnu [env]$ sudo compsize . > Processed 595 files, 1659 regular extents (1659 refs), 0 inline. > Type Perc Disk Usage Uncompressed Referenced > TOTAL 21% 36M 173M 173M > none 100% 16K 16K 16K > zstd 21% 36M 173M 173M Cool! Could you do (or anyone else with btrfs), guix time-machine --commit=d62c9b2671be55ae0305bebfda17b595f33797f2 \ -- describe guix time-machine --commit=a099685659b4bfa6b3218f84953cbb7ff9e88063 \ -- describe then report the size (compsize) of /gnu/store/nqrb3g4l59wd74w8mr9v0b992bj2sd1w-guix-d62c9b267-modules/lib/guile/3.0/site-ccache/gnu /gnu/store/s6rqlhqr750k44ynkqqj5mwjj2cs2yln-guix-a09968565-modules/lib/guile/3.0/site-ccache/gnu ? Cheers, simon ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: How many bytes do we add (closure of guix) when adding one new package? 2023-05-31 8:27 ` Simon Tournier @ 2023-05-31 12:47 ` Guillaume Le Vaillant 0 siblings, 0 replies; 30+ messages in thread From: Guillaume Le Vaillant @ 2023-05-31 12:47 UTC (permalink / raw) To: Simon Tournier; +Cc: Jack Hill, Ludovic Courtès, Andreas Enge, guix-devel [-- Attachment #1: Type: text/plain, Size: 2079 bytes --] Simon Tournier <zimon.toutoune@gmail.com> skribis: > Hi Jack, > > On Tue, 30 May 2023 at 16:55, Jack Hill <jackhill@jackhill.us> wrote: > >> $ ~/.config/guix/current/lib/guile/3.0/site-ccache/gnu [env]$ sudo compsize . >> Processed 595 files, 1659 regular extents (1659 refs), 0 inline. >> Type Perc Disk Usage Uncompressed Referenced >> TOTAL 21% 36M 173M 173M >> none 100% 16K 16K 16K >> zstd 21% 36M 173M 173M > > Cool! Could you do (or anyone else with btrfs), > > guix time-machine --commit=d62c9b2671be55ae0305bebfda17b595f33797f2 \ > -- describe > guix time-machine --commit=a099685659b4bfa6b3218f84953cbb7ff9e88063 \ > -- describe > > then report the size (compsize) of > > /gnu/store/nqrb3g4l59wd74w8mr9v0b992bj2sd1w-guix-d62c9b267-modules/lib/guile/3.0/site-ccache/gnu > /gnu/store/s6rqlhqr750k44ynkqqj5mwjj2cs2yln-guix-a09968565-modules/lib/guile/3.0/site-ccache/gnu > > ? > > Cheers, > simon Hi, With a BTRFS filesystem compressed with zstd:3, I get: --8<---------------cut here---------------start------------->8--- # compsize /gnu/store/nqrb3g4l59wd74w8mr9v0b992bj2sd1w-guix-d62c9b267-modules/lib/guile/3.0/site-ccache/gnu Processed 503 files, 1317 regular extents (1317 refs), 0 inline. Type Perc Disk Usage Uncompressed Referenced TOTAL 27% 40M 144M 144M none 100% 10M 10M 10M zstd 22% 30M 133M 133M # compsize /gnu/store/s6rqlhqr750k44ynkqqj5mwjj2cs2yln-guix-a09968565-modules/lib/guile/3.0/site-ccache/gnu Processed 530 files, 1169 regular extents (1169 refs), 0 inline. Type Perc Disk Usage Uncompressed Referenced TOTAL 19% 22M 116M 116M none 100% 32K 32K 32K zstd 19% 22M 116M 116M --8<---------------cut here---------------end--------------->8--- [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 247 bytes --] ^ permalink raw reply [flat|nested] 30+ messages in thread
end of thread, other threads:[~2023-05-31 12:51 UTC | newest] Thread overview: 30+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2023-04-24 10:13 bug#63050: "guix pull" requires graphical libraries Andreas Enge 2023-04-25 21:48 ` Ludovic Courtès 2023-04-26 7:28 ` Andreas Enge 2023-04-26 8:45 ` Josselin Poiret via Bug reports for GNU Guix 2023-04-26 16:59 ` Liliana Marie Prikler 2023-04-26 17:25 ` Andreas Enge 2023-04-26 18:39 ` Josselin Poiret via Bug reports for GNU Guix 2023-04-26 19:21 ` Andreas Enge 2023-04-26 19:34 ` Liliana Marie Prikler 2023-04-28 15:18 ` Simon Tournier 2023-05-03 19:33 ` Ludovic Courtès 2023-05-04 8:56 ` Simon Tournier 2023-05-05 15:21 ` Csepp 2023-05-09 12:36 ` Simon Tournier 2023-05-11 21:30 ` Csepp 2023-05-03 19:50 ` bug#63050: Reducing the closure size of Graphviz Ludovic Courtès 2023-05-04 9:00 ` Simon Tournier 2023-05-20 16:12 ` bug#63050: "guix pull" requires graphical libraries Ludovic Courtès 2023-05-20 16:38 ` Andreas Enge 2023-05-24 13:10 ` Ludovic Courtès 2023-05-25 18:24 ` How many bytes do we add (closure of guix) when adding one new package? Simon Tournier 2023-05-26 16:21 ` Ludovic Courtès 2023-05-30 12:10 ` Simon Tournier 2023-05-30 19:10 ` Csepp 2023-05-31 8:05 ` Faster “guix search” (was Re: How many bytes do we add (closure of guix) when adding one new package?) Simon Tournier 2023-05-31 11:10 ` Csepp 2023-05-31 11:55 ` Attila Lendvai 2023-05-30 20:55 ` How many bytes do we add (closure of guix) when adding one new package? Jack Hill 2023-05-31 8:27 ` Simon Tournier 2023-05-31 12:47 ` Guillaume Le Vaillant
Code repositories for project(s) associated with this external index https://git.savannah.gnu.org/cgit/guix.git This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.