* bug#47569: ‘qt-build-system’ retains too many references via wrappers
2021-04-06 13:17 ` Ludovic Courtès
@ 2021-04-06 13:43 ` Efraim Flashner
2021-04-06 13:57 ` Efraim Flashner
2021-04-06 17:43 ` Ludovic Courtès
2021-04-06 17:45 ` Leo Famulari
2 siblings, 1 reply; 13+ messages in thread
From: Efraim Flashner @ 2021-04-06 13:43 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 47569, Maxim Cournoyer
[-- Attachment #1: Type: text/plain, Size: 1759 bytes --]
On Tue, Apr 06, 2021 at 03:17:52PM +0200, Ludovic Courtès wrote:
> Hi,
>
> For the record, changing ‘qt-build-system’ would trigger a rebuild of
> less than 400 packages according to the back-of-the-envelope calculation
> below. In that case, it’s tempting to fix on ‘master’ and include it in
> the release.
>
> Thoughts?
>
There are a couple of other things we want to change with the
qt-build-system and the wrappers. Another one is changing the wrappers
so they prefix (or was it suffix?) QT_QPA_PLUGIN (and I think others) so
they can run natively on wayland. I think it would be a good
improvement and worth it.
> Ludo’.
>
> --8<---------------cut here---------------start------------->8---
> scheme@(guile-user)> ,use(gnu)
> scheme@(guile-user)> ,use(guix build-system qt)
> scheme@(guile-user)> ,use(guix)
> scheme@(guile-user)> (define qtpkgs (fold-packages (lambda (pkg lst)
> (if (eq? (package-build-system pkg) qt-build-system)
> (cons pkg lst)
> lst))
> '()))
> scheme@(guile-user)> (length qtpkgs)
> $78 = 186
> scheme@(guile-user)> ,use(guix graph)
> scheme@(guile-user)> ,run-in-store (node-edges (@ (guix scripts graph) %reverse-package-node-type) (fold-packages cons '()))
> $79 = #<procedure 7fe494f11520 at guix/graph.scm:94:17 (node)>
> scheme@(guile-user)> (node-reachable-count qtpkgs $79)
> $80 = 180
> scheme@(guile-user)> (+ 186 180)
> $81 = 366
> --8<---------------cut here---------------end--------------->8---
>
>
>
--
Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#47569: ‘qt-build-system’ retains too many references via wrappers
2021-04-06 13:43 ` Efraim Flashner
@ 2021-04-06 13:57 ` Efraim Flashner
0 siblings, 0 replies; 13+ messages in thread
From: Efraim Flashner @ 2021-04-06 13:57 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 47569, Maxim Cournoyer
[-- Attachment #1: Type: text/plain, Size: 2266 bytes --]
On Tue, Apr 06, 2021 at 04:43:46PM +0300, Efraim Flashner wrote:
> On Tue, Apr 06, 2021 at 03:17:52PM +0200, Ludovic Courtès wrote:
> > Hi,
> >
> > For the record, changing ‘qt-build-system’ would trigger a rebuild of
> > less than 400 packages according to the back-of-the-envelope calculation
> > below. In that case, it’s tempting to fix on ‘master’ and include it in
> > the release.
> >
> > Thoughts?
> >
>
> There are a couple of other things we want to change with the
> qt-build-system and the wrappers. Another one is changing the wrappers
> so they prefix (or was it suffix?) QT_QPA_PLUGIN (and I think others) so
> they can run natively on wayland. I think it would be a good
> improvement and worth it.
A quick search pulls up the following bugs: 43446 44982 45193 45221 45784.
Looks like there's a lot of desire to see it adjusted.
> > Ludo’.
> >
> > --8<---------------cut here---------------start------------->8---
> > scheme@(guile-user)> ,use(gnu)
> > scheme@(guile-user)> ,use(guix build-system qt)
> > scheme@(guile-user)> ,use(guix)
> > scheme@(guile-user)> (define qtpkgs (fold-packages (lambda (pkg lst)
> > (if (eq? (package-build-system pkg) qt-build-system)
> > (cons pkg lst)
> > lst))
> > '()))
> > scheme@(guile-user)> (length qtpkgs)
> > $78 = 186
> > scheme@(guile-user)> ,use(guix graph)
> > scheme@(guile-user)> ,run-in-store (node-edges (@ (guix scripts graph) %reverse-package-node-type) (fold-packages cons '()))
> > $79 = #<procedure 7fe494f11520 at guix/graph.scm:94:17 (node)>
> > scheme@(guile-user)> (node-reachable-count qtpkgs $79)
> > $80 = 180
> > scheme@(guile-user)> (+ 186 180)
> > $81 = 366
> > --8<---------------cut here---------------end--------------->8---
> >
> >
> >
>
> --
> Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר
> GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
> Confidentiality cannot be guaranteed on emails sent or received unencrypted
--
Efraim Flashner <efraim@flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#47569: ‘qt-build-system’ retains too many references via wrappers
2021-04-06 13:17 ` Ludovic Courtès
2021-04-06 13:43 ` Efraim Flashner
@ 2021-04-06 17:43 ` Ludovic Courtès
2021-04-07 19:35 ` Maxim Cournoyer
2021-04-07 21:03 ` Maxim Cournoyer
2021-04-06 17:45 ` Leo Famulari
2 siblings, 2 replies; 13+ messages in thread
From: Ludovic Courtès @ 2021-04-06 17:43 UTC (permalink / raw)
To: Maxim Cournoyer; +Cc: 47569
[-- Attachment #1: Type: text/plain, Size: 5791 bytes --]
Hi again!
The attached patch fixes this problem AFAICS by filtering out of
XDG_DATA_DIRS directories that are unlikely to be of any use. It
follows the same strategy as ‘glib-or-gtk-build-system’, which is to
only include share/ sub-directories that also contain one of the given
“selectors”: /glib-2.0/schemas, /sounds, /themes, etc.
It gives me a working ktouch, with a wrapper sets a much shorter
XDG_DATA_DIR:
--8<---------------cut here---------------start------------->8---
$ head -2 $(./pre-inst-env guix build --no-grafts ktouch)/bin/ktouch
#!/gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/bin/bash
export XDG_DATA_DIRS="/gnu/store/mgzijzw7yn03pbk54zy0f81gyph9jh3k-ktouch-20.12.1/share:/gnu/store/5g95qdh0p46qszv199rmdd2lx4mninm7-kcoreaddons-5.70.0/share"
$ head -2 $(guix build --no-grafts ktouch)/bin/ktouch
#!/gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/bin/bash
export XDG_DATA_DIRS="/gnu/store/f821syw55jd0l04xkmnbvp562flrw6fm-ktouch-20.12.1/share:/gnu/store/3dsl2jalrcyldkrsqab1hc6sv8pyag9z-cmake-minimal-3.16.5/share:/gnu/store/3x63j4ndhbzkjwr6mhnp4h86r550gs7q-extra-cmake-modules-5.70.0/share:/gnu/store/w08sglxbfpmcvz25pfgnq7v27m1rlkf9-kdoctools-5.70.0/share:/gnu/store/krpyb0zi700dcrg9cc8932w4v0qivdg9-pkg-config-0.29.2/share:/gnu/store/v6f44zccwh9z5zk3pjlywjybbi8n2hjh-tar-1.32/share:/gnu/store/ncydgq2znms5n1d2k5yqshhf58nsixwv-gzip-1.10/share:/gnu/store/i8h2pcxqdq07ijm3ibkka8f4smn1w48v-bzip2-1.0.8/share:/gnu/store/9860f1abqj8wjjnwl8a9v54pdcc3bhgf-xz-5.2.4/share:/gnu/store/60g7r3l01fd7c58yjbm6krgcwj1jkpwg-file-5.38/share:/gnu/store/n4n560pfvvw50a9369axw5vj5rrqfj1n-diffutils-3.7/share:/gnu/store/cd5qf3kcnlq35p9k392pjdpdzpsnds70-patch-2.7.6/share:/gnu/store/hic7snhayfl7m6cpfqqr73nmm19bpqkg-findutils-4.7.0/share:/gnu/store/swqdvwri9dbv6zssg6v0by7l05hd6wxp-gawk-5.0.1/share:/gnu/store/ishk7fswcs4gkwcp8mh788z4mvvl9bxh-sed-4.8/share:/gnu/store/bhs4rj58v8j1narb2454raan2ps38xd8-grep-3.4/share:/gnu/store/57xj5gcy1jbl9ai2lnrqnpr0dald9i65-coreutils-8.32/share:/gnu/store/hm40bxnv8jxmbc1lpb7zfimii4xm9m81-make-4.3/share:/gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/share:/gnu/store/m1z7cdbqsqyp9xnjw5cvlb4a7gkcg3m4-binutils-2.34/share:/gnu/store/rn75fm7adgx3pw5j8pg3bczfqq1y17lk-gcc-7.5.0/share:/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/share:/gnu/store/s3dcqzwqaakv1yx37by9chksdbkgih17-glibc-2.31-static/share:/gnu/store/5rj71g09qc9g20mv0lh5a28jhwa5d6dl-kcmutils-5.70.0/share:/gnu/store/3vdmy5gjn1a23ky6l907byc3917xfh2a-kcompletion-5.70.0/share:/gnu/store/ba8nl7wflsf1s95x72kpaj0rzval908b-kconfig-5.70.0/share:/gnu/store/r4120j3v7cnv5qlmp2kqb6xbc63gplpa-kconfigwidgets-5.70.0/share:/gnu/store/5g95qdh0p46qszv199rmdd2lx4mninm7-kcoreaddons-5.70.0/share:/gnu/store/lbq73h2snwpn4wf9wkp8j0rxw0lyhyp4-kdeclarative-5.70.0/share:/gnu/store/k4k0mrgv31clkpl38zjnpb3l0nin2nnl-ki18n-5.70.0/share:/gnu/store/agmzjiqcv1lb77jmr7xwx0br4sy12i6d-kiconthemes-5.70.0/share:/gnu/store/0mbm58p5mybimbvv8ainkzz7mdpynd25-kitemviews-5.70.0/share:/gnu/store/yqpk8haa1z5plgb134f95vryjxghlyjv-kqtquickcharts-20.04.1/share:/gnu/store/5mfz0faj9lc9m1ppijgqkxz03j89dyil-ktextwidgets-5.70.0/share:/gnu/store/fwmp85k5hqy7fw0rnq5560pg4amiblz4-kwidgetsaddons-5.70.0/share:/gnu/store/xslnsy6sxjp66l99mvj67v1x30bh55xi-kwindowsystem-5.70.0/share:/gnu/store/n79ns6y3cc7ny4mkp5qidf57j1cxbg6p-kxmlgui-5.70.0/share:/gnu/store/3x6y8kpbwjmzpndjfhdbn654g7dm85cp-libxcb-1.14/share:/gnu/store/zc9bpa1v46cnqc05xk8njm08bwk8a70m-libxkbfile-1.1.0/share:/gnu/store/j7110r2rhs9pdypx2x0h8nx4r082fbdk-qtbase-5.15.2/share:/gnu/store/8r5y5bnblc4s6w8j5sfxv06ra6lz12j8-qtdeclarative-5.15.2/share:/gnu/store/c020rcd650ikq7gqfnkga46kzis3v2hi-qtgraphicaleffects-5.15.2/share:/gnu/store/2iwman5hbgg15r99rshl6lsxz35dq2b2-qtquickcontrols2-5.15.2/share:/gnu/store/n7dbg5cgcpslrym4zmsimps1ch1p0q8w-qtx11extras-5.15.2/share:/gnu/store/7yjd1h08li3dr2188pfwfnwlq0p3av9p-qtxmlpatterns-5.15.2/share:/gnu/store/gfapkk5c6hvl1d94m4sqnhn7f9l5gqyh-linux-libre-headers-5.4.20/share:/gnu/store/kvdrfyi50zsmy0ykmc9p8qzjlahqfa4p-kservice-5.70.0/share:/gnu/store/yk425lbm0wnjpqm1z964l9l6li5x0wbn-kcodecs-5.70.0/share:/gnu/store/2z5nfmzxhqaalcrswp8nn3yk5ys8q261-kauth-5.70.0/share:/gnu/store/vhmyxywsdy92bgqxy6fnmcpr81xsfcys-kpackage-5.70.0/share:/gnu/store/rz42ba0my9vrgbkjpkzr2drmnjk5ah50-python-3.8.2/share:/gnu/store/n9wx4dp68bfsw9bb1bwmklzl34ggrmgm-gettext-minimal-0.20.1/share:/gnu/store/yqs5mkp20m58sr744cs2pn8sqb5m59sq-sonnet-5.70.0/share:/gnu/store/6cdl970wcv4jhvpgbh8sdj54a5gwhmwj-libxdmcp-1.1.3/share:/gnu/store/h7sy4hr7arjknbyy1aq0xwv6fksnzw9n-libxau-1.0.9/share:/gnu/store/klk98l8czl05lvanrr7xl7a9hksac4ak-libpthread-stubs-0.4/share:/gnu/store/k30a2pgmc6b3ifmq3rn0dr5ng9gr8w81-which-2.21/share:/gnu/store/7disnxqw8i3n4zhw5s0xj9kkdckkmr1p-mesa-20.2.4/share:/gnu/store/wxl57nkbqgamfp73b7v62kk3f1hiv0cz-xorgproto-2019.2/share:/gnu/store/c3zbbynb590yxrpwcxrkll80mn66h3kf-libxxf86vm-1.1.4/share:/gnu/store/jwga98k68l0h5c45jx7z4jdjzhfc34vm-libxshmfence-1.3/share:/gnu/store/9z9gd76la5bww3sqgg2ax0h00msl28k6-libxfixes-5.0.3/share:/gnu/store/2wysr1lr7b5mc444m39qz107n92xm5xq-libxdamage-1.1.5/share:/gnu/store/fx3979c88s9yxdbchyf36qryawgzpwb5-libx11-1.6.10/share:/gnu/store/kx58xc3hxa9b0mp1hl7pzz43bl7gh18h-libdrm-2.4.103/share:/gnu/store/lc8irxnwazq340ygkbqh0ask446jzy9k-util-macros-1.19.2/share:/gnu/store/qrs0p8j3wq6q5a4dm0ndjdavk9gyal5q-libxext-1.3.4/share"
--8<---------------cut here---------------end--------------->8---
The closure is still awfully big but smaller:
--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix size ktouch | tail -1
total: 1298.0 MiB
$ guix size ktouch | tail -1
total: 1752.4 MiB
--8<---------------cut here---------------end--------------->8---
Thoughts?
Ludo’.
[-- Attachment #2: Type: text/x-patch, Size: 3232 bytes --]
diff --git a/guix/build/qt-build-system.scm b/guix/build/qt-build-system.scm
index 005157b0a4..bb17c362d1 100644
--- a/guix/build/qt-build-system.scm
+++ b/guix/build/qt-build-system.scm
@@ -1,6 +1,6 @@
;;; GNU Guix --- Functional package management for GNU
;;; Copyright © 2014 Federico Beffa <beffa@fbengineering.ch>
-;;; Copyright © 2014, 2015 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2014, 2015, 2021 Ludovic Courtès <ludo@gnu.org>
;;; Copyright © 2018 Mark H Weaver <mhw@netris.org>
;;; Copyright © 2019, 2020 Hartmut Goebel <h.goebel@crazy-compilers.com>
;;;
@@ -49,25 +49,44 @@
(define (variables-for-wrapping base-directories)
- (define (collect-sub-dirs base-directories subdirectory)
- (filter-map
- (lambda (dir)
- (let ((directory (string-append dir subdirectory)))
- (if (directory-exists? directory) directory #f)))
- base-directories))
+ (define (collect-sub-dirs base-directories subdirectory
+ selectors)
+ ;; Append SUBDIRECTORY and each of BASE-DIRECTORIES, and return the subset
+ ;; that exists and has at least one of the SELECTORS sub-directories,
+ ;; unless SELECTORS is the empty list.
+ (filter-map (lambda (dir)
+ (let ((directory (string-append dir subdirectory)))
+ (and (directory-exists? directory)
+ (or (null? selectors)
+ (any (lambda (selector)
+ (directory-exists?
+ (string-append directory selector)))
+ selectors))
+ directory)))
+ base-directories))
- (filter
- (lambda (var-to-wrap) (not (null? (last var-to-wrap))))
- (map
- (lambda (var-spec)
- `(,(first var-spec) = ,(collect-sub-dirs base-directories (last var-spec))))
- (list
- ;; these shall match the search-path-specification for Qt and KDE
- ;; libraries
- '("XDG_DATA_DIRS" "/share")
- '("XDG_CONFIG_DIRS" "/etc/xdg")
- '("QT_PLUGIN_PATH" "/lib/qt5/plugins")
- '("QML2_IMPORT_PATH" "/lib/qt5/qml")))))
+ (filter-map
+ (match-lambda
+ ((variable directory selectors ...)
+ (match (collect-sub-dirs base-directories directory
+ selectors)
+ (()
+ #f)
+ (directories
+ `(,variable = ,directories)))))
+
+ ;; These shall match the search-path-specification for Qt and KDE
+ ;; libraries.
+ (list '("XDG_DATA_DIRS" "/share"
+
+ ;; These are "selectors": consider /share if and only if these
+ ;; sub-directories exist. This avoids adding irrelevant packages
+ ;; to XDG_DATA_DIRS just because they have a /share sub-directory.
+ "/glib-2.0/schemas" "/sounds" "/themes"
+ "/cursors" "/wallpapers" "/icons" "/mime")
+ '("XDG_CONFIG_DIRS" "/etc/xdg")
+ '("QT_PLUGIN_PATH" "/lib/qt5/plugins")
+ '("QML2_IMPORT_PATH" "/lib/qt5/qml"))))
(define* (wrap-all-programs #:key inputs outputs
(qt-wrap-excluded-outputs '())
^ permalink raw reply related [flat|nested] 13+ messages in thread* bug#47569: ‘qt-build-system’ retains too many references via wrappers
2021-04-06 17:43 ` Ludovic Courtès
@ 2021-04-07 19:35 ` Maxim Cournoyer
2021-04-08 21:38 ` Ludovic Courtès
2021-04-07 21:03 ` Maxim Cournoyer
1 sibling, 1 reply; 13+ messages in thread
From: Maxim Cournoyer @ 2021-04-07 19:35 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 47569
Hi Ludovic,
Ludovic Courtès <ludo@gnu.org> writes:
> Hi again!
>
> The attached patch fixes this problem AFAICS by filtering out of
> XDG_DATA_DIRS directories that are unlikely to be of any use. It
> follows the same strategy as ‘glib-or-gtk-build-system’, which is to
> only include share/ sub-directories that also contain one of the given
> “selectors”: /glib-2.0/schemas, /sounds, /themes, etc.
>
> It gives me a working ktouch, with a wrapper sets a much shorter
> XDG_DATA_DIR:
>
> $ head -2 $(./pre-inst-env guix build --no-grafts ktouch)/bin/ktouch
> #!/gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/bin/bash
> export XDG_DATA_DIRS="/gnu/store/mgzijzw7yn03pbk54zy0f81gyph9jh3k-ktouch-20.12.1/share:/gnu/store/5g95qdh0p46qszv199rmdd2lx4mninm7-kcoreaddons-5.70.0/share"
> $ head -2 $(guix build --no-grafts ktouch)/bin/ktouch
> #!/gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/bin/bash
> export XDG_DATA_DIRS="/gnu/store/f821syw55jd0l04xkmnbvp562flrw6fm-ktouch-20.12.1/share:/gnu/store/3dsl2jalrcyldkrsqab1hc6sv8pyag9z-cmake-minimal-3.16.5/share:/gnu/store/3x63j4ndhbzkjwr6mhnp4h86r550gs7q-extra-cmake-modules-5.70.0/share:/gnu/store/w08sglxbfpmcvz25pfgnq7v27m1rlkf9-kdoctools-5.70.0/share:/gnu/store/krpyb0zi700dcrg9cc8932w4v0qivdg9-pkg-config-0.29.2/share:/gnu/store/v6f44zccwh9z5zk3pjlywjybbi8n2hjh-tar-1.32/share:/gnu/store/ncydgq2znms5n1d2k5yqshhf58nsixwv-gzip-1.10/share:/gnu/store/i8h2pcxqdq07ijm3ibkka8f4smn1w48v-bzip2-1.0.8/share:/gnu/store/9860f1abqj8wjjnwl8a9v54pdcc3bhgf-xz-5.2.4/share:/gnu/store/60g7r3l01fd7c58yjbm6krgcwj1jkpwg-file-5.38/share:/gnu/store/n4n560pfvvw50a9369axw5vj5rrqfj1n-diffutils-3.7/share:/gnu/store/cd5qf3kcnlq35p9k392pjdpdzpsnds70-patch-2.7.6/share:/gnu/store/hic7snhayfl7m6cpfqqr73nmm19bpqkg-findutils-4.7.0/share:/gnu/store/swqdvwri9dbv6zssg6v0by7l05hd6wxp-gawk-5.0.1/share:/gnu/store/ishk7fswcs4gkwcp8mh788z4mvvl9bxh-sed-4.8/share:/gnu/store/bhs4rj58v8j1narb2454raan2ps38xd8-grep-3.4/share:/gnu/store/57xj5gcy1jbl9ai2lnrqnpr0dald9i65-coreutils-8.32/share:/gnu/store/hm40bxnv8jxmbc1lpb7zfimii4xm9m81-make-4.3/share:/gnu/store/pwcp239kjf7lnj5i4lkdzcfcxwcfyk72-bash-minimal-5.0.16/share:/gnu/store/m1z7cdbqsqyp9xnjw5cvlb4a7gkcg3m4-binutils-2.34/share:/gnu/store/rn75fm7adgx3pw5j8pg3bczfqq1y17lk-gcc-7.5.0/share:/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/share:/gnu/store/s3dcqzwqaakv1yx37by9chksdbkgih17-glibc-2.31-static/share:/gnu/store/5rj71g09qc9g20mv0lh5a28jhwa5d6dl-kcmutils-5.70.0/share:/gnu/store/3vdmy5gjn1a23ky6l907byc3917xfh2a-kcompletion-5.70.0/share:/gnu/store/ba8nl7wflsf1s95x72kpaj0rzval908b-kconfig-5.70.0/share:/gnu/store/r4120j3v7cnv5qlmp2kqb6xbc63gplpa-kconfigwidgets-5.70.0/share:/gnu/store/5g95qdh0p46qszv199rmdd2lx4mninm7-kcoreaddons-5.70.0/share:/gnu/store/lbq73h2snwpn4wf9wkp8j0rxw0lyhyp4-kdeclarative-5.70.0/share:/gnu/store/k4k0mrgv31clkpl38zjnpb3l0nin2nnl-ki18n-5.70.0/share:/gnu/store/agmzjiqcv1lb77jmr7xwx0br4sy12i6d-kiconthemes-5.70.0/share:/gnu/store/0mbm58p5mybimbvv8ainkzz7mdpynd25-kitemviews-5.70.0/share:/gnu/store/yqpk8haa1z5plgb134f95vryjxghlyjv-kqtquickcharts-20.04.1/share:/gnu/store/5mfz0faj9lc9m1ppijgqkxz03j89dyil-ktextwidgets-5.70.0/share:/gnu/store/fwmp85k5hqy7fw0rnq5560pg4amiblz4-kwidgetsaddons-5.70.0/share:/gnu/store/xslnsy6sxjp66l99mvj67v1x30bh55xi-kwindowsystem-5.70.0/share:/gnu/store/n79ns6y3cc7ny4mkp5qidf57j1cxbg6p-kxmlgui-5.70.0/share:/gnu/store/3x6y8kpbwjmzpndjfhdbn654g7dm85cp-libxcb-1.14/share:/gnu/store/zc9bpa1v46cnqc05xk8njm08bwk8a70m-libxkbfile-1.1.0/share:/gnu/store/j7110r2rhs9pdypx2x0h8nx4r082fbdk-qtbase-5.15.2/share:/gnu/store/8r5y5bnblc4s6w8j5sfxv06ra6lz12j8-qtdeclarative-5.15.2/share:/gnu/store/c020rcd650ikq7gqfnkga46kzis3v2hi-qtgraphicaleffects-5.15.2/share:/gnu/store/2iwman5hbgg15r99rshl6lsxz35dq2b2-qtquickcontrols2-5.15.2/share:/gnu/store/n7dbg5cgcpslrym4zmsimps1ch1p0q8w-qtx11extras-5.15.2/share:/gnu/store/7yjd1h08li3dr2188pfwfnwlq0p3av9p-qtxmlpatterns-5.15.2/share:/gnu/store/gfapkk5c6hvl1d94m4sqnhn7f9l5gqyh-linux-libre-headers-5.4.20/share:/gnu/store/kvdrfyi50zsmy0ykmc9p8qzjlahqfa4p-kservice-5.70.0/share:/gnu/store/yk425lbm0wnjpqm1z964l9l6li5x0wbn-kcodecs-5.70.0/share:/gnu/store/2z5nfmzxhqaalcrswp8nn3yk5ys8q261-kauth-5.70.0/share:/gnu/store/vhmyxywsdy92bgqxy6fnmcpr81xsfcys-kpackage-5.70.0/share:/gnu/store/rz42ba0my9vrgbkjpkzr2drmnjk5ah50-python-3.8.2/share:/gnu/store/n9wx4dp68bfsw9bb1bwmklzl34ggrmgm-gettext-minimal-0.20.1/share:/gnu/store/yqs5mkp20m58sr744cs2pn8sqb5m59sq-sonnet-5.70.0/share:/gnu/store/6cdl970wcv4jhvpgbh8sdj54a5gwhmwj-libxdmcp-1.1.3/share:/gnu/store/h7sy4hr7arjknbyy1aq0xwv6fksnzw9n-libxau-1.0.9/share:/gnu/store/klk98l8czl05lvanrr7xl7a9hksac4ak-libpthread-stubs-0.4/share:/gnu/store/k30a2pgmc6b3ifmq3rn0dr5ng9gr8w81-which-2.21/share:/gnu/store/7disnxqw8i3n4zhw5s0xj9kkdckkmr1p-mesa-20.2.4/share:/gnu/store/wxl57nkbqgamfp73b7v62kk3f1hiv0cz-xorgproto-2019.2/share:/gnu/store/c3zbbynb590yxrpwcxrkll80mn66h3kf-libxxf86vm-1.1.4/share:/gnu/store/jwga98k68l0h5c45jx7z4jdjzhfc34vm-libxshmfence-1.3/share:/gnu/store/9z9gd76la5bww3sqgg2ax0h00msl28k6-libxfixes-5.0.3/share:/gnu/store/2wysr1lr7b5mc444m39qz107n92xm5xq-libxdamage-1.1.5/share:/gnu/store/fx3979c88s9yxdbchyf36qryawgzpwb5-libx11-1.6.10/share:/gnu/store/kx58xc3hxa9b0mp1hl7pzz43bl7gh18h-libdrm-2.4.103/share:/gnu/store/lc8irxnwazq340ygkbqh0ask446jzy9k-util-macros-1.19.2/share:/gnu/store/qrs0p8j3wq6q5a4dm0ndjdavk9gyal5q-libxext-1.3.4/share"
>
>
> The closure is still awfully big but smaller:
>
> $ ./pre-inst-env guix size ktouch | tail -1
> total: 1298.0 MiB
> $ guix size ktouch | tail -1
> total: 1752.4 MiB
>
> Thoughts?
>
> Ludo’.
Thanks for the patch! I built jami-qt with it; the closure size was
reduced from 2.7 GiB to 2.2 GiB (there's probably more to do, but it's a
good start!). It seems to work just as well as before the change, from
a cursory look.
LGTM!
Thank you,
Maxim
^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#47569: ‘qt-build-system’ retains too many references via wrappers
2021-04-07 19:35 ` Maxim Cournoyer
@ 2021-04-08 21:38 ` Ludovic Courtès
0 siblings, 0 replies; 13+ messages in thread
From: Ludovic Courtès @ 2021-04-08 21:38 UTC (permalink / raw)
To: Maxim Cournoyer; +Cc: 47569
Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
> Thanks for the patch! I built jami-qt with it; the closure size was
> reduced from 2.7 GiB to 2.2 GiB (there's probably more to do, but it's a
> good start!). It seems to work just as well as before the change, from
> a cursory look.
Regarding the closure size, the top packages and the usual suspects:
--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix size ktouch | head
store item total self
/gnu/store/rv567nsh90bqz5gfh6sh1g84jccxcifa-llvm-11.0.0 220.0 148.6 11.4%
/gnu/store/7disnxqw8i3n4zhw5s0xj9kkdckkmr1p-mesa-20.2.4 389.1 141.6 10.9%
/gnu/store/j7110r2rhs9pdypx2x0h8nx4r082fbdk-qtbase-5.15.2 976.2 80.0 6.2%
/gnu/store/rz42ba0my9vrgbkjpkzr2drmnjk5ah50-python-3.8.2 171.4 79.8 6.1%
/gnu/store/8zvc5mvk0xm3ygrxsgpyy5ilxb5rzjry-perl-5.30.2 146.2 57.1 4.4%
/gnu/store/a5m2xzsv3ff1gsh4pkdxbv99nchmlkhq-mozjs-60.2.3-2 248.3 54.7 4.2%
/gnu/store/18hp7flyb3yid3yp49i6qcdq0sbi5l1n-guile-3.0.2 132.8 53.2 4.1%
/gnu/store/ckwyq3y43i33s5fl9hb6c51418rh1qwj-icu4c-66.1 110.2 38.1 2.9%
/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31 38.4 36.7 2.8%
--8<---------------cut here---------------end--------------->8---
There’s a preliminary patch to improve the LLVM situation at
<https://issues.guix.gnu.org/42576>. I’m sure we can also do a better
job on the size of those other things.
A bit of a shame that mozjs shows up here; that’s because of Polkit:
--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix graph --path ktouch mozjs@60 -t references
/gnu/store/179ifxjb2kvapzydsnkh2vylhnx79kkk-ktouch-20.12.1
/gnu/store/2z5nfmzxhqaalcrswp8nn3yk5ys8q261-kauth-5.70.0
/gnu/store/vg0lwx182c63icfj60bi4gfk4s9bh1rf-polkit-qt-1-0.112.0
/gnu/store/9zijxjyrpg54csx3sam8hmsvrjgqw4zm-polkit-0.116
/gnu/store/a5m2xzsv3ff1gsh4pkdxbv99nchmlkhq-mozjs-60.2.3-2
--8<---------------cut here---------------end--------------->8---
(It’ll be worse when Polkit depends on Rust…)
Ludo’.
^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#47569: ‘qt-build-system’ retains too many references via wrappers
2021-04-06 17:43 ` Ludovic Courtès
2021-04-07 19:35 ` Maxim Cournoyer
@ 2021-04-07 21:03 ` Maxim Cournoyer
2021-04-07 21:23 ` Maxim Cournoyer
1 sibling, 1 reply; 13+ messages in thread
From: Maxim Cournoyer @ 2021-04-07 21:03 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 47569
Hi Ludo,
I just had another thought on this!
Ludovic Courtès <ludo@gnu.org> writes:
[...]
> - (filter
> - (lambda (var-to-wrap) (not (null? (last var-to-wrap))))
> - (map
> - (lambda (var-spec)
> - `(,(first var-spec) = ,(collect-sub-dirs base-directories (last var-spec))))
> - (list
> - ;; these shall match the search-path-specification for Qt and KDE
> - ;; libraries
> - '("XDG_DATA_DIRS" "/share")
> - '("XDG_CONFIG_DIRS" "/etc/xdg")
> - '("QT_PLUGIN_PATH" "/lib/qt5/plugins")
> - '("QML2_IMPORT_PATH" "/lib/qt5/qml")))))
> + (filter-map
> + (match-lambda
> + ((variable directory selectors ...)
> + (match (collect-sub-dirs base-directories directory
> + selectors)
> + (()
> + #f)
> + (directories
> + `(,variable = ,directories)))))
> +
> + ;; These shall match the search-path-specification for Qt and KDE
> + ;; libraries.
> + (list '("XDG_DATA_DIRS" "/share"
> +
> + ;; These are "selectors": consider /share if and only if these
> + ;; sub-directories exist. This avoids adding irrelevant packages
> + ;; to XDG_DATA_DIRS just because they have a /share sub-directory.
> + "/glib-2.0/schemas" "/sounds" "/themes"
> + "/cursors" "/wallpapers" "/icons" "/mime")
> + '("XDG_CONFIG_DIRS" "/etc/xdg")
> + '("QT_PLUGIN_PATH" "/lib/qt5/plugins")
> + '("QML2_IMPORT_PATH" "/lib/qt5/qml"))))
>
> (define* (wrap-all-programs #:key inputs outputs
> (qt-wrap-excluded-outputs '())
Seeing a growing number of packages require a custom wrap phase for
qtwebengine, I think the following additions could make sense to be
incorporated as part as this Qt-world rebuild:
2 files changed, 5 insertions(+), 1 deletion(-)
gnu/packages/qt.scm | 3 +++
guix/build/qt-build-system.scm | 3 ++-
modified gnu/packages/qt.scm
@@ -538,6 +538,9 @@ system, and the core design of Django is reused in Grantlee.")
(search-path-specification
(variable "QT_PLUGIN_PATH")
(files '("lib/qt5/plugins")))
+ (search-path-specification
+ (variable "QTWEBENGINEPROCESS_PATH")
+ (files '("lib/qt5/libexec/QtWebEngineProcess")))
(search-path-specification
(variable "XDG_DATA_DIRS")
(files '("share")))
modified guix/build/qt-build-system.scm
@@ -86,7 +86,8 @@
"/cursors" "/wallpapers" "/icons" "/mime")
'("XDG_CONFIG_DIRS" "/etc/xdg")
'("QT_PLUGIN_PATH" "/lib/qt5/plugins")
- '("QML2_IMPORT_PATH" "/lib/qt5/qml"))))
+ '("QML2_IMPORT_PATH" "/lib/qt5/qml")
+ '("QTWEBENGINEPROCESS_PATH" "lib/qt5/libexec/QtWebEngineProcess"))))
(define* (wrap-all-programs #:key inputs outputs
(qt-wrap-excluded-outputs '())
Thanks,
Maxim
^ permalink raw reply [flat|nested] 13+ messages in thread* bug#47569: ‘qt-build-system’ retains too many references via wrappers
2021-04-07 21:03 ` Maxim Cournoyer
@ 2021-04-07 21:23 ` Maxim Cournoyer
2021-04-08 21:32 ` Ludovic Courtès
0 siblings, 1 reply; 13+ messages in thread
From: Maxim Cournoyer @ 2021-04-07 21:23 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 47569
Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
> Seeing a growing number of packages require a custom wrap phase for
> qtwebengine, I think the following additions could make sense to be
> incorporated as part as this Qt-world rebuild:
>
> 2 files changed, 5 insertions(+), 1 deletion(-)
> gnu/packages/qt.scm | 3 +++
> guix/build/qt-build-system.scm | 3 ++-
>
> modified gnu/packages/qt.scm
> @@ -538,6 +538,9 @@ system, and the core design of Django is reused in Grantlee.")
> (search-path-specification
> (variable "QT_PLUGIN_PATH")
> (files '("lib/qt5/plugins")))
> + (search-path-specification
> + (variable "QTWEBENGINEPROCESS_PATH")
> + (files '("lib/qt5/libexec/QtWebEngineProcess")))
> (search-path-specification
> (variable "XDG_DATA_DIRS")
> (files '("share")))
Actually, scratch that part above, as there's already a search path
defined on the qtwebengine package, and it's more correctly defined as:
(native-search-paths
(list (search-path-specification
(file-type 'regular)
(separator #f)
(variable "QTWEBENGINEPROCESS_PATH")
(files '("lib/qt5/libexec/QtWebEngineProcess")))))
So what I proposed above is not needed.
> modified guix/build/qt-build-system.scm
> @@ -86,7 +86,8 @@
> "/cursors" "/wallpapers" "/icons" "/mime")
> '("XDG_CONFIG_DIRS" "/etc/xdg")
> '("QT_PLUGIN_PATH" "/lib/qt5/plugins")
> - '("QML2_IMPORT_PATH" "/lib/qt5/qml"))))
> + '("QML2_IMPORT_PATH" "/lib/qt5/qml")
> + '("QTWEBENGINEPROCESS_PATH" "lib/qt5/libexec/QtWebEngineProcess"))))
>
> (define* (wrap-all-programs #:key inputs outputs
> (qt-wrap-excluded-outputs '())
Still is still useful I think, but I noticed now it needs a leading
slash in from of the lib/qt5/libexec above.
Thanks,
Maxim
^ permalink raw reply [flat|nested] 13+ messages in thread* bug#47569: ‘qt-build-system’ retains too many references via wrappers
2021-04-07 21:23 ` Maxim Cournoyer
@ 2021-04-08 21:32 ` Ludovic Courtès
0 siblings, 0 replies; 13+ messages in thread
From: Ludovic Courtès @ 2021-04-08 21:32 UTC (permalink / raw)
To: Maxim Cournoyer; +Cc: 47569-done
Hi!
Pushed fix as c5fd1b0bd362f8b8578a76a26a65ba5d00d48992.
Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>> Seeing a growing number of packages require a custom wrap phase for
>> qtwebengine, I think the following additions could make sense to be
>> incorporated as part as this Qt-world rebuild:
[...]
>> modified guix/build/qt-build-system.scm
>> @@ -86,7 +86,8 @@
>> "/cursors" "/wallpapers" "/icons" "/mime")
>> '("XDG_CONFIG_DIRS" "/etc/xdg")
>> '("QT_PLUGIN_PATH" "/lib/qt5/plugins")
>> - '("QML2_IMPORT_PATH" "/lib/qt5/qml"))))
>> + '("QML2_IMPORT_PATH" "/lib/qt5/qml")
>> + '("QTWEBENGINEPROCESS_PATH" "lib/qt5/libexec/QtWebEngineProcess"))))
And done in 06eb21856f9535ab62d0becc92b4146e0620654e. Now we can remove
some of these custom phases!
Ludo’.
^ permalink raw reply [flat|nested] 13+ messages in thread
* bug#47569: ‘qt-build-system’ retains too many references via wrappers
2021-04-06 13:17 ` Ludovic Courtès
2021-04-06 13:43 ` Efraim Flashner
2021-04-06 17:43 ` Ludovic Courtès
@ 2021-04-06 17:45 ` Leo Famulari
2 siblings, 0 replies; 13+ messages in thread
From: Leo Famulari @ 2021-04-06 17:45 UTC (permalink / raw)
To: Ludovic Courtès; +Cc: 47569, Maxim Cournoyer
On Tue, Apr 06, 2021 at 03:17:52PM +0200, Ludovic Courtès wrote:
> Hi,
>
> For the record, changing ‘qt-build-system’ would trigger a rebuild of
> less than 400 packages according to the back-of-the-envelope calculation
> below. In that case, it’s tempting to fix on ‘master’ and include it in
> the release.
>
> Thoughts?
Let's do it.
After the release, I think we should reconsider those numbers. Compared
to when we set those guidelines, we have a lot more packages and a lot
more computing power.
^ permalink raw reply [flat|nested] 13+ messages in thread