* bug#44053: ‘xdg-mime-database’ profile hook is slow [not found] ` <874kmn9adz.fsf_-_@gnu.org> @ 2020-10-27 23:14 ` Joshua Branson via Bug reports for GNU Guix 2020-11-02 11:12 ` Ludovic Courtès 0 siblings, 1 reply; 3+ messages in thread From: Joshua Branson via Bug reports for GNU Guix @ 2020-10-27 23:14 UTC (permalink / raw) To: Ludovic Courtès Cc: guix-devel, Luis Felipe, 44053, Maxim Cournoyer, zimoun I am just posting my performance numbers. Hopefully you all find it helpful. I am running Guix System on a Lenovo Thinkpad T400 with 8GB on a hard drive. #+BEGIN_SRC sh :results output :exports both guix describe #+END_SRC #+RESULTS: #+begin_example Generation 49 Oct 20 2020 01:15:33 (current) my-personal-packages 547d900 repository URL: https://notabug.org/jbranso/guix-packages.git branch: master commit: 547d900c25d2bb9490946162e0fcb315d6bace68 guix 93c2518 repository URL: https://git.savannah.gnu.org/git/guix.git branch: master commit: 93c251818dcd2bf5f60f42e595504293ee83371a GUIX_PACKAGE_PATH="/home/joshua/prog/gnu/guix/guix-packages/" #+end_example The "XDG MIME database" takes a while. #+BEGIN_SRC sh :results output :exports both time guix build --check $(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv) #+END_SRC #+RESULTS: : The following profile hook will be built: : /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv : building XDG MIME database... : successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv : successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv : /gnu/store/x8q8g9l0jhrpmjjm3xsh3ib1z8l79cyx-xdg-mime-database : : real 0m43.716s : user 0m3.626s : sys 0m0.258s Cold cache: #+BEGIN_SRC sh :results output :exports both drv="$(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv)" sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches' time guix build --check "$drv" #+END_SRC #+RESULTS: : The following profile hook will be built: : /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv : building XDG MIME database... : successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv : successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv : /gnu/store/x8q8g9l0jhrpmjjm3xsh3ib1z8l79cyx-xdg-mime-database : : real 1m29.606s : user 0m1.401s : sys 0m0.238s I have 64 packages in my profile. #+BEGIN_SRC sh :results output :exports both guix package -I | wc --lines #+END_SRC #+RESULTS: : 64 If you want to see all packages: #+BEGIN_SRC sh :results output :exports both guix package -I #+END_SRC #+RESULTS: #+begin_example guile-colorized 0.1 out /gnu/store/5h0jfssgyx49cq82wrmw57an7ngb7kkz-guile-colorized-0.1 font-hack 3.003 out /gnu/store/52r8anazd4rnkq9m3vxk700jga5h0i74-font-hack-3.003 font-dejavu 2.37 out /gnu/store/7y3lvk3xf4im8n44337mc6y0ccysvfia-font-dejavu-2.37 font-adobe-source-code-pro 2.030R-ro-1.050R-it out /gnu/store/l5m74158njcan3p784gqhkir6zyk0bcp-font-adobe-source-code-pro-2.030R-ro-1.050R-it sicp 20170703-1.225c172 out /gnu/store/x2safq6fvcbhp2rljny89qiq74y1xd13-sicp-20170703-1.225c172 unzip 6.0 out /gnu/store/1zx8knb7h79q6cr4kwhbz6z6v609309w-unzip-6.0 font-google-noto 20171025 out /gnu/store/g2szydnbvs7qqy2nf7qylba0rapajmd8-font-google-noto-20171025 font-awesome 4.7.0 out /gnu/store/gknn01i1nr80apmcx42w36cqcz3zzri2-font-awesome-4.7.0 texinfo 6.7 out /gnu/store/w8k9hcigvhzrlrblv8lgqj77sm3833rs-texinfo-6.7 gcc-toolchain 10.2.0 out /gnu/store/pknm43xsza6nlc7bn27djip8fis92akd-gcc-toolchain-10.2.0 recutils 1.8 out /gnu/store/qlx179h9mfdv4zywx2ias2jmkygvs834-recutils-1.8 guile-readline 3.0.2 out /gnu/store/va6l1ivclww22fi38w5h99pb4ndn99hg-guile-readline-3.0.2 gtypist 2.9.5 out /gnu/store/03wd707lyz49f2r14cm9ybyzh7vfvi0j-gtypist-2.9.5 scsh 0.0.0-1.1144324 out /gnu/store/57frd08q6gyd7ysq3aqwlxxmify3w53h-scsh-0.0.0-1.1144324 make 4.3 out /gnu/store/4k33n2nhsnnaxk2ip75gj7xiqdjns5hq-make-4.3 isync 1.3.3 out /gnu/store/745qmx9mmnwq0j3zrd50qcdd2bgv5clz-isync-1.3.3 jmacs 27.1 out /gnu/store/0hqdddkvhi2acxjdssdi5as3wj6cgvnz-jmacs-27.1 glibc-locales 2.31 out /gnu/store/x9c4dk4zc9mi5j9hm3lq1q539kkdhkln-glibc-locales-2.31 guile 3.0.4 out /gnu/store/ah16zr8mmfkqy23rr7jy5a842ca1q9h1-guile-3.0.4 guile-json 4.3.2 out /gnu/store/12y1s0p3585ydf336gq85kjh5217pnzp-guile-json-4.3.2 guile-syntax-highlight 0.1 out /gnu/store/dwjqscckpypqfq38hc97x5ipc9fplpp9-guile-syntax-highlight-0.1 haunt 0.2.4 out /gnu/store/ccvqa9a6yclgkpi9izhpgf1vgjaghqvr-haunt-0.2.4 gnutls 3.6.14 out /gnu/store/0f6agk6xk8rcz99907d39nd1vzmc4abq-gnutls-3.6.14 wget 1.20.3 out /gnu/store/d7b16jqrgb1jz65hl7vnz261dbxmjh3s-wget-1.20.3 gnupg 2.2.23 out /gnu/store/89cjqnar4aacg58wl6cm1h41rym9mv4l-gnupg-2.2.23 sassc 3.6.1 out /gnu/store/6pqn2i0r12ypss13mdvsypskkcpaagji-sassc-3.6.1 qterminal 0.15.0 out /gnu/store/y7qrfa78y785xfajb3gvvhwcm2d02l81-qterminal-0.15.0 dmenu 5.0 out /gnu/store/393frjc5xjaab6nvp3wxpajb1rc6gynr-dmenu-5.0 openssh 8.4p1 out /gnu/store/k8lnv9cqk6m15k18hhzg7w696r1vxx1v-openssh-8.4p1 grim 1.3.1 out /gnu/store/9v5i9q7kwcmq1q31qri0icjkhh3csh9r-grim-1.3.1 evince 3.36.5 out /gnu/store/arwvpbrzsch4w6afkpqln6625dfl0bz8-evince-3.36.5 pulseaudio 13.0 out /gnu/store/19wcjfwdr3hzq1a4wcpld1zdrjichck0-pulseaudio-13.0 libreoffice 6.4.6.2 out /gnu/store/7rbk52nzqmakd5whds5a6f07xrd2k870-libreoffice-6.4.6.2 termite 15 out /gnu/store/8lqfy0mxqnq3xg24gwsxvzpxnvv6firh-termite-15 mpv 0.32.0 out /gnu/store/2i2gzn7fa8qq0f1jkqhiyl0xw63fjr4w-mpv-0.32.0 pinentry 1.1.0 out /gnu/store/bdsk9yq48lx1f7y7xr9sv4yrmm4ync9b-pinentry-1.1.0 glib 2.62.6 doc /gnu/store/ql2226knbh4q04bbdiqcppvs5rjyl3ga-glib-2.62.6-doc pidgin 2.13.0 out /gnu/store/fd06njmc5maid7b474v224iazy7bc20h-pidgin-2.13.0 gimp 2.10.20 out /gnu/store/m397hvwhsd0l6v5d0vqxxkm4iyp0p9w3-gimp-2.10.20 sway 1.4 out /gnu/store/27ggd92in0hpgmkgk79n2iy4angjxwvl-sway-1.4 wf-recorder 0.2.1 out /gnu/store/glc6y75l4czpfyrgbrbgxg38ig004v1p-wf-recorder-0.2.1 wofi 1.1.2 out /gnu/store/qaylx0k6wg3awsamn2rqr41q9c8iwypg-wofi-1.1.2 linphoneqt 4.1.1 out /gnu/store/wlcviacpagxqnw1737x131if0fj7kzjv-linphoneqt-4.1.1 qbittorrent 4.2.5 out /gnu/store/29cjbdmz809pdvmylbx307l2p7zv0ma9-qbittorrent-4.2.5 pavucontrol 4.0 out /gnu/store/0ilm4lncr6gpg4h3zdvaxg1pibkm6934-pavucontrol-4.0 git 2.28.0 send-email /gnu/store/g5x559qk5yxs19jcxqplqv8b9y32vak2-git-2.28.0-send-email flatpak 1.8.2 out /gnu/store/ywfsnccvlr0j6afb5vnl3z4ismvcfbz6-flatpak-1.8.2 xdg-desktop-portal 1.7.2 out /gnu/store/jkjx5dd7s3dp58p2hwsm8n2rm15z90ys-xdg-desktop-portal-1.7.2 xdg-desktop-portal-gtk 1.7.1 out /gnu/store/zhssisyp66vggjnqi5y8nb1xnk55vnw1-xdg-desktop-portal-gtk-1.7.1 xeyes 1.1.2 out /gnu/store/4jx8i19xd94d421n4acy2c1p0nn9h6d0-xeyes-1.1.2 emacs-polymode 0.2 out /gnu/store/gmm1y1xahg44yw9ghsf0kwd40w86q7mf-emacs-polymode-0.2 notmuch 0.31 out /gnu/store/0js5dwj8nxj0x0hxfllgzz4zxlsmyrz4-notmuch-0.31 msmtp 1.8.12 out /gnu/store/s1g4nskvnsx9ww0032xg23548h7mnwvq-msmtp-1.8.12 youtube-viewer 3.7.9 out /gnu/store/jm7q53lpda3a4rs2z7xv7cgig7adpncb-youtube-viewer-3.7.9 guile-curl 0.6 out /gnu/store/9nnclxjxakwx9yc381arqrk1nhld21b5-guile-curl-0.6 qemu 5.1.0 out /gnu/store/qzi36f60d2zlrdj9mg6b5y4i00w1fzfq-qemu-5.1.0 icecat 78.3.1-guix0-preview1 out /gnu/store/71fkykdajhkshsvzwrnqm4lsc6qgycg2-icecat-78.3.1-guix0-preview1 git 2.28.0 out /gnu/store/j66k3x1wz0bc6nh09dwa6i4jwahqcabi-git-2.28.0 guix 1.1.0-29.4e3ed9b out /gnu/store/irirj169rncank4y7i6n8qyiiqbs76qr-guix-1.1.0-29.4e3ed9b ffmpeg 4.3.1 out /gnu/store/sg4yq9i0v1lphvs1vx33dw9mn801adrd-ffmpeg-4.3.1 pavucontrol-qt 0.15.0 out /gnu/store/9d0zp4wps5kpbqsl4yvjb96spb5xwz9b-pavucontrol-qt-0.15.0 clementine 1.3.1-2.4619a4c out /gnu/store/5392fysgmmi2jmn98maz9ihzn9ahzhw9-clementine-1.3.1-2.4619a4c mako 1.4 out /gnu/store/k83v9wz6szf6k2p2sz1amvwqm687j3h2-mako-1.4 ungoogled-chromium 86.0.4240.75-0.c34a56d out /gnu/store/m0s9mrbirbssis5nn9agx8k4sl2ck0y9-ungoogled-chromium-86.0.4240.75-0.c34a56d #+end_example Thanks, -- Joshua Branson Sent from Emacs and Gnus https://gnucode.me https://video.hardlimit.com/accounts/joshua_branson/video-channels "You can have whatever you want, as long as you help enough other people get what they want." - Zig Ziglar ^ permalink raw reply [flat|nested] 3+ messages in thread
* bug#44053: ‘xdg-mime-database’ profile hook is slow 2020-10-27 23:14 ` bug#44053: ‘xdg-mime-database’ profile hook is slow Joshua Branson via Bug reports for GNU Guix @ 2020-11-02 11:12 ` Ludovic Courtès [not found] ` <87ft5rncn5.fsf@gnu.org> 0 siblings, 1 reply; 3+ messages in thread From: Ludovic Courtès @ 2020-11-02 11:12 UTC (permalink / raw) To: zimoun; +Cc: guix-devel, Luis Felipe, 44053, Maxim Cournoyer Hi, Joshua Branson <jbranso@dismail.de> skribis: > The "XDG MIME database" takes a while. > > #+BEGIN_SRC sh :results output :exports both > time guix build --check $(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv) > #+END_SRC > > > #+RESULTS: > : The following profile hook will be built: > : /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv > : building XDG MIME database... > : successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv > : successfully built /gnu/store/lmhklgdscbfp5c6gl81skyz0azfg156m-xdg-mime-database.drv > : /gnu/store/x8q8g9l0jhrpmjjm3xsh3ib1z8l79cyx-xdg-mime-database > : > : real 0m43.716s > : user 0m3.626s > : sys 0m0.258s I found that the MIME database is computed only over the subset of the packages in your profile that provide ‘share/mime/packages’, plus ‘shared-mime-info’. In my profile, only LibreOffice provides that directory, so the union is made over these two directories and that’s what ‘update-mime-database’ works on. Apart from LibreOffice, I found that ‘share/mime/packages’ is provided by at least: hugin, gcr, fontforge. Most GUI packages don’t have it. So in practice, we’re often rebuilding the exact same database. Here’s the time taken by ‘update-mime-database’ alone: --8<---------------cut here---------------start------------->8--- ludo@ribbon ~$ mkdir -p /tmp/mime/share/mime/packages ludo@ribbon ~$ cd /tmp/mime/share/mime/packages ludo@ribbon /tmp/mime/share/mime/packages$ for i in $(guix build libreoffice ^C ludo@ribbon /tmp/mime/share/mime/packages$ for i in ~/.guix-profile/share/mime/packages/* ; do ln -s $i ; done ludo@ribbon /tmp/mime/share/mime/packages$ ls -l totalo 8 lrwxrwxrwx 1 ludo users 64 Nov 2 11:46 freedesktop.org.xml -> /home/ludo/.guix-profile/share/mime/packages/freedesktop.org.xml lrwxrwxrwx 1 ludo users 60 Nov 2 11:46 libreoffice.xml -> /home/ludo/.guix-profile/share/mime/packages/libreoffice.xml ludo@ribbon /tmp/mime/share/mime/packages$ cd /tmp/mime/ ludo@ribbon /tmp/mime$ time update-mime-database -V /tmp/mime/share/mime Updating MIME database in /tmp/mime/share/mime... Parsing source file /tmp/mime/share/mime/packages/freedesktop.org.xml... Parsing source file /tmp/mime/share/mime/packages/libreoffice.xml... Wrote 1124 strings at 2c - 6310 Wrote aliases at 6310 - 6be4 Wrote parents at 6be4 - 8468 Wrote literal globs at 8468 - 855c Wrote suffix globs at 855c - 13514 Wrote full globs at 13514 - 13554 Wrote magic at 13554 - 2065c Wrote namespace list at 2065c - 20798 Wrote icons list at 20798 - 2079c Wrote generic icons list at 2079c - 213b8 Wrote types list at 213b8 - 22020 Note that '/tmp/mime/share' is not in the search path set by the XDG_DATA_HOME and XDG_DATA_DIRS environment variables, so applications may not be able to find it until you set them. The directories currently searched are: - /home/ludo/.local/share - /home/ludo/.guix-profile/share - /run/current-system/profile/share - /home/ludo/.guix-profile/share - /run/current-system/profile/share real 0m2.166s user 0m0.278s sys 0m0.056s --8<---------------cut here---------------end--------------->8--- To be compared with: --8<---------------cut here---------------start------------->8--- $ drv="$(guix gc -R $(guix gc --derivers $(readlink -f ~/.guix-profile)) |grep xdg-mime-database.drv)" $ time guix build --check $drv The following profile hook will be built: /gnu/store/jrmwsxpz3wmq37zx29lvb0r9nvcmdviz-xdg-mime-database.drv building XDG MIME database... successfully built /gnu/store/jrmwsxpz3wmq37zx29lvb0r9nvcmdviz-xdg-mime-database.drv /gnu/store/npvj2sr9kxx48znh7zc8zmzwzs6brc90-xdg-mime-database real 0m3.670s user 0m1.428s sys 0m0.029s --8<---------------cut here---------------end--------------->8--- Thus, the build time itself is entirely taken by ‘update-mime-database’ (the 1.4s above is “overhead” in ‘guix build’ it would seem). Ludo’. ^ permalink raw reply [flat|nested] 3+ messages in thread
[parent not found: <87ft5rncn5.fsf@gnu.org>]
[parent not found: <86blgeggo8.fsf@gmail.com>]
[parent not found: <wfCkfOh1kZkBBqR2DYrffbz9xGwrXpB5lNfNNbLywaXO2i620ONaFST7pkyF_vskC20eUnLuV36YyO6t8v6__7oyVbOzaCoba1muCHDP0S4=@protonmail.com>]
* bug#44053: ‘xdg-mime-database’ profile hook is slow [not found] ` <wfCkfOh1kZkBBqR2DYrffbz9xGwrXpB5lNfNNbLywaXO2i620ONaFST7pkyF_vskC20eUnLuV36YyO6t8v6__7oyVbOzaCoba1muCHDP0S4=@protonmail.com> @ 2020-11-06 9:10 ` Ludovic Courtès 0 siblings, 0 replies; 3+ messages in thread From: Ludovic Courtès @ 2020-11-06 9:10 UTC (permalink / raw) To: Luis Felipe; +Cc: guix-devel, 44053@debbugs.gnu.org, Maxim Cournoyer Hi, Luis Felipe <luis.felipe.la@protonmail.com> skribis: > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ > On Tuesday, November 3, 2020 11:32 PM, zimoun <zimon.toutoune@gmail.com> wrote: > >> Hi, >> >> > https://git.savannah.gnu.org/cgit/guix.git/commit/?id=76ea70bd70aeb76570445c11cea2f98139192b54 >> >> Clever workaround! What are now the performances on previous examples >> (same profiles and same packages)? > > In my case there seem to be no improvement (using Guix 5e7cf66fb35780f930ad0bc5fe21ac330df4411d). Please note that the change above addresses only one specific source of slowness, the ‘xdg-mime-database’ hook, and only in specific cases. It’s good to look at the overall timing of ‘guix install’, because that’s what matters in the end, but as we work on optimizing it, we have to look at specific aspects of it. > $ time guix package -i perl --max-jobs=1 [...] > injertando 12 paquetes en /gnu/store/anknpdyhmfirw3rz2k9zm9kiyak8yy1s-cups-filters-1.27.4.drv ... > construyendo la base de datos MIME XDG... > injertando 3 paquetes en /gnu/store/xgny7xbl635g8na8x03x4cdr7abiphiw-cups-2.3.3.drv ... > injertando 20 paquetes en /gnu/store/yhjl68x7kcjbv40v823x4hl8rvv8l50b-gtk+-2.24.32.drv ... > injertando 21 paquetes en /gnu/store/kq37fnw8335f1hqc3j4hhqqcdnhl371p-gtk+-3.24.20.drv ... > creando la caché de temas de iconos de GTK+... > construyendo los ficheros de caché para los métodos de entrada de GTK+... > construyendo perfil con 86 paquetes... > > > real 8m38,121s > user 0m2,742s > sys 0m0,338s Here it’s likely that grafting is what’s taking the most time on a spinning disk. We should hack (guix status) so it can optionally prefix each event with a timestamp. As far as ‘xdg-mime-database’ is concerned, it should be down to 0s, unless your profile contains one of the packages I cited (libreoffice, gcr, hugin, etc.). Thanks, Ludo’. ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2020-11-06 9:11 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <87o8l7pple.fsf@gnu.org>
[not found] ` <sqv1qTRO4PZuOuanFOc1Dw6QlIMeczAQjg9erpoU9CmuWjuqYl7H87lpEfl7w2zNAT755yE_djMRXz8fKFiXZ8ddUJSdtLkSCoWIl9KREnw=@protonmail.com>
[not found] ` <87h7qxhncq.fsf@gnu.org>
[not found] ` <3aQjRn5jGhxHrhAmuB3MfLRjSxAp3PVFu1vL7_GV1ucGibzYURCtMqHN4oeyGb5xez2xYM3loEJIECrVdnAGZBZvf0N1bGXN-80mO6Ymr6k=@protonmail.com>
[not found] ` <87sgagv8qt.fsf@dismail.de>
[not found] ` <87o8l279eb.fsf@gnu.org>
[not found] ` <CAJ3okZ3DR0qDq5qwY-ZBboBUFcCnv5bAYR8RuUSW0dSj8SRyyA@mail.gmail.com>
[not found] ` <4JD1z15e92ccv6WXcMS5IiNJnDwR6P3bkIwtPWLUPYDVCUHYDoekpifyj3GQtnEq1LCKftbL-FV1hp162A1mid2EmjLMbWlxQ05aE9vGyf0=@protonmail.com>
[not found] ` <87v9f842uu.fsf@gmail.com>
[not found] ` <CAJ3okZ2WTcZ+JLYUCEcU6CESpOOjHcdFEWirSOCfrZFbXeLzAg@mail.gmail.com>
[not found] ` <87v9f7gspm.fsf@dismail.de>
[not found] ` <874kmn9adz.fsf_-_@gnu.org>
2020-10-27 23:14 ` bug#44053: ‘xdg-mime-database’ profile hook is slow Joshua Branson via Bug reports for GNU Guix
2020-11-02 11:12 ` Ludovic Courtès
[not found] ` <87ft5rncn5.fsf@gnu.org>
[not found] ` <86blgeggo8.fsf@gmail.com>
[not found] ` <wfCkfOh1kZkBBqR2DYrffbz9xGwrXpB5lNfNNbLywaXO2i620ONaFST7pkyF_vskC20eUnLuV36YyO6t8v6__7oyVbOzaCoba1muCHDP0S4=@protonmail.com>
2020-11-06 9:10 ` Ludovic Courtès
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/guix.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).