From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Optimizing profile creation Date: Mon, 02 Sep 2013 23:14:27 +0200 Message-ID: <874na33ozw.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:40771) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VGbX8-0007cU-4r for guix-devel@gnu.org; Mon, 02 Sep 2013 17:19:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1VGbX0-0000Bk-QG for guix-devel@gnu.org; Mon, 02 Sep 2013 17:19:38 -0400 Received: from hera.aquilenet.fr ([141.255.128.1]:33055) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1VGbX0-0000Bd-JK for guix-devel@gnu.org; Mon, 02 Sep 2013 17:19:30 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 8FD3614AD for ; Mon, 2 Sep 2013 23:14:28 +0200 (CEST) Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xxCFkGn5bnIP for ; Mon, 2 Sep 2013 23:14:28 +0200 (CEST) Received: from pluto (reverse-83.fdn.fr [80.67.176.83]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 2ABB761A for ; Mon, 2 Sep 2013 23:14:28 +0200 (CEST) List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org To: guix-devel@gnu.org At the GHM, Andreas rightfully noted that building a profile with many files, as is the case when TeX Live is installed, can take quite a bit of time; and every time a package is installed/removed/upgraded, you pay that price again since the profile is built from scratch. Creating a profile builds a symlink forest that is the union of all the package directories that make up the profile. But with a package like TeX Live, until now, this would end up traversing all of its =E2=80=98texmf= =E2=80=99 sub-directory, just to create a symlink forest that copies the =E2=80=98tex= mf=E2=80=99 directory structure. So, commit 43dd920 does the (now obvious) optimization: if a sub-directory, like =E2=80=98texmf=E2=80=99, is only in one package, then i= t symlinks that directory without traversing it. This happens to be a quite frequent situation. For instance, with Emacs in my profile, I would before have this: --8<---------------cut here---------------start------------->8--- $ ls -l /nix/var/nix/profiles/per-user/ludo/guix-profile-104-link/share/ema= cs/ total 16 dr-xr-xr-x 6 root nixbld 4096 Jan 1 1970 24.3/ dr-xr-xr-x 3 root nixbld 12288 Jan 1 1970 site-lisp/ $ ls -l /nix/var/nix/profiles/per-user/ludo/guix-profile-104-link/share/ema= cs/24.3/lisp/ total 2096 lrwxrwxrwx 1 nixbld1 nixbld 87 Jan 1 1970 abbrev.elc -> /nix/store/f3p= xa96mg8ws4kfldzarayd8i58ddz5j-emacs-24.3/share/emacs/24.3/lisp/abbrev.elc lrwxrwxrwx 1 nixbld1 nixbld 89 Jan 1 1970 abbrev.el.gz -> /nix/store/f= 3pxa96mg8ws4kfldzarayd8i58ddz5j-emacs-24.3/share/emacs/24.3/lisp/abbrev.el.= gz lrwxrwxrwx 1 nixbld1 nixbld 86 Jan 1 1970 align.elc -> /nix/store/f3px= a96mg8ws4kfldzarayd8i58ddz5j-emacs-24.3/share/emacs/24.3/lisp/align.elc lrwxrwxrwx 1 nixbld1 nixbld 88 Jan 1 1970 align.el.gz -> /nix/store/f3= pxa96mg8ws4kfldzarayd8i58ddz5j-emacs-24.3/share/emacs/24.3/lisp/align.el.gz [...] --8<---------------cut here---------------end--------------->8--- With this optimization, I have directly this: --8<---------------cut here---------------start------------->8--- $ ls -l /nix/var/nix/profiles/per-user/ludo/guix-profile-105-link/share/ema= cs/ total 16 lrwxrwxrwx 1 nixbld1 nixbld 71 Jan 1 1970 24.3 -> /nix/store/f3pxa96mg= 8ws4kfldzarayd8i58ddz5j-emacs-24.3/share/emacs/24.3/ dr-xr-xr-x 2 root nixbld 12288 Jan 1 1970 site-lisp/ --8<---------------cut here---------------end--------------->8--- No need to traverse the crowded =E2=80=9824.3=E2=80=99 directory since ther= e=E2=80=99s only one package in my profile that provides it. Please report any speedups or bugs you may have. :-) Ludo=E2=80=99.