From mboxrd@z Thu Jan 1 00:00:00 1970 From: Maxim Cournoyer Subject: Re: guix package is slow Date: Mon, 02 Jul 2018 11:48:17 -0400 Message-ID: <87h8lhtz9a.fsf@gmail.com> References: <8a4e9b4cb730094a84ef3ff29233c6dd@riseup.net> <87y3evug2w.fsf@gmail.com> <87sh52hrc2.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:33157) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fa140-00017j-Jx for guix-devel@gnu.org; Mon, 02 Jul 2018 11:48:29 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fa13x-000100-Fh for guix-devel@gnu.org; Mon, 02 Jul 2018 11:48:28 -0400 In-Reply-To: <87sh52hrc2.fsf@gnu.org> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Mon, 02 Jul 2018 12:20:13 +0200") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Ludovic =?utf-8?Q?Court=C3=A8s?= Cc: guix-devel Hi! ludo@gnu.org (Ludovic Court=C3=A8s) writes: > Hello! > > Maxim Cournoyer skribis: > >> swedebugia@riseup.net writes: >> >>> Hi >>> >>> I would like guix package -A/s to be faster. >>> There it takes >3s every time I run the command. > > It=E2=80=99s at 1.5s for me on an SSD and a hot cache, but I agree that o= verall > it=E2=80=99s too slow. > >> I agree that Guix is rather slow when comparing it to traditional >> package managers such as apt; I think the main reason, as you found, is >> that it has to load all the byte-compiled package definition files. > > That, and also it generally has =E2=80=9Cmore work=E2=80=9D to do (see = =E2=80=98apt-get update=E2=80=99 > vs. =E2=80=98guix pull=E2=80=99, or even =E2=80=98guix package -i emacs= =E2=80=99 vs. =E2=80=98apt-get install > emacs=E2=80=99.) That=E2=80=99s not meant as an excuse, just a clarifica= tion of what=E2=80=99s > going on. True, especially for guix pull ;) >>> Would it be possible to populate an index or something when pulling so >>> that we have the information in a sqlite-db instead of traversing 40 >>> go-files with guile? >> >> I remember asking Ludovic in #guix if Guile would be able to compete >> with database software in terms of performance, and he said there are >> ways we can make it faster. > > I think we could easily optimize package lookups by name. Instead of > traversing the whole list of package modules to build up a name/package > table, we could have a pre-built cache containing that mapping. That > way, when typing =E2=80=9Cguix build emacs=E2=80=9D, we could essentially= translate that > to =E2=80=9Cguix build -e '(@ (gnu packages emacs) emacs)'=E2=80=9D, whic= h is a little > bit faster. > > We could arrange for the cache to be enough for operations such as =E2=80= =9Cguix > package -A=E2=80=9D, so that we don=E2=80=99t need to load a single packa= ge module. > >> I'm not very knowledgeable myself about Guile but if we could make it >> support static linking, to load only one binary rather than hundreds >> in the case of Guix, that would be one way to speed things. Next thing >> might be to implement native compilation. > > You might like this: > > https://lists.gnu.org/archive/html/guile-devel/2018-06/msg00026.html Interesting developments are coming to Guile, it seems :). I'm happy to see that Andy is still driving Guile forward. Thanks! >>> On an older guixsd with about 4k packages the delay was noticeably >>> shorter.=20 >> >> I guess the time it takes grows linearly (I hope!) with the number of >> packages/modules it has to load. We're now at double that amount of >> packages (IIRC), so Guix might take double the time it used to. > > Yeah, that=E2=80=99s why the cache is probably unavoidable. Thanks for sharing your insights on the matter! Maxim