From mboxrd@z Thu Jan 1 00:00:00 1970 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) Subject: Re: [PATCH] git-download: Speed up 'git-predicate'. Date: Wed, 21 Jun 2017 23:17:02 +0200 Message-ID: <87shitghwx.fsf@gnu.org> References: <20170602070833.25760-1-mail@cbaines.net> <87zidk2c4g.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Return-path: Received: from eggs.gnu.org ([2001:4830:134:3::10]:43894) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dNmzs-0008RA-Qv for guix-devel@gnu.org; Wed, 21 Jun 2017 17:17:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dNmzp-0003dQ-La for guix-devel@gnu.org; Wed, 21 Jun 2017 17:17:08 -0400 In-Reply-To: (Christopher Baines's message of "Mon, 19 Jun 2017 08:24:24 +0100") List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Christopher Baines Cc: guix-devel@gnu.org Hi! Christopher Baines skribis: > On 07/06/17 13:52, Ludovic Court=C3=A8s wrote: >> Christopher Baines skribis: [...] >> Do you know how much the inode optimization vs. the tree structure >> contributes to the performance. > > I've got some more data. I ran the test script for smart-answers, and > let it complete this time: > > real 97m21.291s > user 120m50.400s > sys 0m31.020s > > Just applying the inode optimization gives this result: > > real 90m28.116s > user 100m44.784s > sys 0m18.524s > > I'm going to assume that using the tree structure for directories makes > up the rest of the difference. This will vary between repositories > though, I think smart answers has a unusually large directory to file rat= io. Interesting, thanks for benchmarking! >> Likewise, the (map car =E2=80=A6) call for =E2=80=98match=E2=80=99. :-) > > I'm stuck on this bit though, in the latest patch it reads: > > (list? (member top-directory (map car tree)) > > The list? call is just to turn the list or #f returned by member to #t > or #f. The (map car tree) converts the tree to a list of top level > directories. This bit of code is used when the last directory in the > input list has been reached (e.g. when checking for foo/bar/baz > top-directory would be baz) so the last check to perform is to check if > baz exists at the current level of the tree. Any suggestions on > restructuring this? Probably: (match tree (((heads . _) ...) (->bool (member top-directory heads)))) >>> + (inodes-vhash (alist->vhash >>> + (map >>> + (lambda (file) >>> + (let ((stat >>> + (lstat (string-append directory "/= " file)))) >>> + (cons (stat:ino stat) (stat:dev stat)))) >>> + files))) >>=20 >> I would call it =E2=80=98inodes=E2=80=99 simply. Also, we could use =E2= =80=98list->set=E2=80=99 from >> (guix sets) here. > > I've made the inodes-vhash -> inodes rename, but I was undecided about > using (guix sets), is there a reason you recommended it? Because conceptually it=E2=80=99s a set: what we want to know is whether th= e set contains a given inode/device pair. I=E2=80=99ll comment on v2 of the patch. Thank you, Ludo=E2=80=99.