From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.devel Subject: Re: A project-files implementation for Git projects Date: Fri, 4 Oct 2019 01:49:23 +0300 Message-ID: <1a9a7f78-fbef-176d-9189-7007046d3fb1@yandex.ru> References: <8736h9rdc4.fsf@gnu.org> <87tv9kz2x6.fsf@gnu.org> <87a7bbjdwe.fsf@gnu.org> <87a7ba8uvx.fsf@gnu.org> <87pnk2zvvy.fsf@gnu.org> <87sgows6wy.fsf@gnu.org> <87ef0dy18z.fsf@gnu.org> <87impk675h.fsf@gnu.org> <874l13h30l.fsf@gnu.org> <7386ef98-c151-e1ce-23fa-11470a16f0d3@yandex.ru> <87h84x1zoa.fsf@gnu.org> <4be972f9-45a3-f2aa-f532-d7b8fbe054fd@yandex.ru> <17994e8f-9940-2da5-216e-20a5d3842822@yandex.ru> <87r23u9qjc.fsf@gnu.org> <1ffb046d-f2de-b1d5-d6a9-a6059616cc08@yandex.ru> <87imp5kax0.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="74155"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Fri Oct 04 00:50:55 2019 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1iG9vy-000J5V-AJ for ged-emacs-devel@m.gmane.org; Fri, 04 Oct 2019 00:50:54 +0200 Original-Received: from localhost ([::1]:40670 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iG9vw-0001Tl-C9 for ged-emacs-devel@m.gmane.org; Thu, 03 Oct 2019 18:50:52 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:35837) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iG9uf-0001R5-I4 for emacs-devel@gnu.org; Thu, 03 Oct 2019 18:49:35 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iG9ud-0006gk-7V for emacs-devel@gnu.org; Thu, 03 Oct 2019 18:49:32 -0400 Original-Received: from mail-lj1-x229.google.com ([2a00:1450:4864:20::229]:39301) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iG9uc-0006bI-LT for emacs-devel@gnu.org; Thu, 03 Oct 2019 18:49:31 -0400 Original-Received: by mail-lj1-x229.google.com with SMTP id y3so4501395ljj.6 for ; Thu, 03 Oct 2019 15:49:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=8iPIemIWLH99GSf7vy2WGurD/9y9RDwoi7TK36p2Lvw=; b=WsjBvCW0qRKpy+AAtzthGeDTuYbYfaYumOykA+dmZdLxf/exEsQ8s96Wbum6CVi9jw lx1N2yttGblah+weqxNsYmXOWkphDKTDRPSaMkmQwZfAxTP+MLPvW3Rgbq3lM+MVn+EM NvJYaHVeUPWYaRcx7fhe8ZtZot5/bW/agd+J5WaYuuwL66DRstSZZ1adQEhMqB6WL+Gi 70fTbx2lmjUPXF37iiZ+8Sjm39qqpHVIPRtP1bAW8HfQ3kSJ++gJYdJgpRJjPTKkXYdr wrEmQA3nfGsIiRUcXjnjdhXj1mUdSCiTGGD4ZHHLodqgt0Xt9+5gqaeu6emfHgQ4zimt Ghgw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=8iPIemIWLH99GSf7vy2WGurD/9y9RDwoi7TK36p2Lvw=; b=N9pM/9lDUZlex9CT7HzNMF2fizGn9bm32VrRo2+zgWKCdEa+wqLh7vP8dfhpupWJqR b6kWhIcfSU/WZYWHtL8nn8Bn0BIlje2QjSj7ueoHnMuTPfZ0IFfyMOcYO8zm9o4Ic9sR pWzCjqYVfejd2WvsC95ICnaTQjK26Ks6m+wNXhp/lUFiHZX085JmLL8KAwGVIRUCsLrJ +z6SV797Aj9vGGFSJVcOUIfQ9VR2+pOwm61F8s6coYMphe3n/kIZ2BS/PC5xEav1G1GN WHKeBtf2qdLGMrqRguqexp+Yhv3o2kxMOxtmN3bgL2jhifEBKM75MZZPRA8kA6MGmWN8 gDiQ== X-Gm-Message-State: APjAAAVmwpmGeoKoNZ3YKR3qRuRwaBKRn68FCPD6npnx7zXt/2cLU2QA X/c4ivQh57nkTImLA9Vx4OI6o3S47xU= X-Google-Smtp-Source: APXvYqyZvh8deO22G3CyXH3Oma77pDhgdgNbdjJ2z+NdA//wSF1XL6QZjtQmYrrvQJqtMxIf2Sfhgw== X-Received: by 2002:a05:651c:1032:: with SMTP id w18mr4997372ljm.256.1570142967899; Thu, 03 Oct 2019 15:49:27 -0700 (PDT) Original-Received: from [192.168.10.4] (62-201-46.netrun.cytanet.com.cy. [62.228.201.46]) by smtp.googlemail.com with ESMTPSA id 14sm849592ljs.71.2019.10.03.15.49.26 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 03 Oct 2019 15:49:26 -0700 (PDT) In-Reply-To: <87imp5kax0.fsf@gnu.org> Content-Language: en-US X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::229 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:240538 Archived-At: On 03.10.2019 20:15, Tassilo Horn wrote: > I just tried with the gcc repository with its about 100000 files. Here > it was about equal with ~30secs for both git and find. The git ls-files > invocation on the command line with output piped to /dev/null is done in > one tenth of a second. > > Oh, when dropping the `expand-file-name' call we're doing on every file, > it's ten times faster (~3secs). Nice observation, thanks. It's a significant per-item cost, so no surprise projects with lots of files are (were) doing worse than 'find'. I've changed it to 'concat' now which isn't free, but much faster. I wish someone did some benchmarking with Hg projects. mozilla-central (~200000 files) is still slower on my machine with Hg than with 'find'. We still don't support regexp rules with 'find', but that's probably doesn't affect the speed in this example, and it's a reason to prefer Hg for listing.