From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.devel Subject: Re: A project-files implementation for Git projects Date: Tue, 17 Sep 2019 14:06:03 +0300 Message-ID: References: <8736h9rdc4.fsf@gnu.org> <87mufcfz1u.fsf@gnu.org> <87tv9kz2x6.fsf@gnu.org> <87a7bbjdwe.fsf@gnu.org> <87a7ba8uvx.fsf@gnu.org> <87pnk2zvvy.fsf@gnu.org> <87sgows6wy.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="41811"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 Cc: emacs-devel@gnu.org To: Tassilo Horn Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Tue Sep 17 13:07:58 2019 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1iABKu-000Aij-PG for ged-emacs-devel@m.gmane.org; Tue, 17 Sep 2019 13:07:56 +0200 Original-Received: from localhost ([::1]:44314 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iABKt-0007T1-GH for ged-emacs-devel@m.gmane.org; Tue, 17 Sep 2019 07:07:55 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:52542) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iABJE-0005Rl-H6 for emacs-devel@gnu.org; Tue, 17 Sep 2019 07:06:13 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iABJD-00082t-8s for emacs-devel@gnu.org; Tue, 17 Sep 2019 07:06:12 -0400 Original-Received: from mail-lf1-x12e.google.com ([2a00:1450:4864:20::12e]:41587) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iABJC-000826-W5; Tue, 17 Sep 2019 07:06:11 -0400 Original-Received: by mail-lf1-x12e.google.com with SMTP id r2so2505775lfn.8; Tue, 17 Sep 2019 04:06:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=mOVuspWp79xS3A4Uc2I6fu/4QOOsCK0huVMEKCGpZJI=; b=OXX0rl84tL5YC7PPOU0+DWeUYcLDumbDKPQBRGp+k2I4lkUkqDSl/BmNQwXyNBRX8F CKk71EHeT3FzBMUAsfg2s0Hxcp5RTy7nwbf9zlKlPilUFhU32AvS1dxMxMnbRwNIuvZ9 NFp/iN+NthU2XZJUNf/a0X2ABoLoMI8BZhBl6X1/MW+S4F2Cab/fo6yllQHDQ2CrDV9h f5DKH97/Y4oC4iVPCX0BuhvL+1Cz8zmjjnLt8MGbLfSw5JFmbKgxuqVxpwd/tw1gOQfu GYrP8Qs4ufd8aMthRHfCh/Y/gwP7LXAdmCObKhiD7ezDYXrDtZ+em0wU9Pz+sqqsZgK4 pN8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=mOVuspWp79xS3A4Uc2I6fu/4QOOsCK0huVMEKCGpZJI=; b=feLFshWf8Idc447nSpq3IUWWcC9UPf/fZRQ3V9JxyIQ51lW7Fk4/g6bCEwg/tb0mfU 5/lXlvXU6bRClUCLJmcujioCo8X/vYH7uNU/o18okXjzfVvMM4bkd5kjJUrqDVsdoanO IzCgG6c3GTyB7zq+DUAq8f1pVPHrzHUG5HCfLBHFBB6z+T48j5d38GZ6OjIyFy5thhwS YsWzE1PmzIMAB7d7hmyfVY445NQGc/H46xVqjKSchAVNDT74gxMhvOlzjCIL77GRtBpi mDIiLTcNvDztS4TeLrUuq0AUBbN5d1tM/1OnmkhEFIEO2GtYqNBcB/oEv7eWQdBAyMdl hSxQ== X-Gm-Message-State: APjAAAX82itIqU+6/xlZHhL0YhgwsyWdyd1NI0YQHM9RSygRKYeUujVi ziyfUozjj8KQ9Mtn89JDoJi6NkPJ0+k= X-Google-Smtp-Source: APXvYqySISqfHo28k6jBqVsc6CWmrFciKaN/bLiD81p0FvhO+d5i44S0WiaoBcd1pki9dyLkspl02Q== X-Received: by 2002:ac2:5a4c:: with SMTP id r12mr1736959lfn.52.1568718368619; Tue, 17 Sep 2019 04:06:08 -0700 (PDT) Original-Received: from [192.168.1.142] ([178.252.127.239]) by smtp.googlemail.com with ESMTPSA id z26sm365517ljz.62.2019.09.17.04.06.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 17 Sep 2019 04:06:08 -0700 (PDT) In-Reply-To: <87sgows6wy.fsf@gnu.org> Content-Language: en-US X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::12e X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:240103 Archived-At: Hi Tassilo, On 16.09.2019 16:32, Tassilo Horn wrote: > Ah, "hg status --all" lists all files including their state (untracked, > ignored, you-name-it), so that's the one we should use. Performance > seems to be the same as for "hg files". In my testing the performance difference is about 2x: $ bash -c "time hg status -c >/dev/null" real 0m12,015s user 0m1,899s sys 0m10,113s $ bash -c "time hg files >/dev/null" real 0m5,970s user 0m1,004s sys 0m4,965s (project-files (project-current)) takes ~7 seconds here on the same repo (Mozilla Firefox checkout). But if it's faster than 'find' anyway on some platforms, why not? As long as there's a solution that will handle the adjusted ignore rules in a similarly performant fashion. > I think we can come up with a VC list-files operation which optionally > includes untracked and ignored files (where the latter implies the > former, doesn't it?) Whether it implies or not, depends on which set of ignores we're talking about (Git's own or the modified one). > but I'd leave the filtering according to > project-vc-ignores to project.el. Have you tries benchmarking this approach? E.g. calling 'git ls-files -c -o -z' and then doing all the filtering indicated by .gitignore rules? Try it on the current Emacs repo. IME it's the ignore rules that take up 99% of the CPU time when using 'find'. Without them, 'find .' is instant (though that depends on the disk access speed). If we're going to implement that in Elisp, I'd wager it's going to be even slower. > How would project.el call such a VC list-files operation? I guess you > would include untracked and also ignored files, right? I encourage you to try this approach even with Git only, without the VC facade, and see where we end up. > In my use-case, > the inclusion of ignored files would probably increase the size of the > list of files by a factor of at least 2 because for every *.java file in > our project, there's at least one ignored *.class file (but probably > more like 2-20 *.class files). I'm fairly sure the compilation artefacts aren't going to be the only problem of this approach. > So a new defcustom or include ignored > files only if project-vc-ignores has a non-nil value? I suppose we could end up having different branches of logic for whether the user ends up using this variable, but I'd rather not if the "yes" branch is always going to be slower. It's like punishing them for using a reasonable feature.