From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Dmitry Gutov Newsgroups: gmane.emacs.devel Subject: Re: A project-files implementation for Git projects Date: Thu, 19 Sep 2019 19:01:44 +0300 Message-ID: References: <8736h9rdc4.fsf@gnu.org> <87mufcfz1u.fsf@gnu.org> <87tv9kz2x6.fsf@gnu.org> <87a7bbjdwe.fsf@gnu.org> <87a7ba8uvx.fsf@gnu.org> <87pnk2zvvy.fsf@gnu.org> <87sgows6wy.fsf@gnu.org> <87ef0dy18z.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="249936"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 To: emacs-devel@gnu.org Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Thu Sep 19 18:14:39 2019 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([209.51.188.17]) by blaine.gmane.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1iAz4p-0012vY-Q6 for ged-emacs-devel@m.gmane.org; Thu, 19 Sep 2019 18:14:39 +0200 Original-Received: from localhost ([::1]:46104 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAz4o-0006NO-8m for ged-emacs-devel@m.gmane.org; Thu, 19 Sep 2019 12:14:38 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:41295) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAz4b-00065m-5k for emacs-devel@gnu.org; Thu, 19 Sep 2019 12:14:26 -0400 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iAysP-00012F-IF for emacs-devel@gnu.org; Thu, 19 Sep 2019 12:01:50 -0400 Original-Received: from mail-lf1-x12a.google.com ([2a00:1450:4864:20::12a]:43475) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iAysP-00011z-9J for emacs-devel@gnu.org; Thu, 19 Sep 2019 12:01:49 -0400 Original-Received: by mail-lf1-x12a.google.com with SMTP id u3so2754496lfl.10 for ; Thu, 19 Sep 2019 09:01:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=cNAwCT2TAb0t/5jxHpz7hUsdbv4VIuYOBg5p3T3c3ko=; b=H3Pk8DRIwPxk8PEPp9+N4thchL1c5gnih+FPQ9WcdTK3qKG4U3PX/MPO46yfh7CO3d Geo9CyN92SGbC+cdVJWFZGF9tVtd4YFTX8NbCcNoJxLeDIaCK7EaF5A2+lA5XJNlJBI8 flBL146etIs87F8VzFKE3jT4lJVh+QcjljhMcNGTGMpbuD42iU7h36BTf+LML6Vp8G8j n6XezADf2fdxM/KedGlJx84UbCnHRKBj/vWOqjO1ME6olR940WjmilnI68hbhbm+NCal SDoJV+2R46egEj9Gm8EF9ZbLJ9zQC/u+cYxJj3lOWJHtw+t9Oz77yP4oFI7PDq6nqq+N NuqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=cNAwCT2TAb0t/5jxHpz7hUsdbv4VIuYOBg5p3T3c3ko=; b=rOCiXoAhKhz8Rd9TZuUvm6nAYMy05mdiSbvShQCsD/JHfbJbxeTaCypQv9zk4cJIWS AQY0ADTUGMou6SSXw9pPfi4JZbER6UeORu9mBku1vX1SpXu2HzVSu9S8WuKcbWM1/pB4 If9cqHcE5Ptn5eY4Wfa1drMgFllF0NLCT1QLOg5tHK/Vy9fzlYN18SjM5PiH7g9jfzqD xT18U+GOgXjq20BiY6/29R4Cs5gB+Ds2PyEvofqPFAq35fhkInIHcNN32Etib3VeJqoE Pb9uexB76iTm4QhQKErQay4IiX81Zc+XnN09ZzZr6eCJ+2oZOw0jxHmghqDP7LaHkDyz 3DLA== X-Gm-Message-State: APjAAAWQXdtChwU0/XqJNyJOo1VljRdfE6cN6LPCswJT+ylBil4rPO0W ocHnOthwaWaclpNMfk/0dKNLm+r5e+I= X-Google-Smtp-Source: APXvYqyoxlpUzkpE6GxM92cB5oORmHN3vC2FzRtdy1W4obsSUXGDvzo4N+3ToPBlezd7P3vlofTtuA== X-Received: by 2002:a19:4f5a:: with SMTP id a26mr5350760lfk.116.1568908907213; Thu, 19 Sep 2019 09:01:47 -0700 (PDT) Original-Received: from [192.168.1.142] ([178.252.127.239]) by smtp.googlemail.com with ESMTPSA id m25sm1741588ljg.35.2019.09.19.09.01.45 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 19 Sep 2019 09:01:46 -0700 (PDT) In-Reply-To: <87ef0dy18z.fsf@gnu.org> Content-Language: en-US X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::12a X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:240171 Archived-At: Hi Tassilo, On 18.09.2019 20:15, Tassilo Horn wrote: > Well, ok. I've now played with an interface > > (vc-call-backend (vc-responsible-backend dir) > 'list-files > dir > include-unregistered > extra-includes) Not sure we should use the vc-call-backend route when only 1 or 2 backends will work well enough, but OK, it's not essential. > where extra-includes works in addition to the standard VC ignore rules > (.gitignore, .hgignore). Or do you want to override the VC-internal > rules? I'm afraid it might not work as well if we try to treat all (modified) ignores the same. In my understanding, the speed with which Git lists files to a large extent stems from not having to apply the ignores to the already-registered files. Someone should benchmark this, but I think if we use the "negative pathspec" approach mentioned below for all ignores together, it might slow down file listing by an order of magnitude or several. > At least for Git and Hg, I came up with reasonable implementations: > > --8<---------------cut here---------------start------------->8--- > (defun vc-git-list-files (&optional dir > include-unregistered > extra-ignores) > (let ((default-directory (or dir default-directory)) > (args '("-z"))) > (when include-unregistered > (setq args (nconc args '("-c" "-o" "--exclude-standard")))) > (when extra-ignores > (setq args (nconc args > (mapcan > (lambda (i) > (list "--exclude" i)) > (copy-list extra-ignores))))) > (mapcar > #'expand-file-name > (cl-remove-if > #'string-empty-p > (split-string > (apply #'vc-git--run-command-string nil "ls-files" args) > "\0"))))) > > (defun vc-hg-list-files (&optional dir > include-unregistered > extra-ignores) > (let ((default-directory (or dir default-directory)) > args > files) > (when include-unregistered > (setq args (nconc args '("--all")))) > (when extra-ignores > (setq args (nconc args > (mapcan > (lambda (i) > (list "--exclude" i)) > (copy-list extra-ignores))))) > (with-temp-buffer > (apply #'vc-hg-command t 0 "." > "status" args) > (goto-char (point-min)) > (while (re-search-forward "^[?C]\s+\\(.*\\)$" nil t) > (setq files (cons (expand-file-name (match-string 1)) > files)))) > (nreverse files))) Terrific, thank you! How is Hg's performance with this approach? Does adding a few ignores (like 5 or 10) slow down the output measurably? BTW, can Hg support extra whitelist entries as well? > --8<---------------cut here---------------end--------------->8--- > > There's a semantic difference between Git and Hg in the treatment of > extra-ignores. With Git, the extra-ignores do not rule out committed > files (i.e., they are only effective for untracked files) while for Hg, > they also rule out committed files. I think the Hg semantics are > probably better Better and important, IMO. > but I don't see how to change the Git version so that it > acts the same way (except by re-filtering in lisp, of course), do you? Previously suggested: https://stackoverflow.com/questions/36753573/how-do-i-exclude-files-from-git-ls-files/53083343#53083343 That means converting all extra-ignores into negative pathspec strings. > I haven't looked at the other backends. I guess bzr will probably be > doable, too. However, for SVN, there's no way to list unregistered > files. A correct (but horribly slow) default implementation should also > be doable. Yeah, I wonder if we should treat this as a VC operation. On the other hand, the fallback implementation could just as well use 'find'.