Re: A project-files implementation for Git projects

[Dmitry Gutov <dgutov@yandex.ru>]

[-- Attachment #1: Type: text/plain, Size: 2296 bytes --]

Hi Tassilo,

On 27.09.2019 19:17, Tassilo Horn wrote:

> I have to admit that I've lost track over what we're actually trying to
> do exactly.  My main motivation was just to get a faster project-files
> implementation for the Git repo I have at work.  I've now fixed that on
> my end using a simple caching approach which is good enough for me ATM.

I'm attaching a working patch that will probably need tweaks later. I'm 
not sure how to handle attribution best. If you like it, you can simply 
commit it under your name (up to now, I've mostly done the rearranging 
and some minor tweaks).

> So I think that your current idea is to:
> 
> - No general VC list-files operation since we can come up with good
>    versions just for Git and Hg anyway.  Fine with me.

Great. I think we should start with that that either way. And when some 
other application arises that could use a VC 'ls-files' command we 
should see whether we can extract something that satisfies both. Maybe 
the minimum supported Git version will make it easier for us too then.

> - I'm not sure if we're on the same board when it comes to ignores.
> 
> If you want me to implement something I fear you have to explain the
> ignore story again.  I don't know what you imply by renaming the
> parameter from EXTRA-IGNORES to ALL-IGNORES.  Do you mean that the
> patterns in .{git,hg}ignore (and $XDG_CONFIG_HOME/git/ignore,
> $GIT_DIR/info/exclude) should be part of the ALL-IGNORES list (in
> addition to project-vc-ignores) and parsed from those files?

Good point, I forgot about those (and also about per-directory gitignore 
files). ALL-IGNORES would be a nice semantics for a VC 'ls-files' 
command, but if we're not doing that now, we don't have to try to fit 
that approach.

Still, there could be a performance problem with outputting all ignored 
files and then parsing out only a part of them. Could you try the new 
feature with a test repository from 
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=22481?

> Processing
> 3 files in the case of Git and handling 3 ignore syntaxes (regexp, glob,
> rootglob) in the case of Hg doesn't sound too appealing to me.

Speaking of syntaxes, I was curious which one is used by Mercurial's 
--exclude. But it seems to accept globs okay, including ones rooted with 
'./'.

[-- Attachment #2: vc-list-files.diff --]
[-- Type: text/x-patch, Size: 2709 bytes --]

diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index 4693d07fa8..bbb6310323 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -277,6 +277,66 @@ project-try-vc
      (funcall project-vc-external-roots-function)))
    (project-roots project)))
 
+(cl-defmethod project-files ((project (head vc)) &optional dirs)
+  (cl-mapcan
+   (lambda (dir)
+     (let (backend)
+       (if (and (file-equal-p dir (cdr project))
+                (setq backend (vc-responsible-backend dir))
+                (cond
+                 ((eq backend 'Hg))
+                 ((and (eq backend 'Git)
+                       (or
+                        (not project-vc-ignores)
+                        (version<= "1.9" (vc-git--program-version)))))))
+           (project-vc-list-files dir backend project-vc-ignores)
+         (project--files-in-directory
+          dir
+          (project--dir-ignores project dir)))))
+   (or dirs (project-roots project))))
+
+(defun project-vc-list-files (dir backend extra-ignores)
+  (pcase backend
+    (`Git
+     (let ((default-directory dir)
+           (args '("-z")))
+       (when t ;include-unregistered
+         (setq args (append args '("-c" "-o" "--exclude-standard"))))
+       (when extra-ignores
+         (setq args (append args
+                            (cons "--"
+                                  (mapcar
+                                   (lambda (i)
+                                     (if (string-match "\\./" i)
+                                         (format ":!/:%s" (substring i 2))
+                                       (format ":!:%s" i)))
+                                   extra-ignores)))))
+       (mapcar
+        #'expand-file-name
+        (split-string
+         (apply #'vc-git--run-command-string nil "ls-files" args)
+         "\0" t))))
+    (`Hg
+     (let ((default-directory dir)
+           args
+           files)
+       (when t ;include-unregistered
+         (setq args (nconc args '("--all"))))
+       (when extra-ignores
+         (setq args (nconc args
+                           (mapcan
+                            (lambda (i)
+                              (list "--exclude" i))
+                            (copy-list extra-ignores)))))
+       (with-temp-buffer
+         (apply #'vc-hg-command t 0 "."
+                "status" args)
+         (goto-char (point-min))
+         (while (re-search-forward "^[?C]\s+\\(.*\\)$" nil t)
+           (setq files (cons (expand-file-name (match-string 1))
+                             files))))
+       (nreverse files)))))
+
 (cl-defmethod project-ignores ((project (head vc)) dir)
   (let* ((root (cdr project))
           backend)