bug#59722: 30.0.50; project-find-regexp searches project-ignored files

[Dmitry Gutov <dgutov@yandex.ru>]

On 05/12/2022 01:36, Rudolf Adamkovič via Bug reports for GNU Emacs, the 
Swiss army knife of text editors wrote:

>> Can you step through 'project-find-regexp' with edebug? Does the
>> return value of 'project-files' (saved to 'files') contain the extra
>> entries?
> 
> So, I debugged the problem.

Thanks!

> FINDING 1:
> 
> The first finding may explain why I see the ignored files in C-x p g.
> 
> For my dir-local ignore pattern
> 
>    ((nil . ((project-ignores . ("apps/**/*.scm")))))

project-vc-ignores, you mean.

> Emacs ends up calling
> 
>    git ls-files -z -c --exclude-standard \
>      --no-empty-directory -o -- :(exclude,glob,top)apps/**/*.scm
> 
> which gives a listing that DOES NOT contain (as expected) any '*.scm'
> files in the 'apps' directory, such as
> 
>    apps/lib-app-core/src/main/assets/app/atrium.scm
>    apps/lib-app-core/src/main/assets/app/database.scm
>    ...
> 
> but it still DOES contain "logically empty" parent directories (as not
> expected, perhaps) that would have contained the ignored files if not
> ignored, such as
> 
>    apps/lib-app-core/src/main/assets/app

Huh, interesting. Could you try to show a similar command invocation 
which would make Git output include a "logically empty" directory for 
the Emacs repo? So that we have a common public project to compare.

Or alternatively a project-vc-ignores value which, when put in 
.dir-locals.el at the top, produces such effect on the return value of 
(project-files (project-current)).

As a counter-example, if I add

   (project-vc-ignores . ("./doc/emacs/*"))

there, 'project-files' does not include 'doc/emacs/' at all. Not files 
in it, not it separately as a directory.

> My interpretation:
> 
> git ls-files does not output just files, as one would expect, but also
> directories with no matching files.  grep then gets a list of files and
> possibly also directories, and searches through it all.

That would make sense, if it's indeed what happens. Perhaps it's a bug 
in our code, or this behavior is specific to certain versions of Git.

I currently have Git 2.37.2 installed.

> FINDING 2:
> 
> Given the configuration
> 
>    (setq project-switch-use-entire-map t
>          project-switch-commands '((nil "C-x p")))
> 
> when I type
> 
>    C-x p p <...project...> g
> 
> then Emacs uses the global project ignores and not the ones defined in
> the directory-local file.  However,
> 
>    C-x p g
> 
> inside of any project buffer works OK.

And that one is probably a regression from 2389158a31b4a.

It won't revert cleanly now, unfortunately. Here's a quick patch which 
should fix project-switch-project (but not any other potential uses of 
project-vc values outside of the associated directory):

diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index 3f4a5fb04b..c7fb39b0fb 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -1770,7 +1770,11 @@ project-switch-project
                       project-switch-commands
                     (project--switch-project-command))))
      (let ((project-current-directory-override dir))
-      (call-interactively command))))
+      (with-temp-buffer
+        (let ((default-directory dir)
+              (enable-local-variables :all))
+          (hack-dir-local-variables-non-file-buffer))
+        (call-interactively command)))))

  (provide 'project)
  ;;; project.el ends here

It's mostly to help with your debugging, since it breaks the default 
prompt for 'project-find-regexp' called from 'C-x p p' as well.

I'll have to think of the best way to resolve this: revert said commit 
outright (with updates for the new code), or do it with certain tweaks.

> Notes:
> 
> While probably not the root cause, this bug made debugging much harder,
> until I realized it exists, so I report this finding as well.

Thank you.