bug#41572: 28.0.50; [PATCH] Support plain project marked with file .emacs-project

[Dmitry Gutov <dgutov@yandex.ru>]

On 05.10.2021 21:19, Nikolay Kudryavtsev wrote:
> I currently have a very basic real use case on my hands. There's a 
> particular programming language that has it's own project file type. 
> Since it's a project type, it makes sense to plug it in as project 
> backend. And then on top of this I can implement project target actions 
> like build, compile and debug(this is actually another matter that 
> probably needs to be implemented in(or over) project.el at some point, 
> I'll probably open a discussion about it later).

Right, we are yet to consider this functionality properly.

> So it all makes sense so far, at least conceptually. But here you're 
> saying "you should not add a new file-based backend until you really 
> think about the project file list optimization first". This violates the 
> classic rule of doing the right thing first, then optimizing second. So 
> while I feel this a real issue, it IMHO should be filed and discussed 
> separately and is a nonblocker for this particular task.

Let's talk about correctness. The "right thing".

Suppose we have a large Git repo, which contains a "foo project" (as you 
might see it), marked by Foofile in its 'foo' subdirectory.

And suppose we allow the project-foo backend to come first before the VC 
backend. When we are inside the directory ./project/foo, that's the 
current project. File listing shows ("./project/foo/a", 
"./project/foo/b", etc).

But when we go up a directory, "./project/". And when asked to list its 
files, how do we avoid including "./project/foo/a" in that list? It 
would make sense to exclude any nested projects, right? But we can't do 
that if the project detection logic is so flexible that the project-vc 
backend couldn't find out about subprojects inside it except by visiting 
every subdirectory and querying for the current project there, which 
would obviously be too slow.

So it's a correctness issue as well. Hence the 
simpler-but-easier-to-get-right approach in the other patch (with the 
project-vc-subprojects variable). Even if it might require more effort 
from the end user, unfortunately.

> Now to contradict myself, lets continue discussing this issue. I think 
> this is a local version of a more global multiple backends problem. Lets 
> say we have the same project(more precisely, a set of files) that is 
> served by multiple backends. Roughly we order project-find-functions in 
> this order: major-mode backends, tool backends(eg. GNU Global), generic 
> backends(VC). The preference for the major mode backends over others is 
> due to that "VC has different root" use case. Tool backends are 
> preferred to VC due to that you can start a new Global project as sort 
> of a custom project hack.

Setting project-find-functions in a major mode is a questionable thing 
to do, because then you end up with Emacs where files in the same 
directory belong to different projects. As we say in the commentary:

;; It is a good idea to depend on the
;; directory only, and not on the current major mode, for example.
;; Because the usual expectation is that all files in the directory
;; belong to the same project (even if some/most of them are ignored).

We want to support even backends where this approach is violated (on a 
best-effort basis), but let's not make this the common scenario.

> And here we run into the problem: our major mode while it provides a 
> backend, does not optimize file listing, but there's a backend that 
> does. I think TRT is project.el choosing a different secondary backend 
> in this case as long as it has the same project root.

If there is a next backend which indicates the same root, why do we need 
the first one?

> For this it needs 
> to have some rules, to know which backend better fits a particular 
> situation, which does not.

Why do you need so many backends, anyway? One per language, one per 
tool, etc. That seems to reduce the concept of a project to "this one 
parent directory containing a file some tool cares about".

What would be the meaning of the value (project-current) returns? 
Suppose I call project-find-file, meaning to jump to another file in the 
same Git repository. And instead I am shown only a list of files in the 
current subdirectory because it contains, say, a Makefile. Is that a 
good idea to enact this kind of behavior automatically?

Or suppose we add a backend that looks for 'Makefile', another for 
'Gemfile', another for 'Rakefile', etc. What user-level commands are 
going to benefit from this setup? A command that shows the available 
Makefile tasks? It can just as well call 'locate-dominating-file' to 
find the nearest directory containing it. Same for 'M-x rake', and so on.