bug#41572: 28.0.50; [PATCH] Support plain project marked with file .emacs-project

[Nikolay Kudryavtsev <nikolay.kudryavtsev@gmail.com>]

> What's your stance of having "filemarker" projects honor (or not 
> honor) .gitignore instructions from the containing VCS repo? 
That's a good question. I think at any non-VC project it is fine that it 
does not honor it, since you get what you've signed up for. I'll come 
back to this later in this letter, since another part of this 
conversation sort of gives the answer to it.

> The problem is not being able to invent a variable, but how to honor 
> it when listing project files. If the same backend doesn't have the 
> information about the contained projects. 
Yes, but at this point it's a separate feature request for a problem 
that already exists at this point, it's just that we rely on other tools 
to solve it for us. Let's say I maintain an application, except for a 
module maintained by my colleague Bob. And since that module may be be 
shared by multiple applications, it is version controlled separately. I 
also may or may not want it to be considered a part of my project for 
the sake of project utility commands. Here we rely on .gitignore and the 
like. Lets say with GTAGS I can decide whether to include the bob module 
too. But all of this works only because those backended tools happen to 
implement this ignore functionality. Lets say your VC is broken and just 
won't ignore some files(happend before in practice with VC) even though 
it ordered to? So any such project-level ignore functionality, while 
useful and would be even more useful with more weaker custom backends, 
is not that important in the context of this discussion in my opinion.

> This is an inevitably slow approach. Can be bearable in a local 
> filesystem; might be fatal over Tramp. 
Yes. That's sort of the unfortunate price we have to pay here. I think 
some caching can be implemented, to limit directory reading operations 
to 1 per level.

> It doesn't seem like you want GTAGS to define project root; you'll 
> only want certain commands, like xref-find-definitions, to use the 
> closes GTAGS file. 
If I've made a GTAGS file at this level with the explicit intention to 
exclude everything else I might as well be be interested in running 
project-find-regexp and the like here, right?

> I'm not seeing the concrete usage scenarios yet. 
Well think more about either the 3 marker(vc, build-tool, GTAGS) example 
or the Bob module example. Lets try to put it in your terms of 
"correctness". Is me wanting the build tool to define the project 
boundary over VC "correct"? Is me possibly wanting GTAGS backend to 
define the project boundary "correct"? Is me wanting to possibly NOT 
exclude the Bob module from the project "correct"? Lets say I've found a 
broken function in the general application and I now want to see whether 
the Bob module uses it too along with any other code by calling 
project-find-regexp. You may answer no to all of those and say that Holy 
Correctness can be only provided by the Holy VC backend, but I don't 
think it's a reasonable approach.

> These settings, as I see it, will be on the project-vc backend.
Are you saying that if I want to declare that my make projects can be 
subordinate to another build tool, I have to declare that in the 
project-vc-backend of all places?

> Okay, but we need both correctness and speed.
Lets for a moment assume that my use cases are "correct". Meaning that 
relying on VC here would be doing the wrong thing fast. Is doing the 
wrong thing fast a good idea?

> Answering "see if there is a fast backend behind this one with the 
> same root" is not very useful, since in this scenario we seemingly (!) 
> don't need the first backend anyway. 
There are two possible scenarios.

A. Backend project marker has the same root as VC.

B. Backend project marker has a different root.

You look ONLY at the scenario A and say "why do we need a new backend"? 
Well we need it ONLY for the scenario B, which you are ignoring.

Again, not gonna repeat, myself on the ignore issue, apart from "you get 
what you've signed up for".

> I think it's apparent at this point that we are venturing into the 
> territory of pretty invasive, backward-incompatible changes to the 
> existing project.el API. 
True, but currently the project.el is underutilized due to its 
simplicity and due to this we can still do reasonable invasive changes 
to the API.

> Right, yeah. Having module detection on a hook seems more flexible 
> than putting it on a method dispatches by project type, because then 
> the Maven extension can work with any project backend, not just some 
> specific Maven-supporting one. 
The possible actions for Maven would already work any project backend, 
it's the question of how to ensure the proper root for it.
> FWIW, Steinar asked for being able to choose whether to act on a 
> module or on a project, and for that such separation seems to be 
> necessary. 

I don't think he cares about Emacs having a distinction, more about the 
abiilty to independently act on(compile) those parts.

> Speaking of build tools -- just as well, if the tool is in the same 
> directory as the project root, you don't need any additional backends. 
> But if you do need extra detection, file-finding logic, that could be 
> put into buildtool-find-functions.
That's the thing, you would always need one.

> Debugging seems to be a separate feature from build tools (requiring 
> its own information, and a fair amount of it). Although, depending on 
> a language and environment, it might require integration with build 
> tools as well.
Yeah, and that's one of the conceptual problems with that approach: 
assuming too much. Project markers(or "build-tools" markers in your 
paradigm) are not limited to build tools, it may be a tags tool for example.
> So what's the problem with this approach, then?
The main problem is that it's conceptually wrong IMHO. In trying to make 
anything not VC have a subservient status you're actually elevating it 
to a special one. Not any project backend needs build-tools, for example 
possible GTAGS won't, while it is quite fine that any project is aware 
of its VC regardless of the backend. So the user defined backends should 
be given primacy and VC should enjoy that (elevated secondary 
subservient) special status, with the fallback VC backend staying there 
too of course. And that's also the answer to your original question 
about honoring .gitignore. If VC is always available regardless of the 
backend, we can surely honor it and also use it for whichever 
optimizations are possible.

Even take Zhou's patch. Lets operate on the assumption that we cannot 
let anyone go astray from the paradise of the Holy VCs correctness. 
Every single problem you've mentioned would befall on those trying to 
use a plain project. Adding a new backend would just let the heathens 
win, since they can add those files anywhere and stray away from the 
light. From such point of view, you can never add new backends, you can 
only add new build tools.

Or to put it another way, whenever people want to mess with project 
backends, what they want is to define a scope in which they want to 
operate. That scope may differ from the scope defined by VC. Also 
there's nothing wrong in VC scope being available at the same time at 
the user declared project scope. What I mean by that is that nobody 
would complain if every project-* utility commands gets a sister named 
project-vc-* that would allow you to the same operation, but explicitly 
within the VC scope of the current project.

> The 'project-vc-subprojects' patch.
This a solution to the subprojects hiding problem discussed earlier, 
perfectly acceptable for me, but as I mentioned before is not something 
I'd use much personally.