bug#61817: 30.0.50; Project.el finds incorrect project roots in git worktrees

[Dmitry Gutov <dgutov@yandex.ru>]

On 27/02/2023 16:15, Arthur Miller wrote:

>> We could change project-try-vc to follow the link to the parent repo, but how is
>> the rest of it going to work?
>>
>> If the project root is the parent repo, which set of files would (project-files
>> pr) return? And how could that be implemented?
> 
> I don't know how it works now, so I can't really tell you, but you have called
> git root "actual git root", so perhaps something that differentiate between
> directory hierarchy root (worktree), and real git root (project). Perhaps
> project-vc-root, if you don't already use that name, or something similar?

Do you mean you want a separate helper to get to the "worktree parent" 
repository? We could add something like that to VC, I guess. But your 
own code seems to be working well too.

I haven't had a use for "worktree parent" myself, though. And until we 
get a good idea for how people will use a feature, it's a little 
difficult to choose a place for it.

>>> Yes that is what I wanted? :-).
>>> For automation purpose I need to find the project root, so I can pull sources
>>> to
>>> main, create a clean worktree from main, and switch Emacs to the new worktree
>>> interactively in one command, like M-x make-new-patch. Emacs asks me for a name
>>> and create a clean worktree from the main trunk for current project. Actually
>>> better variant is to ask which branch to patch, but the first one is slightly
>>> faster and works just fine in many cases.
>>
>> It sounds like your code is Git-specific, not project-neutral.
> 
> Yepp. Question is what project.el then really is, if it can't handle vc specific
> "projects", if it is only synomim for operations on directory trees? Perhpas it
> should be called dirtree.el, because that is what it does: it works on
> directory trees and uses "project markers" to know where to start/stop? I am not
> trying to by sarcastic or negative.

It uses VCS-specific information and converts it to a shape that should 
be usable without the caller being aware of which underlying data that 
information is determined by.

In particular, it reads .gitmodules, .git and .gitignore.

And, well, different backends can make different decisions on the 
backends's behavior. But, ideally, it should be good for the majority of 
uses. Such as our built-in commands, for example: project-vc-dir, 
project-eshell, etc.

> I don't mean it is not useful, we need both,
> operations that work as projectile/project, on directory trees, but also those
> that actually understand a possible project structure. To me projectile and now
> project.el seems more like they are an extension to Dired, not in some negative
> connotation, but at least conceptually, then really project handling. I am not
> trying to be negative, I am a bit oversimplistic and exaggerating, becuase I am
> trying to illustrate the difference how I think about "projects".

It a way -- yes, because we describe a project as a set of files. With a 
well-defined root. But it doesn't absolutely have to be in the shape of 
a single directory tree, though it usually is.

Whenever one chooses a different shape, though, they should consider how 
that will affect the common uses.

> Version control has become an integral part of all projects, at least in
> software industry, just as building projects, boilerplate generating projects
> etc. What I am saying is that project-neutral is OK, we don't want just a
> different name for Git commands, but any project library nowadays should be able
> to work with vc systems, build systems etc. At least for software management
> projects.

You seem to be assigning a lot of meaning to the fact that the root 
doesn't point to "worktree root", but to the current repository root 
(modulo the search in the parent directory when the current repo is 
detemined to be a submodule).

But would it be a better choice?

When I use worktrees, it's to carry on development in a certain branch. 
Meaning, I need to be able to pull/push/commit in that branch, all 
within the same worktree.

To do a pull, or to run some commands in that worktree, I can call 
project-eshell. To do a commit, I can call up project-vc-dir. In both 
cases, the useful thing to do for 'project-root' is to return the 
current repository root, not the worktree parent. Because you don't make 
new commits in the parent repo when working on a worktree, you make them 
in the current one.

I'm all for adding new Version Control related features, but they should 
be based off realistic, specific user scenarios.

>> But you still could find the worktree root using project.el, and then read the
>> contents of .git, follow the link and do your automation stuff.
> 
> Yeah, but it what does it saves me? In Git case it is really synonym to:
> 
>      (git-dir (locate-dominating-file directory ".git"))
> 
> I can just as well call built-in function instead of requiring project.el.

Sure. locate-dominating-file is an underrated function. If you just want 
to find the current repository root (no matter if it's a submodule or 
etc), the above will easily work. project-try-vc adds some caching on 
top of that, but you might not need it.

> As said I thought project.el was more general to work with projects on a deeper
> level, sort-of EDE replacement or complement, and I wanted to use it as library,
> but I understand now it is not.

The idea was for a more simple/transparent and extensible EDE 
alternative. Or an extension point to plug Projectile in.

It can still grow some additions (such as build tool support), but for 
now we have what we have.