bug#72300: project.el: detect newly created project contained within another

[Dmitry Gutov <dmitry@gutov.dev>]

On 01/10/2024 23:20, Ship Mints wrote:
> On Mon, Sep 30, 2024 at 7:10 PM Dmitry Gutov <dmitry@gutov.dev 
> <mailto:dmitry@gutov.dev>> wrote:
> 
>     On 30/09/2024 17:31, Ship Mints wrote:
> 
>      >     Git attributes is a possible approach, with a downside of
>     extra process
>      >     calls, which over Tramp (for example) would mean additional
>     latency.
>      >
>      >
>      > (defun project--value-in-dir (var dir)already incurs latency over
>     tramp,
>      > right?
> 
>     Yep, but almost every other separate I/O adds to it. So with other
>     things equal, I'd prefer solutions with fewer disk reads - at least for
>     the default behavior.
> 
> Indeed. It would likely have to be a shell-command-to-string via a tramp 
> file name form.

With a file name handler, then? I suppose it's a possibility.

>      >     How about we just support filtering out submodules using the
>      >     project-vc-ignores var? Which can be assigned in .dir-
>     locals.el or
>      >     through other means.
>      >
>      >
>      > Seems simple. Perhaps an abnormal hook so people can customize
>     buffer
>      > local hook via dir locals? If they want to use git attributes, they
>      > could integrate that approach as an added/replacement function. The
>      > function list could default to a function that implements current
>     behavior.
> 
>     I'd like to say yes, but what would be a good place and name for
>     that hook?
> 
>     Note there is an existing hook called before-hack-local-variables-hook
>     which allows one to alter the applied local variables. But that one
>     might run more often than ideal - perhaps try it out and report back.
> 
> 
> I'll dig back into project.el code and take a look when I have some 
> focus to think this through.

Thanks, looking forward to it.

>     Another approach might use a custom project backend which wraps the
>     existing project-vc - it would need to provide a custom implementation
>     for 'project-ignores' and could delegate the rest of the methods to the
>     parent.
> 
> Also possible. I wonder if this is what Spencer wound up doing to defray 
> the cost of running "find" on a large hierarchy.

Not sure. We're still working on measures aiming to improve performance 
in general, anyway.

>     BTW, are you asking about using git attributes because there is an
>     existing workflow that somehow hooks into it, at some of your clients?
>     If not, then editing dir-locals.el to set the value of
>     project-vc-ignores directly seems like an easier approach.
> 
> 
> I've used git attributes as semaphores for subdirectories in a monorepo 
> to identify subprojects (which are not necessarily git submodules, but 
> we have those, too). This helps with certain tooling external to Emacs. 
> Sadly git-attr, it doesn't have a simple exit with 0 for success, non- 
> zero for failure/missing attribute so needs a little parser for its 
> results. The use cases for git attributes in a monorepo help segregate 
> workflows among front end, back end, firmware subprojects, among others, 
> where tooling is pretty different (and developer skill levels) and there 
> are different workflows. I think I mentioned once before that there is 
> no naming convention for custom git attributes that I'm aware of so I 
> simply prefix ours with an underscore. So far, they're just semaphores 
> where we look for check-attr output of "unspecified" vs. any value for a 
> file/directory of interest.
> 
> I haven't been that focused on this aspect in a while. I think there are 
> some potentially tricky things to think through as I sense that 
> project.el is becoming more closely tied to vc integration than mere 
> project file association.

We can talk about project.el, and we can talk about the project-vc 
backend (the "VC-Aware backend", better called). These two are not the 
same, though naturally related. That's one of the complexities - should 
we add a variable to the one or to the other, for example.

 > In the past (I think on github), I talked > about using a meta 
project to graft together disparate directory
> hierarchies for user convenience beyond just symlinks, though those 
> work, when maintained. e.g., how does one find files that are 
> conceptually common to a project that don't share a directory hierarchy 
> root. The biggest example here is that user/admin/management 
> documentation lives in cloud drives, while source code lives locally 
> (via git, mostly). People work across those boundaries. There are meta 
> project files in each disparate root, .project.meta files which all 
> contain the same text to graft them together using tools we have. Some 
> people put in symlinks (like me, I'm lazy) and that helps but not 
> everyone shares the precise directory hierarchies across every machine 
> making symlinks harder to deal with when checked into git (and there are 
> Windows users). This may be a use case that project.el should never 
> address, not sure.

Maybe not never, but it's currently not very tailored to it. You could 
use project-external-roots, but that part is not so integrated.

Or again, a custom backend could wrap that well enough. Some callers of 
project.el rely on the project being contained in its root dir, but most 
do not.