bug#63870: 29.0.90; project.el can't dynamically populate the project list

[Dmitry Gutov <dmitry@gutov.dev>]

On 28/06/2023 15:56, Eli Zaretskii wrote:
>>> Once again, this is dangerous; users could easily shoot themselves in
>>> the foot, because not many are aware of the pitfall of using file
>>> notifications for many directories.  It makes no sense to warn against
>>> something and at the same time let callers easily stumble upon that.
>> I agree with that, I suppose.  Personally I would be fine with a
>> mandatory 1 or 2 levels of recursion, since I only need 2.  Do you have
>> a suggestion for what that interface could look like?  It feels a bit
>> awkward...
> I'd actually begin by not providing even 1 level.  Let the callers
> call this new function  explicitly for every directory which they want
> watching.  If someone ever complains that this is somehow inconvenient
> (although I don't see why: directory-files is simple to use), then we
> could consider extending the API.

That sounds about right. But I might go a little further in this 
reasoning... (*)

> But that's MO; please wait for Dmitry to chime in.

[ Sorry for the late response, I'm still uncertain about this patch. ]

(*) ... and ask whether this functionality makes sense built-in.

I appreciate that it's succinct, documented and doesn't take a lot of 
space. But would we say that it covers a significantly general use case? 
Do we know many other developers who would appreciate it? Do a lot of 
devs at Jane Street use Emacs and this same workflow? Should we ask 
people somewhere (emacs-devel/Reddit/etc) whether they will find it useful?

If it's just for one user at this point, then it shouldn't be difficult 
to maintain this code inside the init dir.

Here's also some alternative I could potentially suggest: if you have 
some code which checks out new branches for development, or projects to 
start work on, and it's written in Elisp too, could it just call 
project-remember-project at the end? That would circumvent the need for 
using file watches altogether.

Or if we do add this to project.el, we should try to make it safe for an 
average user even with a different directory structure. Suppose they 
have a dir D which they call project-watch on, and then they copy a big 
non-project directory inside. That should trigger many filenotify 
events, and since no search would result in success, I suppose the watch 
stays on, and every directory gets scanned up until the root. So an 
easy-to-enable recursive behavior seems dangerous for this case.

Needless to say, the user could call this function, spend time on other 
stuff, forget, and then get surprised by things taking longer than expected.