bug#63870: 29.0.90; project.el can't dynamically populate the project list

[Spencer Baugh <sbaugh@janestreet.com>]

Dmitry Gutov <dmitry@gutov.dev> writes:

> On 28/06/2023 15:56, Eli Zaretskii wrote:
>>>> Once again, this is dangerous; users could easily shoot themselves in
>>>> the foot, because not many are aware of the pitfall of using file
>>>> notifications for many directories.  It makes no sense to warn against
>>>> something and at the same time let callers easily stumble upon that.
>>> I agree with that, I suppose.  Personally I would be fine with a
>>> mandatory 1 or 2 levels of recursion, since I only need 2.  Do you have
>>> a suggestion for what that interface could look like?  It feels a bit
>>> awkward...
>> I'd actually begin by not providing even 1 level.  Let the callers
>> call this new function  explicitly for every directory which they want
>> watching.  If someone ever complains that this is somehow inconvenient
>> (although I don't see why: directory-files is simple to use), then we
>> could consider extending the API.
>
> That sounds about right. But I might go a little further in this
> reasoning... (*)
>
>> But that's MO; please wait for Dmitry to chime in.
>
> [ Sorry for the late response, I'm still uncertain about this patch. ]
>
> (*) ... and ask whether this functionality makes sense built-in.
>
> I appreciate that it's succinct, documented and doesn't take a lot of
> space. But would we say that it covers a significantly general use
> case? Do we know many other developers who would appreciate it? Do a
> lot of devs at Jane Street use Emacs and this same workflow? Should we
> ask people somewhere (emacs-devel/Reddit/etc) whether they will find
> it useful?

It's something we'd use a lot, but I have no problem keeping it locally
for now.  I agree it has some dangers, maybe I'll come up with a good
way to make it safe for the casual user.

> If it's just for one user at this point, then it shouldn't be
> difficult to maintain this code inside the init dir.
>
> Here's also some alternative I could potentially suggest: if you have
> some code which checks out new branches for development, or projects
> to start work on, and it's written in Elisp too, could it just call
> project-remember-project at the end? That would circumvent the need
> for using file watches altogether.

That works great for stuff written in Elisp too, but alas, there are
command line tools to check out branches for development, etc, and I
want them to update Emacs too.

> Or if we do add this to project.el, we should try to make it safe for
> an average user even with a different directory structure. Suppose
> they have a dir D which they call project-watch on, and then they copy
> a big non-project directory inside. That should trigger many
> filenotify events, and since no search would result in success, I
> suppose the watch stays on, and every directory gets scanned up until
> the root. So an easy-to-enable recursive behavior seems dangerous for
> this case.
>
> Needless to say, the user could call this function, spend time on
> other stuff, forget, and then get surprised by things taking longer
> than expected.

Definitely.  I'll think about whether there's a way to avoid such
footguns...

(Maybe I'll revisit this after writing some scripts like this for Emacs
development - although those will definitely only be in Elisp, so it
would be kind of moot)