* bug#63870: 29.0.90; project.el can't dynamically populate the project list @ 2023-06-03 11:55 Spencer Baugh 2023-06-15 19:30 ` Spencer Baugh ` (2 more replies) 0 siblings, 3 replies; 16+ messages in thread From: Spencer Baugh @ 2023-06-03 11:55 UTC (permalink / raw) To: 63870 project.el wants a list of known projects, for project-switch-project and prompting in project-current. Currently this list is maintained in two ways: - automatically, by remembering any project the user runs project commands within - manually by the user with project-remember-project and other functions In both cases, this list is persisted so that it stays around through Emacs restarts. All this is good. But, I often clone new repositories outside of Emacs, and I also have scripts outside Emacs which make new clones automatically. It would be nice for project.el to know about those clones, so I can switch to them with project-switch-project right away. Instead, today I usually manually navigate to those projects the first time, and I'm only able to use project-switch-project on subsequent times. These new repos are created in relatively predictable places, so I can write code which discovers them all. But I don't have a way to tell project.el about them. I could run code on a timer to project-remember-project these projects, since I create them in predictable locations. But that would mean there's a delay between cloning the repo and being able to use it with project-switch-project, which is annoying especially when I manually cloned the repo and want to use it immediately. The new projects are created while Emacs is running, so just remembering them all at startup doesn't work either. I'd like a customization point where I can supply a function (or list of functions) which project-known-project-roots should run to produce an additional list of project root directories, which should then be appended to project--list. I don't need project.el to specifically remember these projects; they'll be remembered automatically as users use them, and completing-read will nicely deduplicate the project roots anyway. Does that seem reasonable? It would be something like this: diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el index 04c67710d71..cc05cf460ef 100644 --- a/lisp/progmodes/project.el +++ b/lisp/progmodes/project.el @@ -1679,11 +1679,21 @@ project-prompt-project-name (let ((proj (assoc pr-name choices))) (if (stringp proj) proj (project-root (cdr proj))))))) +(defcustom project-dynamic-roots '() + "List of functions to call to dynamically find projects. + +Each is called with no arguments and should return a list of +project root dirs." + :type '(repeat function) + :group 'project + :version "30.1") + ;;;###autoload (defun project-known-project-roots () "Return the list of root directories of all known projects." (project--ensure-read-project-list) - (mapcar #'car project--list)) + (flatten-tree (cons (mapcar #'car project--list) + (mapcar #'funcall project-dynamic-roots)))) ;;;###autoload (defun project-execute-extended-command () ^ permalink raw reply related [flat|nested] 16+ messages in thread
* bug#63870: 29.0.90; project.el can't dynamically populate the project list 2023-06-03 11:55 bug#63870: 29.0.90; project.el can't dynamically populate the project list Spencer Baugh @ 2023-06-15 19:30 ` Spencer Baugh 2023-06-16 5:45 ` Eli Zaretskii 2023-06-17 2:55 ` Dmitry Gutov 2023-06-27 19:27 ` Spencer Baugh 2 siblings, 1 reply; 16+ messages in thread From: Spencer Baugh @ 2023-06-15 19:30 UTC (permalink / raw) To: 63870 Actually I think a file-notify solution works great. I have one working for my use case and I'll post a patch for a generic one soon. ^ permalink raw reply [flat|nested] 16+ messages in thread
* bug#63870: 29.0.90; project.el can't dynamically populate the project list 2023-06-15 19:30 ` Spencer Baugh @ 2023-06-16 5:45 ` Eli Zaretskii 0 siblings, 0 replies; 16+ messages in thread From: Eli Zaretskii @ 2023-06-16 5:45 UTC (permalink / raw) To: Spencer Baugh; +Cc: 63870 > From: Spencer Baugh <sbaugh@janestreet.com> > Date: Thu, 15 Jun 2023 15:30:50 -0400 > > Actually I think a file-notify solution works great. I have one working > for my use case and I'll post a patch for a generic one soon. Please keep in mind that in our experience watching a repository tree is problematic (if that is what you had in mind). ^ permalink raw reply [flat|nested] 16+ messages in thread
* bug#63870: 29.0.90; project.el can't dynamically populate the project list 2023-06-03 11:55 bug#63870: 29.0.90; project.el can't dynamically populate the project list Spencer Baugh 2023-06-15 19:30 ` Spencer Baugh @ 2023-06-17 2:55 ` Dmitry Gutov 2023-06-27 19:29 ` Spencer Baugh 2023-06-27 19:27 ` Spencer Baugh 2 siblings, 1 reply; 16+ messages in thread From: Dmitry Gutov @ 2023-06-17 2:55 UTC (permalink / raw) To: Spencer Baugh, 63870 Hi! On 03/06/2023 14:55, Spencer Baugh wrote: > I'd like a customization point where I can supply a function (or list of > functions) which project-known-project-roots should run to produce an > additional list of project root directories, which should then be > appended to project--list. Are you sure the existing functions won't cut it? Such as project-remember-project and project-forget-project. The names might seem a little wrong, but keeping in mind that project--list is about having a list of projects "remembered" somewhere, they're probably fine. And you could let-bind project--list somewhere at the top level in your function/command/etc, so the list is not altered in the end. > I don't need project.el to specifically remember these projects; they'll > be remembered automatically as users use them, and completing-read will > nicely deduplicate the project roots anyway. We could add some var like project-list-no-write, so that dynamic changes don't get written to disk. Unless you want to keep the "dynamic" list to be used by regular commands, that is. In that case, the above will probably not suffice. We could go with your approach, or even add some project-list-sources hook. But how would the entries from different sources (e.g. the list file and your dynamic list) combined? How will they be sorted? In case we can't come up with a generic way, we could simply add a generic storage abstraction (similar to xref-history-storage), which you would override/advise to combine the lists as needed for your usage. ^ permalink raw reply [flat|nested] 16+ messages in thread
* bug#63870: 29.0.90; project.el can't dynamically populate the project list 2023-06-17 2:55 ` Dmitry Gutov @ 2023-06-27 19:29 ` Spencer Baugh 0 siblings, 0 replies; 16+ messages in thread From: Spencer Baugh @ 2023-06-27 19:29 UTC (permalink / raw) To: Dmitry Gutov; +Cc: 63870 Dmitry Gutov <dmitry@gutov.dev> writes: > Hi! > > On 03/06/2023 14:55, Spencer Baugh wrote: >> I'd like a customization point where I can supply a function (or list of >> functions) which project-known-project-roots should run to produce an >> additional list of project root directories, which should then be >> appended to project--list. > > Are you sure the existing functions won't cut it? Such as > project-remember-project and project-forget-project. > > The names might seem a little wrong, but keeping in mind that > project--list is about having a list of projects "remembered" > somewhere, they're probably fine. And you could let-bind project--list > somewhere at the top level in your function/command/etc, so the list > is not altered in the end. Oh, certainly project-{remember,forget}-project work, indeed I've used them in the patch I just posted. I think I was unclear about what I wanted, perhaps that patch clarifies what exactly I was looking for. >> I don't need project.el to specifically remember these projects; they'll >> be remembered automatically as users use them, and completing-read will >> nicely deduplicate the project roots anyway. > > We could add some var like project-list-no-write, so that dynamic > changes don't get written to disk. > > Unless you want to keep the "dynamic" list to be used by regular > commands, that is. > > In that case, the above will probably not suffice. We could go with > your approach, or even add some project-list-sources hook. > > But how would the entries from different sources (e.g. the list file > and your dynamic list) combined? How will they be sorted? > > In case we can't come up with a generic way, we could simply add a > generic storage abstraction (similar to xref-history-storage), which > you would override/advise to combine the lists as needed for your > usage. I don't think these changes are necessary after all. (Which is good, because they would add a fair bit of complexity.) This file-notify approach I think should work perfectly for a wide variety of uses. ^ permalink raw reply [flat|nested] 16+ messages in thread
* bug#63870: 29.0.90; project.el can't dynamically populate the project list 2023-06-03 11:55 bug#63870: 29.0.90; project.el can't dynamically populate the project list Spencer Baugh 2023-06-15 19:30 ` Spencer Baugh 2023-06-17 2:55 ` Dmitry Gutov @ 2023-06-27 19:27 ` Spencer Baugh 2023-06-28 11:24 ` Eli Zaretskii 2 siblings, 1 reply; 16+ messages in thread From: Spencer Baugh @ 2023-06-27 19:27 UTC (permalink / raw) To: 63870 [-- Attachment #1: Type: text/plain, Size: 488 bytes --] Add project-watch to discover projects with file-notify Projects can be created outside of Emacs, but users might want to be able to switch to them with project-switch-project immediately. This function supports that. If a user calls (project-watch "~/src" 1) then any projects under ~/src will be discovered automatically immediately after their creation. I have strived to make this function usable for a wide set of use-cases, I think it will be a useful addition to project.el. [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #2: 0001-Add-project-watch-to-discover-projects-with-file-not.patch --] [-- Type: text/x-patch, Size: 3362 bytes --] From b283eb7eec428b8b0027a28f8c5f547360386b80 Mon Sep 17 00:00:00 2001 From: Spencer Baugh <sbaugh@janestreet.com> Date: Tue, 27 Jun 2023 15:25:02 -0400 Subject: [PATCH] Add project-watch to discover projects with file-notify Projects can be created outside of Emacs, but users might want to be able to switch to them with project-switch-project immediately. This API supports that. If a user calls (project-watch "~/src" 1) then any projects under ~/src will be discovered automatically immediately after their creation. * lisp/progmodes/project.el (project-check-project) (project--watch-cb-children, project--watch-cb-this) (project--file-notify-watch, project-watch): Add. (bug#63870) --- lisp/progmodes/project.el | 51 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el index 56c524bcab5..ddb033d50f9 100644 --- a/lisp/progmodes/project.el +++ b/lisp/progmodes/project.el @@ -1884,5 +1884,56 @@ project-switch-project (let ((project-current-directory-override dir)) (call-interactively command)))) +(defun project-check-project (dir) + "If there's a project at DIR, remember it; otherwise, forget it. + +Return the found project, if any." + (let ((pr (project--find-in-directory dir))) + (if pr (project-remember-project pr) + (project-forget-project (file-name-as-directory dir))) + pr)) + +(defun project--watch-cb-children (recursive predicate event) + (unless (eq (cl-second event) 'stopped) + (dolist (file (cddr event)) + (condition-case _ (project-watch file recursive predicate) + ((file-error file-notify-error)))))) + +(defun project--watch-cb-this (dir event) + (unless (eq (cl-second event) 'stopped) + (when (project-check-project dir) + (file-notify-rm-watch (cl-first event))))) + +(defun project--file-notify-watch (dir callback &optional init) + "Like `file-notify-add-watch' but also calls CALLBACK immediately." + (let ((watch (file-notify-add-watch dir '(change) callback))) + (funcall callback (append (list watch 'started) init)))) + +;;;###autoload +(defun project-watch (dir &optional recursive predicate) + "Watch DIR until it becomes a project. + +We stop watching DIR once it becomes a project. + +If RECURSIVE is an integer greater than 0, we'll also run +`project-watch' on directories which appear inside DIR, +passing (1- RECURSIVE) as RECURSIVE. To achieve this, we'll +continue watching DIR even if it becomes a project. This can be +expensive, so it's better to pass small values of RECURSIVE, like +1 or 2. + +If PREDICATE is non-nil, it should be a function which will be +called with two arguments, the value of RECURSIVE and a +directory. Only directories for which PREDICATE returns non-nil +will be watched for being a project." + (setq predicate (or predicate (lambda (_recursive dir) t))) + (setq recursive (or recursive 0)) + (when (and (funcall predicate recursive dir) (file-directory-p dir)) + (project--file-notify-watch dir (apply-partially #'project--watch-cb-this dir))) + (when (> recursive 0) + (project--file-notify-watch + dir (apply-partially #'project--watch-cb-children (1- recursive) predicate) + (directory-files dir 'full directory-files-no-dot-files-regexp)))) + (provide 'project) ;;; project.el ends here -- 2.39.3 ^ permalink raw reply related [flat|nested] 16+ messages in thread
* bug#63870: 29.0.90; project.el can't dynamically populate the project list 2023-06-27 19:27 ` Spencer Baugh @ 2023-06-28 11:24 ` Eli Zaretskii 2023-06-28 12:05 ` Spencer Baugh 0 siblings, 1 reply; 16+ messages in thread From: Eli Zaretskii @ 2023-06-28 11:24 UTC (permalink / raw) To: Spencer Baugh; +Cc: 63870 > From: Spencer Baugh <sbaugh@janestreet.com> > Date: Tue, 27 Jun 2023 15:27:30 -0400 > > +(defun project-check-project (dir) > + "If there's a project at DIR, remember it; otherwise, forget it. > + > +Return the found project, if any." > + (let ((pr (project--find-in-directory dir))) > + (if pr (project-remember-project pr) > + (project-forget-project (file-name-as-directory dir))) > + pr)) > + > +(defun project--watch-cb-children (recursive predicate event) > + (unless (eq (cl-second event) 'stopped) > + (dolist (file (cddr event)) > + (condition-case _ (project-watch file recursive predicate) > + ((file-error file-notify-error)))))) > + > +(defun project--watch-cb-this (dir event) > + (unless (eq (cl-second event) 'stopped) > + (when (project-check-project dir) > + (file-notify-rm-watch (cl-first event))))) > + > +(defun project--file-notify-watch (dir callback &optional init) > + "Like `file-notify-add-watch' but also calls CALLBACK immediately." > + (let ((watch (file-notify-add-watch dir '(change) callback))) > + (funcall callback (append (list watch 'started) init)))) Beware of watching a tree recursively: file notifications are not very scalable, for more than one reason. For example, the inotify backend consumes a file descriptor and a slot in the descriptor set monitored by pselect per each file/directory you watch. And watching many directories can overwhelm Emacs if some program (even unrelated to Emacs) performs many file operations in that directory; VCS programs are notorious in this regard, e.g., when you update from upstream. > +(defun project-watch (dir &optional recursive predicate) > + "Watch DIR until it becomes a project. > + > +We stop watching DIR once it becomes a project. This never explains what it means for a directory to "become a project". It should, because this doc string begs that question. > +If RECURSIVE is an integer greater than 0, we'll also run > +`project-watch' on directories which appear inside DIR, > +passing (1- RECURSIVE) as RECURSIVE. To achieve this, we'll > +continue watching DIR even if it becomes a project. This can be > +expensive, so it's better to pass small values of RECURSIVE, like > +1 or 2. Are you sure this feature justifies the risks? When would someone want to use it, while simultaneously limiting the value of RECURSIVE to some small integer? (And what is considered "small" for these purposes?) Thanks. ^ permalink raw reply [flat|nested] 16+ messages in thread
* bug#63870: 29.0.90; project.el can't dynamically populate the project list 2023-06-28 11:24 ` Eli Zaretskii @ 2023-06-28 12:05 ` Spencer Baugh 2023-06-28 12:18 ` Eli Zaretskii 0 siblings, 1 reply; 16+ messages in thread From: Spencer Baugh @ 2023-06-28 12:05 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 63870 Eli Zaretskii <eliz@gnu.org> writes: >> From: Spencer Baugh <sbaugh@janestreet.com> >> Date: Tue, 27 Jun 2023 15:27:30 -0400 >> >> +(defun project-check-project (dir) >> + "If there's a project at DIR, remember it; otherwise, forget it. >> + >> +Return the found project, if any." >> + (let ((pr (project--find-in-directory dir))) >> + (if pr (project-remember-project pr) >> + (project-forget-project (file-name-as-directory dir))) >> + pr)) >> + >> +(defun project--watch-cb-children (recursive predicate event) >> + (unless (eq (cl-second event) 'stopped) >> + (dolist (file (cddr event)) >> + (condition-case _ (project-watch file recursive predicate) >> + ((file-error file-notify-error)))))) >> + >> +(defun project--watch-cb-this (dir event) >> + (unless (eq (cl-second event) 'stopped) >> + (when (project-check-project dir) >> + (file-notify-rm-watch (cl-first event))))) >> + >> +(defun project--file-notify-watch (dir callback &optional init) >> + "Like `file-notify-add-watch' but also calls CALLBACK immediately." >> + (let ((watch (file-notify-add-watch dir '(change) callback))) >> + (funcall callback (append (list watch 'started) init)))) > > Beware of watching a tree recursively: file notifications are not very > scalable, for more than one reason. For example, the inotify backend > consumes a file descriptor and a slot in the descriptor set monitored > by pselect per each file/directory you watch. And watching many > directories can overwhelm Emacs if some program (even unrelated to > Emacs) performs many file operations in that directory; VCS programs > are notorious in this regard, e.g., when you update from upstream. Absolutely. I am trying to be careful about this: project-watch shouldn't create watches on VCS directories. >> +(defun project-watch (dir &optional recursive predicate) >> + "Watch DIR until it becomes a project. >> + >> +We stop watching DIR once it becomes a project. > > This never explains what it means for a directory to "become a > project". It should, because this doc string begs that question. A directory "becomes a project" once some function on project-find-functions returns non-nil for it. I'll include this in the docstring of the next version of the patch. >> +If RECURSIVE is an integer greater than 0, we'll also run >> +`project-watch' on directories which appear inside DIR, >> +passing (1- RECURSIVE) as RECURSIVE. To achieve this, we'll >> +continue watching DIR even if it becomes a project. This can be >> +expensive, so it's better to pass small values of RECURSIVE, like >> +1 or 2. > > Are you sure this feature justifies the risks? When would someone > want to use it, while simultaneously limiting the value of RECURSIVE > to some small integer? (And what is considered "small" for these > purposes?) Imagine, for example, that a user has a directory ~/src. They make all their VCS clones directly under ~/src: ~/src/emacs, ~/src/glibc, etc. And when they work on a new project, they create that new clone under ~/src. If the user wanted all these VCS clones to show up in Emacs as soon as they're made, they could run (project-watch "~/src" 1). This would create a watch on ~/src, which would create watches on new empty directories under ~/src (e.g. ~/src/gdb); the watch on ~/src/gdb would stop if and when ~/src/gdb becomes a project (as defined above). So in the steady state, if ~/src contains only projects, Emacs would run exactly one watch, the one on ~/src. This is definitely okay. If, instead, ~/src has a two-level structure, where ~/src/emacs is not itself a clone but instead contains a clone for each branch, e.g. ~/src/emacs/emacs-29 and ~/src/emacs/trunk, then a user might run (project-watch "~/src" 2). Then in the steady state there would be one watch on ~/src and one watch on each subdirectory of ~/src, e.g. ~/src/emacs. (This is the setup I personally have.) ^ permalink raw reply [flat|nested] 16+ messages in thread
* bug#63870: 29.0.90; project.el can't dynamically populate the project list 2023-06-28 12:05 ` Spencer Baugh @ 2023-06-28 12:18 ` Eli Zaretskii 2023-06-28 12:37 ` Spencer Baugh 0 siblings, 1 reply; 16+ messages in thread From: Eli Zaretskii @ 2023-06-28 12:18 UTC (permalink / raw) To: Spencer Baugh; +Cc: 63870 > From: Spencer Baugh <sbaugh@janestreet.com> > Cc: 63870@debbugs.gnu.org > Date: Wed, 28 Jun 2023 08:05:23 -0400 > > Eli Zaretskii <eliz@gnu.org> writes: > > > Beware of watching a tree recursively: file notifications are not very > > scalable, for more than one reason. For example, the inotify backend > > consumes a file descriptor and a slot in the descriptor set monitored > > by pselect per each file/directory you watch. And watching many > > directories can overwhelm Emacs if some program (even unrelated to > > Emacs) performs many file operations in that directory; VCS programs > > are notorious in this regard, e.g., when you update from upstream. > > Absolutely. I am trying to be careful about this: project-watch > shouldn't create watches on VCS directories. But below you explicitly give an example where it will. And given the fact that the majority of project.el projects use VCS as its backend, I'd say we are already there... > > Are you sure this feature justifies the risks? When would someone > > want to use it, while simultaneously limiting the value of RECURSIVE > > to some small integer? (And what is considered "small" for these > > purposes?) > > Imagine, for example, that a user has a directory ~/src. They make all > their VCS clones directly under ~/src: ~/src/emacs, ~/src/glibc, etc. > And when they work on a new project, they create that new clone under > ~/src. > > If the user wanted all these VCS clones to show up in Emacs as soon as > they're made, they could run (project-watch "~/src" 1). This would > create a watch on ~/src, which would create watches on new empty > directories under ~/src (e.g. ~/src/gdb); the watch on ~/src/gdb would > stop if and when ~/src/gdb becomes a project (as defined above). > > So in the steady state, if ~/src contains only projects, Emacs would run > exactly one watch, the one on ~/src. This is definitely okay. > > If, instead, ~/src has a two-level structure, where ~/src/emacs is not > itself a clone but instead contains a clone for each branch, > e.g. ~/src/emacs/emacs-29 and ~/src/emacs/trunk, then a user might run > (project-watch "~/src" 2). Then in the steady state there would be one > watch on ~/src and one watch on each subdirectory of ~/src, > e.g. ~/src/emacs. (This is the setup I personally have.) If you want to support one or two levels of recursion, let's support just that and remove the too-general RECURSIVE argument. If you think there might be important use cases where there's more than one or two levels of recursion, please describe them. Once again, this is dangerous; users could easily shoot themselves in the foot, because not many are aware of the pitfall of using file notifications for many directories. It makes no sense to warn against something and at the same time let callers easily stumble upon that. ^ permalink raw reply [flat|nested] 16+ messages in thread
* bug#63870: 29.0.90; project.el can't dynamically populate the project list 2023-06-28 12:18 ` Eli Zaretskii @ 2023-06-28 12:37 ` Spencer Baugh 2023-06-28 12:56 ` Eli Zaretskii 0 siblings, 1 reply; 16+ messages in thread From: Spencer Baugh @ 2023-06-28 12:37 UTC (permalink / raw) To: Eli Zaretskii; +Cc: 63870 Eli Zaretskii <eliz@gnu.org> writes: >> From: Spencer Baugh <sbaugh@janestreet.com> >> Cc: 63870@debbugs.gnu.org >> Date: Wed, 28 Jun 2023 08:05:23 -0400 >> >> Eli Zaretskii <eliz@gnu.org> writes: >> >> > Beware of watching a tree recursively: file notifications are not very >> > scalable, for more than one reason. For example, the inotify backend >> > consumes a file descriptor and a slot in the descriptor set monitored >> > by pselect per each file/directory you watch. And watching many >> > directories can overwhelm Emacs if some program (even unrelated to >> > Emacs) performs many file operations in that directory; VCS programs >> > are notorious in this regard, e.g., when you update from upstream. >> >> Absolutely. I am trying to be careful about this: project-watch >> shouldn't create watches on VCS directories. > > But below you explicitly give an example where it will. And given the > fact that the majority of project.el projects use VCS as its backend, > I'd say we are already there... No: the watch on a directory is removed once the directory becomes a VCS directory. >> > Are you sure this feature justifies the risks? When would someone >> > want to use it, while simultaneously limiting the value of RECURSIVE >> > to some small integer? (And what is considered "small" for these >> > purposes?) >> >> Imagine, for example, that a user has a directory ~/src. They make all >> their VCS clones directly under ~/src: ~/src/emacs, ~/src/glibc, etc. >> And when they work on a new project, they create that new clone under >> ~/src. >> >> If the user wanted all these VCS clones to show up in Emacs as soon as >> they're made, they could run (project-watch "~/src" 1). This would >> create a watch on ~/src, which would create watches on new empty >> directories under ~/src (e.g. ~/src/gdb); the watch on ~/src/gdb would >> stop if and when ~/src/gdb becomes a project (as defined above). >> >> So in the steady state, if ~/src contains only projects, Emacs would run >> exactly one watch, the one on ~/src. This is definitely okay. >> >> If, instead, ~/src has a two-level structure, where ~/src/emacs is not >> itself a clone but instead contains a clone for each branch, >> e.g. ~/src/emacs/emacs-29 and ~/src/emacs/trunk, then a user might run >> (project-watch "~/src" 2). Then in the steady state there would be one >> watch on ~/src and one watch on each subdirectory of ~/src, >> e.g. ~/src/emacs. (This is the setup I personally have.) > > If you want to support one or two levels of recursion, let's support > just that and remove the too-general RECURSIVE argument. If you think > there might be important use cases where there's more than one or two > levels of recursion, please describe them. Hm, well, I assume some users might use even more structure than this; for example, some might have ~/src/gnu/emacs/emacs-29. Then they'd want 3 levels of recursion. > Once again, this is dangerous; users could easily shoot themselves in > the foot, because not many are aware of the pitfall of using file > notifications for many directories. It makes no sense to warn against > something and at the same time let callers easily stumble upon that. I agree with that, I suppose. Personally I would be fine with a mandatory 1 or 2 levels of recursion, since I only need 2. Do you have a suggestion for what that interface could look like? It feels a bit awkward... ^ permalink raw reply [flat|nested] 16+ messages in thread
* bug#63870: 29.0.90; project.el can't dynamically populate the project list 2023-06-28 12:37 ` Spencer Baugh @ 2023-06-28 12:56 ` Eli Zaretskii 2023-07-18 2:21 ` Dmitry Gutov 0 siblings, 1 reply; 16+ messages in thread From: Eli Zaretskii @ 2023-06-28 12:56 UTC (permalink / raw) To: Spencer Baugh; +Cc: 63870 > From: Spencer Baugh <sbaugh@janestreet.com> > Cc: 63870@debbugs.gnu.org > Date: Wed, 28 Jun 2023 08:37:58 -0400 > > Eli Zaretskii <eliz@gnu.org> writes: > > >> Absolutely. I am trying to be careful about this: project-watch > >> shouldn't create watches on VCS directories. > > > > But below you explicitly give an example where it will. And given the > > fact that the majority of project.el projects use VCS as its backend, > > I'd say we are already there... > > No: the watch on a directory is removed once the directory becomes a VCS > directory. No, AFAIU the watch is removed once there is a project in the directory. But a VCS can work on a directory regardles of any project creation. > > Once again, this is dangerous; users could easily shoot themselves in > > the foot, because not many are aware of the pitfall of using file > > notifications for many directories. It makes no sense to warn against > > something and at the same time let callers easily stumble upon that. > > I agree with that, I suppose. Personally I would be fine with a > mandatory 1 or 2 levels of recursion, since I only need 2. Do you have > a suggestion for what that interface could look like? It feels a bit > awkward... I'd actually begin by not providing even 1 level. Let the callers call this new function explicitly for every directory which they want watching. If someone ever complains that this is somehow inconvenient (although I don't see why: directory-files is simple to use), then we could consider extending the API. But that's MO; please wait for Dmitry to chime in. ^ permalink raw reply [flat|nested] 16+ messages in thread
* bug#63870: 29.0.90; project.el can't dynamically populate the project list 2023-06-28 12:56 ` Eli Zaretskii @ 2023-07-18 2:21 ` Dmitry Gutov 2023-07-18 16:28 ` Spencer Baugh 0 siblings, 1 reply; 16+ messages in thread From: Dmitry Gutov @ 2023-07-18 2:21 UTC (permalink / raw) To: Eli Zaretskii, Spencer Baugh; +Cc: 63870 On 28/06/2023 15:56, Eli Zaretskii wrote: >>> Once again, this is dangerous; users could easily shoot themselves in >>> the foot, because not many are aware of the pitfall of using file >>> notifications for many directories. It makes no sense to warn against >>> something and at the same time let callers easily stumble upon that. >> I agree with that, I suppose. Personally I would be fine with a >> mandatory 1 or 2 levels of recursion, since I only need 2. Do you have >> a suggestion for what that interface could look like? It feels a bit >> awkward... > I'd actually begin by not providing even 1 level. Let the callers > call this new function explicitly for every directory which they want > watching. If someone ever complains that this is somehow inconvenient > (although I don't see why: directory-files is simple to use), then we > could consider extending the API. That sounds about right. But I might go a little further in this reasoning... (*) > But that's MO; please wait for Dmitry to chime in. [ Sorry for the late response, I'm still uncertain about this patch. ] (*) ... and ask whether this functionality makes sense built-in. I appreciate that it's succinct, documented and doesn't take a lot of space. But would we say that it covers a significantly general use case? Do we know many other developers who would appreciate it? Do a lot of devs at Jane Street use Emacs and this same workflow? Should we ask people somewhere (emacs-devel/Reddit/etc) whether they will find it useful? If it's just for one user at this point, then it shouldn't be difficult to maintain this code inside the init dir. Here's also some alternative I could potentially suggest: if you have some code which checks out new branches for development, or projects to start work on, and it's written in Elisp too, could it just call project-remember-project at the end? That would circumvent the need for using file watches altogether. Or if we do add this to project.el, we should try to make it safe for an average user even with a different directory structure. Suppose they have a dir D which they call project-watch on, and then they copy a big non-project directory inside. That should trigger many filenotify events, and since no search would result in success, I suppose the watch stays on, and every directory gets scanned up until the root. So an easy-to-enable recursive behavior seems dangerous for this case. Needless to say, the user could call this function, spend time on other stuff, forget, and then get surprised by things taking longer than expected. ^ permalink raw reply [flat|nested] 16+ messages in thread
* bug#63870: 29.0.90; project.el can't dynamically populate the project list 2023-07-18 2:21 ` Dmitry Gutov @ 2023-07-18 16:28 ` Spencer Baugh 2023-07-18 17:41 ` Juri Linkov 2023-07-27 1:57 ` Dmitry Gutov 0 siblings, 2 replies; 16+ messages in thread From: Spencer Baugh @ 2023-07-18 16:28 UTC (permalink / raw) To: Dmitry Gutov; +Cc: Eli Zaretskii, 63870 Dmitry Gutov <dmitry@gutov.dev> writes: > On 28/06/2023 15:56, Eli Zaretskii wrote: >>>> Once again, this is dangerous; users could easily shoot themselves in >>>> the foot, because not many are aware of the pitfall of using file >>>> notifications for many directories. It makes no sense to warn against >>>> something and at the same time let callers easily stumble upon that. >>> I agree with that, I suppose. Personally I would be fine with a >>> mandatory 1 or 2 levels of recursion, since I only need 2. Do you have >>> a suggestion for what that interface could look like? It feels a bit >>> awkward... >> I'd actually begin by not providing even 1 level. Let the callers >> call this new function explicitly for every directory which they want >> watching. If someone ever complains that this is somehow inconvenient >> (although I don't see why: directory-files is simple to use), then we >> could consider extending the API. > > That sounds about right. But I might go a little further in this > reasoning... (*) > >> But that's MO; please wait for Dmitry to chime in. > > [ Sorry for the late response, I'm still uncertain about this patch. ] > > (*) ... and ask whether this functionality makes sense built-in. > > I appreciate that it's succinct, documented and doesn't take a lot of > space. But would we say that it covers a significantly general use > case? Do we know many other developers who would appreciate it? Do a > lot of devs at Jane Street use Emacs and this same workflow? Should we > ask people somewhere (emacs-devel/Reddit/etc) whether they will find > it useful? It's something we'd use a lot, but I have no problem keeping it locally for now. I agree it has some dangers, maybe I'll come up with a good way to make it safe for the casual user. > If it's just for one user at this point, then it shouldn't be > difficult to maintain this code inside the init dir. > > Here's also some alternative I could potentially suggest: if you have > some code which checks out new branches for development, or projects > to start work on, and it's written in Elisp too, could it just call > project-remember-project at the end? That would circumvent the need > for using file watches altogether. That works great for stuff written in Elisp too, but alas, there are command line tools to check out branches for development, etc, and I want them to update Emacs too. > Or if we do add this to project.el, we should try to make it safe for > an average user even with a different directory structure. Suppose > they have a dir D which they call project-watch on, and then they copy > a big non-project directory inside. That should trigger many > filenotify events, and since no search would result in success, I > suppose the watch stays on, and every directory gets scanned up until > the root. So an easy-to-enable recursive behavior seems dangerous for > this case. > > Needless to say, the user could call this function, spend time on > other stuff, forget, and then get surprised by things taking longer > than expected. Definitely. I'll think about whether there's a way to avoid such footguns... (Maybe I'll revisit this after writing some scripts like this for Emacs development - although those will definitely only be in Elisp, so it would be kind of moot) ^ permalink raw reply [flat|nested] 16+ messages in thread
* bug#63870: 29.0.90; project.el can't dynamically populate the project list 2023-07-18 16:28 ` Spencer Baugh @ 2023-07-18 17:41 ` Juri Linkov 2023-07-27 1:59 ` Dmitry Gutov 2023-07-27 1:57 ` Dmitry Gutov 1 sibling, 1 reply; 16+ messages in thread From: Juri Linkov @ 2023-07-18 17:41 UTC (permalink / raw) To: Spencer Baugh; +Cc: Dmitry Gutov, Eli Zaretskii, 63870 > (Maybe I'll revisit this after writing some scripts like this for Emacs > development - although those will definitely only be in Elisp, so it > would be kind of moot) Why can't project--ensure-read-project-list scan a list of known roots to find a new project? I.e. the reverse of project-forget-projects-under. Or like there is an item "... (choose a dir)", another item could be "*Rescan*" that will try to find a new project under known roots. ^ permalink raw reply [flat|nested] 16+ messages in thread
* bug#63870: 29.0.90; project.el can't dynamically populate the project list 2023-07-18 17:41 ` Juri Linkov @ 2023-07-27 1:59 ` Dmitry Gutov 0 siblings, 0 replies; 16+ messages in thread From: Dmitry Gutov @ 2023-07-27 1:59 UTC (permalink / raw) To: Juri Linkov, Spencer Baugh; +Cc: Eli Zaretskii, 63870 On 18/07/2023 20:41, Juri Linkov wrote: >> (Maybe I'll revisit this after writing some scripts like this for Emacs >> development - although those will definitely only be in Elisp, so it >> would be kind of moot) > Why can't project--ensure-read-project-list scan a list of known roots > to find a new project? I.e. the reverse of project-forget-projects-under. > > Or like there is an item "... (choose a dir)", another item could be > "*Rescan*" that will try to find a new project under known roots. We have an interactive command called 'project-remember-projects-under', when you want to trigger such a scan. But as for why we don't do that automatically? Performance, for instance. Tree traversal, coupled with arbitrary logic for finding projects, can be too costly. ^ permalink raw reply [flat|nested] 16+ messages in thread
* bug#63870: 29.0.90; project.el can't dynamically populate the project list 2023-07-18 16:28 ` Spencer Baugh 2023-07-18 17:41 ` Juri Linkov @ 2023-07-27 1:57 ` Dmitry Gutov 1 sibling, 0 replies; 16+ messages in thread From: Dmitry Gutov @ 2023-07-27 1:57 UTC (permalink / raw) To: Spencer Baugh; +Cc: Eli Zaretskii, 63870 On 18/07/2023 19:28, Spencer Baugh wrote: >>> But that's MO; please wait for Dmitry to chime in. >> >> [ Sorry for the late response, I'm still uncertain about this patch. ] >> >> (*) ... and ask whether this functionality makes sense built-in. >> >> I appreciate that it's succinct, documented and doesn't take a lot of >> space. But would we say that it covers a significantly general use >> case? Do we know many other developers who would appreciate it? Do a >> lot of devs at Jane Street use Emacs and this same workflow? Should we >> ask people somewhere (emacs-devel/Reddit/etc) whether they will find >> it useful? > > It's something we'd use a lot, but I have no problem keeping it locally > for now. I agree it has some dangers, maybe I'll come up with a good > way to make it safe for the casual user. Thank you. >> If it's just for one user at this point, then it shouldn't be >> difficult to maintain this code inside the init dir. >> >> Here's also some alternative I could potentially suggest: if you have >> some code which checks out new branches for development, or projects >> to start work on, and it's written in Elisp too, could it just call >> project-remember-project at the end? That would circumvent the need >> for using file watches altogether. > > That works great for stuff written in Elisp too, but alas, there are > command line tools to check out branches for development, etc, and I > want them to update Emacs too. Just something to consider: you could invoke those shell scripts from Emacs Lisp, and then they finish, the Lisp program could do some additional steps, like adding projects to the list. Or removing -- in the reverse case. ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2023-07-27 1:59 UTC | newest] Thread overview: 16+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2023-06-03 11:55 bug#63870: 29.0.90; project.el can't dynamically populate the project list Spencer Baugh 2023-06-15 19:30 ` Spencer Baugh 2023-06-16 5:45 ` Eli Zaretskii 2023-06-17 2:55 ` Dmitry Gutov 2023-06-27 19:29 ` Spencer Baugh 2023-06-27 19:27 ` Spencer Baugh 2023-06-28 11:24 ` Eli Zaretskii 2023-06-28 12:05 ` Spencer Baugh 2023-06-28 12:18 ` Eli Zaretskii 2023-06-28 12:37 ` Spencer Baugh 2023-06-28 12:56 ` Eli Zaretskii 2023-07-18 2:21 ` Dmitry Gutov 2023-07-18 16:28 ` Spencer Baugh 2023-07-18 17:41 ` Juri Linkov 2023-07-27 1:59 ` Dmitry Gutov 2023-07-27 1:57 ` Dmitry Gutov
Code repositories for project(s) associated with this public inbox https://git.savannah.gnu.org/cgit/emacs.git This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).