Re: Subprojects in project.el (Was: Eglot, project.el, and python virtual environments)

["João Távora" <joaotavora@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 9222 bytes --]

Hi,

In the previous email, my pseudo-code was all botched, as usual when
writing imaginationware.  Better back up my ideas with real code.  The
patch after my sig allows these kinds of customizations:

  (defun joaot/sub-projects (dir)
    (let* ((super (project-current))
           (super-root (project-root super))
           (relative (file-relative-name dir super-root))
           (d (car (split-string relative "/"))))
      (when (member d '("lisp" "src"))
        (cons 'transient (expand-file-name (concat d "/") super-root)))))

  (add-hook 'project-find-functions
            'joaot/sub-projects
            t nil)

This will define two subprojects inside Emacs's tree.  The "lisp" and
"src" projects.  It's not particularly useful for Emacs, but I hope it
illustrates what I want to do.

Of course, I still haven't coded any commands to access the "super
project" so all commands work on the innermost sub-project, but they
do seem to work unproblematically.

To get the super-project, a seemingly robust way is:

  (project-current nil
    (file-name-directory
       (directory-file-name (project-root (project-current)))))

But I've yet to find where to plug it.  Maybe tomorrow I'll spend
some more time playing around with this.

João

This is the project.el patch:

diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index c7b2c386ccd..1903b7f622f 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -196,13 +196,13 @@ project-current
 See the doc string of `project-find-functions' for the general form
 of the project instance object."
   (unless directory (setq directory default-directory))
-  (let ((pr (project--find-in-directory directory)))
+  (let ((pr (car (project--find-in-directory directory))))
     (cond
      (pr)
      ((unless project-current-inhibit-prompt
         maybe-prompt)
       (setq directory (project-prompt-project-dir)
-            pr (project--find-in-directory directory))))
+            pr (car (project--find-in-directory directory)))))
     (when maybe-prompt
       (if pr
           (project-remember-project pr)
@@ -211,8 +211,22 @@ project-current
         (setq pr (cons 'transient directory))))
     pr))

+(defvar project--overriding-projects nil)
+
 (defun project--find-in-directory (dir)
-  (run-hook-with-args-until-success 'project-find-functions dir))
+  "Find projects that directory DIR belongs to.
+Return them as a list.  If more than one is found, the first is
+usually the innermost."
+  (or project--overriding-projects
+      (let (projects)
+        (run-hook-wrapped
+         'project-find-functions
+         (lambda (fn)
+           (let* ((project--overriding-projects projects)
+                  (prj (funcall fn dir)))
+             (when prj (push prj projects)))
+           nil))
+        projects)))

 (defvar project--within-roots-fallback nil)

On Wed, Nov 23, 2022 at 1:56 PM João Távora <joaotavora@gmail.com> wrote:

> Dmitry Gutov <dgutov@yandex.ru> writes:
>
> >> I've just described in the other thread that I would like to have
> >> finding references and finding files to be able to operate on either
> >> sub-projects or super-projects on demand.  This is the problem I'm
> >> facing, and it's not new.  For example, there is only: find file in the
> >> very large project, and find file in the current directory.  There is no
> >> "find file in this section of the repo, which is a sub-project in
> >> itself".
> >
> > It would be much more helpful in a dedicated bug report where we could
> > discuss the details, collect the votes and see what kind of design
> > will ultimately satisfy the requirements. Instead of drowning it all
> > in this thread, which is only moderately related.
>
> I think we're doing fine here but I've changed the subject line to
> "unbury" it from the thread.
>
> >> IMO, it's not "improper" to describe problems and use cases: in fact I
> >> prefer that people describe over jumping to vapourware solutions.  But
> >> if you're really looking for a suggestion as to _how_ to design it, I
> >> suppose my problem would be well dealt with a negative prefix on C-x p g
> >> and C-x p f giving me a choice of which project to operate on.  But that
> >> is only one possibility: new commands are also acceptable.
> >
> > Note that you can more-or-less do this now: press 'C-x p p', select
> > the parent project from the list, then choose 'f' or 'g' to run the
> > command there. So to justify the added complexity one should say that
> > they do want to use this feature frequently enough to justify the
> > added complexity (which will reduce the number of keystrokes).
>
> Both cases -- focus on super-project/focus on sub-project -- are
> definitely common.  I have no problem in using an existing interface,
> even if it idiosyncratic.
>
> So if the inner-most sub-project is the default but somehow I can
> explicitly call up a "project picker" that shows me the super-project as
> one of the first options, that probably works fine.
>
> > And even if we do, we might not need the additional notion of
> > sub-projects. E.g. 'search in the parent project (if any)' might work
> > as "take the root, go up a directory, search for a project there; if
> > it exists, use it". Though Stephen L. might want a generic for that,
> > since his projects do not correlate with directory tree.
>
> And in my case, sub-projects don't necessarily exist exactly one-level
> down super-project's root.  Sometimes they do, but now always.
>
> Of course, just as data point, I've been solving all my problems outside
> project.el for a good long while now, even before I was faced with this
> gigantic monorepo and even before project.el was a thing.  I use Leo
> Liu's ack-el.el, which greps from the super-project's root by default,
> and from wherever I want given enough C-u.  And I have a similar
> separate command for finding files from arbitrary points in the
> hierarchy.  And I have a similar command for compiling.  I just happen
> to think having this in project.el would make a lot of sense.  I've
> recently experimented with it and found the new project.el features
> pleasant to work in smaller or well-behaved projects.  It's not powerful
> enough for the big stuff, but it ought to be.
>
> > OTOH, if one of the operations will require a step "get all
> > subprojects of the current project", then a separate notion might be
> > required, with a separate hook.
> >
> >> As to how one defines sub-projects, I think having
> >> project-find-functions be used to compose a list of projects (as opposed
> >> to to stopping after finding one) would be a nice way.  It would be nice
> >> if the elements of project-find-functions could be informed of the
> >> projects found so far.  But keeping the "stop after first" behaviour and
> >> having members return a more complex object is maybe also good.  I
> >> really don't care much what you eventually choose, as long as I can
> >> solve my problem.
> >
> > If we wanted to define projects and subprojects through the same hook,
> > the step "get all subprojects of the current project" might not be
> > feasible to implement. Because of performance-related considerations.
>
> Those remain to be seen, I can't guess what things you're imagining.
>
> I'm not sure we need that step at all.  After all there is no "get me
> all the projects in my hard drive" either, and at least I haven't missed
> it.
>
> Anyway, from my POV this type of sub-project definition would not be
> very nice.
>
>   (add-hook 'some-new-subproject-find-hook
>             (lambda (dir)
>               (cons 'super-project
>
> (some-code-which-scans-and-returns-a-list-of-subprojects))))
>
> I would prefer this, which I think is more readable and in-line with
> current project.el:
>
>   (add-hook 'project-find-functions
>             (lambda (dir)
>               (let ((super (project-current)))
>                 (when (and super
>                            (string= (file-name-directory
>                                      (directory-file-name dir))
>                                     (project-root super))
>                            (my-special-condition dir))
>                   (cons 'transient dir)))))
>
>
> Maybe 'my-special-condition' could read a directory local variable to
> decide if sub-projects are enabled or not.
>
> For my idea to work, project-current would have to be changed to
>
> 1. Call all members of project-find-functions: return the innermost
>    project found thus far.
> 2. Not call project-find-functions recursively when called from within a
> function
>    in project-find-functions, just return the innermost project found
>    thus far.
>
> I don't see any big performance problems in this alternative.
>
> But, again, the above are suggestions: _any_ alternative that lets me
> define inner sub-projects within a larger super-project and let me
> choose where I want to grep, find-file, compile, would be an
> improvement.
>
> João
>


-- 
João Távora

[-- Attachment #2: Type: text/html, Size: 11012 bytes --]