bug#63896: [PATCH] Support annotating and sorting the project list during completion

bug#63896: [PATCH] Support annotating and sorting the project list during completion

[Spencer Baugh]

[-- Attachment #1: Type: text/plain, Size: 2289 bytes --]

Tags: patch


This patch adds an annotation-function and display-sort-function to the
completion-table used for project-prompt-project-dir and
project-prompt-project-name, as well as a user customization variable to
customize the behavior of the annotation and sorting functions.

The idea is that projects are annotated with useful information for
deciding which one you want to switch to, and sorted based on how likely
you are to want to switch to or work on them.  For example, a user might
want to know how many buffers they have open in each project, to help
tell them apart at a glance.  Furthermore, a user might be more likely
to switch to projects they've already been working on in this Emacs
instance, so they might want projects with more buffers open to be
sorted before projects with no buffers.

All this is customized by the variable project-annotations which is a
list of functions used to generate the annotations and sorting metadata.
The user can add their own functions to add new annotations and sorting
behavior.  See its docstring for more details.

I added three annotation functions as a starting point, which when added
to project-annotations will annotate with the number of buffers, the
modification time of the root directory, and compilation results.

In this patch I have turned all three of these annotations on by
default, by putting all three in project-annotations, but probably when
this is actually pushed we want project-annotations to be empty by
default.  (Maybe?)

In my own packages, building on this, I hope to add annotation functions
like "number of bugs assigned to you in this project" and "number of
lines of incoming code to review in this project", so that
project-switch-project is a nice way to pick what to work on next.

This patch is still a bit rough around the edges, but I'm posting it now
to get feedback.

In GNU Emacs 29.0.90 (build 6, x86_64-pc-linux-gnu, GTK+ Version
 3.22.30, cairo version 1.15.12) of 2023-06-02 built on igm-qws-u22796a
Repository revision: ff6163fac51759945aa619ca6bf28413be4a53e0
Repository branch: emacs-29
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: CentOS Linux 7 (Core)

Configured using:
 'configure --with-x-toolkit=gtk --with-gif=ifavailable'


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Support-annotating-and-sorting-the-project-list-duri.patch --]
[-- Type: text/patch, Size: 7658 bytes --]

From 11d76029db5f0d9e016f247aac24dd430b729c2a Mon Sep 17 00:00:00 2001
From: Spencer Baugh <sbaugh@janestreet.com>
Date: Sat, 3 Jun 2023 13:21:30 -0400
Subject: [PATCH] Support annotating and sorting the project list during
 completion

---
 lisp/progmodes/project.el | 124 +++++++++++++++++++++++++++++++++++---
 1 file changed, 117 insertions(+), 7 deletions(-)

diff --git a/lisp/progmodes/project.el b/lisp/progmodes/project.el
index 04c67710d71..01ce414221f 100644
--- a/lisp/progmodes/project.el
+++ b/lisp/progmodes/project.el
@@ -317,6 +317,117 @@ project--file-completion-table
      (t
       (complete-with-action action all-files string pred)))))
 
+(defun project-annotation-numbufs (pr)
+  "Annotate PROJECT with the length of `project-buffers'."
+  (let ((numbufs (length (project-buffers pr))))
+    (cons numbufs
+          (if (zerop numbufs)
+              ""
+            (format "%s buf" numbufs)))))
+
+(defun project-annotation-mtime (pr)
+  "Annotate PROJECT with the modification time of its root directory.
+
+Note that the modification time will only change when files
+directly under the root directory are added or deleted.  If you
+only add or delete files in subdirectories, or if you only modify
+existing files, the modification time won't change."
+  (let* ((mtime (file-attribute-modification-time (file-attributes (project-root pr))))
+         (since-change (float-time (time-subtract (current-time) mtime))))
+    (cons (- since-change)
+          (cl-dolist (format '("%x%Y" "%x%D" "%x%H" "%x%M" "%x%S") "")
+            (let ((result (format-seconds format since-change)))
+              (when (not (string-empty-p result))
+                (return (concat result " old"))))))))
+
+(defun project-annotation-compilation (project)
+  "Annotate PROJECT with information from its compilation buffer if any."
+  (let* ((default-directory (project-root project))
+         (name (funcall
+                (or project-compilation-buffer-name-function
+                    compilation-buffer-name-function) "compilation"))
+         (buf (get-buffer name)))
+    (if buf
+        ;; TODO we should include in the sorting number whether the
+        ;; compilation exited non-zero; I don't see where, if
+        ;; anywhere, that's stored, though...
+        (with-current-buffer buf
+          (cons (+ (* 100 compilation-num-errors-found)
+                   (* 10 compilation-num-warnings-found)
+                   compilation-num-infos-found)
+                (format-mode-line mode-line-process nil nil buf)))
+      ;; projects with no errors are less interesting;
+      ;; sort them below projects that haven't been compiled at all
+      '(1 . ""))))
+
+(defcustom project-annotations (list
+                                #'project-annotation-compilation
+                                #'project-annotation-numbufs
+                                #'project-annotation-mtime
+                                )
+  "Functions to call to add annotations when prompting for a project.
+
+While prompting for a project in `project-current' or
+`project-switch-project', these functions are called to annotate
+each completion alternative with information about the project,
+and provide metadata to sort the projects by relevance.  By
+customizing this variable, you can make arbitrary information
+available during project completion, as long as it's fast enough
+to compute that it doesn't slow down completion.
+
+The order of functions in this list determines the order in which
+annotations are used, which determines both their precedence for
+sorting and the order in which they appear as an annotation after
+the completion alternative.
+
+Each function is called with a single argument, a project
+instance.  It should return a cons cell, whose car should be
+numeric and is used to sort the projects, greater values first,
+and whose cdr should be a string to be included as an annotation
+on the project during completion."
+  :type '(repeat (const :tag "Number of buffers"
+                        project-annotation-compilation)
+                 (const :tag "Modification time of root dir"
+                        project-annotation-mtime)
+                 (const :tag "Compilation results"
+                        project-annotation-compilation)
+                 (function :tag "Custom function" nil)))
+
+(defun project--project-completion-table (collection projects)
+  "Completion table for project identifiers in COLLECTION
+
+PROJECTS should be an alist mapping completions from COLLECTION
+to project instances.  Completions which are not in PROJECTS are
+not annotated with `project-annotations'."
+  (let* (annots
+         (get-annot (lambda (completion pr)
+                (or (cdr (assoc completion annots))
+                    (let ((annot (mapcar (lambda (func) (funcall func pr)) project-annotations)))
+                      (push (cons completion annot) annots)
+                      annot))))
+         (annotation-function
+          (lambda (completion)
+            (if-let (pr (cdr (assoc completion projects)))
+                (let ((annotations (mapcar #'cdr (funcall get-annot completion pr))))
+                  (concat " " (string-join (seq-remove #'string-empty-p annotations) ", ")))
+              "")))
+         (display-sort-function
+          (lambda (completions)
+            (let ((with-nums (mapcar (lambda (completion)
+                                       (cons (when-let (pr (cdr (assoc completion projects)))
+                                               (mapcar #'car (funcall get-annot completion pr)))
+                                             completion))
+                                     completions)))
+              (mapcar #'cdr (sort with-nums (lambda (a b) (version-list-< (car b) (car a)))))))))
+    (lambda (string pred action)
+      (cond
+       ((eq action 'metadata)
+        `(metadata . ((category . project-file)
+                      (annotation-function . ,annotation-function)
+                      (display-sort-function . ,display-sort-function))))
+       (t
+        (complete-with-action action collection string pred))))))
+
 (cl-defmethod project-root ((project (head transient)))
   (cdr project))
 
@@ -1640,11 +1751,12 @@ project-prompt-project-dir
 It's also possible to enter an arbitrary directory not in the list."
   (project--ensure-read-project-list)
   (let* ((dir-choice "... (choose a dir)")
+         (projects
+          (mapcar (lambda (dir)
+                    (cons (car dir) (project--find-in-directory (car dir))))
+                  project--list))
          (choices
-          ;; XXX: Just using this for the category (for the substring
-          ;; completion style).
-          (project--file-completion-table
-           (append project--list `(,dir-choice))))
+          (project--project-completion-table (cons dir-choice projects) projects))
          (pr-dir ""))
     (while (equal pr-dir "")
       ;; If the user simply pressed RET, do this again until they don't.
@@ -1667,9 +1779,7 @@ project-prompt-project-name
               (if-let (proj (project--find-in-directory dir))
                   (push (cons (project-name proj) proj) ret)))
             ret))
-         ;; XXX: Just using this for the category (for the substring
-         ;; completion style).
-         (table (project--file-completion-table (cons dir-choice choices)))
+         (table (project--project-completion-table (cons dir-choice choices) choices))
          (pr-name ""))
     (while (equal pr-name "")
       ;; If the user simply pressed RET, do this again until they don't.
-- 
2.39.3

bug#63896: [PATCH] Support annotating and sorting the project list during completion

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Date: Sun, 04 Jun 2023 17:20:19 -0400
> 
> This patch adds an annotation-function and display-sort-function to the
> completion-table used for project-prompt-project-dir and
> project-prompt-project-name, as well as a user customization variable to
> customize the behavior of the annotation and sorting functions.

Thanks.  A few minor comments.

> +(defun project-annotation-numbufs (pr)
> +  "Annotate PROJECT with the length of `project-buffers'."

The argument is named PR, not PROJECT.

Also, instead if too-technical "length of `project-buffers'", how
about saying "number of buffers in the project", or "number of buffers
in `project-buffers'"?

> +(defun project-annotation-mtime (pr)
> +  "Annotate PROJECT with the modification time of its root directory.

PR, not PROJECT.

> +Note that the modification time will only change when files
> +directly under the root directory are added or deleted.  If you
> +only add or delete files in subdirectories, or if you only modify
> +existing files, the modification time won't change."

This is not guaranteed, so I'd suggest to mention that the reliability
of this annotation is questionable, and depends on the underlying
filesystem.

> +(defun project-annotation-compilation (project)
> +  "Annotate PROJECT with information from its compilation buffer if any."

I'd add here something about that "information".  Right now, this doc
string is not very useful.

> +          (cons (+ (* 100 compilation-num-errors-found)
> +                   (* 10 compilation-num-warnings-found)

Why "encode" these numbers in a single value? why not use a cons or a
vector?

> +                (format-mode-line mode-line-process nil nil buf)))

Do you really need to call format-mode-line?  My advice is to stay
away of that function: it could have unpleasant side effects.

bug#63896: [PATCH] Support annotating and sorting the project list during completion

[Spencer Baugh]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Spencer Baugh <sbaugh@janestreet.com>
>> Date: Sun, 04 Jun 2023 17:20:19 -0400
>> 
>> This patch adds an annotation-function and display-sort-function to the
>> completion-table used for project-prompt-project-dir and
>> project-prompt-project-name, as well as a user customization variable to
>> customize the behavior of the annotation and sorting functions.
>
> Thanks.  A few minor comments.

Thanks, fixed.

>> +          (cons (+ (* 100 compilation-num-errors-found)
>> +                   (* 10 compilation-num-warnings-found)
>
> Why "encode" these numbers in a single value? why not use a cons or a
> vector?

I'd be happy to use a cons or a vector, or even a more complicated
structure, but I didn't see an easy way to do comparison of
complicated structures, for the sorting of projects based on their
annotation.  For example, if I have values of the form
(num . (num num num))

there's no way to know what sorting predicate to use for such values - I
need to be able to know which value should sort sort first, when I have
a pair of them.

I could make project-annotations a list of tuples of functions, the
first being the annotation-generating function, the second being the
annotation-comparison function.  Then I wouldn't need to encode the
numbers into a single value.  Does that seem like a reasonable design?
I thought that was a bit overly complex.

>> +                (format-mode-line mode-line-process nil nil buf)))
>
> Do you really need to call format-mode-line?  My advice is to stay
> away of that function: it could have unpleasant side effects.

Annoyingly if I want to include the exit code of the compilation in the
annotation, the only place it's found is as a string in
mode-line-process.  I could extract that string from mode-line-process
and use it, but I thought it would be a bad idea to depend on the exact
structure of what compile.el puts in mode-line-process.  So I just
format-mode-line'd it.

Would it be OK to make compile.el store the exit code as a number in a
variable and then use that?  Then I wouldn't need to touch
mode-line-process at all.

bug#63896: [PATCH] Support annotating and sorting the project list during completion

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Cc: 63896@debbugs.gnu.org
> Date: Tue, 13 Jun 2023 17:19:52 -0400
> 
> >> +          (cons (+ (* 100 compilation-num-errors-found)
> >> +                   (* 10 compilation-num-warnings-found)
> >
> > Why "encode" these numbers in a single value? why not use a cons or a
> > vector?
> 
> I'd be happy to use a cons or a vector, or even a more complicated
> structure, but I didn't see an easy way to do comparison of
> complicated structures, for the sorting of projects based on their
> annotation.  For example, if I have values of the form
> (num . (num num num))

You'd need to write a custom comparison function, but why is that a
problem?

> there's no way to know what sorting predicate to use for such values - I
> need to be able to know which value should sort sort first, when I have
> a pair of them.

But the encoding scheme above provides the answer: you want errors to
sort before the warnings.  So it sounds like you already decided how
to sort those, no?

> >> +                (format-mode-line mode-line-process nil nil buf)))
> >
> > Do you really need to call format-mode-line?  My advice is to stay
> > away of that function: it could have unpleasant side effects.
> 
> Annoyingly if I want to include the exit code of the compilation in the
> annotation, the only place it's found is as a string in
> mode-line-process.  I could extract that string from mode-line-process
> and use it, but I thought it would be a bad idea to depend on the exact
> structure of what compile.el puts in mode-line-process.  So I just
> format-mode-line'd it.
> 
> Would it be OK to make compile.el store the exit code as a number in a
> variable and then use that?  Then I wouldn't need to touch
> mode-line-process at all.

I don't see why you'd need that.  Doesn't process-exit-status give you
that value?  mode-line-process is not some magic, it just accesses
process information exposed via the different primitives.

bug#63896: [PATCH] Support annotating and sorting the project list during completion

[Spencer Baugh]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Spencer Baugh <sbaugh@janestreet.com>
>> Cc: 63896@debbugs.gnu.org
>> Date: Tue, 13 Jun 2023 17:19:52 -0400
>> 
>> >> +          (cons (+ (* 100 compilation-num-errors-found)
>> >> +                   (* 10 compilation-num-warnings-found)
>> >
>> > Why "encode" these numbers in a single value? why not use a cons or a
>> > vector?
>> 
>> I'd be happy to use a cons or a vector, or even a more complicated
>> structure, but I didn't see an easy way to do comparison of
>> complicated structures, for the sorting of projects based on their
>> annotation.  For example, if I have values of the form
>> (num . (num num num))
>
> You'd need to write a custom comparison function, but why is that a
> problem?

Yes, but how does that get configured?

>> there's no way to know what sorting predicate to use for such values - I
>> need to be able to know which value should sort sort first, when I have
>> a pair of them.
>
> But the encoding scheme above provides the answer: you want errors to
> sort before the warnings.  So it sounds like you already decided how
> to sort those, no?

Yes, but I mean that *this function* doesn't know, given some opaque
value returned by a user-provided annotation function, how to sort.

>> >> +                (format-mode-line mode-line-process nil nil buf)))
>> >
>> > Do you really need to call format-mode-line?  My advice is to stay
>> > away of that function: it could have unpleasant side effects.
>> 
>> Annoyingly if I want to include the exit code of the compilation in the
>> annotation, the only place it's found is as a string in
>> mode-line-process.  I could extract that string from mode-line-process
>> and use it, but I thought it would be a bad idea to depend on the exact
>> structure of what compile.el puts in mode-line-process.  So I just
>> format-mode-line'd it.
>> 
>> Would it be OK to make compile.el store the exit code as a number in a
>> variable and then use that?  Then I wouldn't need to touch
>> mode-line-process at all.
>
> I don't see why you'd need that.  Doesn't process-exit-status give you
> that value?  mode-line-process is not some magic, it just accesses
> process information exposed via the different primitives.

For sure, process-exit-status gives me that value.  But how do I get the
process to call it on?  The process is dead at this point, so
(get-buffer-process "*compilation*") returns nil.  Is there a way to get
the process associated with the buffer even though it's killed?

bug#63896: [PATCH] Support annotating and sorting the project list during completion

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Cc: 63896@debbugs.gnu.org
> Date: Thu, 15 Jun 2023 15:04:20 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> I'd be happy to use a cons or a vector, or even a more complicated
> >> structure, but I didn't see an easy way to do comparison of
> >> complicated structures, for the sorting of projects based on their
> >> annotation.  For example, if I have values of the form
> >> (num . (num num num))
> >
> > You'd need to write a custom comparison function, but why is that a
> > problem?
> 
> Yes, but how does that get configured?
> 
> >> there's no way to know what sorting predicate to use for such values - I
> >> need to be able to know which value should sort sort first, when I have
> >> a pair of them.
> >
> > But the encoding scheme above provides the answer: you want errors to
> > sort before the warnings.  So it sounds like you already decided how
> > to sort those, no?
> 
> Yes, but I mean that *this function* doesn't know, given some opaque
> value returned by a user-provided annotation function, how to sort.

You'd need to include the comparison function in the annotation data,
I guess.

> >> Would it be OK to make compile.el store the exit code as a number in a
> >> variable and then use that?  Then I wouldn't need to touch
> >> mode-line-process at all.
> >
> > I don't see why you'd need that.  Doesn't process-exit-status give you
> > that value?  mode-line-process is not some magic, it just accesses
> > process information exposed via the different primitives.
> 
> For sure, process-exit-status gives me that value.  But how do I get the
> process to call it on?  The process is dead at this point, so
> (get-buffer-process "*compilation*") returns nil.  Is there a way to get
> the process associated with the buffer even though it's killed?

If project.el wants to access data from an exited compilation, it
needs to record that when the compilation exits (via the
compilation-finish-functions hook, for example).  Calling
format-mode-line will not help you, because if the process doesn't
exist, its data cannot be accessed, and relying on what's displayed on
the mode line is a bad idea: it could be outdated or even irrelevant.
So please don't use such kludges, even though they might look
convenient at first sight.

bug#63896: [PATCH] Support annotating and sorting the project list during completion

[Spencer Baugh]

Eli Zaretskii <eliz@gnu.org> writes:
>> From: Spencer Baugh <sbaugh@janestreet.com>
>> Cc: 63896@debbugs.gnu.org
>> Date: Thu, 15 Jun 2023 15:04:20 -0400
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> I'd be happy to use a cons or a vector, or even a more complicated
>> >> structure, but I didn't see an easy way to do comparison of
>> >> complicated structures, for the sorting of projects based on their
>> >> annotation.  For example, if I have values of the form
>> >> (num . (num num num))
>> >
>> > You'd need to write a custom comparison function, but why is that a
>> > problem?
>> 
>> Yes, but how does that get configured?
>> 
>> >> there's no way to know what sorting predicate to use for such values - I
>> >> need to be able to know which value should sort sort first, when I have
>> >> a pair of them.
>> >
>> > But the encoding scheme above provides the answer: you want errors to
>> > sort before the warnings.  So it sounds like you already decided how
>> > to sort those, no?
>> 
>> Yes, but I mean that *this function* doesn't know, given some opaque
>> value returned by a user-provided annotation function, how to sort.
>
> You'd need to include the comparison function in the annotation data,
> I guess.

Yes, I'm just not sure the right approach.  It would be wasteful to
return the comparison function from the annotation function every time
it's called, since it's the same for every call.

>> >> Would it be OK to make compile.el store the exit code as a number in a
>> >> variable and then use that?  Then I wouldn't need to touch
>> >> mode-line-process at all.
>> >
>> > I don't see why you'd need that.  Doesn't process-exit-status give you
>> > that value?  mode-line-process is not some magic, it just accesses
>> > process information exposed via the different primitives.
>> 
>> For sure, process-exit-status gives me that value.  But how do I get the
>> process to call it on?  The process is dead at this point, so
>> (get-buffer-process "*compilation*") returns nil.  Is there a way to get
>> the process associated with the buffer even though it's killed?
>
> If project.el wants to access data from an exited compilation, it
> needs to record that when the compilation exits (via the
> compilation-finish-functions hook, for example).  Calling
> format-mode-line will not help you, because if the process doesn't
> exist, its data cannot be accessed, and relying on what's displayed on
> the mode line is a bad idea: it could be outdated or even irrelevant.
> So please don't use such kludges, even though they might look
> convenient at first sight.

Would it be OK for compile.el to start storing this data in a variable?
The number of errors/warnings/infos is already stored; also storing the
exit status would probably be useful for all kinds of things.

bug#63896: [PATCH] Support annotating and sorting the project list during completion

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Cc: 63896@debbugs.gnu.org
> Date: Fri, 16 Jun 2023 10:26:05 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> >
> > If project.el wants to access data from an exited compilation, it
> > needs to record that when the compilation exits (via the
> > compilation-finish-functions hook, for example).  Calling
> > format-mode-line will not help you, because if the process doesn't
> > exist, its data cannot be accessed, and relying on what's displayed on
> > the mode line is a bad idea: it could be outdated or even irrelevant.
> > So please don't use such kludges, even though they might look
> > convenient at first sight.
> 
> Would it be OK for compile.el to start storing this data in a variable?
> The number of errors/warnings/infos is already stored; also storing the
> exit status would probably be useful for all kinds of things.

I think it would be okay to store data about the process in a
buffer-local variable, but we need to make sure we store everything
that might be useful.

bug#63896: [PATCH] Support annotating and sorting the project list during completion

[Spencer Baugh]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Spencer Baugh <sbaugh@janestreet.com>
>> Cc: 63896@debbugs.gnu.org
>> Date: Fri, 16 Jun 2023 10:26:05 -0400
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> >
>> > If project.el wants to access data from an exited compilation, it
>> > needs to record that when the compilation exits (via the
>> > compilation-finish-functions hook, for example).  Calling
>> > format-mode-line will not help you, because if the process doesn't
>> > exist, its data cannot be accessed, and relying on what's displayed on
>> > the mode line is a bad idea: it could be outdated or even irrelevant.
>> > So please don't use such kludges, even though they might look
>> > convenient at first sight.
>> 
>> Would it be OK for compile.el to start storing this data in a variable?
>> The number of errors/warnings/infos is already stored; also storing the
>> exit status would probably be useful for all kinds of things.
>
> I think it would be okay to store data about the process in a
> buffer-local variable, but we need to make sure we store everything
> that might be useful.

Sure, how about this?

diff --git a/lisp/progmodes/compile.el b/lisp/progmodes/compile.el
index ccf64fb670b..b1f6d146d01 100644
--- a/lisp/progmodes/compile.el
+++ b/lisp/progmodes/compile.el
@@ -171,6 +171,10 @@ compilation-error
 (defvar compilation-arguments nil
   "Arguments that were given to `compilation-start'.")
 
+(defvar compilation-exit-status nil
+  "The `process-exit-status' of the compilation process, or nil if not exited yet.")
+(defvar compilation-process-status nil
+  "The latest `process-status' of the compilation process.")
 (defvar compilation-num-errors-found 0)
 (defvar compilation-num-warnings-found 0)
 (defvar compilation-num-infos-found 0)
@@ -2457,6 +2461,7 @@ compilation-minor-mode
 
 (defun compilation-handle-exit (process-status exit-status msg)
   "Write MSG in the current buffer and hack its `mode-line-process'."
+  (setq-local compilation-exit-status exit-status)
   (let ((inhibit-read-only t)
 	(status (if compilation-exit-message-function
 		    (funcall compilation-exit-message-function
@@ -2500,6 +2505,7 @@ compilation-handle-exit
 ;; Called when compilation process changes state.
 (defun compilation-sentinel (proc msg)
   "Sentinel for compilation buffers."
+  (setq-local compilation-process-status (process-status proc))
   (if (memq (process-status proc) '(exit signal))
       (unwind-protect
           (let ((buffer (process-buffer proc)))

bug#63896: [PATCH] Support annotating and sorting the project list during completion

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Cc: 63896@debbugs.gnu.org
> Date: Tue, 27 Jun 2023 16:30:38 -0400
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> +(defvar compilation-exit-status nil
> +  "The `process-exit-status' of the compilation process, or nil if not exited yet.")

This should clarify that this is the exit status of the _last_
compilation in this buffer.  It should also say that the variable is
always buffer-local.  I'd also add a reference to
compilation-process-status, because the value of nil is not
informative enough unless one looks at that other variable.

> +(defvar compilation-process-status nil
> +  "The latest `process-status' of the compilation process.")

Likewise here, about the buffer-local part.

I think these variables warrant to be called out in NEWS.

bug#63896: [PATCH] Support annotating and sorting the project list during completion

[Dmitry Gutov]

On 05/06/2023 14:23, Eli Zaretskii wrote:
>> +                (format-mode-line mode-line-process nil nil buf)))
> Do you really need to call format-mode-line?  My advice is to stay
> away of that function: it could have unpleasant side effects.

Does calling format-mode-line trigger extra redisplays or something? 
That sounds unexpected.

OTOH, relying on the format of the value of mode-line-process is iffy 
(submodes could change it; though I haven't really seen it done so far).

bug#63896: [PATCH] Support annotating and sorting the project list during completion

[Eli Zaretskii]

> Date: Thu, 24 Aug 2023 04:54:44 +0300
> Cc: 63896@debbugs.gnu.org
> From: Dmitry Gutov <dmitry@gutov.dev>
> 
> On 05/06/2023 14:23, Eli Zaretskii wrote:
> >> +                (format-mode-line mode-line-process nil nil buf)))
> > Do you really need to call format-mode-line?  My advice is to stay
> > away of that function: it could have unpleasant side effects.
> 
> Does calling format-mode-line trigger extra redisplays or something? 
> That sounds unexpected.

It reuses parts of the redisplay's code, yes.  Depending what's in the
format variable passed to format-mode-line, that could cause a variety
of surprises, especially if :eval forms are involved.  It could also
temporarily select another window and/or buffer, if the respective
arguments are not omitted, which could also be unexpected.

IMO, this function should be used sparing and with caution.

bug#63896: [PATCH] Support annotating and sorting the project list during completion

[Dmitry Gutov]

On 24/08/2023 08:29, Eli Zaretskii wrote:
>> Date: Thu, 24 Aug 2023 04:54:44 +0300
>> Cc:63896@debbugs.gnu.org
>> From: Dmitry Gutov<dmitry@gutov.dev>
>>
>> On 05/06/2023 14:23, Eli Zaretskii wrote:
>>>> +                (format-mode-line mode-line-process nil nil buf)))
>>> Do you really need to call format-mode-line?  My advice is to stay
>>> away of that function: it could have unpleasant side effects.
>> Does calling format-mode-line trigger extra redisplays or something?
>> That sounds unexpected.
> It reuses parts of the redisplay's code, yes.  Depending what's in the
> format variable passed to format-mode-line, that could cause a variety
> of surprises, especially if :eval forms are involved.  It could also
> temporarily select another window and/or buffer, if the respective
> arguments are not omitted, which could also be unexpected.
> 
> IMO, this function should be used sparing and with caution.

All right, so I guess it's the same concern: when mode-line-process 
contains something unusual, possibly assigned by a derived mode.

bug#63896: [PATCH] Support annotating and sorting the project list during completion

[Eli Zaretskii]

> Date: Thu, 24 Aug 2023 16:08:04 +0300
> Cc: sbaugh@janestreet.com, 63896@debbugs.gnu.org
> From: Dmitry Gutov <dmitry@gutov.dev>
> 
> On 24/08/2023 08:29, Eli Zaretskii wrote:
> >> Date: Thu, 24 Aug 2023 04:54:44 +0300
> >> Cc:63896@debbugs.gnu.org
> >> From: Dmitry Gutov<dmitry@gutov.dev>
> >>
> >> On 05/06/2023 14:23, Eli Zaretskii wrote:
> >>>> +                (format-mode-line mode-line-process nil nil buf)))
> >>> Do you really need to call format-mode-line?  My advice is to stay
> >>> away of that function: it could have unpleasant side effects.
> >> Does calling format-mode-line trigger extra redisplays or something?
> >> That sounds unexpected.
> > It reuses parts of the redisplay's code, yes.  Depending what's in the
> > format variable passed to format-mode-line, that could cause a variety
> > of surprises, especially if :eval forms are involved.  It could also
> > temporarily select another window and/or buffer, if the respective
> > arguments are not omitted, which could also be unexpected.
> > 
> > IMO, this function should be used sparing and with caution.
> 
> All right, so I guess it's the same concern: when mode-line-process 
> contains something unusual, possibly assigned by a derived mode.

Yes.

bug#63896: [PATCH] Support annotating and sorting the project list during completion

[Dmitry Gutov]

Hi Spencer,

Sorry for replying late on this one. It looks like an interesting 
feature, going a bit further in the IDE department ("denser" UI).

I've tried the patch a little, here are some notes.

On 05/06/2023 00:20, Spencer Baugh wrote:
> Tags: patch
> 
> 
> This patch adds an annotation-function and display-sort-function to the
> completion-table used for project-prompt-project-dir and
> project-prompt-project-name, as well as a user customization variable to
> customize the behavior of the annotation and sorting functions.
> The idea is that projects are annotated with useful information for
> deciding which one you want to switch to, and sorted based on how likely
> you are to want to switch to or work on them.  For example, a user might
> want to know how many buffers they have open in each project, to help
> tell them apart at a glance.  Furthermore, a user might be more likely
> to switch to projects they've already been working on in this Emacs
> instance, so they might want projects with more buffers open to be
> sorted before projects with no buffers.

Or they might still like the MRU order, or some combination of that 
together with the number of open issues or compilation errors.

> All this is customized by the variable project-annotations which is a
> list of functions used to generate the annotations and sorting metadata.
> The user can add their own functions to add new annotations and sorting
> behavior.  See its docstring for more details.

What do you think about splitting it into two vars?

One for annotations (just text) and another for "sorters" (or 
something). Unless the performance difference is noticeable, decoupling 
the two will lead to better customizability: the fact that I want to see 
the numbers of errors doesn't necessarily mean I want them to be used 
for sorting. Or vice versa: I could move that number closer to the end 
in display, but prefer to have it weigh more prominently when sorting.

> I added three annotation functions as a starting point, which when added
> to project-annotations will annotate with the number of buffers, the
> modification time of the root directory, and compilation results.

Regarding the mtime one, we could also look at mtime of .git or .hg 
subdirectory when present, and take the max.

About the "compilation" one -- and this is nothing urgent -- I launch my 
compilation submode buffers using rspec-mode, not through 
project-compile. Might be useful to have that supported ootb without 
having to tweak the corresponding function. Just a search for 
compiletion-mode derived buffers belonging to the given project could work.

> In this patch I have turned all three of these annotations on by
> default, by putting all three in project-annotations, but probably when
> this is actually pushed we want project-annotations to be empty by
> default.  (Maybe?)

Probably. Up for later discussion, I guess.

We could also run an experiment.

> In my own packages, building on this, I hope to add annotation functions
> like "number of bugs assigned to you in this project" and "number of
> lines of incoming code to review in this project", so that
> project-switch-project is a nice way to pick what to work on next.

Very cool. Nothing that we could include ootb, but if you have a system 
which fetches the info over time and caches locally, that would be a 
great addition.

> This patch is still a bit rough around the edges, but I'm posting it now
> to get feedback.

Two problems I encountered when testing:

- It tries to connect to remote directories which have no currenct 
connection. This line:

+                    (cons (car dir) (project--find-in-directory (car 
dir))))

I fixed it by adding

   (when (or (not (file-remote-p (car dir)))
                  (file-remote-p (car dir) nil t))
     ...

but there might be a more efficient solution. We could drop remote 
projects from this altogether, but OTOH some users could work 
exclusively over Tramp (e.g. in local Docker VMs). Something to think about.

- The (return ...) call needed the cl- prefix. It doesn't compile 
otherwise (and breaks in 'emacs -Q').