bug#65520: 30.0.50; [FR Xref] Project-wide operations

bug#65520: 30.0.50; [FR Xref] Project-wide operations

[Gerd Möllmann]

In C mode, Xref works using Eglot, in ELisp mode it uses something else
(I have no tags tables; donÄt know how that works).  Both work really
well, but I apparently have to be in an ELisp buffer to use one and in a
C buffer to use the other.

Feature request: Is it possible to use more than one backend at the same
time?  So that I could C-M-. to find an ELisp function while being in a
C file?  I couldn't find something like that mentioned in the docs, so I
guess it's not yet possible.

In GNU Emacs 30.0.50 (build 2, aarch64-apple-darwin22.6.0, NS
 appkit-2299.70 Version 13.5 (Build 22G74)) of 2023-08-24 built on
 Mini.fritz.box
Repository revision: 53c07bd04bf59f63e49af2c626714bf3fdd03ad6
Repository branch: scratch/pkg
Windowing system distributor 'Apple', version 10.3.2299
System Description:  macOS 13.5

bug#65520: 30.0.50; [FR Xref] Project-wide operations

[Gerd Möllmann]

Dmitry Gutov <dmitry@gutov.dev> writes:

>> In C mode, Xref works using Eglot, in ELisp mode it uses something
>> else
>> (I have no tags tables).  Both work really well, but I apparently have
>> to be in an ELisp buffer to use one and in a C buffer to use the other.
>> Feature request: Is it possible to use more than one backend at the
>> same
>> time?  So that I could C-M-. to find an ELisp function while being in a
>> C file?  I couldn't find something like that mentioned in the docs, so I
>> guess it's not yet possible.
>
> No such capability at the moment, but we indeed have an ELPA package
> xref-union which implements the "merging" feature that has been
> requested in the past.

Thanks also to Visuwesh for pointing me to xref-union.

> You are asking for something a little different, though: to be able to
> switch to a different language/project that's also present in the
> current Emacs session. Its backend function cannot be found in the
> xref-backend-functions in the current buffer (local or global), unlike
> the examples with mixing in etags or imenu results, like mentioned by
> Visuwesh.
>
> A naive approach, or for the new code to search across different open
> buffers and look for different available xref-backend-functions
> elements. And then prompt the user, and then call (?) the said
> function in one of the buffers it was found in, because in all
> likelihood it would fail in unrelated ones.
>
> Perhaps the solution is to add a new facility to Xref, where different
> "projects" would be able to register globally in. A feature request
> indeed.

I'm not sure about the "different project in the same session" thing.

My mental model is like so: I'm in a Git repo (Emacs), which is, I think
also a project in the project.el sense.  This project contains differnt
sets of files for which information is available using different
backends (Eglot, Etags, others depending on the kind of project).  What
U'd like to have is something on the level of such a project, if you
know what mean.  That is, M-. would take all available info for such a
project into account.

bug#65520: 30.0.50; [FR Xref] Project-wide operations

[Eshel Yaron via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

>>> Feature request: Is it possible to use more than one backend at the
>>> same time?  So that I could C-M-. to find an ELisp function while
>>> being in a C file?  I couldn't find something like that mentioned in
>>> the docs, so I guess it's not yet possible.

Note that specifically for Elisp functions, you can use `find-function`
from anywhere (including your C buffers).

>> A naive approach, or for the new code to search across different open
>> buffers and look for different available xref-backend-functions
>> elements. And then prompt the user, and then call (?) the said
>> function in one of the buffers it was found in, because in all
>> likelihood it would fail in unrelated ones.
>>
>> Perhaps the solution is to add a new facility to Xref, where different
>> "projects" would be able to register globally in. A feature request
>> indeed.

> My mental model is like so: I'm in a Git repo (Emacs), which is, I think
> also a project in the project.el sense.  This project contains differnt
> sets of files for which information is available using different
> backends (Eglot, Etags, others depending on the kind of project).  What
> U'd like to have is something on the level of such a project, if you
> know what mean.  That is, M-. would take all available info for such a
> project into account.

That sounds really interesting!  The way I'd suggest structuring it is
introducing a new command `project-find-definitions` that prompts for an
identifier defined in the current project and jumps to it, similarly to
`xref-find-definitions` but limited to the current project and not
limited to the current `xref` backend.  The way
`project-find-definitions` finds this definitions would be up to the
`project` backend.  Crucially, ISTM that it would be better to build
this as an extension of `project`, not `xref`.  Of course, the "naive
approach" based on `xref` described above would be a perfectly good
starting point for any `project` backend and may be used as a default.


Best,

Eshl

bug#65520: 30.0.50; [FR Xref] Project-wide operations

[Dmitry Gutov]

On 25/08/2023 10:48, Eshel Yaron wrote:
> The way I'd suggest structuring it is
> introducing a new command `project-find-definitions` that prompts for an
> identifier defined in the current project and jumps to it, similarly to
> `xref-find-definitions` but limited to the current project and not
> limited to the current `xref` backend.

To make this work ootb (as opposed to custom user setup), this would 
require some global-ish registry of languages available in a project.

project.el itself is unlikely to help too much, but it could host the 
new variable(s).

bug#65520: 30.0.50; [FR Xref] Project-wide operations

[Gerd Möllmann]

Gerd Möllmann <gerd.moellmann@gmail.com> writes:

> My mental model is like so: I'm in a Git repo (Emacs), which is, I think
> also a project in the project.el sense.  This project contains differnt
> sets of files for which information is available using different
> backends

Just want to add, because I didn't clearly state that: these set of
files don't have to be disjoint, i.e. more than one backend might have
information about the same file.

bug#65520: 30.0.50; [FR Xref] Project-wide operations

[Dmitry Gutov]

On 25/08/2023 10:03, Gerd Möllmann wrote:
> My mental model is like so: I'm in a Git repo (Emacs), which is, I think
> also a project in the project.el sense.  This project contains differnt
> sets of files for which information is available using different
> backends (Eglot, Etags, others depending on the kind of project).  What
> U'd like to have is something on the level of such a project, if you
> know what mean.  That is, M-. would take all available info for such a
> project into account.

One possible alternative is to treat this situation not like a new 
feature, and write a specialized new Xref backend which would collect 
both the info from Lisp and from TAGS when you are anywhere inside the 
Emacs sources. It shouldn't take too many lines either.

The current existing alternative for that, like Eli pointed out, is 
xref-etags-mode: it disables Elisp's own backend and just uses TAGS 
everywhere ('make tags' in Emacs generates tags for Lisp functions as 
well). With the natural downside that you would need to regen tags 
manually for both types of files now. And that you're using Eglot 
instead ;-(.

bug#65520: 30.0.50; [FR Xref] Project-wide operations

[Gerd Möllmann]

Dmitry Gutov <dmitry@gutov.dev> writes:

> On 25/08/2023 10:03, Gerd Möllmann wrote:
>> My mental model is like so: I'm in a Git repo (Emacs), which is, I think
>> also a project in the project.el sense.  This project contains differnt
>> sets of files for which information is available using different
>> backends (Eglot, Etags, others depending on the kind of project).  What
>> U'd like to have is something on the level of such a project, if you
>> know what mean.  That is, M-. would take all available info for such a
>> project into account.
>
> One possible alternative is to treat this situation not like a new
> feature, and write a specialized new Xref backend which would collect
> both the info from Lisp and from TAGS when you are anywhere inside the
> Emacs sources. It shouldn't take too many lines either.
>
> The current existing alternative for that, like Eli pointed out, is
> xref-etags-mode: it disables Elisp's own backend and just uses TAGS
> everywhere ('make tags' in Emacs generates tags for Lisp functions as
> well). With the natural downside that you would need to regen tags
> manually for both types of files now. And that you're using Eglot
> instead ;-(.

Yeah ;-).  That's not what I was looking for. 

bug#65520: 30.0.50; [FR Xref] Project-wide operations

[Dmitry Gutov]

On 25/08/2023 10:03, Gerd Möllmann wrote:
> Dmitry Gutov <dmitry@gutov.dev> writes:
> 
>>> In C mode, Xref works using Eglot, in ELisp mode it uses something
>>> else
>>> (I have no tags tables).  Both work really well, but I apparently have
>>> to be in an ELisp buffer to use one and in a C buffer to use the other.
>>> Feature request: Is it possible to use more than one backend at the
>>> same
>>> time?  So that I could C-M-. to find an ELisp function while being in a
>>> C file?  I couldn't find something like that mentioned in the docs, so I
>>> guess it's not yet possible.
>>
>> No such capability at the moment, but we indeed have an ELPA package
>> xref-union which implements the "merging" feature that has been
>> requested in the past.
> 
> Thanks also to Visuwesh for pointing me to xref-union.

Have you tried it? By setting up both it and the xref-backend-functions 
manually (in a hook) in every relevant buffer you might get more or less 
the requested behavior. If not, what does feel unsatisfying still?

>> A naive approach, or for the new code to search across different open
>> buffers and look for different available xref-backend-functions
>> elements. And then prompt the user, and then call (?) the said
>> function in one of the buffers it was found in, because in all
>> likelihood it would fail in unrelated ones.
>>
>> Perhaps the solution is to add a new facility to Xref, where different
>> "projects" would be able to register globally in. A feature request
>> indeed.
> 
> I'm not sure about the "different project in the same session" thing.
> 
> My mental model is like so: I'm in a Git repo (Emacs), which is, I think
> also a project in the project.el sense.  This project contains differnt
> sets of files for which information is available using different
> backends (Eglot, Etags, others depending on the kind of project).  What
> U'd like to have is something on the level of such a project, if you
> know what mean.  That is, M-. would take all available info for such a
> project into account.

Indeed, it's also a question of mental model, which we might have 
different. When I was saying "different projects" and "register 
globally", I was thinking of being able to jump to Emacs' sources and 
hack them from anywhere: from any other project I might be working on at 
the moment.

If we're talking about Emacs development only, a "combined" backend 
might make more sense (see the other email).

bug#65520: 30.0.50; [FR Xref] Project-wide operations

[Gerd Möllmann]

Dmitry Gutov <dmitry@gutov.dev> writes:

> On 25/08/2023 10:03, Gerd Möllmann wrote:
>> Dmitry Gutov <dmitry@gutov.dev> writes:
>> 
>>>> In C mode, Xref works using Eglot, in ELisp mode it uses something
>>>> else
>>>> (I have no tags tables).  Both work really well, but I apparently have
>>>> to be in an ELisp buffer to use one and in a C buffer to use the other.
>>>> Feature request: Is it possible to use more than one backend at the
>>>> same
>>>> time?  So that I could C-M-. to find an ELisp function while being in a
>>>> C file?  I couldn't find something like that mentioned in the docs, so I
>>>> guess it's not yet possible.
>>>
>>> No such capability at the moment, but we indeed have an ELPA package
>>> xref-union which implements the "merging" feature that has been
>>> requested in the past.
>> Thanks also to Visuwesh for pointing me to xref-union.
>
> Have you tried it?

Not yet.  I'm still hoping someone else (tm) builds something, so that I
don't have to do anything.

> By setting up both it and the
> xref-backend-functions manually (in a hook) in every relevant buffer
> you might get more or less the requested behavior. If not, what does
> feel unsatisfying still?
>
>>> A naive approach, or for the new code to search across different open
>>> buffers and look for different available xref-backend-functions
>>> elements. And then prompt the user, and then call (?) the said
>>> function in one of the buffers it was found in, because in all
>>> likelihood it would fail in unrelated ones.
>>>
>>> Perhaps the solution is to add a new facility to Xref, where different
>>> "projects" would be able to register globally in. A feature request
>>> indeed.
>> I'm not sure about the "different project in the same session"
>> thing.
>> My mental model is like so: I'm in a Git repo (Emacs), which is, I
>> think
>> also a project in the project.el sense.  This project contains differnt
>> sets of files for which information is available using different
>> backends (Eglot, Etags, others depending on the kind of project).  What
>> U'd like to have is something on the level of such a project, if you
>> know what mean.  That is, M-. would take all available info for such a
>> project into account.
>
> Indeed, it's also a question of mental model, which we might have
> different. When I was saying "different projects" and "register
> globally", I was thinking of being able to jump to Emacs' sources and
> hack them from anywhere: from any other project I might be working on
> at the moment.
>
> If we're talking about Emacs development only, a "combined" backend
> might make more sense (see the other email).

No, I didn't mean Emacs development specifically, although that's
probably the only things I'd use such a facility for.

The whole thing would be like in some IDEs, with their "usual" idea of
project.  I'm not 100% sure, but I think I've seen vscode being used
with different languages (JS, CSS, Java...) and offering such
project-wide stuff.

bug#65520: 30.0.50; [FR Xref] Project-wide operations

[Dmitry Gutov]

On 26/08/2023 08:31, Gerd Möllmann wrote:

>>>> No such capability at the moment, but we indeed have an ELPA package
>>>> xref-union which implements the "merging" feature that has been
>>>> requested in the past.
>>> Thanks also to Visuwesh for pointing me to xref-union.
>>
>> Have you tried it?
> 
> Not yet.  I'm still hoping someone else (tm) builds something, so that I
> don't have to do anything.

xref-union is a thing that somebody has already built. Either you try it 
and like it, or you can come back with more targeted feedback.

Which we could then use to improve either it, or the core xref, of course.

>> Indeed, it's also a question of mental model, which we might have
>> different. When I was saying "different projects" and "register
>> globally", I was thinking of being able to jump to Emacs' sources and
>> hack them from anywhere: from any other project I might be working on
>> at the moment.
>>
>> If we're talking about Emacs development only, a "combined" backend
>> might make more sense (see the other email).
> 
> No, I didn't mean Emacs development specifically, although that's
> probably the only things I'd use such a facility for.
> 
> The whole thing would be like in some IDEs, with their "usual" idea of
> project.  I'm not 100% sure, but I think I've seen vscode being used
> with different languages (JS, CSS, Java...) and offering such
> project-wide stuff.

IIRC the LSP protocol includes project type detection, and so Eglot can 
automatically spin up several language servers already.

Without the addition of a similar registry, which all backends would 
have to plug into, the best we could do is loop across the open project 
buffers and see what backends exist there. But that would e.g. miss 
Elisp if you haven't opened any .el files in the current session yet.

bug#65520: 30.0.50; [FR Xref] Project-wide operations

[Gerd Möllmann]

Dmitry Gutov <dmitry@gutov.dev> writes:

>>> Have you tried it?
>> Not yet.  I'm still hoping someone else (tm) builds something, so
>> that I
>> don't have to do anything.
>
> xref-union is a thing that somebody has already built. Either you try
> it and like it, or you can come back with more targeted feedback.
>
> Which we could then use to improve either it, or the core xref, of
> course.

I know, I know. I'm procrastinating ;-).  

...
>> No, I didn't mean Emacs development specifically, although that's
>> probably the only things I'd use such a facility for.
>> The whole thing would be like in some IDEs, with their "usual" idea
>> of
>> project.  I'm not 100% sure, but I think I've seen vscode being used
>> with different languages (JS, CSS, Java...) and offering such
>> project-wide stuff.
>
> IIRC the LSP protocol includes project type detection, and so Eglot
> can automatically spin up several language servers already.
>
> Without the addition of a similar registry, which all backends would
> have to plug into, the best we could do is loop across the open
> project buffers and see what backends exist there. But that would
> e.g. miss Elisp if you haven't opened any .el files in the current
> session yet.

Can't say anything to that...  BTW, I'm not planning on working on
anything in that are.  Just wishing to have something like I described.

bug#65520: 30.0.50; [FR Xref] Project-wide operations

[Eli Zaretskii]

> From: Gerd Möllmann <gerd.moellmann@gmail.com>
> Date: Fri, 25 Aug 2023 08:49:01 +0200
> 
> In C mode, Xref works using Eglot, in ELisp mode it uses something else
> (I have no tags tables; don't know how that works).

It works by using the ELisp's own backend.

> Both work really
> well, but I apparently have to be in an ELisp buffer to use one and in a
> C buffer to use the other.
> 
> Feature request: Is it possible to use more than one backend at the same
> time?  So that I could C-M-. to find an ELisp function while being in a
> C file?  I couldn't find something like that mentioned in the docs, so I
> guess it's not yet possible.

Probably not what you had in mind, but "M-x xref-etags-mode" will
cause ELisp to use etags as well, and then you can have what you want
(at least in Emacs) with a single backend.