Adding support for xref jumping to headers/interfaces

Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

Hi,

xref provides support for jump-to-definition, but no support for
jump-to-interface.  The latter would jump to the header or interface
file for an identifier.  This functionality exists in some packages
already because it is particularly important in some languages, and I'm
looking for suggestions on how to expose it through xref.

For example, merlin, for OCaml, supports both jump-to-definition and
jump-to-interface.  However, merlin's xref support only exposes
jump-to-definition (through xref-find-definitions), because there's no
natural way to expose jump-to-interface through xref.

It would be straightforward to extend xref with a new command
xref-find-interfaces.  The main difficulty is picking a good UI to
expose this command.  I've thought of a few options, but nothing that
seems obviously correct, so I welcome opinions.

Suppose there's a new command xref-find-interfaces.  Then some options are:

1. xref-find-interfaces could be bound by default with some new binding.
   I'm not sure what a good binding would be.

2. xref-find-definitions could call xref-find-interfaces if called with
   a prefix argument.  Note that this would change the current prefix
   argument behavior of xref-find-definitions.

3. xref-find-definitions could call xref-find-interfaces if called after
   some custom prefix is typed, like how other-window-prefix works.

4. xref-find-definitions could call xref-find-interfaces if a certain
   customization variable is set.

5. Like option 5, but there could also be a new binding to
   toggle that variable on and off.

I think the most satisfying of these is option 3, but maybe that's too
unusual of an interface for core Emacs?

Thoughts?

Thanks,
Spencer Baugh

Re: Adding support for xref jumping to headers/interfaces

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Cc: azeng@janestreet.com
> Date: Fri, 24 Feb 2023 14:56:59 -0500
> 
> 
> xref provides support for jump-to-definition, but no support for
> jump-to-interface.  The latter would jump to the header or interface
> file for an identifier.  This functionality exists in some packages
> already because it is particularly important in some languages, and I'm
> looking for suggestions on how to expose it through xref.
> 
> For example, merlin, for OCaml, supports both jump-to-definition and
> jump-to-interface.  However, merlin's xref support only exposes
> jump-to-definition (through xref-find-definitions), because there's no
> natural way to expose jump-to-interface through xref.
> 
> It would be straightforward to extend xref with a new command
> xref-find-interfaces.  The main difficulty is picking a good UI to
> expose this command.  I've thought of a few options, but nothing that
> seems obviously correct, so I welcome opinions.
> 
> Suppose there's a new command xref-find-interfaces.  Then some options are:
> 
> 1. xref-find-interfaces could be bound by default with some new binding.
>    I'm not sure what a good binding would be.
> 
> 2. xref-find-definitions could call xref-find-interfaces if called with
>    a prefix argument.  Note that this would change the current prefix
>    argument behavior of xref-find-definitions.
> 
> 3. xref-find-definitions could call xref-find-interfaces if called after
>    some custom prefix is typed, like how other-window-prefix works.
> 
> 4. xref-find-definitions could call xref-find-interfaces if a certain
>    customization variable is set.
> 
> 5. Like option 5, but there could also be a new binding to
>    toggle that variable on and off.
> 
> I think the most satisfying of these is option 3, but maybe that's too
> unusual of an interface for core Emacs?

I think 1 is the best alternative.  C-M-? is available, AFAIK.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 24/02/2023 22:04, Eli Zaretskii wrote:
>> From: Spencer Baugh<sbaugh@janestreet.com>
>> Cc:azeng@janestreet.com
>> Date: Fri, 24 Feb 2023 14:56:59 -0500
>>
>>
>> xref provides support for jump-to-definition, but no support for
>> jump-to-interface.  The latter would jump to the header or interface
>> file for an identifier.  This functionality exists in some packages
>> already because it is particularly important in some languages, and I'm
>> looking for suggestions on how to expose it through xref.
>>
>> For example, merlin, for OCaml, supports both jump-to-definition and
>> jump-to-interface.  However, merlin's xref support only exposes
>> jump-to-definition (through xref-find-definitions), because there's no
>> natural way to expose jump-to-interface through xref.
>>
>> It would be straightforward to extend xref with a new command
>> xref-find-interfaces.  The main difficulty is picking a good UI to
>> expose this command.  I've thought of a few options, but nothing that
>> seems obviously correct, so I welcome opinions.
>>
>> Suppose there's a new command xref-find-interfaces.  Then some options are:
>>
>> 1. xref-find-interfaces could be bound by default with some new binding.
>>     I'm not sure what a good binding would be.
>>
>> 2. xref-find-definitions could call xref-find-interfaces if called with
>>     a prefix argument.  Note that this would change the current prefix
>>     argument behavior of xref-find-definitions.
>>
>> 3. xref-find-definitions could call xref-find-interfaces if called after
>>     some custom prefix is typed, like how other-window-prefix works.
>>
>> 4. xref-find-definitions could call xref-find-interfaces if a certain
>>     customization variable is set.
>>
>> 5. Like option 5, but there could also be a new binding to
>>     toggle that variable on and off.
>>
>> I think the most satisfying of these is option 3, but maybe that's too
>> unusual of an interface for core Emacs?
> I think 1 is the best alternative.  C-M-? is available, AFAIK.

Option 1 sounds best to me too.

About the name, though -- does "jump to interfaces" make sense for many 
languages? To my ear, the naming seems specific to OCaml.

"jump to declaration(s)", perhaps?

Re: Adding support for xref jumping to headers/interfaces

[Stephen Leake]

Dmitry Gutov <dgutov@yandex.ru> writes:

> About the name, though -- does "jump to interfaces" make sense for
> many languages? To my ear, the naming seems specific to OCaml.
>
> "jump to declaration(s)", perhaps?

In Ada, the term is "specification(s)". I don't know of other languages
that use it, but the English meaning is clear.


-- 
-- Stephe

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 28/02/2023 01:12, Stephen Leake wrote:
> Dmitry Gutov<dgutov@yandex.ru>  writes:
> 
>> About the name, though -- does "jump to interfaces" make sense for
>> many languages? To my ear, the naming seems specific to OCaml.
>>
>> "jump to declaration(s)", perhaps?
> In Ada, the term is "specification(s)". I don't know of other languages
> that use it, but the English meaning is clear.

That sounds clear enough, but if it's only an Ada term, it might not be 
the best choice for other languages. E.g. from completion POV.

This seems like a good occasion for a popular vote. So everyone is 
welcome to state their preferred naming.

Re: Adding support for xref jumping to headers/interfaces

[Yuan Fu]

> On Feb 27, 2023, at 3:34 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
> 
> On 28/02/2023 01:12, Stephen Leake wrote:
>> Dmitry Gutov<dgutov@yandex.ru>  writes:
>>> About the name, though -- does "jump to interfaces" make sense for
>>> many languages? To my ear, the naming seems specific to OCaml.
>>> 
>>> "jump to declaration(s)", perhaps?
>> In Ada, the term is "specification(s)". I don't know of other languages
>> that use it, but the English meaning is clear.
> 
> That sounds clear enough, but if it's only an Ada term, it might not be the best choice for other languages. E.g. from completion POV.
> 
> This seems like a good occasion for a popular vote. So everyone is welcome to state their preferred naming.
> 

I think “interface” is widely used and conveys the meaning well. Java, Clojure and Javascript call them interfaces, too.

Yuan

Re: Adding support for xref jumping to headers/interfaces

[Filipp Gunbin]

On 27/02/2023 16:18 -0800, Yuan Fu wrote:

>> On Feb 27, 2023, at 3:34 PM, Dmitry Gutov <dgutov@yandex.ru> wrote:
>> 
>> On 28/02/2023 01:12, Stephen Leake wrote:
>>> Dmitry Gutov<dgutov@yandex.ru>  writes:
>>>> About the name, though -- does "jump to interfaces" make sense for
>>>> many languages? To my ear, the naming seems specific to OCaml.
>>>> 
>>>> "jump to declaration(s)", perhaps?
>>> In Ada, the term is "specification(s)". I don't know of other languages
>>> that use it, but the English meaning is clear.
>> 
>> That sounds clear enough, but if it's only an Ada term, it might not be the best choice for other languages. E.g. from completion POV.
>> 
>> This seems like a good occasion for a popular vote. So everyone is welcome to state their preferred naming.
>> 
>
> I think “interface” is widely used and conveys the meaning well. Java, Clojure and Javascript call them interfaces, too.

In Java it makes more sense to talk about abstract methods, if we're on
the method level.  Interfaces are type-level construct, and they include
not only abstract methods, but also default and static methods (with
implementation), constants etc.

TBH, I don't see how this could be named well to fit all languages.  I
like "declaration" word more, if we need a name.  However, it's
confusable with "definition".

Re: Adding support for xref jumping to headers/interfaces

[John Yates]

On Mon, Feb 27, 2023 at 7:19 PM Yuan Fu <casouri@gmail.com> wrote:
>
> I think “interface” is widely used and conveys the meaning well. Java, Clojure and Javascript call them interfaces, too.

In the C++ community the term 'interface' is part of the
vernacular, especially within the TLAs API and ABI.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 28/02/2023 23:40, John Yates wrote:
> On Mon, Feb 27, 2023 at 7:19 PM Yuan Fu<casouri@gmail.com>  wrote:
>> I think “interface” is widely used and conveys the meaning well. Java, Clojure and Javascript call them interfaces, too.
> In the C++ community the term 'interface' is part of the
> vernacular, especially within the TLAs API and ABI.

But would you call that navigation "jumping to method's interface(s)"?

AFAIK, in Java you still call the method definitions inside an interface 
(file/entity) "method declarations". Unless it's a "default" method, 
available with Java 8+.

Clojure is similarly able to define Java interfaces (with no special 
term for methods enumerated inside, AFAIK), or Protocols (methods inside 
are called "protocol methods", but it would probably be fair to call 
them "declarations" as well).

More importantly, I guess, in both Java or C/C++ you can have method 
declarations that are not part of an "interface". E.g. you have some 
class with abstract method or several.

The LSP protocol uses the term "declaration", so we probably won't make 
too much of a mistake reusing that term. But, indeed, it sounds similar 
enough to "definition".

Re: Adding support for xref jumping to headers/interfaces

[Konstantin Kharlamov]

I vote for "declaration". I like both "interface" and "declaration", but given 
Dmitry's reasoning below, I'm also inclined to vote for "declaration".

On Tue, 2023-02-28 at 23:53 +0200, Dmitry Gutov wrote:
> On 28/02/2023 23:40, John Yates wrote:
> > On Mon, Feb 27, 2023 at 7:19 PM Yuan Fu<casouri@gmail.com>  wrote:
> > > I think “interface” is widely used and conveys the meaning well. Java,
> > > Clojure and Javascript call them interfaces, too.
> > In the C++ community the term 'interface' is part of the
> > vernacular, especially within the TLAs API and ABI.
> 
> But would you call that navigation "jumping to method's interface(s)"?
> 
> AFAIK, in Java you still call the method definitions inside an interface 
> (file/entity) "method declarations". Unless it's a "default" method, 
> available with Java 8+.
> 
> Clojure is similarly able to define Java interfaces (with no special 
> term for methods enumerated inside, AFAIK), or Protocols (methods inside 
> are called "protocol methods", but it would probably be fair to call 
> them "declarations" as well).
> 
> More importantly, I guess, in both Java or C/C++ you can have method 
> declarations that are not part of an "interface". E.g. you have some 
> class with abstract method or several.
> 
> The LSP protocol uses the term "declaration", so we probably won't make 
> too much of a mistake reusing that term. But, indeed, it sounds similar 
> enough to "definition".

Re: Adding support for xref jumping to headers/interfaces

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > TBH, I don't see how this could be named well to fit all languages.  I
  > like "declaration" word more, if we need a name.

I think that "declaration" will be clear, even for users of the language
where the precise term is "interface".

If I understand right, an "interface" would describe an entire package's
calling conventions.  Is that correct?  If so, then since we are talking
about the calling specs for one specific function, "declaration"
would fit that better than "interface".

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Adding support for xref jumping to headers/interfaces

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: casouri@gmail.com, dgutov@yandex.ru,
> 	stephen_leake@stephe-leake.org, eliz@gnu.org,
> 	sbaugh@janestreet.com, emacs-devel@gnu.org,
> 	azeng@janestreet.com
> Date: Tue, 28 Feb 2023 23:33:47 -0500
> 
>   > TBH, I don't see how this could be named well to fit all languages.  I
>   > like "declaration" word more, if we need a name.
> 
> I think that "declaration" will be clear, even for users of the language
> where the precise term is "interface".
> 
> If I understand right, an "interface" would describe an entire package's
> calling conventions.  Is that correct?  If so, then since we are talking
> about the calling specs for one specific function, "declaration"
> would fit that better than "interface".

We can mention the alternative terminology in the doc string, so we
don't necessarily have to find a one-fits-all name.

RE: [External] : Re: Adding support for xref jumping to headers/interfaces

[Drew Adams]

>   > TBH, I don't see how this could be named well to fit all languages.
>   > I like "declaration" word more, if we need a name.
> 
> I think that "declaration" will be clear, even for users of the language
> where the precise term is "interface".
> 
> If I understand right, an "interface" would describe an entire package's
> calling conventions.  Is that correct?  If so, then since we are talking
> about the calling specs for one specific function, "declaration"
> would fit that better than "interface".

Caveat: I haven't been following this.  And I have no problem with "declaration" for what I think you're describing. But just to mention this:

A function's "calling specs", meaning a spec of its arguments and its return value, are often called its "signature".  This is the case for both programming in general and theory.

Re: Adding support for xref jumping to headers/interfaces

[John Yates]

Earlier in this thread I mentioned that 'interface' is common
in the vernacular of the C/C++ community.  That was not
to express a preference for the term 'interface', only no strong
objection.

My preference is for 'declaration', for which recent postings
are making a good case.

Re: Adding support for xref jumping to headers/interfaces

[Stephen Leake]

I'm ok with "declaration"

-- 
-- Stephe

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 04/03/2023 20:54, Stephen Leake wrote:
> I'm ok with "declaration"

Thanks all!

Seems like we're going with 'declaration' after all.

Now, the next question is will we be able to provide some implementation 
of this method using one of the default backends?

elisp -- probably not.

Can we have one for 'etags'? Even if we just limit it to a pre-defined 
set of languages?

Otherwise, I guess, we'll only add to Eglot's xref backend.

Re: Adding support for xref jumping to headers/interfaces

[Eli Zaretskii]

> Date: Sun, 5 Mar 2023 00:24:01 +0200
> Cc: Eli Zaretskii <eliz@gnu.org>, rms@gnu.org, fgunbin@fastmail.fm,
>  casouri@gmail.com, sbaugh@janestreet.com, emacs-devel@gnu.org,
>  azeng@janestreet.com
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> Now, the next question is will we be able to provide some implementation 
> of this method using one of the default backends?
> 
> elisp -- probably not.
> 
> Can we have one for 'etags'? Even if we just limit it to a pre-defined 
> set of languages?

For C-like languages, we'd need to run "etags --declarations" on *.h
files for that, I think.  And the resulting TAGS table will have stuff
other than just declarations, so we'd need some way of distinguishing
between them and the rest.

Alternatively, perhaps it should be possible to add a special
command-line option (--declarations-only ?) to etags, to cause it to
output tags only for declarations.

Re: Adding support for xref jumping to headers/interfaces

[Helmut Eller]

On Fri, Feb 24 2023, Spencer Baugh wrote:

> Hi,
>
> xref provides support for jump-to-definition, but no support for
> jump-to-interface.  The latter would jump to the header or interface
> file for an identifier.  This functionality exists in some packages
> already because it is particularly important in some languages, and I'm
> looking for suggestions on how to expose it through xref.

I think that it's a bad idea to add language specific commands to xref.
The next thing is that we have commands for jump-to-variable and
jump-to-constant and jump-to-macro ...

It would be better to leave those commands in the language specific mode
with the C-c prefix.

Helmut

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 05/03/2023 11:59, Helmut Eller wrote:
>> xref provides support for jump-to-definition, but no support for
>> jump-to-interface.  The latter would jump to the header or interface
>> file for an identifier.  This functionality exists in some packages
>> already because it is particularly important in some languages, and I'm
>> looking for suggestions on how to expose it through xref.
> I think that it's a bad idea to add language specific commands to xref.
> The next thing is that we have commands for jump-to-variable and
> jump-to-constant and jump-to-macro ...
> 
> It would be better to leave those commands in the language specific mode
> with the C-c prefix.

Is it still a problem if the command applies to a lot (most) of languages?

Macros usually use the same namespace as functions, so having a separate 
command is not essential.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 05/03/2023 08:11, Eli Zaretskii wrote:
>> Date: Sun, 5 Mar 2023 00:24:01 +0200
>> Cc: Eli Zaretskii<eliz@gnu.org>,rms@gnu.org,fgunbin@fastmail.fm,
>>   casouri@gmail.com,sbaugh@janestreet.com,emacs-devel@gnu.org,
>>   azeng@janestreet.com
>> From: Dmitry Gutov<dgutov@yandex.ru>
>>
>> Now, the next question is will we be able to provide some implementation
>> of this method using one of the default backends?
>>
>> elisp -- probably not.
>>
>> Can we have one for 'etags'? Even if we just limit it to a pre-defined
>> set of languages?
> For C-like languages, we'd need to run "etags --declarations" on *.h
> files for that, I think.  And the resulting TAGS table will have stuff
> other than just declarations, so we'd need some way of distinguishing
> between them and the rest.
> 
> Alternatively, perhaps it should be possible to add a special
> command-line option (--declarations-only ?) to etags, to cause it to
> output tags only for declarations.

The etags xref backend usually works with pre-generated tags files. So a 
way to distinguish between them and the rest would be best.

With the second approach, I suppose we'd have to have some new naming 
convention for tags files (like TAGS and TAGS.decl), and load two of 
them automatically.

If the feature is useful enough to go to all this trouble, of course.

Re: Adding support for xref jumping to headers/interfaces

[Helmut Eller]

On Sun, Mar 05 2023, Dmitry Gutov wrote:

> On 05/03/2023 11:59, Helmut Eller wrote:
>> I think that it's a bad idea to add language specific commands to xref.
>> The next thing is that we have commands for jump-to-variable and
>> jump-to-constant and jump-to-macro ...
>> It would be better to leave those commands in the language specific
>> mode
>> with the C-c prefix.
>
> Is it still a problem if the command applies to a lot (most) of languages?

It still creates this problem: as a user I don't know which command to
use, the more specific or the more general or both.

Helmut

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 05/03/2023 17:57, Helmut Eller wrote:
> On Sun, Mar 05 2023, Dmitry Gutov wrote:
> 
>> On 05/03/2023 11:59, Helmut Eller wrote:
>>> I think that it's a bad idea to add language specific commands to xref.
>>> The next thing is that we have commands for jump-to-variable and
>>> jump-to-constant and jump-to-macro ...
>>> It would be better to leave those commands in the language specific
>>> mode
>>> with the C-c prefix.
>> Is it still a problem if the command applies to a lot (most) of languages?
> It still creates this problem: as a user I don't know which command to
> use, the more specific or the more general or both.

When you need a declaration, use xref-find-declarations, and otherwise, 
xref-find-definitions, I guess? Spencer, who started this thread, 
apparently had a use case where he needs to "jump to interface".

I'm not sure if "definitions" should, in general, include 
"declarations". But we probably have included them up to now, at least 
as far as the etags backend is concerned.

Re: Adding support for xref jumping to headers/interfaces

[Stephen Leake]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 05/03/2023 17:57, Helmut Eller wrote:
>> On Sun, Mar 05 2023, Dmitry Gutov wrote:
>> 
>>> On 05/03/2023 11:59, Helmut Eller wrote:
>>>> I think that it's a bad idea to add language specific commands to xref.
>>>> The next thing is that we have commands for jump-to-variable and
>>>> jump-to-constant and jump-to-macro ...
>>>> It would be better to leave those commands in the language specific
>>>> mode
>>>> with the C-c prefix.
>>> Is it still a problem if the command applies to a lot (most) of languages?
>> It still creates this problem: as a user I don't know which command to
>> use, the more specific or the more general or both.
>
> When you need a declaration, use xref-find-declarations, and
> otherwise, xref-find-definitions, I guess? Spencer, who started this
> thread, apparently had a use case where he needs to "jump to
> interface".

ada-mode has "wisi-goto-spec/body"; if on a body (definition), goto
specification (declaration), if on a spec, goto body, if on a reference,
goto spec. 

-- 
-- Stephe

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 473 bytes --]

On Sun, Mar 5, 2023, 10:22 Helmut Eller <eller.helmut@gmail.com> wrote:.

>
> I think that it's a bad idea to add language specific commands to xref.
> The next thing is that we have commands for jump-to-variable and
> jump-to-constant and jump-to-macro ...
>
> It would be better to leave those commands in the language specific mode
> with the C-c prefix.
>

I agree, but the xref backend responsible for answering the request can
still be Eglot's.

João

[-- Attachment #2: Type: text/html, Size: 850 bytes --]

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 866 bytes --]

On Sun, Mar 5, 2023, 20:10 João Távora <joaotavora@gmail.com> wrote:

> On Sun, Mar 5, 2023, 10:22 Helmut Eller <eller.helmut@gmail.com> wrote:.
>
>>
>> I think that it's a bad idea to add language specific commands to xref.
>> The next thing is that we have commands for jump-to-variable and
>> jump-to-constant and jump-to-macro ...
>>
>> It would be better to leave those commands in the language specific mode
>> with the C-c prefix.
>>
>
> I agree, but the xref backend responsible for answering the request can
> still be Eglot's.
>

To clarify, i think the thin commands and the keybinds should be in the
major modes, but these commands should be thin and use xref as a library.
And the backend should be Eglot's,  if it happens to be managing the buffer
via LSP, else out can be whatever the major modes sets up.

João

Josh

>

[-- Attachment #2: Type: text/html, Size: 1768 bytes --]

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 05/03/2023 22:32, João Távora wrote:
> To clarify, i think the thin commands and the keybinds should be in the 
> major modes, but these commands should be thin and use xref as a 
> library. And the backend should be Eglot's,  if it happens to be 
> managing the buffer via LSP, else out can be whatever the major modes 
> sets up.

xref-find-declarations should be bound by major modes? Which modes?

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1191 bytes --]

On Sun, Mar 5, 2023, 20:40 Dmitry Gutov <dgutov@yandex.ru> wrote:

> On 05/03/2023 22:32, João Távora wrote:
> > To clarify, i think the thin commands and the keybinds should be in the
> > major modes, but these commands should be thin and use xref as a
> > library. And the backend should be Eglot's,  if it happens to be
> > managing the buffer via LSP, else out can be whatever the major modes
> > sets up.
>
> xref-find-declarations should be bound by major modes? Which modes?
>

No, you misunderstand. I wrote "commands and keybinds should be in the
major mode". Xref-find-declarations should not exist, because declaration
is not a universal thing common to all languages. That was Helmut's point
AFAIU, and i agree with it.

Which modes? The ones where "declaration" is a thing.

IOW xref should try to stay language-agnostic, but provide common framework
for language-specific things like major modes to reach backends like Eglot
or anything else

Mind you the same point made for xref can be made about LSP,
language-agnostic on paper, but many a language-specific concept poorly
disguised in there. But I see no reason to copy that flaw.

João

>

[-- Attachment #2: Type: text/html, Size: 1902 bytes --]

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 05/03/2023 23:04, João Távora wrote:
> 
> On Sun, Mar 5, 2023, 20:40 Dmitry Gutov <dgutov@yandex.ru 
> <mailto:dgutov@yandex.ru>> wrote:
> 
>     On 05/03/2023 22:32, João Távora wrote:
>      > To clarify, i think the thin commands and the keybinds should be
>     in the
>      > major modes, but these commands should be thin and use xref as a
>      > library. And the backend should be Eglot's,  if it happens to be
>      > managing the buffer via LSP, else out can be whatever the major
>     modes
>      > sets up.
> 
>     xref-find-declarations should be bound by major modes? Which modes?
> 
> 
> No, you misunderstand. I wrote "commands and keybinds should be in the 
> major mode". Xref-find-declarations should not exist, because 
> declaration is not a universal thing common to all languages. That was 
> Helmut's point AFAIU, and i agree with it.

Okay, let's follow on with that idea.

> Which modes? The ones where "declaration" is a thing.

So we'll have a dozen different commands called 'xxx-find-declarations' 
which will be implemented the same (?) way? Bound to the same key sequence?

If there a particular point to doing that? Do we expect other modes to 
reuse the same binding to do something else? That will reduce UI 
consistency, at the very least.

> IOW xref should try to stay language-agnostic, but provide common 
> framework for language-specific things like major modes to reach 
> backends like Eglot or anything else

Most programming languages have something that corresponds to the term 
"declaration".

> Mind you the same point made for xref can be made about LSP, 
> language-agnostic on paper, but many a language-specific concept poorly 
> disguised in there. But I see no reason to copy that flaw.

I don't mind the idea of extensibility (and packages like lsp-mode do 
this already: IIRC they have commands called lsp-find-declarations and 
lsp-find-implementations).

But insofar as languages do have shared syntactic concepts, I don't see 
why we wouldn't want to include more common ones.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1884 bytes --]

On Sun, Mar 5, 2023, 21:21 Dmitry Gutov <dgutov@yandex.ru> wrote:

> > Which modes? The ones where "declaration" is a thing.
>
> So we'll have a dozen different commands called 'xxx-find-declarations'
> which will be implemented the same (?) way? Bound to the same key sequence?
>
> If there a particular point to doing that? Do we expect other modes to
> reuse the same binding to do something else? That will reduce UI
> consistency, at the very least.
>

Implementation could be trivial, xref could even provide a command-defining
macro.

Consistency should be evaluated considering the two approaches. Is it more
consistent to have an xref command that doesn't really make sense in a
another dozen languages?

> IOW xref should try to stay language-agnostic, but provide common
> > framework for language-specific things like major modes to reach
> > backends like Eglot or anything else
>
> Most programming languages have something that corresponds to the term
> "declaration".
>

Not sure about that. But if one squints very hard, everything does indeed
look like a nail to one's hammer.

> Mind you the same point made for xref can be made about LSP,
> > language-agnostic on paper, but many a language-specific concept poorly
> > disguised in there. But I see no reason to copy that flaw.
>
> I don't mind the idea of extensibility (and packages like lsp-mode do
> this already: IIRC they have commands called lsp-find-declarations and
> lsp-find-implementations).
>

As commands, this would be bloat in Eglot. But Eglot can give access to
these LSP interfaces.

But insofar as languages do have shared syntactic concepts, I don't see
> why we wouldn't want to include more common ones.
>

"Definition" and "references" are universal enough. "Declaration", "class"
and others, not so much. But that's just my 2c.

João

>

[-- Attachment #2: Type: text/html, Size: 3318 bytes --]

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 05/03/2023 23:48, João Távora wrote:
> On Sun, Mar 5, 2023, 21:21 Dmitry Gutov <dgutov@yandex.ru 
> <mailto:dgutov@yandex.ru>> wrote:
> 
>      > Which modes? The ones where "declaration" is a thing.
> 
>     So we'll have a dozen different commands called 'xxx-find-declarations'
>     which will be implemented the same (?) way? Bound to the same key
>     sequence?
> 
>     If there a particular point to doing that? Do we expect other modes to
>     reuse the same binding to do something else? That will reduce UI
>     consistency, at the very least.
> 
> 
> Implementation could be trivial, xref could even provide a 
> command-defining macro.

Right. So we'll have a dozen commands with the same two-line definition, 
different names, but the same binding.

> Consistency should be evaluated considering the two approaches. Is it 
> more consistent to have an xref command that doesn't really make sense 
> in a another dozen languages?

Having a common binding and a common xref backend method to dispatch to 
simplifies things. The major modes which don't have the notion of 
'declaration' will have options: do nothing (and simply have the command 
return an error), or rebind the keys to a different command.

Note that whether the command works will also depend on the current Xref 
backend. etags will likely support declarations in a smaller set of 
languages than LSP clients. And yet some specialized packages can cover 
some of the remaining languages that LSP doesn't reach.

A major mode author won't necessarily be able to tell in advance whether 
an Xref backend will exist that would cover their language with this 
particular feature.

>      > Mind you the same point made for xref can be made about LSP,
>      > language-agnostic on paper, but many a language-specific concept
>     poorly
>      > disguised in there. But I see no reason to copy that flaw.
> 
>     I don't mind the idea of extensibility (and packages like lsp-mode do
>     this already: IIRC they have commands called lsp-find-declarations and
>     lsp-find-implementations).
> 
> 
> As commands, this would be bloat in Eglot. But Eglot can give access to 
> these LSP interfaces.

If we don't want to extend xref.el with such commands, packages like 
Eglot seem like the natural alternative, though. Because they can know 
which commands the LSP protocol can support. Eglot could even query for 
that support at runtime for a particular language, I think.

>     But insofar as languages do have shared syntactic concepts, I don't see
>     why we wouldn't want to include more common ones.
> 
> 
> "Definition" and "references" are universal enough. "Declaration", 
> "class" and others, not so much. But that's just my 2c.

declarations/implementations are definitely less universal than the 
first two. But I don't see how they will hurt: if we decide to extend 
the list, the main difficulty will just be choosing the bindings, I think.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

Dmitry Gutov <dgutov@yandex.ru> writes:

>> Implementation could be trivial, xref could even provide a
>> command-defining macro.
>
> Right. So we'll have a dozen commands with the same two-line
> definition, different names, but the same binding.

Lisp macros are great for cutting down repetition.  If all we want to
say is "this mode for language foo supports finding xref finding of
'declaration' it could even be as simple as

   (xref-define-find-command :declaration foo-mode-map)

at the top-level of foo-mode.el.

An Eglot xref backend would pick up on this ':declaration and hook it up
to LSP's 'textDocument/declaration' (if it makes sense).  Other xref
backends would do whatever.  A default binding in foo-mode-map could be
chosen by xref.el, answering your other objection.  Finding these lines
in major-modes be as common as say, easy-menu-define.

Additionally, if the Foo language additionally has "interfaces", and
"macros" and "gremlins" and the FooLS server supports those (via LSP
extensions which are somewhat common), then foo-mode.el can use some
similar xref.el macro for definiting specific commands for those.  Their
implementation would use jsonrpc-request for the custom
'x/foo-find-gremlins' JSONRPC request, eglot-current-server for finding
the server, and xref-make-match for making the objects.

>> Consistency should be evaluated considering the two approaches. Is
>> it more consistent to have an xref command that doesn't really make
>> sense in a another dozen languages?
>
> Having a common binding and a common xref backend method to dispatch
> to simplifies things. The major modes which don't have the notion of
> 'declaration' will have options: do nothing (and simply have the
> command return an error), or rebind the keys to a different command.

"This command is not supported" errors or commands with same bindings
that do something else are perfectly possible and pretty common in
Emacs.  But they are not the best examples of UI consistency.

> If we don't want to extend xref.el with such commands, packages like
> Eglot seem like the natural alternative, though. Because they can know
> which commands the LSP protocol can support.

Eglot doesn't know about languages, nor should it.  It's also not
"natural" for Eglot to provide commands and UI: its job is to hook up
Emacs' facilities with LSP interfaces.  The first bullet in eglot.el's
commentary explains this.

Anyway, if you have like your approach, don't mind me.  I'm just
suggesting alternatives in hopes they can be useful, not standing in
your way.  If you want add things to Eglot's xref backend, that's fine
in principle, but it's better coordinate first.

João

Re: Adding support for xref jumping to headers/interfaces

[John Yates]

On Sun, Mar 5, 2023 at 5:09 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> Having a common binding and a common xref backend method to dispatch to
> simplifies things. The major modes which don't have the notion of
> 'declaration' will have options: do nothing (and simply have the command
> return an error), or rebind the keys to a different command.

For languages that do not distinguish declaration from definition,
another alternative is to have both key bindings reference the same
find-definition functionality.

As a user, I suspect that I would find that easier to master and
remember as I switch from one language to another.  Rebinding to a
different command feels hacky, and special case.  I fear, if that
option exists, different languages will bind it to entirely different
functionalities, exactly the phenomenon I would hope we are trying to
avoid.

Re: Adding support for xref jumping to headers/interfaces

[Felician Nemeth]

Can you, please, consider adding more than just one new xref command?
The Language Server Protocol defines jump-to-declaration,
jump-to-implementation, and jump-to-typeDefinition commands as well.

----

I can image a new xref-find-extra command that provides a
`tmm-menubar'-like single key selection for the specific jump type.  The
list of jump-types could be filled up by Eglot or major-modes (ada-mode
would add "specifications", ocaml-mode would add "interface").  If C-M-?
is bound to this new xref-find-extra, then "C-M-? i" would jump to the
interface definition in ada-mode, and "C-M-? d" would ask the LSP server
for declarations in case of Eglot.

(Users who find "C-M-?" awkward can rebind xref-find-extra to "s-.")

What do you think?

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Mon, Mar 6, 2023 at 12:23 PM Felician Nemeth
<felician.nemeth@gmail.com> wrote:
>
> Can you, please, consider adding more than just one new xref command?
> The Language Server Protocol defines jump-to-declaration,
> jump-to-implementation, and jump-to-typeDefinition commands as well.
>
> ----
>
> I can image a new xref-find-extra command that provides a
> `tmm-menubar'-like single key selection for the specific jump type.  The
> list of jump-types could be filled up by Eglot or major-modes (ada-mode
> would add "specifications", ocaml-mode would add "interface").  If C-M-?
> is bound to this new xref-find-extra, then "C-M-? i" would jump to the
> interface definition in ada-mode, and "C-M-? d" would ask the LSP server
> for declarations in case of Eglot.
>
> (Users who find "C-M-?" awkward can rebind xref-find-extra to "s-.")
>
> What do you think?

To me, the "find extra" trampoline idea sounds nice too.  It doesn't
suffer from the drawback of adding "interface", "implementation",
"typeDefinition", "superSpecificLanguageConstruct" concepts to xref.el
in poorly-scalable fashion.

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 06/03/2023 15:15, João Távora wrote:
> On Mon, Mar 6, 2023 at 12:23 PM Felician Nemeth
> <felician.nemeth@gmail.com>  wrote:
>> Can you, please, consider adding more than just one new xref command?
>> The Language Server Protocol defines jump-to-declaration,
>> jump-to-implementation, and jump-to-typeDefinition commands as well.
>>
>> ----
>>
>> I can image a new xref-find-extra command that provides a
>> `tmm-menubar'-like single key selection for the specific jump type.  The
>> list of jump-types could be filled up by Eglot or major-modes (ada-mode
>> would add "specifications", ocaml-mode would add "interface").  If C-M-?
>> is bound to this new xref-find-extra, then "C-M-? i" would jump to the
>> interface definition in ada-mode, and "C-M-? d" would ask the LSP server
>> for declarations in case of Eglot.
>>
>> (Users who find "C-M-?" awkward can rebind xref-find-extra to "s-.")
>>
>> What do you think?
> To me, the "find extra" trampoline idea sounds nice too.  It doesn't
> suffer from the drawback of adding "interface", "implementation",
> "typeDefinition", "superSpecificLanguageConstruct" concepts to xref.el
> in poorly-scalable fashion.

Could Eglot dynamically report on which of these operations are 
supported in the current buffer?

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Mon, Mar 6, 2023 at 1:24 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 06/03/2023 15:15, João Távora wrote:
> > On Mon, Mar 6, 2023 at 12:23 PM Felician Nemeth
> > <felician.nemeth@gmail.com>  wrote:
> >> Can you, please, consider adding more than just one new xref command?
> >> The Language Server Protocol defines jump-to-declaration,
> >> jump-to-implementation, and jump-to-typeDefinition commands as well.
> >>
> >> ----
> >>
> >> I can image a new xref-find-extra command that provides a
> >> `tmm-menubar'-like single key selection for the specific jump type.  The
> >> list of jump-types could be filled up by Eglot or major-modes (ada-mode
> >> would add "specifications", ocaml-mode would add "interface").  If C-M-?
> >> is bound to this new xref-find-extra, then "C-M-? i" would jump to the
> >> interface definition in ada-mode, and "C-M-? d" would ask the LSP server
> >> for declarations in case of Eglot.
> >>
> >> (Users who find "C-M-?" awkward can rebind xref-find-extra to "s-.")
> >>
> >> What do you think?
> > To me, the "find extra" trampoline idea sounds nice too.  It doesn't
> > suffer from the drawback of adding "interface", "implementation",
> > "typeDefinition", "superSpecificLanguageConstruct" concepts to xref.el
> > in poorly-scalable fashion.
>
> Could Eglot dynamically report on which of these operations are
> supported in the current buffer?

I think that is doable, yes.  The question is whether to use the LSP
names for them or our "own" names.  I think I prefer the former, to
avoid concept creep.  So M-x xref-find-extra would offer
"textDocument/findImplementation", "textDocument/findDeclaration",
etc when backend reports these operation names (which in turn it does
if the server has the associated LSP capability).

Other non-Eglot backends would report other operation names,
but that's fine as long as whoever reports them also knows
how to handle them.

The consistency would be in the single xref-find-extra binding.
The LSP names are clear enough and we can even decamelify them
automatically to be nicer to the camelically disinclined like
myself.

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 06/03/2023 15:38, João Távora wrote:
> I think that is doable, yes.  The question is whether to use the LSP
> names for them or our "own" names.  I think I prefer the former, to
> avoid concept creep.  So M-x xref-find-extra would offer
> "textDocument/findImplementation", "textDocument/findDeclaration",
> etc when backend reports these operation names (which in turn it does
> if the server has the associated LSP capability).

I think we'll need readable names -- e.g. "implementations", 
"declarations", "type definitions" (with a space or dash, maybe), 
without the confusing "text document" stuff.

Anyway, if the list of types will be extensible, there'll be nothing to 
stop you from using any particular naming format.

> Other non-Eglot backends would report other operation names,
> but that's fine as long as whoever reports them also knows
> how to handle them.

I think the idea is to report the names of "things" that can be found.

> The consistency would be in the single xref-find-extra binding.
> The LSP names are clear enough and we can even decamelify them
> automatically to be nicer to the camelically disinclined like
> myself.

And remove the "textDocument/find" prefix. But, again, that's up to you.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 06/03/2023 15:38, João Távora wrote:
> The LSP names are clear enough and we can even decamelify them
> automatically to be nicer to the camelically disinclined like
> myself.

There's also the issue of assigning keys to them (for tmm-menubar alike 
interface), which might be harder to do automatically.

We could just do completing-read, but that's bound to be slower to work 
with.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Mon, Mar 6, 2023 at 1:46 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 06/03/2023 15:38, João Távora wrote:
> > I think that is doable, yes.  The question is whether to use the LSP
> > names for them or our "own" names.  I think I prefer the former, to
> > avoid concept creep.  So M-x xref-find-extra would offer
> > "textDocument/findImplementation", "textDocument/findDeclaration",
> > etc when backend reports these operation names (which in turn it does
> > if the server has the associated LSP capability).
>
> I think we'll need readable names -- e.g. "implementations",
> "declarations", "type definitions" (with a space or dash, maybe),
> without the confusing "text document" stuff.

We don't _need_ them, but it's generally nice to have readable names.

> I think the idea is to report the names of "things" that can be found.
>
> > The consistency would be in the single xref-find-extra binding.
> > The LSP names are clear enough and we can even decamelify them
> > automatically to be nicer to the camelically disinclined like
> > myself.
>
> And remove the "textDocument/find" prefix.

Right.  That's a LSP thing, and Eglot knows LSP, so perfectly
doable.

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 06/03/2023 01:40, John Yates wrote:
> For languages that do not distinguish declaration from definition,
> another alternative is to have both key bindings reference the same
> find-definition functionality.

I think I would generally prefer for the "find declarations" binding to 
end with an error when unsupported rather than repeat an existing 
command which is already accessible through an easier key binding.

Otherwise some users might waste time wondering over the difference, 
testing, and comparing -- I myself would, probably.

Anyway, the question is moot if we're going the way of dynamically 
detecting the set of possible navigations.

But I suppose some simpler languages might have no "extra" searches to 
suggest. I wouldn't necessarily oppose rebinding xref-find-extra in such 
major mode to something different, as long as the command is doing 
something related. The key bindings are a limited set, after all.

Re: Adding support for xref jumping to headers/interfaces

[John Yates]

On Mon, Mar 6, 2023 at 9:09 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> Anyway, the question is moot if we're going the way of dynamically
> detecting the set of possible navigations.

Agreed.

> But I suppose some simpler languages might have no "extra" searches to
> suggest. I wouldn't necessarily oppose rebinding xref-find-extra in such
> major mode to something different, as long as the command is doing
> something related.

At the very least that expectation should be documented.  Better yet
would be to have a concrete example of an appropriate rebinding.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 06/03/2023 16:15, John Yates wrote:
>> But I suppose some simpler languages might have no "extra" searches to
>> suggest. I wouldn't necessarily oppose rebinding xref-find-extra in such
>> major mode to something different, as long as the command is doing
>> something related.
> At the very least that expectation should be documented.  Better yet
> would be to have a concrete example of an appropriate rebinding.

I don't have one yet.

But this issue is not limited to Xref commands: major modes have been 
rebinding global key bindings for a long time.

Re: Adding support for xref jumping to headers/interfaces

[John Yates]

On Mon, Mar 6, 2023 at 9:23 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> But this issue is not limited to Xref commands: major modes have been
> rebinding global key bindings for a long time.

Agreed.  Still, package authors should be encouraged, where possible,
to support emacs UI consistency.  All else being equal, violating a widely
followed pattern is likely not a good local UI choice.

Pointing out the existence of an overarching key binding scheme, even
if it does not apply to a particular package, may prompt more thoughtful
choices of key bindings.

Re: Adding support for xref jumping to headers/interfaces

[Yuan Fu]

> On Mar 6, 2023, at 5:50 AM, Dmitry Gutov <dgutov@yandex.ru> wrote:
> 
> On 06/03/2023 15:38, João Távora wrote:
>> The LSP names are clear enough and we can even decamelify them
>> automatically to be nicer to the camelically disinclined like
>> myself.
> 
> There's also the issue of assigning keys to them (for tmm-menubar alike interface), which might be harder to do automatically.
> 
> We could just do completing-read, but that's bound to be slower to work with.
> 

I like the idea of xref-find-extra. For keys we can support both auto-compute (use first character) and backend-chosen keys, similar to read-multiple-choice. We can even allow users to customize it.

Completing-read is a bad idea, as you said, it’s going to be slow, much slower and less pleasant to use. With keys I can just type the binding for xref-find-extra and type the key, which would feels just like compound keybindings.



Yuan

Re: Adding support for xref jumping to headers/interfaces

[Felician Nemeth]

Yuan Fu <casouri@gmail.com> writes:

>> There's also the issue of assigning keys to them (for tmm-menubar
>> alike interface), which might be harder to do automatically.
>> 
>> We could just do completing-read, but that's bound to be slower to work with.
>> 
>
> I like the idea of xref-find-extra. For keys we can support both
> auto-compute (use first character) and backend-chosen keys, similar to
> read-multiple-choice. We can even allow users to customize it.

I think this approach might be problematic if the major-mode provides a
"declaration" type and, say, Eglot provides a "definition" type.  But
maybe, as you write, providing customization options to the users is a
simple solution in this rare case.

> Completing-read is a bad idea, as you said, it’s going to be slow,
> much slower and less pleasant to use.

Currently all I write are emails, but long ago I wrote a simple,
tmm-based completing-read function to overcome this problem.  Maybe that
approach is useful here as well:
https://github.com/nemethf/single-key-completion/

Re: Adding support for xref jumping to headers/interfaces

[Yuan Fu]

> On Mar 7, 2023, at 11:58 AM, Felician Nemeth <felician.nemeth@gmail.com> wrote:
> 
> Yuan Fu <casouri@gmail.com> writes:
> 
>>> There's also the issue of assigning keys to them (for tmm-menubar
>>> alike interface), which might be harder to do automatically.
>>> 
>>> We could just do completing-read, but that's bound to be slower to work with.
>>> 
>> 
>> I like the idea of xref-find-extra. For keys we can support both
>> auto-compute (use first character) and backend-chosen keys, similar to
>> read-multiple-choice. We can even allow users to customize it.
> 
> I think this approach might be problematic if the major-mode provides a
> "declaration" type and, say, Eglot provides a "definition" type.  But
> maybe, as you write, providing customization options to the users is a
> simple solution in this rare case.

We can provide a standard set of names & keys for common types and describe them in the docstring and manual, and ask major modes to follow it to their best effort. This way the common ones should be more or less consistent, and major modes are free to add specialized types.

> 
>> Completing-read is a bad idea, as you said, it’s going to be slow,
>> much slower and less pleasant to use.
> 
> Currently all I write are emails, but long ago I wrote a simple,
> tmm-based completing-read function to overcome this problem.  Maybe that
> approach is useful here as well:
> https://github.com/nemethf/single-key-completion/
> 

Very cool! I’ve been thinking about a “reduced M-x” that only shows a selected subset of the command that I use regularly, and use fast keys to pick candidates. This seems to fit that purpose.

Yuan

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 07/03/2023 21:58, Felician Nemeth wrote:
> Yuan Fu <casouri@gmail.com> writes:
> 
>>> There's also the issue of assigning keys to them (for tmm-menubar
>>> alike interface), which might be harder to do automatically.
>>>
>>> We could just do completing-read, but that's bound to be slower to work with.
>>>
>>
>> I like the idea of xref-find-extra. For keys we can support both
>> auto-compute (use first character) and backend-chosen keys, similar to
>> read-multiple-choice. We can even allow users to customize it.
> 
> I think this approach might be problematic if the major-mode provides a
> "declaration" type and, say, Eglot provides a "definition" type.  But
> maybe, as you write, providing customization options to the users is a
> simple solution in this rare case.

I'd rather the backend did the disambiguation rather than make it the 
responsibility of the user.

Or we can have a registry of extra types, but then we'll have to either 
think of making of extensible or discuss additions from time to time.

>> Completing-read is a bad idea, as you said, it’s going to be slow,
>> much slower and less pleasant to use.
> 
> Currently all I write are emails, but long ago I wrote a simple,
> tmm-based completing-read function to overcome this problem.  Maybe that
> approach is useful here as well:
> https://github.com/nemethf/single-key-completion/

This could work as the automated disambiguation approach, I guess?

Anyway, I think we'll make the reading UI customizable.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 05/03/2023 14:06, Dmitry Gutov wrote:
>> For C-like languages, we'd need to run "etags --declarations" on *.h
>> files for that, I think.  And the resulting TAGS table will have stuff
>> other than just declarations, so we'd need some way of distinguishing
>> between them and the rest.
>>
> 
> The etags xref backend usually works with pre-generated tags files. So a 
> way to distinguish between them and the rest would be best.

To expand on that thought: Universal Ctags has a bunch of extensions for 
the output -- in particular the "kind" field.

After 'sudo apt install universal-ctags', I can see this:

   $ ctags --list-kinds=c
   d  macro definitions
   e  enumerators (values inside an enumeration)
   f  function definitions
   g  enumeration names
   h  included header files
   l  local variables [off]
   m  struct, and union members
   p  function prototypes [off]
   s  structure names
   t  typedefs
   u  union names
   v  variable definitions
   x  external and forward variable declarations [off]
   z  function parameters inside function or prototype definitions [off]
   L  goto labels [off]
   D  parameters inside macro definitions [off]

That seems to mean that it can distinguish "function prototypes" and 
"forward variable declarations", although their scanning seems to be 
disabled by default (can be toggled via --kinds-c=+p+x).

So I suppose the question here is how hard it would be to add to 
existing etags parsers something like that, and whether we can extend 
the TAGS format in a backward compatible way with this info.

Re: Adding support for xref jumping to headers/interfaces

[Eli Zaretskii]

> Date: Wed, 8 Mar 2023 00:41:49 +0200
> From: Dmitry Gutov <dgutov@yandex.ru>
> Cc: stephen_leake@stephe-leake.org, john@yates-sheets.org, rms@gnu.org,
>  fgunbin@fastmail.fm, casouri@gmail.com, sbaugh@janestreet.com,
>  emacs-devel@gnu.org, azeng@janestreet.com
> 
> After 'sudo apt install universal-ctags', I can see this:
> 
>    $ ctags --list-kinds=c
>    d  macro definitions
>    e  enumerators (values inside an enumeration)
>    f  function definitions
>    g  enumeration names
>    h  included header files
>    l  local variables [off]
>    m  struct, and union members
>    p  function prototypes [off]
>    s  structure names
>    t  typedefs
>    u  union names
>    v  variable definitions
>    x  external and forward variable declarations [off]
>    z  function parameters inside function or prototype definitions [off]
>    L  goto labels [off]
>    D  parameters inside macro definitions [off]
> 
> That seems to mean that it can distinguish "function prototypes" and 
> "forward variable declarations", although their scanning seems to be 
> disabled by default (can be toggled via --kinds-c=+p+x).
> 
> So I suppose the question here is how hard it would be to add to 
> existing etags parsers something like that, and whether we can extend 
> the TAGS format in a backward compatible way with this info.

How do you envision a backward-compatible extension of TAGS format to
support such flags?  AFAIK, TAGS doesn't distinguish between the tags
by their kind, it just records all of them, and leaves the semantics
to the program which reads them.  The current format of TAGS is
described in etc/ETAGS.EBNF, in case you didn't know.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 08/03/2023 16:58, Eli Zaretskii wrote:
>> Date: Wed, 8 Mar 2023 00:41:49 +0200
>> From: Dmitry Gutov<dgutov@yandex.ru>
>> Cc:stephen_leake@stephe-leake.org,john@yates-sheets.org,rms@gnu.org,
>>   fgunbin@fastmail.fm,casouri@gmail.com,sbaugh@janestreet.com,
>>   emacs-devel@gnu.org,azeng@janestreet.com
>>
>> After 'sudo apt install universal-ctags', I can see this:
>>
>>     $ ctags --list-kinds=c
>>     d  macro definitions
>>     e  enumerators (values inside an enumeration)
>>     f  function definitions
>>     g  enumeration names
>>     h  included header files
>>     l  local variables [off]
>>     m  struct, and union members
>>     p  function prototypes [off]
>>     s  structure names
>>     t  typedefs
>>     u  union names
>>     v  variable definitions
>>     x  external and forward variable declarations [off]
>>     z  function parameters inside function or prototype definitions [off]
>>     L  goto labels [off]
>>     D  parameters inside macro definitions [off]
>>
>> That seems to mean that it can distinguish "function prototypes" and
>> "forward variable declarations", although their scanning seems to be
>> disabled by default (can be toggled via --kinds-c=+p+x).
>>
>> So I suppose the question here is how hard it would be to add to
>> existing etags parsers something like that, and whether we can extend
>> the TAGS format in a backward compatible way with this info.
> How do you envision a backward-compatible extension of TAGS format to
> support such flags?  AFAIK, TAGS doesn't distinguish between the tags
> by their kind, it just records all of them, and leaves the semantics
> to the program which reads them.  The current format of TAGS is
> described in etc/ETAGS.EBNF, in case you didn't know.

etags-snarf-tag doesn't seem to call (line-end-position), at least not 
in the latest version. So we can probably add

   DEL kind

to the end of both directtag and patterntag.

At least, if I just add "\x7f^?f" to the end of a random line, 
(etags-snarf-tag) still returns the same value right now.

Re: Adding support for xref jumping to headers/interfaces

[Eli Zaretskii]

> Date: Wed, 8 Mar 2023 20:23:21 +0200
> Cc: stephen_leake@stephe-leake.org, john@yates-sheets.org, rms@gnu.org,
>  fgunbin@fastmail.fm, casouri@gmail.com, sbaugh@janestreet.com,
>  emacs-devel@gnu.org, azeng@janestreet.com
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> On 08/03/2023 16:58, Eli Zaretskii wrote:
> >> Date: Wed, 8 Mar 2023 00:41:49 +0200
> >> From: Dmitry Gutov<dgutov@yandex.ru>
> >> Cc:stephen_leake@stephe-leake.org,john@yates-sheets.org,rms@gnu.org,
> >>   fgunbin@fastmail.fm,casouri@gmail.com,sbaugh@janestreet.com,
> >>   emacs-devel@gnu.org,azeng@janestreet.com
> >>
> >> After 'sudo apt install universal-ctags', I can see this:
> >>
> >>     $ ctags --list-kinds=c
> >>     d  macro definitions
> >>     e  enumerators (values inside an enumeration)
> >>     f  function definitions
> >>     g  enumeration names
> >>     h  included header files
> >>     l  local variables [off]
> >>     m  struct, and union members
> >>     p  function prototypes [off]
> >>     s  structure names
> >>     t  typedefs
> >>     u  union names
> >>     v  variable definitions
> >>     x  external and forward variable declarations [off]
> >>     z  function parameters inside function or prototype definitions [off]
> >>     L  goto labels [off]
> >>     D  parameters inside macro definitions [off]
> >>
> >> That seems to mean that it can distinguish "function prototypes" and
> >> "forward variable declarations", although their scanning seems to be
> >> disabled by default (can be toggled via --kinds-c=+p+x).
> >>
> >> So I suppose the question here is how hard it would be to add to
> >> existing etags parsers something like that, and whether we can extend
> >> the TAGS format in a backward compatible way with this info.
> > How do you envision a backward-compatible extension of TAGS format to
> > support such flags?  AFAIK, TAGS doesn't distinguish between the tags
> > by their kind, it just records all of them, and leaves the semantics
> > to the program which reads them.  The current format of TAGS is
> > described in etc/ETAGS.EBNF, in case you didn't know.
> 
> etags-snarf-tag doesn't seem to call (line-end-position), at least not 
> in the latest version. So we can probably add
> 
>    DEL kind
> 
> to the end of both directtag and patterntag.
> 
> At least, if I just add "\x7f^?f" to the end of a random line, 
> (etags-snarf-tag) still returns the same value right now.

There's more than just etags-snarf-tag that know about the format of
the file.  If all the rest are also okay with such a change, then yes,
this obstacle is down.  The next one is to teach etags itself produce
those flags.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 08/03/2023 21:49, Eli Zaretskii wrote:
> If all the rest are also okay with such a change, then yes,
> this obstacle is down.

Hmm, tag-implicit-name-match-p uses the $ anchor in its regexp (line 
1652), seemingly unnecessarily. That condition can break.

I suppose that we could ensure to only produce explicit tags when kinds 
are added.

Or add a new switch to etags which would add kinds, default to off. Then 
the new feature would be supported (by etags backend) only for users who 
made sure to generate that info.

Re: Adding support for xref jumping to headers/interfaces

[Eli Zaretskii]

> Date: Wed, 8 Mar 2023 22:15:18 +0200
> Cc: stephen_leake@stephe-leake.org, john@yates-sheets.org, rms@gnu.org,
>  fgunbin@fastmail.fm, casouri@gmail.com, sbaugh@janestreet.com,
>  emacs-devel@gnu.org, azeng@janestreet.com
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> On 08/03/2023 21:49, Eli Zaretskii wrote:
> > If all the rest are also okay with such a change, then yes,
> > this obstacle is down.
> 
> Hmm, tag-implicit-name-match-p uses the $ anchor in its regexp (line 
> 1652), seemingly unnecessarily. That condition can break.
> 
> I suppose that we could ensure to only produce explicit tags when kinds 
> are added.
> 
> Or add a new switch to etags which would add kinds, default to off. Then 
> the new feature would be supported (by etags backend) only for users who 
> made sure to generate that info.

Maybe an easier way would be to have separate tags files for each
supported category (e.g., one for declarations and prototypes, one for
definitions, etc.).  Then we don't need to change the format of the
tags file, only make changes in etags, and instruct etags.el to load
the relevant file as needed.

The list of categories provided by universal-ctags is quite long, but
I think we only need to support a small subset of that for our
purposes, right?  That would mean perhaps 3 or 4 separate tags files,
which might not be unreasonable.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 09/03/2023 08:13, Eli Zaretskii wrote:
>> Date: Wed, 8 Mar 2023 22:15:18 +0200
>> Cc: stephen_leake@stephe-leake.org, john@yates-sheets.org, rms@gnu.org,
>>   fgunbin@fastmail.fm, casouri@gmail.com, sbaugh@janestreet.com,
>>   emacs-devel@gnu.org, azeng@janestreet.com
>> From: Dmitry Gutov <dgutov@yandex.ru>
>>
>> On 08/03/2023 21:49, Eli Zaretskii wrote:
>>> If all the rest are also okay with such a change, then yes,
>>> this obstacle is down.
>>
>> Hmm, tag-implicit-name-match-p uses the $ anchor in its regexp (line
>> 1652), seemingly unnecessarily. That condition can break.
>>
>> I suppose that we could ensure to only produce explicit tags when kinds
>> are added.
>>
>> Or add a new switch to etags which would add kinds, default to off. Then
>> the new feature would be supported (by etags backend) only for users who
>> made sure to generate that info.
> 
> Maybe an easier way would be to have separate tags files for each
> supported category (e.g., one for declarations and prototypes, one for
> definitions, etc.).  Then we don't need to change the format of the
> tags file, only make changes in etags, and instruct etags.el to load
> the relevant file as needed.

What would the user interface for loading the tags look like?

Altering our format sounds much easier to me from that standpoint, and 
from the added complexity in managing the files/buffers containing 
indexes for different types.

> The list of categories provided by universal-ctags is quite long, but
> I think we only need to support a small subset of that for our
> purposes, right?  That would mean perhaps 3 or 4 separate tags files,
> which might not be unreasonable.

The granularity in that list seems useful to me, so I don't think we 
should try to merge the kinds into some large categories, if possible.

We might not support distinguishing between certain kinds from the 
outset (e.g. between enums and structs, or between macro and function 
definitions), and we probably don't support outright a number of kinds 
like goto labels, parameter definitions and local variables. That leaves 
more than 3-4 kinds remaining, though.

Re: Adding support for xref jumping to headers/interfaces

[Eli Zaretskii]

> Date: Thu, 9 Mar 2023 15:04:09 +0200
> Cc: stephen_leake@stephe-leake.org, john@yates-sheets.org, rms@gnu.org,
>  fgunbin@fastmail.fm, casouri@gmail.com, sbaugh@janestreet.com,
>  emacs-devel@gnu.org, azeng@janestreet.com
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> > Maybe an easier way would be to have separate tags files for each
> > supported category (e.g., one for declarations and prototypes, one for
> > definitions, etc.).  Then we don't need to change the format of the
> > tags file, only make changes in etags, and instruct etags.el to load
> > the relevant file as needed.
> 
> What would the user interface for loading the tags look like?

I'm not sure we'd need any UI.  The command that works with
declarations (say) would need to know that the default name of the
tags file is TAGS.decl (say), and either ask the user or naybe even
load that silently.

> Altering our format sounds much easier to me from that standpoint, and 
> from the added complexity in managing the files/buffers containing 
> indexes for different types.

Is it really that more complex?

OTOH, having those in a single file will need Xref to cope with
multiple matches, something that currently we largely avoid.  You'd
have the same symbol mentioned once for declaration, another time for
definition, perhaps yet another time for interface, etc.

> > The list of categories provided by universal-ctags is quite long, but
> > I think we only need to support a small subset of that for our
> > purposes, right?  That would mean perhaps 3 or 4 separate tags files,
> > which might not be unreasonable.
> 
> The granularity in that list seems useful to me, so I don't think we 
> should try to merge the kinds into some large categories, if possible.
> 
> We might not support distinguishing between certain kinds from the 
> outset (e.g. between enums and structs, or between macro and function 
> definitions), and we probably don't support outright a number of kinds 
> like goto labels, parameter definitions and local variables. That leaves 
> more than 3-4 kinds remaining, though.

My point was that we won't need dozen separate files.  How we get to
that conclusion, whether by analysis or by synthesis, is less
important ;-)

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 09/03/2023 17:36, Eli Zaretskii wrote:
>> Date: Thu, 9 Mar 2023 15:04:09 +0200
>> Cc: stephen_leake@stephe-leake.org, john@yates-sheets.org, rms@gnu.org,
>>   fgunbin@fastmail.fm, casouri@gmail.com, sbaugh@janestreet.com,
>>   emacs-devel@gnu.org, azeng@janestreet.com
>> From: Dmitry Gutov <dgutov@yandex.ru>
>>
>>> Maybe an easier way would be to have separate tags files for each
>>> supported category (e.g., one for declarations and prototypes, one for
>>> definitions, etc.).  Then we don't need to change the format of the
>>> tags file, only make changes in etags, and instruct etags.el to load
>>> the relevant file as needed.
>>
>> What would the user interface for loading the tags look like?
> 
> I'm not sure we'd need any UI.  The command that works with
> declarations (say) would need to know that the default name of the
> tags file is TAGS.decl (say), and either ask the user or naybe even
> load that silently.

There is also the issue of having those files generated and saved to 
appropriate files. If I run 'etags -o TAGS', I might not expect that 
additional files will appear. Or existing ones -- overwritten.

>> Altering our format sounds much easier to me from that standpoint, and
>> from the added complexity in managing the files/buffers containing
>> indexes for different types.
> 
> Is it really that more complex?

The buffer state seems to be the least clear part of etags.el to me.

I can easily imagine how to implement the feature the way I'm 
suggesting, and your way -- not so much.

> OTOH, having those in a single file will need Xref to cope with
> multiple matches, something that currently we largely avoid.  You'd
> have the same symbol mentioned once for declaration, another time for
> definition, perhaps yet another time for interface, etc.

We already support duplicates, don't we? E.g. definitions of the same 
method in different files, or even in the same file but in different 
classes (thus on different lines).

The main thing to decide is whether declarations will be included in the 
list of definitions. But that will just affect the behavior, not the 
ease of implementation.

>> We might not support distinguishing between certain kinds from the
>> outset (e.g. between enums and structs, or between macro and function
>> definitions), and we probably don't support outright a number of kinds
>> like goto labels, parameter definitions and local variables. That leaves
>> more than 3-4 kinds remaining, though.
> 
> My point was that we won't need dozen separate files.  How we get to
> that conclusion, whether by analysis or by synthesis, is less
> important ;-)

If it's going to be a file per kind, then let's see how many we could 
support:

  d, e, f, g, m, p, s, t, u, v, x.

Not a dozen, but almost.

Consider also this: someday we might want to stop supporting etags 
ourselves, as soon as we find an external tool which knows how to scan 
the same languages and outputs in the same format. At that point it will 
be an advantage if our output matches the information that some existing 
tool uses already. Like universal-ctags. Which could extend its TAGS 
output format to include our changes with little effort.

Or yet another approach: we switch to another file format right now 
which already supports the additional fields: ctags.

Re: Adding support for xref jumping to headers/interfaces

[Eli Zaretskii]

> Date: Thu, 9 Mar 2023 18:53:42 +0200
> Cc: stephen_leake@stephe-leake.org, john@yates-sheets.org, rms@gnu.org,
>  fgunbin@fastmail.fm, casouri@gmail.com, sbaugh@janestreet.com,
>  emacs-devel@gnu.org, azeng@janestreet.com
> From: Dmitry Gutov <dgutov@yandex.ru>
> 
> There is also the issue of having those files generated and saved to 
> appropriate files. If I run 'etags -o TAGS', I might not expect that 
> additional files will appear. Or existing ones -- overwritten.

That's not what I had in mind.  I thought we'd run etags several
times, as in

  etags --enums-only -o TAGS.enum
  etags --decls-only -o TAGS.decl

etc.

> > OTOH, having those in a single file will need Xref to cope with
> > multiple matches, something that currently we largely avoid.  You'd
> > have the same symbol mentioned once for declaration, another time for
> > definition, perhaps yet another time for interface, etc.
> 
> We already support duplicates, don't we?

Only when there's no way around that, and we consider that unfortunate
when it does happen.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 09/03/2023 19:31, Eli Zaretskii wrote:
>> Date: Thu, 9 Mar 2023 18:53:42 +0200
>> Cc: stephen_leake@stephe-leake.org, john@yates-sheets.org, rms@gnu.org,
>>   fgunbin@fastmail.fm, casouri@gmail.com, sbaugh@janestreet.com,
>>   emacs-devel@gnu.org, azeng@janestreet.com
>> From: Dmitry Gutov <dgutov@yandex.ru>
>>
>> There is also the issue of having those files generated and saved to
>> appropriate files. If I run 'etags -o TAGS', I might not expect that
>> additional files will appear. Or existing ones -- overwritten.
> 
> That's not what I had in mind.  I thought we'd run etags several
> times, as in
> 
>    etags --enums-only -o TAGS.enum
>    etags --decls-only -o TAGS.decl
> 
> etc.

Okay. That sounds like additional hassle for the users, though.

>>> OTOH, having those in a single file will need Xref to cope with
>>> multiple matches, something that currently we largely avoid.  You'd
>>> have the same symbol mentioned once for declaration, another time for
>>> definition, perhaps yet another time for interface, etc.
>>
>> We already support duplicates, don't we?
> 
> Only when there's no way around that, and we consider that unfortunate
> when it does happen.

Sure.

But whether we will show duplicates to the user, in any scenario, 
shouldn't depend on how we store information (in multiple files or not), 
only on what we do with it.

Re: Adding support for xref jumping to headers/interfaces

[Ergus]

Ho Dmitry and Joao

On March 6, 2023 3:09:40 PM GMT+01:00, Dmitry Gutov <dgutov@yandex.ru> wrote:
>On 06/03/2023 01:40, John Yates wrote:
>> For languages that do not distinguish declaration from definition,
>> another alternative is to have both key bindings reference the same
>> find-definition functionality.

Yes please, just fall back to find-definition (or find-references) + a warning when find-declaration or the the others are not supported? It is simpler and practical.

Alternatively there could be a default fallback command similar to find-references but reordering to list definitions first (just an example)

>I think I would generally prefer for the "find declarations" binding to end with an error when unsupported rather than repeat an existing command which is already accessible through an easier key binding.
>
>Otherwise some users might waste time wondering over the difference, testing, and comparing -- I myself would, probably.
>

Before binding to a key maybe let's start again with the  simpler: to add them in the right click context menu... And we can find a binding later.



-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 10/03/2023 02:57, Ergus wrote:
> Ho Dmitry and Joao
> 
> On March 6, 2023 3:09:40 PM GMT+01:00, Dmitry Gutov <dgutov@yandex.ru> wrote:
>> On 06/03/2023 01:40, John Yates wrote:
>>> For languages that do not distinguish declaration from definition,
>>> another alternative is to have both key bindings reference the same
>>> find-definition functionality.
> 
> Yes please, just fall back to find-definition (or find-references) + a warning when find-declaration or the the others are not supported? It is simpler and practical.

Could you explain this usage scenario? Why not just report "unsupported" 
and let the user try again with the simpler key binding (M-.)?

It's not like people will prefer to reach for C-M-?, or will hit it by 
accident.

> Alternatively there could be a default fallback command similar to find-references but reordering to list definitions first (just an example)

That sounds harder because at the moment our "reference" items don't 
have indication of which type they are. So if we wanted this to work 
well, we'd have to mandate that additional information, and we don't 
have any good implementation options for either etags or elisp backend 
that would do that.

>> I think I would generally prefer for the "find declarations" binding to end with an error when unsupported rather than repeat an existing command which is already accessible through an easier key binding.
>>
>> Otherwise some users might waste time wondering over the difference, testing, and comparing -- I myself would, probably.
>>
> 
> Before binding to a key maybe let's start again with the  simpler: to add them in the right click context menu... And we can find a binding later.

We can do both.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Fri, Mar 10, 2023 at 9:55 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 10/03/2023 02:57, Ergus wrote:
> > Ho Dmitry and Joao
> >
> > On March 6, 2023 3:09:40 PM GMT+01:00, Dmitry Gutov <dgutov@yandex.ru> wrote:
> >> On 06/03/2023 01:40, John Yates wrote:
> >>> For languages that do not distinguish declaration from definition,
> >>> another alternative is to have both key bindings reference the same
> >>> find-definition functionality.
> >
> > Yes please, just fall back to find-definition (or find-references) + a warning when find-declaration or the the others are not supported? It is simpler and practical.
>
> Could you explain this usage scenario? Why not just report "unsupported"
> and let the user try again with the simpler key binding (M-.)?
>
> It's not like people will prefer to reach for C-M-?, or will hit it by
> accident.

I haven't been following this discussion very closely for the last
few days, but last I followed it, we were on the M-x xref-find-extra
idea of Felicián. There -- at least as far as i understood -
Dmitry's idea was to ask the xref backend what kind of syntactic
constructs  it supports finding, say via some xref-extra-constructs
generic function.

Eglot would have an answer for that, and so could other backends
like etags, SLY,  and any other backend presumably.  Xref could
then keep track of who supports what and the M-x xref-find-extra
would ask those backends for those things that they support.

I think this would be a pretty good idea to avoid the concept
creep problem and still provide a nice user interface.

So, unless we are talking about the case were absolutely 0
"extras" are supported in M-x xref-find-extra. I don't get the
fallback idea nor the "unsupported" message idea.  The problem
would simply not exist.

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 11/03/2023 00:18, João Távora wrote:
> So, unless we are talking about the case were absolutely 0
> "extras" are supported in M-x xref-find-extra

I think we are.

Re: Adding support for xref jumping to headers/interfaces

[Ergus]

On March 10, 2023 10:55:24 PM GMT+01:00, Dmitry Gutov <dgutov@yandex.ru> wrote:
>On 10/03/2023 02:57, Ergus wrote:
>> Ho Dmitry and Joao
>> 
>> On March 6, 2023 3:09:40 PM GMT+01:00, Dmitry Gutov <dgutov@yandex.ru> wrote:
>>> On 06/03/2023 01:40, John Yates wrote:
>>>> For languages that do not distinguish declaration from definition,
>>>> another alternative is to have both key bindings reference the same
>>>> find-definition functionality.
>> 
>> Yes please, just fall back to find-definition (or find-references) + a warning when find-declaration or the the others are not supported? It is simpler and practical.
>
>Could you explain this usage scenario? Why not just report "unsupported" and let the user try again with the simpler key binding (M-.)?
>
>It's not like people will prefer to reach for C-M-?, or will hit it by accident.
>

It is mostly conceptual. In the languages without declarations generally the declaration IS the definition itself.


>> Alternatively there could be a default fallback command similar to find-references but reordering to list definitions first (just an example)
>
>That sounds harder because at the moment our "reference" items don't have indication of which type they are. So if we wanted this to work well, we'd have to mandate that additional information, and we don't have any good implementation options for either etags or elisp backend that would do that.

I understand... I was thinking in a dumb check of coincidences between references and definitions list. Implying to call both and a side to side comparison ... But it doesn't worth it probably....


>
>>> I think I would generally prefer for the "find declarations" binding to end with an error when unsupported rather than repeat an existing command which is already accessible through an easier key binding.
>>> 
>>> Otherwise some users might waste time wondering over the difference, testing, and comparing -- I myself would, probably.
>>> 
>> 
>> Before binding to a key maybe let's start again with the  simpler: to add them in the right click context menu... And we can find a binding later.
>
>We can do both.

I am very excited to see this feature working...

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

Dmitry Gutov <dgutov@yandex.ru> writes:

> On 11/03/2023 00:18, João Távora wrote:
>> So, unless we are talking about the case were absolutely 0
>> "extras" are supported in M-x xref-find-extra
>
> I think we are.

Ah, great.  In that case, sth like `(user-error "unsupported")` is fine,
for interactive calls to xref-find-extra.  For lisp calls you can make
the function return nil instead, so users like Ergus can code up their
desired fallback behaviour in a new command.

João

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

An option I didn't mention originally: Perhaps xref-find-definitions
could just show both implementation and interface.  That would remove
the need for any new keybindings.  The UI might be worse but perhaps
it could be improved.

This is inspired by this comment:
https://github.com/joaotavora/eglot/issues/302#issuecomment-534202561

The relevant excerpt:
> By the way, on a tangent, I think it's a mistake on LSP's part to
> import C++/Java's ecosystem of possible symbol loci. In Common Lisp,
> there are many different types of construct associated with a symbol
> and they are all "definitions". A good IDE like SLIME will use
> something akin to xref-find-definitions (indeed xref is modelled after
> SLIME) and categorize them properly in the result buffer if there is
> more than one. So the command should have been textDocument/definition
> with some kind of subtype parameter, or at least this is where xref
> should evolve to, i.e. xref should keep the single command
> xref-find-definitions and maybe augment it with some "definition-type"
> parameter.

Re: Adding support for xref jumping to headers/interfaces

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Date: Tue, 16 May 2023 17:10:40 -0400
> Cc: Eli Zaretskii <eliz@gnu.org>, stephen_leake@stephe-leake.org, john@yates-sheets.org, 
>  	rms@gnu.org, fgunbin@fastmail.fm, casouri@gmail.com, emacs-devel@gnu.org, 
>  	azeng@janestreet.com
> 
> An option I didn't mention originally: Perhaps xref-find-definitions
> could just show both implementation and interface.  That would remove
> the need for any new keybindings.  The UI might be worse but perhaps
> it could be improved.
> 
> This is inspired by this comment:
> https://github.com/joaotavora/eglot/issues/302#issuecomment-534202561
> 
> The relevant excerpt:
> > By the way, on a tangent, I think it's a mistake on LSP's part to
> > import C++/Java's ecosystem of possible symbol loci. In Common Lisp,
> > there are many different types of construct associated with a symbol
> > and they are all "definitions". A good IDE like SLIME will use
> > something akin to xref-find-definitions (indeed xref is modelled after
> > SLIME) and categorize them properly in the result buffer if there is
> > more than one. So the command should have been textDocument/definition
> > with some kind of subtype parameter, or at least this is where xref
> > should evolve to, i.e. xref should keep the single command
> > xref-find-definitions and maybe augment it with some "definition-type"
> > parameter.

IME, in at least some OOP environments, definition and interface are
considered to be very different things.  It is possible that CL is not
one of them (after all, the OOP model of Lisp is quite different from
that of, say, C++ and Java).

So I'm not sure lumping these two together will be welcome by
everyone.  Especially since you seem to suggest that the format of the
*XREF* buffer also be changed to include the indication of which is
which.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 17/05/2023 00:10, Spencer Baugh wrote:
> An option I didn't mention originally: Perhaps xref-find-definitions
> could just show both implementation and interface.  That would remove
> the need for any new keybindings.  The UI might be worse but perhaps
> it could be improved.
> 
> This is inspired by this comment:
> https://github.com/joaotavora/eglot/issues/302#issuecomment-534202561
> 
> The relevant excerpt:
>> By the way, on a tangent, I think it's a mistake on LSP's part to
>> import C++/Java's ecosystem of possible symbol loci. In Common Lisp,
>> there are many different types of construct associated with a symbol
>> and they are all "definitions". A good IDE like SLIME will use
>> something akin to xref-find-definitions (indeed xref is modelled after
>> SLIME) and categorize them properly in the result buffer if there is
>> more than one. So the command should have been textDocument/definition
>> with some kind of subtype parameter, or at least this is where xref
>> should evolve to, i.e. xref should keep the single command
>> xref-find-definitions and maybe augment it with some "definition-type"
>> parameter.

It is of course the prerogative of the backend to choose which locations 
to return as the list of definitions. Some backends/languages might as 
well include interfaces in the list.

One problem with that, though, is that we're doing some things 
differently from SLIME. In particular, we try to make it easy to jump to 
a specific definition as quickly as possible. If there is just one 
definition found with a given name, we don't even prompt to choose. We 
also have an option for xref-show-definitions-function like 
xref-show-definitions-completing-read which also tries to make things 
faster using completing-read. If other kinds of locations are added to 
the list, the odds of getting duplicates get higher, slowing down the 
interaction. So it's probably a good idea to hide some location kinds 
behind prefix argument, or a separate binding.

That brings me to another question, though. Can we assume that all 
"extra" searches should be of the "definition" type? Meaning that the 
results display should go through xref-show-definitions-function and not 
through xref-show-xrefs-function. If yes (and the ones that have been 
brought up in this discussion -- "implementations", "declarations", 
"type definitions" -- seem to fit that pattern), then good.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Sat, Jun 17, 2023 at 2:54 AM Dmitry Gutov <dgutov@yandex.ru> wrote:

> It is of course the prerogative of the backend to choose which locations
> to return as the list of definitions. Some backends/languages might as
> well include interfaces in the list.
>
> One problem with that, though, is that we're doing some things
> differently from SLIME. In particular, we try to make it easy to jump to
> a specific definition as quickly as possible.

I see no problem with that, but it might be interesting to keep the
backend plug-in mechanisms generic enough to eventually enable
different types of UI.

> If there is just one
> definition found with a given name, we don't even prompt to choose. We
> also have an option for xref-show-definitions-function like
> xref-show-definitions-completing-read which also tries to make things
> faster using completing-read. If other kinds of locations are added to
> the list, the odds of getting duplicates get higher, slowing down the
> interaction. So it's probably a good idea to hide some location kinds
> behind prefix argument, or a separate binding.
>
> That brings me to another question, though. Can we assume that all
> "extra" searches should be of the "definition" type? Meaning that the
> results display should go through xref-show-definitions-function and not
> through xref-show-xrefs-function. If yes (and the ones that have been
> brought up in this discussion -- "implementations", "declarations",
> "type definitions" -- seem to fit that pattern), then good.

If I remember correctly, SLIME (and SLY) have at least two types of
"reference" searches: "who calls" and "who expands".  They create the
typical ((file -> list of matches)...) listing in an SLIME xref buffer.

In connection to the earlier comment, SLIME uses M-? to
slime-edit-uses, which is similar, but not equivalent to our
xref-find-references.  Here, if I remember correctly, the xref
buffer is built like this:

who-calls
   file-less-function-that-calls-1
   file-less-function-that-calls-2
other-type-of-reference
   file-less-function-that-expands-3
   file-less-function-that-expands-4

of course who-calls and who-expands never share an xref buffer.

João

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

João Távora <joaotavora@gmail.com> writes:
> On Sat, Jun 17, 2023 at 2:54 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
>> It is of course the prerogative of the backend to choose which locations
>> to return as the list of definitions. Some backends/languages might as
>> well include interfaces in the list.
>>
>> One problem with that, though, is that we're doing some things
>> differently from SLIME. In particular, we try to make it easy to jump to
>> a specific definition as quickly as possible.
>
> I see no problem with that, but it might be interesting to keep the
> backend plug-in mechanisms generic enough to eventually enable
> different types of UI.

Do you envision that eglot-find-typeDefinition would also be covered by
xref-find-definitions in this way?  The type defintion feels a bit
different from others.

In any case, it seems to me that C-u M-. would be a good way to say
"return all different kinds of definitions" to xref-find-definitions.
Then the user could just switch to the one they want in the *xref*
buffer.  This keeps the basic M-. operation fast, while making it easy
to jump to other kinds of definitions.  Seems perfect.

(Well, literally C-u M-. won't work, because prefix argument for
xref-find-definitions already is used to cause it to prompt for an
identifier.  But something close to that, perhaps.  Maybe this can be
C-M-?)

Re: Adding support for xref jumping to headers/interfaces

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Date: Thu, 22 Jun 2023 15:22:25 -0400
> 
> João Távora <joaotavora@gmail.com> writes:
> > On Sat, Jun 17, 2023 at 2:54 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
> >> It is of course the prerogative of the backend to choose which locations
> >> to return as the list of definitions. Some backends/languages might as
> >> well include interfaces in the list.
> >>
> >> One problem with that, though, is that we're doing some things
> >> differently from SLIME. In particular, we try to make it easy to jump to
> >> a specific definition as quickly as possible.
> >
> > I see no problem with that, but it might be interesting to keep the
> > backend plug-in mechanisms generic enough to eventually enable
> > different types of UI.
> 
> Do you envision that eglot-find-typeDefinition would also be covered by
> xref-find-definitions in this way?  The type defintion feels a bit
> different from others.
> 
> In any case, it seems to me that C-u M-. would be a good way to say
> "return all different kinds of definitions" to xref-find-definitions.
> Then the user could just switch to the one they want in the *xref*
> buffer.  This keeps the basic M-. operation fast, while making it easy
> to jump to other kinds of definitions.  Seems perfect.

Why does this need a separate key sequence?  M-. already knows how to
cope with more than a single candidate returned by the backend.  How
is this case different?

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

On Fri, Jun 23, 2023 at 1:52 AM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > From: Spencer Baugh <sbaugh@janestreet.com>
> > Date: Thu, 22 Jun 2023 15:22:25 -0400
> >
> > João Távora <joaotavora@gmail.com> writes:
> > > On Sat, Jun 17, 2023 at 2:54 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
> > >> It is of course the prerogative of the backend to choose which locations
> > >> to return as the list of definitions. Some backends/languages might as
> > >> well include interfaces in the list.
> > >>
> > >> One problem with that, though, is that we're doing some things
> > >> differently from SLIME. In particular, we try to make it easy to jump to
> > >> a specific definition as quickly as possible.
> > >
> > > I see no problem with that, but it might be interesting to keep the
> > > backend plug-in mechanisms generic enough to eventually enable
> > > different types of UI.
> >
> > Do you envision that eglot-find-typeDefinition would also be covered by
> > xref-find-definitions in this way?  The type defintion feels a bit
> > different from others.
> >
> > In any case, it seems to me that C-u M-. would be a good way to say
> > "return all different kinds of definitions" to xref-find-definitions.
> > Then the user could just switch to the one they want in the *xref*
> > buffer.  This keeps the basic M-. operation fast, while making it easy
> > to jump to other kinds of definitions.  Seems perfect.
>
> Why does this need a separate key sequence?  M-. already knows how to
> cope with more than a single candidate returned by the backend.  How
> is this case different?

Merely that M-. currently usually returns only one candidate, and when
it returns multiple candidates it pops up a buffer, which is annoying
if you only ever want one of the candidates. Most of the time, for
most users, a user will only want the single candidate which is
currently returned by M-. (which is the implementation/definition of
the identifier), so we should avoid adding multiple candidates by
default.

Re: Adding support for xref jumping to headers/interfaces

[Eli Zaretskii]

> From: Spencer Baugh <sbaugh@janestreet.com>
> Date: Fri, 23 Jun 2023 10:37:30 -0400
> Cc: emacs-devel@gnu.org
> 
> On Fri, Jun 23, 2023 at 1:52 AM Eli Zaretskii <eliz@gnu.org> wrote:
> >
> > > From: Spencer Baugh <sbaugh@janestreet.com>
> > > Date: Thu, 22 Jun 2023 15:22:25 -0400
> > >
> > > João Távora <joaotavora@gmail.com> writes:
> > > > On Sat, Jun 17, 2023 at 2:54 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
> > > >> It is of course the prerogative of the backend to choose which locations
> > > >> to return as the list of definitions. Some backends/languages might as
> > > >> well include interfaces in the list.
> > > >>
> > > >> One problem with that, though, is that we're doing some things
> > > >> differently from SLIME. In particular, we try to make it easy to jump to
> > > >> a specific definition as quickly as possible.
> > > >
> > > > I see no problem with that, but it might be interesting to keep the
> > > > backend plug-in mechanisms generic enough to eventually enable
> > > > different types of UI.
> > >
> > > Do you envision that eglot-find-typeDefinition would also be covered by
> > > xref-find-definitions in this way?  The type defintion feels a bit
> > > different from others.
> > >
> > > In any case, it seems to me that C-u M-. would be a good way to say
> > > "return all different kinds of definitions" to xref-find-definitions.
> > > Then the user could just switch to the one they want in the *xref*
> > > buffer.  This keeps the basic M-. operation fast, while making it easy
> > > to jump to other kinds of definitions.  Seems perfect.
> >
> > Why does this need a separate key sequence?  M-. already knows how to
> > cope with more than a single candidate returned by the backend.  How
> > is this case different?
> 
> Merely that M-. currently usually returns only one candidate, and when
> it returns multiple candidates it pops up a buffer, which is annoying
> if you only ever want one of the candidates.

Isn't that what you wanted when you said "return all different kinds
of definitions"?

M-. shows a buffer with several candidates when it cannot find only
one that fits the name you typed.  That is rare, but it does happen.
AFAIU, you have described one situation where this will happen.  So I
think the behavior of M-. is exactly right in that case, and we don't
need any new key sequences or commands.  Or what am I missing?

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Fri, Jun 23, 2023 at 6:18 AM Spencer Baugh <sbaugh@janestreet.com> wrote:
>
> João Távora <joaotavora@gmail.com> writes:
> > On Sat, Jun 17, 2023 at 2:54 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
> >> It is of course the prerogative of the backend to choose which locations
> >> to return as the list of definitions. Some backends/languages might as
> >> well include interfaces in the list.
> >>
> >> One problem with that, though, is that we're doing some things
> >> differently from SLIME. In particular, we try to make it easy to jump to
> >> a specific definition as quickly as possible.
> >
> > I see no problem with that, but it might be interesting to keep the
> > backend plug-in mechanisms generic enough to eventually enable
> > different types of UI.
>
> Do you envision that eglot-find-typeDefinition would also be covered by
> xref-find-definitions in this way?  The type defintion feels a bit
> different from others.


Depends on "the way" but yes, eglot-find-typeDefinition shouldn't exist.  Two
possibilities at least and we don't have to implement them all at once.

1. "type" should be one of the "extras" for the new xref-find-extra command.
   This is what I think is the current thinking here.

2. if we're willing to add another level of hierarchy to the current *xref*
   buffer and segregate results by file and category instead of just by file
   as is done currently.  Or make this configurable so the user can
   segregate just by category.  Or present the category as a tag.  Many
   ways to skin this cat.

> In any case, it seems to me that C-u M-. would be a good way to say
> "return all different kinds of definitions" to xref-find-definitions.
> Then the user could just switch to the one they want in the *xref*
> buffer.  This keeps the basic M-. operation fast, while making it easy
> to jump to other kinds of definitions.  Seems perfect.

That is not the SLIME way and C-u M-. already does something else.  But
I get your point.

> (Well, literally C-u M-. won't work, because prefix argument for
> xref-find-definitions already is used to cause it to prompt for an
> identifier.

Yup exactly.  SOmetimes this is solved by adding two C-u.  Sometimes
it makes sense.  Not sure here.


> But something close to that, perhaps.  Maybe this can be
> C-M-?)

That is xref-find-references.  Which curiously, in SLIME/SLY is "find
uses" and segregates results by category, hiding the file.

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

João Távora <joaotavora@gmail.com> writes:
>> But something close to that, perhaps.  Maybe this can be
>> C-M-?)
>
> That is xref-find-references.  Which curiously, in SLIME/SLY is "find
> uses" and segregates results by category, hiding the file.

FYI M-? is xref-find-references, C-M-? is not bound.  (But somewhat
awkward to type)

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 20/06/2023 18:31, João Távora wrote:
> If I remember correctly, SLIME (and SLY) have at least two types of
> "reference" searches: "who calls" and "who expands".  They create the
> typical ((file -> list of matches)...) listing in an SLIME xref buffer.

Interesting. And AFAICS SLIME also has another kind of non-definitions 
search: slime-who-specializes.

Speaking of the two former ones, though: is there a point in using two 
different commands, if their outputs never "intersect"? That is, one is 
for the macros, and another for regular functions. Why wouldn't 
'slime-who-calls' also work on macros, printing their expansion sites?

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Sat, Nov 4, 2023 at 10:00 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 20/06/2023 18:31, João Távora wrote:
> > If I remember correctly, SLIME (and SLY) have at least two types of
> > "reference" searches: "who calls" and "who expands".  They create the
> > typical ((file -> list of matches)...) listing in an SLIME xref buffer.
>
> Interesting. And AFAICS SLIME also has another kind of non-definitions
> search: slime-who-specializes.
>
> Speaking of the two former ones, though: is there a point in using two
> different commands, if their outputs never "intersect"? That is, one is
> for the macros, and another for regular functions. Why wouldn't
> 'slime-who-calls' also work on macros, printing their expansion sites?

I think maybe just to limit the signal to noise ratio.  Maybe I'm
specifically interested in macro expansions only.  But I kind of agree
that, at least  in Lisp, calling and expanding isn't usually very different
in terms of what I want to know about such a site.

Anyway, does this mean that xref.el is soon to have a solution for
this?  The one I vaguely remember discussing in this thread.  Something
like a single xref-find-other command + a selector for the type of
reference, right?

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 05/11/2023 00:24, João Távora wrote:
> On Sat, Nov 4, 2023 at 10:00 PM Dmitry Gutov<dgutov@yandex.ru>  wrote:
>> On 20/06/2023 18:31, João Távora wrote:
>>> If I remember correctly, SLIME (and SLY) have at least two types of
>>> "reference" searches: "who calls" and "who expands".  They create the
>>> typical ((file -> list of matches)...) listing in an SLIME xref buffer.
>> Interesting. And AFAICS SLIME also has another kind of non-definitions
>> search: slime-who-specializes.
>>
>> Speaking of the two former ones, though: is there a point in using two
>> different commands, if their outputs never "intersect"? That is, one is
>> for the macros, and another for regular functions. Why wouldn't
>> 'slime-who-calls' also work on macros, printing their expansion sites?
> I think maybe just to limit the signal to noise ratio.  Maybe I'm
> specifically interested in macro expansions only.  But I kind of agree
> that, at least  in Lisp, calling and expanding isn't usually very different
> in terms of what I want to know about such a site.

But if who-calls only works for functions and who-expands only works for 
macros, only one of them would work for a given symbol, right? The other 
would return an empty list.

> Anyway, does this mean that xref.el is soon to have a solution for
> this?  The one I vaguely remember discussing in this thread.  Something
> like a single xref-find-other command + a selector for the type of
> reference, right?

Maybe it will, maybe actually not. See one of my next few emails with a 
patch to try out (with the things you mentioned) and some further 
questions to ponder.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Sat, Nov 4, 2023 at 10:29 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 05/11/2023 00:24, João Távora wrote:
> > On Sat, Nov 4, 2023 at 10:00 PM Dmitry Gutov<dgutov@yandex.ru>  wrote:
> >> On 20/06/2023 18:31, João Távora wrote:
> >>> If I remember correctly, SLIME (and SLY) have at least two types of
> >>> "reference" searches: "who calls" and "who expands".  They create the
> >>> typical ((file -> list of matches)...) listing in an SLIME xref buffer.
> >> Interesting. And AFAICS SLIME also has another kind of non-definitions
> >> search: slime-who-specializes.
> >>
> >> Speaking of the two former ones, though: is there a point in using two
> >> different commands, if their outputs never "intersect"? That is, one is
> >> for the macros, and another for regular functions. Why wouldn't
> >> 'slime-who-calls' also work on macros, printing their expansion sites?
> > I think maybe just to limit the signal to noise ratio.  Maybe I'm
> > specifically interested in macro expansions only.  But I kind of agree
> > that, at least  in Lisp, calling and expanding isn't usually very different
> > in terms of what I want to know about such a site.
>
> But if who-calls only works for functions and who-expands only works for
> macros, only one of them would work for a given symbol, right? The other
> would return an empty list.

Yeah, you're right, I guess.  IDK, SLIME has always worked like that.

> > Anyway, does this mean that xref.el is soon to have a solution for
> > this?  The one I vaguely remember discussing in this thread.  Something
> > like a single xref-find-other command + a selector for the type of
> > reference, right?
>
> Maybe it will, maybe actually not. See one of my next few emails with a
> patch to try out (with the things you mentioned) and some further
> questions to ponder.

Will the patch have changes to Eglot as well?  Hoping it does, but if it
doesn't leave some instructions on how to do that.  That's mostly what
I'm interested in, i.e.  how easy it is to plug into.  The interface
is easier to tweak once we have concrete use cases to work with.

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 23/06/2023 18:03, João Távora wrote:
> 2. if we're willing to add another level of hierarchy to the current*xref*
>     buffer and segregate results by file and category instead of just by file
>     as is done currently.  Or make this configurable so the user can
>     segregate just by category.  Or present the category as a tag.  Many
>     ways to skin this cat.

Adding multiple levels of grouping is definitely an option. I'm not sure 
it's ideal, though (usability issues like +1 extra level of indentation, 
and if there are enough elements in a group, you scroll down and don't 
see the group's heading anymore unless you make it "sticky").

Note that actually Elisp's find-definitions output is already 
"annotated" with categories: the first symbol in the summary 
(defun/defvar/defface/cl-defmethod) unambiguously denotes the definition 
type. I wonder how often that is the case with the other backends.

Maybe Xref backends for C/C++ (etags/clang/lsp) just show the line where 
the definition resides, which depending on the style might or might not 
include the beginning of the definition. I think for those langs, the 
main cases where there would be multiples definitions, is in the case of 
inheritance (also in Java), and it would either should the definition in 
a superclass, or definitions in subclasses with overrides. In Java, at 
least, those would be usually disambiguated by the file name 
(conventionally, they correspond to class names more or less 1-to-1).

All that aside, a "sufficiently smart" backends for any of those 
languages would be able to render the Xref summary string in any fashion 
they want (for definitions, the summary doesn't need to copy any 
physical line). E.g. prepending "@override " for overrides. Not sure how 
realistic that is for LSP, but something more rough could likely be 
done. E.g. specific words attenuated with color prepended to the summary 
for the "special" definitions. [type] <typeDefinition Xref's summary>. 
Something to try out, anyway, if

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

[-- Attachment #1: Type: text/plain, Size: 3564 bytes --]

On 17/05/2023 00:10, Spencer Baugh wrote:
> An option I didn't mention originally: Perhaps xref-find-definitions
> could just show both implementation and interface.  That would remove
> the need for any new keybindings.  The UI might be worse but perhaps
> it could be improved.
> 
> This is inspired by this comment:
> https://github.com/joaotavora/eglot/issues/302#issuecomment-534202561
> 
> The relevant excerpt:
>> By the way, on a tangent, I think it's a mistake on LSP's part to
>> import C++/Java's ecosystem of possible symbol loci. In Common Lisp,
>> there are many different types of construct associated with a symbol
>> and they are all "definitions". A good IDE like SLIME will use
>> something akin to xref-find-definitions (indeed xref is modelled after
>> SLIME) and categorize them properly in the result buffer if there is
>> more than one. So the command should have been textDocument/definition
>> with some kind of subtype parameter, or at least this is where xref
>> should evolve to, i.e. xref should keep the single command
>> xref-find-definitions and maybe augment it with some "definition-type"
>> parameter.

Perhaps it /should/ (xref-find-definitions to include both 
implementation and interface). And include the typeDefinition entries 
for langs that have those. Etc.

Would the UI be worse? Let's try to find out.

Attached is the patch along the lines that we discussed: a new command 
xref-find-extra currently bound to M-' just for the sake of this 
experiment (eliminating the need to set up define-key to try out the 
"fast" workflow). It uses the symbol at point or asks for it, then polls 
the backend for available kinds, does completing-read and asks the 
backend for definitions of that kind. And shows the list or jumps to the 
unique one.

To have something to test, I implemented this for Elisp. However, all 
known kinds are already printed by the regular xref-find-definitions 
output in that backend. So the result turned out interesting, but a 
little impractical: with Elisp, we already make the choice between the 
kinds in the Xref output buffer, when that buffer is shown at all (as 
Eli pointed out, actually). So... we could proceed in that direction and 
recommend that all kinds of definitions would be added there.

But let's try to answer the question: which workflows would the attached 
approach work better for? I could give one example: in Java, you 
sometimes (often?) want to find the locations where the current 
definition near point is overridden by subclasses. We could call it 
"find overrides", for example. But it does not just filter by kind (e.g. 
method overrides), it also filters by class hierarchy, i.e. showing only 
definitions in the subclasses of the current class, or interface.

Examples of such searches:
 
https://stackoverflow.com/questions/11849631/how-to-find-the-overridden-method-in-eclipse
   https://stackoverflow.com/questions/1143267/finding-overriding-methods

Is this still something people care about? Does jdt-ls even support this?

Or perhaps the main value would be in actually having separate commands 
which could be bound to a submap with faster key sequences and/or to the 
context menu (with context-menu-mode on). Then our solution should be 
more in line with either defining a number of additional named commands 
(for mode universal kinds) and/or making it easier to define new such 
commands for users/3rd-party packages/etc.

Please try out the attached (with implementation for Eglot hopefully 
coming soon) and let me know your thoughts (you all).

[-- Attachment #2: xref-find-extras.diff --]
[-- Type: text/x-patch, Size: 8315 bytes --]

diff --git a/lisp/progmodes/elisp-mode.el b/lisp/progmodes/elisp-mode.el
index ff90a744ea3..ead8b618599 100644
--- a/lisp/progmodes/elisp-mode.el
+++ b/lisp/progmodes/elisp-mode.el
@@ -1037,6 +1037,26 @@ xref-backend-definitions
                 (elisp--xref-filter-definitions defs 'any sym))
           (elisp--xref-filter-definitions defs namespace sym))))))
 
+(cl-defmethod xref-backend-extra-kinds ((_backend (eql 'elisp)) identifier)
+  (require 'find-func)
+  (let ((sym (intern-soft identifier)))
+    (when sym
+      (let* ((pos (get-text-property 0 'pos identifier))
+             (namespace (if pos
+                            (elisp--xref-infer-namespace pos)
+                          'any))
+             (defs (elisp--xref-find-definitions sym)))
+        (pcase-exhaustive namespace
+          ('function '(function defalias define-type
+                       cl-defgeneric cl-defmethod))
+          ('variable '(defvar))
+          ('face '(defface))
+          ('feature '(feature)))
+        (if (eq namespace 'maybe-variable)
+            (or (elisp--xref-filter-definitions defs 'variable sym)
+                (elisp--xref-filter-definitions defs 'any sym))
+          (elisp--xref-filter-definitions defs namespace sym))))))
+
 (defun elisp--xref-filter-definitions (definitions namespace symbol)
   (if (eq namespace 'any)
       (if (memq symbol minor-mode-list)
@@ -1212,6 +1232,66 @@ elisp--xref-find-definitions
 
     xrefs))
 
+(cl-defmethod xref-backend-extra-kinds ((_backend (eql 'elisp)) identifier)
+  ;; The file name is not known when `symbol' is defined via interactive eval.
+  (let ((symbol (intern-soft identifier))
+        kinds)
+    ;; alphabetical by result type symbol
+
+    ;; FIXME: advised function; list of advice functions
+    ;; FIXME: aliased variable
+
+    ;; Coding system symbols do not appear in ‘load-history’,
+    ;; so we can’t get a location for them.
+    (when (and (symbolp symbol)
+               (symbol-function symbol)
+               (symbolp (symbol-function symbol)))
+      (push "defalias" kinds))
+
+    (when (facep symbol)
+      (push "face" kinds))
+
+    (when (fboundp symbol)
+      (let ((file (find-lisp-object-file-name symbol (symbol-function symbol)))
+            doc)
+        (when file
+          (cond
+           ((eq file 'C-source)
+            (push "function" kinds))
+           ((and (setq doc (documentation symbol t))
+                 ;; This doc string is defined in cl-macs.el cl-defstruct
+                 (string-match "Constructor for objects of type `\\(.*\\)'" doc))
+            (push "constructor" kinds))
+           ((cl--generic symbol)
+            (push "generic" kinds))
+           (t
+            (push "function" kinds))))))
+    (when (boundp symbol)
+      (push "variable" kinds))
+    (when (featurep symbol)
+      (push "feature" kinds))
+    (nreverse kinds)))
+
+(cl-defmethod xref-backend-extra-defs ((_backend (eql 'elisp)) identifier kind)
+  (require 'find-func)
+  (let ((sym (intern-soft identifier)))
+    (when sym
+      (let* ((defs (elisp--xref-find-definitions sym))
+             (expected-kind
+              (assoc-default kind
+                             '(("defalias" . defalias)
+                               ("face" . defface)
+                               ("function" . nil)
+                               ("variable" . defvar)
+                               ("constructor" . define-type)
+                               ("generic" . generic)))))
+        (cl-loop for d in defs
+                 for def-kind = (xref-elisp-location-type (xref-item-location d))
+                 when (if (eq expected-kind 'generic)
+                          (memq def-kind '(cl-defgeneric cl-defmethod))
+                        (eq def-kind expected-kind))
+                 collect d)))))
+
 (declare-function xref-apropos-regexp "xref" (pattern))
 
 (cl-defmethod xref-backend-apropos ((_backend (eql 'elisp)) pattern)
diff --git a/lisp/progmodes/xref.el b/lisp/progmodes/xref.el
index 81618428bf3..e1e3862256c 100644
--- a/lisp/progmodes/xref.el
+++ b/lisp/progmodes/xref.el
@@ -314,6 +314,17 @@ xref-backend-identifier-completion-ignore-case
   "Return t if case is not significant in identifier completion."
   completion-ignore-case)
 
+(cl-defgeneric xref-backend-extra-kinds (_backend _identifier)
+  "Return the other definition types BACKEND could show for IDENTIFIER."
+  (user-error "Extra definitions not supported by the backend"))
+
+(cl-defgeneric xref-backend-extra-defs (_backend _identifier _kind)
+  "Find definitions of extra KIND for IDENTIFIER.
+
+The result must be a list of xref objects.  Refer to
+`xref-backend-definitions' for other details."
+  nil)
+
 \f
 ;;; misc utilities
 (defun xref--alistify (list key)
@@ -364,7 +375,8 @@ xref-marker-ring-length
 
 (defcustom xref-prompt-for-identifier '(not xref-find-definitions
                                             xref-find-definitions-other-window
-                                            xref-find-definitions-other-frame)
+                                            xref-find-definitions-other-frame
+                                            xref-find-extra)
   "If non-nil, prompt for the identifier to find.
 
 When t, always prompt for the identifier name.
@@ -1569,11 +1581,11 @@ xref--find-definitions
    (xref--create-fetcher id 'definitions id)
    display-action))
 
-(defun xref--create-fetcher (input kind arg)
+(defun xref--create-fetcher (input kind &rest args)
   "Return an xref list fetcher function.
 
 It revisits the saved position and delegates the finding logic to
-the xref backend method indicated by KIND and passes ARG to it."
+the xref backend method indicated by KIND and passes ARGS to it."
   (let* ((orig-buffer (current-buffer))
          (orig-position (point))
          (backend (xref-find-backend))
@@ -1589,7 +1601,7 @@ xref--create-fetcher
         (when (buffer-live-p orig-buffer)
           (set-buffer orig-buffer)
           (ignore-errors (goto-char orig-position)))
-        (let ((xrefs (funcall method backend arg)))
+        (let ((xrefs (apply method backend args)))
           (unless xrefs
             (xref--not-found-error kind input))
           xrefs)))))
@@ -1624,6 +1636,35 @@ xref-find-definitions-other-frame
   (interactive (list (xref--read-identifier "Find definitions of: ")))
   (xref--find-definitions identifier 'frame))
 
+;;;###autoload
+(defun xref-find-extra (identifier)
+  "Find some specific kind of definition of the identifier at point.
+With prefix argument or when there's no identifier at point,
+prompt for the identifier.
+
+If only one location is found, display it in the selected window.
+Otherwise, display the list of the possible definitions in a
+buffer where the user can select from the list.
+
+Use \\[xref-go-back] to return back to where you invoked this command."
+  (interactive (list
+                ;; XXX: Choose kind of "extra" first? That would fail
+                ;; to take advantage of the symbol-at-point, though.
+                (xref--read-identifier "Find definitions of: ")))
+  (let* ((kinds (xref-backend-extra-kinds (xref-find-backend) identifier))
+         ;; FIXME: We should probably skip asking when there's just
+         ;; one available kind, but let's keep completing-read while
+         ;; collecting the initial feedback about the interface.
+         (kind ;; (if (cdr kinds)
+          (completing-read "Definition kind: " kinds nil t nil nil (car kinds))
+          ;; (car kinds)
+          ;; )
+          ))
+    (unless kind (user-error "No supported kinds"))
+    (xref--show-defs
+     (xref--create-fetcher identifier 'extra-defs identifier kind)
+     nil)))
+
 ;;;###autoload
 (defun xref-find-references (identifier)
   "Find references to the identifier at point.
@@ -1724,6 +1765,7 @@ xref-apropos-regexp
 ;;;###autoload (define-key esc-map [?\C-,] #'xref-go-forward)
 ;;;###autoload (define-key esc-map "?" #'xref-find-references)
 ;;;###autoload (define-key esc-map [?\C-.] #'xref-find-apropos)
+;;;###autoload (define-key esc-map "'" #'xref-find-extra)
 ;;;###autoload (define-key ctl-x-4-map "." #'xref-find-definitions-other-window)
 ;;;###autoload (define-key ctl-x-5-map "." #'xref-find-definitions-other-frame)
 

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 05/11/2023 00:36, João Távora wrote:

>>>> Speaking of the two former ones, though: is there a point in using two
>>>> different commands, if their outputs never "intersect"? That is, one is
>>>> for the macros, and another for regular functions. Why wouldn't
>>>> 'slime-who-calls' also work on macros, printing their expansion sites?
>>> I think maybe just to limit the signal to noise ratio.  Maybe I'm
>>> specifically interested in macro expansions only.  But I kind of agree
>>> that, at least  in Lisp, calling and expanding isn't usually very different
>>> in terms of what I want to know about such a site.
>>
>> But if who-calls only works for functions and who-expands only works for
>> macros, only one of them would work for a given symbol, right? The other
>> would return an empty list.
> 
> Yeah, you're right, I guess.  IDK, SLIME has always worked like that.

I suppose one benefit would be in narrowed down completion, if the user 
inputs the symbol to search by hand. But that seems very minor.

>>> Anyway, does this mean that xref.el is soon to have a solution for
>>> this?  The one I vaguely remember discussing in this thread.  Something
>>> like a single xref-find-other command + a selector for the type of
>>> reference, right?
>>
>> Maybe it will, maybe actually not. See one of my next few emails with a
>> patch to try out (with the things you mentioned) and some further
>> questions to ponder.
> 
> Will the patch have changes to Eglot as well?  Hoping it does, but if it
> doesn't leave some instructions on how to do that.  That's mostly what
> I'm interested in, i.e.  how easy it is to plug into.  The interface
> is easier to tweak once we have concrete use cases to work with.

Please see xref-find-extras.diff in the other email. Eglot -- or any 
other backend -- would need to provide overrides for 
xref-backend-extra-kinds and xref-backend-extra-defs.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 05/11/2023 01:16, Dmitry Gutov wrote:
> Please try out the attached

Never mind that there are two definitions of xref-backend-extra-kinds 
for elisp-mode inside the patch: the wrong version goes first, and thus 
is unused.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Sat, Nov 4, 2023 at 11:17 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> Please try out the attached (with implementation for Eglot hopefully
> coming soon) and let me know your thoughts (you all).

Does this mean you're going to come up with this implementation or
are you waiting for me to get to it?  I'm fine with either (if you
have the momentum and understand eglot.el's use case I'll be glad to
accept your patch).

João Távora

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 05/11/2023 01:24, João Távora wrote:
> On Sat, Nov 4, 2023 at 11:17 PM Dmitry Gutov<dgutov@yandex.ru>  wrote:
> 
>> Please try out the attached (with implementation for Eglot hopefully
>> coming soon) and let me know your thoughts (you all).
> Does this mean you're going to come up with this implementation or
> are you waiting for me to get to it?  I'm fine with either (if you
> have the momentum and understand eglot.el's use case I'll be glad to
> accept your patch).

If you have the time, please do it. I expect that will increase the 
amount of feedback from your side. Also, you know the codebase better, 
and probably the capabilities of some existing language servers.

Re: Adding support for xref jumping to headers/interfaces

[Eshel Yaron]

Hi Dmitry,

Thanks for working on this feature.  I'm looking forward to using it.

Dmitry Gutov <dgutov@yandex.ru> writes:

> Attached is the patch along the lines that we discussed: a new command
> xref-find-extra currently bound to M-' just for the sake of this
> experiment (eliminating the need to set up define-key to try out the
> "fast" workflow).

That's was indeed convenient for testing the patch, although in the long
run there should probably be a different binding since `M-'` is already
bound to `abbrev-prefix-mark`.

> It uses the symbol at point or asks for it, then
> polls the backend for available kinds,

So far so good.

> does completing-read and asks the backend for definitions of that
> kind. And shows the list or jumps to the unique one.

Personally, I would prefer a `read-multiple-choice` alternative to
`completing-read` here, that would allow me to select the "kind" with a
single key press.  Since there are rarely more than a few kinds to
choose from, in many cases `completing-read` introduces unnecessary
friction.  Ideally, the backend should be able to choose whether to do a
`completing-read` or a "quick" `read-multiple-choice` on a case-by-case
basis.  The backend can also associate stable `read-multiple-choice`
keys to the different kinds it support.  WDYT?

> To have something to test, I implemented this for Elisp. However, all
> known kinds are already printed by the regular xref-find-definitions
> output in that backend. So the result turned out interesting, but a
> little impractical: with Elisp, we already make the choice between the
> kinds in the Xref output buffer, when that buffer is shown at all (as
> Eli pointed out, actually). So... we could proceed in that direction
> and recommend that all kinds of definitions would be added there.

Yes, I've noticed that redundancy of sorts while testing this patch, but
I think that selecting the definition kind explicitly with this new
command still provides a distinct experience compared to the
`xref-find-definitions`.

> But let's try to answer the question: which workflows would the
> attached approach work better for?

My above suggestion to provide "quick kind selection" with something
like `read-multiple-choice` would also be a general advantage here.

> Or perhaps the main value would be in actually having separate
> commands which could be bound to a submap with faster key sequences
> and/or to the context menu (with context-menu-mode on).

That's indeed an added value IMO.  In that spirit, it might be better for
`xref-find-extra` to take the "kind" as an argument, so we can define
`xref-find-foo` as `(apply-partially #'xref-find-extra 'foo)`.
Interactively, `xref-find-extra` would still pick up the identifier first
and then prompt for the kind.

> Please try out the attached (with implementation for Eglot hopefully
> coming soon) and let me know your thoughts (you all).

One issue I had was with selecting "feature" as the definition kind.
Basically it doesn't seem to work for some reason, so for example typing
`M-' isearch RET feature RET` says "No extra-defs found for: isearch".

Also, for minor mode names, we currently suggest both "function" and
"variable" kinds, but they both point to the same `define-minor-mode`.
Ideally, we'd unify the two and simply suggest "minor-mode" as the
definition kind.


Best,

Eshel

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

Hi Eshel,

On 05/11/2023 10:16, Eshel Yaron wrote:

> Thanks for working on this feature.  I'm looking forward to using it.

Thanks for testing! Just to be clear, it's an experiment so far, to try 
to determine the best direction to go with. Whether it will be a new 
command, or several, or augmentation of some existing one(s).

> Dmitry Gutov <dgutov@yandex.ru> writes:
> 
>> Attached is the patch along the lines that we discussed: a new command
>> xref-find-extra currently bound to M-' just for the sake of this
>> experiment (eliminating the need to set up define-key to try out the
>> "fast" workflow).
> 
> That's was indeed convenient for testing the patch, although in the long
> run there should probably be a different binding since `M-'` is already
> bound to `abbrev-prefix-mark`.

Honestly, it's a pretty obscure command, and the binding is nice. 
Anyway, it's not a given we'll have a new command like this, so there's 
no point in discussing bindings too early.

>> It uses the symbol at point or asks for it, then
>> polls the backend for available kinds,
> 
> So far so good.
> 
>> does completing-read and asks the backend for definitions of that
>> kind. And shows the list or jumps to the unique one.
> 
> Personally, I would prefer a `read-multiple-choice` alternative to
> `completing-read` here, that would allow me to select the "kind" with a
> single key press.  Since there are rarely more than a few kinds to
> choose from, in many cases `completing-read` introduces unnecessary
> friction.  Ideally, the backend should be able to choose whether to do a
> `completing-read` or a "quick" `read-multiple-choice` on a case-by-case
> basis.

People override completing-read using modes like 
icomplete-vertical-mode, but there are no popular augmentations for 
read-multiple-choice. Also, sometimes the types will start with the same 
characters, and sometimes not. Just using completing-read at least 
provides a stable UI for all cases: if you remember the type, you can 
input the first part of the name without thinking and press RET.

> The backend can also associate stable `read-multiple-choice`
> keys to the different kinds it support.  WDYT?

Perhaps you are instead thinking of a solution when there is a stable 
keymap with known commands assigned to the same characters.

>> To have something to test, I implemented this for Elisp. However, all
>> known kinds are already printed by the regular xref-find-definitions
>> output in that backend. So the result turned out interesting, but a
>> little impractical: with Elisp, we already make the choice between the
>> kinds in the Xref output buffer, when that buffer is shown at all (as
>> Eli pointed out, actually). So... we could proceed in that direction
>> and recommend that all kinds of definitions would be added there.
> 
> Yes, I've noticed that redundancy of sorts while testing this patch, but
> I think that selecting the definition kind explicitly with this new
> command still provides a distinct experience compared to the
> `xref-find-definitions`.

Yes, but is it better? Does it bring much to the table? For Elisp, my 
own answer is "probably not": all the kinds are available in 
xref-find-definitions, and the selection UI is adequate already (isn't it?)

But maybe it will be more useful for some Eglot languages, specific 
contexts and specific "kinds" to search for?

>> But let's try to answer the question: which workflows would the
>> attached approach work better for?
> 
> My above suggestion to provide "quick kind selection" with something
> like `read-multiple-choice` would also be a general advantage here.
> 
>> Or perhaps the main value would be in actually having separate
>> commands which could be bound to a submap with faster key sequences
>> and/or to the context menu (with context-menu-mode on).
> 
> That's indeed an added value IMO.  In that spirit, it might be better for
> `xref-find-extra` to take the "kind" as an argument, so we can define
> `xref-find-foo` as `(apply-partially #'xref-find-extra 'foo)`.
> Interactively, `xref-find-extra` would still pick up the identifier first
> and then prompt for the kind.

A helper for defining new commands is definitely doable. For that route, 
though, I would like to ask whether we'll want to define any of them 
inside core Xref. And if not, then where would those definitions reside.

>> Please try out the attached (with implementation for Eglot hopefully
>> coming soon) and let me know your thoughts (you all).
> 
> One issue I had was with selecting "feature" as the definition kind.
> Basically it doesn't seem to work for some reason, so for example typing
> `M-' isearch RET feature RET` says "No extra-defs found for: isearch".

That's just a missing mapping inside xref-backend-extra-defs, I didn't 
want to spend too much time polishing the prototype.

> Also, for minor mode names, we currently suggest both "function" and
> "variable" kinds, but they both point to the same `define-minor-mode`.
> Ideally, we'd unify the two and simply suggest "minor-mode" as the
> definition kind.

Also easy enough to do, e.g. by borrowing the solution from 
elisp--xref-filter-definitions.

Still, though. It's not clear to me that improving this particular 
feature for Elisp will bring a lot to the table.

Does it really do certain things better than simply using 'M-.' and then 
choosing between all the available definition types inside the buffer 
(and then navigating to one with TAB)?

Or if one prefers a more dynamic interface with completions, (setq 
xref-xref-show-definitions-function #'show-definitions-completing-read) 
could fit that bill.

There is a tradeoff of not being able to just press a single char, but 
OTOH one doesn't always know which definition kind they want in advance, 
and here, the full list helps one make that choice.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

[Adding back emacs-devel, which I didn't mean to remove from my previous
email.  This is also why I quote your message fully.  ]

On Tue, Nov 7, 2023 at 2:31 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> Hi Joao,
>
> On 06/11/2023 02:22, João Távora wrote:
>
> > Alright, so I added Eglot support for the new xref-find-extra feature of
> > Xref.  I pushed two commits to that branch, feature/xref-find-extra.
>
> I'm happy to see the implementation was short enough. Hopefully that
> will make it easier to experiment, but also potentially pivot to
> similar/different solutions, abandoning most of the code, too.

Sure.

> > It wasn't very difficult, but I did have to scratch my head a bit until
> > I found what I think is the key.  So first I did a reasonably simple
> > commit to xref.el, which gives xref-find-extra a KIND argument, making
> > it more versatile since it can now be used non-interactively.  I think
> > the interactive behavior is completely unchanged.  The docstring might
> > need a tweak or two.
>
> There is an omission in being able to input the identifier (fixable, of
> course).

Can you explain exactly the problem?  I'm happy to fix it.

> Though I'm not sure if that capability will be helpful for
> those searches with an LSP client (with language servers providing
> relatively limited identifier completion).

Indeed, it's not.  For every reference kind _except_ the "definition"
kind, where it does come in handy, since I have a hack for that
too.  Indeed very handy (and very hackish, but that's life).

> I still note, though, that you kept those commands around:
> eglot-find-declaration/implementation/typeDefinition. Whereas previously
> I seem to recall you opined that the commands themselves shouldn't be in
> Eglot.
>
> Would you say it's a backward compatibility concern, or something else
> as well?

The former.  IOW if xref.el had this capability in the past, I would have
used it.

But I still nonetheless think functions should be as versatile as possible,
so that even what we designed as an interactive entry-point can be used
as a library entry point.

> > Anyway, this new xref-find-extra aided greatly in adding Eglot support.
> > It removes the need for a very hackish implementation of the existing
> > 'eglot-find-declaration', 'eglot-find-implementation' and
> > 'eglot-find-typeDefinition' commands.  And of course 'xref-find-extra'
> > as an interactive entry point also works for finding these things.
>
> Whatever solution we decide to go with in the end, it should be
> straightforward enough to create a helper that would assist in defining
> such commands.

Yes.  I do think this one is pretty good at doing that.  But I guess
even with this one, you could offer a 'xref-define-interactive-finder-for-kind'
macro that would automate it even further and help xref.el retain finer
control over how exactly that finder behaves, for example vis-a-vis
prefix arguments.

> > One thing that is missing IMO, but is reasonably easy to fix, is an
> > indirection to allow non-string objects to be used as KIND, but still
> > have these objects be displayed to the user with a pretty string.  Of
> > course, this is not essential -- I could just use strings as KINDs in
> > eglot.el and be done with it -- but I feel it'd be a little nicer.
> > Right now I'm using the symbols 'eglot--xref-implementation',
> > 'eglot--xref-declaration', etc, which are printed as is to the user in
> > the completing-read of 'xref-find-extra'.  Works for now.
>
> Using strings as KINDs is what allows us to use completing-read on them.
> Internally, there could be of course some alist mapping them to complex
> objects. If you have a better idea, I'm open.

Yes, that indirection is what is needed.  It could be done via symbol
properties (in which case KIND would be a symbol or a string), via
a specific prescription that KIND can be (OBJ . DISPLAY), a specific
prescription like (OBJ . DISPLAY-FN-TO-BE-CALLED-ON-OBJ) or just a
generic function whose default implementation is #'identity thus
letting completing read do whatever it does for that OBJ, which in
the case of symbols is convert them to string and return that string).
Very many ways to skin this cat, so just pick one.

But just to be clear: whatever the technique used, that "mapping" you
speak of (which I call "indirection"), could _not_ be "internal".  It
would have to be exposed so the backend can control how the OBJs it
returns from 'xref-backend-extra-kinds' are converted to strings.

> > Another idea I had while doing this is that the name xref-find-extra
> > could be actually something like xref-find-by-kind.  What I mean is that
> > I found myself adding the existing kinds "definition" and "references"
> > to that "extra" set.  I think this is more comfortable for the user that
> > even though they are not "extras".
>
> This is one of the questions I'd like to see answered:
>
> - Should a command like this include existing kinds (that
> xref-find-definitions already lists)?

My answer is yes.  But we could leave that up to the backend.  I
intend to remap M-?  to xref-find-extra myself, losing the speed of
xref-find-references for "all references", but gaining in versatility.
Of course a similar effect could be achieved via some kind of prefix
argument or variable controlling the behaviour of xref-find-references.

Another thing that I think should be changed is the name of the generic
function xref-backend-extra-defs.  It should be IMO xref-backend-extra-refs
because a "definition" is a type of "reference" to a symbol/name.  Likewise
to mentions of "definition" in the new docstrings.

> - Should there be other kinds in it, absent from xref-find-definitions?
> If yes, of kinds of "kinds" (:-)) should be in one set but not the other?

Up to the backend.

> - Should xref-find-definitions include the results of
> eglot-find-declaration and eglot-find-typeDefinition? Does it include
> them already?

That's up to the backend, Eglot in this case.  And the Eglot backend
leaves that up to the LSP server, so it's a question that can't be answered.

> > Finally, I really like the M-' binding.  It'd be awesome if we could
> > somehow make it stick (who even uses 'abbrev-prefix-mark'), but I don't
> > have high hopes for it?
>
> Concur on all three points. I've never even heard of
> 'abbrev-prefix-mark' before, but it might be a difficult discussion anyway.
>
> > Or maybe xref-find-references could lead to xref-find-extra's menu when
> > a certain flag variable is set.  Then the M-? binding could stay.  Of
> > course this makes more sense if the current "references" kind is also
> > part of the choices offered in that menu.
>
> Actually, I imagined that a distinction between "find definition" and
> "find references" will remain. And so "xref-find-references" would not
> be one of the "extras" available, if only because it seems to make sense
> to use xref-show-xrefs-function for one and
> xref-show-definitions-function for the other (sometimes they are minorly
> different; sometimes there values do really different things).
>
> All this is still negotiable, of course.
>
> (Please feel free to reply Cc'ing emacs-devel, these are high-level
> details hopefully deserving public discussion.)

Yes, I forgot to add it in the first mail.

João

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

Dmitry Gutov <dgutov@yandex.ru> writes:
> On 17/05/2023 00:10, Spencer Baugh wrote:
>> An option I didn't mention originally: Perhaps xref-find-definitions
>> could just show both implementation and interface.  That would remove
>> the need for any new keybindings.  The UI might be worse but perhaps
>> it could be improved.
>> This is inspired by this comment:
>> https://github.com/joaotavora/eglot/issues/302#issuecomment-534202561
>> The relevant excerpt:
>>> By the way, on a tangent, I think it's a mistake on LSP's part to
>>> import C++/Java's ecosystem of possible symbol loci. In Common Lisp,
>>> there are many different types of construct associated with a symbol
>>> and they are all "definitions". A good IDE like SLIME will use
>>> something akin to xref-find-definitions (indeed xref is modelled after
>>> SLIME) and categorize them properly in the result buffer if there is
>>> more than one. So the command should have been textDocument/definition
>>> with some kind of subtype parameter, or at least this is where xref
>>> should evolve to, i.e. xref should keep the single command
>>> xref-find-definitions and maybe augment it with some "definition-type"
>>> parameter.
>
> Perhaps it /should/ (xref-find-definitions to include both
> implementation and interface). And include the typeDefinition entries
> for langs that have those. Etc.
>
> Would the UI be worse? Let's try to find out.
>
> Attached is the patch along the lines that we discussed: a new command
> xref-find-extra currently bound to M-' just for the sake of this
> experiment (eliminating the need to set up define-key to try out the
> "fast" workflow). It uses the symbol at point or asks for it, then
> polls the backend for available kinds, does completing-read and asks
> the backend for definitions of that kind. And shows the list or jumps
> to the unique one.
>
> To have something to test, I implemented this for Elisp. However, all
> known kinds are already printed by the regular xref-find-definitions
> output in that backend. So the result turned out interesting, but a
> little impractical: with Elisp, we already make the choice between the
> kinds in the Xref output buffer, when that buffer is shown at all (as
> Eli pointed out, actually). So... we could proceed in that direction
> and recommend that all kinds of definitions would be added there.

This (and Joao's patch adding support for eglot) is very interesting,
thank you for implementing it!

We also discussed a UI which shows all kinds of definitions in a single
buffer.  Maybe the prompt for KIND should default to "all" which shows
all kinds of definitions?

I notice the API doesn't support "fetch all kinds of definitions";
extra-defs requires specifying a specific kind.  Perhaps just specifying
a nil KIND should request all kinds, and extra-defs should tag each
returned object with the kind?  We could also just make a separate call
to extra-defs for each kind returned by extra-kinds, but that would be
allow less space for the backend to batch its operations.

> But let's try to answer the question: which workflows would the
> attached approach work better for? I could give one example: in Java,
> you sometimes (often?) want to find the locations where the current
> definition near point is overridden by subclasses. We could call it
> "find overrides", for example. But it does not just filter by kind
> (e.g. method overrides), it also filters by class hierarchy,
> i.e. showing only definitions in the subclasses of the current class,
> or interface.
>
> Examples of such searches:
>
> https://stackoverflow.com/questions/11849631/how-to-find-the-overridden-method-in-eclipse
>   https://stackoverflow.com/questions/1143267/finding-overriding-methods
>
> Is this still something people care about? Does jdt-ls even support this?
>
> Or perhaps the main value would be in actually having separate
> commands which could be bound to a submap with faster key sequences
> and/or to the context menu (with context-menu-mode on). Then our
> solution should be more in line with either defining a number of
> additional named commands (for mode universal kinds) and/or making it
> easier to define new such commands for users/3rd-party packages/etc.

That's an interesting idea.  So maybe M-' (or whatever) could be a new
prefix for a prefix-map which contains both universal and mode-specific
kinds of lookups.

So maybe something which looks kinda like (not suggesting final
bindings, just trying to feel out what it would be like):

generic eglot-find-implementation:   M-' i
generic eglot-find-declaration:      M-' d
generic eglot-find-typeDefinition:   M-' t
xref-extra-kind, prompting for kind: M-' M-'
xref-extra-kind, showing all:        M-' a

And if there was something mode-specific, like the Java overriding
method thing, it could be e.g. M-' o

I like this sort of design a lot actually:

- it's faster to type than having to complete the kind name (IMO, having
  to complete the kind name makes the current patch quite clumsy)
- it allows a common set of keys between all modes
- it's extensible by individual modes without too much trouble
- it works naturally with context-menu-mode and other menus
  (although I suppose we could automatically populate the context-menu
  with the output from xref-backend-extra-kinds)
- users could still use completing-read to type the kind

Plus, if we do use M-' or any other short binding for this, we should
almost certainly make it the start of a new prefix-map rather than bind
M-' directly to the new command; doing otherwise would be wasteful of
valuable keybinding space.

> Please try out the attached (with implementation for Eglot hopefully
> coming soon) and let me know your thoughts (you all).

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

Dmitry Gutov <dgutov@yandex.ru> writes:
> Still, though. It's not clear to me that improving this particular
> feature for Elisp will bring a lot to the table.
>
> Does it really do certain things better than simply using 'M-.' and
> then choosing between all the available definition types inside the
> buffer (and then navigating to one with TAB)?
>
> Or if one prefers a more dynamic interface with completions, (setq
> xref-xref-show-definitions-function
> #'show-definitions-completing-read) could fit that bill.
>
> There is a tradeoff of not being able to just press a single char, but
> OTOH one doesn't always know which definition kind they want in
> advance, and here, the full list helps one make that choice.

I guess one possible issue is that requesting all definitions and then
choosing between them could be expensive in certain languages.  Knowing
what kind of "definition" you want in advance could make things faster.
I don't have any concrete example of this, just putting out the thought.

Re: Adding support for xref jumping to headers/interfaces

[Eshel Yaron]

Dmitry Gutov <dgutov@yandex.ru> writes:

> Hi Eshel,
>
> On 05/11/2023 10:16, Eshel Yaron wrote:
>
>>> Attached is the patch along the lines that we discussed: a new command
>>> xref-find-extra currently bound to M-' just for the sake of this
>>> experiment (eliminating the need to set up define-key to try out the
>>> "fast" workflow).
>> That's was indeed convenient for testing the patch, although in the
>> long
>> run there should probably be a different binding since `M-'` is already
>> bound to `abbrev-prefix-mark`.
>
> Honestly, it's a pretty obscure command, and the binding is
> nice. Anyway, it's not a given we'll have a new command like this, so
> there's no point in discussing bindings too early.

I agree, both that this `abbrev-prefix-mark` hardly deserves such a
precious global default binding, and that it's futile to discuss
changing that right now.

>> Personally, I would prefer a `read-multiple-choice` alternative to
>> `completing-read` here, that would allow me to select the "kind" with a
>> single key press.  Since there are rarely more than a few kinds to
>> choose from, in many cases `completing-read` introduces unnecessary
>> friction.  Ideally, the backend should be able to choose whether to do a
>> `completing-read` or a "quick" `read-multiple-choice` on a case-by-case
>> basis.
>
> People override completing-read using modes like
> icomplete-vertical-mode, but there are no popular augmentations for
> read-multiple-choice.

Sure, I also tweak and customize `completing-read`, and it's definitely
up for the task of selecting among a few candidates, but nevertheless I
think that `completing-read` is not the best tool for this job.  With
`read-multiple-choice` you make a selection with a single key, and you
don't need to press TAB to see which options are available--the prompt
tells you what each key does in case you don't know.  That's hard to
beat even with a highly optimized `completing-read` interface IME.

> Also, sometimes the types will start with the same characters, and
> sometimes not. Just using completing-read at least provides a stable
> UI for all cases: if you remember the type, you can input the first
> part of the name without thinking and press RET.

That's true, but to know which kinds are available you need to see the
list at least once, even if only to know how exactly the backend chose
to name these kinds (is it `defvar` or `variable`?).  Anyway, I don't
think this is such a crucial issue.  We can always add a
`read-multiple-choice` wrapper later, as long as the API permits it.

>> The backend can also associate stable `read-multiple-choice`
>> keys to the different kinds it support.  WDYT?
>
> Perhaps you are instead thinking of a solution when there is a stable
> keymap with known commands assigned to the same characters.

That could work too, perhaps, but what I meant is for the backend to
provide along with each kind the unique key that you press to select it.
So you always use the same key to the select the same kind with a given
backend, regardless of the set of kinds available for a specific identifier.

>>> To have something to test, I implemented this for Elisp. However, all
>>> known kinds are already printed by the regular xref-find-definitions
>>> output in that backend. So the result turned out interesting, but a
>>> little impractical: with Elisp, we already make the choice between the
>>> kinds in the Xref output buffer, when that buffer is shown at all (as
>>> Eli pointed out, actually). So... we could proceed in that direction
>>> and recommend that all kinds of definitions would be added there.
>> Yes, I've noticed that redundancy of sorts while testing this patch,
>> but I think that selecting the definition kind explicitly with this
>> new command still provides a distinct experience compared to the
>> `xref-find-definitions`.
>
> Yes, but is it better? Does it bring much to the table? For Elisp, my
> own answer is "probably not": all the kinds are available in
> xref-find-definitions, and the selection UI is adequate already (isn't
> it?)

I think that there's some benefit in being able to designate the kind of
definition you want in advance, especially if I already know which kind
I want.  In the current state of affairs, I get all definitions and then
possibly skip one or two and get to select the one I wanted.  But I
don't know in advance if the one I want will come first, or last, so I
need to take a look at the results and respond accordingly.
`xref-find-extra` (or similar) removes that bit of friction, and IMO
that's an advantage over the current UI.

>> That's indeed an added value IMO.  In that spirit, it might be better
>> for `xref-find-extra` to take the "kind" as an argument, so we can
>> define `xref-find-foo` as `(apply-partially #'xref-find-extra 'foo)`.
>> Interactively, `xref-find-extra` would still pick up the identifier
>> first and then prompt for the kind.
>
> A helper for defining new commands is definitely doable. For that
> route, though, I would like to ask whether we'll want to define any of
> them inside core Xref. And if not, then where would those definitions
> reside.
>

I would imagine that the backends should provide such commands in the
appropriate contexts, so maybe those definitions could reside alongside
the backend code?

>>> Please try out the attached (with implementation for Eglot hopefully
>>> coming soon) and let me know your thoughts (you all).
>> One issue I had was with selecting "feature" as the definition kind.
>> Basically it doesn't seem to work for some reason, so for example typing
>> `M-' isearch RET feature RET` says "No extra-defs found for: isearch".
>
> That's just a missing mapping inside xref-backend-extra-defs, I didn't
> want to spend too much time polishing the prototype.
>
>> Also, for minor mode names, we currently suggest both "function" and
>> "variable" kinds, but they both point to the same `define-minor-mode`.
>> Ideally, we'd unify the two and simply suggest "minor-mode" as the
>> definition kind.
>
> Also easy enough to do, e.g. by borrowing the solution from
> elisp--xref-filter-definitions.
>
> Still, though. It's not clear to me that improving this particular
> feature for Elisp will bring a lot to the table.

Yes, the general API is probably more interesting to then the specifics
of the Elisp backend.


Cheers,

Eshel

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 07/11/2023 11:36, João Távora wrote:
> [Adding back emacs-devel, which I didn't mean to remove from my previous
> email.  This is also why I quote your message fully.  ]

Perfect.

>>> It wasn't very difficult, but I did have to scratch my head a bit until
>>> I found what I think is the key.  So first I did a reasonably simple
>>> commit to xref.el, which gives xref-find-extra a KIND argument, making
>>> it more versatile since it can now be used non-interactively.  I think
>>> the interactive behavior is completely unchanged.  The docstring might
>>> need a tweak or two.
>>
>> There is an omission in being able to input the identifier (fixable, of
>> course).
> 
> Can you explain exactly the problem?  I'm happy to fix it.

Just that it does not ask for the identifier (with C-u or without); that 
it cannot. But if that won't be useful anyway (as you state below), 
there's no point in trying.

>> Though I'm not sure if that capability will be helpful for
>> those searches with an LSP client (with language servers providing
>> relatively limited identifier completion).
> 
> Indeed, it's not.  For every reference kind _except_ the "definition"
> kind, where it does come in handy, since I have a hack for that
> too.  Indeed very handy (and very hackish, but that's life).

So... there's no way to take a "definition" symbol name (and associated 
metadata) and jump to the "declaration", for example? Using that hack of 
yours, or with a slight tweak.

>> I still note, though, that you kept those commands around:
>> eglot-find-declaration/implementation/typeDefinition. Whereas previously
>> I seem to recall you opined that the commands themselves shouldn't be in
>> Eglot.
>>
>> Would you say it's a backward compatibility concern, or something else
>> as well?
> 
> The former.  IOW if xref.el had this capability in the past, I would have
> used it.
> 
> But I still nonetheless think functions should be as versatile as possible,
> so that even what we designed as an interactive entry-point can be used
> as a library entry point.

Well, making those commands easy to write is a small task. We already 
have 'xref-show-xrefs' which you could call, and if my thinking is right 
(the bottom of the previous email), a new function called 
'xref-show-defs' which internally uses xref-show-definitions-function, 
would be just what the doctor ordered. No new backend methods necessary.

>>> Anyway, this new xref-find-extra aided greatly in adding Eglot support.
>>> It removes the need for a very hackish implementation of the existing
>>> 'eglot-find-declaration', 'eglot-find-implementation' and
>>> 'eglot-find-typeDefinition' commands.  And of course 'xref-find-extra'
>>> as an interactive entry point also works for finding these things.
>>
>> Whatever solution we decide to go with in the end, it should be
>> straightforward enough to create a helper that would assist in defining
>> such commands.
> 
> Yes.  I do think this one is pretty good at doing that.  But I guess
> even with this one, you could offer a 'xref-define-interactive-finder-for-kind'
> macro that would automate it even further and help xref.el retain finer
> control over how exactly that finder behaves, for example vis-a-vis
> prefix arguments.

Right. Though if the backend cannot work on arbitrary symbol names (only 
on symbol at point), that kind of negates most of the benefits.

>>> One thing that is missing IMO, but is reasonably easy to fix, is an
>>> indirection to allow non-string objects to be used as KIND, but still
>>> have these objects be displayed to the user with a pretty string.  Of
>>> course, this is not essential -- I could just use strings as KINDs in
>>> eglot.el and be done with it -- but I feel it'd be a little nicer.
>>> Right now I'm using the symbols 'eglot--xref-implementation',
>>> 'eglot--xref-declaration', etc, which are printed as is to the user in
>>> the completing-read of 'xref-find-extra'.  Works for now.
>>
>> Using strings as KINDs is what allows us to use completing-read on them.
>> Internally, there could be of course some alist mapping them to complex
>> objects. If you have a better idea, I'm open.
> 
> Yes, that indirection is what is needed.  It could be done via symbol
> properties (in which case KIND would be a symbol or a string), via
> a specific prescription that KIND can be (OBJ . DISPLAY), a specific
> prescription like (OBJ . DISPLAY-FN-TO-BE-CALLED-ON-OBJ) or just a
> generic function whose default implementation is #'identity thus
> letting completing read do whatever it does for that OBJ, which in
> the case of symbols is convert them to string and return that string).
> Very many ways to skin this cat, so just pick one.

I suppose xref-backend-definition-kinds could return any data structure 
that completing-read could use? A.g. an alist.

> But just to be clear: whatever the technique used, that "mapping" you
> speak of (which I call "indirection"), could _not_ be "internal".  It
> would have to be exposed so the backend can control how the OBJs it
> returns from 'xref-backend-extra-kinds' are converted to strings.

Hm, why not? You set up an internal var with the mapping, return a list 
of strings, get one of those stings, look up the "complex" var in the 
mapping, and use it. Text properties are a good option too, but IIRC 
there was a problem with completing-read stripping them.

>>> Another idea I had while doing this is that the name xref-find-extra
>>> could be actually something like xref-find-by-kind.  What I mean is that
>>> I found myself adding the existing kinds "definition" and "references"
>>> to that "extra" set.  I think this is more comfortable for the user that
>>> even though they are not "extras".
>>
>> This is one of the questions I'd like to see answered:
>>
>> - Should a command like this include existing kinds (that
>> xref-find-definitions already lists)?
> 
> My answer is yes.  But we could leave that up to the backend.

We would ultimately leave it all to the backend, of course, but we 
should provide some guidelines. And these choices should inform the 
commands we add and how we organize them.

> I
> intend to remap M-?  to xref-find-extra myself, losing the speed of
> xref-find-references for "all references", but gaining in versatility.
> Of course a similar effect could be achieved via some kind of prefix
> argument or variable controlling the behaviour of xref-find-references.
 >
> Another thing that I think should be changed is the name of the generic
> function xref-backend-extra-defs.  It should be IMO xref-backend-extra-refs
> because a "definition" is a type of "reference" to a symbol/name.  Likewise
> to mentions of "definition" in the new docstrings.

We use xref-show-xrefs-function for "references" and 
xref-show-definitions-function for "definitions". By default they differ 
in what happens if there is just 1 match (the latter just jumps to it). 
But one can also customize xref-show-definitions-function to 
xref-show-definitions-completing-read (which I did), and then you get a 
direct jump (with completion) for definitions, and a list of all matches 
for references. A definition could be called a subtype of a reference, 
but they often carry different information in practice as well.

Perhaps we could add two commands: xref-find-defs-by-kind and 
xref-find-xrefs-by-kind. I wonder which "kinds" would get sorted in the 
second category along with "references", though.

>> - Should there be other kinds in it, absent from xref-find-definitions?
>> If yes, of kinds of "kinds" (:-)) should be in one set but not the other?
> 
> Up to the backend.
> 
>> - Should xref-find-definitions include the results of
>> eglot-find-declaration and eglot-find-typeDefinition? Does it include
>> them already?
> 
> That's up to the backend, Eglot in this case.  And the Eglot backend
> leaves that up to the LSP server, so it's a question that can't be answered.

Eglot could make several queries and append the results. It might be 
slower in certain cases, but I'm not sure it will be slow enough for the 
users to take notice. In most cases, anyway.

That might lead to a better UX as well. For example, what kind of 
symbols does eglot-find-typeDefinition work on? Type references? If you 
invoke xref-find-definitions instead, does it find any other kind of 
definitions for them?

Speaking of guidelines and semantics again:

1. If we were to expect that "xref-find-definitions" finds all kinds of 
definitions, we could instead just add a new method to xref-item to 
indicate kind. And then filter. Theoretically less efficient, but if 
xref-find-definitions will remain the main tool for people, saving on 
work in rare situations might not be optimal.

2. If we say that xref-find-extra includes kinds that are already in 
"definitions", then the name "extra" is invalid, and 
xref-find-...-by-kind seems more apt as the name of a command. If it 
only included "extra" kinds, OTOH, the names would hold.

3. If xref-find-...-by-kind allows you to find both kinds included in 
the set of "definitions" and not, that becomes the most awkward option 
to describe, in the docstrings and the manual. In any case, we'd have to 
use the most careful naming.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 07/11/2023 19:09, Spencer Baugh wrote:
> Dmitry Gutov<dgutov@yandex.ru>  writes:
>> Still, though. It's not clear to me that improving this particular
>> feature for Elisp will bring a lot to the table.
>>
>> Does it really do certain things better than simply using 'M-.' and
>> then choosing between all the available definition types inside the
>> buffer (and then navigating to one with TAB)?
>>
>> Or if one prefers a more dynamic interface with completions, (setq
>> xref-xref-show-definitions-function
>> #'show-definitions-completing-read) could fit that bill.
>>
>> There is a tradeoff of not being able to just press a single char, but
>> OTOH one doesn't always know which definition kind they want in
>> advance, and here, the full list helps one make that choice.
> I guess one possible issue is that requesting all definitions and then
> choosing between them could be expensive in certain languages.  Knowing
> what kind of "definition" you want in advance could make things faster.
> I don't have any concrete example of this, just putting out the thought.

Indeed it could, but is that a practical concern? For certain languages, 
or large projects, or etc?

In general, I'm not a fan of solving performance problems with requiring 
more manual control on the part of the user. We can do that, but 
hopefully only when there really are no better options.

And making the choice which kind to search for might save the machine 
500ms during the search while requiring the user to spend several more 
seconds typing and thinking.

I was hoping people could try out my patch, and Joao's addition as well, 
and perhaps report with concrete examples.

And arguments can be unrelated to performance: e.g. certain "kinds" 
might really be unsuitable for the default set. Perhaps "find 
implementations" might fall in this variety.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 07/11/2023 23:30, Eshel Yaron wrote:

>>> Personally, I would prefer a `read-multiple-choice` alternative to
>>> `completing-read` here, that would allow me to select the "kind" with a
>>> single key press.  Since there are rarely more than a few kinds to
>>> choose from, in many cases `completing-read` introduces unnecessary
>>> friction.  Ideally, the backend should be able to choose whether to do a
>>> `completing-read` or a "quick" `read-multiple-choice` on a case-by-case
>>> basis.
>>
>> People override completing-read using modes like
>> icomplete-vertical-mode, but there are no popular augmentations for
>> read-multiple-choice.
> 
> Sure, I also tweak and customize `completing-read`, and it's definitely
> up for the task of selecting among a few candidates, but nevertheless I
> think that `completing-read` is not the best tool for this job.  With
> `read-multiple-choice` you make a selection with a single key, and you
> don't need to press TAB to see which options are available--the prompt
> tells you what each key does in case you don't know.  That's hard to
> beat even with a highly optimized `completing-read` interface IME.

But for read-multiple-choice you have to move your eyes to the echo 
area, read the char->value mappings, choose one in your head, and type 
it. Typing the first 2-3 chars of the value you wanted without looking 
anywhere might take less time.

>> Also, sometimes the types will start with the same characters, and
>> sometimes not. Just using completing-read at least provides a stable
>> UI for all cases: if you remember the type, you can input the first
>> part of the name without thinking and press RET.
> 
> That's true, but to know which kinds are available you need to see the
> list at least once, even if only to know how exactly the backend chose
> to name these kinds (is it `defvar` or `variable`?).  Anyway, I don't
> think this is such a crucial issue.  We can always add a
> `read-multiple-choice` wrapper later, as long as the API permits it.

An advice binding completing-read to something using 
read-multiple-choice internally should do it. Or a user option, etc.

>>> The backend can also associate stable `read-multiple-choice`
>>> keys to the different kinds it support.  WDYT?
>>
>> Perhaps you are instead thinking of a solution when there is a stable
>> keymap with known commands assigned to the same characters.
> 
> That could work too, perhaps, but what I meant is for the backend to
> provide along with each kind the unique key that you press to select it.
> So you always use the same key to the select the same kind with a given
> backend, regardless of the set of kinds available for a specific identifier.

Imagine you are switching between different buffers using different 
languages. If the backends are different, your set of keys that could be 
used would have to be looked up every time, because the sets of values 
are different. That could even happen when using the same client, if 
e.g. Eglot should different kinds of different languages.

If, however, we managed to work out a fixed set of commands that cover 
99% of the languages and assigned them to a map, old-style, that would 
make things more stable: at least the same char would always mean the same.

Just spitballing again, with a different take on the problem (a less 
flexible and extensible one).

>>>> To have something to test, I implemented this for Elisp. However, all
>>>> known kinds are already printed by the regular xref-find-definitions
>>>> output in that backend. So the result turned out interesting, but a
>>>> little impractical: with Elisp, we already make the choice between the
>>>> kinds in the Xref output buffer, when that buffer is shown at all (as
>>>> Eli pointed out, actually). So... we could proceed in that direction
>>>> and recommend that all kinds of definitions would be added there.
>>> Yes, I've noticed that redundancy of sorts while testing this patch,
>>> but I think that selecting the definition kind explicitly with this
>>> new command still provides a distinct experience compared to the
>>> `xref-find-definitions`.
>>
>> Yes, but is it better? Does it bring much to the table? For Elisp, my
>> own answer is "probably not": all the kinds are available in
>> xref-find-definitions, and the selection UI is adequate already (isn't
>> it?)
> 
> I think that there's some benefit in being able to designate the kind of
> definition you want in advance, especially if I already know which kind
> I want.

It sometimes happens that the Elisp Xref backend guesses the "kind" 
automatically more correctly than I might (usually because of missing or 
extra parens), ultimately hinting at problems with code.

> In the current state of affairs, I get all definitions and then
> possibly skip one or two and get to select the one I wanted.  But I
> don't know in advance if the one I want will come first, or last, so I
> need to take a look at the results and respond accordingly.

Does that happen often? With which languages? Are there "kinds" that you 
more often would prefer not to see in the list?

>>> That's indeed an added value IMO.  In that spirit, it might be better
>>> for `xref-find-extra` to take the "kind" as an argument, so we can
>>> define `xref-find-foo` as `(apply-partially #'xref-find-extra 'foo)`.
>>> Interactively, `xref-find-extra` would still pick up the identifier
>>> first and then prompt for the kind.
>>
>> A helper for defining new commands is definitely doable. For that
>> route, though, I would like to ask whether we'll want to define any of
>> them inside core Xref. And if not, then where would those definitions
>> reside.
>>
> 
> I would imagine that the backends should provide such commands in the
> appropriate contexts, so maybe those definitions could reside alongside
> the backend code?

Joao didn't want to have such definitions in Xref. I'm still on the 
fence, personally.

But these are two different/separate features:
- Add command that can ask you which kind you want, and show matches.
- Define different new commands, one for each "kind".

>>> Also, for minor mode names, we currently suggest both "function" and
>>> "variable" kinds, but they both point to the same `define-minor-mode`.
>>> Ideally, we'd unify the two and simply suggest "minor-mode" as the
>>> definition kind.
>>
>> Also easy enough to do, e.g. by borrowing the solution from
>> elisp--xref-filter-definitions.
>>
>> Still, though. It's not clear to me that improving this particular
>> feature for Elisp will bring a lot to the table.
> 
> Yes, the general API is probably more interesting to then the specifics
> of the Elisp backend.

What I meant is, using it with the Elisp implementation didn't convince 
me of the usefulness of the feature. Perhaps you disagree?

Or it would be nice to hear from someone who have tried out Eglot's 
integration and found more upsides there.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 07/11/2023 18:51, Spencer Baugh wrote:

>> Perhaps it /should/ (xref-find-definitions to include both
>> implementation and interface). And include the typeDefinition entries
>> for langs that have those. Etc.
>>
>> Would the UI be worse? Let's try to find out.
>>
>> Attached is the patch along the lines that we discussed: a new command
>> xref-find-extra currently bound to M-' just for the sake of this
>> experiment (eliminating the need to set up define-key to try out the
>> "fast" workflow). It uses the symbol at point or asks for it, then
>> polls the backend for available kinds, does completing-read and asks
>> the backend for definitions of that kind. And shows the list or jumps
>> to the unique one.
>>
>> To have something to test, I implemented this for Elisp. However, all
>> known kinds are already printed by the regular xref-find-definitions
>> output in that backend. So the result turned out interesting, but a
>> little impractical: with Elisp, we already make the choice between the
>> kinds in the Xref output buffer, when that buffer is shown at all (as
>> Eli pointed out, actually). So... we could proceed in that direction
>> and recommend that all kinds of definitions would be added there.
> 
> This (and Joao's patch adding support for eglot) is very interesting,
> thank you for implementing it!
> 
> We also discussed a UI which shows all kinds of definitions in a single
> buffer.  Maybe the prompt for KIND should default to "all" which shows
> all kinds of definitions?

The natural question is whether 'xref-find-definitions' would be that 
UI, and if not, why not.

Also, if "references" are included in the list of kinds (as the current 
patch for Eglot does, I think), then "show all kinds" is not likely to 
be very useful -- all will drown in "references".

> I notice the API doesn't support "fetch all kinds of definitions";
> extra-defs requires specifying a specific kind.  Perhaps just specifying
> a nil KIND should request all kinds, and extra-defs should tag each
> returned object with the kind?  We could also just make a separate call
> to extra-defs for each kind returned by extra-kinds, but that would be
> allow less space for the backend to batch its operations.

That's a question for later.

>> Or perhaps the main value would be in actually having separate
>> commands which could be bound to a submap with faster key sequences
>> and/or to the context menu (with context-menu-mode on). Then our
>> solution should be more in line with either defining a number of
>> additional named commands (for mode universal kinds) and/or making it
>> easier to define new such commands for users/3rd-party packages/etc.
> 
> That's an interesting idea.  So maybe M-' (or whatever) could be a new
> prefix for a prefix-map which contains both universal and mode-specific
> kinds of lookups.
> 
> So maybe something which looks kinda like (not suggesting final
> bindings, just trying to feel out what it would be like):
> 
> generic eglot-find-implementation:   M-' i
> generic eglot-find-declaration:      M-' d
> generic eglot-find-typeDefinition:   M-' t

Then we will more-or-less nail down the whole set of "available Xref 
kinds" in the core by having these commands defined.

That's not very flexible, but if the set doesn't actually change too 
much, and doesn't vary between languages a lot, it could work. Would 
lead to a more straightforward design, too.

> xref-extra-kind, prompting for kind: M-' M-'

Would we need this command, if we had separate commands for each kind 
already?

> xref-extra-kind, showing all:        M-' a
> 
> And if there was something mode-specific, like the Java overriding
> method thing, it could be e.g. M-' o

Most of them are mode-specific already. Some languages don't have 
separate "declarations", some don't know what "type definitions" are, 
and some, indeed, don't have method overrides.

> I like this sort of design a lot actually:

> - it's faster to type than having to complete the kind name (IMO, having
>    to complete the kind name makes the current patch quite clumsy)
> - it allows a common set of keys between all modes
> - it's extensible by individual modes without too much trouble
> - it works naturally with context-menu-mode and other menus

Indeed.

>    (although I suppose we could automatically populate the context-menu
>    with the output from xref-backend-extra-kinds)

Probably.

> - users could still use completing-read to type the kind
> 
> Plus, if we do use M-' or any other short binding for this, we should
> almost certainly make it the start of a new prefix-map rather than bind
> M-' directly to the new command; doing otherwise would be wasteful of
> valuable keybinding space.

If we're going to have separate commands for kinds, that is indeed a 
good idea.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Tue, Nov 7, 2023 at 10:56 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 07/11/2023 11:36, João Távora wrote:
> > [Adding back emacs-devel, which I didn't mean to remove from my previous
> > email.  This is also why I quote your message fully.  ]
>
> Perfect.
>
> >>> It wasn't very difficult, but I did have to scratch my head a bit until
> >>> I found what I think is the key.  So first I did a reasonably simple
> >>> commit to xref.el, which gives xref-find-extra a KIND argument, making
> >>> it more versatile since it can now be used non-interactively.  I think
> >>> the interactive behavior is completely unchanged.  The docstring might
> >>> need a tweak or two.
> >>
> >> There is an omission in being able to input the identifier (fixable, of
> >> course).
> >
> > Can you explain exactly the problem?  I'm happy to fix it.
>
> Just that it does not ask for the identifier (with C-u or without); that
> it cannot.

I just tried my changed xref-find-extra in Elisp mode and it always asks
for the identifier.  In Eglot, which sets xref-prompt-for-identifier
to nil while
managing a buffer it doesn't.  But it does when you give it C-u.  So I
don't understand
the problem.  Can you give a reference.

> But if that won't be useful anyway (as you state below),
> there's no point in trying.

No, it _is_ useful.  But only for the eglot--xref-definition kind.

> >> Though I'm not sure if that capability will be helpful for
> >> those searches with an LSP client (with language servers providing
> >> relatively limited identifier completion).
> >
> > Indeed, it's not.  For every reference kind _except_ the "definition"
> > kind, where it does come in handy, since I have a hack for that
> > too.  Indeed very handy (and very hackish, but that's life).
>
> So... there's no way to take a "definition" symbol name (and associated
> metadata) and jump to the "declaration", for example? Using that hack of
> yours, or with a slight tweak.

No there isn't.  The "hack" works like this.  The LSP method workspace/symbols
when passed an empty string gives you a listing of all the symbols in the
workspace.  Great, that's what you need to give users the choice to select
a symbol.   Also that's something previous version of LSP didn't do, you
may recall.

Anyway, the response to workspace/symbols also happens to supply, at least
in some servers, the "location" of every symbol and this
usually/always matches what
you would have gotten if you had used the LSP method textDocument/definition
when putting point on that symbol in source code.

So, my hack takes advantage of this and if "location" is available from
the workspace/symbols that triggers with C-u, I don't even have to
issue textDocument/definition:  instead I go straight to "location" locus.
And that's pretty pretty useful in C++ and clangd at least.

But -- at least for now -- "location" is the only thing you get from
workspace/symbols.  No "declaration", "implementation", "typeDefinition", etc.
So at least for now, only the "definition" kind can take advantage of the C-u.

But who knows in the future?  It's a very good thing to be future-proof, when
feasible.  And it certainly sounds feasible here.

> >> I still note, though, that you kept those commands around:
> >> eglot-find-declaration/implementation/typeDefinition. Whereas previously
> >> I seem to recall you opined that the commands themselves shouldn't be in
> >> Eglot.
> >>
> >> Would you say it's a backward compatibility concern, or something else
> >> as well?
> >
> > The former.  IOW if xref.el had this capability in the past, I would have
> > used it.
> >
> > But I still nonetheless think functions should be as versatile as possible,
> > so that even what we designed as an interactive entry-point can be used
> > as a library entry point.
>
> Well, making those commands easy to write is a small task. We already
> have 'xref-show-xrefs' which you could call, and if my thinking is right
> (the bottom of the previous email), a new function called
> 'xref-show-defs' which internally uses xref-show-definitions-function,
> would be just what the doctor ordered. No new backend methods necessary.

I'd rather not call xref.el's display code and worry about what to pass
to that function.  I want a one-liner like there is today.

> >>> Anyway, this new xref-find-extra aided greatly in adding Eglot support.
> >>> It removes the need for a very hackish implementation of the existing
> >>> 'eglot-find-declaration', 'eglot-find-implementation' and
> >>> 'eglot-find-typeDefinition' commands.  And of course 'xref-find-extra'
> >>> as an interactive entry point also works for finding these things.
> >>
> >> Whatever solution we decide to go with in the end, it should be
> >> straightforward enough to create a helper that would assist in defining
> >> such commands.
> >
> > Yes.  I do think this one is pretty good at doing that.  But I guess
> > even with this one, you could offer a 'xref-define-interactive-finder-for-kind'
> > macro that would automate it even further and help xref.el retain finer
> > control over how exactly that finder behaves, for example vis-a-vis
> > prefix arguments.
>
> Right. Though if the backend cannot work on arbitrary symbol names (only
> on symbol at point), that kind of negates most of the benefits.

Not really, prefix argument handling was just one example.  And it's
not certain at all that LSP will forever be unable to ask for different
kinds of refs for a given symbol.  It recently gained the ability to
do that for the "definition" kind, so there's no reason to be so
pessimistic.  Not to mention other backends, say like SLY or SLIME may
well want to leverage xref-find-extra to simplify, say their sly-edit-uses.

So please don't deprecate your creation.

> >>> One thing that is missing IMO, but is reasonably easy to fix, is an
> >>> indirection to allow non-string objects to be used as KIND, but still
> >>> have these objects be displayed to the user with a pretty string.  Of
> >>> course, this is not essential -- I could just use strings as KINDs in
> >>> eglot.el and be done with it -- but I feel it'd be a little nicer.
> >>> Right now I'm using the symbols 'eglot--xref-implementation',
> >>> 'eglot--xref-declaration', etc, which are printed as is to the user in
> >>> the completing-read of 'xref-find-extra'.  Works for now.
> >>
> >> Using strings as KINDs is what allows us to use completing-read on them.
> >> Internally, there could be of course some alist mapping them to complex
> >> objects. If you have a better idea, I'm open.
> >
> > Yes, that indirection is what is needed.  It could be done via symbol
> > properties (in which case KIND would be a symbol or a string), via
> > a specific prescription that KIND can be (OBJ . DISPLAY), a specific
> > prescription like (OBJ . DISPLAY-FN-TO-BE-CALLED-ON-OBJ) or just a
> > generic function whose default implementation is #'identity thus
> > letting completing read do whatever it does for that OBJ, which in
> > the case of symbols is convert them to string and return that string).
> > Very many ways to skin this cat, so just pick one.
>
> I suppose xref-backend-definition-kinds could return any data structure
> that completing-read could use? A.g. an alist.

That's option 2 in my list.  Works, but the generic function is much
easier.  Return whatever you want from xref-backend-definition-kinds
and if your're interested in conversion, add a method to a generic
function.  The set of all kinds is very small, and this is all
interactive, so no performance problem.  And most of the xref API
is already generic functions, so we should just be consistent.

> > But just to be clear: whatever the technique used, that "mapping" you
> > speak of (which I call "indirection"), could _not_ be "internal".  It
> > would have to be exposed so the backend can control how the OBJs it
> > returns from 'xref-backend-extra-kinds' are converted to strings.
>
> Hm, why not? You set up an internal var with the mapping, return a list
> of strings, get one of those stings, look up the "complex" var in the
> mapping, and use it.

If by "you" you mean the "backend", OK.  An internal xref mapping won't
work, since the backend needs to see it.

> Text properties are a good option too, but IIRC
> there was a problem with completing-read stripping them.

Yes, also works.  The problem here is not really the text-properties
but the age-old problem that completing-read will return a string even if
you give it list, an alist or a hash-table and that string isn't
even guaranteed to be eq to the keys of the maps you passed in.  So you
have to re-lookup with equal.  I've done this a million times.  Not elegant
but as long as the mapping is bijective and the map is small enough
you're absolutely fine.  Which is the case here.

> >>> Another idea I had while doing this is that the name xref-find-extra
> >>> could be actually something like xref-find-by-kind.  What I mean is that
> >>> I found myself adding the existing kinds "definition" and "references"
> >>> to that "extra" set.  I think this is more comfortable for the user that
> >>> even though they are not "extras".
> >>
> >> This is one of the questions I'd like to see answered:
> >>
> >> - Should a command like this include existing kinds (that
> >> xref-find-definitions already lists)?
> >
> > My answer is yes.  But we could leave that up to the backend.
>
> We would ultimately leave it all to the backend, of course, but we
> should provide some guidelines. And these choices should inform the
> commands we add and how we organize them.

OK, then the guideline should be: "yes, backends, please include existing
'definition' and 'references' things that you also feed to
xref-backend-definitions and xref-backend-references".

Or if you don't feel like guidelining, you could have xref force that
or force that according to a variable, which Eglot would most definitely
set to t in Eglot-managed buffers.

> > I
> > intend to remap M-?  to xref-find-extra myself, losing the speed of
> > xref-find-references for "all references", but gaining in versatility.
> > Of course a similar effect could be achieved via some kind of prefix
> > argument or variable controlling the behaviour of xref-find-references.
>  >
> > Another thing that I think should be changed is the name of the generic
> > function xref-backend-extra-defs.  It should be IMO xref-backend-extra-refs
> > because a "definition" is a type of "reference" to a symbol/name.  Likewise
> > to mentions of "definition" in the new docstrings.
>
> We use xref-show-xrefs-function for "references" and
> xref-show-definitions-function for "definitions". By default they differ
> in what happens if there is just 1 match (the latter just jumps to it).
> But one can also customize xref-show-definitions-function to
> xref-show-definitions-completing-read (which I did), and then you get a
> direct jump (with completion) for definitions, and a list of all matches
> for references. A definition could be called a subtype of a reference,
> but they often carry different information in practice as well.
>
> Perhaps we could add two commands: xref-find-defs-by-kind and
> xref-find-xrefs-by-kind. I wonder which "kinds" would get sorted in the
> second category along with "references", though.

I don't understand but I think you're overcomplicating or at least
misreading what I suggested.  I don't want any of that.  It just suggested
a name change the sake of clarity.

In my view, a definition of a symbol is a particular type of
reference to a symbol.  It's a "manifestation" if you want.
If it makes it any clear,   I suggest to rename

   xref-backend-extra-defs

to

   xref-backend-identifier-manifestations

Same arguments, protocol, better name (if somewhat long-winded).  And gets rid
of the "extra" while we're at it.

> >> - Should there be other kinds in it, absent from xref-find-definitions?
> >> If yes, of kinds of "kinds" (:-)) should be in one set but not the other?
> >
> > Up to the backend.
> >
> >> - Should xref-find-definitions include the results of
> >> eglot-find-declaration and eglot-find-typeDefinition? Does it include
> >> them already?
> >
> > That's up to the backend, Eglot in this case.  And the Eglot backend
> > leaves that up to the LSP server, so it's a question that can't be answered.
>
> Eglot could make several queries and append the results. It might be
> slower in certain cases, but I'm not sure it will be slow enough for the
> users to take notice.

That's overcomplicating and unneeded complexity.  Eglot should work like
other LSP clients and if those clients don't do that amalgamation neither
should Eglot.  What you're suggesting negates the purpose of having different
kinds of manifestations which is the end is about letting backends
organize those kinds into sets that the backend maintainer deems
useful.  Not to mention I just don't want to introduce this complexity.

> That might lead to a better UX as well. For example, what kind of
> symbols does eglot-find-typeDefinition work on? Type references? If you
> invoke xref-find-definitions instead, does it find any other kind of
> definitions for them?

textDocument/definitions only finds one definition, whatever the server
thinks is the "main" one.  xref-find-definitions has always worked like
that and when existing Eglot users for a specific server users read
"definition", that's what they expect.  So I don't want xref to force
this model.

> Speaking of guidelines and semantics again:
>
> 1. If we were to expect that "xref-find-definitions" finds all kinds of
> definitions, we could instead just add a new method to xref-item to
> indicate kind. And then filter. Theoretically less efficient, but if
> xref-find-definitions will remain the main tool for people, saving on
> work in rare situations might not be optimal.
>
> 2. If we say that xref-find-extra includes kinds that are already in
> "definitions", then the name "extra" is invalid, and
> xref-find-...-by-kind seems more apt as the name of a command. If it
> only included "extra" kinds, OTOH, the names would hold.
>
> 3. If xref-find-...-by-kind allows you to find both kinds included in
> the set of "definitions" and not, that becomes the most awkward option
> to describe, in the docstrings and the manual. In any case, we'd have to
> use the most careful naming.

Sure, maybe.  I admit I don't understand much of this, sorry.  I'll have to
read it better.  But should I?  I don't understand why you are complicating
things.   This feature isn't complex, it's simple. The patch shows it.
That's a good sign.  I certainly don't want to revamp Eglot's code,
merging request return values somehow, etc, just for this feature.  I'd sooner
reimplement the current UX entirely in eglot.el than do that.  But of course
I'd prefer xref.el: I hate adding UI things to eglot.el.

Your patch is almost perfect, I just tweaked a miniscule thing.  And I
while I would _like_ to see the strings thing addressed and the naming
reviewed, but I can certainly work with strings for kinds and there's no
shortage of ghastly naming in Emacs anyway, so I'll gladly part with
that suggestion as well.  Same with the command-definining macro I suggested.
Take the suggestion or don't, it doesn't make an enormous difference.

Sure you like to debate, and that's good, but we've already debated
this back then, and finally you gave us something that is working, useful
and -- with my little tweak -- a wee bit more versatile.  So let's stop
splitting these hair, push this small new feature and move on to more
interesting things.  We can always change it on master later on if
hordes of users
come out  screaming they hate it.

João

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Tue, Nov 7, 2023 at 11:18 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> Joao didn't want to have such definitions in Xref. I'm still on the
> fence, personally.

What don't I want?  I haven't read this wordy subthread but it seems
Eshel is proposing exactly what I changed in xref-find-extra: take "kind"
as argument.

> Or it would be nice to hear from someone who have tried out Eglot's
> integration and found more upsides there.

If you want that to happen, the only realistic way to have good
feedback from anyone else other than emacs-devel nerds like me and you
is to release an Eglot version with  this feature, which we can change
later (even non-backward-compatibly, within a reasonable time frame).

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 08/11/2023 02:21, João Távora wrote:
> On Tue, Nov 7, 2023 at 11:18 PM Dmitry Gutov<dgutov@yandex.ru>  wrote:
> 
>> Joao didn't want to have such definitions in Xref. I'm still on the
>> fence, personally.
> What don't I want?  I haven't read this wordy subthread but it seems
> Eshel is proposing exactly what I changed in xref-find-extra: take "kind"
> as argument.

Sorry: you didn't want those definitions in _Eglot_. Not in Xref, that 
was a mistype.

 > I haven't read this wordy subthread

Too bad.

>> Or it would be nice to hear from someone who have tried out Eglot's
>> integration and found more upsides there.
> If you want that to happen, the only realistic way to have good
> feedback from anyone else other than emacs-devel nerds like me and you
> is to release an Eglot version with  this feature, which we can change
> later (even non-backward-compatibly, within a reasonable time frame).

That seems like a last-resort type of approach, for this particular 
discussion.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Wed, Nov 8, 2023 at 12:33 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 08/11/2023 02:21, João Távora wrote:
> > On Tue, Nov 7, 2023 at 11:18 PM Dmitry Gutov<dgutov@yandex.ru>  wrote:
> >
> >> Joao didn't want to have such definitions in Xref. I'm still on the
> >> fence, personally.
> > What don't I want?  I haven't read this wordy subthread but it seems
> > Eshel is proposing exactly what I changed in xref-find-extra: take "kind"
> > as argument.
>
> Sorry: you didn't want those definitions in _Eglot_. Not in Xref, that
> was a mistype.

Ah. Well, first, it a fact that I _have_ them in Eglot, and I'll
keep having them, because they've been there a long time.

And it's not really relevant that I didn't _want_ them.  If I had
the choice back then between recommending existing xref commands
and adding new commands to Eglot, of course I wouldn't have added
them.  But I hadn't, so I did. Pretty simple.  So I don't understand
the argument you're making against Eshel's and my suggestion.

> >> Or it would be nice to hear from someone who have tried out Eglot's
> >> integration and found more upsides there.
> > If you want that to happen, the only realistic way to have good
> > feedback from anyone else other than emacs-devel nerds like me and you
> > is to release an Eglot version with  this feature, which we can change
> > later (even non-backward-compatibly, within a reasonable time frame).
> That seems like a last-resort type of approach, for this particular
> discussion.

Not really, just being pragmatic and suggesting a way for you to
get the opinions from Eglot users that you wrote you wanted.  There
isn't a significant amount of engaged Eglot users in this discussion
that are going to try the patch.

João

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

João Távora <joaotavora@gmail.com> writes:
> On Tue, Nov 7, 2023 at 11:18 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>> What I meant is, using it with the Elisp implementation didn't
>> convince me of the usefulness of the feature. Perhaps you disagree?
>>
>> Or it would be nice to hear from someone who have tried out Eglot's
>> integration and found more upsides there.
>
> If you want that to happen, the only realistic way to have good
> feedback from anyone else other than emacs-devel nerds like me and you
> is to release an Eglot version with  this feature, which we can change
> later (even non-backward-compatibly, within a reasonable time frame).
>
> João

I tried the Eglot integration, my comments elsewhere are inspired by
that.

I found it rather clumsy out of the box.  However, it works well with an
approach of adding bindings for individual kinds, as is currently how I
(and other Eglot users at Jane Street, and presumably most other Eglot
users everywhere) use Eglot.  So that compatibility with an
already-common approach is an upside of this integration.

However, the alternative UI which shows all kinds of definitions in a
single buffer does not exist for Eglot in the same way it exists for
Elisp.  So... I can't really compare it to that.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1779 bytes --]

On Wed, Nov 8, 2023, 16:40 Spencer Baugh <sbaugh@janestreet.com> wrote:

> João Távora <joaotavora@gmail.com> writes:
> > On Tue, Nov 7, 2023 at 11:18 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
> >> What I meant is, using it with the Elisp implementation didn't
> >> convince me of the usefulness of the feature. Perhaps you disagree?
> >>
> >> Or it would be nice to hear from someone who have tried out Eglot's
> >> integration and found more upsides there.
> >
> > If you want that to happen, the only realistic way to have good
> > feedback from anyone else other than emacs-devel nerds like me and you
> > is to release an Eglot version with  this feature, which we can change
> > later (even non-backward-compatibly, within a reasonable time frame).
>
> I tried the Eglot integration, my comments elsewhere are inspired by
> that.
>

Can you point me to this "elsewhere"?

>
> I found it rather clumsy out of the box.


What exactly is clumsy? Please state the command you used and what the
clumsy effects were.

However, it works well with an
> approach of adding bindings for individual kinds, as is currently how I
> (and other Eglot users at Jane Street, and presumably most other Eglot
> users everywhere) use Eglot.  So that compatibility with an
> already-common approach is an upside of this integration.
>

Bindings to what commands exactly?

However, the alternative UI which shows all kinds of definitions in a
> single buffer does not exist for Eglot in the same way it exists for
> Elisp.  So... I can't really compare it to that.
>

What "alternative UI" are you talking about exactly? How can I trigger it
for Elisp? Please give a full example as I don't know what you are taking
about.

Thanks!
João

>

[-- Attachment #2: Type: text/html, Size: 3292 bytes --]

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

Dmitry Gutov <dmitry@gutov.dev> writes:
> On 07/11/2023 18:51, Spencer Baugh wrote:
>>> Perhaps it /should/ (xref-find-definitions to include both
>>> implementation and interface). And include the typeDefinition entries
>>> for langs that have those. Etc.
>>>
>>> Would the UI be worse? Let's try to find out.
>>>
>>> Attached is the patch along the lines that we discussed: a new command
>>> xref-find-extra currently bound to M-' just for the sake of this
>>> experiment (eliminating the need to set up define-key to try out the
>>> "fast" workflow). It uses the symbol at point or asks for it, then
>>> polls the backend for available kinds, does completing-read and asks
>>> the backend for definitions of that kind. And shows the list or jumps
>>> to the unique one.
>>>
>>> To have something to test, I implemented this for Elisp. However, all
>>> known kinds are already printed by the regular xref-find-definitions
>>> output in that backend. So the result turned out interesting, but a
>>> little impractical: with Elisp, we already make the choice between the
>>> kinds in the Xref output buffer, when that buffer is shown at all (as
>>> Eli pointed out, actually). So... we could proceed in that direction
>>> and recommend that all kinds of definitions would be added there.
>> This (and Joao's patch adding support for eglot) is very
>> interesting,
>> thank you for implementing it!
>> We also discussed a UI which shows all kinds of definitions in a
>> single
>> buffer.  Maybe the prompt for KIND should default to "all" which shows
>> all kinds of definitions?
>
> The natural question is whether 'xref-find-definitions' would be that
> UI, and if not, why not.

As in, xref-find-definitions (M-.) *by default* shows all kinds of
definitions?  That seems bad for one big reason: 90% of the time, in 90%
of languages, there's a clear meaning of "definition" and I want M-. to
take me there directly rather than prompting in any way.  But there are
other kinds of "definitions" which plausibly I could want, just
sometimes, not by default.

If you mean something a non-default version of xref-find-definitions, or
another binding for another basically-identical command, sure, I think
that would be good.  We can have both that command *and* have a command
which prompts for KIND and defaults to "all".

> Also, if "references" are included in the list of kinds (as the
> current patch for Eglot does, I think), then "show all kinds" is not
> likely to be very useful -- all will drown in "references".

True, interesting point.  I wonder if we could reasonably distinguish
"definition" kinds (a small number per identifier) from "reference"
kinds (no limit on how many per identifier).  Although I suppose there's
also no limit on how many implementations of a generic function there
can be, and that's definitely something that fits in "definitions".

Maybe the backend could decide what kinds get included in "all".  Then
it could deliberately avoid including anything "reference-like".

>> I notice the API doesn't support "fetch all kinds of definitions";
>> extra-defs requires specifying a specific kind.  Perhaps just specifying
>> a nil KIND should request all kinds, and extra-defs should tag each
>> returned object with the kind?  We could also just make a separate call
>> to extra-defs for each kind returned by extra-kinds, but that would be
>> allow less space for the backend to batch its operations.
>
> That's a question for later.
>
>>> Or perhaps the main value would be in actually having separate
>>> commands which could be bound to a submap with faster key sequences
>>> and/or to the context menu (with context-menu-mode on). Then our
>>> solution should be more in line with either defining a number of
>>> additional named commands (for mode universal kinds) and/or making it
>>> easier to define new such commands for users/3rd-party packages/etc.
>> That's an interesting idea.  So maybe M-' (or whatever) could be a
>> new
>> prefix for a prefix-map which contains both universal and mode-specific
>> kinds of lookups.
>> So maybe something which looks kinda like (not suggesting final
>> bindings, just trying to feel out what it would be like):
>> generic eglot-find-implementation:   M-' i
>> generic eglot-find-declaration:      M-' d
>> generic eglot-find-typeDefinition:   M-' t
>
> Then we will more-or-less nail down the whole set of "available Xref
> kinds" in the core by having these commands defined.
>
> That's not very flexible, but if the set doesn't actually change too
> much, and doesn't vary between languages a lot, it could work. Would
> lead to a more straightforward design, too.

Hm, why do you say this?  This approach seems very extensible to me -
modes could add their own bindings/commands/kinds for things specific to
that mode.  But they build on top of a set of common kinds which are
provided by the core, which don't vary between language too much.

I do think that if we go with an API which has any notion of "kinds", we
should have some standard "kinds" in the core like implementation,
declaration, type-definition.  I don't see any reason not to do that.

>> xref-extra-kind, prompting for kind: M-' M-'
>
> Would we need this command, if we had separate commands for each kind
> already?

This would support kinds which:
- are language-specific,
- or are more rarely used and don't need a dedicated command,
- or are both.

>> xref-extra-kind, showing all:        M-' a
>> And if there was something mode-specific, like the Java overriding
>> method thing, it could be e.g. M-' o
>
> Most of them are mode-specific already. Some languages don't have
> separate "declarations", some don't know what "type definitions" are,
> and some, indeed, don't have method overrides.

Most languages have something that can fit into "declaration" or "type
definitions", though.

For Elisp generics, find-implementation could show the cl-defmethod, and
find-declaration could jump to the cl-defgeneric.  I would love that,
actually - I already find it rather annoying to have to navigate in the
xref buffer to the cl-defgeneric when I M-. on a generic method, if I
know up front that I want to jump to the cl-defgeneric.

Actually, this example has just convinced me that I definitely want
"kind-specific commands", even for Elisp.  That would be great.

For functions without separate declarations, find-implementation and
find-declaration could jump to the same place.  In some languages, maybe
that's just always what happens.

As for type definitions - even Elisp has things that could reasonably be
called type definitions, we just don't support jump-to-definition for
them right now.  I have no idea how we'd implement it, but we could
support jumping to cl-defstruct definitions, for example.  Maybe we
could use information from native compilation to guess the type of the
identifier at point?  But anyway, for now we'd just error if the user
ran such a command in emacs-lisp-mode or other modes that don't support
jumping to the type definition.

>> I like this sort of design a lot actually:
>
>> - it's faster to type than having to complete the kind name (IMO, having
>>    to complete the kind name makes the current patch quite clumsy)
>> - it allows a common set of keys between all modes
>> - it's extensible by individual modes without too much trouble
>> - it works naturally with context-menu-mode and other menus
>
> Indeed.
>
>>    (although I suppose we could automatically populate the context-menu
>>    with the output from xref-backend-extra-kinds)
>
> Probably.
>
>> - users could still use completing-read to type the kind
>> Plus, if we do use M-' or any other short binding for this, we
>> should
>> almost certainly make it the start of a new prefix-map rather than bind
>> M-' directly to the new command; doing otherwise would be wasteful of
>> valuable keybinding space.
>
> If we're going to have separate commands for kinds, that is indeed a
> good idea.

I almost want to say that we should have it be a prefix regardless of
whether we have separate commands for kinds.  I guess it depends:

- if we use C-M-?, that's already such a hard key to hit that maybe it's
  okay if we bind it directly to a command

- if we use M-' or something similarly convenient, it would be really
  tragic to not reclaim all that premium keybinding space.  (And
  probably if we use M-' we will need to put the old command on
  something under M-' anyway)




  

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

João Távora <joaotavora@gmail.com> writes:

> On Wed, Nov 8, 2023, 16:40 Spencer Baugh <sbaugh@janestreet.com> wrote:
>
>  João Távora <joaotavora@gmail.com> writes:
>  > On Tue, Nov 7, 2023 at 11:18 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>  >> What I meant is, using it with the Elisp implementation didn't
>  >> convince me of the usefulness of the feature. Perhaps you disagree?
>  >>
>  >> Or it would be nice to hear from someone who have tried out Eglot's
>  >> integration and found more upsides there.
>  >
>  > If you want that to happen, the only realistic way to have good
>  > feedback from anyone else other than emacs-devel nerds like me and you
>  > is to release an Eglot version with  this feature, which we can change
>  > later (even non-backward-compatibly, within a reasonable time frame).
>
>  I tried the Eglot integration, my comments elsewhere are inspired by
>  that.
>
> Can you point me to this "elsewhere"?

Elsewhere in this thread - my message on Tue, 07 Nov 2023 11:51:49 -0500

>  I found it rather clumsy out of the box. 
>
> What exactly is clumsy? Please state the command you used and what the clumsy effects were. 

Specifically xref-find-extras is clumsy, because it's annoying to have
to type the full "kind" in to request a specific "kind" of definition.

eglot-find-{declaration,implementation,typeDefinition} is good as
always.

>  However, it works well with an
>  approach of adding bindings for individual kinds, as is currently how I
>  (and other Eglot users at Jane Street, and presumably most other Eglot
>  users everywhere) use Eglot.  So that compatibility with an
>  already-common approach is an upside of this integration.
>
> Bindings to what commands exactly?

eglot-find-{declaration,implementation,typeDefinition}

>  However, the alternative UI which shows all kinds of definitions in a
>  single buffer does not exist for Eglot in the same way it exists for
>  Elisp.  So... I can't really compare it to that.
>
> What "alternative UI" are you talking about exactly? How can I trigger it for Elisp? Please give a full example as I don't know what
> you are taking about.

In Elisp, M-. (xref-find-definitions) already shows all the kinds of
definitions that are requestable through xref-find-extras.  But there's
no similar way to show all kinds of definitions for Eglot - M-. only
includes the proper definition, not the declaration and typeDefinition
and so on.

I know you would prefer not to implement such a thing, I'm just
explaining for Dmitry that since this kind of UI doesn't exist, I can't
compare xref-find-extras against that kind of UI for Eglot in the same
way that I can with Elisp.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 08/11/2023 03:19, João Távora wrote:
> On Wed, Nov 8, 2023 at 12:33 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
>>
>> On 08/11/2023 02:21, João Távora wrote:
>>> On Tue, Nov 7, 2023 at 11:18 PM Dmitry Gutov<dgutov@yandex.ru>  wrote:
>>>
>>>> Joao didn't want to have such definitions in Xref. I'm still on the
>>>> fence, personally.
>>> What don't I want?  I haven't read this wordy subthread but it seems
>>> Eshel is proposing exactly what I changed in xref-find-extra: take "kind"
>>> as argument.
>>
>> Sorry: you didn't want those definitions in _Eglot_. Not in Xref, that
>> was a mistype.
> 
> Ah. Well, first, it a fact that I _have_ them in Eglot, and I'll
> keep having them, because they've been there a long time.

The normal practice is to wait a little while until the new "core" 
feature is available for a lot of users (perhaps skipping a few Emacs 
versions, or - more bravely - wait for the next one and then just raise 
the minimum xref version required), and then declare the built-in 
versions obsolete in favor of the upstream.

The period before obsoletion is not as important as the agreement to do 
that - because then we will be talking whether the "proper" 
functionality should be shaped as a number of new commands (one per 
kind), or a dispatching command, or both. Or something else.

For the purpose of such design discussion, simply using Eglot's commands 
is not a good choice.

> And it's not really relevant that I didn't _want_ them.  If I had
> the choice back then between recommending existing xref commands
> and adding new commands to Eglot, of course I wouldn't have added
> them.  But I hadn't, so I did. Pretty simple.  So I don't understand
> the argument you're making against Eshel's and my suggestion.

Since you didn't want them and argued that they shouldn't be in Eglot, 
it seems natural to assume that you would be happy to obsolete them in 
favor of built-in functionality.

>>>> Or it would be nice to hear from someone who have tried out Eglot's
>>>> integration and found more upsides there.
>>> If you want that to happen, the only realistic way to have good
>>> feedback from anyone else other than emacs-devel nerds like me and you
>>> is to release an Eglot version with  this feature, which we can change
>>> later (even non-backward-compatibly, within a reasonable time frame).
>> That seems like a last-resort type of approach, for this particular
>> discussion.
> 
> Not really, just being pragmatic and suggesting a way for you to
> get the opinions from Eglot users that you wrote you wanted.  There
> isn't a significant amount of engaged Eglot users in this discussion
> that are going to try the patch.

Feedback is good, but I prefer to arrive at some sound design first, 
even if it doesn't do everything anyone might want. And then test it on 
a wider audience.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 08/11/2023 18:11, Spencer Baugh wrote:
> However, the alternative UI which shows all kinds of definitions in a
> single buffer does not exist for Eglot in the same way it exists for
> Elisp.  So... I can't really compare it to that.

I wonder what kind of forms we would call "declarations" in Emacs Lisp.

cl-defgeneric form seems to meet that description, but it can also 
contain the default definition, so on average it seems good to have it 
in the "find definitions" set by default.

We also have (declare-function ...) calls which seem to fit the 
description even better. Unfortunately, we don't know how to search for 
them (yet?), so they only feature in the search for "references". And 
that's probably good that they are not in the "find definitions" set.

Anything else? I think that's it.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Wed, Nov 8, 2023 at 5:45 PM Spencer Baugh <sbaugh@janestreet.com> wrote:
>
> João Távora <joaotavora@gmail.com> writes:
>
> > On Wed, Nov 8, 2023, 16:40 Spencer Baugh <sbaugh@janestreet.com> wrote:
> >
> >  João Távora <joaotavora@gmail.com> writes:
> >  > On Tue, Nov 7, 2023 at 11:18 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
> >  >> What I meant is, using it with the Elisp implementation didn't
> >  >> convince me of the usefulness of the feature. Perhaps you disagree?
> >  >>
> >  >> Or it would be nice to hear from someone who have tried out Eglot's
> >  >> integration and found more upsides there.
> >  >
> >  > If you want that to happen, the only realistic way to have good
> >  > feedback from anyone else other than emacs-devel nerds like me and you
> >  > is to release an Eglot version with  this feature, which we can change
> >  > later (even non-backward-compatibly, within a reasonable time frame).
> >
> >  I tried the Eglot integration, my comments elsewhere are inspired by
> >  that.
> >
> > Can you point me to this "elsewhere"?
>
> Elsewhere in this thread - my message on Tue, 07 Nov 2023 11:51:49 -0500

OK, I found it.  It's useful, and very commonly done, if you CC the
participants in this discussion, particularly when addressing their
contributions.
Your emails are sent only to emacs-devel and it's easier for me to
miss them.

> >  I found it rather clumsy out of the box.
> >
> > What exactly is clumsy? Please state the command you used and what the clumsy effects were.
>
> Specifically xref-find-extras is clumsy, because it's annoying to have
> to type the full "kind" in to request a specific "kind" of definition.

I don't find it annoying.  But since we settled long ago that there can
be no standard set of all kinds that makes sense for all languages, then
I don't see a better way than what the current xref patch does now: asking
the backend for kinds and then having one command with an additional selection
step.  The little change I did to Dmitry's xref.el allows particular modes
such as Eglot to implement their own "quick-access" commands for a particular
kind more easily.

> eglot-find-{declaration,implementation,typeDefinition} is good as
> always.

Right.  And they're not going away anytime soon.
>
> >  However, it works well with an
> >  approach of adding bindings for individual kinds, as is currently how I
> >  (and other Eglot users at Jane Street, and presumably most other Eglot
> >  users everywhere) use Eglot.  So that compatibility with an
> >  already-common approach is an upside of this integration.
> >
> > Bindings to what commands exactly?
>
> eglot-find-{declaration,implementation,typeDefinition}
>
> >  However, the alternative UI which shows all kinds of definitions in a
> >  single buffer does not exist for Eglot in the same way it exists for
> >  Elisp.  So... I can't really compare it to that.
> >
> > What "alternative UI" are you talking about exactly? How can I trigger it for Elisp? Please give a full example as I don't know what
> > you are taking about.
>
> In Elisp, M-. (xref-find-definitions) already shows all the kinds of
> definitions that are requestable through xref-find-extras.  But there's
> no similar way to show all kinds of definitions for Eglot - M-. only
> includes the proper definition, not the declaration and typeDefinition
> and so on.

As it always did.  And you yourself wrote

 > 90% of the time, in 90%
 > of languages, there's a clear meaning of "definition" and I want M-. to
 > take me there directly rather than prompting in any way.

And I fully agree, btw.  So I don't understand why you sometimes want
prompting and sometimes no prompting.

But you can discuss that with Dmitry about the UI parts. I'm just
really interested
in the xref.el <-> eglot.el API.  I want it to be as simple and
versatile as possible,
much like it is now.

> I know you would prefer not to implement such a thing, I'm just
> explaining for Dmitry that since this kind of UI doesn't exist, I can't
> compare xref-find-extras against that kind of UI for Eglot in the same
> way that I can with Elisp.

It's not really about not wanting.  It's an incompatible change.  Users
used to M-. on an identifier and going to the definition would now
see an *xref* buffer to click on.  And it would break the only command
where C-u M-. to prompt for the identifier first can possibly work in LSP.

But, I think your idea that passing nil (or 't' or 'all') as KIND
to xref-find-extra makes it ask xref-backend-extra-defs (which should
be renamed to not have that misleading "defs") for all kinds previously
reported by the backend in xref-backend-extra-kinds.

Then the UI can organize the results as it sees fit, and I don't have
any strong opinions on how it should do that, nor any strong opinions
as to how many C-u or variables should modify the new command behaviour.

That's not to say I don't have opinions, just not opinions strong enough to
justify not pushing a draft version of the current patch into master.

João

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Wed, Nov 8, 2023 at 10:59 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>
> On 08/11/2023 03:19, João Távora wrote:
> > On Wed, Nov 8, 2023 at 12:33 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
> >>
> >> On 08/11/2023 02:21, João Távora wrote:
> >>> On Tue, Nov 7, 2023 at 11:18 PM Dmitry Gutov<dgutov@yandex.ru>  wrote:
> >>>
> >>>> Joao didn't want to have such definitions in Xref. I'm still on the
> >>>> fence, personally.
> >>> What don't I want?  I haven't read this wordy subthread but it seems
> >>> Eshel is proposing exactly what I changed in xref-find-extra: take "kind"
> >>> as argument.
> >>
> >> Sorry: you didn't want those definitions in _Eglot_. Not in Xref, that
> >> was a mistype.
> >
> > Ah. Well, first, it a fact that I _have_ them in Eglot, and I'll
> > keep having them, because they've been there a long time.
>
> The normal practice is to wait a little while until the new "core"
> feature is available for a lot of users (perhaps skipping a few Emacs
> versions, or - more bravely - wait for the next one and then just raise
> the minimum xref version required), and then declare the built-in
> versions obsolete in favor of the upstream.
>
> The period before obsoletion is not as important as the agreement to do
> that - because then we will be talking whether the "proper"
> functionality should be shaped as a number of new commands (one per
> kind), or a dispatching command, or both. Or something else.
>
> For the purpose of such design discussion, simply using Eglot's commands
> is not a good choice.

I don't get this talk of obsoletion.  I don't want to obsolete anything.
The Eglot commands are there because they fulfill a need that
generic xref.el commands will never be able to fulfill.

> > And it's not really relevant that I didn't _want_ them.  If I had
> > the choice back then between recommending existing xref commands
> > and adding new commands to Eglot, of course I wouldn't have added
> > them.  But I hadn't, so I did. Pretty simple.  So I don't understand
> > the argument you're making against Eshel's and my suggestion.
>
> Since you didn't want them and argued that they shouldn't be in Eglot,
> it seems natural to assume that you would be happy to obsolete them in
> favor of built-in functionality.

You assume wrong, or at least it is not something really relevant
to this discussion.  If xref-find-extras had been around at the time, I
would have argued that users could use that or simply that xref
command-defining macro to make their own johndoe/find-thingy commads.
Or major modes could do it, perhaps even better.

But neither was available at the time, so I did those commands and they won't
be obsoleted any time soon.

> >>>> Or it would be nice to hear from someone who have tried out Eglot's
> >>>> integration and found more upsides there.
> >>> If you want that to happen, the only realistic way to have good
> >>> feedback from anyone else other than emacs-devel nerds like me and you
> >>> is to release an Eglot version with  this feature, which we can change
> >>> later (even non-backward-compatibly, within a reasonable time frame).
> >> That seems like a last-resort type of approach, for this particular
> >> discussion.
> >
> > Not really, just being pragmatic and suggesting a way for you to
> > get the opinions from Eglot users that you wrote you wanted.  There
> > isn't a significant amount of engaged Eglot users in this discussion
> > that are going to try the patch.
>
> Feedback is good, but I prefer to arrive at some sound design first,
> even if it doesn't do everything anyone might want. And then test it on
> a wider audience.

The base of the current design, the xref-backend-extra-defs and
xref-backend-extra-kinds, is more than sound enough. IMO could
some micro-enhancements like better names and support for non-string
kinds, but that's completely secondary.

I also think the xref-find-extra you created and I tweaked is pretty
good.  completing-read is definitely the correct choice to select
the kind.  Not clumsy at all IMHO.  But the ideas you've been offered
(by Spencer?) about enhancing it via a prefix argument or extra variable to
ask for all kinds all at once are pretty good too.  And you can do this
later, probably even backward-compatibly.

João Távora

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 09/11/2023 01:22, João Távora wrote:
> On Wed, Nov 8, 2023 at 10:59 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>>
>> On 08/11/2023 03:19, João Távora wrote:
>>> On Wed, Nov 8, 2023 at 12:33 AM Dmitry Gutov <dgutov@yandex.ru> wrote:
>>>>
>>>> On 08/11/2023 02:21, João Távora wrote:
>>>>> On Tue, Nov 7, 2023 at 11:18 PM Dmitry Gutov<dgutov@yandex.ru>  wrote:
>>>>>
>>>>>> Joao didn't want to have such definitions in Xref. I'm still on the
>>>>>> fence, personally.
>>>>> What don't I want?  I haven't read this wordy subthread but it seems
>>>>> Eshel is proposing exactly what I changed in xref-find-extra: take "kind"
>>>>> as argument.
>>>>
>>>> Sorry: you didn't want those definitions in _Eglot_. Not in Xref, that
>>>> was a mistype.
>>>
>>> Ah. Well, first, it a fact that I _have_ them in Eglot, and I'll
>>> keep having them, because they've been there a long time.
>>
>> The normal practice is to wait a little while until the new "core"
>> feature is available for a lot of users (perhaps skipping a few Emacs
>> versions, or - more bravely - wait for the next one and then just raise
>> the minimum xref version required), and then declare the built-in
>> versions obsolete in favor of the upstream.
>>
>> The period before obsoletion is not as important as the agreement to do
>> that - because then we will be talking whether the "proper"
>> functionality should be shaped as a number of new commands (one per
>> kind), or a dispatching command, or both. Or something else.
>>
>> For the purpose of such design discussion, simply using Eglot's commands
>> is not a good choice.
> 
> I don't get this talk of obsoletion.  I don't want to obsolete anything.
> The Eglot commands are there because they fulfill a need that
> generic xref.el commands will never be able to fulfill.

If Eglot keeps its commands (which I don't mind that much), then any 
comparable package should also be allowed and even encouraged to define 
its own commands. And we should document that somewhere, too.

But that design ends up far from the original "Eglot shouldn't do this" 
sentiment that I have read before.

And also, Eglot keeping its commands seems incompatible with (or at 
least counter to) the approach where is an existing set of "extra" 
commands that is bound to some prefix map (e.g. one assigned to M-'). 
Because if there are many different commands called *-find-declarations, 
it seems difficult to put them all on the same key.

>>> And it's not really relevant that I didn't _want_ them.  If I had
>>> the choice back then between recommending existing xref commands
>>> and adding new commands to Eglot, of course I wouldn't have added
>>> them.  But I hadn't, so I did. Pretty simple.  So I don't understand
>>> the argument you're making against Eshel's and my suggestion.
>>
>> Since you didn't want them and argued that they shouldn't be in Eglot,
>> it seems natural to assume that you would be happy to obsolete them in
>> favor of built-in functionality.
> 
> You assume wrong, or at least it is not something really relevant
> to this discussion.  If xref-find-extras had been around at the time, I
> would have argued that users could use that or simply that xref
> command-defining macro to make their own johndoe/find-thingy commads.
> Or major modes could do it, perhaps even better.
> 
> But neither was available at the time, so I did those commands and they won't
> be obsoleted any time soon.

Reasons being..?

>>>>>> Or it would be nice to hear from someone who have tried out Eglot's
>>>>>> integration and found more upsides there.
>>>>> If you want that to happen, the only realistic way to have good
>>>>> feedback from anyone else other than emacs-devel nerds like me and you
>>>>> is to release an Eglot version with  this feature, which we can change
>>>>> later (even non-backward-compatibly, within a reasonable time frame).
>>>> That seems like a last-resort type of approach, for this particular
>>>> discussion.
>>>
>>> Not really, just being pragmatic and suggesting a way for you to
>>> get the opinions from Eglot users that you wrote you wanted.  There
>>> isn't a significant amount of engaged Eglot users in this discussion
>>> that are going to try the patch.
>>
>> Feedback is good, but I prefer to arrive at some sound design first,
>> even if it doesn't do everything anyone might want. And then test it on
>> a wider audience.
> 
> The base of the current design, the xref-backend-extra-defs and
> xref-backend-extra-kinds, is more than sound enough. IMO could
> some micro-enhancements like better names and support for non-string
> kinds, but that's completely secondary.

It's not secondary if people will start adapting their backends to it.

E.g. the term "extra" seems more like a misnomer now, given that people 
seem to want that command to include the kinds already present in 
"definitions". And they might constitute the majority of those "kinds".

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 08/11/2023 19:25, Spencer Baugh wrote:

>>> This (and Joao's patch adding support for eglot) is very
>>> interesting,
>>> thank you for implementing it!
>>> We also discussed a UI which shows all kinds of definitions in a
>>> single
>>> buffer.  Maybe the prompt for KIND should default to "all" which shows
>>> all kinds of definitions?
>>
>> The natural question is whether 'xref-find-definitions' would be that
>> UI, and if not, why not.
> 
> As in, xref-find-definitions (M-.) *by default* shows all kinds of
> definitions?  That seems bad for one big reason: 90% of the time, in 90%
> of languages, there's a clear meaning of "definition" and I want M-. to
> take me there directly rather than prompting in any way.  But there are
> other kinds of "definitions" which plausibly I could want, just
> sometimes, not by default.

Fair enough. So "definitions" are the places we want to see 90% of the 
time when learning about a funciton or a variable. And the other 
reference kinds (BTW, what to call that? "definition kinds"? "reference 
kinds"?) must be apparently less useful.

Do you know which category does "eglot-find-typeDefinition" falls into, 
and why? Aside from the fact that it, historically, uses a separate 
endpoint.

> If you mean something a non-default version of xref-find-definitions, or
> another binding for another basically-identical command, sure, I think
> that would be good.  We can have both that command *and* have a command
> which prompts for KIND and defaults to "all".

We could indeed, if we decide what to call it. "extras" seems out (since 
it would include both definitions and additional reference kinds). Just 
"xref-find-by-kind"? Then it's less obvious to have the default behavior 
showing all.

>> Also, if "references" are included in the list of kinds (as the
>> current patch for Eglot does, I think), then "show all kinds" is not
>> likely to be very useful -- all will drown in "references".
> 
> True, interesting point.  I wonder if we could reasonably distinguish
> "definition" kinds (a small number per identifier) from "reference"
> kinds (no limit on how many per identifier).

OTOH, maybe "declarations" are a little closer to "references"? E.g. in 
Elisp the declare-function forms seem to be more in that category. And 
if we're talking about C/C++ forward declarations, I suppose that 
depends on whether the declaration is for an "extern"

 > Although I suppose there's
 > also no limit on how many implementations of a generic function there
 > can be, and that's definitely something that fits in "definitions".

The fact that there are often many definitions for a generic function 
indeed is an argument toward including more "kinds" into 
xref-find-definitions' output by default. But only as many as it would 
be actually useful, especially if the kinds are mutually exclusive (a 
symbol is either one or the other) or intersect very rarely, or can be 
distinguished by context.

> Maybe the backend could decide what kinds get included in "all".  Then
> it could deliberately avoid including anything "reference-like".

Could we want several such commands? E.g. one for "all definition-like 
hits" and another for "all reference-like hits"? With separate sets of 
kinds for definitions and references?

>>>> Or perhaps the main value would be in actually having separate
>>>> commands which could be bound to a submap with faster key sequences
>>>> and/or to the context menu (with context-menu-mode on). Then our
>>>> solution should be more in line with either defining a number of
>>>> additional named commands (for mode universal kinds) and/or making it
>>>> easier to define new such commands for users/3rd-party packages/etc.
>>> That's an interesting idea.  So maybe M-' (or whatever) could be a
>>> new
>>> prefix for a prefix-map which contains both universal and mode-specific
>>> kinds of lookups.
>>> So maybe something which looks kinda like (not suggesting final
>>> bindings, just trying to feel out what it would be like):
>>> generic eglot-find-implementation:   M-' i
>>> generic eglot-find-declaration:      M-' d
>>> generic eglot-find-typeDefinition:   M-' t
>>
>> Then we will more-or-less nail down the whole set of "available Xref
>> kinds" in the core by having these commands defined.
>>
>> That's not very flexible, but if the set doesn't actually change too
>> much, and doesn't vary between languages a lot, it could work. Would
>> lead to a more straightforward design, too.
> 
> Hm, why do you say this?  This approach seems very extensible to me -
> modes could add their own bindings/commands/kinds for things specific to
> that mode.  But they build on top of a set of common kinds which are
> provided by the core, which don't vary between language too much.

It is less flexible because any time a backend wants to use a new 
"kind", it will need to expend some effort and add it to the core 
somehow. Define a new command or two and assign them to the said prefix 
map. This could also lead to conflicts if backend authors don't do this 
carefully enough.

> I do think that if we go with an API which has any notion of "kinds", we
> should have some standard "kinds" in the core like implementation,
> declaration, type-definition.  I don't see any reason not to do that.

We could have a "registry" of kinds, associating each of them with a 
key. Then the result could be more dynamic, e.g. M-' could be bound to a 
command that reads the key and performs the dispatch to the 
corresponding search (similar to project-switch-project).

And the users (though probably not packages) would later be able to 
customize that mapping, adding new kinds or modifying the keys.

With this approach we don't end up with many xref-find-xyz commands, 
most of which just clutter the namespace, staying unused for a large 
proportion of the users. OTOH, one wouldn't be able to examine the 
prefix map and its contents and definitions with (M-' C-h) -- that's a 
minor reduction in usability.

>>> xref-extra-kind, prompting for kind: M-' M-'
>>
>> Would we need this command, if we had separate commands for each kind
>> already?
> 
> This would support kinds which:
> - are language-specific,
> - or are more rarely used and don't need a dedicated command,
> - or are both.

What would be the more frequently used, less language-specific commands?

If we could agree on such list, we could indeed have those 
xref-find-a/b/c definitions and a command with completing-read for the rest.

>>> xref-extra-kind, showing all:        M-' a
>>> And if there was something mode-specific, like the Java overriding
>>> method thing, it could be e.g. M-' o
>>
>> Most of them are mode-specific already. Some languages don't have
>> separate "declarations", some don't know what "type definitions" are,
>> and some, indeed, don't have method overrides.
> 
> Most languages have something that can fit into "declaration" or "type
> definitions", though.
> 
> For Elisp generics, find-implementation could show the cl-defmethod, and
> find-declaration could jump to the cl-defgeneric.  I would love that,
> actually - I already find it rather annoying to have to navigate in the
> xref buffer to the cl-defgeneric when I M-. on a generic method, if I
> know up front that I want to jump to the cl-defgeneric.

Interesting. I usually find cl-defgeneric entries to be useful in that 
list. But we *are* able to distinguish those that don't contain the 
default implementation.

> Actually, this example has just convinced me that I definitely want
> "kind-specific commands", even for Elisp.  That would be great.
> 
> For functions without separate declarations, find-implementation and
> find-declaration could jump to the same place.  In some languages, maybe
> that's just always what happens.

Aren't find-implementation included in find-definition anyway? Or *are* 
the same as find-definition, for example, in LSP's approach? Aside from 
the fact that the latter also works for variables and other symbols.

> As for type definitions - even Elisp has things that could reasonably be
> called type definitions, we just don't support jump-to-definition for
> them right now.  I have no idea how we'd implement it, but we could
> support jumping to cl-defstruct definitions, for example.  Maybe we
> could use information from native compilation to guess the type of the
> identifier at point?  But anyway, for now we'd just error if the user
> ran such a command in emacs-lisp-mode or other modes that don't support
> jumping to the type definition.

I think the aforementioned type definitions in Elisp would be better 
included in the "find definitions" set because they generally don't 
clash with other kinds.

>>> - users could still use completing-read to type the kind
>>> Plus, if we do use M-' or any other short binding for this, we
>>> should
>>> almost certainly make it the start of a new prefix-map rather than bind
>>> M-' directly to the new command; doing otherwise would be wasteful of
>>> valuable keybinding space.
>>
>> If we're going to have separate commands for kinds, that is indeed a
>> good idea.
> 
> I almost want to say that we should have it be a prefix regardless of
> whether we have separate commands for kinds.  I guess it depends:

You probably meant "have it be the binding".

> - if we use C-M-?, that's already such a hard key to hit that maybe it's
>    okay if we bind it directly to a command
> 
> - if we use M-' or something similarly convenient, it would be really
>    tragic to not reclaim all that premium keybinding space.

It's possible that it would work better the other way around: if we 
discover that people really want a prefix map, then it would be easier 
to argue for an easy-to-hit binding like the above.

> (And
>    probably if we use M-' we will need to put the old command on
>    something under M-' anyway)

I think it would be odd to put an 'abbrev-...' command inside an 
Xref-specific prefix map. But I suppose reserving the "M-' M-'" sequence 
for this is not out of the question.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 09/11/2023 02:08, Dmitry Gutov wrote:
> Do you know which category does "eglot-find-typeDefinition" falls into, 
> and why? Aside from the fact that it, historically, uses a separate 
> endpoint.

Hm, according to 
https://github.com/rust-lang/rust-analyzer/issues/2541#issuecomment-565166660, 
"find-typeDefinition" goes to the definition of the type of the current 
variable (at point).

That probably means it doesn't really fit with the rest of the commands 
we're discussing, e.g. because the identifier it works on is actually a 
type name (so the completion should be across types? ideally, at least, 
if LSP supported that). Anyway, it shouldn't be mixed into "find 
definition", at least if we go by rust-analyzer's meaning of it.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Wed, Nov 8, 2023 at 11:34 PM Dmitry Gutov <dgutov@yandex.ru> wrote:

> >> For the purpose of such design discussion, simply using Eglot's commands
> >> is not a good choice.
> >
> > I don't get this talk of obsoletion.  I don't want to obsolete anything.
> > The Eglot commands are there because they fulfill a need that
> > generic xref.el commands will never be able to fulfill.
>
> If Eglot keeps its commands (which I don't mind that much), then any
> comparable package should also be allowed and even encouraged to define
> its own commands. And we should document that somewhere, too.

Not necessarily encouraged OR discouraged.  Just possible.  Why
complicate things?

> But that design ends up far from the original "Eglot shouldn't do this"
> sentiment that I have read before.

It might be useful to check this code in eglot.el.  Code actions are
very similar to what we have with definition finding.  Look:

  (eglot--code-action eglot-code-action-organize-imports
"source.organizeImports")
  (eglot--code-action eglot-code-action-extract "refactor.extract")
  (eglot--code-action eglot-code-action-inline "refactor.inline")
  (eglot--code-action eglot-code-action-rewrite "refactor.rewrite")
  (eglot--code-action eglot-code-action-quickfix "quickfix")

This is exactly the same, with the exception that refactor.el doesn't
yet exist (working on it) and 'eglot--code-action' is thus not
refactor-define-code-action

Now, I don't use those commands, I use eglot-code-actions directly or
Flymake clicky diagnostics, but users requested those commands, so I put
them in eglot.el.  They could be in some eglot-ui.el.  But it doesn't
mean eglot-code-action (soon to be refactor-code-action and which, like
xref-find-extras, uses completing-read) is any less useful or
incompatible with the commands created by these one-liners.

To make myself clear: when I say I don't like to add UI to eglot.el,
it means I don't like to do it when a clean abstraction for a similar
concept can be cleanly implemented, or even already exists, elsewhere
for other non-LSP things.  But sometimes those two conditions simply
aren't true and can't easily be made true, like here.  So instead
of endless soul-searching about what I would like a zillion language
servers and an Microsoft-owoned VScodish couldnt-care-less-for-Emacs
protocol to look like, I create the simplest possible Eglot UI and
reuse as much code of non-Eglot UI libraries as possible.

> And also, Eglot keeping its commands seems incompatible with (or at
> least counter to) the approach where is an existing set of "extra"
> commands that is bound to some prefix map (e.g. one assigned to M-').
> Because if there are many different commands called *-find-declarations,
> it seems difficult to put them all on the same key.

Eglot doesn't put any commands in any keymap.  It just offers them.

> > But neither was available at the time, so I did those commands and they won't
> > be obsoleted any time soon.
>
> Reasons being..?

That we can't come up with alternatives to exactly that interface,
obviously.  When you do, I'll obsolete them.  But do you want to hardcode
things like LSP "typeDefinition" and Sly's "who-macroexpands" somewhere
in xref.el?  How would that work?

> > The base of the current design, the xref-backend-extra-defs and
> > xref-backend-extra-kinds, is more than sound enough. IMO could
> > some micro-enhancements like better names and support for non-string
> > kinds, but that's completely secondary.
>
> It's not secondary if people will start adapting their backends to it.

If that's even a problem -- like there's going to be a plethora
of backends doing that (how many third party xref backends are there?)
-- strings-for-kinds can still be supported and  you have obsolete alias.
So yes, completely secondary.

> E.g. the term "extra" seems more like a misnomer now, given that people
> seem to want that command to include the kinds already present in
> "definitions". And they might constitute the majority of those "kinds".

Yeah, "extra" should be or "by-kind".

João

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Thu, Nov 9, 2023 at 12:27 AM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 09/11/2023 02:08, Dmitry Gutov wrote:
> > Do you know which category does "eglot-find-typeDefinition" falls into,
> > and why? Aside from the fact that it, historically, uses a separate
> > endpoint.
>
> Hm, according to
> https://github.com/rust-lang/rust-analyzer/issues/2541#issuecomment-565166660,
> "find-typeDefinition" goes to the definition of the type of the current
> variable (at point).

That's rust-analyzer interpretation.  In practice, servers can do what they
want.  This is the LSP description

    The go to type definition request is sent from the client to the server
    to resolve the type definition location of a symbol at a given text
    document position.

For example, an Elisp language server could use that to go to the place
where the  `typeDefinition` of a given symbol is set on that symbol.

> That probably means it doesn't really fit with the rest of the commands
> we're discussing, e.g. because the identifier it works on is actually a
> type name

Not in rust-analyzer, at least judging by what you described.  Seems
like the identifier it works on is a variable name, which rust-analyzer
then associates with the type (being a strongly typed language it knows
that) and then you get to the place where that type is defined.
Super-reasonable, for rust.

> (so the completion should be across types? ideally, at least,
> if LSP supported that). Anyway, it shouldn't be mixed into "find
> definition", at least if we go by rust-analyzer's meaning of it.

Actually, no.   Precisely in rust-analyzer's case it could be mixed.
I see somewhere a reference to a variable, I do that command you're thinking
of and get the definition of the variable and the definition of the type of the
variable, both useful things.

But you're right that it won't make sense for every possible
conceivable language or server.  But just don't worry.  Leave that
concern to server developers and LSP users.  You're not going to
magically fix LSP with the xref.el feature.  The Emacs-LSP marriage
is never quite perfect, but it gets a lot of love.

Exactly like any good marriage should :-)

João

CCing thread participants through gnus+gmane

[Spencer Baugh]

João Távora <joaotavora@gmail.com> writes:
> On Wed, Nov 8, 2023 at 5:45 PM Spencer Baugh <sbaugh@janestreet.com> wrote:
>>
>> João Távora <joaotavora@gmail.com> writes:
>>
>> > On Wed, Nov 8, 2023, 16:40 Spencer Baugh <sbaugh@janestreet.com> wrote:
>> >
>> >  João Távora <joaotavora@gmail.com> writes:
>> >  > On Tue, Nov 7, 2023 at 11:18 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>> >  >> What I meant is, using it with the Elisp implementation didn't
>> >  >> convince me of the usefulness of the feature. Perhaps you disagree?
>> >  >>
>> >  >> Or it would be nice to hear from someone who have tried out Eglot's
>> >  >> integration and found more upsides there.
>> >  >
>> >  > If you want that to happen, the only realistic way to have good
>> >  > feedback from anyone else other than emacs-devel nerds like me and you
>> >  > is to release an Eglot version with  this feature, which we can change
>> >  > later (even non-backward-compatibly, within a reasonable time frame).
>> >
>> >  I tried the Eglot integration, my comments elsewhere are inspired by
>> >  that.
>> >
>> > Can you point me to this "elsewhere"?
>>
>> Elsewhere in this thread - my message on Tue, 07 Nov 2023 11:51:49 -0500
>
> OK, I found it.  It's useful, and very commonly done, if you CC the
> participants in this discussion, particularly when addressing their
> contributions.
> Your emails are sent only to emacs-devel and it's easier for me to
> miss them.

Sorry about that... I fully agree about CCing, but for $REASONS I have
to use gmane to interact with emacs-devel.  Does anyone happen to know
if there's a way to get gmane+gnus to CC participants?

(If there's not a way to do that, I should really look into not using
gmane...  Maybe I can use a public-inbox mirror of emacs-devel like
https://yhetil.org/emacs-devel/ )

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

João Távora <joaotavora@gmail.com> writes:
> On Wed, Nov 8, 2023 at 11:34 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>> > But neither was available at the time, so I did those commands and they won't
>> > be obsoleted any time soon.
>>
>> Reasons being..?
>
> That we can't come up with alternatives to exactly that interface,
> obviously.  When you do, I'll obsolete them.  But do you want to hardcode
> things like LSP "typeDefinition" and Sly's "who-macroexpands" somewhere
> in xref.el?  How would that work?

Right, so if we have kinds defined in the core for "declaration",
"implementation", and "type-definition" (maybe not with those exact
names), then we can have xref-find-declaration,
xref-find-implementation, and xref-find-type-definition (again maybe not
with those exact names), and just do

(define-obsolete-function-alias 'eglot-find-declaration 'xref-find-declaration)
(define-obsolete-function-alias 'eglot-find-implementation 'xref-find-implementation)
(define-obsolete-function-alias 'eglot-find-typeDefinition 'xref-find-type-definition)

Is that possible?

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

João Távora <joaotavora@gmail.com> writes:

> On Wed, Nov 8, 2023 at 5:45 PM Spencer Baugh <sbaugh@janestreet.com> wrote:
>> João Távora <joaotavora@gmail.com> writes:
>> > On Wed, Nov 8, 2023, 16:40 Spencer Baugh <sbaugh@janestreet.com> wrote:
>> >  I found it rather clumsy out of the box.
>> >
>> > What exactly is clumsy? Please state the command you used and what the clumsy effects were.
>>
>> Specifically xref-find-extras is clumsy, because it's annoying to have
>> to type the full "kind" in to request a specific "kind" of definition.
>
> I don't find it annoying.  But since we settled long ago that there can
> be no standard set of all kinds that makes sense for all languages, then
> I don't see a better way than what the current xref patch does now: asking
> the backend for kinds and then having one command with an additional selection
> step.  The little change I did to Dmitry's xref.el allows particular modes
> such as Eglot to implement their own "quick-access" commands for a particular
> kind more easily.

Here is the better way: have both a standard set of kinds, and the
ability for a given backend to make their own custom kinds.

A "standard set of kinds" would just be standard kind symbols.  Instead
of Eglot returning 'eglot--xref-declaration as a supported kind, it
would return 'declaration.  And we could have a standard command with a
standard binding which requests definitions of kind 'declaration.

The only change to the current patch would be changing
'eglot--xref-declaration to 'declaration, and adding an
xref-find-declaration.

Custom kinds would still require you to type in the kind symbol.  (Or a
backend or mode can make a "quick-access" command specific to that kind,
if they want.)

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

Dmitry Gutov <dmitry@gutov.dev> writes:
> On 08/11/2023 19:25, Spencer Baugh wrote:
>
>>>> This (and Joao's patch adding support for eglot) is very
>>>> interesting,
>>>> thank you for implementing it!
>>>> We also discussed a UI which shows all kinds of definitions in a
>>>> single
>>>> buffer.  Maybe the prompt for KIND should default to "all" which shows
>>>> all kinds of definitions?
>>>
>>> The natural question is whether 'xref-find-definitions' would be that
>>> UI, and if not, why not.
>> As in, xref-find-definitions (M-.) *by default* shows all kinds of
>> definitions?  That seems bad for one big reason: 90% of the time, in 90%
>> of languages, there's a clear meaning of "definition" and I want M-. to
>> take me there directly rather than prompting in any way.  But there are
>> other kinds of "definitions" which plausibly I could want, just
>> sometimes, not by default.
>
> Fair enough. So "definitions" are the places we want to see 90% of the
> time when learning about a funciton or a variable. And the other
> reference kinds (BTW, what to call that? "definition kinds"?
> "reference kinds"?) must be apparently less useful.

Right.

> Do you know which category does "eglot-find-typeDefinition" falls
> into, and why? Aside from the fact that it, historically, uses a
> separate endpoint.

It's on the less-useful, more-rarely-used side.

>> If you mean something a non-default version of xref-find-definitions, or
>> another binding for another basically-identical command, sure, I think
>> that would be good.  We can have both that command *and* have a command
>> which prompts for KIND and defaults to "all".
>
> We could indeed, if we decide what to call it. "extras" seems out
> (since it would include both definitions and additional reference
> kinds). Just "xref-find-by-kind"? Then it's less obvious to have the
> default behavior showing all.

Actually, maybe this should be a backend-specific thing.  The backend
could specify a default along with its list of kinds.  And if we request
that default kind (which might be 'all), then the backend will decide
what kinds to send us.

That neatly avoids us having to make any kind of design decisions about
the semantic meaning of different kinds, and what sorts of kinds exist,
and all that.

(Other than having a few standard kinds, which I still think we should
do, if only for the sake of getting the eglot-find-* commands out of
eglot and into xref.  But we don't have to really think about what those
standard kinds mean semantically)

>>> Also, if "references" are included in the list of kinds (as the
>>> current patch for Eglot does, I think), then "show all kinds" is not
>>> likely to be very useful -- all will drown in "references".
>> True, interesting point.  I wonder if we could reasonably
>> distinguish
>> "definition" kinds (a small number per identifier) from "reference"
>> kinds (no limit on how many per identifier).
>
> OTOH, maybe "declarations" are a little closer to "references"?
> E.g. in Elisp the declare-function forms seem to be more in that
> category. And if we're talking about C/C++ forward declarations, I
> suppose that depends on whether the declaration is for an "extern"
>
>> Although I suppose there's
>> also no limit on how many implementations of a generic function there
>> can be, and that's definitely something that fits in "definitions".
>
> The fact that there are often many definitions for a generic function
> indeed is an argument toward including more "kinds" into
> xref-find-definitions' output by default. But only as many as it would
> be actually useful, especially if the kinds are mutually exclusive (a
> symbol is either one or the other) or intersect very rarely, or can be
> distinguished by context.

I suggest this should be up to the backend.  IMO xref-find-definitions
should stay fast in the common case, but sometimes it will unavoidably
have to include multiple things, like for cl-defgenerics in Elisp.  We
can't make the decision about how and when to do that for every backend.

Actually, it occurs to me that if we had an xref-find-implementations
command from the start, with a convenient binding, maybe
xref-find-definitions would just only show the cl-defgeneric, and jump
to it right away.  And only if you hit xref-find-implementations would
you jump to the cl-defmethods.  We can't make that change now, but I
don't think it would be worse!  And if a backend wants a design like
that, I think the backend should be able to have it.

>> Maybe the backend could decide what kinds get included in "all".  Then
>> it could deliberately avoid including anything "reference-like".
>
> Could we want several such commands? E.g. one for "all definition-like
> hits" and another for "all reference-like hits"? With separate sets of
> kinds for definitions and references?

Possibly, but my suggestion is that "all definition-like hits" and "all
reference-like hits" should just be kinds exposed by the backend.
*Maybe* we'll have them be "standard kinds" with a command and binding
by default, but maybe not.  (We could always add more standard kinds and
commands and bindings later.)

>>>>> Or perhaps the main value would be in actually having separate
>>>>> commands which could be bound to a submap with faster key sequences
>>>>> and/or to the context menu (with context-menu-mode on). Then our
>>>>> solution should be more in line with either defining a number of
>>>>> additional named commands (for mode universal kinds) and/or making it
>>>>> easier to define new such commands for users/3rd-party packages/etc.
>>>> That's an interesting idea.  So maybe M-' (or whatever) could be a
>>>> new
>>>> prefix for a prefix-map which contains both universal and mode-specific
>>>> kinds of lookups.
>>>> So maybe something which looks kinda like (not suggesting final
>>>> bindings, just trying to feel out what it would be like):
>>>> generic eglot-find-implementation:   M-' i
>>>> generic eglot-find-declaration:      M-' d
>>>> generic eglot-find-typeDefinition:   M-' t
>>>
>>> Then we will more-or-less nail down the whole set of "available Xref
>>> kinds" in the core by having these commands defined.
>>>
>>> That's not very flexible, but if the set doesn't actually change too
>>> much, and doesn't vary between languages a lot, it could work. Would
>>> lead to a more straightforward design, too.
>> Hm, why do you say this?  This approach seems very extensible to me
>> -
>> modes could add their own bindings/commands/kinds for things specific to
>> that mode.  But they build on top of a set of common kinds which are
>> provided by the core, which don't vary between language too much.
>
> It is less flexible because any time a backend wants to use a new
> "kind", it will need to expend some effort and add it to the core
> somehow. Define a new command or two and assign them to the said
> prefix map. This could also lead to conflicts if backend authors don't
> do this carefully enough.

No, I suggest that xref-backend-extra-kinds should be able to return
whatever arbitrary symbols it wants.  Just, if its "kinds" match the
kinds used in the core, then it will benefit from the commands and
bindings defined by the core.

Perfectly extensible, but still benefiting from standardization.

>> I do think that if we go with an API which has any notion of "kinds", we
>> should have some standard "kinds" in the core like implementation,
>> declaration, type-definition.  I don't see any reason not to do that.
>
> We could have a "registry" of kinds, associating each of them with a
> key. Then the result could be more dynamic, e.g. M-' could be bound to
> a command that reads the key and performs the dispatch to the
> corresponding search (similar to project-switch-project).
>
> And the users (though probably not packages) would later be able to
> customize that mapping, adding new kinds or modifying the keys.
>
> With this approach we don't end up with many xref-find-xyz commands,
> most of which just clutter the namespace, staying unused for a large
> proportion of the users. OTOH, one wouldn't be able to examine the
> prefix map and its contents and definitions with (M-' C-h) -- that's a
> minor reduction in usability.

The only difference between this and a regular keymap which maps keys to
commands is:

> we don't end up with many xref-find-xyz commands

But we can also avoid that by just... not defining many such commands
and kinds.  A mode will always have the ability to define its own
mode-specific kinds in the mode's own namespace, which don't clutter
xref-*.

>>>> xref-extra-kind, prompting for kind: M-' M-'
>>>
>>> Would we need this command, if we had separate commands for each kind
>>> already?
>> This would support kinds which:
>> - are language-specific,
>> - or are more rarely used and don't need a dedicated command,
>> - or are both.
>
> What would be the more frequently used, less language-specific commands?
>
> If we could agree on such list, we could indeed have those
> xref-find-a/b/c definitions and a command with completing-read for the
> rest.

I suggest that we should attempt to provide the following standard
kinds:

'declaration
'implementation
'type-definition

I personally think these would work well for a wide range of languages.
Yes, these are the ones which LSP has; but just because they're from
LSP, doesn't mean they're completely wrong.  They seem to me to strike
the right balance of commonality between languages.

>>>> xref-extra-kind, showing all:        M-' a
>>>> And if there was something mode-specific, like the Java overriding
>>>> method thing, it could be e.g. M-' o
>>>
>>> Most of them are mode-specific already. Some languages don't have
>>> separate "declarations", some don't know what "type definitions" are,
>>> and some, indeed, don't have method overrides.
>> Most languages have something that can fit into "declaration" or
>> "type
>> definitions", though.
>> For Elisp generics, find-implementation could show the cl-defmethod,
>> and
>> find-declaration could jump to the cl-defgeneric.  I would love that,
>> actually - I already find it rather annoying to have to navigate in the
>> xref buffer to the cl-defgeneric when I M-. on a generic method, if I
>> know up front that I want to jump to the cl-defgeneric.
>
> Interesting. I usually find cl-defgeneric entries to be useful in that
> list. But we *are* able to distinguish those that don't contain the
> default implementation.
>
>> Actually, this example has just convinced me that I definitely want
>> "kind-specific commands", even for Elisp.  That would be great.
>> For functions without separate declarations, find-implementation and
>> find-declaration could jump to the same place.  In some languages, maybe
>> that's just always what happens.
>
> Aren't find-implementation included in find-definition anyway? Or
> *are* the same as find-definition, for example, in LSP's approach?
> Aside from the fact that the latter also works for variables and other
> symbols.

No, find-implementation shows all the implementations of a generic
function/interface method.  (e.g. cl-defmethod)

find-definition (as I understand it) jumps to the interface definition
itself (e.g. cl-defgeneric).  Which in my experience in Elisp, is
usually what I want; I want to see the documentation and default
implementation and surrounding code, before I look at a specific
implementation.

I think that's a pretty reasonable way for it to work.  Actually, maybe
we could add a customization point for the Elisp backend so that a user
can choose to make find-definition work that way, if we do add a
find-implementation command.

There's also find-declaration - for Elisp, I think that one *is* the
mostly same as find-definition, since Elisp doesn't have separate
declarations.  (declare-function is kind of a different thing.  Maybe
find-declaration should jump to the declare-function instance in the
current file, if it can, but for now it's fine.)

>> As for type definitions - even Elisp has things that could reasonably be
>> called type definitions, we just don't support jump-to-definition for
>> them right now.  I have no idea how we'd implement it, but we could
>> support jumping to cl-defstruct definitions, for example.  Maybe we
>> could use information from native compilation to guess the type of the
>> identifier at point?  But anyway, for now we'd just error if the user
>> ran such a command in emacs-lisp-mode or other modes that don't support
>> jumping to the type definition.
>
> I think the aforementioned type definitions in Elisp would be better
> included in the "find definitions" set because they generally don't
> clash with other kinds.

I think they actually do clash in the important case.

For global variables, e.g. defcustoms, I want M-. to jump to the
defcustom, and find-type-definition would do the same thing.  So there's
no clash.  And in fact this works today, nothing new.

For local variables with no global definition though, I would like
M-. to jump straight to the local binding inside the function, but
find-type-definition should jump to the cl-defstruct defining the value
of the variable.  That's how it works in Eglot today, at least in Rust
and OCaml.

Actually supporting "jump to local binding" and "jump to type
definition" in Elisp is of course quite hard, but if we did someday find
a way to support it, I see no reason the behavior shouldn't match Eglot.

>>>> - users could still use completing-read to type the kind
>>>> Plus, if we do use M-' or any other short binding for this, we
>>>> should
>>>> almost certainly make it the start of a new prefix-map rather than bind
>>>> M-' directly to the new command; doing otherwise would be wasteful of
>>>> valuable keybinding space.
>>>
>>> If we're going to have separate commands for kinds, that is indeed a
>>> good idea.
>> I almost want to say that we should have it be a prefix regardless
>> of
>> whether we have separate commands for kinds.  I guess it depends:
>
> You probably meant "have it be the binding".
>
>> - if we use C-M-?, that's already such a hard key to hit that maybe it's
>>    okay if we bind it directly to a command
>> - if we use M-' or something similarly convenient, it would be
>> really
>>    tragic to not reclaim all that premium keybinding space.
>
> It's possible that it would work better the other way around: if we
> discover that people really want a prefix map, then it would be easier
> to argue for an easy-to-hit binding like the above.

True, true.  And if we end up with a single command design, then C-M-?
will suffice.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Thu, Nov 9, 2023 at 7:23 PM Spencer Baugh <sbaugh@janestreet.com> wrote:
>
> João Távora <joaotavora@gmail.com> writes:
> > On Wed, Nov 8, 2023 at 11:34 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
> >> > But neither was available at the time, so I did those commands and they won't
> >> > be obsoleted any time soon.
> >>
> >> Reasons being..?
> >
> > That we can't come up with alternatives to exactly that interface,
> > obviously.  When you do, I'll obsolete them.  But do you want to hardcode
> > things like LSP "typeDefinition" and Sly's "who-macroexpands" somewhere
> > in xref.el?  How would that work?
>
> Right, so if we have kinds defined in the core for "declaration",
> "implementation", and "type-definition" (maybe not with those exact
> names), then we can have xref-find-declaration,
> xref-find-implementation, and xref-find-type-definition (again maybe not
> with those exact names), and just do
>
> (define-obsolete-function-alias 'eglot-find-declaration 'xref-find-declaration)
> (define-obsolete-function-alias 'eglot-find-implementation 'xref-find-implementation)
> (define-obsolete-function-alias 'eglot-find-typeDefinition 'xref-find-type-definition)
>
> Is that possible?

For Dmitry to answer, but I think we had established that is a
slippery slope.  Many languages have different exotic concepts, like
Lisp and macroexpansion, C++ template instantiations, or annotations.

And conversely many other languages don't even have the distinction
between the three you mention.

And i'm only talking about the handful of languages I'm
more of less familiar with, there are many many more.  And many
more can be invented with different types of manifestation, there's
no shortage of imagination is language designer-land.

Not to mention Xref can in theory be used to provide
cross-referencing support in any type of work, not just
programming.

So I thought, that about 6 months ago we had established that
"definition" and "reference" are two relatively safe concept to
keep in xref.el, but other concepts should not be in there,
because this doesn't scale well and could imposes awkward
structure and hacks into an unknown number of backends.

Has everyone changed their mind?  What exactly bothers you
about eglot-find-declaration/implementation/typeDefinition?
In LSP-land these concepts _do_ make sense, because the
LSP standard prescribes what servers should do with them.

What practical problem are you trying to solve?

João

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

João Távora <joaotavora@gmail.com> writes:
> On Thu, Nov 9, 2023 at 7:23 PM Spencer Baugh <sbaugh@janestreet.com> wrote:
>>
>> João Távora <joaotavora@gmail.com> writes:
>> > On Wed, Nov 8, 2023 at 11:34 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>> >> > But neither was available at the time, so I did those commands and they won't
>> >> > be obsoleted any time soon.
>> >>
>> >> Reasons being..?
>> >
>> > That we can't come up with alternatives to exactly that interface,
>> > obviously.  When you do, I'll obsolete them.  But do you want to hardcode
>> > things like LSP "typeDefinition" and Sly's "who-macroexpands" somewhere
>> > in xref.el?  How would that work?
>>
>> Right, so if we have kinds defined in the core for "declaration",
>> "implementation", and "type-definition" (maybe not with those exact
>> names), then we can have xref-find-declaration,
>> xref-find-implementation, and xref-find-type-definition (again maybe not
>> with those exact names), and just do
>>
>> (define-obsolete-function-alias 'eglot-find-declaration 'xref-find-declaration)
>> (define-obsolete-function-alias 'eglot-find-implementation 'xref-find-implementation)
>> (define-obsolete-function-alias 'eglot-find-typeDefinition 'xref-find-type-definition)
>>
>> Is that possible?
>
> For Dmitry to answer, but I think we had established that is a
> slippery slope.  Many languages have different exotic concepts, like
> Lisp and macroexpansion, C++ template instantiations, or annotations.
>
> And conversely many other languages don't even have the distinction
> between the three you mention.

Well, Elisp for example has "declarations" and "implementations":
cl-defgeneric and cl-defmethod.  I'd like, in Elisp, a command which
jumps to the cl-defgeneric, and another command which shows the
cl-defmethods.  This doesn't have anything to do with LSP, it's a
feature purely for Elisp.

I think we can support this without going down any slippery slopes.

> And i'm only talking about the handful of languages I'm
> more of less familiar with, there are many many more.  And many
> more can be invented with different types of manifestation, there's
> no shortage of imagination is language designer-land.
>
> Not to mention Xref can in theory be used to provide
> cross-referencing support in any type of work, not just
> programming.
>
> So I thought, that about 6 months ago we had established that
> "definition" and "reference" are two relatively safe concept to
> keep in xref.el, but other concepts should not be in there,
> because this doesn't scale well and could imposes awkward
> structure and hacks into an unknown number of backends.

We don't have to include the concepts in xref.el per-se.  All I suggest
is that instead of supporting 'eglot--xref-implementation as a kind,
eglot should support 'implementation as a kind.  Likewise other backends
which support something they want to call an "implementation" can
support 'implementation.  We don't have to say what an "implementation"
is, any more than we currently specify what a "definition" or a
"reference" is.

If a backend doesn't support 'implementation, that works fine with the
current patch - the backend just errors when kind=implementation is
passed to xref-find-extra.  No awkward structure, no hacks.

> Has everyone changed their mind?  What exactly bothers you
> about eglot-find-declaration/implementation/typeDefinition?
> In LSP-land these concepts _do_ make sense, because the
> LSP standard prescribes what servers should do with them.
>
> What practical problem are you trying to solve?

Practically, I don't want to have a different UI for these commands in
every individual language and mode, I'd like a common set of bindings
which are usable for every language which supports these concepts.
Which I think is most languages - including Elisp.

There's also bug#64714 - I have to pick bindings for these commands for
my hundreds of users, and I don't want to pick ones which will conflict
with later changes upstream.  I expect there will eventually be bindings
for these commands in Emacs.  So I'd like to just do it now.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Fri, Nov 10, 2023 at 6:23 AM Spencer Baugh <sbaugh@janestreet.com> wrote:

> Well, Elisp for example has "declarations" and "implementations":
> cl-defgeneric and cl-defmethod.  I'd like, in Elisp, a command which
> jumps to the cl-defgeneric, and another command which shows the
> cl-defmethods.  This doesn't have anything to do with LSP, it's a
> feature purely for Elisp.
>
> I think we can support this without going down any slippery slopes.

Really, what if the language du jour sometime in the future becomes
something like C++ again? Will you add 'type-instantiation,
'type-specialization,  'partial-specialization,  'concept, and so on?
Or the C++ backend somehow manages to index the loci of memory
allocation to a given arena names at point.  Will 'memory-allocations'
also appear in xref.el?

These are all things a C++ backend may legitimately want to organize
much more than those 3 extra compartments, just like Lisp users
like to see who-macroexpands and who-calls.

So yes, quite slippery.

LSP, the protocol has the same problem, and their 5 categories
(definitions, references, implementation, declaration, typeDefinition)
are awkward as heck and don't make any sense for many languages.
They're probably coming from large commercial languages such as Java
and C#,  Typescript, etc not suprising given it's all a Microso$t
gig. But it's just poor design that I'd rather not have bleed into
Emacs xref.el.

Emacs is older than LSP and my bet is that it will outlive LSP.

Of course Dmitry is the person to convince here, not me.

> We don't have to include the concepts in xref.el per-se.  All I suggest
> is that instead of supporting 'eglot--xref-implementation as a kind,
> eglot should support 'implementation as a kind.

By the the symbol you quoted is an internal implementation detail.
The fact that it can't be represented well under the current
patch is under discussion with Dmitry and it's super-easy to fix
whatever the outcome of that discussion is.  IOW that prompt should
show "Implementation".

> If a backend doesn't support 'implementation, that works fine with the
> current patch - the backend just errors when kind=implementation is
> passed to xref-find-extra.  No awkward structure, no hacks.

Oh, big awkward:

1. You lose the consistency you wanted so much.  I can't
   take my muscle memory from backend A to backend B.  I'll just get
   errors.

2. If backend C supports many more things other than implementation
   typeDefinition, etc, they're either going to be awkwardly
   organized into those drawers or available as backend-specific
   commands you wanted to avoid in the first place.  Unless
   you enshrine them into xref.el.  Then you'll get more of
   problem 1.

The only clean solution to this is to go knock on language
designer doors ;-).  But I thought we had reached this conclusion
already, why are we rehashing it?

And if you didn't catch it, the current eglot-find-* commands
can be cleanly implemented with no hacks and with a common UI
between them.

Just like the Eglot code-action commands for common actions such
as "organizeImports", "quickfix".

When I get around to finishing refactor.el which will inherit most of
Eglot's UI for doing refactorings, I don't plan to burn the concepts of
"organizeImports" and "quickfix" into it.  They are LSP things.

> Practically, I don't want to have a different UI for these commands in
> every individual language and mode, I'd like a common set of bindings
> which are usable for every language which supports these concepts.

And I'd like speakers of foreign languages to understand Portuguese
"go there go!".  And I yet I fail, and it's impossible to  explain.


> Which I think is most languages - including Elisp.

"declarations" really?  Sure you could cram declare-function into there,
and it kind of emulates the forward-declaration meaning of it in C/C++
which is clearly LSP got the idea from.  Fine.  But then what about
compiler-macro declarations and edebug declarations, etc, the ones you use
with `(declare (debug...))` in the beginning of functions (sometimes not)?
They're an entirely different concept and yet known as "declarations" for
any Elisp  (or Common Lisp) user. It's awkward to find both mixed when
you're thinking of just one.

> There's also bug#64714 - I have to pick bindings for these commands for
> my hundreds of users, and I don't want to pick ones which will conflict
> with later changes upstream.  I expect there will eventually be bindings
> for these commands in Emacs.  So I'd like to just do it now.

This isn't an Eglot/Xref, etc problem.  It's a problem of whatever you
are trying to do.  Eglot uses no keybindings and plans to keep that
promise.  Then, there are keybindings reserved for users, major modes,
specialized modes, etc.  I don't know the rules by heart, you can consult
them.

João Távora

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

João Távora <joaotavora@gmail.com> writes:
> On Fri, Nov 10, 2023 at 6:23 AM Spencer Baugh <sbaugh@janestreet.com> wrote:
>
>> Well, Elisp for example has "declarations" and "implementations":
>> cl-defgeneric and cl-defmethod.  I'd like, in Elisp, a command which
>> jumps to the cl-defgeneric, and another command which shows the
>> cl-defmethods.  This doesn't have anything to do with LSP, it's a
>> feature purely for Elisp.
>>
>> I think we can support this without going down any slippery slopes.
>
> Really, what if the language du jour sometime in the future becomes
> something like C++ again? Will you add 'type-instantiation,
> 'type-specialization,  'partial-specialization,  'concept, and so on?
> Or the C++ backend somehow manages to index the loci of memory
> allocation to a given arena names at point.  Will 'memory-allocations'
> also appear in xref.el?
>
> These are all things a C++ backend may legitimately want to organize
> much more than those 3 extra compartments, just like Lisp users
> like to see who-macroexpands and who-calls.
>
> So yes, quite slippery.

None of those make sense for Elisp, and "declarations" and
"implementations" do make sense for Elisp (as I'll describe below).
That's the difference.

I think we should be somewhat "selfish", and let our design here be
guided by "Things which make sense for Elisp, and also many other
languages".  Declarations and implementations, I think, are an instance
of that.

> LSP, the protocol has the same problem, and their 5 categories
> (definitions, references, implementation, declaration, typeDefinition)
> are awkward as heck and don't make any sense for many languages.
> They're probably coming from large commercial languages such as Java
> and C#,  Typescript, etc not suprising given it's all a Microso$t
> gig. But it's just poor design that I'd rather not have bleed into
> Emacs xref.el.
>
> Emacs is older than LSP and my bet is that it will outlive LSP.

Absolutely, of course, but just because LSP has "declarations" and
"implementations" doesn't mean it's not good for Emacs.

> Of course Dmitry is the person to convince here, not me.
>
>> We don't have to include the concepts in xref.el per-se.  All I suggest
>> is that instead of supporting 'eglot--xref-implementation as a kind,
>> eglot should support 'implementation as a kind.
>
> By the the symbol you quoted is an internal implementation detail.
> The fact that it can't be represented well under the current
> patch is under discussion with Dmitry and it's super-easy to fix
> whatever the outcome of that discussion is.  IOW that prompt should
> show "Implementation".
>
>> If a backend doesn't support 'implementation, that works fine with the
>> current patch - the backend just errors when kind=implementation is
>> passed to xref-find-extra.  No awkward structure, no hacks.
>
> Oh, big awkward:
>
> 1. You lose the consistency you wanted so much.  I can't
>    take my muscle memory from backend A to backend B.  I'll just get
>    errors.

Eh?  You'll only get an error if backend B doesn't support one of the
core kinds.

I also get errors today if the backend doesn't support
xref-find-references.  That doesn't make M-? (xref-find-references) any
less of a consistent UI for modes that support it.

> 2. If backend C supports many more things other than implementation
>    typeDefinition, etc, they're either going to be awkwardly
>    organized into those drawers or available as backend-specific
>    commands you wanted to avoid in the first place.

That is fine and exactly what I want.  I see an area where a degree of
consistency is possible, so I see no reason not to get that degree of
consistency.

"This other thing can't be consistent" doesn't make sense to me as an
argument against making this somewhat consistent.

>    Unless you enshrine them into xref.el.  Then you'll get more of
>    problem 1.

Definitely not.  Let's keep it minimal.

> The only clean solution to this is to go knock on language
> designer doors ;-).  But I thought we had reached this conclusion
> already, why are we rehashing it?
>
> And if you didn't catch it, the current eglot-find-* commands
> can be cleanly implemented with no hacks and with a common UI
> between them.
>
> Just like the Eglot code-action commands for common actions such
> as "organizeImports", "quickfix".
>
> When I get around to finishing refactor.el which will inherit most of
> Eglot's UI for doing refactorings, I don't plan to burn the concepts of
> "organizeImports" and "quickfix" into it.  They are LSP things.

Sure, that sounds good.  But do you hope to eventually add a standard
keybinding for accessing refactor.el?  I hope so.  And I'd like the same
for these xref commands.  (but not for Eglot and LSP's sake!  My
inspiration for making this thread is not Eglot and LSP, it's my own
xref backend which also has these concepts)

>> Practically, I don't want to have a different UI for these commands in
>> every individual language and mode, I'd like a common set of bindings
>> which are usable for every language which supports these concepts.
>
> And I'd like speakers of foreign languages to understand Portuguese
> "go there go!".  And I yet I fail, and it's impossible to  explain.
>
>
>> Which I think is most languages - including Elisp.
>
> "declarations" really?  Sure you could cram declare-function into
> there, and it kind of emulates the forward-declaration meaning of it
> in C/C++ which is clearly LSP got the idea from.  Fine.  But then what
> about compiler-macro declarations and edebug declarations, etc, the
> ones you use with `(declare (debug...))` in the beginning of functions
> (sometimes not)?  They're an entirely different concept and yet known
> as "declarations" for any Elisp (or Common Lisp) user. It's awkward to
> find both mixed when you're thinking of just one.

That's not what I'm suggesting.  I suggest that find-declarations when
run on a generic should give the cl-defgeneric, and find-implementations
should give the cl-defmethods.

I think that would be a really helpful new feature, apart from all the
rest of this.

If the term "declarations" and "implementations" are a problem, we could
totally give them another name.

How about "find-generic" and "find-methods"?  Just literally copying
what Elisp calls them.  Then we can set aside all this worry about
sharing terms with LSP.

So let me restate my proposal in those terms:

I'd like an xref-find-generic which in Elisp jumps to the cl-defgeneric,
and an xref-find-methods which in Elisp jumps to the cl-defmethods.
When run on regular functions, both will just jump to the defun.  And I
would like default keybindings for those, ideally under M-'.  I think
this is useful independent of anything else.  I want this purely for
programming in Emacs Lisp.

If and when we added those, I think it would be reasonable for some
language modes (but not others) to connect xref-find-generic to LSP
declarations, and to connect xref-find-methods to LSP implementations.
Since in some languages those concepts match up decently well.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Fri, Nov 10, 2023 at 12:15 PM Spencer Baugh <sbaugh@catern.com> wrote:

> I think we should be somewhat "selfish", and let our design here be
> guided by "Things which make sense for Elisp, and also many other
> languages".  Declarations and implementations, I think, are an instance
> of that.

Not really.  "Declarations" don't make much sense in Elisp. I'll
explain below.

And I don't think we should code xref.el with "selfishness" in mind
at least if we're serious about making it reusable.  elisp-mode.el
should have things that make sense fo Elisp.  Vast amounts of Emacs
users couldn't care less for Elisp, and that's fine.

> > LSP, the protocol has the same problem, and their 5 categories
> > (definitions, references, implementation, declaration, typeDefinition)
> > are awkward as heck and don't make any sense for many languages.
> > They're probably coming from large commercial languages such as Java
> > and C#,  Typescript, etc not suprising given it's all a Microso$t
> > gig. But it's just poor design that I'd rather not have bleed into
> > Emacs xref.el.
> >
> > Emacs is older than LSP and my bet is that it will outlive LSP.
>
> Absolutely, of course, but just because LSP has "declarations" and
> "implementations" doesn't mean it's not good for Emacs.

Doesn't mean it's any good either.  Emacs precedes LSP by many
years in being flexible enough to support many languages.
And it absolutely trounces LSP in that respect.  LSP's main point
here is that it supports many editors, and so reusable language servers
appear for many languages not motivated by Emacs, and Emacs,
using Eglot, profits from that.  But LSP makes tremendous tradeoffs
in flexibility, tradeoff that we don't have to make or even consider,
because we support only one editor, ourselves.

> > Of course Dmitry is the person to convince here, not me.
> >
> >> We don't have to include the concepts in xref.el per-se.  All I suggest
> >> is that instead of supporting 'eglot--xref-implementation as a kind,
> >> eglot should support 'implementation as a kind.
> >
> > By the the symbol you quoted is an internal implementation detail.
> > The fact that it can't be represented well under the current
> > patch is under discussion with Dmitry and it's super-easy to fix
> > whatever the outcome of that discussion is.  IOW that prompt should
> > show "Implementation".
> >
> >> If a backend doesn't support 'implementation, that works fine with the
> >> current patch - the backend just errors when kind=implementation is
> >> passed to xref-find-extra.  No awkward structure, no hacks.
> >
> > Oh, big awkward:
> >
> > 1. You lose the consistency you wanted so much.  I can't
> >    take my muscle memory from backend A to backend B.  I'll just get
> >    errors.
>
> Eh?  You'll only get an error if backend B doesn't support one of the
> core kinds.

Exactly, and you wrote "the backend just errors when kind=implemetation"
So you admit it's going to happen.

> I also get errors today if the backend doesn't support
> xref-find-references.  That doesn't make M-? (xref-find-references) any
> less of a consistent UI for modes that support it.

Yes, the notion of definition and reference are much less
controversial.

> > 2. If backend C supports many more things other than implementation
> >    typeDefinition, etc, they're either going to be awkwardly
> >    organized into those drawers or available as backend-specific
> >    commands you wanted to avoid in the first place.
>
> That is fine and exactly what I want.  I see an area where a degree of
> consistency is possible, so I see no reason not to get that degree of
> consistency.

But incomplete consistency contains, by definition, inconsistency!  IOW
"degree of consistency" is a fine example of an oxymoron.

> > When I get around to finishing refactor.el which will inherit most of
> > Eglot's UI for doing refactorings, I don't plan to burn the concepts of
> > "organizeImports" and "quickfix" into it.  They are LSP things.
>
> Sure, that sounds good.  But do you hope to eventually add a standard
> keybinding for accessing refactor.el?  I hope so.

Sure, for refactor-code-actions, which will work much like xref-find-extras,
aka xref-find-by-kind (or maybe it should be named only xref-find, or
why not only
xref).

But there won't be a command refactor-quickfix or refactor-organizeImports
because not all potential refactor.el backends support those LSP concepts.

It's exactly the same here.

> for these xref commands.  (but not for Eglot and LSP's sake!  My
> inspiration for making this thread is not Eglot and LSP, it's my own
> xref backend which also has these concepts)
>
> >> Practically, I don't want to have a different UI for these commands in
> >> every individual language and mode, I'd like a common set of bindings
> >> which are usable for every language which supports these concepts.
> >
> > And I'd like speakers of foreign languages to understand Portuguese
> > "go there go!".  And I yet I fail, and it's impossible to  explain.
> >
> >
> >> Which I think is most languages - including Elisp.
> >
> > "declarations" really?  Sure you could cram declare-function into
> > there, and it kind of emulates the forward-declaration meaning of it
> > in C/C++ which is clearly LSP got the idea from.  Fine.  But then what
> > about compiler-macro declarations and edebug declarations, etc, the
> > ones you use with `(declare (debug...))` in the beginning of functions
> > (sometimes not)?  They're an entirely different concept and yet known
> > as "declarations" for any Elisp (or Common Lisp) user. It's awkward to
> > find both mixed when you're thinking of just one.
>
> That's not what I'm suggesting.  I suggest that find-declarations when
> run on a generic should give the cl-defgeneric, and find-implementations
> should give the cl-defmethods.

Doesn't make sense to me, sorry.  In Elisp you work with symbols.  So
a given symbol has many possible meanings (a so-called lisp-2 has two
standard ones: variable and function bindings).  But you can add your
own, class definition, generic function, setf expander, autoload-function,
docstring, whatever crosses your mind.  In current Elisp-- but not
necessarily future Elisp--you alreadyhave two completely different concepts
of "declaration".   One is declaring functions early so the byte-compiler
doesn't complain about them and another is optional `declare` forms you find
in the first form of a function's body.  Completely different concepts,
miles apart.  Now you want to add a symbol's generic function definition
to the "declaration" family. Even more miles apart. So one day I find
a symbol, I press your desired xref-find-declarations on it and I get
this awkward salad of totally unrelated things.

I'd certainly like to be able to use xref to find the source code
loci for all these different things that a symbol might mean, of course
I do.  But xref-find-declaration is a awful way to do it.

Mind you that xref-find-references is much less problematic.
A "reference" is in essence any manifestation of a given
symbol/name/identifier in the source material, _regardless_ of
the specific kind of manifestation.

> I think that would be a really helpful new feature, apart from all the
> rest of this.
>
> If the term "declarations" and "implementations" are a problem, we could
> totally give them another name.
>
> How about "find-generic" and "find-methods"?  Just literally copying
> what Elisp calls them.  Then we can set aside all this worry about
> sharing terms with LSP.
>
> So let me restate my proposal in those terms:
>
> I'd like an xref-find-generic which in Elisp jumps to the cl-defgeneric,

So what would xref-find-generic do in C++ code?  It has no generics.
Most languages have no generics.  Most programmers I know have never
even heard of "generics".  "generic functions" even less so.  Why do
you want to burden these non-Lisp people with this new concept
from Common Lisp that they'll have to map to their own favourite
language they want to do their work on?  And no, never will "generic"
map clearly to the fundamental C/C++  concept of a declaration, for
example.  Not to mention Java has templates called "generics".
And C++ has templates called templates. Completely different concepts.

So IMO this is just a very bad idea, I'm afraid.

But I think we have made both our positions clear.  I understand what
you want to do and I am completely against it, but this is not just
up for me to decide.  The only thing that is, is that if I like the
new interfaces in xref.el I will implement them in eglot.el.  The
UI that xref.el provides I can't control.  Of course I recommend
keeping at least the current UI of xref-find-extra, not least because
it's something that has actually been implemented and I have
spent time integrating.  And I also like your idea of KIND=nil to
make it show a sectioned listing using whatever kinds the backend
declares as section headings.

In the end, people will be able to continue using LSP and xref.el
without any problems, with eglot-find-declaration, etc, that's a
given.

Joao

Re: CCing thread participants through gnus+gmane

[Eric Abrahamsen]

Spencer Baugh <sbaugh@janestreet.com> writes:

> João Távora <joaotavora@gmail.com> writes:
>> On Wed, Nov 8, 2023 at 5:45 PM Spencer Baugh <sbaugh@janestreet.com> wrote:
>>>
>>> João Távora <joaotavora@gmail.com> writes:
>>>
>>> > On Wed, Nov 8, 2023, 16:40 Spencer Baugh <sbaugh@janestreet.com> wrote:
>>> >
>>> >  João Távora <joaotavora@gmail.com> writes:
>>> >  > On Tue, Nov 7, 2023 at 11:18 PM Dmitry Gutov <dgutov@yandex.ru> wrote:
>>> >  >> What I meant is, using it with the Elisp implementation didn't
>>> >  >> convince me of the usefulness of the feature. Perhaps you disagree?
>>> >  >>
>>> >  >> Or it would be nice to hear from someone who have tried out Eglot's
>>> >  >> integration and found more upsides there.
>>> >  >
>>> >  > If you want that to happen, the only realistic way to have good
>>> >  > feedback from anyone else other than emacs-devel nerds like me and you
>>> >  > is to release an Eglot version with  this feature, which we can change
>>> >  > later (even non-backward-compatibly, within a reasonable time frame).
>>> >
>>> >  I tried the Eglot integration, my comments elsewhere are inspired by
>>> >  that.
>>> >
>>> > Can you point me to this "elsewhere"?
>>>
>>> Elsewhere in this thread - my message on Tue, 07 Nov 2023 11:51:49 -0500
>>
>> OK, I found it.  It's useful, and very commonly done, if you CC the
>> participants in this discussion, particularly when addressing their
>> contributions.
>> Your emails are sent only to emacs-devel and it's easier for me to
>> miss them.
>
> Sorry about that... I fully agree about CCing, but for $REASONS I have
> to use gmane to interact with emacs-devel.  Does anyone happen to know
> if there's a way to get gmane+gnus to CC participants?

Usually what you do (as I've done here) is hit "S W". That ends up
replying To the original author and Cc'ing emacs-devel, but same difference...

Re: CCing thread participants through gnus+gmane

[Visuwesh]

[வியாழன் நவம்பர் 09, 2023] Spencer Baugh wrote:

> Sorry about that... I fully agree about CCing, but for $REASONS I have
> to use gmane to interact with emacs-devel.  Does anyone happen to know
> if there's a way to get gmane+gnus to CC participants?

S W (wide reply) puts everyone participating in the CC list for me.  I
read the list through gmane.

> (If there's not a way to do that, I should really look into not using
> gmane...  Maybe I can use a public-inbox mirror of emacs-devel like
> https://yhetil.org/emacs-devel/ )

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

João Távora <joaotavora@gmail.com> writes:
> On Fri, Nov 10, 2023 at 12:15 PM Spencer Baugh <sbaugh@catern.com> wrote:
>> > When I get around to finishing refactor.el which will inherit most of
>> > Eglot's UI for doing refactorings, I don't plan to burn the concepts of
>> > "organizeImports" and "quickfix" into it.  They are LSP things.
>>
>> Sure, that sounds good.  But do you hope to eventually add a standard
>> keybinding for accessing refactor.el?  I hope so.
>
> Sure, for refactor-code-actions, which will work much like xref-find-extras,
> aka xref-find-by-kind (or maybe it should be named only xref-find, or
> why not only
> xref).
>
> But there won't be a command refactor-quickfix or refactor-organizeImports
> because not all potential refactor.el backends support those LSP concepts.
>
> It's exactly the same here.

And refactor-code-actions will behave like the current
eglot-code-actions, and prompt for code actions iff there are multiple
possible actions that can be taken?  I know you prefer that workflow
personally, but some people like keybindings to do common tasks
directly.

For example, do you expect to support symbol renaming in refactor.el?
That if anything seems like something which deserves its own binding.

That being said, maybe we could have mode-specific commands and bindings
for these things.  So if something is supported in a language, a major
mode could make a binding for that.  And then, just, we might encourage
using the same bindings in each mode that binds something.

>> I think that would be a really helpful new feature, apart from all the
>> rest of this.
>>
>> If the term "declarations" and "implementations" are a problem, we could
>> totally give them another name.
>>
>> How about "find-generic" and "find-methods"?  Just literally copying
>> what Elisp calls them.  Then we can set aside all this worry about
>> sharing terms with LSP.
>>
>> So let me restate my proposal in those terms:
>>
>> I'd like an xref-find-generic which in Elisp jumps to the cl-defgeneric,
>
> So what would xref-find-generic do in C++ code?  It has no generics.
> Most languages have no generics.  Most programmers I know have never
> even heard of "generics".  "generic functions" even less so.  Why do
> you want to burden these non-Lisp people with this new concept
> from Common Lisp that they'll have to map to their own favourite
> language they want to do their work on?  And no, never will "generic"
> map clearly to the fundamental C/C++  concept of a declaration, for
> example.  Not to mention Java has templates called "generics".
> And C++ has templates called templates. Completely different concepts.
>
> So IMO this is just a very bad idea, I'm afraid.
>
> But I think we have made both our positions clear.  I understand what
> you want to do and I am completely against it, but this is not just
> up for me to decide.  The only thing that is, is that if I like the
> new interfaces in xref.el I will implement them in eglot.el.  The
> UI that xref.el provides I can't control.  Of course I recommend
> keeping at least the current UI of xref-find-extra, not least because
> it's something that has actually been implemented and I have
> spent time integrating.  And I also like your idea of KIND=nil to
> make it show a sectioned listing using whatever kinds the backend
> declares as section headings.

OK, fair enough, and I can see your point.  I think we're both clear.

Putting any talk about common xref APIs aside... Just to be clear, my
understanding is that you have no problem with the idea of:

- Teaching the elisp--xref-backend about separate 'cl-defgeneric and
  'cl-defmethods kinds, which return the obvious things

- Adding commands elisp-find-cl-{defgeneric,defmethods} which use those
  kinds (and which fall back to the regular definitions)

- Adding bindings in emacs-lisp-mode for calling those commands

I'd like to do that independent of whatever we end up with, so just
making sure you're not opposed to that, even though you might not be
interested in using it.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 09/11/2023 19:07, Spencer Baugh wrote:
> Custom kinds would still require you to type in the kind symbol.  (Or a
> backend or mode can make a "quick-access" command specific to that kind,
> if they want.)
If we leave all the UI decisions to the backends (both extra commands 
and the dispatch which kind to use in the "unifying" command as well), 
then there might be little point in adding extra infrastructure to 
xref.el itself.

xref-show-xrefs is a public function, FETCHER just needs to return a 
list of xrefs, the same data that backend functions like 
xref-backend-definitions return.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 09/11/2023 03:06, João Távora wrote:
> On Thu, Nov 9, 2023 at 12:27 AM Dmitry Gutov <dmitry@gutov.dev> wrote:
>>
>> On 09/11/2023 02:08, Dmitry Gutov wrote:
>>> Do you know which category does "eglot-find-typeDefinition" falls into,
>>> and why? Aside from the fact that it, historically, uses a separate
>>> endpoint.
>>
>> Hm, according to
>> https://github.com/rust-lang/rust-analyzer/issues/2541#issuecomment-565166660,
>> "find-typeDefinition" goes to the definition of the type of the current
>> variable (at point).
> 
> That's rust-analyzer interpretation.  In practice, servers can do what they
> want.  This is the LSP description
> 
>      The go to type definition request is sent from the client to the server
>      to resolve the type definition location of a symbol at a given text
>      document position.
> 
> For example, an Elisp language server could use that to go to the place
> where the  `typeDefinition` of a given symbol is set on that symbol.

Perhaps. But that's still not the definition of the symbol. It would 
more-or-less align with rust-analyzer's interpretation, as I see it.

>> That probably means it doesn't really fit with the rest of the commands
>> we're discussing, e.g. because the identifier it works on is actually a
>> type name
> 
> Not in rust-analyzer, at least judging by what you described.  Seems
> like the identifier it works on is a variable name, which rust-analyzer
> then associates with the type (being a strongly typed language it knows
> that) and then you get to the place where that type is defined.
> Super-reasonable, for rust.

Okay. That's an interesting indirection, and indeed it might be 
difficult to make it work otherwise via LSP.

>> (so the completion should be across types? ideally, at least,
>> if LSP supported that). Anyway, it shouldn't be mixed into "find
>> definition", at least if we go by rust-analyzer's meaning of it.
> 
> Actually, no.   Precisely in rust-analyzer's case it could be mixed.
> I see somewhere a reference to a variable, I do that command you're thinking
> of and get the definition of the variable and the definition of the type of the
> variable, both useful things.

It's less useful if one considers that in the resulting interface you 
never just jump to the definition of the said variable, even if it's 
found unambiguously. You would always have to choose between the 
variable and its type to jump to.

In that context, an additional command which would collect "everything 
to show about a symbol", including type definition, makes sense. For the 
less frequent cases when you want a more through overview. It could 
default to that behavior (asking for individual kinds otherwise), or 
have some kind called 'all' which would do this. I'd probably choose the 
former.

> But you're right that it won't make sense for every possible
> conceivable language or server.  But just don't worry.  Leave that
> concern to server developers and LSP users.  You're not going to
> magically fix LSP with the xref.el feature.  The Emacs-LSP marriage
> is never quite perfect, but it gets a lot of love.
> 
> Exactly like any good marriage should :-)

That's poetic.

Still, this change needs to work well for both LSP (an important backend 
these days) and for others. If we find ways to make switching between 
backends more seamless, it's an improvement.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 09/11/2023 19:57, Spencer Baugh wrote:

>> Fair enough. So "definitions" are the places we want to see 90% of the
>> time when learning about a funciton or a variable. And the other
>> reference kinds (BTW, what to call that? "definition kinds"?
>> "reference kinds"?) must be apparently less useful.
> 
> Right.
> 
>> Do you know which category does "eglot-find-typeDefinition" falls
>> into, and why? Aside from the fact that it, historically, uses a
>> separate endpoint.
> 
> It's on the less-useful, more-rarely-used side.

It seems like typeDefinition would still be in the category of 
"definitions", i.e. preferably dispatched through 
xref-show-definitions-function, so e.g. if only one type definition is 
found, navigation doesn't show the list and jumps directly.

But not in the output of xref-find-definitions.

>>> If you mean something a non-default version of xref-find-definitions, or
>>> another binding for another basically-identical command, sure, I think
>>> that would be good.  We can have both that command *and* have a command
>>> which prompts for KIND and defaults to "all".
>>
>> We could indeed, if we decide what to call it. "extras" seems out
>> (since it would include both definitions and additional reference
>> kinds). Just "xref-find-by-kind"? Then it's less obvious to have the
>> default behavior showing all.
> 
> Actually, maybe this should be a backend-specific thing.  The backend
> could specify a default along with its list of kinds.  And if we request
> that default kind (which might be 'all), then the backend will decide
> what kinds to send us.

If the backend specifies the default behavior of xref-find-by-kind, then 
it's what, another generic method to override?

> That neatly avoids us having to make any kind of design decisions about
> the semantic meaning of different kinds, and what sorts of kinds exist,
> and all that.

Well, neatly is in the eye of the beholder - we would end up with more 
choices for backend authors and less consistent experience for the users.

> (Other than having a few standard kinds, which I still think we should
> do, if only for the sake of getting the eglot-find-* commands out of
> eglot and into xref.  But we don't have to really think about what those
> standard kinds mean semantically)

Ultimately, the backend decides what "implementations" or "type 
definitions" are anyway. But we get to make suggestions.

>>> Although I suppose there's
>>> also no limit on how many implementations of a generic function there
>>> can be, and that's definitely something that fits in "definitions".
>>
>> The fact that there are often many definitions for a generic function
>> indeed is an argument toward including more "kinds" into
>> xref-find-definitions' output by default. But only as many as it would
>> be actually useful, especially if the kinds are mutually exclusive (a
>> symbol is either one or the other) or intersect very rarely, or can be
>> distinguished by context.
> 
> I suggest this should be up to the backend.  IMO xref-find-definitions
> should stay fast in the common case, but sometimes it will unavoidably
> have to include multiple things, like for cl-defgenerics in Elisp.  We
> can't make the decision about how and when to do that for every backend.

It will be up to the backend: the xref-backend-definitions won't be 
going away. We'll need stronger reasons to change that.

> Actually, it occurs to me that if we had an xref-find-implementations
> command from the start, with a convenient binding, maybe
> xref-find-definitions would just only show the cl-defgeneric, and jump
> to it right away.  And only if you hit xref-find-implementations would
> you jump to the cl-defmethods.  We can't make that change now, but I
> don't think it would be worse!  And if a backend wants a design like
> that, I think the backend should be able to have it.

Not sure I would like the result that you are describing, but indeed, if 
a backend wants to do it like this, nothing will be stopping it.

Are LSP servers behaving like this? E.g. jdt-ls jumping to the method 
definition in an interface, even when the owner type is a statically 
known class?

>>> Maybe the backend could decide what kinds get included in "all".  Then
>>> it could deliberately avoid including anything "reference-like".
>>
>> Could we want several such commands? E.g. one for "all definition-like
>> hits" and another for "all reference-like hits"? With separate sets of
>> kinds for definitions and references?
> 
> Possibly, but my suggestion is that "all definition-like hits" and "all
> reference-like hits" should just be kinds exposed by the backend.

Two different generics returning lists of kinds, or just one? If two, 
then could one give an example of a "reference-like" kind except for 
"references"? I've taken a couple of guesses, but I'm exactly sure of them.

> *Maybe* we'll have them be "standard kinds" with a command and binding
> by default, but maybe not.  (We could always add more standard kinds and
> commands and bindings later.)

Yes.

>> It is less flexible because any time a backend wants to use a new
>> "kind", it will need to expend some effort and add it to the core
>> somehow. Define a new command or two and assign them to the said
>> prefix map. This could also lead to conflicts if backend authors don't
>> do this carefully enough.
> 
> No, I suggest that xref-backend-extra-kinds should be able to return
> whatever arbitrary symbols it wants.  Just, if its "kinds" match the
> kinds used in the core, then it will benefit from the commands and
> bindings defined by the core.
> 
> Perfectly extensible, but still benefiting from standardization.

Would the user be able to input/use the kinds that are in the list 
returned by the backend but are not in the set of "kinds used in the 
core"? If yes, how? Using a backend-provided command?

>>> I do think that if we go with an API which has any notion of "kinds", we
>>> should have some standard "kinds" in the core like implementation,
>>> declaration, type-definition.  I don't see any reason not to do that.
>>
>> We could have a "registry" of kinds, associating each of them with a
>> key. Then the result could be more dynamic, e.g. M-' could be bound to
>> a command that reads the key and performs the dispatch to the
>> corresponding search (similar to project-switch-project).
>>
>> And the users (though probably not packages) would later be able to
>> customize that mapping, adding new kinds or modifying the keys.
>>
>> With this approach we don't end up with many xref-find-xyz commands,
>> most of which just clutter the namespace, staying unused for a large
>> proportion of the users. OTOH, one wouldn't be able to examine the
>> prefix map and its contents and definitions with (M-' C-h) -- that's a
>> minor reduction in usability.
> 
> The only difference between this and a regular keymap which maps keys to
> commands is:
> 
>> we don't end up with many xref-find-xyz commands
> 
> But we can also avoid that by just... not defining many such commands
> and kinds.  A mode will always have the ability to define its own
> mode-specific kinds in the mode's own namespace, which don't clutter
> xref-*.

Indeed.

But if we don't define a dynamic dispatcher at all, it seems we won't 
need the backend to tell us about the supported kinds either. It would 
just support some and perhaps return error for unknown kinds.

>>>>> xref-extra-kind, prompting for kind: M-' M-'
>>>>
>>>> Would we need this command, if we had separate commands for each kind
>>>> already?
>>> This would support kinds which:
>>> - are language-specific,
>>> - or are more rarely used and don't need a dedicated command,
>>> - or are both.
>>
>> What would be the more frequently used, less language-specific commands?
>>
>> If we could agree on such list, we could indeed have those
>> xref-find-a/b/c definitions and a command with completing-read for the
>> rest.
> 
> I suggest that we should attempt to provide the following standard
> kinds:
> 
> 'declaration
> 'implementation
> 'type-definition
> 
> I personally think these would work well for a wide range of languages.
> Yes, these are the ones which LSP has; but just because they're from
> LSP, doesn't mean they're completely wrong.  They seem to me to strike
> the right balance of commonality between languages.

Fine with me. I don't see any problem with borrowing the list from LSP.

>>> Actually, this example has just convinced me that I definitely want
>>> "kind-specific commands", even for Elisp.  That would be great.
>>> For functions without separate declarations, find-implementation and
>>> find-declaration could jump to the same place.  In some languages, maybe
>>> that's just always what happens.
>>
>> Aren't find-implementation included in find-definition anyway? Or
>> *are* the same as find-definition, for example, in LSP's approach?
>> Aside from the fact that the latter also works for variables and other
>> symbols.
> 
> No, find-implementation shows all the implementations of a generic
> function/interface method.  (e.g. cl-defmethod)
> 
> find-definition (as I understand it) jumps to the interface definition
> itself (e.g. cl-defgeneric).  Which in my experience in Elisp, is
> usually what I want; I want to see the documentation and default
> implementation and surrounding code, before I look at a specific
> implementation.

I usually want to jump to where the function is implemented. Preferably, 
for the given type (though we don't support that). Perhaps I would use 
the proposed xref-find-implementations for this purpose, but it's bound 
to have a more complex key sequence than 'M-.'.

> I think that's a pretty reasonable way for it to work.  Actually, maybe
> we could add a customization point for the Elisp backend so that a user
> can choose to make find-definition work that way, if we do add a
> find-implementation command.

We could.

> There's also find-declaration - for Elisp, I think that one *is* the
> mostly same as find-definition, since Elisp doesn't have separate
> declarations.  (declare-function is kind of a different thing.  Maybe
> find-declaration should jump to the declare-function instance in the
> current file, if it can, but for now it's fine.)

Yep. Though for cases where there is no corresponding meaningful 
implementation, I would rather abort with an error. Doing otherwise 
seems misleading (like we have special handing of this case, but we don't).

>> I think the aforementioned type definitions in Elisp would be better
>> included in the "find definitions" set because they generally don't
>> clash with other kinds.
> 
> I think they actually do clash in the important case.
> 
> For global variables, e.g. defcustoms, I want M-. to jump to the
> defcustom, and find-type-definition would do the same thing.  So there's
> no clash.  And in fact this works today, nothing new.
> 
> For local variables with no global definition though, I would like
> M-. to jump straight to the local binding inside the function, but
> find-type-definition should jump to the cl-defstruct defining the value
> of the variable.  That's how it works in Eglot today, at least in Rust
> and OCaml.
> 
> Actually supporting "jump to local binding" and "jump to type
> definition" in Elisp is of course quite hard, but if we did someday find
> a way to support it, I see no reason the behavior shouldn't match Eglot.

Yes, agreed.

>>>>> - users could still use completing-read to type the kind
>>>>> Plus, if we do use M-' or any other short binding for this, we
>>>>> should
>>>>> almost certainly make it the start of a new prefix-map rather than bind
>>>>> M-' directly to the new command; doing otherwise would be wasteful of
>>>>> valuable keybinding space.
>>>>
>>>> If we're going to have separate commands for kinds, that is indeed a
>>>> good idea.
>>> I almost want to say that we should have it be a prefix regardless
>>> of
>>> whether we have separate commands for kinds.  I guess it depends:
>>
>> You probably meant "have it be the binding".
>>
>>> - if we use C-M-?, that's already such a hard key to hit that maybe it's
>>>     okay if we bind it directly to a command
>>> - if we use M-' or something similarly convenient, it would be
>>> really
>>>     tragic to not reclaim all that premium keybinding space.
>>
>> It's possible that it would work better the other way around: if we
>> discover that people really want a prefix map, then it would be easier
>> to argue for an easy-to-hit binding like the above.
> 
> True, true.  And if we end up with a single command design, then C-M-?
> will suffice.

I'm guessing people would object to using C-M-? because in the terminal 
it just maps to M-? which is taken.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 09/11/2023 22:44, João Távora wrote:
> So I thought, that about 6 months ago we had established that
> "definition" and "reference" are two relatively safe concept to
> keep in xref.el, but other concepts should not be in there,
> because this doesn't scale well and could imposes awkward
> structure and hacks into an unknown number of backends.
> 
> Has everyone changed their mind?

My impression is that the first feedback from our patches actually made 
people excited about things that _weren't_ included in the 
previously-discussed plan, so it seems like a good idea to re-evaluate 
it. Though not necessarily redo it all.

Though I guess this particular mailing list might be biased in favor of 
particular type of users (keyboard-driven, faster interaction as a 
priority).

> What exactly bothers you
> about eglot-find-declaration/implementation/typeDefinition?
> In LSP-land these concepts_do_  make sense, because the
> LSP standard prescribes what servers should do with them.

I think we've actually discovered that these kind of make sense for 
Elisp too. They might not match the current implementation, but 
conceptually, in some future, they might.

Again, not that we will necessarily change Elisp's backend to align with 
LSP, but this makes the notions universal enough, it seems. And yes, 
there might be more specific kinds which I don't know if we're going to 
support directly.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 10/11/2023 14:02, Spencer Baugh wrote:
>> "declarations" really?  Sure you could cram declare-function into
>> there, and it kind of emulates the forward-declaration meaning of it
>> in C/C++ which is clearly LSP got the idea from.  Fine.  But then what
>> about compiler-macro declarations and edebug declarations, etc, the
>> ones you use with `(declare (debug...))` in the beginning of functions
>> (sometimes not)?  They're an entirely different concept and yet known
>> as "declarations" for any Elisp (or Common Lisp) user. It's awkward to
>> find both mixed when you're thinking of just one.
> That's not what I'm suggesting.  I suggest that find-declarations when
> run on a generic should give the cl-defgeneric, and find-implementations
> should give the cl-defmethods.
> 
> I think that would be a really helpful new feature, apart from all the
> rest of this.
> 
> If the term "declarations" and "implementations" are a problem, we could
> totally give them another name.
> 
> How about "find-generic" and "find-methods"?  Just literally copying
> what Elisp calls them.  Then we can set aside all this worry about
> sharing terms with LSP.

Perhaps I should state also here that we shouldn't worry about sharing 
LSP's terms. Not these ones, anyway.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 10/11/2023 12:06, João Távora wrote:
> Really, what if the language du jour sometime in the future becomes
> something like C++ again? Will you add 'type-instantiation,
> 'type-specialization,  'partial-specialization,  'concept, and so on?
> Or the C++ backend somehow manages to index the loci of memory
> allocation to a given arena names at point.  Will 'memory-allocations'
> also appear in xref.el?

BTW, do you know of any LSP servers for C/C++ have 
implementation-specific actions/endpoints for any of the things that you 
mentioned above?

 > When I get around to finishing refactor.el which will inherit most of
 > Eglot's UI for doing refactorings, I don't plan to burn the concepts
 > of "organizeImports" and "quickfix" into it.  They are LSP things.

I things refactorings are inherently slower process, so having less 
dedicated commands and more typing won't be a problem.

Do you have any WIP code/patch/branch for refactor.el? I was thinking of 
giving it a shot soon-ish, but it would start with extracting some code 
from Eglot anyway.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 09/11/2023 22:44, João Távora wrote:
> On Thu, Nov 9, 2023 at 7:23 PM Spencer Baugh<sbaugh@janestreet.com>  wrote:
>> João Távora<joaotavora@gmail.com>  writes:
>>> On Wed, Nov 8, 2023 at 11:34 PM Dmitry Gutov<dgutov@yandex.ru>  wrote:
>>>>> But neither was available at the time, so I did those commands and they won't
>>>>> be obsoleted any time soon.
>>>> Reasons being..?
>>> That we can't come up with alternatives to exactly that interface,
>>> obviously.  When you do, I'll obsolete them.  But do you want to hardcode
>>> things like LSP "typeDefinition" and Sly's "who-macroexpands" somewhere
>>> in xref.el?  How would that work?
>> Right, so if we have kinds defined in the core for "declaration",
>> "implementation", and "type-definition" (maybe not with those exact
>> names), then we can have xref-find-declaration,
>> xref-find-implementation, and xref-find-type-definition (again maybe not
>> with those exact names), and just do
>>
>> (define-obsolete-function-alias 'eglot-find-declaration 'xref-find-declaration)
>> (define-obsolete-function-alias 'eglot-find-implementation 'xref-find-implementation)
>> (define-obsolete-function-alias 'eglot-find-typeDefinition 'xref-find-type-definition)
>>
>> Is that possible?
> For Dmitry to answer

FWIW, the above is one of the alternatives that I had in mind when I 
mentioned obsoleting the said commands.

The other alternative looked like this, though:

(make-obsolete 'eglot-find-declaration "use `xref-find-extra'" "...")
(make-obsolete 'eglot-find-implementation "use `xref-find-extra'" "...")
(make-obsolete 'eglot-find-typeDefinition "use `xref-find-extra'" "...")

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 10/11/2023 19:36, Spencer Baugh wrote:

> And refactor-code-actions will behave like the current
> eglot-code-actions, and prompt for code actions iff there are multiple
> possible actions that can be taken?  I know you prefer that workflow
> personally, but some people like keybindings to do common tasks
> directly.
> 
> For example, do you expect to support symbol renaming in refactor.el?
> That if anything seems like something which deserves its own binding.

I think the plan was to have a binding for refactor-rename, and dispatch 
the rest of action dynamically.

> That being said, maybe we could have mode-specific commands and bindings
> for these things.  So if something is supported in a language, a major
> mode could make a binding for that.  And then, just, we might encourage
> using the same bindings in each mode that binds something.

That could be discussed.

>> But I think we have made both our positions clear.  I understand what
>> you want to do and I am completely against it, but this is not just
>> up for me to decide.  The only thing that is, is that if I like the
>> new interfaces in xref.el I will implement them in eglot.el.  The
>> UI that xref.el provides I can't control.  Of course I recommend
>> keeping at least the current UI of xref-find-extra, not least because
>> it's something that has actually been implemented and I have
>> spent time integrating.  And I also like your idea of KIND=nil to
>> make it show a sectioned listing using whatever kinds the backend
>> declares as section headings.
> 
> OK, fair enough, and I can see your point.  I think we're both clear.
> 
> Putting any talk about common xref APIs aside... Just to be clear, my
> understanding is that you have no problem with the idea of:
> 
> - Teaching the elisp--xref-backend about separate 'cl-defgeneric and
>    'cl-defmethods kinds, which return the obvious things
> 
> - Adding commands elisp-find-cl-{defgeneric,defmethods} which use those
>    kinds (and which fall back to the regular definitions)
> 
> - Adding bindings in emacs-lisp-mode for calling those commands
> 
> I'd like to do that independent of whatever we end up with, so just
> making sure you're not opposed to that, even though you might not be
> interested in using it.

I think your idea of using 
xref-find-declarations/xref-find-implementations for that would be a 
little more compact. With a user option or not.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Fri, Nov 10, 2023 at 5:36 PM Spencer Baugh <sbaugh@janestreet.com> wrote:
>
> João Távora <joaotavora@gmail.com> writes:
> > On Fri, Nov 10, 2023 at 12:15 PM Spencer Baugh <sbaugh@catern.com> wrote:
> >> > When I get around to finishing refactor.el which will inherit most of
> >> > Eglot's UI for doing refactorings, I don't plan to burn the concepts of
> >> > "organizeImports" and "quickfix" into it.  They are LSP things.
> >>
> >> Sure, that sounds good.  But do you hope to eventually add a standard
> >> keybinding for accessing refactor.el?  I hope so.
> >
> > Sure, for refactor-code-actions, which will work much like xref-find-extras,
> > aka xref-find-by-kind (or maybe it should be named only xref-find, or
> > why not only
> > xref).
> >
> > But there won't be a command refactor-quickfix or refactor-organizeImports
> > because not all potential refactor.el backends support those LSP concepts.
> >
> > It's exactly the same here.
>
> And refactor-code-actions will behave like the current
> eglot-code-actions, and prompt for code actions iff there are multiple
> possible actions that can be taken?  I know you prefer that workflow
> personally, but some people like keybindings to do common tasks
> directly.

Yes, and there will be eglot-* commands for those common LSP-specific
tasks yet, just like there are these commands today.

eglot-code-action-quickfix
eglot-code-action-organize-imports

etc.  The only difference is not user visibile and is that they
are to be defined internally with a macro from refactor.el.

> For example, do you expect to support symbol renaming in refactor.el?
> That if anything seems like something which deserves its own binding.

Yes, it will have its own command, and you can discuss with the
bindings people to bind it to some key, I won't oppose it.
refactor-rename is analogous xref-find-definition, it's a concept
that we're very confident that we can provide in all refactoring
backends, LSP or not.

> That being said, maybe we could have mode-specific commands and bindings
> for these things.  So if something is supported in a language, a major
> mode could make a binding for that.  And then, just, we might encourage
> using the same bindings in each mode that binds something.

Aha!  Yes, of course.  That's the Emacs way.  The advent of LSP sometimes
makes us forget that LSP's tradeoffs are there because LSP doesn't have
the concept of mode or any kind of programming language.  That's why
it is so stiff sometimes.  But we do, we have a existing abstraction
for dealing with languages, the major mode, a pretty flexible one.  And
we have a mature programming language.


This is why indiscriminately borrowing stuff from LSP should really
be frowned upon.  Doesn't mean flat-out reject it, but scrutinized
because tying ourselves down to LSP's inherent impotence is not
something we want to do in an editor with our time-proven strengths.

Instead, what we should do is lobby LSP to become more powerful.
We won't get it to run Elisp, but a lot of things could be done to
make Emacs/LSP integration even better.  Just one example, be able
to pass an arbitrary identifier to LSP's `textDocument/findFoo`.

But I'm the first one who should be doing that, and I'm not, because
I just don't have time, and the sad truth is that Emacs doesn't
carry that much weight  over there, unfortunately.

So what we should do in the long IMO is take advantage of Emacs
inherent advantage for language integrations so undeniably good
that more users come use it and then we have more negotiating
power.

So yes, c++-ts-mode can and should define, and possibly bind,  all those
xref.el and refactor.el  things that make sense for C++.  And if binding
consistency is a goal, then major mode authors should coordinate.  And we
can even have Elisp constructs that help this coordination and make
uncoordination difficult.

- Putting any talk about common xref APIs aside... Just to be clear, my
> understanding is that you have no problem with the idea of:
>
> - Teaching the elisp--xref-backend about separate 'cl-defgeneric and
>   'cl-defmethods kinds, which return the obvious things

Yes, no problem I think.  Didn't think much about this, but bear in mind
defgeneric forms can also have specific methods inside them.  So there
are at least to ways to put methods on generic function, not just
cl-defmethod.

Then macros come along.  I'm not sure the Elisp backend is good enough
to know that a macro expanding to cl-defmethod is a method-defining macro.
SLIME and SLY can most definitely do it.  But that's another problem.

> - Adding commands elisp-find-cl-{defgeneric,defmethods} which use those
>   kinds (and which fall back to the regular definitions)

No problem.  But have a look at how SLY and SLIME do this.  It's been
tested for a very long time today there.

I hope I can update SLY to use xref.el .  I tried once many years ago,
but failed, not really because of xref.el but because SLY's code is
very convoluted.>

> - Adding bindings in emacs-lisp-mode for calling those commands
> I'd like to do that independent of whatever we end up with, so just
> making sure you're not opposed to that, even though you might not be
> interested in using it.

No, not opposed at all to doing it.  Whether I'm interested in
using it it varies in time greatly.  My personal workflow changes
slowly over time, and of course it is influenced by the design
decisions, it is not set in stone.  But it does stick, to tell you
the truth, I'm still using C-h f foo RET C-x o TAB RET to go to
functions definitions when I don't have the symbol name under point.
I know there's C-u M-. but I'm so used to that other workflow that
I've used for 20 years, that I just can't let go.  Plus I
like that it makes me gloss over the docstring briefly.  When
programing C++ with Eglot, I do use C-u M-. of course.


João

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Sat, Nov 11, 2023 at 1:08 AM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 09/11/2023 22:44, João Távora wrote:
> > On Thu, Nov 9, 2023 at 7:23 PM Spencer Baugh<sbaugh@janestreet.com>  wrote:
> >> João Távora<joaotavora@gmail.com>  writes:
> >>> On Wed, Nov 8, 2023 at 11:34 PM Dmitry Gutov<dgutov@yandex.ru>  wrote:
> >>>>> But neither was available at the time, so I did those commands and they won't
> >>>>> be obsoleted any time soon.
> >>>> Reasons being..?
> >>> That we can't come up with alternatives to exactly that interface,
> >>> obviously.  When you do, I'll obsolete them.  But do you want to hardcode
> >>> things like LSP "typeDefinition" and Sly's "who-macroexpands" somewhere
> >>> in xref.el?  How would that work?
> >> Right, so if we have kinds defined in the core for "declaration",
> >> "implementation", and "type-definition" (maybe not with those exact
> >> names), then we can have xref-find-declaration,
> >> xref-find-implementation, and xref-find-type-definition (again maybe not
> >> with those exact names), and just do
> >>
> >> (define-obsolete-function-alias 'eglot-find-declaration 'xref-find-declaration)
> >> (define-obsolete-function-alias 'eglot-find-implementation 'xref-find-implementation)
> >> (define-obsolete-function-alias 'eglot-find-typeDefinition 'xref-find-type-definition)
> >>
> >> Is that possible?
> > For Dmitry to answer
>
> FWIW, the above is one of the alternatives that I had in mind when I
> mentioned obsoleting the said commands.
>
> The other alternative looked like this, though:
>
> (make-obsolete 'eglot-find-declaration "use `xref-find-extra'" "...")
> (make-obsolete 'eglot-find-implementation "use `xref-find-extra'" "...")
> (make-obsolete 'eglot-find-typeDefinition "use `xref-find-extra'" "...")

No, I'm not obsoleting those commands, sorry.  Not least because
new LSP things may come along that xref.el won't be able to respond
in time with.  So those commands are staying there.

The "declaration/implementation/typeDefinition" triplet is an LSP
invention and Eglot is for LSP, so eglot- commands they will be,
even if sometimes later in an eglot-commands.el file along with
other ones.  And that's even if we do import that LSP invention
into xref (which I think we shouldn't -- see my reply to Spencer).

João

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Sat, Nov 11, 2023 at 1:05 AM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 10/11/2023 12:06, João Távora wrote:
> > Really, what if the language du jour sometime in the future becomes
> > something like C++ again? Will you add 'type-instantiation,
> > 'type-specialization,  'partial-specialization,  'concept, and so on?
> > Or the C++ backend somehow manages to index the loci of memory
> > allocation to a given arena names at point.  Will 'memory-allocations'
> > also appear in xref.el?
>
> BTW, do you know of any LSP servers for C/C++ have
> implementation-specific actions/endpoints for any of the things that you
> mentioned above?

Clangd, whose source code I've recently become familiar with,
has the capability of doing all of those but my vapourware
'memory-allocations'.  It doesn't expose endpoints to them, I think,
but it's not fundamentally hard to do (clangd has a very rich index
derived from the LLVM AST processing machinery, as you probably know).

>  > When I get around to finishing refactor.el which will inherit most of
>  > Eglot's UI for doing refactorings, I don't plan to burn the concepts
>  > of "organizeImports" and "quickfix" into it.  They are LSP things.
>
> I things refactorings are inherently slower process, so having less
> dedicated commands and more typing won't be a problem.
>
> Do you have any WIP code/patch/branch for refactor.el? I was thinking of
> giving it a shot soon-ish, but it would start with extracting some code
> from Eglot anyway.

Yes, I have a refactor.el started, but  ot sidetracked.  I'll pull it
up soon, it's still unfinished but I would love some help and am
fairly certain we can collaborate to get at least the basics working.
I think the xref-backend-extra-kinds/xref-backend-extra-defs is
the analogue mechanism I was looking for and couldn't quite commit
to but your patch and  this discussion has helped me see it is a
simple and viable way.

Anyway, will try to do that coming weeks, do ping me if I forget,
we can even pair-program via tmate or something.

João

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Sat, Nov 11, 2023 at 12:58 AM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 09/11/2023 22:44, João Távora wrote:
> > So I thought, that about 6 months ago we had established that
> > "definition" and "reference" are two relatively safe concept to
> > keep in xref.el, but other concepts should not be in there,
> > because this doesn't scale well and could imposes awkward
> > structure and hacks into an unknown number of backends.
> >
> > Has everyone changed their mind?
>
> My impression is that the first feedback from our patches actually made
> people excited about things that _weren't_ included in the
> previously-discussed plan, so it seems like a good idea to re-evaluate
> it. Though not necessarily redo it all.
>
> Though I guess this particular mailing list might be biased in favor of
> particular type of users (keyboard-driven, faster interaction as a
> priority).

yeah, the sample size is so small that's it's more than certainly
biased.

> > What exactly bothers you
> > about eglot-find-declaration/implementation/typeDefinition?
> > In LSP-land these concepts_do_  make sense, because the
> > LSP standard prescribes what servers should do with them.
>
> I think we've actually discovered that these kind of make sense for
> Elisp too. They might not match the current implementation, but
> conceptually, in some future, they might.

Yeah, they "kind of" make sense, but not very well and IMHO that's
not a detail, but a big red flag.  It's clear, and I understand
your point, that if we were to squint to contort to them we could
have xref provide a consistent "works everywhere" UI.  But as
Spencer noted it would be "works _mostly_ everywhere".  And, as I
noted, the contortion might lead us up awkward alley, such as
finding a list mixing `declare-function` and `declare (compiler-macro`
in the bag of xref-find-declarations.

And Spencer even wanted to put cl-defgeneric in that bag, which I
can fully understand, as cl-defgeneric can also be viewed as a
declaration.  But its a completely different concept from those
two.

And then SLY would probably make the bag slightly different yet.

And every time you use Eglot to connect to a different LSP language
server you see a different bag of concepts yet again (but that's
not our fault).

So that sought-for consistency vanishes quickly.

So, I think that if what we want is ultimately to promote UI
consistency for programmers of different languages, we can come
up with xref.el helpers for major modes (and eglot.el) to help
achieve coordinate "a degree of consistency", as Spencer wishes.

But we have to note that full consistency is an utopia only fixed
by fixing the root problem, which is that languages are different,
and I think we don't want to "fix" that.

João

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

João Távora <joaotavora@gmail.com> writes:
> On Fri, Nov 10, 2023 at 5:36 PM Spencer Baugh <sbaugh@janestreet.com> wrote:
>> For example, do you expect to support symbol renaming in refactor.el?
>> That if anything seems like something which deserves its own binding.
>
> Yes, it will have its own command, and you can discuss with the
> bindings people to bind it to some key, I won't oppose it.
> refactor-rename is analogous xref-find-definition, it's a concept
> that we're very confident that we can provide in all refactoring
> backends, LSP or not.
>
>> That being said, maybe we could have mode-specific commands and bindings
>> for these things.  So if something is supported in a language, a major
>> mode could make a binding for that.  And then, just, we might encourage
>> using the same bindings in each mode that binds something.
>
> Aha!  Yes, of course.  That's the Emacs way.  The advent of LSP sometimes
> makes us forget that LSP's tradeoffs are there because LSP doesn't have
> the concept of mode or any kind of programming language.  That's why
> it is so stiff sometimes.  But we do, we have a existing abstraction
> for dealing with languages, the major mode, a pretty flexible one.  And
> we have a mature programming language.
>
>
> This is why indiscriminately borrowing stuff from LSP should really
> be frowned upon.  Doesn't mean flat-out reject it, but scrutinized
> because tying ourselves down to LSP's inherent impotence is not
> something we want to do in an editor with our time-proven strengths.
>
> Instead, what we should do is lobby LSP to become more powerful.
> We won't get it to run Elisp, but a lot of things could be done to
> make Emacs/LSP integration even better.  Just one example, be able
> to pass an arbitrary identifier to LSP's `textDocument/findFoo`.
>
> But I'm the first one who should be doing that, and I'm not, because
> I just don't have time, and the sad truth is that Emacs doesn't
> carry that much weight  over there, unfortunately.
>
> So what we should do in the long IMO is take advantage of Emacs
> inherent advantage for language integrations so undeniably good
> that more users come use it and then we have more negotiating
> power.
>
> So yes, c++-ts-mode can and should define, and possibly bind,  all those
> xref.el and refactor.el  things that make sense for C++.  And if binding
> consistency is a goal, then major mode authors should coordinate.  And we
> can even have Elisp constructs that help this coordination and make
> uncoordination difficult.

Oh, this is very convincing.  In fact, I am fully convinced.

Major modes should define their own commands for the functionality
supported by that language.  So e.g. c-ts-mode could define
c-find-signature, which has a clear meaning for C.  And likewise for
refactorings which only make sense in a given language.

And we could reserve single-character bindings under M-' (or something)
for the major mode.  For example:
(keymap-set emacs-lisp-mode-map "M-' g" emacs-lisp-find-generic)
(keymap-set c-mode-map "M-' s" c-find-signature)

Reliably using bindings under M-' even if those bindings point to
mode-specific commands is enough consistency for me.  Using literally
the same command in each major-mode is not necessary.

That's everything that I wanted from adding xref-find-declaration and so
on in the core, so I agree, actually, we don't need standard kinds in
the core or anything, as long as we actually do something to encourage
common bindings.

(And maybe we could use M-' M-r as a prefix for mode-specific
refactoring commands.  Perhaps if we use this M-' prefix it can be the
start of encouraging more binding consistency between programming
language major modes, which I find is something sadly lacking.  Maybe at
first there could be a customization which changes M-', and we could
hope to change it by default in a subsequent release.)

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

Dmitry Gutov <dmitry@gutov.dev> writes:
> On 09/11/2023 19:07, Spencer Baugh wrote:
>> Custom kinds would still require you to type in the kind symbol.  (Or a
>> backend or mode can make a "quick-access" command specific to that kind,
>> if they want.)
> If we leave all the UI decisions to the backends (both extra commands
> and the dispatch which kind to use in the "unifying" command as well),
> then there might be little point in adding extra infrastructure to
> xref.el itself.
>
> xref-show-xrefs is a public function, FETCHER just needs to return a
> list of xrefs, the same data that backend functions like
> xref-backend-definitions return.

I think we plausibly leave UI decisions to the *major-modes*.

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

Dmitry Gutov <dmitry@gutov.dev> writes:
>>>> If you mean something a non-default version of xref-find-definitions, or
>>>> another binding for another basically-identical command, sure, I think
>>>> that would be good.  We can have both that command *and* have a command
>>>> which prompts for KIND and defaults to "all".
>>>
>>> We could indeed, if we decide what to call it. "extras" seems out
>>> (since it would include both definitions and additional reference
>>> kinds). Just "xref-find-by-kind"? Then it's less obvious to have the
>>> default behavior showing all.
>> Actually, maybe this should be a backend-specific thing.  The
>> backend
>> could specify a default along with its list of kinds.  And if we request
>> that default kind (which might be 'all), then the backend will decide
>> what kinds to send us.
>
> If the backend specifies the default behavior of xref-find-by-kind,
> then it's what, another generic method to override?

The default could just be the first in the list of kinds returned by the
backend.

>> Actually, it occurs to me that if we had an xref-find-implementations
>> command from the start, with a convenient binding, maybe
>> xref-find-definitions would just only show the cl-defgeneric, and jump
>> to it right away.  And only if you hit xref-find-implementations would
>> you jump to the cl-defmethods.  We can't make that change now, but I
>> don't think it would be worse!  And if a backend wants a design like
>> that, I think the backend should be able to have it.
>
> Not sure I would like the result that you are describing, but indeed,
> if a backend wants to do it like this, nothing will be stopping it.
>
> Are LSP servers behaving like this? E.g. jdt-ls jumping to the method
> definition in an interface, even when the owner type is a statically
> known class?

I don't know about jdt-ls or indeed about LSP servers, but the xref
backend for Merlin (used for OCaml) by default has xref-find-definitions
jump to the signature of a function, not its implementation.

There's a user customization to toggle whether it jumps to the signature
or the implementation.  And if the customization is in "implementation"
mode, there's no way (through xref) to jump to the signature, and vice
versa.  Adding a way for xref to support jumping to the signature in
cases like that is in fact why I made this thread in the first place. :)

>>>> Maybe the backend could decide what kinds get included in "all".  Then
>>>> it could deliberately avoid including anything "reference-like".
>>>
>>> Could we want several such commands? E.g. one for "all definition-like
>>> hits" and another for "all reference-like hits"? With separate sets of
>>> kinds for definitions and references?
>> Possibly, but my suggestion is that "all definition-like hits" and
>> "all
>> reference-like hits" should just be kinds exposed by the backend.
>
> Two different generics returning lists of kinds, or just one? If two,
> then could one give an example of a "reference-like" kind except for
> "references"? I've taken a couple of guesses, but I'm exactly sure of
> them.

One generic.  Just "all-references" is a kind, and "all-definitions" is
another kind.

>>> It is less flexible because any time a backend wants to use a new
>>> "kind", it will need to expend some effort and add it to the core
>>> somehow. Define a new command or two and assign them to the said
>>> prefix map. This could also lead to conflicts if backend authors don't
>>> do this carefully enough.
>> No, I suggest that xref-backend-extra-kinds should be able to return
>> whatever arbitrary symbols it wants.  Just, if its "kinds" match the
>> kinds used in the core, then it will benefit from the commands and
>> bindings defined by the core.
>> Perfectly extensible, but still benefiting from standardization.
>
> Would the user be able to input/use the kinds that are in the list
> returned by the backend but are not in the set of "kinds used in the
> core"? If yes, how? Using a backend-provided command?

With xref-find-extras, which prompts for a kind.

>>>> I do think that if we go with an API which has any notion of "kinds", we
>>>> should have some standard "kinds" in the core like implementation,
>>>> declaration, type-definition.  I don't see any reason not to do that.
>>>
>>> We could have a "registry" of kinds, associating each of them with a
>>> key. Then the result could be more dynamic, e.g. M-' could be bound to
>>> a command that reads the key and performs the dispatch to the
>>> corresponding search (similar to project-switch-project).
>>>
>>> And the users (though probably not packages) would later be able to
>>> customize that mapping, adding new kinds or modifying the keys.
>>>
>>> With this approach we don't end up with many xref-find-xyz commands,
>>> most of which just clutter the namespace, staying unused for a large
>>> proportion of the users. OTOH, one wouldn't be able to examine the
>>> prefix map and its contents and definitions with (M-' C-h) -- that's a
>>> minor reduction in usability.
>> The only difference between this and a regular keymap which maps
>> keys to
>> commands is:
>> 
>>> we don't end up with many xref-find-xyz commands
>> But we can also avoid that by just... not defining many such
>> commands
>> and kinds.  A mode will always have the ability to define its own
>> mode-specific kinds in the mode's own namespace, which don't clutter
>> xref-*.
>
> Indeed.
>
> But if we don't define a dynamic dispatcher at all, it seems we won't
> need the backend to tell us about the supported kinds either. It would
> just support some and perhaps return error for unknown kinds.

True, in theory we could do that.  We could even provide some way to ask
the backend, "do you support kind X?" which could be useful to avoid
making bindings for something not supported by the backend.

So we could start without xref-backend-extra-kinds, and probably could
get decently far.

However, I think the dynamic dispatcher is useful for one very important
thing: when the backend adds support for a new kind which the major-mode
and/or core has not yet added support for.  Users will definitely want
to be able to use features of the backend that their major-mode/the core
doesn't know about.

>>>> Actually, this example has just convinced me that I definitely want
>>>> "kind-specific commands", even for Elisp.  That would be great.
>>>> For functions without separate declarations, find-implementation and
>>>> find-declaration could jump to the same place.  In some languages, maybe
>>>> that's just always what happens.
>>>
>>> Aren't find-implementation included in find-definition anyway? Or
>>> *are* the same as find-definition, for example, in LSP's approach?
>>> Aside from the fact that the latter also works for variables and other
>>> symbols.
>> No, find-implementation shows all the implementations of a generic
>> function/interface method.  (e.g. cl-defmethod)
>> find-definition (as I understand it) jumps to the interface
>> definition
>> itself (e.g. cl-defgeneric).  Which in my experience in Elisp, is
>> usually what I want; I want to see the documentation and default
>> implementation and surrounding code, before I look at a specific
>> implementation.
>
> I usually want to jump to where the function is
> implemented. Preferably, for the given type (though we don't support
> that). Perhaps I would use the proposed xref-find-implementations for
> this purpose, but it's bound to have a more complex key sequence than
> 'M-.'.
>
>> I think that's a pretty reasonable way for it to work.  Actually, maybe
>> we could add a customization point for the Elisp backend so that a user
>> can choose to make find-definition work that way, if we do add a
>> find-implementation command.
>
> We could.
>
>> There's also find-declaration - for Elisp, I think that one *is* the
>> mostly same as find-definition, since Elisp doesn't have separate
>> declarations.  (declare-function is kind of a different thing.  Maybe
>> find-declaration should jump to the declare-function instance in the
>> current file, if it can, but for now it's fine.)
>
> Yep. Though for cases where there is no corresponding meaningful
> implementation, I would rather abort with an error. Doing otherwise
> seems misleading (like we have special handing of this case, but we
> don't).

BTW, talking with Joao has convinced me that actually these should be
major-mode specific.  So we can have emacs-lisp-find-generic or
whatever, and that can have all kinds of special behavior and a useful
docstring that takes about Elisp things, but ultimately it calls into
the xref backend with a specific kind.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 11/11/2023 14:39, Spencer Baugh wrote:
> Dmitry Gutov<dmitry@gutov.dev>  writes:
>> On 09/11/2023 19:07, Spencer Baugh wrote:
>>> Custom kinds would still require you to type in the kind symbol.  (Or a
>>> backend or mode can make a "quick-access" command specific to that kind,
>>> if they want.)
>> If we leave all the UI decisions to the backends (both extra commands
>> and the dispatch which kind to use in the "unifying" command as well),
>> then there might be little point in adding extra infrastructure to
>> xref.el itself.
>>
>> xref-show-xrefs is a public function, FETCHER just needs to return a
>> list of xrefs, the same data that backend functions like
>> xref-backend-definitions return.
> I think we plausibly leave UI decisions to the*major-modes*.

I don't think so: major modes provide information to common facilities 
(other examples are font-lock or indent-line-function or syntax tables), 
and then the common code uses that information in the same fashion (e.g. 
obeys font-lock-maximum-decoration).

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 11/11/2023 14:38, Spencer Baugh wrote:
> So e.g. c-ts-mode could define
> c-find-signature, which has a clear meaning for C.

And c-mode will not.

> And likewise for
> refactorings which only make sense in a given language.

And then it will fail because the user didn't enable a minor mode which 
would give access to said functionality.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 11/11/2023 13:22, João Távora wrote:

>>>> Is that possible?
>>> For Dmitry to answer
>>
>> FWIW, the above is one of the alternatives that I had in mind when I
>> mentioned obsoleting the said commands.
>>
>> The other alternative looked like this, though:
>>
>> (make-obsolete 'eglot-find-declaration "use `xref-find-extra'" "...")
>> (make-obsolete 'eglot-find-implementation "use `xref-find-extra'" "...")
>> (make-obsolete 'eglot-find-typeDefinition "use `xref-find-extra'" "...")
> 
> No, I'm not obsoleting those commands, sorry.  Not least because
> new LSP things may come along that xref.el won't be able to respond
> in time with.  So those commands are staying there.

So what if that happens? You'll add the new commands to Eglot then. Just 
like any other backend is supposed to do if they support commands not in 
the built-in set.

> The "declaration/implementation/typeDefinition" triplet is an LSP
> invention and Eglot is for LSP, so eglot- commands they will be,
> even if sometimes later in an eglot-commands.el file along with
> other ones.  And that's even if we do import that LSP invention
> into xref (which I think we shouldn't -- see my reply to Spencer).

Eglot is not the only LSP client for Emacs. I think there are about 4 of 
them at the moment...? Each with their own command definitions.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 11/11/2023 13:30, João Távora wrote:

>> BTW, do you know of any LSP servers for C/C++ have
>> implementation-specific actions/endpoints for any of the things that you
>> mentioned above?
> 
> Clangd, whose source code I've recently become familiar with,
> has the capability of doing all of those but my vapourware
> 'memory-allocations'.  It doesn't expose endpoints to them, I think,
> but it's not fundamentally hard to do (clangd has a very rich index
> derived from the LLVM AST processing machinery, as you probably know).

So these features that are not available in the VS Code plugin yet? I 
guess we'll wait until that happens. But then it seems like Eglot will 
have to define the corresponding commands.

>>   > When I get around to finishing refactor.el which will inherit most of
>>   > Eglot's UI for doing refactorings, I don't plan to burn the concepts
>>   > of "organizeImports" and "quickfix" into it.  They are LSP things.
>>
>> I things refactorings are inherently slower process, so having less
>> dedicated commands and more typing won't be a problem.
>>
>> Do you have any WIP code/patch/branch for refactor.el? I was thinking of
>> giving it a shot soon-ish, but it would start with extracting some code
>> from Eglot anyway.
> 
> Yes, I have a refactor.el started, but  ot sidetracked.  I'll pull it
> up soon, it's still unfinished but I would love some help and am
> fairly certain we can collaborate to get at least the basics working.

I'm happy to take a look.

> I think the xref-backend-extra-kinds/xref-backend-extra-defs is
> the analogue mechanism I was looking for and couldn't quite commit
> to but your patch and  this discussion has helped me see it is a
> simple and viable way.

Indeed, I think that approach should fit it well.

> Anyway, will try to do that coming weeks, do ping me if I forget,
> we can even pair-program via tmate or something.

Sounds interesting. My Emacs doesn't really function in the terminal, 
but we could try.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 11/11/2023 13:44, João Távora wrote:
> On Sat, Nov 11, 2023 at 12:58 AM Dmitry Gutov <dmitry@gutov.dev> wrote:
>>
>> On 09/11/2023 22:44, João Távora wrote:
>>> So I thought, that about 6 months ago we had established that
>>> "definition" and "reference" are two relatively safe concept to
>>> keep in xref.el, but other concepts should not be in there,
>>> because this doesn't scale well and could imposes awkward
>>> structure and hacks into an unknown number of backends.
>>>
>>> Has everyone changed their mind?
>>
>> My impression is that the first feedback from our patches actually made
>> people excited about things that _weren't_ included in the
>> previously-discussed plan, so it seems like a good idea to re-evaluate
>> it. Though not necessarily redo it all.
>>
>> Though I guess this particular mailing list might be biased in favor of
>> particular type of users (keyboard-driven, faster interaction as a
>> priority).
> 
> yeah, the sample size is so small that's it's more than certainly
> biased.

Most likely. It's too bad our actual audience here is quite small, and 
we don't have other reliable ways to collect feedback to prototypes.

Dropping an implementation into master is one way to test out a 
hypothesis, but it has inherent problems: the feedback comes late 
anyway, and quite often we'd end up having to maintain half-bakes 
features or unfortunate capabilities this way simply because they had 
been added and time had passed.

>>> What exactly bothers you
>>> about eglot-find-declaration/implementation/typeDefinition?
>>> In LSP-land these concepts_do_  make sense, because the
>>> LSP standard prescribes what servers should do with them.
>>
>> I think we've actually discovered that these kind of make sense for
>> Elisp too. They might not match the current implementation, but
>> conceptually, in some future, they might.
> 
> Yeah, they "kind of" make sense, but not very well and IMHO that's
> not a detail, but a big red flag.  It's clear, and I understand
> your point, that if we were to squint to contort to them we could
> have xref provide a consistent "works everywhere" UI.  But as
> Spencer noted it would be "works _mostly_ everywhere".  And, as I
> noted, the contortion might lead us up awkward alley, such as
> finding a list mixing `declare-function` and `declare (compiler-macro`
> in the bag of xref-find-declarations.

Why contort? From where I'm standing, 
declarations/implementations/typeDefinitions apply to almost any 
object-oriented language (C++/Java/Ruby/Python/JS), and even some 
functional ones.

Declarations might be less of a thing in some of those (mostly dynamic 
ones), but overall the list even makes a certain sense for Elisp (and 
similarly CL, I guess), at least in the OOP parts of them. And where 
they don't, one or two of these commands will fail to work, big deal.

What's the alternative? To have many similar commands

   ruby-find-implementation
   c++-find-implementations
   python-find-implementations
   js-find-implementations

that all do the same thing inside: call the Xref backend's "find 
implementations" mechanism? It will be especially great to use if some 
major mode authors bind that command to one key, and others to another.

> And Spencer even wanted to put cl-defgeneric in that bag, which I
> can fully understand, as cl-defgeneric can also be viewed as a
> declaration.  But its a completely different concept from those
> two.

If Elisp doesn't have any other kind of declarations, what is the worry? 
Let's put the ones it has.

Elisp's bag is different in that it will have separate kinds for 
faces/features/methods and, potentially, variables as well. But those 
are already in find-func.el and don't need new bindings.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 11/11/2023 15:00, Spencer Baugh wrote:

>>>> We could indeed, if we decide what to call it. "extras" seems out
>>>> (since it would include both definitions and additional reference
>>>> kinds). Just "xref-find-by-kind"? Then it's less obvious to have the
>>>> default behavior showing all.
>>> Actually, maybe this should be a backend-specific thing.  The
>>> backend
>>> could specify a default along with its list of kinds.  And if we request
>>> that default kind (which might be 'all), then the backend will decide
>>> what kinds to send us.
>>
>> If the backend specifies the default behavior of xref-find-by-kind,
>> then it's what, another generic method to override?
> 
> The default could just be the first in the list of kinds returned by the
> backend.

So... one backend would show implementations, another "all kinds", and 
the third one "definitions"? That seems odd.

Moreover, if I don't have any idea what the average backend would do, 
it's harder to write the docstring, or describe the command in the 
manual (check out the current manual entries related to Xref; that 
wasn't me who wrote them).

I was also thinking to suggest the command name 
xref-find-all-definitions, but that one clearly implies what its 
behavior will be when called without prefix.

>>> Actually, it occurs to me that if we had an xref-find-implementations
>>> command from the start, with a convenient binding, maybe
>>> xref-find-definitions would just only show the cl-defgeneric, and jump
>>> to it right away.  And only if you hit xref-find-implementations would
>>> you jump to the cl-defmethods.  We can't make that change now, but I
>>> don't think it would be worse!  And if a backend wants a design like
>>> that, I think the backend should be able to have it.
>>
>> Not sure I would like the result that you are describing, but indeed,
>> if a backend wants to do it like this, nothing will be stopping it.
>>
>> Are LSP servers behaving like this? E.g. jdt-ls jumping to the method
>> definition in an interface, even when the owner type is a statically
>> known class?
> 
> I don't know about jdt-ls or indeed about LSP servers, but the xref
> backend for Merlin (used for OCaml) by default has xref-find-definitions
> jump to the signature of a function, not its implementation.

Unfortunately, I don't have an intuition for how this works for OCaml in 
other editors, or worked in the past, to compare.

> There's a user customization to toggle whether it jumps to the signature
> or the implementation.  And if the customization is in "implementation"
> mode, there's no way (through xref) to jump to the signature, and vice
> versa.  Adding a way for xref to support jumping to the signature in
> cases like that is in fact why I made this thread in the first place. :)

Sounds like a good goal.

>>>>> Maybe the backend could decide what kinds get included in "all".  Then
>>>>> it could deliberately avoid including anything "reference-like".
>>>>
>>>> Could we want several such commands? E.g. one for "all definition-like
>>>> hits" and another for "all reference-like hits"? With separate sets of
>>>> kinds for definitions and references?
>>> Possibly, but my suggestion is that "all definition-like hits" and
>>> "all
>>> reference-like hits" should just be kinds exposed by the backend.
>>
>> Two different generics returning lists of kinds, or just one? If two,
>> then could one give an example of a "reference-like" kind except for
>> "references"? I've taken a couple of guesses, but I'm exactly sure of
>> them.
> 
> One generic.  Just "all-references" is a kind, and "all-definitions" is
> another kind.

Then how will xref-find-by-kind (or xref-find-extras) decide whether to 
use xref-show-definitions-function or xref-show-xrefs-function to 
display the result?

>> But if we don't define a dynamic dispatcher at all, it seems we won't
>> need the backend to tell us about the supported kinds either. It would
>> just support some and perhaps return error for unknown kinds.
> 
> True, in theory we could do that.  We could even provide some way to ask
> the backend, "do you support kind X?" which could be useful to avoid
> making bindings for something not supported by the backend.
> 
> So we could start without xref-backend-extra-kinds, and probably could
> get decently far.
> 
> However, I think the dynamic dispatcher is useful for one very important
> thing: when the backend adds support for a new kind which the major-mode
> and/or core has not yet added support for.  Users will definitely want
> to be able to use features of the backend that their major-mode/the core
> doesn't know about.

I thought we might have decided that the backend would create its own 
specific command for that (e.g. called 
eglot-find-higher-order-template-cxx-macroexpansion-definitions).

We could also create a command for dynamic dispatch. Supplying both is 
also fine, I suppose, but a little redundant.

>>> There's also find-declaration - for Elisp, I think that one *is* the
>>> mostly same as find-definition, since Elisp doesn't have separate
>>> declarations.  (declare-function is kind of a different thing.  Maybe
>>> find-declaration should jump to the declare-function instance in the
>>> current file, if it can, but for now it's fine.)
>>
>> Yep. Though for cases where there is no corresponding meaningful
>> implementation, I would rather abort with an error. Doing otherwise
>> seems misleading (like we have special handing of this case, but we
>> don't).
> 
> BTW, talking with Joao has convinced me that actually these should be
> major-mode specific.  So we can have emacs-lisp-find-generic or
> whatever, and that can have all kinds of special behavior and a useful
> docstring that takes about Elisp things, but ultimately it calls into
> the xref backend with a specific kind.

Useful docstrings are nice, and I suppose some modes would define their 
own special commands, but it seems difficult to assign them to the same 
prefix-map, for one thing. If we use a prefix map on M-', 
emacs-lisp-mode couldn't add a binding to it without modifying it globally.

OTOH, modes can use the [remap] binding, but only when the original map 
contains a binding and a command that the current major or minor mode is 
interested in installing a local variant of.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Sun, Nov 12, 2023 at 1:51 AM Dmitry Gutov <dmitry@gutov.dev> wrote:

> I thought we might have decided that the backend would create its own
> specific command for that (e.g. called
> eglot-find-higher-order-template-cxx-macroexpansion-definitions).

Noooo.  Eglot only does LSP things!  The major mode may do that if
it has another alternative backend, or if it knows it is using LSP
with a specific language server, like 'superduperclangdfork'.  So it would be

  c++-ts-mode-find-higher-order-template-cxx-macroexpansion-definitions

, never eglot-foo.

In my proposal, for your example, if the major mode knows LSP is being
used with 'superduperclangdfork' fetching xrefs _may_ be simplified using
some existing eglot.el helpers to be externalized later.  If LSP is
not being used,  the major mode uses whatever it knows to use from
its backend.  The commands _should_ be defined by using xref.el
command-defining macros for a consistent interface to the command
(or defun+xref-show-refs for a potentially less consistent one).

Good grief, so many mails to answer and statements to address.  I'll
be back later with more but I just wanted to quickly correct
this misperception.  This is not what's being proposed, at least by
me, and not by Spencer, I think.

Eglot will only ever do LSP things so the
eglot-find-{implemtation|declaration|typeDefinition} trio is all
you'll find there, at least until LSP invents a fourth or a fifth.

The major mode _may_ bind these as well of course.  Maybe it
may bind them in a xref-defined prefix map _along_ with
<major-mode>-find-shiiz

Anyway, hope to get back to this later.

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 12/11/2023 04:08, João Távora wrote:
> Noooo.  Eglot only does LSP things!

Server-specific extensions are LSP things, in my book. Okay, it might 
not be in eglot.el itself, but then someone will create 
eglot-superduperclang.el to support those extensions.

> The major mode may do that if
> it has another alternative backend, or if it knows it is using LSP
> with a specific language server, like 'superduperclangdfork'.

I'm pretty sure major modes that are specific to the language server in 
use are a bad idea.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Sun, Nov 12, 2023 at 2:10 AM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 12/11/2023 04:08, João Távora wrote:
> > Noooo.  Eglot only does LSP things!
>
> Server-specific extensions are LSP things, in my book. Okay, it might
> not be in eglot.el itself, but then someone will create
> eglot-superduperclang.el to support those extensions.

Could be, but why can't this code live in c++-ts-mode?

> > The major mode may do that if
> > it has another alternative backend, or if it knows it is using LSP
> > with a specific language server, like 'superduperclangdfork'.
>
> I'm pretty sure major modes that are specific to the language server in
> use are a bad idea.

You may have misunderstood my suggestion.

I'm saying code specific to certain languages, LSP-based on not,
should always live in major mode files and have major-mode prefixes.
It's always been like this.  The LSP server program doesn't have
to be available for themajor mode to work, but it works better
with it.

Major modes already do this.  For example, you don't _have_ to
have a Python interpreter program to use python-mode.el to edit
Python code,  but having one enables M-x run-python of course.
And run-python uses comint.el which is a library that python.el
relies on.

Just like it could and should rely on eglot.el if it wants to
offer M-x python-find-super-special-annotation with the
'superduperpyright' LSP server program.

I'm fairly sure other major modes do this.  octave-mode
comes to mind, and probably others.

João

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Sat, Nov 11, 2023 at 9:01 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 11/11/2023 13:30, João Távora wrote:
>
> >> BTW, do you know of any LSP servers for C/C++ have
> >> implementation-specific actions/endpoints for any of the things that you
> >> mentioned above?
> >
> > Clangd, whose source code I've recently become familiar with,
> > has the capability of doing all of those but my vapourware
> > 'memory-allocations'.  It doesn't expose endpoints to them, I think,
> > but it's not fundamentally hard to do (clangd has a very rich index
> > derived from the LLVM AST processing machinery, as you probably know).
>
> So these features that are not available in the VS Code plugin yet? I
> guess we'll wait until that happens. But then it seems like Eglot will
> have to define the corresponding commands.

Not Eglot, no, unless they become part of LSP.  See my other email.

> > Anyway, will try to do that coming weeks, do ping me if I forget,
> > we can even pair-program via tmate or something.
>
> Sounds interesting. My Emacs doesn't really function in the terminal,
> but we could try.

Emacs -Q would function, right? :-) Anyway mine does, fairly reliably and I
have very few personal keybindings.

João

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Sat, Nov 11, 2023 at 9:37 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 11/11/2023 13:44, João Távora wrote:
> > On Sat, Nov 11, 2023 at 12:58 AM Dmitry Gutov <dmitry@gutov.dev> wrote:
> >>
> >> On 09/11/2023 22:44, João Távora wrote:
> >>> So I thought, that about 6 months ago we had established that
> >>> "definition" and "reference" are two relatively safe concept to
> >>> keep in xref.el, but other concepts should not be in there,
> >>> because this doesn't scale well and could imposes awkward
> >>> structure and hacks into an unknown number of backends.
> >>>
> >>> Has everyone changed their mind?
> >>
> >> My impression is that the first feedback from our patches actually made
> >> people excited about things that _weren't_ included in the
> >> previously-discussed plan, so it seems like a good idea to re-evaluate
> >> it. Though not necessarily redo it all.
> >>
> >> Though I guess this particular mailing list might be biased in favor of
> >> particular type of users (keyboard-driven, faster interaction as a
> >> priority).
> >
> > yeah, the sample size is so small that's it's more than certainly
> > biased.
>
> Most likely. It's too bad our actual audience here is quite small, and
> we don't have other reliable ways to collect feedback to prototypes.
>
> Dropping an implementation into master is one way to test out a
> hypothesis, but it has inherent problems: the feedback comes late
> anyway, and quite often we'd end up having to maintain half-bakes
> features or unfortunate capabilities this way simply because they had
> been added and time had passed.

Maybe.  Well Eglot has a fair enough audience, though not sure for
this specific feature.  And I don't see any better alternatives if
collecting feedback is what we want.  IOW, we shouldn't fall prey
to "analysis-paralysis".

> >>> What exactly bothers you
> >>> about eglot-find-declaration/implementation/typeDefinition?
> >>> In LSP-land these concepts_do_  make sense, because the
> >>> LSP standard prescribes what servers should do with them.
> >>
> >> I think we've actually discovered that these kind of make sense for
> >> Elisp too. They might not match the current implementation, but
> >> conceptually, in some future, they might.
> >
> > Yeah, they "kind of" make sense, but not very well and IMHO that's
> > not a detail, but a big red flag.  It's clear, and I understand
> > your point, that if we were to squint to contort to them we could
> > have xref provide a consistent "works everywhere" UI.  But as
> > Spencer noted it would be "works _mostly_ everywhere".  And, as I
> > noted, the contortion might lead us up awkward alley, such as
> > finding a list mixing `declare-function` and `declare (compiler-macro`
> > in the bag of xref-find-declarations.
>
> Why contort? From where I'm standing,
> declarations/implementations/typeDefinitions apply to almost any
> object-oriented language (C++/Java/Ruby/Python/JS), and even some
> functional ones.

What are "declarations" in JS, Java or Ruby and Python?  Perhaps
nothing perhaps entirely different things. Also, there  are a lot of
sub-kinds of declaration in C++.  Is a forward declaration of a
type in C++ also a "declaration" here?  What about a C++20 concept,
is it a type definition?  No right?  Or maybe, depends on who you ask
some say it's a type for types.  Not to mention I think in Elisp we
have at least 3 completely different kinds of "declaration": forward
declarations, (declare ...) forms and defgeneric.  It's odd to mix
them.

Anyway, this is LSP's taxonomy so it's LSP's problem (or rather,
the problem of the poor servers that have to invent ways to shove
things in these categories.  But not our problem by and large if we
don't let  it spread any further than Eglot.  IMO Eglot is where
LSP ends and Emacs starts.

> What's the alternative? To have many similar commands
>
>    ruby-find-implementation
>    c++-find-implementations
>    python-find-implementations
>    js-find-implementations

Depends.  If you are using Eglot/LSP as a backend, you don't need to.
Eglot already gives you the LSP trio.

If you are using a non Eglot backend, then yes.  If you are using
specific backend based on Eglot but with more servers, then yes too,
but only for the extra types you support.

> that all do the same thing inside: call the Xref backend's "find
> implementations" mechanism? It will be especially great to use if some
> major mode authors bind that command to one key, and others to another.

I don't think it'll happen that often, and xref can define a good prefix
map to use.  It's unlikely we or major modes would find keybinding space
outside such a map anyway.

> > And Spencer even wanted to put cl-defgeneric in that bag, which I
> > can fully understand, as cl-defgeneric can also be viewed as a
> > declaration.  But its a completely different concept from those
> > two.
>
> If Elisp doesn't have any other kind of declarations, what is the worry?
> Let's put the ones it has.

But it does, at least for some understandings of "declaration".  And it
could get even more.  A declaration is of course a reference to a thing
that characterizes a part, but not the main part or totality of the thing.
Besides the other two I mentioned already, any source code locus
setting a property on a symbol could be viewed as that.  Are we prepared
to handle this?  We could be.  If we are, do we really want to put this all
in a one-size-fits-all "declarations" bag?

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 12/11/2023 20:40, João Távora wrote:
>>> Anyway, will try to do that coming weeks, do ping me if I forget,
>>> we can even pair-program via tmate or something.
>> Sounds interesting. My Emacs doesn't really function in the terminal,
>> but we could try.
> Emacs -Q would function, right? 😄

Hard to say!

Anyway, it's pretty difficult to edit Lisp without Paredit.

> Anyway mine does, fairly reliably and I
> have very few personal keybindings.

Cool.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 12/11/2023 04:25, João Távora wrote:
> On Sun, Nov 12, 2023 at 2:10 AM Dmitry Gutov <dmitry@gutov.dev> wrote:
>>
>> On 12/11/2023 04:08, João Távora wrote:
>>> Noooo.  Eglot only does LSP things!
>>
>> Server-specific extensions are LSP things, in my book. Okay, it might
>> not be in eglot.el itself, but then someone will create
>> eglot-superduperclang.el to support those extensions.
> 
> Could be, but why can't this code live in c++-ts-mode?

What about c++-mode users? Or the devoted sm-c-mode aficionados?

And it would be odd if c++-ts-mode would only function as expected 
together with Eglot -- at the very least, it will be a challenge to 
explain this to the users.

>>> The major mode may do that if
>>> it has another alternative backend, or if it knows it is using LSP
>>> with a specific language server, like 'superduperclangdfork'.
>>
>> I'm pretty sure major modes that are specific to the language server in
>> use are a bad idea.
> 
> You may have misunderstood my suggestion.
> 
> I'm saying code specific to certain languages, LSP-based on not,
> should always live in major mode files and have major-mode prefixes.

We're not just talking about features specific to a language, but 
features specific to a certain potential future version of a language 
server, which might or might not be available through a certain LSP 
client/Xref backend, someday.

Otherwise we could just add those commands now, right?

Aside from the issue of multiple major modes existing for a number of 
important languages.

> It's always been like this.  The LSP server program doesn't have
> to be available for themajor mode to work, but it works better
> with it.

And the "...but it works better with it" has often been expressed in 
terms of minor modes. E.g. SLIME/Sly/CIDER/lots of others.

Etags might be put into that category too (you have to build tags first 
to use it) -- and all of its language-specific concerns (how to build 
tags, for example? which flags are preferred) never found its way its 
major modes.

> Major modes already do this.  For example, you don't _have_ to
> have a Python interpreter program to use python-mode.el to edit
> Python code,  but having one enables M-x run-python of course.
> And run-python uses comint.el which is a library that python.el
> relies on.

The Python interpreter always came pre-installed with Python.

Anyway... this was a digression that's probably not important for now. 
Whether the new advanced commands are defined in Eglot or in major 
modes, should have no bearing on what we do now, or whether we add a new 
set of "common" commands to xref.el.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Mon, Nov 13, 2023 at 12:27 AM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 12/11/2023 20:40, João Távora wrote:
> >>> Anyway, will try to do that coming weeks, do ping me if I forget,
> >>> we can even pair-program via tmate or something.
> >> Sounds interesting. My Emacs doesn't really function in the terminal,
> >> but we could try.
> > Emacs -Q would function, right? 😄
>
> Hard to say!
>
> Anyway, it's pretty difficult to edit Lisp without Paredit.

Yes, but electric-pair-mode right? ;-) Though I have Paredit lying
around here somewhere.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 13/11/2023 03:03, João Távora wrote:
> On Mon, Nov 13, 2023 at 12:27 AM Dmitry Gutov<dmitry@gutov.dev>  wrote:
>> On 12/11/2023 20:40, João Távora wrote:
>>>>> Anyway, will try to do that coming weeks, do ping me if I forget,
>>>>> we can even pair-program via tmate or something.
>>>> Sounds interesting. My Emacs doesn't really function in the terminal,
>>>> but we could try.
>>> Emacs -Q would function, right? 😄
>> Hard to say!
>>
>> Anyway, it's pretty difficult to edit Lisp without Paredit.
> Yes, but electric-pair-mode right? 😉 Though I have Paredit lying
> around here somewhere.

electric-pair-mode is necessary but not sufficient.

I'm the structured editing kind of guy. With all the accompanying 
slurping and barfing.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Mon, Nov 13, 2023 at 1:05 AM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 13/11/2023 03:03, João Távora wrote:

> electric-pair-mode is necessary but not sufficient.
>
> I'm the structured editing kind of guy. With all the accompanying
> slurping and barfing.

I slurp.  I barf.  C-M-k,f,b,u mostly works.  I have an extract-list
function too.

But I would love that paredit.el would just supply the slurp/barf
commands separate from the broken (IMHO) auto-pairing logic that I'm
not interested in.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Mon, Nov 13, 2023 at 1:03 AM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 12/11/2023 04:25, João Távora wrote:
> > On Sun, Nov 12, 2023 at 2:10 AM Dmitry Gutov <dmitry@gutov.dev> wrote:
> >>
> >> On 12/11/2023 04:08, João Távora wrote:
> >>> Noooo.  Eglot only does LSP things!
> >>
> >> Server-specific extensions are LSP things, in my book. Okay, it might
> >> not be in eglot.el itself, but then someone will create
> >> eglot-superduperclang.el to support those extensions.
> >
> > Could be, but why can't this code live in c++-ts-mode?
>
> What about c++-mode users? Or the devoted sm-c-mode aficionados?

Happy to put it an a superduperclang.el file required by both of
them.  I have my doubts Alan would like that, it took too much
discussion just to convince him to set a flymake variable some years
ago.

> And it would be odd if c++-ts-mode would only function as expected
> together with Eglot -- at the very least, it will be a challenge to
> explain this to the users.

It would function OK without it.  Just as it does now.  Just no
cross-referencing support.  Much like python.el works without a
python interpreter, but better with it.

> >>> The major mode may do that if
> >>> it has another alternative backend, or if it knows it is using LSP
> >>> with a specific language server, like 'superduperclangdfork'.
> >>
> >> I'm pretty sure major modes that are specific to the language server in
> >> use are a bad idea.
> >
> > You may have misunderstood my suggestion.
> >
> > I'm saying code specific to certain languages, LSP-based on not,
> > should always live in major mode files and have major-mode prefixes.
>
> We're not just talking about features specific to a language, but
> features specific to a certain potential future version of a language
> server, which might or might not be available through a certain LSP
> client/Xref backend, someday.
>
> Otherwise we could just add those commands now, right?
>
> Aside from the issue of multiple major modes existing for a number of
> important languages.
>
> > It's always been like this.  The LSP server program doesn't have
> > to be available for themajor mode to work, but it works better
> > with it.
>
> And the "...but it works better with it" has often been expressed in
> terms of minor modes. E.g. SLIME/Sly/CIDER/lots of others.

Yes, you can use an extra minor mode if you want or not.  But
Eglot by itself is not that minor mode.

> Etags might be put into that category too (you have to build tags first
> to use it) -- and all of its language-specific concerns (how to build
> tags, for example? which flags are preferred) never found its way its
> major modes.
>
> > Major modes already do this.  For example, you don't _have_ to
> > have a Python interpreter program to use python-mode.el to edit
> > Python code,  but having one enables M-x run-python of course.
> > And run-python uses comint.el which is a library that python.el
> > relies on.
>
> The Python interpreter always came pre-installed with Python.

Yes, but you don't have to install Python to edit python code in
Emacs, do you?

> Anyway... this was a digression that's probably not important for now.
> Whether the new advanced commands are defined in Eglot or in major
> modes, should have no bearing on what we do now, or whether we add a new
> set of "common" commands to xref.el.

I think it does, because the new advanced commands may become just as
popular or more as what you say as "common" commands.  This is the
"slippery slope" part of my argument.  The other part of my argument
is that the "common" commands you want to import from LSP isn't really
that great and that you don't _need_ to, because major modes.

Also I think that this particular decision doesn't need to be taken
now.  Whichever way we go, it won't invalidate what's in the Git
branch right now.

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 12/11/2023 20:59, João Távora wrote:

>> Dropping an implementation into master is one way to test out a
>> hypothesis, but it has inherent problems: the feedback comes late
>> anyway, and quite often we'd end up having to maintain half-bakes
>> features or unfortunate capabilities this way simply because they had
>> been added and time had passed.
> 
> Maybe.  Well Eglot has a fair enough audience, though not sure for
> this specific feature.  And I don't see any better alternatives if
> collecting feedback is what we want.  IOW, we shouldn't fall prey
> to "analysis-paralysis".

Hopefully not analysis paralysis. What we can do first, though, is 
iterate through a bunch of prototypes.

I was hoping to reach some sort of consensus before pushing the next 
one, but it's not really necessary for experimenting. So I'll just push 
the one I've been thinking of lately and see what everyone thinks.

>> Why contort? From where I'm standing,
>> declarations/implementations/typeDefinitions apply to almost any
>> object-oriented language (C++/Java/Ruby/Python/JS), and even some
>> functional ones.
> 
> What are "declarations" in JS, Java or Ruby and Python?

It's a pretty common term.

In Java, I'd be surprised if some interpreted it as something other than 
method definitions inside of an interface. And indeed: 
https://github.com/eclipse-jdtls/eclipse.jdt.ls/pull/2702

In JS, Mozilla's codebase actually had interface definitions, so a 
specialized backend could point to those. Some frontend frameworks had a 
thing similar to that.

No idea about Python's practices, but for Ruby there also are classes 
and mixins that have "abstract" methods. There are no syntactic 
annotations for that in the base language (you just throw a certain 
exception in the implementation), but the type-definition projects like 
Sorbet will likely change that.

The other two (implementations and typeDefinitions) make sense for the 
all of the above languages. Although "implementations" are of course 
only meningful to have when "declarations" exist.

> Perhaps
> nothing perhaps entirely different things. Also, there  are a lot of
> sub-kinds of declaration in C++.  Is a forward declaration of a
> type in C++ also a "declaration" here?

If the type is the entered identifier, why not?

I would first ask, are those definitions? Are they in the list returned 
by xref-find-definitions for the same input? If not, they might be 
useful to have in xref-find-declarations.

> What about a C++20 concept,
> is it a type definition?  No right?

I have no idea, but if it maps to the concept, and the users will find 
the binding in the common map using "M-' C-h", why not?

The only big downside is when xref-find-type-definitions already returns 
some different things, conceptually incompatible with the potential 
addition.

Or to look at it from another side, if xref-find-type-definitions is on 
"M-' t" in java-mode, would c++-mode prefer to have some other command 
on that bindings? If yes, would we want it to be arbitrarily incompatible?

> Or maybe, depends on who you ask
> some say it's a type for types.  Not to mention I think in Elisp we
> have at least 3 completely different kinds of "declaration": forward
> declarations, (declare ...) forms and defgeneric.  It's odd to mix
> them.

They can be returned together in xref-find-declarations (I would 
actually use that), and Elisp could have separate commands for more 
fine-grained navigation. Though the latter doesn't seem too much in 
demand, given they haven't materialized thus far.

> Anyway, this is LSP's taxonomy so it's LSP's problem (or rather,
> the problem of the poor servers that have to invent ways to shove
> things in these categories.  But not our problem by and large if we
> don't let  it spread any further than Eglot.  IMO Eglot is where
> LSP ends and Emacs starts.

The practice of shoving things into categories didn't appear from 
nowhere. Having common bindings and menu entries and easy transition 
between language modes is nice.

>> What's the alternative? To have many similar commands
>>
>>     ruby-find-implementation
>>     c++-find-implementations
>>     python-find-implementations
>>     js-find-implementations
> 
> Depends.  If you are using Eglot/LSP as a backend, you don't need to.
> Eglot already gives you the LSP trio.

ruby-mode doesn't choose an Xref backend. It just uses the configured 
one. Same for other modes in the above list.

> If you are using a non Eglot backend, then yes.  If you are using
> specific backend based on Eglot but with more servers, then yes too,
> but only for the extra types you support.

And for the rest? Unoccupied bindings? "undefined is not a function"? 
There be dragons?

Suppose a language doesn't support "declarations" in particular? Will it 
find a much better use for the corresponding binding in the prefix map?

>> that all do the same thing inside: call the Xref backend's "find
>> implementations" mechanism? It will be especially great to use if some
>> major mode authors bind that command to one key, and others to another.
> 
> I don't think it'll happen that often, and xref can define a good prefix
> map to use.  It's unlikely we or major modes would find keybinding space
> outside such a map anyway.

I'm saying it would be bad if java-mode binds 
java-find-interface-declarations to "M-' i" and c++-mode binds 
c++-find-forward-declarations to "M-' f", while they are more similar 
than different and could be both put on "M-' d".

>>> And Spencer even wanted to put cl-defgeneric in that bag, which I
>>> can fully understand, as cl-defgeneric can also be viewed as a
>>> declaration.  But its a completely different concept from those
>>> two.
>>
>> If Elisp doesn't have any other kind of declarations, what is the worry?
>> Let's put the ones it has.
> 
> But it does, at least for some understandings of "declaration".  And it
> could get even more.  A declaration is of course a reference to a thing
> that characterizes a part, but not the main part or totality of the thing.
> Besides the other two I mentioned already, any source code locus
> setting a property on a symbol could be viewed as that.  Are we prepared
> to handle this?  We could be.  If we are, do we really want to put this all
> in a one-size-fits-all "declarations" bag?

We are able to offer both the common "declarations" bag and separate 
lookup commands for finer searches. I would likely use the former.

electric-pair-mode vs paredit, was: Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 13/11/2023 03:16, João Távora wrote:
> On Mon, Nov 13, 2023 at 1:05 AM Dmitry Gutov <dmitry@gutov.dev> wrote:
>>
>> On 13/11/2023 03:03, João Távora wrote:
> 
>> electric-pair-mode is necessary but not sufficient.
>>
>> I'm the structured editing kind of guy. With all the accompanying
>> slurping and barfing.
> 
> I slurp.  I barf.  C-M-k,f,b,u mostly works.  I have an extract-list
> function too.

Perhaps I missed the addition of the slurping and barfing commands to 
elec-pair? Sexp navigation works indeed.

> But I would love that paredit.el would just supply the slurp/barf
> commands separate from the broken (IMHO) auto-pairing logic that I'm
> not interested in.

But I like this one which keeps the code paired and makes sure C-k 
doesn't kill too much.

Re: electric-pair-mode vs paredit, was: Re: Adding support for xref jumping to headers/interfaces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 2437 bytes --]

On Mon, Nov 13, 2023 at 1:41 AM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 13/11/2023 03:16, João Távora wrote:
> > On Mon, Nov 13, 2023 at 1:05 AM Dmitry Gutov <dmitry@gutov.dev> wrote:
> >>
> >> On 13/11/2023 03:03, João Távora wrote:
> >
> >> electric-pair-mode is necessary but not sufficient.
> >>
> >> I'm the structured editing kind of guy. With all the accompanying
> >> slurping and barfing.
> >
> > I slurp.  I barf.  C-M-k,f,b,u mostly works.  I have an extract-list
> > function too.
>
> Perhaps I missed the addition of the slurping and barfing commands to
> elec-pair? Sexp navigation works indeed.

At least i didn't add anything.  I just said that I use sexp stuff
for doing those structural operations.  Along with the command
after my sig, which I've had for almost 20 years.

> > But I would love that paredit.el would just supply the slurp/barf
> > commands separate from the broken (IMHO) auto-pairing logic that I'm
> > not interested in.
>
> But I like this one which keeps the code paired and makes sure C-k
> doesn't kill too much.

electric-pair-mode keeps the code paired.  I like C-k killing
whole lines, even in lisp.  I hate those harnesses, I just put
add a thumb on the alt key when I need to work with sexps.

Of course to each his own.  Doesn't seem you need e-p-m at all.

João

(defun joaot/extract-list-or-region (arg)
  "Take the list after the point and remove the surrounding list.  With
argument ARG do it that many times."
  (interactive "p")
  (let* ((start (if mark-active
                    (region-beginning)
                  (point)))
         (end (if mark-active
                  (region-end)
                (save-excursion
                  (forward-sexp)
                  (point))))
         (extracted (buffer-substring start end))
         (stop nil))
    (backward-up-list (or arg 1))
    (progn
      (delete-region start end)
      (condition-case err
          (kill-region (point)
                       (save-excursion
                         (forward-sexp)
                         (point)))
        (error (goto-char start)
               (insert extracted)
               (message "ooops....")
               (setq stop t)))
      (unless stop
        (indent-region
         (point)
         (progn
           (save-excursion
             (insert extracted)
             (point))))))))

[-- Attachment #2: Type: text/html, Size: 3287 bytes --]

Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh]

Dmitry Gutov <dmitry@gutov.dev> writes:
> On 11/11/2023 14:38, Spencer Baugh wrote:
>> So e.g. c-ts-mode could define
>> c-find-signature, which has a clear meaning for C.
>
> And c-mode will not.

Some major modes will support things which other major modes do not
support.  Users will use the best one, whatever it is, and we'll change
the default to whatever it is, eventually.  I don't see an issue.

>> And likewise for
>> refactorings which only make sense in a given language.
>
> And then it will fail because the user didn't enable a minor mode
> which would give access to said functionality.

That doesn't seem like much of a problem.

Alternatively, maybe such commands can autostart Eglot or whatever other
backend it needs?

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 15/11/2023 23:32, Spencer Baugh wrote:
> Dmitry Gutov <dmitry@gutov.dev> writes:
>> On 11/11/2023 14:38, Spencer Baugh wrote:
>>> So e.g. c-ts-mode could define
>>> c-find-signature, which has a clear meaning for C.
>>
>> And c-mode will not.
> 
> Some major modes will support things which other major modes do not
> support.  Users will use the best one, whatever it is, and we'll change
> the default to whatever it is, eventually.  I don't see an issue.

I don't see why we have to limit the users to specific major mode, when 
one might have functionality others don't have, and vice versa.

Like now, for example, many will likely stay with CC Mode for a while 
because of it more lax behavior in macro-heavy codebases, where 
c-ts-mode chokes and shows syntax errors.

While others will hopefully like c/c++-ts-mode with its more accurate 
and faster parsing and highlighting.

Anyway, I've pushed an update to the same branch 
(feature/xref-find-extra). Again, it's something to try out, not a done 
deal:

* The command is called xref-find-all-definitions, with appropriate 
behavior when invoked without prefix argument (appends the results from 
all kinds and shows them together, with duplicates removed).

* The Eglot implementation doesn't include 'references' in the list of 
kinds anymore, just because those are not definitions.

* New commands for the 3 popular kinds, bound to "M-' e", "M-' i" and 
"M-'t". The command which shows all moved to "M-' M-'".

What does everyone think?

IMO the "all kinds together" view is interesting, but it all depends on 
what people actually find useful.

Re: Adding support for xref jumping to headers/interfaces

[Felician Nemeth]

Dmitry Gutov <dmitry@gutov.dev> writes:

> Anyway, I've pushed an update to the same branch
> (feature/xref-find-extra).
>
> What does everyone think?

I haven't followed the discussion, but the extensibility of it is just
great.

I think it is a minor inconvenience that it is not possible to use
directly the symbol-at-point as identifier while asking the user for the
kind.  That is a prefix argument causes xref-find-all-definitions to ask
the user for both.

eglot-find-declaration, eglot-find-implementation,
eglot-find-typeDefinition still use the now non-existent
xref-find-extra.

Thank you.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 24/11/2023 23:43, Felician Nemeth wrote:
> Dmitry Gutov <dmitry@gutov.dev> writes:
> 
>> Anyway, I've pushed an update to the same branch
>> (feature/xref-find-extra).
>>
>> What does everyone think?
> 
> I haven't followed the discussion, but the extensibility of it is just
> great.

Here's a refresher on the points of contention, in case you'll have 
something to add:

1. Whether Xref should have separate commands for 
declarations/implementations/type-definitions. With default global 
bindings (probably -- if we manage to get agreement on taking M-', for 
example). Or e.g. have a registry for kinds and letters, for quick 
selection between kinds (alternative for completing-read). Or just use 
completing-read and leave the definitions of all new commands (including 
the counterparts to declarations/implementations/type-definitions, of 
which then there will be multiple copies of) to the Xref backends.
2. Whether Eglot should, in time, deprecate its corresponding three 
commands if Xref adds its own versions.
3. How the new command (currently named xref-find-all-definitions) 
should behave: should it always ask for the kind, or have a default one 
(determined how?), or fetch all kinds by default and show them together, 
like the latest version does.

> I think it is a minor inconvenience that it is not possible to use
> directly the symbol-at-point as identifier while asking the user for the
> kind.  That is a prefix argument causes xref-find-all-definitions to ask
> the user for both.

I'm open to suggestions. Using 'C-u C-u' for one of the choices? Adding 
a separate command with different behavior (and different binding)?

> eglot-find-declaration, eglot-find-implementation,
> eglot-find-typeDefinition still use the now non-existent
> xref-find-extra.

Now fixed, thanks (with a rebase).

Re: Adding support for xref jumping to headers/interfaces

[Felician Nemeth]

Dmitry Gutov <dmitry@gutov.dev> writes:

> Here's a refresher on the points of contention, in case you'll have
> something to add:
>
> 1. Whether Xref should have separate commands for
> declarations/implementations/type-definitions. With default global
> bindings (probably -- if we manage to get agreement on taking M-', for
> example). Or e.g. have a registry for kinds and letters, for quick
> selection between kinds (alternative for completing-read). Or just use
> completing-read and leave the definitions of all new commands
> (including the counterparts to
> declarations/implementations/type-definitions, of which then there
> will be multiple copies of) to the Xref backends.

I have a completing-read-function that creates a single keystroke
selection, so the two alternatives are almost the same to me.

However, in the past it made sense to find the first xref backend that
knew the definition of an identifier.  Now let's assume backend A
provides definitions with letter "d" and backend B provides
documentation locations also with letter "d".  I think it useful to
allow the user to select between the two when xref-find-all-definitions
is invoked.  And it is easier to create a flat list for completing-read
than for read-multiple-choice, for example: ("definition/A"
"documentation/B").

> 2. Whether Eglot should, in time, deprecate its corresponding three
> commands if Xref adds its own versions.

I think one of João's design goals of Eglot is to rely on existing Emacs
features as much as possible.

> 3. How the new command (currently named xref-find-all-definitions)
> should behave: should it always ask for the kind, or have a default
> one (determined how?), or fetch all kinds by default and show them
> together, like the latest version does.

I don't have a strong opinion about this.  In elisp-mode fetching all
kinds by default seems to be the most useful.  Usually there are no more
than a handful of items and it is easy to see their kind.

>> I think it is a minor inconvenience that it is not possible to use
>> directly the symbol-at-point as identifier while asking the user for the
>> kind.  That is a prefix argument causes xref-find-all-definitions to ask
>> the user for both.
>
> I'm open to suggestions. Using 'C-u C-u' for one of the choices?
> Adding a separate command with different behavior (and different
> binding)?

If I'm the only one who'd be happy to have this, then a separate command
without a keybinding would be fine.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 26/11/2023 18:08, Felician Nemeth wrote:
> Dmitry Gutov <dmitry@gutov.dev> writes:
> 
>> Here's a refresher on the points of contention, in case you'll have
>> something to add:
>>
>> 1. Whether Xref should have separate commands for
>> declarations/implementations/type-definitions. With default global
>> bindings (probably -- if we manage to get agreement on taking M-', for
>> example). Or e.g. have a registry for kinds and letters, for quick
>> selection between kinds (alternative for completing-read). Or just use
>> completing-read and leave the definitions of all new commands
>> (including the counterparts to
>> declarations/implementations/type-definitions, of which then there
>> will be multiple copies of) to the Xref backends.
> 
> I have a completing-read-function that creates a single keystroke
> selection, so the two alternatives are almost the same to me.

I think the first question is what will be best for the default UI. And 
the follow-up will be, if needed, is whether anything else is needed to 
make it more easily customizable.

A completing-read-function, or simply a UI which does what you describe, 
is not out of the question for this built-in command too. But if we 
tried to add it, then it's a question for how to choose the letters, and 
whether we expect them to be more-or-less stable over different 
languages and backends. If no, that's a bit of a bummer, given that one 
would often want to input their choice quickly, even without reading. If 
yes, then I suppose that means that the global set of "kinds" is 
actually stable, and it's either possible to create a mapping registry 
(characters to kinds), or choose a very stable dynamic algorithm (given 
that we know most of the inputs).

> However, in the past it made sense to find the first xref backend that
> knew the definition of an identifier.  Now let's assume backend A
> provides definitions with letter "d" and backend B provides
> documentation locations also with letter "d".  I think it useful to
> allow the user to select between the two when xref-find-all-definitions
> is invoked.  And it is easier to create a flat list for completing-read
> than for read-multiple-choice, for example: ("definition/A"
> "documentation/B").

Eglot's integration shows that we don't in advance know whether a 
backend actually supports a specific kind. At least, we cannot know 
whether that kind is supported for a specific symbol at a given position 
(my tests with Eglot in the Emacs repo and clangd more than often ended 
up in "nothing found"). So to provide a list like ("definition/A"
"documentation/B" ...) we'd have to first query all available backends 
about all kinds for the symbol at point. I think somebody said that this 
can be slow at least for some kind/backend/ls combinations.

>> 2. Whether Eglot should, in time, deprecate its corresponding three
>> commands if Xref adds its own versions.
> 
> I think one of João's design goals of Eglot is to rely on existing Emacs
> features as much as possible.

That was my expectation indeed, but lately he said that since the 
commands are already in Eglot, they will remain there. Among other 
things, it will likely mean that they won't be getting key bindings 
(aside from those assigned by the users manually).

>> 3. How the new command (currently named xref-find-all-definitions)
>> should behave: should it always ask for the kind, or have a default
>> one (determined how?), or fetch all kinds by default and show them
>> together, like the latest version does.
> 
> I don't have a strong opinion about this.  In elisp-mode fetching all
> kinds by default seems to be the most useful.  Usually there are no more
> than a handful of items and it is easy to see their kind.

In Elisp, though, the whole addition doesn't make much sense. At least I 
don't see the benefit: xref-find-definition in Elisp already shows all 
kinds together, and it's rare than one needs to choose between them. And 
together with Mattias E.'s work for detecting the symbol-at-point's 
context ('elisp--xref-infer-namespace', etc) this works faster 
automatically 99% of the time.

My testing with the Elisp backend is what prompted me to try to 
re-evaluate how this addition should work, anyway.

>>> I think it is a minor inconvenience that it is not possible to use
>>> directly the symbol-at-point as identifier while asking the user for the
>>> kind.  That is a prefix argument causes xref-find-all-definitions to ask
>>> the user for both.
>>
>> I'm open to suggestions. Using 'C-u C-u' for one of the choices?
>> Adding a separate command with different behavior (and different
>> binding)?
> 
> If I'm the only one who'd be happy to have this, then a separate command
> without a keybinding would be fine.

It could also be an automatically generated kind called "all" in the 
list to choose from.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Fri, Nov 24, 2023 at 1:37 AM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 15/11/2023 23:32, Spencer Baugh wrote:
> > Dmitry Gutov <dmitry@gutov.dev> writes:
> >> On 11/11/2023 14:38, Spencer Baugh wrote:
> >>> So e.g. c-ts-mode could define
> >>> c-find-signature, which has a clear meaning for C.
> >>
> >> And c-mode will not.
> >
> > Some major modes will support things which other major modes do not
> > support.  Users will use the best one, whatever it is, and we'll change
> > the default to whatever it is, eventually.  I don't see an issue.
>
> I don't see why we have to limit the users to specific major mode, when
> one might have functionality others don't have, and vice versa.

It's unclear what you are understanding of Spencer's suggestion, which
is also mine.

I hope it's clear that in mine and Spencer's view if there is
support for a given finding a given thing of kind foo for a given
language, then there is an easy command for doing so.  And if there
isn't that foo, there isn't a command for doing so.  This is the way
it should be.

In my view, again, we should not limit ourselves to a triad
of kinds clumsily inherited from LSP.  We should limit that LSP
concept to Eglot.  It is not flexible enough, because it can't
be.  But Xref is for much more than LSP.

> Like now, for example, many will likely stay with CC Mode for a while
> because of it more lax behavior in macro-heavy codebases, where
> c-ts-mode chokes and shows syntax errors.

Whatever you are suggesting, I don't understand.  Whatever xref backend
is written for c++-mode with its special-purpose commands for finding
C++ things can be used for both c++-mode and c++-ts-mode.  Just put that
code in a special file.

> Anyway, I've pushed an update to the same branch
> (feature/xref-find-extra). Again, it's something to try out, not a done
> deal:
>
> * The command is called xref-find-all-definitions, with appropriate
> behavior when invoked without prefix argument (appends the results from
> all kinds and shows them together, with duplicates removed).

I think this is a mistake and a regression from xref-find-extra.
The command should be called xref-find-all, xref-find, or simply
xref, since definition is a category that doesn't span a lot of
useful cross-referenceable constructs in many languages. In C++,
for one, a declaration is absolutely _not_ a definition.

> * The Eglot implementation doesn't include 'references' in the list of
> kinds anymore, just because those are not definitions.

A mistake, but at least I can roll it back easily.
I would like the new interactive "find all" command (whatever you decide)
to show me "references" as one of the kinds of thing to search for.

Why are you rolling back this useful stuff? Are you even testing?  Because
the current branch doesn't even work in Eglot.

> * New commands for the 3 popular kinds, bound to "M-' e", "M-' i" and
> "M-'t". The command which shows all moved to "M-' M-'".

A mistake, again.  I wonder why you are doing these opinionated
changes to the branch you yourself proposed and which I put work
into.

If you are proposing this is brought to master for experimentation,
and if you are wary of this step, then may I suggest we push the older
more conservative version, perhaps with some naming changes?

This new version isn't something we can roll back easily, while the
more conservative approach could eventually be enhanced with the three
"popular kinds" later on.

> What does everyone think?

> xref-find-all-definitions is an interactive native-compiled Lisp
> function in `xref.el'.

> at point, prompt for the identifier.  Interactively, show matches
> for all supported kinds.  When invoked with prefix argument,
> prompt for KIND.

I can't get this to work btw.   If I add a prefx argument, it
prompts me for the identifier, which I don't want.

So will you please roll back some of your changes so we can
at least get something working again?

João

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Sun, Nov 26, 2023 at 8:15 PM Dmitry Gutov <dmitry@gutov.dev> wrote:

> Eglot's integration shows that we don't in advance know whether a
> backend actually supports a specific kind. At least, we cannot know
> whether that kind is supported for a specific symbol at a given position
> (my tests with Eglot in the Emacs repo and clangd more than often ended
> up in "nothing found"). So to provide a list like ("definition/A"
> "documentation/B" ...) we'd have to first query all available backends
> about all kinds for the symbol at point. I think somebody said that this
> can be slow at least for some kind/backend/ls combinations.

Who?  Was it me?  If so (I don't think so), I take it back.  This is
absolutely fine.  Eglot takes this list, from the capabilities reported
at the start of the LSP session and is very fast.

> >> 2. Whether Eglot should, in time, deprecate its corresponding three
> >> commands if Xref adds its own versions.
> >
> > I think one of João's design goals of Eglot is to rely on existing Emacs
> > features as much as possible.
>
> That was my expectation indeed, but lately he said that since the
> commands are already in Eglot, they will remain there.

This is not my argument, as you well know.

João

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Sun, Nov 26, 2023 at 4:08 PM Felician Nemeth
<felician.nemeth@gmail.com> wrote:
>
> Dmitry Gutov <dmitry@gutov.dev> writes:

> However, in the past it made sense to find the first xref backend that
> knew the definition of an identifier.  Now let's assume backend A
> provides definitions with letter "d" and backend B provides
> documentation locations also with letter "d".

I think according to Dmitry these documentation locations
should be crammed somewhere in either "definition", "declaration" or
"typeDefinition" or "implementation".  Go figure.

> > 2. Whether Eglot should, in time, deprecate its corresponding three
> > commands if Xref adds its own versions.
>
> I think one of João's design goals of Eglot is to rely on existing Emacs
> features as much as possible.

I don't think these commands should exist in Xref though.  They
directly reflect a limited LSP-specific categorization of
cross-referenceable things that should be confined to Emacs's
LSP interface, which is Eglot.  So I will not be deprecating these
commands, unless LSP deprecates them.

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 26/11/2023 22:37, João Távora wrote:
> On Sun, Nov 26, 2023 at 8:15 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
> 
>> Eglot's integration shows that we don't in advance know whether a
>> backend actually supports a specific kind. At least, we cannot know
>> whether that kind is supported for a specific symbol at a given position
>> (my tests with Eglot in the Emacs repo and clangd more than often ended
>> up in "nothing found"). So to provide a list like ("definition/A"
>> "documentation/B" ...) we'd have to first query all available backends
>> about all kinds for the symbol at point. I think somebody said that this
>> can be slow at least for some kind/backend/ls combinations.
> 
> Who?  Was it me?  If so (I don't think so), I take it back.  This is
> absolutely fine.  Eglot takes this list, from the capabilities reported
> at the start of the LSP session and is very fast.

What I'm saying is that *actually fetching* those might be slow 
(allegedly) for some combinations of kind/backend/ls.

And to implement the kind of fallback Felician described, it seems we 
would first have to know what actual kinds of definitions are available 
for a given reference before providing the aforementioned. For optimal 
user experience, at least.

The Elisp backend actually does that (only show the applicable kinds), 
but Eglot returns a fixed list for all symbols.

>>>> 2. Whether Eglot should, in time, deprecate its corresponding three
>>>> commands if Xref adds its own versions.
>>>
>>> I think one of João's design goals of Eglot is to rely on existing Emacs
>>> features as much as possible.
>>
>> That was my expectation indeed, but lately he said that since the
>> commands are already in Eglot, they will remain there.
> 
> This is not my argument, as you well know.

It was one of your explanations, among others, and it was one that I 
understood best.

But one of the latest emails mentioned a separate Xref backend for c/c++ 
-- should I assume that every such backend would define its own set of 
*-find-* commands suitable for its languages?

Would such backends each come with an automatically-enabled keymap? Or 
would the user have to set up key bindings for every such command, for 
every special backend like that?

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 26/11/2023 22:40, João Távora wrote:
> On Sun, Nov 26, 2023 at 4:08 PM Felician Nemeth
> <felician.nemeth@gmail.com> wrote:
>>
>> Dmitry Gutov <dmitry@gutov.dev> writes:
> 
>> However, in the past it made sense to find the first xref backend that
>> knew the definition of an identifier.  Now let's assume backend A
>> provides definitions with letter "d" and backend B provides
>> documentation locations also with letter "d".
> 
> I think according to Dmitry these documentation locations
> should be crammed somewhere in either "definition", "declaration" or
> "typeDefinition" or "implementation".  Go figure.

Err, of course not. Every backend is free to return additional kinds 
(that's what the completing-read approach is for), and whenever we find 
out that particular kinds get supported by many backends (or by most of 
the available/popular ones), we could "pull them into the core", adding 
a global command with one binding which would work across languages.

That's one of the balances to pick: both choosing a set of commands to 
support across the backends, and to allow frictionless extensions for 
special capabilities.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Mon, Nov 27, 2023 at 2:43 PM Dmitry Gutov <dmitry@gutov.dev> wrote:

> (that's what the completing-read approach is for), and whenever we find
> out that particular kinds get supported by many backends (or by most of
> the available/popular ones), we could "pull them into the core", adding
> a global command with one binding which would work across languages.

Alright, so since this is contentious (who else but you is pushing
this idea?) what about we start with 0 in xref.el.  We can always
add to 0, no problem, but taking away  from some other number
isn't so easy.

>  and to allow frictionless extensions for special capabilities.

As to frictionless extension, the macro I proposed already
xref-define-finder seems the easiest way by far.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 27/11/2023 16:49, João Távora wrote:
> On Mon, Nov 27, 2023 at 2:43 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
> 
>> (that's what the completing-read approach is for), and whenever we find
>> out that particular kinds get supported by many backends (or by most of
>> the available/popular ones), we could "pull them into the core", adding
>> a global command with one binding which would work across languages.
> 
> Alright, so since this is contentious (who else but you is pushing
> this idea?)

Indeed, I wonder if nobody else is interested in having the additional 
commands have pre-defined bindings, or having the same bindings across 
languages.

> what about we start with 0 in xref.el.  We can always
> add to 0, no problem, but taking away  from some other number
> isn't so easy.

We could indeed start with 0, but then I already see the same set of 
extra commands supported in Eglot, Elisp, lsp-mode

>>   and to allow frictionless extensions for special capabilities.
> 
> As to frictionless extension, the macro I proposed already
> xref-define-finder seems the easiest way by far.

Sorry, I can't find it.

But the definition of xref-find-declarations takes about 3 lines. There 
is not much potential for making it even shorter.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Mon, Nov 27, 2023 at 2:35 PM Dmitry Gutov <dmitry@gutov.dev> wrote:

> But one of the latest emails mentioned a separate Xref backend for c/c++
> -- should I assume that every such backend would define its own set of
> *-find-* commands suitable for its languages?

Of course, a cross-referencing backend should know what kind of
cross-references it can find.  Eglot knows how to find 3 or 4 different
kinds, language server permitting of course.  Eglot will never know how
to find "documentation", and I believe the Elisp backend should
never purport to know how to find "declaration", since that's a very
fuzzy concept in Lisp with no accepted meaning.  As is
"typeDefinition", doesn't make much sense in dinamically typed
languages (but "declaration" is worse).

> Would such backends each come with an automatically-enabled keymap? Or
> would the user have to set up key bindings for every such command, for
> every special backend like that?

The major modes could do that, for example.  They could do it by
linking the backend-created "one key/command" keymap into a special
prefix created by xref.el, doing so by calling a load-time xref.el helper.
So maybe some backends would have d for  "documentation" and others would
have d for definitions, so what? Maybe that's precisely what a user working
on a documentation heavy system expects "d" to do. And if these two backends
happen to be enabled simultaneously,  even this conflict could be
resolved, by adjusting the keymap during the linking process.

But however this works, I think the bindings issue is completely
secondary.  Secondary not in the sense that it's less important,
but in the sense that we don't have to commit to something right
now, so I don't understand the tortuous route this feature is taking into
what-if territory.

So whatever solution we find to the bindings issue (if it is indeed
an issue)

It's much more pressing to decide if/why you want a command named
xref-find-all-definitions to return things that aren't "definitions"
by any measure or what is pressuring us to enshrine
"declaration/implementation/typeDefinition" in one of our
most generic libraries.

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 26/11/2023 22:30, João Távora wrote:
>>> Some major modes will support things which other major modes do not
>>> support.  Users will use the best one, whatever it is, and we'll change
>>> the default to whatever it is, eventually.  I don't see an issue.
>>
>> I don't see why we have to limit the users to specific major mode, when
>> one might have functionality others don't have, and vice versa.
> 
> It's unclear what you are understanding of Spencer's suggestion, which
> is also mine.
> 
> I hope it's clear that in mine and Spencer's view if there is
> support for a given finding a given thing of kind foo for a given
> language, then there is an easy command for doing so.  And if there
> isn't that foo, there isn't a command for doing so.  This is the way
> it should be.
> 
> In my view, again, we should not limit ourselves to a triad
> of kinds clumsily inherited from LSP.  We should limit that LSP
> concept to Eglot.  It is not flexible enough, because it can't
> be.  But Xref is for much more than LSP.

What it seems you are saying is that we never should try to extend the 
set of definition-finding commands above what's already in xref.el.

Because it seems clear that we're unlikely to find a more 
widely-supported set of additional kinds that the aforementioned triad, 
in any medium-term future.

>> Like now, for example, many will likely stay with CC Mode for a while
>> because of it more lax behavior in macro-heavy codebases, where
>> c-ts-mode chokes and shows syntax errors.
> 
> Whatever you are suggesting, I don't understand.  Whatever xref backend
> is written for c++-mode with its special-purpose commands for finding
> C++ things can be used for both c++-mode and c++-ts-mode.  Just put that
> code in a special file.

What about default key bindings?

>> Anyway, I've pushed an update to the same branch
>> (feature/xref-find-extra). Again, it's something to try out, not a done
>> deal:
>>
>> * The command is called xref-find-all-definitions, with appropriate
>> behavior when invoked without prefix argument (appends the results from
>> all kinds and shows them together, with duplicates removed).
> 
> I think this is a mistake and a regression from xref-find-extra.
> The command should be called xref-find-all, xref-find, or simply
> xref, since definition is a category that doesn't span a lot of
> useful cross-referenceable constructs in many languages. In C++,
> for one, a declaration is absolutely _not_ a definition.

It's a "definition" in at least one Xref-related sense: it should be 
dispatched through xref-show-definitions-function.

Further, I'm not sure that when a user looks up all definition-related 
things for a symbol, they wouldn't want to see the "declaration". In 
fact, if we classify 'defgeneric' as declarations and 'defmethod' as 
implementations, I'm pretty sure I would want to see the former in the list.

And you yourself mentioned that "type definitions" might be suitable for 
that list (which I found surprising at first). So it seems clear that 
there is no single red line.

>> * The Eglot implementation doesn't include 'references' in the list of
>> kinds anymore, just because those are not definitions.
> 
> A mistake, but at least I can roll it back easily.
> I would like the new interactive "find all" command (whatever you decide)
> to show me "references" as one of the kinds of thing to search for.

Then the list of results would drown in "references", wouldn't it?

And its output would (or should) be the same as 'xref-find-references'. 
That would make it fairly pointless, I believe.

> Why are you rolling back this useful stuff? Are you even testing?  Because
> the current branch doesn't even work in Eglot.

Sorry, I didn't test the Eglot-specific commands. But their definitions 
should be 100% the same as the new Xref ones.

>> * New commands for the 3 popular kinds, bound to "M-' e", "M-' i" and
>> "M-'t". The command which shows all moved to "M-' M-'".
> 
> A mistake, again.  I wonder why you are doing these opinionated
> changes to the branch you yourself proposed and which I put work
> into.

For experimentation.

> If you are proposing this is brought to master for experimentation,
> and if you are wary of this step, then may I suggest we push the older
> more conservative version, perhaps with some naming changes?

I'm not yet seeing a common basic version for which there would be 
agreement. You just called a renaming a mistake and disagreed with the 
principle of what "definition kinds" should be.

> This new version isn't something we can roll back easily, while the
> more conservative approach could eventually be enhanced with the three
> "popular kinds" later on.

I'm not proposing any merge to master yet.

>> What does everyone think?
> 
>> xref-find-all-definitions is an interactive native-compiled Lisp
>> function in `xref.el'.
> 
>> at point, prompt for the identifier.  Interactively, show matches
>> for all supported kinds.  When invoked with prefix argument,
>> prompt for KIND.
> 
> I can't get this to work btw.   If I add a prefx argument, it
> prompts me for the identifier, which I don't want.

What's what the docstring says, and it's what Felician brought up as a 
problem. Do you want two separate commands? And one of them would always 
prompt for the kind to use?

> So will you please roll back some of your changes so we can
> at least get something working again?

Everything is "working" there, except for Eglot-specific commands, which 
are trivial to fix. I just did that, you can test.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Mon, Nov 27, 2023 at 3:02 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 27/11/2023 16:49, João Távora wrote:
> > On Mon, Nov 27, 2023 at 2:43 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
> >
> >> (that's what the completing-read approach is for), and whenever we find
> >> out that particular kinds get supported by many backends (or by most of
> >> the available/popular ones), we could "pull them into the core", adding
> >> a global command with one binding which would work across languages.
> >
> > Alright, so since this is contentious (who else but you is pushing
> > this idea?)
>
> Indeed, I wonder if nobody else is interested in having the additional
> commands have pre-defined bindings, or having the same bindings across
> languages.

Of course if you put it like that then EVERYBODY is interested in that.
Everybody likes consistency, but this consistency you're proposing
is a mirage.  An naive Xref user would be very proud to use
"xref-find-declaration" in 3 different languages but then would
start to doubt xref that the things he and the backend author meant
by declaration are entirely different.

Again, leave this to major modes, which is what we always do (or to backend
which in 95% of cases would live in the major mode when they are
language-specific).

> > what about we start with 0 in xref.el.  We can always
> > add to 0, no problem, but taking away  from some other number
> > isn't so easy.
>
> We could indeed start with 0, but then I already see the same set of
> extra commands supported in Eglot, Elisp, lsp-mode

They only exist in Eglot and LSP-mode because these are LSP things,
it makes full sense they would exist in any Emacs/LSP interface.

Elisp?  What commands are you talking about, only if you force them.
I see no elisp-find-{declaration,type-definition,implementation}
or anything of the sort.

The 7 Elisp cross-reference kinds you defined in your original
patch are

  "defalias"
  "face"
  "function"
  "constructor"
  "generic"
  "variable"
  "feature'"

This is exactly what a language-specific kind list should look like by
the way.  These are the interesting things to cross-reference to in Elisp,
not "implementation", "declaration" and "typeDefinition".  Sure we
can squint very hard and try put those into those three drawers, but
it's such a meaningless exercise with no practical value.

> >>   and to allow frictionless extensions for special capabilities.
> >
> > As to frictionless extension, the macro I proposed already
> > xref-define-finder seems the easiest way by far.
>
> Sorry, I can't find it.
>
> But the definition of xref-find-declarations takes about 3 lines. There
> is not much potential for making it even shorter.

It's very similar and analogous to:

(defmacro eglot--code-action (name kind)
  "Define NAME to execute KIND code action."
  `(defun ,name (beg &optional end)
     ,(format "Execute `%s' code actions between BEG and END." kind)
     (interactive (eglot--code-action-bounds))
     (eglot-code-actions beg end ,kind t)))

And allows us to control exactly the 'interactive' spec of such
commands to give us consistency among them.

João

Re: Adding support for xref jumping to headers/interfaces

[Eli Zaretskii]

> Date: Mon, 27 Nov 2023 17:01:58 +0200
> Cc: Felician Nemeth <felician.nemeth@gmail.com>,
>  Spencer Baugh <sbaugh@janestreet.com>, emacs-devel@gnu.org,
>  Eshel Yaron <me@eshelyaron.com>, John Yates <john@yates-sheets.org>,
>  Ergus <spacibba@aol.com>, Filipp Gunbin <fgunbin@fastmail.fm>
> From: Dmitry Gutov <dmitry@gutov.dev>
> 
> On 27/11/2023 16:49, João Távora wrote:
> > On Mon, Nov 27, 2023 at 2:43 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
> > 
> >> (that's what the completing-read approach is for), and whenever we find
> >> out that particular kinds get supported by many backends (or by most of
> >> the available/popular ones), we could "pull them into the core", adding
> >> a global command with one binding which would work across languages.
> > 
> > Alright, so since this is contentious (who else but you is pushing
> > this idea?)
> 
> Indeed, I wonder if nobody else is interested in having the additional 
> commands have pre-defined bindings, or having the same bindings across 
> languages.

I might be interested, but you guys have a tendency to drown the
important issues in an ocean of second- and third-order stuff.  It is
very hard to follow this discussion, with its longish posts and a lot
of barely-related issues and arguments.

Maybe it's time to post some kind of summary?

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 27/11/2023 17:03, João Távora wrote:
> On Mon, Nov 27, 2023 at 2:35 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
> 
>> But one of the latest emails mentioned a separate Xref backend for c/c++
>> -- should I assume that every such backend would define its own set of
>> *-find-* commands suitable for its languages?
> 
> Of course, a cross-referencing backend should know what kind of
> cross-references it can find.  Eglot knows how to find 3 or 4 different
> kinds, language server permitting of course.

That didn't answer the above question. To signal "what kind of 
cross-references it can find", it would define 
xref-backend-definition-kinds with corresponding return value.

> Eglot will never know how
> to find "documentation", and I believe the Elisp backend should
> never purport to know how to find "declaration", since that's a very
> fuzzy concept in Lisp with no accepted meaning.  As is
> "typeDefinition", doesn't make much sense in dinamically typed
> languages (but "declaration" is worse).

Dynamic languages have type checkers and inferencers implemented all the 
time. E.g. There's nothing stopping someone from analyzing a stack of 
definitions in Elisp, seeing that a variable is either defined with a 
cl-struct's constructor, or accessed with a cl-struct's accessor, and 
assuming that struct's definition as its type definition. Same for EIEIO 
and its counterparts in Common Lisp. In Ruby and Python, we have 
optional type annotations as well, provided by libraries (Ruby) and the 
core language (Python).

>> Would such backends each come with an automatically-enabled keymap? Or
>> would the user have to set up key bindings for every such command, for
>> every special backend like that?
> 
> The major modes could do that, for example.  They could do it by
> linking the backend-created "one key/command" keymap into a special
> prefix created by xref.el, doing so by calling a load-time xref.el helper.

Okay, so we would create and occupy a new common prefix. But without any 
commands in it? How do you suggest we advocate for that around here?

> So maybe some backends would have d for  "documentation" and others would
> have d for definitions, so what? Maybe that's precisely what a user working
> on a documentation heavy system expects "d" to do. And if these two backends
> happen to be enabled simultaneously,  even this conflict could be
> resolved, by adjusting the keymap during the linking process.

That's easily solvable in other ways.

First of all, xref-find-definitions is bound already, and I'm sure 
text-related backends have its own "things" found by it, ones that you 
wouldn't readily call "definitions" (there was a bug report related to 
LaTeX not too long ago about a situation like that, IIRC).

But if you wanted to take "d" from "declarations" instead, then, with 
core Xref commands it's easy to override. Something like:

   (define-key well-learned-mode-map [remap xref-find-declarations] 
#'well-learned-find-special-documentation)

With the benefit that a known key binding is used.

> But however this works, I think the bindings issue is completely
> secondary.  Secondary not in the sense that it's less important,
> but in the sense that we don't have to commit to something right
> now, so I don't understand the tortuous route this feature is taking into
> what-if territory.

If we don't have built-in bindings (or a prefix map), we miss an 
opportunity to make life for Xref backends' implementors easier. But 
indeed, it could be deferred.

But another issue is that if we drop the commands, we lose the fastest 
way to access the definitions for each kind OOtB. Then we're back to 
discussing how the new command which allows accessing different kinds 
(the unlimited set) could be made faster/more predictable than just 
using completing-read. E.g. with a customizable registry of key->kind 
mappings.

And I didn't just invent the need for faster interaction by myself - 
just look at the first responses to the prototype. Okay, perhaps some 
power users in this thread don't mind adding personal tweaks to their 
config to speed things up. But that means the OOtB experience will be 
lacking.

> So whatever solution we find to the bindings issue (if it is indeed
> an issue)
> 
> It's much more pressing to decide if/why you want a command named
> xref-find-all-definitions to return things that aren't "definitions"
> by any measure or what is pressuring us to enshrine
> "declaration/implementation/typeDefinition" in one of our
> most generic libraries.

It's a framework, not so much a library. Although it has the latter's 
capabilities too. As a framework, we should consider what the end user 
experience would look like.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Mon, Nov 27, 2023 at 3:17 PM Dmitry Gutov <dmitry@gutov.dev> wrote:

> > In my view, again, we should not limit ourselves to a triad
> > of kinds clumsily inherited from LSP.  We should limit that LSP
> > concept to Eglot.  It is not flexible enough, because it can't
> > be.  But Xref is for much more than LSP.
>
> What it seems you are saying is that we never should try to extend the
> set of definition-finding commands above what's already in xref.el.

Not necessarily.  I'm saying we shouldn't make that a priority
or the rest of this useful feature contingent on that.  If we
someday come to the blissful epiphany that every language ever to
be and to have been has the fundamental concept of a watermelon in it,
then we can think about xref-find-watermelon.

> Because it seems clear that we're unlikely to find a more
> widely-supported set of additional kinds that the aforementioned triad,
> in any medium-term future.

They're not particularly well supported, they won't age well.  LSP
uses that trio for two reasons: it doesn't have major modes or a
programming language, and it's driven in part by commercial vendors of
specific large language implementations.  But Editors like VSCode still
have "plugins" akin to our major modes.

I wouldn't be surprised if these plugins don't have total freedom over
what part of LSP they expose to the user.

Opening VSCode on a C++ file for example, I see "find definitions" and
"find references" have dedicated bindings, but "declaration" and
"typeDefinition" do NOT. And "implementation" isn't even there as it
doesn't make much sense in C++.

> >> Like now, for example, many will likely stay with CC Mode for a while
> >> because of it more lax behavior in macro-heavy codebases, where
> >> c-ts-mode chokes and shows syntax errors.
> >
> > Whatever you are suggesting, I don't understand.  Whatever xref backend
> > is written for c++-mode with its special-purpose commands for finding
> > C++ things can be used for both c++-mode and c++-ts-mode.  Just put that
> > code in a special file.
>
> What about default key bindings?

I've already answered that in a nearby email.

> > I think this is a mistake and a regression from xref-find-extra.
> > The command should be called xref-find-all, xref-find, or simply
> > xref, since definition is a category that doesn't span a lot of
> > useful cross-referenceable constructs in many languages. In C++,
> > for one, a declaration is absolutely _not_ a definition.
>
> It's a "definition" in at least one Xref-related sense: it should be
> dispatched through xref-show-definitions-function.

So it's a "definition" because the xref.el author called an
implementation detail a "definition"?  What does the user possibly
care about that?

> Further, I'm not sure that when a user looks up all definition-related
> things for a symbol, they wouldn't want to see the "declaration". In
> fact, if we classify 'defgeneric' as declarations and 'defmethod' as
> implementations, I'm pretty sure I would want to see the former in the list.

Of course, if the world is painted in the three LSP primary colors,
every other color will have to be truncated to that.  But it doesn't
mean the world gets any prettier.

> And you yourself mentioned that "type definitions" might be suitable for
> that list (which I found surprising at first). So it seems clear that
> there is no single red line.

Precisely.  So don't go making those lines.

> o show me "references" as one of the kinds of thing to search for.
>
> Then the list of results would drown in "references", wouldn't it?

But if I want to see references to the given symbol at point, that's what
I want!  I press M-? xref-find-references all the time in Eglot.

> For experimentation.

Then perhaps we could cut another less contentious branch off
279203199a2d10677e42747476b39394a4184a78

Over that branch we can rename Eglot kinds to strings and "extra"
to "all".  I'm sure that's less contentious, a good candidate
for master and not incompatible with evolving into whatever
results of your experimentation.

> > If you are proposing this is brought to master for experimentation,
> > and if you are wary of this step, then may I suggest we push the older
> > more conservative version, perhaps with some naming changes?
>
> I'm not yet seeing a common basic version for which there would be
> agreement. You just called a renaming a mistake and disagreed with the
> principle of what "definition kinds" should be.
>
> > This new version isn't something we can roll back easily, while the
> > more conservative approach could eventually be enhanced with the three
> > "popular kinds" later on.
>
> I'm not proposing any merge to master yet.

Well I am, but not of that code, at least not yet of that code.
Merge to master something useful that doesn't compromise us.

> >> What does everyone think?
> >
> >> xref-find-all-definitions is an interactive native-compiled Lisp
> >> function in `xref.el'.
> >
> >> at point, prompt for the identifier.  Interactively, show matches
> >> for all supported kinds.  When invoked with prefix argument,
> >> prompt for KIND.
> >
> > I can't get this to work btw.   If I add a prefx argument, it
> > prompts me for the identifier, which I don't want.
>
> What's what the docstring says, and it's what Felician brought up as a
> problem. Do you want two separate commands? And one of them would always
> prompt for the kind to use?

No, it's not.  It starts prompting me for identifiers which for LSP
is useless.  Eglot has no way to know which kinds of reference it has
for arbitrary identifiers not at point.

The only useful way for this to work in Eglot would be to prompt
for KIND and _then_ show the results, which may be empty.

Anything else doesn't make sense for Eglot, for LSP-specific
reasons beyond my control which I've already explained.

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 27/11/2023 17:45, João Távora wrote:
> On Mon, Nov 27, 2023 at 3:17 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
> 
>>> In my view, again, we should not limit ourselves to a triad
>>> of kinds clumsily inherited from LSP.  We should limit that LSP
>>> concept to Eglot.  It is not flexible enough, because it can't
>>> be.  But Xref is for much more than LSP.
>>
>> What it seems you are saying is that we never should try to extend the
>> set of definition-finding commands above what's already in xref.el.
> 
> Not necessarily.  I'm saying we shouldn't make that a priority
> or the rest of this useful feature contingent on that.  If we
> someday come to the blissful epiphany that every language ever to
> be and to have been has the fundamental concept of a watermelon in it,
> then we can think about xref-find-watermelon.

In 20-30 years or so?

>> Because it seems clear that we're unlikely to find a more
>> widely-supported set of additional kinds that the aforementioned triad,
>> in any medium-term future.
> 
> They're not particularly well supported, they won't age well.  LSP
> uses that trio for two reasons: it doesn't have major modes or a
> programming language, and it's driven in part by commercial vendors of
> specific large language implementations.  But Editors like VSCode still
> have "plugins" akin to our major modes.

I don't know about aging well: most of the aforementioned ones have had 
meaning for the whole 15 years of my career (that's not too long, but 
still), and typeDefinitions is apparently something more helpful in 
newer popular languages which rely on type inference (so there's rarely 
a type annotation nearby to 'M-.' on).

> I wouldn't be surprised if these plugins don't have total freedom over
> what part of LSP they expose to the user.
> 
> Opening VSCode on a C++ file for example, I see "find definitions" and
> "find references" have dedicated bindings, but "declaration" and
> "typeDefinition" do NOT. And "implementation" isn't even there as it
> doesn't make much sense in C++.

Ultimately, we could also show-hide definitions depending on which kinds 
the current backend(s) expose. I.e. a set of commands might be 
dynamically generated via dispatch (like project-switch-project does), 
or they indeed can be traditional commands. The latter has advantages: 
the users can examine and rebind them using the traditional methods.

Indeed, some commands will be "unsupported" in some languages. I don't 
see much issue with it: if the language mode (major or minor) has a 
better command to use instead, they can [remap ...] it. And they will 
have a common prefix to add their own additional commands too.

>>> I think this is a mistake and a regression from xref-find-extra.
>>> The command should be called xref-find-all, xref-find, or simply
>>> xref, since definition is a category that doesn't span a lot of
>>> useful cross-referenceable constructs in many languages. In C++,
>>> for one, a declaration is absolutely _not_ a definition.
>>
>> It's a "definition" in at least one Xref-related sense: it should be
>> dispatched through xref-show-definitions-function.
> 
> So it's a "definition" because the xref.el author called an
> implementation detail a "definition"?  What does the user possibly
> care about that?

Why don't you read up on the difference between 
xref-show-definitions-function and xref-show-xrefs-function. Those are 
not implementation details but something that affects user experience. 
And users can customize one or the other, which correspondingly affects 
dispatches which go through one or the other.

>> Further, I'm not sure that when a user looks up all definition-related
>> things for a symbol, they wouldn't want to see the "declaration". In
>> fact, if we classify 'defgeneric' as declarations and 'defmethod' as
>> implementations, I'm pretty sure I would want to see the former in the list.
> 
> Of course, if the world is painted in the three LSP primary colors,
> every other color will have to be truncated to that.  But it doesn't
> mean the world gets any prettier.

Would it be better if the user has to type 'dec' in C mode but 'defg' in 
emacs-lisp-mode to get to a semantically similar set of results?

>> And you yourself mentioned that "type definitions" might be suitable for
>> that list (which I found surprising at first). So it seems clear that
>> there is no single red line.
> 
> Precisely.  So don't go making those lines.
> 
>> o show me "references" as one of the kinds of thing to search for.
>>
>> Then the list of results would drown in "references", wouldn't it?
> 
> But if I want to see references to the given symbol at point, that's what
> I want!  I press M-? xref-find-references all the time in Eglot.

And that's great -- please continue to press M-? for that purpose.

But if 'references' joins the "definition kinds", then the command to 
"find all definition kinds" will become pointless. And we already have 
command to "find all references".

>> For experimentation.
> 
> Then perhaps we could cut another less contentious branch off
> 279203199a2d10677e42747476b39394a4184a78
> 
> Over that branch we can rename Eglot kinds to strings and "extra"
> to "all".  I'm sure that's less contentious, a good candidate
> for master and not incompatible with evolving into whatever
> results of your experimentation.

See above.

>>> If you are proposing this is brought to master for experimentation,
>>> and if you are wary of this step, then may I suggest we push the older
>>> more conservative version, perhaps with some naming changes?
>>
>> I'm not yet seeing a common basic version for which there would be
>> agreement. You just called a renaming a mistake and disagreed with the
>> principle of what "definition kinds" should be.
>>
>>> This new version isn't something we can roll back easily, while the
>>> more conservative approach could eventually be enhanced with the three
>>> "popular kinds" later on.
>>
>> I'm not proposing any merge to master yet.
> 
> Well I am, but not of that code, at least not yet of that code.
> Merge to master something useful that doesn't compromise us.

We're yet finding that.

>>>> What does everyone think?
>>>
>>>> xref-find-all-definitions is an interactive native-compiled Lisp
>>>> function in `xref.el'.
>>>
>>>> at point, prompt for the identifier.  Interactively, show matches
>>>> for all supported kinds.  When invoked with prefix argument,
>>>> prompt for KIND.
>>>
>>> I can't get this to work btw.   If I add a prefx argument, it
>>> prompts me for the identifier, which I don't want.
>>
>> What's what the docstring says, and it's what Felician brought up as a
>> problem. Do you want two separate commands? And one of them would always
>> prompt for the kind to use?
> 
> No, it's not.  It starts prompting me for identifiers which for LSP
> is useless.  Eglot has no way to know which kinds of reference it has
> for arbitrary identifiers not at point.

I said that it matches the docstring, not that it's ideal for Eglot.

> The only useful way for this to work in Eglot would be to prompt
> for KIND and _then_ show the results, which may be empty.
> 
> Anything else doesn't make sense for Eglot, for LSP-specific
> reasons beyond my control which I've already explained.

What about showing the combined set?

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Mon, Nov 27, 2023 at 3:45 PM Dmitry Gutov <dmitry@gutov.dev> wrote:

> That didn't answer the above question. To signal "what kind of
> cross-references it can find", it would define
> xref-backend-definition-kinds with corresponding return value.

The answer was in the first two words.  "Of course", and the reason
is in the words following that.

So to be clear, yes, c++-ts-mode, if it ever gets such capabilities
should define c++-ts-find-declaration, c++-ts-find-partial-specializations
and so forth, but probably no c++-ts-find-implementation (fuzzy concept
in C++).  It should (or rather, it may) bind them in a given xref-provided
prefix map and they should become active when c++-ts-mode is active.

For "references" and "definitions" the existing xref commands will do,
but the phrases "references" and "definitions" should well be in the
"prompt for kind" list as options.

And no, it doesn't take much imagination to do this for CC-mode as
well, if the backend supports both C++ modes.

> > The major modes could do that, for example.  They could do it by
> > linking the backend-created "one key/command" keymap into a special
> > prefix created by xref.el, doing so by calling a load-time xref.el helper.
>
> Okay, so we would create and occupy a new common prefix. But without any
> commands in it? How do you suggest we advocate for that around here?

What's the difference between no commands and useless commands that return
errors?

But you could put the existing xref-find-definition and xref-find-references
commands in it.  Even the "find-all" could be there.  Also, as far as I can
see, you are already ogling the M-' binding in your branch anyway.

> > So maybe some backends would have d for  "documentation" and others would
> > have d for definitions, so what? Maybe that's precisely what a user working
> > on a documentation heavy system expects "d" to do. And if these two backends
> > happen to be enabled simultaneously,  even this conflict could be
> > resolved, by adjusting the keymap during the linking process.
>
> That's easily solvable in other ways.

OK.  I just suggested one, keep it in your catalog maybe.

> And I didn't just invent the need for faster interaction by myself -
> just look at the first responses to the prototype. Okay, perhaps some
> power users in this thread don't mind adding personal tweaks to their
> config to speed things up. But that means the OOtB experience will be
> lacking.

I'm not saying the bindings issue isn't important, I'm saying it's
(1) secondary to the communication mechanism and the functionality
which is already pretty useful in your original patch in this
20-years+ Emacs user's experience (and I don't keybind things).
(2) not something we should rush to solve by copying an awkward
categorization that doesn't fit the needs of languages.
(3) solvable in all practical aspects by judicious coordination
with backends and major modes.

> > It's much more pressing to decide if/why you want a command named
> > xref-find-all-definitions to return things that aren't "definitions"
> > by any measure or what is pressuring us to enshrine
> > "declaration/implementation/typeDefinition" in one of our
> > most generic libraries.
>
> It's a framework, not so much a library. Although it has the latter's
> capabilities too. As a framework, we should consider what the end user
> experience would look like.

Sure, of course.  And it already does that it excellently in so many
good ways (IMHO the decision to copy SLIME's UI was excellent). But
don't try to make it the uberframework for finding things to the point
of making Emacs-wide commands copied from LSP that don't fit a bunch
of the vast amount of Emacs use cases.  It's too heavy handed,
a step too far, not elegant.

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 27/11/2023 18:04, João Távora wrote:
> On Mon, Nov 27, 2023 at 3:45 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
> 
>> That didn't answer the above question. To signal "what kind of
>> cross-references it can find", it would define
>> xref-backend-definition-kinds with corresponding return value.
> 
> The answer was in the first two words.  "Of course", and the reason
> is in the words following that.
> 
> So to be clear, yes, c++-ts-mode, if it ever gets such capabilities
> should define c++-ts-find-declaration, c++-ts-find-partial-specializations
> and so forth, but probably no c++-ts-find-implementation (fuzzy concept
> in C++).  It should (or rather, it may) bind them in a given xref-provided
> prefix map and they should become active when c++-ts-mode is active.
> 
> For "references" and "definitions" the existing xref commands will do,
> but the phrases "references" and "definitions" should well be in the
> "prompt for kind" list as options.

We in the end we'll have dozens of similarly-named commands with the 
same implementation, right? Some of them in the core, others in 
third-party addons.

> And no, it doesn't take much imagination to do this for CC-mode as
> well, if the backend supports both C++ modes.
> 
>>> The major modes could do that, for example.  They could do it by
>>> linking the backend-created "one key/command" keymap into a special
>>> prefix created by xref.el, doing so by calling a load-time xref.el helper.
>>
>> Okay, so we would create and occupy a new common prefix. But without any
>> commands in it? How do you suggest we advocate for that around here?
> 
> What's the difference between no commands and useless commands that return
> errors?

Existing binding that one could [remap]. Or an opportinity for an Xref 
backend to extend itself into corresponding types.

> But you could put the existing xref-find-definition and xref-find-references
> commands in it.  Even the "find-all" could be there.

That... doesn't sound very convincing, TBH.

 > Also, as far as I can
 > see, you are already ogling the M-' binding in your branch anyway.

M-' has an existing binding, and the contents of the branch could help 
make a case to change it. And then, if we don't add other new commands, 
why force people to use "M-' M-'" if they could just use the "M-'" for 
the only new command.

>> And I didn't just invent the need for faster interaction by myself -
>> just look at the first responses to the prototype. Okay, perhaps some
>> power users in this thread don't mind adding personal tweaks to their
>> config to speed things up. But that means the OOtB experience will be
>> lacking.
> 
> I'm not saying the bindings issue isn't important, I'm saying it's
> (1) secondary to the communication mechanism and the functionality
> which is already pretty useful in your original patch in this
> 20-years+ Emacs user's experience (and I don't keybind things).
> (2) not something we should rush to solve by copying an awkward
> categorization that doesn't fit the needs of languages.
> (3) solvable in all practical aspects by judicious coordination
> with backends and major modes.

True. Aside from the issues of 1) naming, 2) presence of "references" in 
the set, 3) interface to "fetch all" which we still haven't come to 
accord on.

>>> It's much more pressing to decide if/why you want a command named
>>> xref-find-all-definitions to return things that aren't "definitions"
>>> by any measure or what is pressuring us to enshrine
>>> "declaration/implementation/typeDefinition" in one of our
>>> most generic libraries.
>>
>> It's a framework, not so much a library. Although it has the latter's
>> capabilities too. As a framework, we should consider what the end user
>> experience would look like.
> 
> Sure, of course.  And it already does that it excellently in so many
> good ways (IMHO the decision to copy SLIME's UI was excellent). But
> don't try to make it the uberframework for finding things to the point
> of making Emacs-wide commands copied from LSP that don't fit a bunch
> of the vast amount of Emacs use cases.  It's too heavy handed,
> a step too far, not elegant.

We don't have to copy LSP, or follow any of its subsequent additions, or 
even copy the 3 "kinds" already mentioned. But any tentative plan for 
making user experience across backends faster and more unified would be 
good to have. Even if we don't act on it yet, code-wise.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Mon, Nov 27, 2023 at 4:04 PM Dmitry Gutov <dmitry@gutov.dev> wrote:

> Why don't you read up on the difference between
> xref-show-definitions-function and xref-show-xrefs-function. Those are
> not implementation details but something that affects user experience.
> And users can customize one or the other, which correspondingly affects
> dispatches which go through one or the other.

<eyeroll> First, who does that?  I thought you were concerned about
the OOtB experience for non-powerusers.  Secondly, if something called
"definitions" is used for things that are patently not "definitions"
that thing is incorrectly named, period.  It's not because that editor
thing was misnamed by its creator that magically my C++ declaration is
now a definition, too.

> >> Further, I'm not sure that when a user looks up all definition-related
> >> things for a symbol, they wouldn't want to see the "declaration". In
> >> fact, if we classify 'defgeneric' as declarations and 'defmethod' as
> >> implementations, I'm pretty sure I would want to see the former in the list.
> >
> > Of course, if the world is painted in the three LSP primary colors,
> > every other color will have to be truncated to that.  But it doesn't
> > mean the world gets any prettier.
>
> Would it be better if the user has to type 'dec' in C mode but 'defg' in
> emacs-lisp-mode to get to a semantically similar set of results?

They're absolutely NOT semantically similar, only very superficially,
like, yes, most users of this list are vertebrates. I don't mean to
be pedantic about languages but there's really a world of difference
between a C declaration and the things you can say with a DEFGENERIC
call, or what their specific intended purposes are for the language
designer. Any user of these two languages (C and Common Lisp) languages
will know this very well.  In my 15 years CL experience I've known exactly
0 people calling defgeneric forms "declarations".

But I presume you're talking about completing-read?  I would type 'dec'
in the first and 'gene' in the second and so would probably the masses.
But the list is so small, I'd probably just scroll down one or two
positions.

> >> And you yourself mentioned that "type definitions" might be suitable for
> >> that list (which I found surprising at first). So it seems clear that
> >> there is no single red line.
> >
> > Precisely.  So don't go making those lines.
> >
> >> o show me "references" as one of the kinds of thing to search for.
> >>
> >> Then the list of results would drown in "references", wouldn't it?
> >
> > But if I want to see references to the given symbol at point, that's what
> > I want!  I press M-? xref-find-references all the time in Eglot.
>
> And that's great -- please continue to press M-? for that purpose.
>
> But if 'references' joins the "definition kinds", then the command to
> "find all definition kinds" will become pointless. And we already have
> command to "find all references".

No, not pointless at all.  My main use case for this is to
first invoke the command on a given place of interest and _then_
ask myself what I want to see of that symbol. "All references",
"definitions", "declarations"?

> >> For experimentation.
> >
> > Then perhaps we could cut another less contentious branch off
> > 279203199a2d10677e42747476b39394a4184a78
> >
> > Over that branch we can rename Eglot kinds to strings and "extra"
> > to "all".  I'm sure that's less contentious, a good candidate
> > for master and not incompatible with evolving into whatever
> > results of your experimentation.
>
> See above.

So, awright.  Do you really really want to rename "extra" to
"definitions"??  is that the blocker?   Makes 0 sense, but if
there's no talking you out of it then I guess users like me
can swallow  the awkwardness and translate "definition" to
"thing" in their  minds.  And we have strategies for renaming
things anyway.

> > Well I am, but not of that code, at least not yet of that code.
> > Merge to master something useful that doesn't compromise us.
>
> We're yet finding that.

What exactly is compromised by that patch?  And do you not agree
it is minimally useful?  You wrote the thing! Noone forced you to.

> What about showing the combined set?

Sure, that can be done.  Just have xref invoke the generic function
for as many kinds as the backend reports.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 27/11/2023 17:28, Eli Zaretskii wrote:
>> Date: Mon, 27 Nov 2023 17:01:58 +0200
>> Cc: Felician Nemeth<felician.nemeth@gmail.com>,
>>   Spencer Baugh<sbaugh@janestreet.com>,emacs-devel@gnu.org,
>>   Eshel Yaron<me@eshelyaron.com>, John Yates<john@yates-sheets.org>,
>>   Ergus<spacibba@aol.com>, Filipp Gunbin<fgunbin@fastmail.fm>
>> From: Dmitry Gutov<dmitry@gutov.dev>
>>
>> On 27/11/2023 16:49, João Távora wrote:
>>> On Mon, Nov 27, 2023 at 2:43 PM Dmitry Gutov<dmitry@gutov.dev>  wrote:
>>>
>>>> (that's what the completing-read approach is for), and whenever we find
>>>> out that particular kinds get supported by many backends (or by most of
>>>> the available/popular ones), we could "pull them into the core", adding
>>>> a global command with one binding which would work across languages.
>>> Alright, so since this is contentious (who else but you is pushing
>>> this idea?)
>> Indeed, I wonder if nobody else is interested in having the additional
>> commands have pre-defined bindings, or having the same bindings across
>> languages.
> I might be interested, but you guys have a tendency to drown the
> important issues in an ocean of second- and third-order stuff.  It is
> very hard to follow this discussion, with its longish posts and a lot
> of barely-related issues and arguments.

That's moderately insulting. To me, at least.

> Maybe it's time to post some kind of summary?

I just posted a summary on 24/11/2023, 03:37 +02:00. Do you perhaps have 
any questions related to it?

One of the disagreements is on whether Xref should have any new 
kind-commands, such as xref-find-declarations, xref-find-implemenetation 
and xref-find-type-definitions which you can find on the branch 
feature/xref-find-extra (the diff is not big, you can take a look).

Or whether definitions like that should be reimplemented by every Xref 
backend that wants to use them, and the backends would themselves pick 
the key bindings for them (or leave unbound).

The usefulness of this also depends on whether we can appoint a 
comfortable binding for the prefix map for the new commands. In the 
prototype I've used "M-'" -- a binding currently occupied by a command 
that I've never used in my life, which others here seemed to agree with 
-- but any other alternative can be considered, of course.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Mon, Nov 27, 2023 at 4:23 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 27/11/2023 18:04, João Távora wrote:
> > On Mon, Nov 27, 2023 at 3:45 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
> >
> >> That didn't answer the above question. To signal "what kind of
> >> cross-references it can find", it would define
> >> xref-backend-definition-kinds with corresponding return value.
> >
> > The answer was in the first two words.  "Of course", and the reason
> > is in the words following that.
> >
> > So to be clear, yes, c++-ts-mode, if it ever gets such capabilities
> > should define c++-ts-find-declaration, c++-ts-find-partial-specializations
> > and so forth, but probably no c++-ts-find-implementation (fuzzy concept
> > in C++).  It should (or rather, it may) bind them in a given xref-provided
> > prefix map and they should become active when c++-ts-mode is active.
> >
> > For "references" and "definitions" the existing xref commands will do,
> > but the phrases "references" and "definitions" should well be in the
> > "prompt for kind" list as options.
>
> We in the end we'll have dozens of similarly-named commands with the
> same implementation, right? Some of them in the core, others in
> third-party addons.

What same implementation?? They only common part is is literally just
the string "xref-define-finder", the opening and the closing parenthesis.
Then comes the backend specifics.  A JSONRPC-based transport with a specific
protocol, an HTTP transport with another protocol, a ask the oracle of Delphos
protocol, etc.

Realistically, most people will be using Eglot anyway, so the existing
eglot-find-* commands are what they're going to see and use.

> > But you could put the existing xref-find-definition and xref-find-references
> > commands in it.  Even the "find-all" could be there.
>
> That... doesn't sound very convincing, TBH.

That's obvious.  It's clear to me that you are already convinced
of whatever you want and all this discussion leads nowhere.

So do what you want.  If I like it, Eglot will use it.  Else I will
do my own commands in eglot.el.  It'd be nice to reuse as much
of Emacs as possible, but when it's not suitable (like many parts)
or takes years to bikeshed to an agreement, I don't have to.

I think I've given this discussion enough energy argument, etc.
You're familiar with all the technical limitations of Eglot, etc.

So, to summarize,  I will branch from `feature/xref-find-extra` in
that well-known point, polish some things there to a mergeable
point.  If you come around and want to merge what is basically your
own patch of a month ago, great!

João

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Mon, Nov 27, 2023 at 4:37 PM Dmitry Gutov <dmitry@gutov.dev> wrote:

> > I might be interested, but you guys have a tendency to drown the
> > important issues in an ocean of second- and third-order stuff.  It is
> > very hard to follow this discussion, with its longish posts and a lot
> > of barely-related issues and arguments.
>
> That's moderately insulting. To me, at least.

Yeah, to me too, a bit. But it's also kinda true, if I may be honest.

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 27/11/2023 18:41, João Távora wrote:
> On Mon, Nov 27, 2023 at 4:23 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
>>
>> On 27/11/2023 18:04, João Távora wrote:
>>> On Mon, Nov 27, 2023 at 3:45 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
>>>
>>>> That didn't answer the above question. To signal "what kind of
>>>> cross-references it can find", it would define
>>>> xref-backend-definition-kinds with corresponding return value.
>>>
>>> The answer was in the first two words.  "Of course", and the reason
>>> is in the words following that.
>>>
>>> So to be clear, yes, c++-ts-mode, if it ever gets such capabilities
>>> should define c++-ts-find-declaration, c++-ts-find-partial-specializations
>>> and so forth, but probably no c++-ts-find-implementation (fuzzy concept
>>> in C++).  It should (or rather, it may) bind them in a given xref-provided
>>> prefix map and they should become active when c++-ts-mode is active.
>>>
>>> For "references" and "definitions" the existing xref commands will do,
>>> but the phrases "references" and "definitions" should well be in the
>>> "prompt for kind" list as options.
>>
>> We in the end we'll have dozens of similarly-named commands with the
>> same implementation, right? Some of them in the core, others in
>> third-party addons.
> 
> What same implementation?? They only common part is is literally just
> the string "xref-define-finder", the opening and the closing parenthesis.
> Then comes the backend specifics.  A JSONRPC-based transport with a specific
> protocol, an HTTP transport with another protocol, a ask the oracle of Delphos
> protocol, etc.
> 
> Realistically, most people will be using Eglot anyway, so the existing
> eglot-find-* commands are what they're going to see and use.

So that's what your argument comes down to in practical sense -- keep 
the commands in Eglot, and not worry much about other Xref backends. 
Whether they are alternative LSP clients or not.

Setting aside the issue of us binding ourselves to Eglot too closely, 
that *would* mean keeping those commands unbound for the foreseeable 
future. And I do remember users asking about the default bindings.

>>> But you could put the existing xref-find-definition and xref-find-references
>>> commands in it.  Even the "find-all" could be there.
>>
>> That... doesn't sound very convincing, TBH.
> 
> That's obvious.  It's clear to me that you are already convinced
> of whatever you want and all this discussion leads nowhere.

I'm saying it doesn't look good as an argument that I could make to 
convince the current head maintainers of Emacs to free up the "M-'" 
binding. Or any other short key sequence, probably.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Mon, Nov 27, 2023 at 5:05 PM Dmitry Gutov <dmitry@gutov.dev> wrote:

> > What same implementation?? They only common part is is literally just
> > the string "xref-define-finder", the opening and the closing parenthesis.
> > Then comes the backend specifics.  A JSONRPC-based transport with a specific
> > protocol, an HTTP transport with another protocol, a ask the oracle of Delphos
> > protocol, etc.
> >
> > Realistically, most people will be using Eglot anyway, so the existing
> > eglot-find-* commands are what they're going to see and use.
>
> So that's what your argument comes down to in practical sense -- keep
> the commands in Eglot, and not worry much about other Xref backends.
> Whether they are alternative LSP clients or not.

No, of course not.  And frankly it's a bit silly to try to
caricaturize my position with this strawman, given how much of
my text you've read.  I provided ample support for other backends
and  I myself want to write such a non-Eglot backend for SLY
Common Lisp support.

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 27/11/2023 18:27, João Távora wrote:
> On Mon, Nov 27, 2023 at 4:04 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
> 
>> Why don't you read up on the difference between
>> xref-show-definitions-function and xref-show-xrefs-function. Those are
>> not implementation details but something that affects user experience.
>> And users can customize one or the other, which correspondingly affects
>> dispatches which go through one or the other.
> 
> <eyeroll> First, who does that?  I thought you were concerned about
> the OOtB experience for non-powerusers.

It doesn't take a power user to use the Customize interface to set 
xref-show-definitions-function to xref-show-definitions-completing-read.

Who uses that? I do, for example. And the people who asked for it (see 
https://lists.gnu.org/archive/html/emacs-devel/2020-11/msg00824.html). 
There are also third-party packages which provide alternative UIs 
through these options (e.g. ivy-xref and helm-xref).

> Secondly, if something called
> "definitions" is used for things that are patently not "definitions"
> that thing is incorrectly named, period.  It's not because that editor
> thing was misnamed by its creator that magically my C++ declaration is
> now a definition, too.

It's about "set of things where I usually want to pick only one and jump 
to it" versus "set of things which I usually want to see the full list 
of, which stays around".

You can say that "definition" is perhaps not an ideal term for the 
former. But the distinction is useful, and if you have better naming 
suggestions, go ahead.

Until now the only built-in commands which went through that dispatcher 
were the variations of xref-find-definitions, so the name wasn't a problem.

>>>> o show me "references" as one of the kinds of thing to search for.
>>>>
>>>> Then the list of results would drown in "references", wouldn't it?
>>>
>>> But if I want to see references to the given symbol at point, that's what
>>> I want!  I press M-? xref-find-references all the time in Eglot.
>>
>> And that's great -- please continue to press M-? for that purpose.
>>
>> But if 'references' joins the "definition kinds", then the command to
>> "find all definition kinds" will become pointless. And we already have
>> command to "find all references".
> 
> No, not pointless at all.  My main use case for this is to
> first invoke the command on a given place of interest and _then_
> ask myself what I want to see of that symbol. "All references",
> "definitions", "declarations"?

The combined view, as implemented in the current 
xref-find-all-definitions by default, will then just show references.

Why would you want to also be able to reach "references" through that 
interface anyway?

>>>> For experimentation.
>>>
>>> Then perhaps we could cut another less contentious branch off
>>> 279203199a2d10677e42747476b39394a4184a78
>>>
>>> Over that branch we can rename Eglot kinds to strings and "extra"
>>> to "all".  I'm sure that's less contentious, a good candidate
>>> for master and not incompatible with evolving into whatever
>>> results of your experimentation.
>>
>> See above.
> 
> So, awright.  Do you really really want to rename "extra" to
> "definitions"??  is that the blocker?   Makes 0 sense, but if
> there's no talking you out of it then I guess users like me
> can swallow  the awkwardness and translate "definition" to
> "thing" in their  minds.  And we have strategies for renaming
> things anyway.

Like I said, they are not "extra" because they will on most cases 
include the kinds already shown by xref-find-definitions. I'm open to 
other names, but please keep the intended semantics in mind (see above).

>>> Well I am, but not of that code, at least not yet of that code.
>>> Merge to master something useful that doesn't compromise us.
>>
>> We're yet finding that.
> 
> What exactly is compromised by that patch?  And do you not agree
> it is minimally useful?  You wrote the thing! Noone forced you to.

I voiced my doubts in the very first message that patch was attached to. 
And that it's experimental. It's also like 20 lines total. Let's keep a 
perspective.

Re: Adding support for xref jumping to headers/interfaces

[Eli Zaretskii]

> Date: Mon, 27 Nov 2023 18:37:28 +0200
> Cc: joaotavora@gmail.com, felician.nemeth@gmail.com, sbaugh@janestreet.com,
>  emacs-devel@gnu.org, me@eshelyaron.com, john@yates-sheets.org,
>  spacibba@aol.com, fgunbin@fastmail.fm
> From: Dmitry Gutov <dmitry@gutov.dev>
> 
> >> Indeed, I wonder if nobody else is interested in having the additional
> >> commands have pre-defined bindings, or having the same bindings across
> >> languages.
> > I might be interested, but you guys have a tendency to drown the
> > important issues in an ocean of second- and third-order stuff.  It is
> > very hard to follow this discussion, with its longish posts and a lot
> > of barely-related issues and arguments.
> 
> That's moderately insulting. To me, at least.

Sorry, I will keep silence from now on.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Mon, Nov 27, 2023 at 5:22 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
>
> On 27/11/2023 18:27, João Távora wrote:
> > On Mon, Nov 27, 2023 at 4:04 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
> >
> >> Why don't you read up on the difference between
> >> xref-show-definitions-function and xref-show-xrefs-function. Those are
> >> not implementation details but something that affects user experience.
> >> And users can customize one or the other, which correspondingly affects
> >> dispatches which go through one or the other.
> >
> > <eyeroll> First, who does that?  I thought you were concerned about
> > the OOtB experience for non-powerusers.
>
> It doesn't take a power user to use the Customize interface to set
> xref-show-definitions-function to xref-show-definitions-completing-read.
>
> Who uses that? I do, for example. And the people who asked for it (see
> https://lists.gnu.org/archive/html/emacs-devel/2020-11/msg00824.html).
> There are also third-party packages which provide alternative UIs
> through these options (e.g. ivy-xref and helm-xref).
>
> > Secondly, if something called
> > "definitions" is used for things that are patently not "definitions"
> > that thing is incorrectly named, period.  It's not because that editor
> > thing was misnamed by its creator that magically my C++ declaration is
> > now a definition, too.
>
> It's about "set of things where I usually want to pick only one and jump
> to it" versus "set of things which I usually want to see the full list
> of, which stays around".
>
> You can say that "definition" is perhaps not an ideal term for the
> former. But the distinction is useful, and if you have better naming
> suggestions, go ahead.
>
> Until now the only built-in commands which went through that dispatcher
> were the variations of xref-find-definitions, so the name wasn't a problem.

This whole thing started because you used that misnaming of
existing things as a justification for misnaming new things.

> > No, not pointless at all.  My main use case for this is to
> > first invoke the command on a given place of interest and _then_
> > ask myself what I want to see of that symbol. "All references",
> > "definitions", "declarations"?
>
> The combined view, as implemented in the current
> xref-find-all-definitions by default, will then just show references.

Not necessarily, it depends on the backend's.  But yes, it might.
But I presume they will be categorized by kind.

> Why would you want to also be able to reach "references" through that
> interface anyway?

Why not? To avoid the noise of that categorization. To get to some
reference type not explicitly covered in those other categories.

> Like I said, they are not "extra" because they will on most cases
> include the kinds already shown by xref-find-definitions. I'm open to
> other names, but please keep the intended semantics in mind (see above).

How bout "cross-reference", or simply xref?  There's a reason why your xref.el
which you maintain inherited that name from SLIME by whoever ported the UI
(Helmut it was, IIRC);-)  There's a lot of wisdom in that name.

xref-backend-extra-kinds ->
                   xref-backend-xref-kinds OR
                   xref-backend-xref-cross-reference-kinds

xref-backend-extra-defs ->
                   xref-backend-xref-defs OR
                   xref-backend-cross-reference-defs OR
                   xref-backend-xrefs (this method literally returns xrefs)

xref-find-extra ->
                   xref-find-all (more people suggested this, I think)
                   xref-find
                   xref
                   xref-find-combined
                   xref-find-by-kind


> > What exactly is compromised by that patch?  And do you not agree
> > it is minimally useful?  You wrote the thing! Noone forced you to.
>
> I voiced my doubts in the very first message that patch was attached to.
> And that it's experimental. It's also like 20 lines total. Let's keep a
> perspective.

Yes! a simple workable patch that someone else liked, spent time
on, worked on, solves real-world problems, and doesn't lock us in
anything.   "Doesn't come with keybindings! Blasphemy! Let's
bikeshed, make it very complicated!"

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 27/11/2023 19:46, João Távora wrote:

>> It's about "set of things where I usually want to pick only one and jump
>> to it" versus "set of things which I usually want to see the full list
>> of, which stays around".
>>
>> You can say that "definition" is perhaps not an ideal term for the
>> former. But the distinction is useful, and if you have better naming
>> suggestions, go ahead.
>>
>> Until now the only built-in commands which went through that dispatcher
>> were the variations of xref-find-definitions, so the name wasn't a problem.
> 
> This whole thing started because you used that misnaming of
> existing things as a justification for misnaming new things.

And "implementations" are not definitions? Are "type definitions" not 
"definitions"?

What goes for declarations in a lot of cases will also be "definitions 
of some sort". E.g. in Java a method declaration is its abstract 
definition in the interface or parent class. There are also abstract 
method definitions in C++ and so on.

>>> No, not pointless at all.  My main use case for this is to
>>> first invoke the command on a given place of interest and _then_
>>> ask myself what I want to see of that symbol. "All references",
>>> "definitions", "declarations"?
>>
>> The combined view, as implemented in the current
>> xref-find-all-definitions by default, will then just show references.
> 
> Not necessarily, it depends on the backend's.  But yes, it might.

How wouldn't it be?

>> Why would you want to also be able to reach "references" through that
>> interface anyway?
> 
> Why not? To avoid the noise of that categorization. To get to some
> reference type not explicitly covered in those other categories.

To avoid the noise, you can just use an existing separate command.

>> Like I said, they are not "extra" because they will on most cases
>> include the kinds already shown by xref-find-definitions. I'm open to
>> other names, but please keep the intended semantics in mind (see above).
> 
> How bout "cross-reference", or simply xref?  There's a reason why your xref.el
> which you maintain inherited that name from SLIME by whoever ported the UI
> (Helmut it was, IIRC);-)  There's a lot of wisdom in that name.

That's the general term for all thingies Xref shows. The "xrefs" in the 
name "xref-show-xrefs-function" stands for pretty much that.

If you don't like "definitions" in "xref-show-definitions-function", 
then it would need to be a term that describes "special kind of 
cross-references". Probably referring to the kinds of locations those 
cross-references point to (which determines the optimal UI).

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 27/11/2023 19:40, Eli Zaretskii wrote:
>> Date: Mon, 27 Nov 2023 18:37:28 +0200
>> Cc:joaotavora@gmail.com,felician.nemeth@gmail.com,sbaugh@janestreet.com,
>>   emacs-devel@gnu.org,me@eshelyaron.com,john@yates-sheets.org,
>>   spacibba@aol.com,fgunbin@fastmail.fm
>> From: Dmitry Gutov<dmitry@gutov.dev>
>>
>>>> Indeed, I wonder if nobody else is interested in having the additional
>>>> commands have pre-defined bindings, or having the same bindings across
>>>> languages.
>>> I might be interested, but you guys have a tendency to drown the
>>> important issues in an ocean of second- and third-order stuff.  It is
>>> very hard to follow this discussion, with its longish posts and a lot
>>> of barely-related issues and arguments.
>> That's moderately insulting. To me, at least.
> Sorry, I will keep silence from now on.

I made an effort to provide you a personal summary.

If the right to address my emails in an insulting fashion is more 
important, so be it.

Re: Adding support for xref jumping to headers/interfaces

[Eli Zaretskii]

> Date: Mon, 27 Nov 2023 20:26:34 +0200
> Cc: joaotavora@gmail.com, felician.nemeth@gmail.com, sbaugh@janestreet.com,
>  emacs-devel@gnu.org, me@eshelyaron.com, john@yates-sheets.org,
>  spacibba@aol.com, fgunbin@fastmail.fm
> From: Dmitry Gutov <dmitry@gutov.dev>
> 
> On 27/11/2023 19:40, Eli Zaretskii wrote:
> >>> I might be interested, but you guys have a tendency to drown the
> >>> important issues in an ocean of second- and third-order stuff.  It is
> >>> very hard to follow this discussion, with its longish posts and a lot
> >>> of barely-related issues and arguments.
> >> That's moderately insulting. To me, at least.
> > Sorry, I will keep silence from now on.
> 
> I made an effort to provide you a personal summary.
> 
> If the right to address my emails in an insulting fashion is more 
> important, so be it.

You see an insult where there is none, and only in what others say.
Your own insults you never see.

You asked for opinions, and I tried to tell what I would need to have
an opinion.  If the only response to that is you taking offense (for
no good reason), I prefer to shut up, thank you very much.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 27/11/2023 22:50, Eli Zaretskii wrote:
> If the only response to that is you taking offense

Was it?

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Mon, Nov 27, 2023 at 6:12 PM Dmitry Gutov <dmitry@gutov.dev> wrote:

> There are also abstract method definitions in C++ and so on.

No, there aren't.  You probably mean "(pure) virtual member function".
And yes, I can assure you that's the jargon that professional and aspiring
C++ developers prefer when communicating with colleagues and compilers,
certainly not wishy-washy Javaisms.

> What goes for declarations in a lot of cases will also be "definitions
of some sort".

But if you squint hard enough, everything's "definition".  In fact,
definition definition definition.  Definition?  Definition!

> If you don't like "definitions" in "xref-show-definitions-function",
> then it would need to be a term that describes "special kind of
> cross-references". Probably referring to the kinds of locations those
> cross-references point to (which determines the optimal UI).

I'm fine with that name myself, it's specifically for xref
-find-definitions, right?  I just find it nonsensical to use that
to justify the "definitions" in your proposed xref-find-all-definitions
which potentially serves to find C declarations, C++ deduction
guides, Elisp feature-providing forms, CL macroexpansion loci,
etc, etc.  I've given plenty of alternatives.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 27/11/2023 17:19, João Távora wrote:

>>> Alright, so since this is contentious (who else but you is pushing
>>> this idea?)
>>
>> Indeed, I wonder if nobody else is interested in having the additional
>> commands have pre-defined bindings, or having the same bindings across
>> languages.
> 
> Of course if you put it like that then EVERYBODY is interested in that.
> Everybody likes consistency, but this consistency you're proposing
> is a mirage.  An naive Xref user would be very proud to use
> "xref-find-declaration" in 3 different languages but then would
> start to doubt xref that the things he and the backend author meant
> by declaration are entirely different.

It's not like things are very different now: a backend might fail to 
implement "references", or the new "kinds" thingy, leaving the most 
recent addition not working anyway.

Or, again, different languages might have different notions for what a 
"definition" is. E.g. C vs. TeX.

> Again, leave this to major modes, which is what we always do (or to backend
> which in 95% of cases would live in the major mode when they are
> language-specific).

I wonder where you're getting this statistic from: advanced language 
analysis features don't usually live in major modes, not in the current 
state of the Emacs ecosystem.

>>> what about we start with 0 in xref.el.  We can always
>>> add to 0, no problem, but taking away  from some other number
>>> isn't so easy.
>>
>> We could indeed start with 0, but then I already see the same set of
>> extra commands supported in Eglot, Elisp, lsp-mode
> 
> They only exist in Eglot and LSP-mode because these are LSP things,
> it makes full sense they would exist in any Emacs/LSP interface.

Before LSP, they would similarly make sense for eclim, or omnisharp, or 
rtags/irony-mode, and etc. And that would be the case after LSP, unless 
the current crop of popular languages is simply washed away. We might 
have taken the names from LSP where they're nicely categorized, but they 
correspond to existing navigation features in IDEs that's been there for 
years.

> Elisp?  What commands are you talking about, only if you force them.
> I see no elisp-find-{declaration,type-definition,implementation}
> or anything of the sort.
> 
> The 7 Elisp cross-reference kinds you defined in your original
> patch are
> 
>    "defalias"
>    "face"
>    "function"
>    "constructor"
>    "generic"
>    "variable"
>    "feature'"

That's a good critique. But the list above is a different ontology than 
the declaration/implementation/type-definition "drawers" we were talking 
about. I wonder how much of it really language-specific.

E.g., a C++/Java/Ruby program also has symbols like 
modules/namespaces/classes/methods/functions... Variables, too. Some 
have global ones, some allow one local variables.

What makes the drawers called 
declaration/definition/implementation/type-definition different? I think 
that these ones, in many cases, apply several at once to the same 
symbol, and not just the same name, but the same entity in the code, 
such as for example a method -- one method can have a declaration and an 
implementation (which is these are the "definition" -- perhaps both -- 
is up to interpretation). Or a variable can have a definition (or 
declaration -- for variables they're usually the same) -- and a type 
definition. Point is, there is no way for the Xref backend to decide 
which of these you want to see when invoking "M-.", that's why there are 
separate actions for them in the protocol.

Whereas for Elisp's 
defalias/face/function/constructor/generic/variable/feature, 95% of the 
time the choice is easy to detect from the usage context. Same for 
modules/classes/methods for Ruby or C++. That's actually the reason why 
I felt my implementation for Elisp's "kinds" was useless -- it's simply 
slower and more awkward to work with than the existing 'M-.' (which has 
the automatic "infer namespace" functionality since 2021 courtesy of 
Mattias).

So we could also say that 
declaration/definition/implementation/type-definition are the names of 
"search actions" whereas namespace/class/face/function are "symbol 
namespaces". And to make interaction with "symbol namespaces" more 
useful, I'm guessing we would need something more - e.g. 
namespace-specific symbol completion. For xref-find-all-definitions (or 
whatever new name it has) to prompt for the namespace first and then 
fetch its specific identifier completion table (all functions or all 
faces, etc). And that, by the way, could also work in different 
languages, with very similar nomenclature (I think only "face" in the 
above list is really Elisp-specific).

I concede that the "search actions" such as 
find-declaration/implementation (and especially type-definition) are 
less useful in Elisp than in OO languages where inheritance (or at least 
traits/mixins) are used more widely. Maybe just for the cl-generic code, 
which is a minority. OTOH, perhaps they would be more relevant in CL?

>>>>    and to allow frictionless extensions for special capabilities.
>>>
>>> As to frictionless extension, the macro I proposed already
>>> xref-define-finder seems the easiest way by far.
>>
>> Sorry, I can't find it.
>>
>> But the definition of xref-find-declarations takes about 3 lines. There
>> is not much potential for making it even shorter.
> 
> It's very similar and analogous to:
> 
> (defmacro eglot--code-action (name kind)
>    "Define NAME to execute KIND code action."
>    `(defun ,name (beg &optional end)
>       ,(format "Execute `%s' code actions between BEG and END." kind)
>       (interactive (eglot--code-action-bounds))
>       (eglot-code-actions beg end ,kind t)))
> 
> And allows us to control exactly the 'interactive' spec of such
> commands to give us consistency among them.

I'm wary of using macros as framework/library interface in Elisp because 
unless they are very stable, they can cause versioning problems when 
upgrading or reverting to an earlier version. You change a macro, but a 
dependent package is already installed and byte-compiled, and there 
won't be a recompilation until its next update. We might not even intend 
to change them much, but then you don't always anticipate a bugfix.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 28/11/2023 01:25, João Távora wrote:
> On Mon, Nov 27, 2023 at 6:12 PM Dmitry Gutov <dmitry@gutov.dev> wrote:
> 
>> There are also abstract method definitions in C++ and so on.
> 
> No, there aren't.  You probably mean "(pure) virtual member function".
> And yes, I can assure you that's the jargon that professional and aspiring
> C++ developers prefer when communicating with colleagues and compilers,
> certainly not wishy-washy Javaisms.

My deepest apologies.

>> What goes for declarations in a lot of cases will also be "definitions
> of some sort".
> 
> But if you squint hard enough, everything's "definition".  In fact,
> definition definition definition.  Definition?  Definition!

Unless it's a reference (call site of a function or a place where a 
variable's value is looked up; or referred to in a doc). So yes, these 
are basically two categories of xrefs we distinguish: definitions (only 
one or a few of them for a given name; usually want to jump to one) and 
references (usually many of them for a given name; and we would often 
want to see the full list, to read or search/replace against it).

So in that terminology, both declarations and implementations are "kinds 
of definitions".

If there are suggestion for replacement of that term, I'm happy to discuss.

>> If you don't like "definitions" in "xref-show-definitions-function",
>> then it would need to be a term that describes "special kind of
>> cross-references". Probably referring to the kinds of locations those
>> cross-references point to (which determines the optimal UI).
> 
> I'm fine with that name myself, it's specifically for xref
> -find-definitions, right?

And for declarations/implementations, and so on. Again, 
xref-find-all-definitions might be renamed, but what it does by default 
is find all definition-looking sites for the entity at point. Which 
seems useful enough to me. And which also implies that "references" of 
any sort should not be in that list.

Just to reiterate: xref-find-all-definitions calls xref--show-defs which 
displays the results using xref-show-definitions-function.

> I just find it nonsensical to use that
> to justify the "definitions" in your proposed xref-find-all-definitions
> which potentially serves to find C declarations, C++ deduction
> guides, Elisp feature-providing forms, CL macroexpansion loci,
> etc, etc.  I've given plenty of alternatives.

The only alternative I remember you suggesting is using "xrefs", which 
was missing the point.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Tue, Nov 28, 2023 at 12:30 AM Dmitry Gutov <dmitry@gutov.dev> wrote:

>  I'm happy to discuss.

Well that's obvious.  But I've had enough, sorry.

João

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Tue, Nov 28, 2023 at 12:18 AM Dmitry Gutov <dmitry@gutov.dev> wrote:

> > (defmacro eglot--code-action (name kind)
> >    "Define NAME to execute KIND code action."
> >    `(defun ,name (beg &optional end)
> >       ,(format "Execute `%s' code actions between BEG and END." kind)
> >       (interactive (eglot--code-action-bounds))
> >       (eglot-code-actions beg end ,kind t)))
> >
> > And allows us to control exactly the 'interactive' spec of such
> > commands to give us consistency among them.
>
> I'm wary of using macros as framework/library interface in Elisp because
> unless they are very stable, they can cause versioning problems when
> upgrading or reverting to an earlier version. You change a macro, but a
> dependent package is already installed and byte-compiled, and there
> won't be a recompilation until its next update. We might not even intend
> to change them much, but then you don't always anticipate a bugfix.

BTW, these tangents and distractions are possibly what Eli complained
about when trying to follow this discussion about xref, but in the
interest of Lisp programmers out there, what you describe is both a
very real problem and a problem which is trivial to eradicate.   Just
expand to functions.  Here's a general form of a command-defining macro
suitable for a library interface.

(defmacro mylib-define-command (name &rest margs)
   `(defun ,name (&rest args)
       (interactive (mylib--interactive ,@margs))
       (apply #'mylib--doit args)))

Then you can freely change the implementations of mylib--interactive
and mylib--doit in mylib.el.

And there are tricks to concoct the docstring at
consultation time too (see 13.2.4 Documentation Strings of Functions)

João

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 28/11/2023 11:51, João Távora wrote:
>> I'm wary of using macros as framework/library interface in Elisp because
>> unless they are very stable, they can cause versioning problems when
>> upgrading or reverting to an earlier version. You change a macro, but a
>> dependent package is already installed and byte-compiled, and there
>> won't be a recompilation until its next update. We might not even intend
>> to change them much, but then you don't always anticipate a bugfix.
> BTW, these tangents and distractions are possibly what Eli complained
> about when trying to follow this discussion about xref, but in the
> interest of Lisp programmers out there, what you describe is both a
> very real problem and a problem which is trivial to eradicate.   Just
> expand to functions.  Here's a general form of a command-defining macro
> suitable for a library interface.
> 
> (defmacro mylib-define-command (name &rest margs)
>     `(defun ,name (&rest args)
>         (interactive (mylib--interactive ,@margs))
>         (apply #'mylib--doit args)))
> 
> Then you can freely change the implementations of mylib--interactive
> and mylib--doit in mylib.el.
> 
> And there are tricks to concoct the docstring at
> consultation time too (see 13.2.4 Documentation Strings of Functions)

That's a good practice, and it works (and mitigates the issues) until we 
need to change the underlying function, for some reasons or other. In 
that sense, having the expansion use private functions (with "--" in the 
name) doesn't do much because those cannot be safely renamed, extended, 
etc. Unlike when they're called by a public function.

Anyway, adding such macro is even less of a priority concern than the 
addition (or not) of new key bindings. We could always do it later.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

On Tue, Nov 28, 2023 at 12:45 PM Dmitry Gutov <dmitry@gutov.dev> wrote:

> That's a good practice, and it works (and mitigates the issues) until we
> need to change the underlying function, for some reasons or other. In
> that sense, having the expansion use private functions (with "--" in the
> name) doesn't do much because those cannot be safely renamed, extended,
> etc. Unlike when they're called by a public function.

Sorry, that completely unspecified and vague scenario is irrelevant
in both theory and practice.  Why would you need to rename them?
They're internal functions: noone else but you needs to know or care
about them.  This technique doesn't suffer from any other problem
a function interface wouldn't have as well.  Can't tell why you're
trying to poke holes in such a well known and widely used Lisp
technique, but I must be off.

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 28/11/2023 17:02, João Távora wrote:
> On Tue, Nov 28, 2023 at 12:45 PM Dmitry Gutov<dmitry@gutov.dev>  wrote:
> 
>> That's a good practice, and it works (and mitigates the issues) until we
>> need to change the underlying function, for some reasons or other. In
>> that sense, having the expansion use private functions (with "--" in the
>> name) doesn't do much because those cannot be safely renamed, extended,
>> etc. Unlike when they're called by a public function.
> Sorry, that completely unspecified and vague scenario is irrelevant
> in both theory and practice.  Why would you need to rename them?
> They're internal functions: noone else but you needs to know or care
> about them.

Being able to rename, or change arguments, or split into several, or 
severely rework, are all benefits of using internal functions, which are 
thus taken away.

So essentially we have to choose stable public function(s) instead, no 
matter if they have "--" in the name. Which is what we need to do first.

> This technique doesn't suffer from any other problem
> a function interface wouldn't have as well.  Can't tell why you're
> trying to poke holes in such a well known and widely used Lisp
> technique, but I must be off.

Like you said, it's a tangent, so maybe split off a separate thread if 
you want to continue.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 367 bytes --]

On Tue, Nov 28, 2023, 16:33 Dmitry Gutov <dmitry@gutov.dev> wrote:

Being able to rename, or change arguments, or split into several, or
> severely rework, are all benefits of using internal functions, which are
> thus taken away.
>

No. And you can't come up with an actual scenario that reveals a problem
you wouldn't have when compared to a function interface.

>

[-- Attachment #2: Type: text/html, Size: 906 bytes --]

Re: Adding support for xref jumping to headers/interfaces

[Dmitry Gutov]

On 28/11/2023 19:16, João Távora wrote:
> On Tue, Nov 28, 2023, 16:33 Dmitry Gutov <dmitry@gutov.dev 
> <mailto:dmitry@gutov.dev>> wrote:
> 
>     Being able to rename, or change arguments, or split into several, or
>     severely rework, are all benefits of using internal functions, which
>     are
>     thus taken away.
> 
> 
> No. And you can't come up with an actual scenario that reveals a problem 
> you wouldn't have when compared to a function interface.

I said that introducing a macro requires a public function anyway. So we 
should start with one.

Re: Adding support for xref jumping to headers/interfaces

[João Távora]

[-- Attachment #1: Type: text/plain, Size: 1595 bytes --]

On Tue, Nov 28, 2023, 17:25 Dmitry Gutov <dmitry@gutov.dev> wrote:

> On 28/11/2023 19:16, João Távora wrote:
> > On Tue, Nov 28, 2023, 16:33 Dmitry Gutov <dmitry@gutov.dev
> > <mailto:dmitry@gutov.dev>> wrote:
> >
> >     Being able to rename, or change arguments, or split into several, or
> >     severely rework, are all benefits of using internal functions, which
> >     are
> >     thus taken away.
> >
> >
> > No. And you can't come up with an actual scenario that reveals a problem
> > you wouldn't have when compared to a function interface.
>
> I said that introducing a macro requires a public function anyway.
>

Just because you said it doesn't mean it's true. It's not, that's not
required at all. Again, you can't come up with a real upgrade scenario
where the technique I presented is in any way more problematic than a
normal function interface.

Look, I don't care about xref anymore, I'm really just defending this
technique from your so far 100% unsubstantiated discredit because it's
really a standard watertight technique for presenting macros to a library's
user, and I'd like people here to be aware of it, so the usual problems of
call site recompilation become things of the past. I've used it tens of
times, and seen it used many more. It's also known as the "CALL-WITH"
idiom. The only situation where it "fails" is when you're using macros for
speed, which functions don't give you anyway, and the indirection negates
that. And you rarely should be using macros for that reason anyway,
certainly not in these types of interfaces.

[-- Attachment #2: Type: text/html, Size: 2472 bytes --]