Re: Adding support for xref jumping to headers/interfaces

[Spencer Baugh <sbaugh@janestreet.com>]

Dmitry Gutov <dmitry@gutov.dev> writes:
> On 08/11/2023 19:25, Spencer Baugh wrote:
>
>>>> This (and Joao's patch adding support for eglot) is very
>>>> interesting,
>>>> thank you for implementing it!
>>>> We also discussed a UI which shows all kinds of definitions in a
>>>> single
>>>> buffer.  Maybe the prompt for KIND should default to "all" which shows
>>>> all kinds of definitions?
>>>
>>> The natural question is whether 'xref-find-definitions' would be that
>>> UI, and if not, why not.
>> As in, xref-find-definitions (M-.) *by default* shows all kinds of
>> definitions?  That seems bad for one big reason: 90% of the time, in 90%
>> of languages, there's a clear meaning of "definition" and I want M-. to
>> take me there directly rather than prompting in any way.  But there are
>> other kinds of "definitions" which plausibly I could want, just
>> sometimes, not by default.
>
> Fair enough. So "definitions" are the places we want to see 90% of the
> time when learning about a funciton or a variable. And the other
> reference kinds (BTW, what to call that? "definition kinds"?
> "reference kinds"?) must be apparently less useful.

Right.

> Do you know which category does "eglot-find-typeDefinition" falls
> into, and why? Aside from the fact that it, historically, uses a
> separate endpoint.

It's on the less-useful, more-rarely-used side.

>> If you mean something a non-default version of xref-find-definitions, or
>> another binding for another basically-identical command, sure, I think
>> that would be good.  We can have both that command *and* have a command
>> which prompts for KIND and defaults to "all".
>
> We could indeed, if we decide what to call it. "extras" seems out
> (since it would include both definitions and additional reference
> kinds). Just "xref-find-by-kind"? Then it's less obvious to have the
> default behavior showing all.

Actually, maybe this should be a backend-specific thing.  The backend
could specify a default along with its list of kinds.  And if we request
that default kind (which might be 'all), then the backend will decide
what kinds to send us.

That neatly avoids us having to make any kind of design decisions about
the semantic meaning of different kinds, and what sorts of kinds exist,
and all that.

(Other than having a few standard kinds, which I still think we should
do, if only for the sake of getting the eglot-find-* commands out of
eglot and into xref.  But we don't have to really think about what those
standard kinds mean semantically)

>>> Also, if "references" are included in the list of kinds (as the
>>> current patch for Eglot does, I think), then "show all kinds" is not
>>> likely to be very useful -- all will drown in "references".
>> True, interesting point.  I wonder if we could reasonably
>> distinguish
>> "definition" kinds (a small number per identifier) from "reference"
>> kinds (no limit on how many per identifier).
>
> OTOH, maybe "declarations" are a little closer to "references"?
> E.g. in Elisp the declare-function forms seem to be more in that
> category. And if we're talking about C/C++ forward declarations, I
> suppose that depends on whether the declaration is for an "extern"
>
>> Although I suppose there's
>> also no limit on how many implementations of a generic function there
>> can be, and that's definitely something that fits in "definitions".
>
> The fact that there are often many definitions for a generic function
> indeed is an argument toward including more "kinds" into
> xref-find-definitions' output by default. But only as many as it would
> be actually useful, especially if the kinds are mutually exclusive (a
> symbol is either one or the other) or intersect very rarely, or can be
> distinguished by context.

I suggest this should be up to the backend.  IMO xref-find-definitions
should stay fast in the common case, but sometimes it will unavoidably
have to include multiple things, like for cl-defgenerics in Elisp.  We
can't make the decision about how and when to do that for every backend.

Actually, it occurs to me that if we had an xref-find-implementations
command from the start, with a convenient binding, maybe
xref-find-definitions would just only show the cl-defgeneric, and jump
to it right away.  And only if you hit xref-find-implementations would
you jump to the cl-defmethods.  We can't make that change now, but I
don't think it would be worse!  And if a backend wants a design like
that, I think the backend should be able to have it.

>> Maybe the backend could decide what kinds get included in "all".  Then
>> it could deliberately avoid including anything "reference-like".
>
> Could we want several such commands? E.g. one for "all definition-like
> hits" and another for "all reference-like hits"? With separate sets of
> kinds for definitions and references?

Possibly, but my suggestion is that "all definition-like hits" and "all
reference-like hits" should just be kinds exposed by the backend.
*Maybe* we'll have them be "standard kinds" with a command and binding
by default, but maybe not.  (We could always add more standard kinds and
commands and bindings later.)

>>>>> Or perhaps the main value would be in actually having separate
>>>>> commands which could be bound to a submap with faster key sequences
>>>>> and/or to the context menu (with context-menu-mode on). Then our
>>>>> solution should be more in line with either defining a number of
>>>>> additional named commands (for mode universal kinds) and/or making it
>>>>> easier to define new such commands for users/3rd-party packages/etc.
>>>> That's an interesting idea.  So maybe M-' (or whatever) could be a
>>>> new
>>>> prefix for a prefix-map which contains both universal and mode-specific
>>>> kinds of lookups.
>>>> So maybe something which looks kinda like (not suggesting final
>>>> bindings, just trying to feel out what it would be like):
>>>> generic eglot-find-implementation:   M-' i
>>>> generic eglot-find-declaration:      M-' d
>>>> generic eglot-find-typeDefinition:   M-' t
>>>
>>> Then we will more-or-less nail down the whole set of "available Xref
>>> kinds" in the core by having these commands defined.
>>>
>>> That's not very flexible, but if the set doesn't actually change too
>>> much, and doesn't vary between languages a lot, it could work. Would
>>> lead to a more straightforward design, too.
>> Hm, why do you say this?  This approach seems very extensible to me
>> -
>> modes could add their own bindings/commands/kinds for things specific to
>> that mode.  But they build on top of a set of common kinds which are
>> provided by the core, which don't vary between language too much.
>
> It is less flexible because any time a backend wants to use a new
> "kind", it will need to expend some effort and add it to the core
> somehow. Define a new command or two and assign them to the said
> prefix map. This could also lead to conflicts if backend authors don't
> do this carefully enough.

No, I suggest that xref-backend-extra-kinds should be able to return
whatever arbitrary symbols it wants.  Just, if its "kinds" match the
kinds used in the core, then it will benefit from the commands and
bindings defined by the core.

Perfectly extensible, but still benefiting from standardization.

>> I do think that if we go with an API which has any notion of "kinds", we
>> should have some standard "kinds" in the core like implementation,
>> declaration, type-definition.  I don't see any reason not to do that.
>
> We could have a "registry" of kinds, associating each of them with a
> key. Then the result could be more dynamic, e.g. M-' could be bound to
> a command that reads the key and performs the dispatch to the
> corresponding search (similar to project-switch-project).
>
> And the users (though probably not packages) would later be able to
> customize that mapping, adding new kinds or modifying the keys.
>
> With this approach we don't end up with many xref-find-xyz commands,
> most of which just clutter the namespace, staying unused for a large
> proportion of the users. OTOH, one wouldn't be able to examine the
> prefix map and its contents and definitions with (M-' C-h) -- that's a
> minor reduction in usability.

The only difference between this and a regular keymap which maps keys to
commands is:

> we don't end up with many xref-find-xyz commands

But we can also avoid that by just... not defining many such commands
and kinds.  A mode will always have the ability to define its own
mode-specific kinds in the mode's own namespace, which don't clutter
xref-*.

>>>> xref-extra-kind, prompting for kind: M-' M-'
>>>
>>> Would we need this command, if we had separate commands for each kind
>>> already?
>> This would support kinds which:
>> - are language-specific,
>> - or are more rarely used and don't need a dedicated command,
>> - or are both.
>
> What would be the more frequently used, less language-specific commands?
>
> If we could agree on such list, we could indeed have those
> xref-find-a/b/c definitions and a command with completing-read for the
> rest.

I suggest that we should attempt to provide the following standard
kinds:

'declaration
'implementation
'type-definition

I personally think these would work well for a wide range of languages.
Yes, these are the ones which LSP has; but just because they're from
LSP, doesn't mean they're completely wrong.  They seem to me to strike
the right balance of commonality between languages.

>>>> xref-extra-kind, showing all:        M-' a
>>>> And if there was something mode-specific, like the Java overriding
>>>> method thing, it could be e.g. M-' o
>>>
>>> Most of them are mode-specific already. Some languages don't have
>>> separate "declarations", some don't know what "type definitions" are,
>>> and some, indeed, don't have method overrides.
>> Most languages have something that can fit into "declaration" or
>> "type
>> definitions", though.
>> For Elisp generics, find-implementation could show the cl-defmethod,
>> and
>> find-declaration could jump to the cl-defgeneric.  I would love that,
>> actually - I already find it rather annoying to have to navigate in the
>> xref buffer to the cl-defgeneric when I M-. on a generic method, if I
>> know up front that I want to jump to the cl-defgeneric.
>
> Interesting. I usually find cl-defgeneric entries to be useful in that
> list. But we *are* able to distinguish those that don't contain the
> default implementation.
>
>> Actually, this example has just convinced me that I definitely want
>> "kind-specific commands", even for Elisp.  That would be great.
>> For functions without separate declarations, find-implementation and
>> find-declaration could jump to the same place.  In some languages, maybe
>> that's just always what happens.
>
> Aren't find-implementation included in find-definition anyway? Or
> *are* the same as find-definition, for example, in LSP's approach?
> Aside from the fact that the latter also works for variables and other
> symbols.

No, find-implementation shows all the implementations of a generic
function/interface method.  (e.g. cl-defmethod)

find-definition (as I understand it) jumps to the interface definition
itself (e.g. cl-defgeneric).  Which in my experience in Elisp, is
usually what I want; I want to see the documentation and default
implementation and surrounding code, before I look at a specific
implementation.

I think that's a pretty reasonable way for it to work.  Actually, maybe
we could add a customization point for the Elisp backend so that a user
can choose to make find-definition work that way, if we do add a
find-implementation command.

There's also find-declaration - for Elisp, I think that one *is* the
mostly same as find-definition, since Elisp doesn't have separate
declarations.  (declare-function is kind of a different thing.  Maybe
find-declaration should jump to the declare-function instance in the
current file, if it can, but for now it's fine.)

>> As for type definitions - even Elisp has things that could reasonably be
>> called type definitions, we just don't support jump-to-definition for
>> them right now.  I have no idea how we'd implement it, but we could
>> support jumping to cl-defstruct definitions, for example.  Maybe we
>> could use information from native compilation to guess the type of the
>> identifier at point?  But anyway, for now we'd just error if the user
>> ran such a command in emacs-lisp-mode or other modes that don't support
>> jumping to the type definition.
>
> I think the aforementioned type definitions in Elisp would be better
> included in the "find definitions" set because they generally don't
> clash with other kinds.

I think they actually do clash in the important case.

For global variables, e.g. defcustoms, I want M-. to jump to the
defcustom, and find-type-definition would do the same thing.  So there's
no clash.  And in fact this works today, nothing new.

For local variables with no global definition though, I would like
M-. to jump straight to the local binding inside the function, but
find-type-definition should jump to the cl-defstruct defining the value
of the variable.  That's how it works in Eglot today, at least in Rust
and OCaml.

Actually supporting "jump to local binding" and "jump to type
definition" in Elisp is of course quite hard, but if we did someday find
a way to support it, I see no reason the behavior shouldn't match Eglot.

>>>> - users could still use completing-read to type the kind
>>>> Plus, if we do use M-' or any other short binding for this, we
>>>> should
>>>> almost certainly make it the start of a new prefix-map rather than bind
>>>> M-' directly to the new command; doing otherwise would be wasteful of
>>>> valuable keybinding space.
>>>
>>> If we're going to have separate commands for kinds, that is indeed a
>>> good idea.
>> I almost want to say that we should have it be a prefix regardless
>> of
>> whether we have separate commands for kinds.  I guess it depends:
>
> You probably meant "have it be the binding".
>
>> - if we use C-M-?, that's already such a hard key to hit that maybe it's
>>    okay if we bind it directly to a command
>> - if we use M-' or something similarly convenient, it would be
>> really
>>    tragic to not reclaim all that premium keybinding space.
>
> It's possible that it would work better the other way around: if we
> discover that people really want a prefix map, then it would be easier
> to argue for an easy-to-hit binding like the above.

True, true.  And if we end up with a single command design, then C-M-?
will suffice.